system

Abstract

The system according to this embodiment aims to generate a story based on character settings and a synopsis, and convert it into a video. [Solution] The system according to the embodiment comprises a reception unit, a generation unit, a modification unit, and a video generation unit. The reception unit receives input of character settings and a synopsis. The generation unit generates a story based on the information received by the reception unit. The modification unit presents the story generated by the generation unit to the user and accepts modification instructions. The video generation unit converts the story modified by the modification unit into a video.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, although character settings and outlines can be considered, there is a problem that it is difficult to convert them into specific stories or videos.

[0005] The system according to the embodiment aims to generate a story based on character settings and an outline and convert it into a video.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a reception unit, a generation unit, a modification unit, and a video generation unit. The reception unit receives input of character settings and a synopsis. The generation unit generates a story based on the information received by the reception unit. The modification unit presents the story generated by the generation unit to the user and accepts modification instructions. The video generation unit converts the story modified by the modification unit into a video. [Effects of the Invention]

[0007] The system according to this embodiment can generate a story based on character settings and a synopsis, and convert it into a video. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The manga / anime generation system according to an embodiment of the present invention is a system that utilizes AI to provide a mechanism for anyone to easily become the original author of manga or anime. In this system, the user inputs character settings and a synopsis, and the AI ​​generates a manga or anime story based on those settings, so even people who are not good at drawing or who find paneling difficult can easily create their own works. For example, the user inputs character settings. For example, the protagonist is a hot-blooded and justice-driven police officer belonging to the traffic department, but will act freely to solve cases. Also, the protagonist's junior is cool and logical, but is often led astray by the protagonist. The heroine is a female detective with keen insight and memory, who supports the protagonist and his junior. Furthermore, the protagonist's senior is the head of the traffic department, a strict but warm superior who watches over the protagonist and his colleagues. The police bureaucrat is very strict and can be harsh with his subordinates, but is basically caring towards them and supports them both professionally and personally. Next, the user inputs a synopsis. For example, it depicts the detectives of the Tokyo XX Police Station Traffic Division solving everything from everyday traffic incidents to major crime cases. The protagonist and his companions, while enforcing traffic laws and handling accidents, also become involved in incidents that go beyond the scope of the traffic department, such as murder and terrorist attacks. A heartwarming and humorous human drama unfolds, and at times the story becomes a tense and serious crime suspense. When a user inputs character settings and a synopsis, the AI ​​generates a manga or anime story based on that. For example, the AI ​​automatically generates character dialogue, facial expressions, and panel layouts and presents them to the user. The user can review the generated story and give instructions for revisions as needed. For example, they can give instructions such as "Make the protagonist's running more dynamic," "Make the heroine's facial expression a closer shot," or "Make the food on the table a little more colorful." Furthermore, the AI ​​also has a function to convert the generated manga or anime into a video. For example, it can lower the protagonist's gaze to create a serious atmosphere, turn past scenes in sepia tones, make slow motion even slower, or change the music to sad music to match the heroine's mood. This system makes it easy for anyone to become the original author of a manga or anime.Even those who aren't skilled at drawing or find panel layouts difficult can use AI to bring their ideas to life. This is expected to further grow the manga and anime market. The manga / anime generation system will allow users to simply input character settings and a synopsis, and the AI ​​will automatically generate, revise, and convert the story into a video.

[0029] The manga / anime generation system according to this embodiment comprises a reception unit, a generation unit, a correction unit, and a video generation unit. The reception unit accepts input of character settings and a synopsis. For example, the reception unit can save the character settings entered by the user and record the synopsis. The reception unit can save, for example, the character's name, personality, and background information entered by the user in a database. The reception unit can also save the synopsis entered by the user as text data. Furthermore, the reception unit can analyze the information entered by the user and use it as basic data for story generation. The generation unit generates a story based on the information received by the reception unit. The generation unit generates character dialogue and facial expressions using, for example, AI. The generation unit can generate character dialogue using, for example, natural language processing technology. The generation unit can also generate character facial expressions using image generation technology. Furthermore, the generation unit can automatically generate panel layouts. The generation unit can automatically arrange panel layouts according to the progress of the story. The correction unit presents the story generated by the generation unit to the user and accepts correction instructions. The editing unit provides, for example, an interface for the user to review the generated story and specify areas for revision. The editing unit also provides tools for the user to revise character dialogue and facial expressions. Furthermore, the editing unit can provide an interface for the user to revise panel layouts. Additionally, the editing unit can save the user's revisions and use them as data for regenerating the story. The video generation unit converts the story revised by the editing unit into a video. The video generation unit can, for example, apply effects to the generated story. It can also add effects to match scenes in the story. Furthermore, the video generation unit can save the generated story as a video file. Finally, the video generation unit can provide an interface for delivering the generated video to the user.As a result, the manga / anime generation system according to the embodiment can consistently perform everything from inputting character settings and plot summaries to generating, revising, and converting the story into a video.

[0030] The reception desk accepts input for character settings and synopses. For example, the reception desk can save the character settings entered by the user and record the synopsis. The reception desk can store the character's name, personality, and background information entered by the user in a database. The reception desk can also save the synopsis entered by the user as text data. Furthermore, the reception desk can analyze the information entered by the user and use it as basic data for story generation. Specifically, the reception desk provides an interface for receiving detailed character setting information entered by the user. This interface includes fields for entering detailed information such as the character's appearance, personality, background, special skills, and relationships. For example, for the character's appearance, detailed attributes such as hair color, eye color, height, and body type can be entered. For personality, personality traits such as introverted, extroverted, brave, and timid can be selected. For background information, the character's past events, family structure, occupation, etc. can be entered. This information is stored in a database and used in the subsequent story generation process. Furthermore, the reception department analyzes the synopsis entered by the user and uses natural language processing technology to extract the main elements and plot points of the story. For example, it extracts information such as main characters, events, places, and times from the synopsis and organizes it as basic data for story generation. This allows the reception department to efficiently collect the information provided by the user and prepare it for story generation.

[0031] The generation unit generates a story based on the information received by the reception unit. The generation unit generates character dialogue and facial expressions, for example, using AI. The generation unit can generate character dialogue using natural language processing technology, for example. The generation unit can also generate character facial expressions using image generation technology. Furthermore, the generation unit can automatically generate panel layouts. The generation unit can automatically arrange panel layouts according to the progression of the story, for example. Specifically, the generation unit generates a detailed plot of the story using AI based on the character settings and synopsis entered by the user. The AI ​​generates character dialogue using natural language processing technology and selects appropriate expressions according to the character's personality and situation. For example, in scenes where the character is tense, it generates short, fragmented dialogue, and in scenes where the character is relaxed, it generates long, detailed dialogue. It also generates character facial expressions using image generation technology and depicts appropriate expressions to match the dialogue. Furthermore, the generation unit uses an algorithm to automatically arrange panel layouts according to the progression of the story. This algorithm adjusts the size and placement of panels to highlight important scenes and action sequences in the story. For example, it uses larger panels in action scenes to emphasize character movements, while using smaller panels in dialogue scenes to focus on character expressions and lines. This allows the generator to automatically produce a compelling and consistent story based on the information provided by the user.

[0032] The editing unit presents the story generated by the generation unit to the user and accepts editing instructions. For example, the editing unit provides an interface for the user to review the generated story and specify areas for revision. It also provides tools for the user to revise character dialogue and expressions. Furthermore, the editing unit can provide an interface for the user to revise panel layouts. Additionally, the editing unit can save the user's revisions and use them as data for regenerating the story. Specifically, the editing unit provides an interface that allows the user to visually review the generated story. This interface displays the content of each panel, character dialogue, and expressions, making it easy for the user to identify areas for revision. Users can make revisions directly on the interface, for example, by editing character dialogue or changing expressions. The editing unit also provides a drag-and-drop function for users to revise panel layouts, allowing for easy adjustment of panel positions and sizes. Furthermore, the editing unit reflects the user's revisions in real time and saves them as data for regenerating the story. This allows the user to customize the generated story to their liking and complete the final work.

[0033] The video generation unit converts the story, corrected by the correction unit, into a video. The video generation unit can, for example, apply effects to the generated story. It can also add effects to match the scenes in the story. Furthermore, the video generation unit can save the generated story as a video file. In addition, the video generation unit can provide an interface for users to access the generated video. Specifically, the video generation unit uses an algorithm to animate each frame based on the corrected story. This algorithm smoothly reproduces character movements and facial expressions, maintaining the continuity of the scenes. The video generation unit also adds appropriate effects to each scene to enhance its visual appeal. For example, it adds explosion and light effects to action scenes, and blurs the background and adjusts lighting in dialogue scenes. Furthermore, the video generation unit saves the generated video in high resolution, allowing users to view it on various devices. The video file is saved in a common video format (e.g., MP4 or AVI) for easy access by users. The video generation unit also provides an interface for sharing the generated video, allowing users to easily upload it to social media and video sharing sites. This allows the video generation unit to provide the revised story as a high-quality video, helping users to widely share their work.

[0034] The generation unit includes a dialogue generation unit that generates character dialogue and facial expressions. The dialogue generation unit can generate dialogue based on, for example, the character's tone of voice and writing style. The dialogue generation unit can generate character dialogue using, for example, natural language processing technology. The dialogue generation unit can also adjust the content of the dialogue according to the character's emotions. For example, if the character is angry, the dialogue generation unit can generate dialogue in a strong tone. Also, if the character is sad, the dialogue generation unit can generate dialogue that expresses that emotion. Furthermore, the dialogue generation unit can also generate dialogue based on the relationship between the characters. For example, in a scene between the protagonist and the heroine, the dialogue generation unit can generate dialogue that emphasizes emotions. As a result, the generation unit can automatically generate character dialogue and facial expressions.

[0035] The generation unit includes a panel layout generation unit that generates panel layouts. The panel layout generation unit can, for example, automatically arrange panel layouts according to the progression of the story. The panel layout generation unit can, for example, place large panels to emphasize important scenes in the story. The panel layout generation unit can also adjust the size and placement of panels to match the tempo of the story. For example, in action scenes, small panels can be placed in succession to depict fast movement. The panel layout generation unit can also place large panels in emotional scenes of the story to emphasize emotions. Furthermore, the panel layout generation unit can adjust the level of detail in the panel layouts based on the development of the story. For example, detailed depictions can be used in important scenes, while other scenes can be simplified. As a result, the generation unit can automatically generate panel layouts.

[0036] The editing unit includes an interface unit for users to review the generated story and issue revision instructions. The interface unit can, for example, provide tools for users to review the generated story and specify areas for revision. It can also provide an interface for revising character dialogue and facial expressions. Furthermore, the interface unit can provide tools for revising panel layouts. For example, it can provide tools for users to adjust panel size and placement. Additionally, the interface unit can save the user's revisions and use them as data for regenerating the story. This allows the editing unit to review the generated story and issue revision instructions.

[0037] The video generation unit includes an effects application unit that applies effects to the generated story. The effects application unit can, for example, add effects to match the scenes in the story. For example, the effects application unit can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to emotional scenes. For example, the effects application unit can apply a sepia-toned effect to sad scenes. It can also apply bright-colored effects to happy scenes. Furthermore, the effects application unit can adjust the level of detail of the effects based on the importance of the scene. For example, it can apply detailed effects to important scenes and simplified effects to other scenes. In this way, the video generation unit can apply effects to the generated story.

[0038] The generation unit includes a customization unit for inputting different character settings and plot summaries. The customization unit can, for example, provide an interface for users to input different character settings. For instance, it can provide tools for users to input character names, personalities, and background information. It can also provide an interface for users to input different plot summaries. For example, it can provide tools for users to input a story outline and major events. Furthermore, the customization unit can save the information entered by the user and use it as data for story generation. This allows the generation unit to input different character settings and plot summaries.

[0039] The reception desk can analyze the user's past input history and suggest the optimal input method. For example, it can automatically suggest similar character settings based on the user's past input. It can also prioritize suggesting input methods (voice, text, etc.) that the user has used in the past. Furthermore, it can predict and suggest input methods to be used during specific time periods based on the user's past input history. This allows the reception desk to suggest the optimal input method based on the user's past input history.

[0040] The input system can filter character settings and plot summaries based on the user's current interests. For example, it can suggest relevant character settings based on genres the user has recently been interested in (e.g., fantasy or science fiction). It can also suggest similar plot summaries based on manga or anime the user has recently read. Furthermore, the input system can analyze the content of accounts the user follows on social media and suggest settings based on their interests. This allows the input system to filter based on the user's interests.

[0041] The input system can prioritize inputting highly relevant settings when users enter character settings and plot summaries, taking into account their geographical location. For example, if a user lives in a specific region, the input system can suggest character settings related to that region. Similarly, if a user is traveling, the input system can suggest plot summaries related to the places they are visiting. Furthermore, if a user is participating in a specific event, the input system can suggest settings related to that event. This allows the input system to prioritize inputting highly relevant settings based on the user's geographical location.

[0042] The reception desk can analyze the user's social media activity when they input character settings and synopses, and suggest relevant settings. For example, it can suggest relevant character settings based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest synopses based on their areas of interest. Furthermore, it can suggest relevant settings based on the content of articles and videos the user has shared. In this way, the reception desk can suggest relevant settings based on the user's social media activity.

[0043] The generation unit can adjust the level of detail in story generation based on the importance of each character. For example, it can provide detailed descriptions in scenes featuring the protagonist and simplify scenes featuring supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those of other characters. Furthermore, it can focus on depicting scenes featuring major characters while omitting background characters. In this way, the generation unit can adjust the level of detail in story generation based on the importance of each character.

[0044] The generation unit can apply different generation algorithms depending on the character category when generating a story. For example, the generation unit can apply an algorithm that provides detailed character descriptions to the main character. It can also apply an algorithm that provides simplified descriptions to supporting characters. Furthermore, it can apply an algorithm that creates a sense of tension to enemy characters. In this way, the generation unit can apply different generation algorithms depending on the character category.

[0045] The generation unit can determine the priority of character creation based on when character settings were submitted during story generation. For example, the generation unit can prioritize incorporating character settings submitted early into the story. It can also simplify character settings submitted close to the deadline before incorporating them into the story. Furthermore, the generation unit can adjust the level of detail in the character settings depending on when they were submitted. This allows the generation unit to determine the priority of character creation based on when the character settings were submitted.

[0046] The generation unit can adjust the generation order based on the relevance of character settings when generating a story. For example, the generation unit can prioritize the incorporation of settings for major characters into the story. It can also group together highly relevant character settings into the story. Furthermore, it can postpone the incorporation of less relevant character settings into the story. In this way, the generation unit can adjust the generation order based on the relevance of character settings.

[0047] The correction unit can analyze the user's past correction history and suggest the optimal correction method during the correction process. For example, it can suggest similar correction methods based on the user's past corrections. It can also prioritize suggesting correction tools the user has used in the past. Furthermore, it can predict and suggest correction methods to be used during specific time periods based on the user's past correction history. This allows the correction unit to suggest the optimal correction method based on the user's past correction history.

[0048] The editing unit can customize the editing process based on the user's current areas of interest. For example, it can suggest relevant editing options based on genres the user has recently been interested in. It can also suggest similar editing methods based on the content of manga or anime the user has recently read. Furthermore, it can analyze the content of accounts the user follows on social media and suggest editing options based on their areas of interest. This allows the editing unit to customize the editing process based on the user's areas of interest.

[0049] The correction unit can suggest the optimal correction method when making corrections, taking into account the user's geographical location. For example, if the user lives in a specific region, the correction unit can suggest correction options related to that region. Furthermore, if the user is traveling, the correction unit can suggest correction options related to the place they are visiting. Additionally, if the user is participating in a specific event, the correction unit can suggest correction options related to that event. This allows the correction unit to suggest the optimal correction method based on the user's geographical location.

[0050] The editing function can analyze the user's social media activity and suggest editing options during the editing process. For example, it can suggest relevant editing options based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest editing options based on their areas of interest. Furthermore, it can suggest relevant editing options based on the content of articles and videos the user has shared. In this way, the editing function can suggest editing options based on the user's social media activity.

[0051] The video generation unit can adjust the level of detail generated based on the importance of the story during video generation. For example, it can provide detailed depictions in important scenes and simplify other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can provide simple depictions in everyday scenes and concentrate resources on important scenes. This allows the video generation unit to adjust the level of detail generated based on the importance of the story.

[0052] The video generation unit can apply different generation algorithms depending on the story category during video generation. For example, it can apply an algorithm that produces fast-paced action scenes. It can also apply an algorithm that emphasizes emotions in romance scenes. Furthermore, it can apply an algorithm that produces humorous scenes in comedy scenes. In this way, the video generation unit can apply different generation algorithms depending on the story category.

[0053] The video generation unit can determine the generation priority based on when the stories were submitted. For example, it can prioritize converting stories submitted early into videos. It can also simplify stories submitted close to the deadline before converting them into videos. Furthermore, the video generation unit can adjust the level of detail in the stories according to the submission date. This allows the video generation unit to determine the generation priority based on when the stories were submitted.

[0054] The video generation unit can adjust the generation order based on the relevance of the stories during video generation. For example, the video generation unit can prioritize converting the main story into a video. It can also convert highly relevant stories together into a video. Furthermore, the video generation unit can postpone the conversion of less relevant stories. In this way, the video generation unit can adjust the generation order based on the relevance of the stories.

[0055] The dialogue generation unit can adjust the level of detail in the dialogue based on the character's personality. For example, it can provide detailed descriptions for the protagonist's dialogue and simplify the dialogue for supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those for other characters. Furthermore, it can focus on describing scenes featuring main characters while omitting background characters. In this way, the dialogue generation unit can adjust the level of detail in the dialogue based on the character's personality.

[0056] The dialogue generation unit can apply different generation algorithms depending on the relationships between the characters when generating dialogue. For example, the dialogue generation unit can apply an algorithm that emphasizes emotions in scenes between the protagonist and the heroine. It can also apply an algorithm that creates tension in scenes with enemy characters. Furthermore, it can apply an algorithm that provides humorous descriptions in comedic scenes. In this way, the dialogue generation unit can apply different generation algorithms depending on the relationships between the characters.

[0057] The dialogue generation unit can determine the priority of dialogue based on when the character settings were submitted. For example, the dialogue generation unit can prioritize the inclusion of character settings submitted early in the dialogue. It can also simplify character settings submitted close to the deadline before incorporating them into the dialogue. Furthermore, the dialogue generation unit can adjust the level of detail in the character settings according to the submission date. This allows the dialogue generation unit to determine the priority of dialogue based on when the character settings were submitted.

[0058] The dialogue generation unit can adjust the order of dialogue based on the relevance of character settings during the dialogue generation process. For example, the dialogue generation unit can prioritize reflecting the settings of main characters in the dialogue. It can also group together highly relevant character settings and reflect them in the dialogue. Furthermore, it can postpone reflecting less relevant character settings in the dialogue. In this way, the dialogue generation unit can adjust the order of dialogue based on the relevance of character settings.

[0059] The panel layout generator can adjust the level of detail in the panels based on the story's progression. For example, it can provide detailed depictions in important scenes and simplify other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can provide simple depictions in everyday scenes and concentrate resources on important scenes. In this way, the panel layout generator can adjust the level of detail in the panels based on the story's progression.

[0060] The panel layout generation unit can apply different generation algorithms depending on the importance of the scene during panel layout generation. For example, it can apply an algorithm that depicts fast-moving action to action scenes. It can also apply an algorithm that emphasizes emotions to romance scenes. Furthermore, it can apply an algorithm that depicts humor to comedy scenes. In this way, the panel layout generation unit can apply different generation algorithms depending on the importance of the scene.

[0061] The panel layout generation unit can determine the priority of panel layouts based on the submission timing of the stories. For example, it can prioritize incorporating stories submitted early into the panel layouts. It can also simplify stories submitted close to the deadline before incorporating them into the panel layouts. Furthermore, the panel layout generation unit can adjust the level of detail in the stories according to the submission timing. This allows the panel layout generation unit to determine the priority of panel layouts based on the submission timing of the stories.

[0062] The panel layout generation unit can adjust the order of panels based on the relevance of the story during panel generation. For example, the panel layout generation unit can prioritize reflecting the main story in the panel layout. It can also group highly related storylines together and reflect them in the panel layout. Furthermore, it can postpone reflecting less relevant storylines in the panel layout. In this way, the panel layout generation unit can adjust the order of panels based on the relevance of the story.

[0063] The interface unit can select the optimal display method by referring to the user's past operation history when displaying the interface. For example, the interface unit can prioritize displaying interface designs that the user has used in the past. Furthermore, the interface unit can customize the interface based on the user's preferred colors and layouts. In addition, the interface unit can predict and suggest interface designs to be used during specific time periods based on the user's past operation history. This allows the interface unit to select the optimal display method based on the user's past operation history.

[0064] The interface unit can select the optimal display method when displaying the interface, taking into account the user's device information. For example, if the user is using a smartphone, the interface unit can provide a display method that matches the screen size. Furthermore, if the user is using a tablet, the interface unit can provide a display method optimized for a larger screen. Additionally, if the user is using a smartwatch, the interface unit can provide a concise and highly visible display method. In this way, the interface unit can select the optimal display method based on the user's device information.

[0065] The effects application unit can adjust the level of detail of an effect based on the importance of the scene when applying it. For example, it can apply detailed effects to important scenes and simplify them to other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can apply simple effects to everyday scenes and concentrate resources on important scenes. In this way, the effects application unit can adjust the level of detail of an effect based on the importance of the scene.

[0066] The effects application unit can apply different effects depending on the scene category when applying effects. For example, it can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to romance scenes. Furthermore, it can apply humorous effects to comedy scenes. In this way, the effects application unit can apply different effects depending on the scene category.

[0067] The effects application unit can determine the priority of effects based on the scene submission date when applying effects. For example, the effects application unit can prioritize the application of effects to scenes submitted early. Furthermore, the effects application unit can simplify the effects applied to scenes submitted close to the deadline. In addition, the effects application unit can adjust the level of detail of the scene according to the submission date. This allows the effects application unit to determine the priority of effects based on the scene submission date.

[0068] The effect application unit can adjust the order of effects based on the relevance of the scenes when applying effects. For example, the effect application unit can prioritize applying effects to major scenes. It can also apply effects to highly relevant scenes together. Furthermore, the effect application unit can postpone applying effects to less relevant scenes. In this way, the effect application unit can adjust the order of effects based on the relevance of the scenes.

[0069] The customization department can analyze the user's past customization history to propose the optimal customization method during the customization process. For example, it can suggest similar customization methods based on the user's past customizations. It can also prioritize suggesting customization tools the user has used in the past. Furthermore, it can predict and suggest customization methods to be used at specific times based on the user's past customization history. This allows the customization department to propose the optimal customization method based on the user's past customization history.

[0070] The customization unit can customize the means of customization based on the user's current areas of interest during the customization process. For example, the customization unit can suggest relevant customization options based on genres the user has recently been interested in. It can also suggest similar customization methods based on the content of manga or anime the user has recently read. Furthermore, the customization unit can analyze the content of accounts the user follows on social media and suggest customization options based on areas of interest. In this way, the customization unit can customize the means of customization based on the user's areas of interest.

[0071] The customization unit can suggest the optimal customization method by considering the user's geographical location during the customization process. For example, if the user lives in a specific region, the customization unit can suggest customization options related to that region. Furthermore, if the user is traveling, the customization unit can suggest customization options related to the place they are visiting. In addition, if the user is participating in a specific event, the customization unit can suggest customization options related to that event. This allows the customization unit to suggest the optimal customization method based on the user's geographical location.

[0072] The customization department can analyze the user's social media activity during the customization process and suggest customization options. For example, it can suggest relevant customization options based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest customization options based on their areas of interest. Furthermore, it can suggest relevant customization options based on the content of articles and videos the user has shared. In this way, the customization department can suggest customization options based on the user's social media activity.

[0073] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0074] The generation unit can adjust the level of detail in story generation based on the relationships between characters. For example, it can provide detailed descriptions in scenes with main characters and simplify scenes with supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those of other characters. Furthermore, it can focus on depicting scenes with main characters while omitting background characters. In this way, the generation unit can adjust the level of detail in story generation based on the relationships between characters.

[0075] The video generation unit includes an effects application unit that applies effects to the generated story. The effects application unit can, for example, add effects to match the scenes in the story. For example, the effects application unit can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to emotional scenes. For example, the effects application unit can apply a sepia-toned effect to sad scenes. It can also apply bright-colored effects to happy scenes. Furthermore, the effects application unit can adjust the level of detail of the effects based on the importance of the scene. For example, it can apply detailed effects to important scenes and simplified effects to other scenes. In this way, the video generation unit can apply effects to the generated story.

[0076] The generation unit can apply different generation algorithms depending on the character category when generating a story. For example, the main character can be given an algorithm that provides detailed character descriptions. Sub-characters can be given an algorithm that provides simplified descriptions. Furthermore, enemy characters can be given an algorithm that creates a sense of tension. In this way, the generation unit can apply different generation algorithms depending on the character category.

[0077] The reception desk can analyze a user's past input history and suggest the optimal input method. For example, it can automatically suggest similar settings based on character settings the user has entered in the past. It can also prioritize suggesting input methods the user has used in the past (voice, text, etc.). Furthermore, it can predict and suggest input methods that the user will use at specific times of the day based on their past input history. In this way, the reception desk can suggest the optimal input method based on the user's past input history.

[0078] The correction unit can analyze the user's past correction history and propose the optimal correction method during the correction process. For example, it can suggest similar correction methods based on the user's past corrections. It can also prioritize suggesting correction tools the user has used in the past. Furthermore, it can predict and suggest correction methods to be used during specific time periods based on the user's past correction history. In this way, the correction unit can propose the optimal correction method based on the user's past correction history.

[0079] The following briefly describes the processing flow for example form 1.

[0080] Step 1: The reception desk accepts character settings and synopsis input. For example, it saves the character's name, personality, and background information entered by the user to a database, and saves the synopsis as text data. It also analyzes the information entered by the user and uses it as basic data for story generation. Step 2: The generation unit generates a story based on the information received by the reception unit. For example, it uses AI to generate character dialogue and facial expressions, and implements this using natural language processing and image generation technologies. It also automatically arranges the panels according to the progression of the story. Step 3: The editing unit presents the story generated by the generation unit to the user and accepts editing instructions. For example, it provides an interface and tools for the user to edit character dialogue, facial expressions, and panel layouts, and saves the edited content to use as data for generating the story again. Step 4: The video generation unit converts the story modified by the modification unit into a video. For example, it applies effects to the generated story and saves it as a video file. It also provides an interface for delivering the generated video to the user.

[0081] (Example of form 2) The manga / anime generation system according to an embodiment of the present invention is a system that utilizes AI to provide a mechanism for anyone to easily become the original author of manga or anime. In this system, the user inputs character settings and a synopsis, and the AI ​​generates a manga or anime story based on those settings, so even people who are not good at drawing or who find paneling difficult can easily create their own works. For example, the user inputs character settings. For example, the protagonist is a hot-blooded and justice-driven police officer belonging to the traffic department, but will act freely to solve cases. Also, the protagonist's junior is cool and logical, but is often led astray by the protagonist. The heroine is a female detective with keen insight and memory, who supports the protagonist and his junior. Furthermore, the protagonist's senior is the head of the traffic department, a strict but warm superior who watches over the protagonist and his colleagues. The police bureaucrat is very strict and can be harsh with his subordinates, but is basically caring towards them and supports them both professionally and personally. Next, the user inputs a synopsis. For example, it depicts the detectives of the Tokyo XX Police Station Traffic Division solving everything from everyday traffic incidents to major crime cases. The protagonist and his companions, while enforcing traffic laws and handling accidents, also become involved in incidents that go beyond the scope of the traffic department, such as murder and terrorist attacks. A heartwarming and humorous human drama unfolds, and at times the story becomes a tense and serious crime suspense. When a user inputs character settings and a synopsis, the AI ​​generates a manga or anime story based on that. For example, the AI ​​automatically generates character dialogue, facial expressions, and panel layouts and presents them to the user. The user can review the generated story and give instructions for revisions as needed. For example, they can give instructions such as "Make the protagonist's running more dynamic," "Make the heroine's facial expression a closer shot," or "Make the food on the table a little more colorful." Furthermore, the AI ​​also has a function to convert the generated manga or anime into a video. For example, it can lower the protagonist's gaze to create a serious atmosphere, turn past scenes in sepia tones, make slow motion even slower, or change the music to sad music to match the heroine's mood. This system makes it easy for anyone to become the original author of a manga or anime.Even those who aren't skilled at drawing or find panel layouts difficult can use AI to bring their ideas to life. This is expected to further grow the manga and anime market. The manga / anime generation system will allow users to simply input character settings and a synopsis, and the AI ​​will automatically generate, revise, and convert the story into a video.

[0082] The manga / anime generation system according to this embodiment comprises a reception unit, a generation unit, a correction unit, and a video generation unit. The reception unit accepts input of character settings and a synopsis. For example, the reception unit can save the character settings entered by the user and record the synopsis. The reception unit can save, for example, the character's name, personality, and background information entered by the user in a database. The reception unit can also save the synopsis entered by the user as text data. Furthermore, the reception unit can analyze the information entered by the user and use it as basic data for story generation. The generation unit generates a story based on the information received by the reception unit. The generation unit generates character dialogue and facial expressions using, for example, AI. The generation unit can generate character dialogue using, for example, natural language processing technology. The generation unit can also generate character facial expressions using image generation technology. Furthermore, the generation unit can automatically generate panel layouts. The generation unit can automatically arrange panel layouts according to the progress of the story. The correction unit presents the story generated by the generation unit to the user and accepts correction instructions. The editing unit provides, for example, an interface for the user to review the generated story and specify areas for revision. The editing unit also provides tools for the user to revise character dialogue and facial expressions. Furthermore, the editing unit can provide an interface for the user to revise panel layouts. Additionally, the editing unit can save the user's revisions and use them as data for regenerating the story. The video generation unit converts the story revised by the editing unit into a video. The video generation unit can, for example, apply effects to the generated story. It can also add effects to match scenes in the story. Furthermore, the video generation unit can save the generated story as a video file. Finally, the video generation unit can provide an interface for delivering the generated video to the user.As a result, the manga / anime generation system according to the embodiment can consistently perform everything from inputting character settings and plot summaries to generating, revising, and converting the story into a video.

[0083] The reception desk accepts input for character settings and synopses. For example, the reception desk can save the character settings entered by the user and record the synopsis. The reception desk can store the character's name, personality, and background information entered by the user in a database. The reception desk can also save the synopsis entered by the user as text data. Furthermore, the reception desk can analyze the information entered by the user and use it as basic data for story generation. Specifically, the reception desk provides an interface for receiving detailed character setting information entered by the user. This interface includes fields for entering detailed information such as the character's appearance, personality, background, special skills, and relationships. For example, for the character's appearance, detailed attributes such as hair color, eye color, height, and body type can be entered. For personality, personality traits such as introverted, extroverted, brave, and timid can be selected. For background information, the character's past events, family structure, occupation, etc. can be entered. This information is stored in a database and used in the subsequent story generation process. Furthermore, the reception department analyzes the synopsis entered by the user and uses natural language processing technology to extract the main elements and plot points of the story. For example, it extracts information such as main characters, events, places, and times from the synopsis and organizes it as basic data for story generation. This allows the reception department to efficiently collect the information provided by the user and prepare it for story generation.

[0084] The generation unit generates a story based on the information received by the reception unit. The generation unit generates character dialogue and facial expressions, for example, using AI. The generation unit can generate character dialogue using natural language processing technology, for example. The generation unit can also generate character facial expressions using image generation technology. Furthermore, the generation unit can automatically generate panel layouts. The generation unit can automatically arrange panel layouts according to the progression of the story, for example. Specifically, the generation unit generates a detailed plot of the story using AI based on the character settings and synopsis entered by the user. The AI ​​generates character dialogue using natural language processing technology and selects appropriate expressions according to the character's personality and situation. For example, in scenes where the character is tense, it generates short, fragmented dialogue, and in scenes where the character is relaxed, it generates long, detailed dialogue. It also generates character facial expressions using image generation technology and depicts appropriate expressions to match the dialogue. Furthermore, the generation unit uses an algorithm to automatically arrange panel layouts according to the progression of the story. This algorithm adjusts the size and placement of panels to highlight important scenes and action sequences in the story. For example, it uses larger panels in action scenes to emphasize character movements, while using smaller panels in dialogue scenes to focus on character expressions and lines. This allows the generator to automatically produce a compelling and consistent story based on the information provided by the user.

[0085] The editing unit presents the story generated by the generation unit to the user and accepts editing instructions. For example, the editing unit provides an interface for the user to review the generated story and specify areas for revision. It also provides tools for the user to revise character dialogue and expressions. Furthermore, the editing unit can provide an interface for the user to revise panel layouts. Additionally, the editing unit can save the user's revisions and use them as data for regenerating the story. Specifically, the editing unit provides an interface that allows the user to visually review the generated story. This interface displays the content of each panel, character dialogue, and expressions, making it easy for the user to identify areas for revision. Users can make revisions directly on the interface, for example, by editing character dialogue or changing expressions. The editing unit also provides a drag-and-drop function for users to revise panel layouts, allowing for easy adjustment of panel positions and sizes. Furthermore, the editing unit reflects the user's revisions in real time and saves them as data for regenerating the story. This allows the user to customize the generated story to their liking and complete the final work.

[0086] The video generation unit converts the story, corrected by the correction unit, into a video. The video generation unit can, for example, apply effects to the generated story. It can also add effects to match the scenes in the story. Furthermore, the video generation unit can save the generated story as a video file. In addition, the video generation unit can provide an interface for users to access the generated video. Specifically, the video generation unit uses an algorithm to animate each frame based on the corrected story. This algorithm smoothly reproduces character movements and facial expressions, maintaining the continuity of the scenes. The video generation unit also adds appropriate effects to each scene to enhance its visual appeal. For example, it adds explosion and light effects to action scenes, and blurs the background and adjusts lighting in dialogue scenes. Furthermore, the video generation unit saves the generated video in high resolution, allowing users to view it on various devices. The video file is saved in a common video format (e.g., MP4 or AVI) for easy access by users. The video generation unit also provides an interface for sharing the generated video, allowing users to easily upload it to social media and video sharing sites. This allows the video generation unit to provide the revised story as a high-quality video, helping users to widely share their work.

[0087] The generation unit includes a dialogue generation unit that generates character dialogue and facial expressions. The dialogue generation unit can generate dialogue based on, for example, the character's tone of voice and writing style. The dialogue generation unit can generate character dialogue using, for example, natural language processing technology. The dialogue generation unit can also adjust the content of the dialogue according to the character's emotions. For example, if the character is angry, the dialogue generation unit can generate dialogue in a strong tone. Also, if the character is sad, the dialogue generation unit can generate dialogue that expresses that emotion. Furthermore, the dialogue generation unit can also generate dialogue based on the relationship between the characters. For example, in a scene between the protagonist and the heroine, the dialogue generation unit can generate dialogue that emphasizes emotions. As a result, the generation unit can automatically generate character dialogue and facial expressions.

[0088] The generation unit includes a panel layout generation unit that generates panel layouts. The panel layout generation unit can, for example, automatically arrange panel layouts according to the progression of the story. The panel layout generation unit can, for example, place large panels to emphasize important scenes in the story. The panel layout generation unit can also adjust the size and placement of panels to match the tempo of the story. For example, in action scenes, small panels can be placed in succession to depict fast movement. The panel layout generation unit can also place large panels in emotional scenes of the story to emphasize emotions. Furthermore, the panel layout generation unit can adjust the level of detail in the panel layouts based on the development of the story. For example, detailed depictions can be used in important scenes, while other scenes can be simplified. As a result, the generation unit can automatically generate panel layouts.

[0089] The editing unit includes an interface unit for users to review the generated story and issue revision instructions. The interface unit can, for example, provide tools for users to review the generated story and specify areas for revision. The interface unit can also provide an interface for revising character dialogue and facial expressions. Furthermore, the interface unit can provide tools for revising panel layouts. For example, the interface unit can provide tools for users to adjust panel size and placement. In addition, the interface unit can save the user's revisions and use them as data for regenerating the story. This allows the editing unit to review the generated story and issue revision instructions.

[0090] The video generation unit includes an effects application unit that applies effects to the generated story. The effects application unit can, for example, add effects to match the scenes in the story. For example, the effects application unit can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to emotional scenes. For example, the effects application unit can apply a sepia-toned effect to sad scenes. It can also apply bright-colored effects to happy scenes. Furthermore, the effects application unit can adjust the level of detail of the effects based on the importance of the scene. For example, it can apply detailed effects to important scenes and simplified effects to other scenes. In this way, the video generation unit can apply effects to the generated story.

[0091] The generation unit includes a customization unit for inputting different character settings and plot summaries. The customization unit can, for example, provide an interface for users to input different character settings. For instance, it can provide tools for users to input character names, personalities, and background information. It can also provide an interface for users to input different plot summaries. For example, it can provide tools for users to input a story outline and major events. Furthermore, the customization unit can save the information entered by the user and use it as data for story generation. This allows the generation unit to input different character settings and plot summaries.

[0092] The reception system can estimate the user's emotions and adjust the timing of character setting and synopsis input based on the estimated emotions. For example, if the user is excited, the reception system can immediately prompt for character setting input, allowing the user to input ideas while their emotions are heightened. If the user is relaxed, the reception system can allow them to input the synopsis first and proceed with character setting at a more leisurely pace. Furthermore, if the user is tired, the reception system can provide a simplified input form to allow for quicker completion. This allows the reception system to adjust the input timing according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0093] The reception desk can analyze the user's past input history and suggest the optimal input method. For example, it can automatically suggest similar character settings based on the user's past input. It can also prioritize suggesting input methods (voice, text, etc.) that the user has used in the past. Furthermore, it can predict and suggest input methods to be used during specific time periods based on the user's past input history. This allows the reception desk to suggest the optimal input method based on the user's past input history.

[0094] The input system can filter character settings and plot summaries based on the user's current interests. For example, it can suggest relevant character settings based on genres the user has recently been interested in (e.g., fantasy or science fiction). It can also suggest similar plot summaries based on manga or anime the user has recently read. Furthermore, the input system can analyze the content of accounts the user follows on social media and suggest settings based on their interests. This allows the input system to filter based on the user's interests.

[0095] The input system can estimate the user's emotions and, based on those emotions, determine the priority of character settings and plot outlines to be entered. For example, if the user is excited, the input system can prioritize character settings and allow them to enter ideas while their emotions are heightened. If the user is relaxed, the input system can allow them to enter the plot outline first and proceed with character settings at a more leisurely pace. Furthermore, if the user is tired, the input system can provide a simplified input form to allow for quicker completion. This allows the input system to prioritize inputs according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0096] The input system can prioritize inputting highly relevant settings when users enter character settings and plot summaries, taking into account their geographical location. For example, if a user lives in a specific region, the input system can suggest character settings related to that region. Similarly, if a user is traveling, the input system can suggest plot summaries related to the places they are visiting. Furthermore, if a user is participating in a specific event, the input system can suggest settings related to that event. This allows the input system to prioritize inputting highly relevant settings based on the user's geographical location.

[0097] The reception desk can analyze the user's social media activity when they input character settings and synopses, and suggest relevant settings. For example, it can suggest relevant character settings based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest synopses based on their areas of interest. Furthermore, it can suggest relevant settings based on the content of articles and videos the user has shared. In this way, the reception desk can suggest relevant settings based on the user's social media activity.

[0098] The generation unit can estimate the user's emotions and adjust the story generation method based on the estimated emotions. For example, if the user is relaxed, the generation unit can generate a story that progresses at a leisurely pace. If the user is in a hurry, the generation unit can generate a story that emphasizes the shortest route. Furthermore, if the user is excited, the generation unit can generate a story with visually stimulating effects. In this way, the generation unit can adjust the story generation method according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generation AI. Generation AIs include, but are not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0099] The generation unit can adjust the level of detail in story generation based on the importance of each character. For example, it can provide detailed descriptions in scenes featuring the protagonist and simplify scenes featuring supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those of other characters. Furthermore, it can focus on depicting scenes featuring major characters while omitting background characters. In this way, the generation unit can adjust the level of detail in story generation based on the importance of each character.

[0100] The generation unit can apply different generation algorithms depending on the character category when generating a story. For example, the generation unit can apply an algorithm that provides detailed character descriptions to the main character. It can also apply an algorithm that provides simplified descriptions to supporting characters. Furthermore, it can apply an algorithm that creates a sense of tension to enemy characters. In this way, the generation unit can apply different generation algorithms depending on the character category.

[0101] The generation unit can estimate the user's emotions and adjust the story length based on the estimated emotions. For example, if the user is in a hurry, the generation unit can generate a short, concise story. If the user is relaxed, the generation unit can generate a longer story with detailed explanations. Furthermore, if the user is excited, the generation unit can generate a story with visually stimulating effects. In this way, the generation unit can adjust the story length according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generation AI. Generation AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0102] The generation unit can determine the priority of character creation based on when character settings were submitted during story generation. For example, the generation unit can prioritize incorporating character settings submitted early into the story. It can also simplify character settings submitted close to the deadline before incorporating them into the story. Furthermore, the generation unit can adjust the level of detail in the character settings depending on when they were submitted. This allows the generation unit to determine the priority of character creation based on when the character settings were submitted.

[0103] The generation unit can adjust the generation order based on the relevance of character settings when generating a story. For example, the generation unit can prioritize the incorporation of settings for major characters into the story. It can also group together highly relevant character settings into the story. Furthermore, it can postpone the incorporation of less relevant character settings into the story. In this way, the generation unit can adjust the generation order based on the relevance of character settings.

[0104] The editing unit can estimate the user's emotions and adjust the editing method based on the estimated emotions. For example, if the user is relaxed, the editing unit can provide detailed editing options. If the user is in a hurry, it can provide simplified editing options. Furthermore, if the user is excited, it can provide visually stimulating editing options. In this way, the editing unit can adjust the editing method according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0105] The correction unit can analyze the user's past correction history and suggest the optimal correction method during the correction process. For example, it can suggest similar correction methods based on the user's past corrections. It can also prioritize suggesting correction tools the user has used in the past. Furthermore, it can predict and suggest correction methods to be used during specific time periods based on the user's past correction history. This allows the correction unit to suggest the optimal correction method based on the user's past correction history.

[0106] The editing unit can customize the editing process based on the user's current areas of interest. For example, it can suggest relevant editing options based on genres the user has recently been interested in. It can also suggest similar editing methods based on the content of manga or anime the user has recently read. Furthermore, it can analyze the content of accounts the user follows on social media and suggest editing options based on their areas of interest. This allows the editing unit to customize the editing process based on the user's areas of interest.

[0107] The editing unit can estimate the user's emotions and determine the priority of corrections based on the estimated emotions. For example, if the user is excited, the editing unit can prioritize important corrections. If the user is relaxed, the editing unit can also prioritize detailed corrections. Furthermore, if the user is tired, the editing unit can prioritize simplified corrections. In this way, the editing unit can determine the priority of corrections according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0108] The correction unit can suggest the optimal correction method when making corrections, taking into account the user's geographical location. For example, if the user lives in a specific region, the correction unit can suggest correction options related to that region. Furthermore, if the user is traveling, the correction unit can suggest correction options related to the place they are visiting. Additionally, if the user is participating in a specific event, the correction unit can suggest correction options related to that event. This allows the correction unit to suggest the optimal correction method based on the user's geographical location.

[0109] The editing function can analyze the user's social media activity and suggest editing options during the editing process. For example, it can suggest relevant editing options based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest editing options based on their areas of interest. Furthermore, it can suggest relevant editing options based on the content of articles and videos the user has shared. In this way, the editing function can suggest editing options based on the user's social media activity.

[0110] The video generation unit can estimate the user's emotions and adjust the video generation method based on the estimated emotions. For example, if the user is relaxed, the video generation unit can generate a video that progresses at a leisurely pace. If the user is in a hurry, the video generation unit can generate a video that emphasizes the shortest route. Furthermore, if the user is excited, the video generation unit can generate a video with visually stimulating effects. In this way, the video generation unit can adjust the video generation method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0111] The video generation unit can adjust the level of detail generated based on the importance of the story during video generation. For example, it can provide detailed depictions in important scenes and simplify other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can provide simple depictions in everyday scenes and concentrate resources on important scenes. This allows the video generation unit to adjust the level of detail generated based on the importance of the story.

[0112] The video generation unit can apply different generation algorithms depending on the story category during video generation. For example, it can apply an algorithm that produces fast-paced action scenes. It can also apply an algorithm that emphasizes emotions in romance scenes. Furthermore, it can apply an algorithm that produces humorous scenes in comedy scenes. In this way, the video generation unit can apply different generation algorithms depending on the story category.

[0113] The video generation unit can estimate the user's emotions and adjust the length of the generated video based on the estimated emotions. For example, if the user is in a hurry, the video generation unit can generate a short, concise video. If the user is relaxed, the video generation unit can generate a longer video with detailed explanations. Furthermore, if the user is excited, the video generation unit can generate a video with visually stimulating effects. In this way, the video generation unit can adjust the length of the generated video according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0114] The video generation unit can determine the generation priority based on when the stories were submitted. For example, it can prioritize converting stories submitted early into videos. It can also simplify stories submitted close to the deadline before converting them into videos. Furthermore, the video generation unit can adjust the level of detail in the stories according to the submission date. This allows the video generation unit to determine the generation priority based on when the stories were submitted.

[0115] The video generation unit can adjust the generation order based on the relevance of the stories during video generation. For example, the video generation unit can prioritize converting the main story into a video. It can also convert highly relevant stories together into a video. Furthermore, the video generation unit can postpone the conversion of less relevant stories. In this way, the video generation unit can adjust the generation order based on the relevance of the stories.

[0116] The dialogue generation unit can estimate the user's emotions and adjust the dialogue generation method based on the estimated emotions. For example, if the user is relaxed, the dialogue generation unit can generate dialogue that proceeds at a leisurely pace. If the user is in a hurry, the dialogue generation unit can generate short, concise dialogue. Furthermore, if the user is excited, the dialogue generation unit can generate dialogue with visually stimulating effects. In this way, the dialogue generation unit can adjust the dialogue generation method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0117] The dialogue generation unit can adjust the level of detail in the dialogue based on the character's personality. For example, it can provide detailed descriptions for the protagonist's dialogue and simplify the dialogue for supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those for other characters. Furthermore, it can focus on describing scenes featuring main characters while omitting background characters. In this way, the dialogue generation unit can adjust the level of detail in the dialogue based on the character's personality.

[0118] The dialogue generation unit can apply different generation algorithms depending on the relationships between the characters when generating dialogue. For example, the dialogue generation unit can apply an algorithm that emphasizes emotions in scenes between the protagonist and the heroine. It can also apply an algorithm that creates tension in scenes with enemy characters. Furthermore, it can apply an algorithm that provides humorous descriptions in comedic scenes. In this way, the dialogue generation unit can apply different generation algorithms depending on the relationships between the characters.

[0119] The dialogue generation unit can estimate the user's emotions and adjust the length of the dialogue based on those emotions. For example, if the user is in a hurry, the dialogue generation unit can generate short, concise dialogue. Conversely, if the user is relaxed, it can generate longer dialogue that includes detailed explanations. Furthermore, if the user is excited, it can generate dialogue with visually stimulating effects. In this way, the dialogue generation unit can adjust the length of the dialogue according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0120] The dialogue generation unit can determine the priority of dialogue based on when the character settings were submitted. For example, the dialogue generation unit can prioritize the inclusion of character settings submitted early in the dialogue. It can also simplify character settings submitted close to the deadline before incorporating them into the dialogue. Furthermore, the dialogue generation unit can adjust the level of detail in the character settings according to the submission date. This allows the dialogue generation unit to determine the priority of dialogue based on when the character settings were submitted.

[0121] The dialogue generation unit can adjust the order of dialogue based on the relevance of character settings during the dialogue generation process. For example, the dialogue generation unit can prioritize reflecting the settings of main characters in the dialogue. It can also group together highly relevant character settings and reflect them in the dialogue. Furthermore, it can postpone reflecting less relevant character settings in the dialogue. In this way, the dialogue generation unit can adjust the order of dialogue based on the relevance of character settings.

[0122] The panel layout generation unit can estimate the user's emotions and adjust the panel layout generation method based on the estimated emotions. For example, if the user is relaxed, the panel layout generation unit can generate a panel layout that progresses at a leisurely pace. If the user is in a hurry, the panel layout generation unit can generate a short, concise panel layout. Furthermore, if the user is excited, the panel layout generation unit can generate a panel layout with visually stimulating effects. In this way, the panel layout generation unit can adjust the panel layout generation method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0123] The panel layout generator can adjust the level of detail in the panels based on the story's progression. For example, it can provide detailed depictions in important scenes and simplify other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can provide simple depictions in everyday scenes and concentrate resources on important scenes. In this way, the panel layout generator can adjust the level of detail in the panels based on the story's progression.

[0124] The panel layout generation unit can apply different generation algorithms depending on the importance of the scene during panel layout generation. For example, it can apply an algorithm that depicts fast-moving action to action scenes. It can also apply an algorithm that emphasizes emotions to romance scenes. Furthermore, it can apply an algorithm that depicts humor to comedy scenes. In this way, the panel layout generation unit can apply different generation algorithms depending on the importance of the scene.

[0125] The panel layout generation unit can estimate the user's emotions and adjust the length of the panels based on those emotions. For example, if the user is in a hurry, the panel layout generation unit can generate short, concise panels. If the user is relaxed, the panel layout generation unit can generate longer panels with detailed explanations. Furthermore, if the user is excited, the panel layout generation unit can generate panels with visually stimulating effects. In this way, the panel layout generation unit can adjust the length of the panels according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0126] The panel layout generation unit can determine the priority of panel layouts based on the submission timing of the stories. For example, it can prioritize incorporating stories submitted early into the panel layouts. It can also simplify stories submitted close to the deadline before incorporating them into the panel layouts. Furthermore, the panel layout generation unit can adjust the level of detail in the stories according to the submission timing. This allows the panel layout generation unit to determine the priority of panel layouts based on the submission timing of the stories.

[0127] The panel layout generation unit can adjust the order of panels based on the relevance of the story during panel generation. For example, the panel layout generation unit can prioritize reflecting the main story in the panel layout. It can also group highly related storylines together and reflect them in the panel layout. Furthermore, it can postpone reflecting less relevant storylines in the panel layout. In this way, the panel layout generation unit can adjust the order of panels based on the relevance of the story.

[0128] The interface unit can estimate the user's emotions and adjust the interface display method based on the estimated user emotions. For example, if the user is tense, the interface unit can provide an interface with calming colors to reduce visual stress. Conversely, if the user is enjoying themselves, the interface unit can provide an interface with bright colors to make the input process more enjoyable. Furthermore, if the user is tired, the interface unit can provide a simple and highly visible interface to facilitate the input process. Thus, the interface unit can adjust the interface display method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0129] The interface unit can select the optimal display method by referring to the user's past operation history when displaying the interface. For example, the interface unit can prioritize displaying interface designs that the user has used in the past. Furthermore, the interface unit can customize the interface based on the user's preferred colors and layouts. In addition, the interface unit can predict and suggest interface designs to be used during specific time periods based on the user's past operation history. This allows the interface unit to select the optimal display method based on the user's past operation history.

[0130] The interface unit can estimate the user's emotions and adjust the interface's operation procedures based on the estimated emotions. For example, if the user is tense, the interface unit can provide simple and intuitive operation procedures. If the user is enjoying themselves, the interface unit can provide detailed operation procedures to make the experience more enjoyable. Furthermore, if the user is tired, the interface unit can provide procedures that can be completed with minimal operation. In this way, the interface unit can adjust the interface's operation procedures according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0131] The interface unit can select the optimal display method when displaying the interface, taking into account the user's device information. For example, if the user is using a smartphone, the interface unit can provide a display method that matches the screen size. Furthermore, if the user is using a tablet, the interface unit can provide a display method optimized for a larger screen. Additionally, if the user is using a smartwatch, the interface unit can provide a concise and highly visible display method. In this way, the interface unit can select the optimal display method based on the user's device information.

[0132] The effect application unit can estimate the user's emotions and adjust the application method of effects based on the estimated emotions. For example, if the user is relaxed, the effect application unit can apply a gentle effect. If the user is excited, the effect application unit can also apply a visually stimulating effect. Furthermore, if the user is sad, the effect application unit can apply an effect that enhances that emotion. In this way, the effect application unit can adjust the application method of effects according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0133] The effects application unit can adjust the level of detail of an effect based on the importance of the scene when applying it. For example, it can apply detailed effects to important scenes and simplify them to other scenes. It can also add visually appealing effects to climactic scenes. Furthermore, it can apply simple effects to everyday scenes and concentrate resources on important scenes. In this way, the effects application unit can adjust the level of detail of an effect based on the importance of the scene.

[0134] The effects application unit can apply different effects depending on the scene category when applying effects. For example, it can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to romance scenes. Furthermore, it can apply humorous effects to comedy scenes. In this way, the effects application unit can apply different effects depending on the scene category.

[0135] The effect application unit can estimate the user's emotions and adjust the length of the effect based on the estimated emotions. For example, if the user is in a hurry, the effect application unit can apply a short, concise effect. If the user is relaxed, the effect application unit can apply a longer effect that includes detailed explanations. Furthermore, if the user is excited, the effect application unit can apply a visually stimulating effect. In this way, the effect application unit can adjust the length of the effect according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0136] The effects application unit can determine the priority of effects based on the scene submission date when applying effects. For example, the effects application unit can prioritize the application of effects to scenes submitted early. Furthermore, the effects application unit can simplify the effects applied to scenes submitted close to the deadline. In addition, the effects application unit can adjust the level of detail of the scene according to the submission date. This allows the effects application unit to determine the priority of effects based on the scene submission date.

[0137] The effect application unit can adjust the order of effects based on the relevance of the scenes when applying effects. For example, the effect application unit can prioritize applying effects to major scenes. It can also apply effects to highly relevant scenes together. Furthermore, the effect application unit can postpone applying effects to less relevant scenes. In this way, the effect application unit can adjust the order of effects based on the relevance of the scenes.

[0138] The customization unit can estimate the user's emotions and adjust the customization method based on the estimated emotions. For example, if the user is relaxed, the customization unit can provide detailed customization options. If the user is in a hurry, it can provide simplified customization options. Furthermore, if the user is excited, it can provide visually stimulating customization options. In this way, the customization unit can adjust the customization method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0139] The customization department can analyze the user's past customization history to propose the optimal customization method during the customization process. For example, it can suggest similar customization methods based on the user's past customizations. It can also prioritize suggesting customization tools the user has used in the past. Furthermore, it can predict and suggest customization methods to be used at specific times based on the user's past customization history. This allows the customization department to propose the optimal customization method based on the user's past customization history.

[0140] The customization unit can customize the means of customization based on the user's current areas of interest during the customization process. For example, the customization unit can suggest relevant customization options based on genres the user has recently been interested in. It can also suggest similar customization methods based on the content of manga or anime the user has recently read. Furthermore, the customization unit can analyze the content of accounts the user follows on social media and suggest customization options based on areas of interest. In this way, the customization unit can customize the means of customization based on the user's areas of interest.

[0141] The customization unit can estimate the user's emotions and determine the priority of customizations based on those emotions. For example, if the user is excited, the customization unit can prioritize important customizations. If the user is relaxed, the customization unit can prioritize detailed customizations. Furthermore, if the user is tired, the customization unit can prioritize simplified customizations. In this way, the customization unit can determine the priority of customizations according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0142] The customization unit can suggest the optimal customization method by considering the user's geographical location during the customization process. For example, if the user lives in a specific region, the customization unit can suggest customization options related to that region. Furthermore, if the user is traveling, the customization unit can suggest customization options related to the place they are visiting. In addition, if the user is participating in a specific event, the customization unit can suggest customization options related to that event. This allows the customization unit to suggest the optimal customization method based on the user's geographical location.

[0143] The customization department can analyze the user's social media activity during the customization process and suggest customization options. For example, it can suggest relevant customization options based on posts the user has recently "liked." It can also analyze the content of accounts the user follows and suggest customization options based on their areas of interest. Furthermore, it can suggest relevant customization options based on the content of articles and videos the user has shared. In this way, the customization department can suggest customization options based on the user's social media activity.

[0144] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0145] The generation unit can estimate the user's emotions and adjust the story's pace based on those emotions. For example, if the user is relaxed, it can generate a story that progresses at a leisurely pace. If the user is in a hurry, it can generate a short, concise story. Furthermore, if the user is excited, it can generate a story with visually stimulating effects. In this way, the generation unit can adjust the story's pace according to the user's emotions.

[0146] The generation unit can adjust the level of detail in story generation based on the relationships between characters. For example, it can provide detailed descriptions in scenes with main characters and simplify scenes with supporting characters. It can also generate detailed dialogue and expressions for important characters while simplifying those of other characters. Furthermore, it can focus on depicting scenes with main characters while omitting background characters. In this way, the generation unit can adjust the level of detail in story generation based on the relationships between characters.

[0147] The editing unit can estimate the user's emotions and determine the priority of modifications based on those emotions. For example, if the user is excited, important modifications can be prioritized. If the user is relaxed, detailed modifications can be prioritized. Furthermore, if the user is tired, simplified modifications can be prioritized. In this way, the editing unit can determine the priority of modifications according to the user's emotions.

[0148] The video generation unit includes an effects application unit that applies effects to the generated story. The effects application unit can, for example, add effects to match the scenes in the story. For example, the effects application unit can apply fast-moving effects to action scenes. It can also apply effects that emphasize emotions to emotional scenes. For example, the effects application unit can apply a sepia-toned effect to sad scenes. It can also apply bright-colored effects to happy scenes. Furthermore, the effects application unit can adjust the level of detail of the effects based on the importance of the scene. For example, it can apply detailed effects to important scenes and simplified effects to other scenes. In this way, the video generation unit can apply effects to the generated story.

[0149] The reception desk can estimate the user's emotions and adjust the timing of character setting and synopsis input based on those estimates. For example, if the user is excited, it can immediately prompt them to input character settings, allowing them to enter ideas while their emotions are heightened. If the user is relaxed, it can allow them to input the synopsis first and then proceed with character setting at a more leisurely pace. Furthermore, if the user is tired, a simplified input form can be provided to allow them to complete the input in a shorter time. In this way, the reception desk can adjust the input timing according to the user's emotions.

[0150] The generation unit can apply different generation algorithms depending on the character category when generating a story. For example, the main character can be given an algorithm that provides detailed character descriptions. Sub-characters can be given an algorithm that provides simplified descriptions. Furthermore, enemy characters can be given an algorithm that creates a sense of tension. In this way, the generation unit can apply different generation algorithms depending on the character category.

[0151] The reception desk can analyze a user's past input history and suggest the optimal input method. For example, it can automatically suggest similar settings based on character settings the user has entered in the past. It can also prioritize suggesting input methods the user has used in the past (voice, text, etc.). Furthermore, it can predict and suggest input methods that the user will use at specific times of the day based on their past input history. In this way, the reception desk can suggest the optimal input method based on the user's past input history.

[0152] The generation unit can estimate the user's emotions and adjust the story length based on those emotions. For example, if the user is in a hurry, it can generate a short, concise story. If the user is relaxed, it can generate a longer story with more detailed explanations. Furthermore, if the user is excited, it can generate a story with visually stimulating effects. In this way, the generation unit can adjust the story length according to the user's emotions.

[0153] The correction unit can analyze the user's past correction history and propose the optimal correction method during the correction process. For example, it can suggest similar correction methods based on the user's past corrections. It can also prioritize suggesting correction tools the user has used in the past. Furthermore, it can predict and suggest correction methods to be used during specific time periods based on the user's past correction history. In this way, the correction unit can propose the optimal correction method based on the user's past correction history.

[0154] The video generation unit can estimate the user's emotions and adjust the video generation method based on those emotions. For example, if the user is relaxed, it can generate a video that progresses at a leisurely pace. If the user is in a hurry, it can generate a video that emphasizes the shortest route. Furthermore, if the user is excited, it can generate a video with visually stimulating effects. In this way, the video generation unit can adjust the video generation method according to the user's emotions.

[0155] The following briefly describes the processing flow for example form 2.

[0156] Step 1: The reception desk accepts character settings and synopsis input. For example, it saves the character's name, personality, and background information entered by the user to a database, and saves the synopsis as text data. It also analyzes the information entered by the user and uses it as basic data for story generation. Step 2: The generation unit generates a story based on the information received by the reception unit. For example, it uses AI to generate character dialogue and facial expressions, and implements this using natural language processing and image generation technologies. It also automatically arranges the panels according to the progression of the story. Step 3: The editing unit presents the story generated by the generation unit to the user and accepts editing instructions. For example, it provides an interface and tools for the user to edit character dialogue, facial expressions, and panel layouts, and saves the edited content to use as data for generating the story again. Step 4: The video generation unit converts the story modified by the modification unit into a video. For example, it applies effects to the generated story and saves it as a video file. It also provides an interface for delivering the generated video to the user.

[0157] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0158] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0159] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements described above, including the reception unit, generation unit, modification unit, and video generation unit, is implemented in at least one of the smart device 14 and the data processing device 12. For example, the reception unit is implemented by the control unit 46A of the smart device 14 and stores the character settings and synopsis entered by the user in a database. The generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates character dialogue and facial expressions using AI. The modification unit is implemented by the control unit 46A of the smart device 14 and provides an interface for the user to check the generated story and give modification instructions. The video generation unit is implemented by the specific processing unit 290 of the data processing device 12 and converts the generated story into a video. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0161] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0162] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0163] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0164] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0165] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0167] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0168] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0169] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0170] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0171] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0172] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0173] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0174] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0175] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0176] Each of the multiple elements described above, including the reception unit, generation unit, modification unit, and video generation unit, is implemented in at least one of the smart glasses 214 and the data processing device 12. For example, the reception unit is implemented by the control unit 46A of the smart glasses 214 and stores the character settings and synopsis entered by the user in a database. The generation unit is implemented by the specific processing unit 290 of the data processing device 12 and generates character dialogue and facial expressions using AI. The modification unit is implemented by the control unit 46A of the smart glasses 214 and provides an interface for the user to check the generated story and give modification instructions. The video generation unit is implemented by the specific processing unit 290 of the data processing device 12 and converts the generated story into a video. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0177] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0178] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0179] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0180] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0181] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0182] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0183] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0184] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0185] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0186] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0187] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0188] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0189] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0190] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0191] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0192] Each of the multiple elements described above, including the reception unit, generation unit, modification unit, and video generation unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the headset terminal 314 and stores the character settings and synopsis entered by the user in a database. The generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates character dialogue and facial expressions using AI. The modification unit is implemented by the control unit 46A of the headset terminal 314 and provides an interface for the user to check the generated story and give modification instructions. The video generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and converts the generated story into a video. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0193] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0194] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0195] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0196] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0197] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0198] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0199] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0200] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0201] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0202] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0203] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0204] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0205] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0206] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0207] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0208] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0209] Each of the multiple elements described above, including the reception unit, generation unit, modification unit, and video generation unit, is implemented by, for example, at least one of the robot 414 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the robot 414 and stores the character settings and synopsis entered by the user in a database. The generation unit is implemented by, for example, the specific processing unit 290 of the data processing unit 12 and generates the character's lines and expressions using AI. The modification unit is implemented by, for example, the control unit 46A of the robot 414 and provides an interface for the user to check the generated story and give modification instructions. The video generation unit is implemented by, for example, the specific processing unit 290 of the data processing unit 12 and converts the generated story into a video. The correspondence between each unit and the device or control unit is not limited to the example described above and can be changed in various ways.

[0210] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0211] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0212] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0213] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0214] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0215] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0216] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0217] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0218] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0219] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0220] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0221] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0222] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0223] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0224] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0225] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0226] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0227] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0228] (Note 1) A reception desk that accepts input of character settings and synopsis, A generation unit that generates a story based on the information received by the reception unit, A revision unit presents the story generated by the generation unit to the user and accepts revision instructions. The system includes a video generation unit that converts the story modified by the modification unit into a video. A system characterized by the following features. (Note 2) The generating unit is It features a dialogue generation unit that generates character lines and facial expressions. The system described in Appendix 1, characterized by the features described herein. (Note 3) The generating unit is It includes a panel layout generation unit that generates panel layouts. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned modification section is, It includes an interface section for users to review generated stories and provide correction instructions. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned video generation unit, It includes an effects application section that applies effects to the generated story. The system described in Appendix 1, characterized by the features described herein. (Note 6) The generating unit is It features a customization section for entering different character settings and plot summaries. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of character settings and plot input based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned reception unit is It analyzes the user's past input history and suggests the optimal input method. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned reception unit is When entering character settings and plot summaries, filtering is performed based on the user's current areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned reception unit is It estimates the user's emotions and determines the priority of character settings and plot points to be entered based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned reception unit is When entering character settings and plot summaries, the system prioritizes inputting settings that are highly relevant, taking into account the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned reception unit is When users input character settings and plot summaries, the system analyzes their social media activity and suggests relevant settings. The system described in Appendix 1, characterized by the features described herein. (Note 13) The generating unit is It estimates the user's emotions and adjusts how the story is generated based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The generating unit is When generating a story, adjust the level of detail based on the importance of the characters. The system described in Appendix 1, characterized by the features described herein. (Note 15) The generating unit is When generating a story, different generation algorithms are applied depending on the character category. The system described in Appendix 1, characterized by the features described herein. (Note 16) The generating unit is It estimates the user's emotions and adjusts the story length based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The generating unit is When generating a story, the priority of generation is determined based on when the character settings were submitted. The system described in Appendix 1, characterized by the features described herein. (Note 18) The generating unit is When generating the story, the generation order is adjusted based on the relevance of the character settings. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned modification section is, It estimates the user's emotions and adjusts the correction method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned modification section is, When making corrections, we analyze the user's past correction history and suggest the optimal correction method. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned modification section is, During the modification process, the modification method is customized based on the user's current areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned modification section is, It estimates user sentiment and determines the priority of modifications based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned modification section is, When making corrections, we will suggest the optimal correction method considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned modification section is, During the correction process, we analyze the user's social media activity and propose corrective measures. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned video generation unit, It estimates the user's emotions and adjusts the video generation method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned video generation unit, When generating a video, adjust the level of detail based on the importance of the story. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned video generation unit, When generating videos, different generation algorithms are applied depending on the story category. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned video generation unit, It estimates the user's emotions and adjusts the length of video generation based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned video generation unit, When generating videos, the generation priority is determined based on when the stories were submitted. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned video generation unit, When generating videos, the generation order is adjusted based on the relevance of the story. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned dialogue generation unit, It estimates the user's emotions and adjusts the dialogue generation method based on the estimated user emotions. The system described in Appendix 2, characterized by the features described herein. (Note 32) The aforementioned dialogue generation unit, When generating dialogue, adjust the level of detail in the dialogue based on the character's personality. The system described in Appendix 2, characterized by the features described herein. (Note 33) The aforementioned dialogue generation unit, When generating dialogue, different generation algorithms are applied depending on the relationship between the characters. The system described in Appendix 2, characterized by the features described herein. (Note 34) The aforementioned dialogue generation unit, It estimates the user's emotions and adjusts the length of the dialogue based on those emotions. The system described in Appendix 2, characterized by the features described herein. (Note 35) The aforementioned dialogue generation unit, When generating dialogue, the priority of the lines is determined based on when the character settings were submitted. The system described in Appendix 2, characterized by the features described herein. (Note 36) The aforementioned dialogue generation unit, When generating dialogue, the order of lines is adjusted based on the relevance of the character settings. The system described in Appendix 2, characterized by the features described herein. (Note 37) The aforementioned panel layout generation unit, The system estimates the user's emotions and adjusts the panel layout generation method based on the estimated user emotions. The system described in Appendix 3, characterized by the features described herein. (Note 38) The aforementioned panel layout generation unit, When generating panel layouts, adjust the level of detail in the layouts based on the story's progression. The system described in Appendix 3, characterized by the features described herein. (Note 39) The aforementioned panel layout generation unit, When generating panel layouts, different generation algorithms are applied depending on the importance of each scene. The system described in Appendix 3, characterized by the features described herein. (Note 40) The aforementioned panel layout generation unit, It estimates the user's emotions and adjusts the length of the panel based on the estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Supplementary Note 41) When generating the frame division, the frame division generation unit determines the priority of the frame division based on the submission time of the story The system according to Supplementary Note 3, characterized in that (Supplementary Note 42) When generating the frame division, the frame division generation unit adjusts the order of the frame division based on the relevance of the story The system according to Supplementary Note 3, characterized in that (Supplementary Note 43) The interface unit estimates the user's emotion and adjusts the display method of the interface based on the estimated user's emotion The system according to Supplementary Note 4, characterized in that (Supplementary Note 44) The interface unit selects an optimal display method by referring to the user's past operation history when displaying the interface The system according to Supplementary Note 4, characterized in that (Supplementary Note 45) The interface unit estimates the user's emotion and adjusts the operation procedure of the interface based on the estimated user's emotion The system according to Supplementary Note 4, characterized in that (Supplementary Note 46) The interface unit selects an optimal display method by considering the user's device information when displaying the interface The system according to Supplementary Note 4, characterized in that (Supplementary Note 47) The effect application unit estimates the user's emotion and adjusts the application method of the effect based on the estimated user's emotion The system according to Supplementary Note 5, characterized in that (Supplementary Note 48) The effect application unit adjusts the detail level of the effect based on the importance of the scene when applying the effect The system described in Appendix 5, characterized by the features described herein. (Note 49) The aforementioned effect application unit, When applying effects, different effects are applied depending on the scene category. The system described in Appendix 5, characterized by the features described herein. (Note 50) The aforementioned effect application unit, It estimates the user's emotions and adjusts the length of the effect based on the estimated emotions. The system described in Appendix 5, characterized by the features described herein. (Note 51) The aforementioned effect application unit, When applying effects, prioritize effects based on when the scene was submitted. The system described in Appendix 5, characterized by the features described herein. (Note 52) The aforementioned effect application unit, When applying effects, the order of effects is adjusted based on their relevance in the scene. The system described in Appendix 5, characterized by the features described herein. (Note 53) The aforementioned customization unit is It estimates the user's emotions and adjusts the customization method based on the estimated user emotions. The system described in Appendix 6, characterized by the features described herein. (Note 54) The aforementioned customization unit is During customization, the system analyzes the user's past customization history to suggest the optimal customization method. The system described in Appendix 6, characterized by the features described herein. (Note 55) The aforementioned customization unit is During customization, the means of customization are tailored based on the user's current areas of interest. The system described in Appendix 6, characterized by the features described herein. (Note 56) The aforementioned customization unit is Estimate the user's emotion and determine the customization priority based on the estimated user emotion The system according to appendix 6, characterized in that it has the above function (Appendix 57) The customization unit Proposes an optimal customization method considering the user's geographical location information during customization The system according to appendix 6, characterized in that it has the above function (Appendix 58) The customization unit Analyzes the user's social media activities and proposes means for customization during customization The system according to appendix 6, characterized in that it has the above function

Explanation of reference numerals

[0229] 10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset-type terminal 414 Robot

Claims

1. A reception desk that accepts input of character settings and synopsis, A generation unit that generates a story based on the information received by the reception unit, A revision unit presents the story generated by the generation unit to the user and accepts revision instructions. The system includes a video generation unit that converts the story modified by the modification unit into a video. A system characterized by the following features.

2. The generating unit is It features a dialogue generation unit that generates character lines and facial expressions. The system according to feature 1.

3. The generating unit is It includes a panel layout generation unit that generates panel layouts. The system according to feature 1.

4. The aforementioned modification section is, It includes an interface section for users to review generated stories and provide correction instructions. The system according to feature 1.

5. The aforementioned video generation unit, It includes an effects application section that applies effects to the generated story. The system according to feature 1.

6. The generating unit is It features a customization section for entering different character settings and plot summaries. The system according to feature 1.

7. The aforementioned reception unit is The system estimates the user's emotions and adjusts the timing of character settings and plot input based on those estimated emotions. The system according to feature 1.

8. The aforementioned reception unit is It analyzes the user's past input history and suggests the optimal input method. The system according to feature 1.

9. The aforementioned reception unit is When entering character settings and plot summaries, filtering is performed based on the user's current areas of interest. The system according to feature 1.

Images