system

Abstract

To provide a system.SOLUTION: A system includes: means for inputting a story setting containing information on a theme, a stage and characters of a story; means for automatically creating a composition of a story on the basis of the inputted story setting; means for generating elements corresponding to the created composition of the story; means for automatically producing graphics and music corresponding to the story; means for recognizing a user's feelings; and means for adjusting the scenes and music of the story in accordance with the user's feelings recognized.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Creating a story involves many intertwining elements, such as story structure, character dialogue, scenic descriptions, plot twists, and creating moving scenes. Only by properly combining these elements can an interesting story emerge. However, creating all of these elements by hand takes a great deal of time and effort. Furthermore, creating graphics and music to accompany a story requires specialized knowledge and skills, making it difficult for one person to do all of this alone. [Means for solving the problem]

[0005] This invention automatically creates character dialogue, scenery, and story structure when story settings such as the story's theme, setting, and characters are input. It also automatically generates plot twists, mystery elements, and moving scenes to make the story more interesting. It then automatically creates graphics and music that match each scene in the story. This enhances the dramatic effect of the story structure, character dialogue, images, and music, effectively stirring the viewer's emotions. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention receives input from the user about the story setting, such as the theme, setting, and characters. This input is provided in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenery, and a story structure. This creation is performed using existing story databases and artificial intelligence technology, generating dialogue and scenery that match the story flow, character personalities, and theme.

[0029] "Example 2"

[0030] Furthermore, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done in accordance with the story's theme, the characters' personalities, and the progression of the story, laying hints at the right time, sprinkling mystery elements, and creating moving scenes. For example, it automatically generates hints that people the knight meets during his adventure will play an important role in the story later on, elements that the knight must solve, and moving scenes when the knight overcomes difficulties.

[0031] "Example 3"

[0032] The system then automatically creates graphics and music that match each scene in the story. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story structure, character dialogue, images, and music, effectively stirring the emotions of the viewer.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: The user provides textual input of the story setting, such as the theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

[0036] Step 2: Based on the input, the system automatically generates character dialogue, scenery, and story structure, using existing story databases and artificial intelligence techniques.

[0037] Step 3: Lines and scenes are generated that fit the story flow, the characters' personalities, and the story's theme.

[0038] "Example 2"

[0039] Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and other elements to make the story more interesting.

[0040] Step 2: Lay out hints at the right time, sprinkle in elements of mystery, and create moving scenes in line with the story's theme, the characters' personalities, and the progression of the story.

[0041] Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing an important role in the story later on, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

[0042] "Example 3"

[0043] Step 1: The system automatically creates graphics and music that match each scene in the story you create.

[0044] Step 2: Graphics and music are created taking into account the mood of the scene and the emotional state of the characters.

[0045] Step 3: For example, powerful graphics and music that heightens the tension are produced for battle scenes, while beautiful landscape graphics and moving music are produced for moving scenes. This enhances the dramatic effect of the story structure, the characters' conversations, images and music, and effectively stirs the viewer's emotions.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] With conventional story generation systems, when automatically generating a story based on a story setting entered by the user, it is difficult to maintain consistency in the flow of the story, the dialogue of the characters, and the scenery. Furthermore, there is a lack of ingenuity to ensure that the generated story is moving and entertaining for the viewer. Furthermore, there is an issue that it is not possible to automatically create graphics and music that match each scene of the story, making it difficult to effectively move the viewer's emotions.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect of the story structure, character dialogue, and video and music to effectively stir the viewer's emotions; means for receiving input from a user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into a generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

[0051] The "theme of a story" refers to the central subject or theme of a story, and is an element that determines the overall direction and atmosphere of the story.

[0052] The "setting" refers to the place and historical background in which the story unfolds, and is an important element in forming the setting and atmosphere of the story.

[0053] A "character" is someone who acts in a story and moves the story forward, and is an element that has a major influence on the development and theme of the story.

[0054] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and is the element that forms the framework of the story.

[0055] "Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[0056] "Scenes" refer to the landscapes and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

[0057] "Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the development of a story effective.

[0058] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that adds consistency and depth to a story.

[0059] A "mystery element" is an element that presents a mystery or question in a story to attract the interest of readers or viewers.

[0060] A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

[0061] "Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotions of the story.

[0062] "Songs" are melodies and sounds that are musically expressed to match each scene in the story, and are elements that musically complement the atmosphere and emotions of the story.

[0063] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

[0064] A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

[0065] A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content that the model generates.

[0066] "Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read before it is presented to the user.

[0067] This invention is a system that automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[0068] User input

[0069] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe."

[0070] The server receives and analyzes the input data

[0071] The server receives the text data sent by the user. It then analyzes the input data using natural language processing technology. Specifically, it uses the Python (registered trademark) NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

[0072] Generate prompt statement

[0073] The server generates a prompt to be input to the generative AI model based on the extracted information. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

[0074] Generating stories with generative AI models

[0075] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4 (registered trademark)). The generative AI model automatically generates the lines, scenery, and structure of a story based on the prompt. For example, it generates text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

[0076] Formatting and serving the generated results

[0077] The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

[0078] Specific examples

[0079] Consider the user's input, "A knight's adventure set in medieval Europe." Based on this input, the server generates the following prompt:

[0080] "Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

[0081] The generative AI model generates a story based on this prompt, outputting text such as the following:

[0082] "The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

[0083] In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

[0084] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0085] Step 1:

[0086] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe." This input data is then sent to the server.

[0087] Step 2:

[0088] The server receives the text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

[0089] Step 3:

[0090] The server generates a prompt sentence to be input to the generative AI model based on the analysis results. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt sentence would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt sentence becomes the input for the next step.

[0091] Step 4:

[0092] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it generates text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'" This generated text becomes the input for the next step.

[0093] Step 5:

[0094] The server receives the narrative text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted narrative is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

[0095] Step 6:

[0096] The server provides the formatted story to the user, who can then view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

[0097] (Application example 1)

[0098] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0099] Conventional story generation systems automatically generate stories based on themes and settings entered by users, but the generation of story structure, character dialogue, and scenery is insufficient, making it difficult to effectively move the viewer's emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories results in poor user convenience.

[0100] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0101] In this invention, the server includes: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating character lines, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user and effectively stir the viewer's emotions.

[0102] A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

[0103] The "setting" refers to the place and historical background in which the story unfolds, and is an element that forms the setting and atmosphere of the story.

[0104] "Characters" are characters that appear in a story and play an important role in the progression and development of the story.

[0105] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and forms the framework of the story.

[0106] "Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[0107] "Scenes" are the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

[0108] "Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

[0109] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

[0110] "Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

[0111] A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer.

[0112] "Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

[0113] A "song" is music created to accompany each scene in a story, and is intended to musically express the atmosphere and emotions of the story.

[0114] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

[0115] A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

[0116] "Preview" is a function that allows users to check the generated story in advance and check the content.

[0117] "Preservation" refers to recording the generated stories in a digital format so that they can be reused and shared later.

[0118] "Sharing" refers to sharing the generated story with other users and platforms, in order to promote the spread and appreciation of the story.

[0119] A system for carrying out this invention automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[0120] System Configuration

[0121] This system is mainly composed of the following hardware and software.

[0122] Hardware: Smartphones, servers

[0123] Software: Python, OpenAI API

[0124] Program processing

[0125] User Input

[0126] Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they can enter the theme "A knight's adventure story set in medieval Europe."

[0127] Prompt Generation

[0128] The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input to the generative AI model. As a specific example, the following prompt is generated:

[0129] Theme: A knight's adventure set in medieval Europe

[0130] Setting: Medieval Europe

[0131] Characters: Brave knights, wizards, and dragons

[0132] Generate a story.

[0133] Story Generation

[0134] The server automatically generates a story based on the generated prompt sentence using a generative AI model (e.g., GPT-4). This generative AI model is called through the OpenAI API.

[0135] Preview, save, and share

[0136] The generated story is displayed for users to preview through the smartphone application, and users can save the story and share it on social media.

[0137] Specific examples

[0138] If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

[0139] Theme: A knight's adventure set in medieval Europe

[0140] Setting: Medieval Europe

[0141] Characters: Brave knights, wizards, and dragons

[0142] Generate a story.

[0143] By inputting this prompt into a generative AI model, a story is automatically generated, which users can preview, save, and share.

[0144] Hardware and software used

[0145] Hardware: Smartphones, servers

[0146] Software: Python, OpenAI API

[0147] This makes it possible to automatically generate high-quality stories based on the settings entered by the user, effectively stirring the emotions of the viewer.

[0148] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0149] Step 1:

[0150] The user uses a smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

[0151] Step 2:

[0152] The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt suitable for the generative AI model. The generated prompt looks like this:

[0153] Theme: A knight's adventure set in medieval Europe

[0154] Setting: Medieval Europe

[0155] Characters: Brave knights, wizards, and dragons

[0156] Generate a story.

[0157] This prompt sentence is generated.

[0158] Step 3:

[0159] The server calls a generative AI model (e.g., GPT-4) with the generated prompt sentence. It sends the prompt sentence using the OpenAI API and requests the generation of a story. The input is the prompt sentence, and the output is the generated story text.

[0160] Step 4:

[0161] The generative AI model automatically generates a story based on the prompt sentence. The generated story text is returned to the server. For example, the following story is generated:

[0162] Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight named Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

[0163] Step 5:

[0164] The server sends the generated story text to the smartphone terminal, and the user can preview the generated story through the smartphone terminal. The input is the generated story text, and the output is the preview screen displayed on the user's terminal.

[0165] Step 6:

[0166] Users can save the created story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

[0167] By following the steps above, it is possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the emotions of the viewer.

[0168] Example 2

[0169] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] Conventional story generation systems can automatically generate stories based on themes and character settings, but they struggle to properly generate plot twists, mystery elements, and moving scenes to make a story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively utilize the generated story. Furthermore, they lack the means to enhance the dramatic effect of story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0172] In this invention, the server includes means for inputting story settings such as the theme, setting, and characters of the story, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for organizing the generated story elements and providing them to users, and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewers' emotions. This makes it possible to make the story more interesting and enable users to effectively use the created story.

[0173] A "story theme" refers to the central subject or theme of a story.

[0174] "Setting" refers to the place or environment in which the story unfolds.

[0175] "Characters" refers to the characters and people who appear in a story.

[0176] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[0177] "Dialogue" refers to the words or lines spoken by characters.

[0178] "Scene" refers to the scenery or scene depicted in the story.

[0179] "Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

[0180] A "foreshadowing" is an element or event placed in the first half of a story in such a way that it will have an important meaning later on.

[0181] "Mystery elements" refer to elements that add mystery or detective elements to a story, thereby attracting the interest of readers and viewers.

[0182] A "moving scene" is a scene in a story that stirs strong emotions.

[0183] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[0184] "Song" refers to the music or background music that plays along with the story scenes.

[0185] "Staging effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and responses from the viewer.

[0186] "Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

[0187] This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the personalities of the characters, and the story's progression. A specific embodiment of this system is described below.

[0188] System configuration

[0189] Hardware

[0190] The server uses a cloud server equipped with a high-performance GPU.

[0191] software

[0192] The server uses a generative AI model, specifically one such as OpenAI's GPT-4.

[0193] Program processing

[0194] User Input

[0195] The user uses the terminal to input information about the theme of the story, the personalities of the characters, and the progress of the story. For example, the user may input the following information:

[0196] Theme: Medieval Knight's Adventure

[0197] Characters: A brave knight, a mysterious old man, and a princess

[0198] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[0199] Generate prompt statement

[0200] The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

[0201] Theme: Medieval Knight's Adventure

[0202] Characters: A brave knight, a mysterious old man, and a princess

[0203] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[0204] Generating story elements

[0205] The server inputs the generated prompt sentence into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt sentence.

[0206] Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

[0207] Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

[0208] Emotional Scene: The knight saves the princess and fights a final battle to protect the kingdom.

[0209] Organizing and presenting generated story elements

[0210] The server organizes the story elements output by the generative AI model and provides them to the user, who can then review the generated story elements via their device and make corrections or additions as needed.

[0211] Specific examples

[0212] Below is a concrete example of how a user can use the system to generate a story.

[0213] Theme: Medieval Knight's Adventure

[0214] Characters: A brave knight, a mysterious old man, and a princess

[0215] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[0216] The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

[0217] In this way, it is possible to make the story more interesting and enable the user to effectively use the generated story.

[0218] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0219] Step 1:

[0220] The user inputs information about the story's theme and characters.

[0221] Input: Information about the story's theme, character personalities, and story progression.

[0222] Specific operation: The user enters the theme "Adventures of a Medieval Knight" and the characters "A Brave Knight, a Mysterious Old Man, and a Princess" into the input form on the terminal. When the input is complete, the user clicks the "Submit" button.

[0223] Output: The entered information is sent to the server.

[0224] Step 2:

[0225] The server receives the input information and generates a prompt.

[0226] Input: User-submitted information about the story's theme, character personalities, and story progression.

[0227] Specific operation: Based on the received information, the server generates a prompt sentence to be input into the generative AI model.

[0228] Output: The generated prompt statement.

[0229] Theme: Medieval Knight's Adventure

[0230] Characters: A brave knight, a mysterious old man, and a princess

[0231] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[0232] Step 3:

[0233] The server inputs prompts into the generative AI model to generate story elements.

[0234] Input: The generated prompt statement.

[0235] Specific operation: The server inputs prompt text into the generative AI model to generate plot twists, mystery elements, and moving scenes.

[0236] Output: Generated story elements.

[0237] Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

[0238] Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

[0239] Emotional Scene: The knight saves the princess and fights a final battle to protect the kingdom.

[0240] Step 4:

[0241] The server organizes the generated story elements and provides them to the user.

[0242] Input: Story elements output from a generative AI model.

[0243] Specific operation: The server organizes the generated story elements and provides them to the user. The user can check the generated story elements through their terminal and make corrections or additions as necessary.

[0244] Output: The organized story elements are presented to the user.

[0245] (Application example 2)

[0246] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0247] While conventional story generation systems can automatically create stories based on themes and character settings, they struggle to properly generate plot twists, mystery elements, and moving scenes to make a story more interesting. They also struggle to automatically create graphics and music that match each scene in the generated story and effectively sway the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompts and input them into the model, which poses a challenge in providing a satisfying story to users.

[0248] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0249] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions, means for generating story elements using a generative AI model, means for generating prompt sentences and inputting them into the generative AI model, and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

[0250] A "story theme" is the central subject or theme of a story.

[0251] The "setting" is the place or environment in which the story unfolds.

[0252] "Characters" are the characters that appear in the story.

[0253] "Story setting" refers to the basic elements of a story, such as its theme, setting, and characters.

[0254] "Dialogue" refers to the words spoken by the characters.

[0255] "Scene" refers to the scenery or scene depicted in the story.

[0256] "Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

[0257] A "foreshadowing" is the advance indication of an important element related to later developments.

[0258] A "mystery element" is the inclusion of a mystery or mysterious event in a story.

[0259] A "moving scene" is one that strongly stirs the emotions of the viewer.

[0260] "Graphics" refers to visual images or illustrations.

[0261] A "song" is a piece of music or melody.

[0262] "Stage effects" are visual and auditory devices used to make a story more appealing.

[0263] The "audience" is the person who watches the story.

[0264] A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[0265] A "prompt sentence" is an instruction sentence to be input into a generative AI model.

[0266] "User" means a person who uses the system.

[0267] A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the emotions of the viewer, means for generating story elements using a generative AI model, means for generating prompt sentences and inputting them into the generative AI model, and means for displaying the generated story elements to the user.

[0268] The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3 (registered trademark)) to generate story elements. The generated story elements are then displayed to the user.

[0269] Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

[0270] The hardware used includes servers and user devices, and the software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

[0271] As a concrete example, the following prompt sentence is generated:

[0272] Theme: Knight's Adventure

[0273] Characters: Brave knight, mysterious old man, wizard

[0274] Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

[0275] By inputting this prompt into a generative AI model, story elements are automatically generated. These story elements are then displayed to the user, enhancing the story structure, character dialogue, visuals, and music for dramatic effect.

[0276] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0277] Step 1:

[0278] The user inputs information about the story's theme and characters.

[0279] Input: Story theme, character information

[0280] Output: Input theme and character information

[0281] Specific operation: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

[0282] Step 2:

[0283] The server receives the input information about the theme and characters and generates a prompt sentence.

[0284] Input: Theme and character information entered

[0285] Output: Generated prompt statement

[0286] Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

[0287] Step 3:

[0288] The server inputs the generated prompt sentence into the generative AI model.

[0289] Input: Generated prompt statement

[0290] Output: Story elements generated by the generative AI model

[0291] Specific operation: The server sends the generated prompt sentence to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, and moving scenes).

[0292] Step 4:

[0293] The server receives the generated story elements and displays them to the user.

[0294] Input: Story elements generated by a generative AI model

[0295] Output: The story elements displayed to the user

[0296] Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user, who can then review the generated story elements.

[0297] Step 5:

[0298] The server automatically creates graphics and music that match each scene in the story.

[0299] Input: Generated story elements

[0300] Output: Graphics and music that match each scene

[0301] Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

[0302] Step 6:

[0303] The server enhances the dramatic effect through the story structure, character dialogue, images and music.

[0304] Input: Graphics, music, and story elements that match each scene

[0305] Output: A narrative with enhanced effects

[0306] Specific operation: The server combines the story structure, the characters' dialogue, images and music to enhance the production effect, effectively stirring the viewer's emotions.

[0307] Example 3

[0308] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] Conventional story generation systems have difficulty automatically creating graphics and music that match each scene in a story, and have been unable to enhance the dramatic effect of effectively swaying the viewer's emotions. In addition, since graphics and music that take into account the atmosphere of the scene and the emotional state of the characters are generated manually, this is time-consuming and labor-intensive.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0311] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt statements to be input to a generative AI model based on the extracted information; means for inputting the prompt statements to the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data to generate a final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and enhance the dramatic effect that effectively stirs the viewer's emotions.

[0312] "Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

[0313] "Input means" refers to the interface that allows the user to provide the story setting, such as the theme, setting, and characters, to the system.

[0314] "Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of the story based on the input story setting.

[0315] "Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

[0316] "Means for automatically creating graphics and music" refers to the ability to automatically generate visual and auditory elements to match each scene in a story.

[0317] "Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

[0318] "Means for generating prompt sentences to be input to the generative AI model" refers to a function that creates text to provide appropriate instructions to the generative AI model based on analyzed scene data.

[0319] "A means for inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to the function of inputting a prompt sentence into a generative AI model and, as a result, generating graphics and music that are appropriate for the scene.

[0320] "Means for integrating generated graphics and music into scene data and producing final output" refers to functionality that incorporates generated visual and auditory elements into scene data to create final narrative output.

[0321] "Means of effectively stirring the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the structure of the story, the dialogue of the characters, images, and music.

[0322] This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates plot twists, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

[0323] Hardware and software used

[0324] The system uses the following hardware and software:

[0325] Server: Analyzes data and runs generative AI models.

[0326] Terminal: Provides an interface for users to input the story setting of a story.

[0327] Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

[0328] Data processing and calculation

[0329] The server receives the story settings entered by the user through the device. The story settings include the theme, setting, and character traits of the story. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

[0330] Next, the server automatically generates plot twists, mystery elements, and moving scenes based on the story, making the story more interesting and moving for viewers.

[0331] The server then analyzes the scene data to automatically create graphics and music that match each scene in the story. The analysis extracts the scene's atmosphere and the characters' emotional states. Based on the extracted information, the server generates prompts to be input into the generative AI model.

[0332] Specific examples

[0333] For example, in the case of an emotional scene, the server might input the following prompt sentence into the generative AI model:

[0334] Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional and the character is in tears."

[0335] Example prompt for music generation: "Generate moving, poignant music. The scene is emotional and the characters are in tears."

[0336] The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The terminal displays the integrated results to the user.

[0337] In this way, the user can automatically create graphics and music that match each scene of the story, thereby enhancing the dramatic effect that effectively stirs the emotions of the viewer. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0338] Step 1:

[0339] The user inputs the story setting, such as the theme, setting, and characters, into the terminal.

[0340] Specific operation: The user enters information such as the story theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into an input form on the device. The device then sends the entered story settings to the server.

[0341] Input: Story setting such as theme, setting, characters, etc.

[0342] Output: Story setting data sent to the server

[0343] Step 2:

[0344] The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

[0345] Specific operation: The server uses natural language processing (NLP) techniques to analyze the story setting data and generate character dialogue, scene descriptions, and the overall flow of the story.

[0346] Input: Story setting data

[0347] Output: Character dialogue, scenery, and story structure data

[0348] Step 3:

[0349] Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

[0350] Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

[0351] Input: Story composition data

[0352] Output: Story data including hints, mystery elements, and moving scenes

[0353] Step 4:

[0354] The server analyzes the scene data to automatically create graphics and music that match each scene in the story.

[0355] Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

[0356] Input: Narrative data

[0357] Output: Scene atmosphere and characters' emotional state data

[0358] Step 5:

[0359] The server generates prompt sentences to input into the generative AI model based on the extracted information.

[0360] Specific operation: The server creates a prompt sentence according to the atmosphere of the scene and the emotional state of the characters. For example, in the case of a touching scene, the server generates a prompt sentence such as "Generate a graphic of a beautiful landscape. The scene is touching, and the characters are crying."

[0361] Input: Scene atmosphere and character emotional state data

[0362] Output: A prompt to be input to the generative AI model

[0363] Step 6:

[0364] The server inputs prompt text into the generative AI model and generates graphics and music that match the scene.

[0365] Specific operation: The server sends prompts to the generative AI model (e.g., DALL-E) to generate graphics that match the scene. Similarly, it sends prompts to the music generation model to generate music that matches the scene.

[0366] Input: A prompt to be input to the generative AI model

[0367] Output: Generated graphics and music

[0368] Step 7:

[0369] The server integrates the generated graphics and music with the scene data to generate the final output.

[0370] Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

[0371] Input: Generated graphics, music, and scene data

[0372] Output: Final integrated narrative data

[0373] Step 8:

[0374] The terminal displays the final integrated story data to the user.

[0375] Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

[0376] Input: Final consolidated narrative data

[0377] Output: The visual and auditory content of the story that is displayed to the user

[0378] (Application example 3)

[0379] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Conventional story generation systems have difficulty automatically creating graphics and music that match each scene in the story, limiting the visual and auditory effects they can produce. Even when users input their own stories, there is a lack of means to generate graphics and music that are appropriate for that story, which leads to a problem of reduced content quality. Furthermore, the lack of functionality to preview and export the generated content makes it less convenient for users.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0382] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions, means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters, means for generating graphics based on scene descriptions, means for generating music based on scene descriptions, means for previewing the generated content, and means for exporting the completed content. This makes it possible to automatically generate visually and aurally rich content based on a story input by a user, and preview and export it.

[0383] A "story theme" refers to the central subject or message of a story.

[0384] "Setting" refers to the place or environment in which the story unfolds.

[0385] "Characters" refers to the characters that appear in the story.

[0386] "Story setting" refers to basic elements such as the story's theme, setting, and characters.

[0387] "Dialogue" refers to the words spoken by the characters.

[0388] "Scene" refers to the scenery or scene depicted in the story.

[0389] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[0390] A "foreshadowing" is an element that provides advance information or events that will become important later in the story.

[0391] "Mystery elements" refer to elements that add mystery or suspense to a story.

[0392] A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

[0393] "Graphics" refers to visual images and illustrations generated to accompany the story scenes.

[0394] "Song" refers to music generated to match the scenes in the story.

[0395] "Mood of the scene" refers to the overall feeling or mood of a particular scene in a story.

[0396] "Character emotional state" refers to the emotions that the characters are feeling in the story.

[0397] A "scene description" is a detailed description or explanation of a particular moment in a story.

[0398] "Preview" refers to checking the generated content in advance.

[0399] "Export" refers to saving the generated content as an external file.

[0400] A system for implementing this invention includes means for inputting story settings such as the theme, setting, and characters of the story, means for automatically creating character lines and scenes and a story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically producing graphics and music that match each scene of the created story, means for enhancing the story structure, character conversations, and the dramatic effect of using images and music to effectively stir the emotions of viewers, means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters, means for generating graphics based on a description of the scene, means for generating music based on a description of the scene, means for previewing the generated content, and means for exporting the completed content.

[0401] Hardware and Software Configuration

[0402] Hardware:

[0403] Smartphone

[0404] software:

[0405] Python

[0406] OpenAI API

[0407] PIL (Python Imaging Library)

[0408] Music Generation Library

[0409] Data processing and calculation

[0410] Ways to enter story settings such as theme, setting, and characters:

[0411] Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

[0412] How to automatically generate character dialogue, scenes, and story structure:

[0413] Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

[0414] How to automatically generate plot twists, mystery elements, and moving scenes:

[0415] The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

[0416] How to automatically create graphics and music that match each scene in the story:

[0417] The server analyzes each scene in the story and extracts the mood of the scene and the emotional state of the characters, based on which it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

[0418] To preview the generated content:

[0419] Through a smartphone application, users can preview each scene of the story, including the generated graphics and music.

[0420] To export your finished content:

[0421] Users can export the completed content as a video file through a smartphone application.

[0422] Specific examples

[0423] For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

[0424] Example prompt sentence:

[0425] Analyze the following story scene and print the mood of the scene and the emotional state of the characters:

[0426] The hero climbed the mountain to fight the dragon.

[0427] In this way, users can enjoy their stories as visually and aurally rich content.

[0428] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0429] Step 1:

[0430] Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

[0431] Input: Story setting, including theme, setting, and characters

[0432] Output: Story setting data in text format

[0433] Step 2:

[0434] Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

[0435] Input: Story setting data in text format

[0436] Output: Character dialogue, scenery, and story structure data

[0437] Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

[0438] Step 3:

[0439] The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

[0440] Input: Character dialogue, scenery, and story structure data

[0441] Output: Foreshadowing, mystery elements, and moving scene data

[0442] Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

[0443] Step 4:

[0444] The server analyzes each scene in the story and extracts the mood of the scene and the emotional state of the characters.

[0445] Input: Foreshadowing, mystery elements, and moving scene data

[0446] Output: Scene mood, characters' emotional states

[0447] Specific operation: The server sends a prompt to the generative AI model and receives the analysis result.

[0448] Step 5:

[0449] The server generates graphics based on the scene description.

[0450] Input: Scene mood, character emotional state data

[0451] Output: Graphic data

[0452] Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

[0453] Step 6:

[0454] The server generates music based on the scene description.

[0455] Input: Scene mood, character emotional state data

[0456] Output: Music data

[0457] Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

[0458] Step 7:

[0459] Through a smartphone application, users can preview each scene of the story, including the generated graphics and music.

[0460] Input: Graphic data, music data

[0461] Output: Preview screen

[0462] What it does: The smartphone application displays the generated data and gives the user a preview.

[0463] Step 8:

[0464] Users can export the completed content as a video file through a smartphone application.

[0465] Input: Preview screen

[0466] Output: Video file

[0467] Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

[0468] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0469] "Example 1"

[0470] One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Then, based on the story settings, it automatically creates character dialogue, scenery, and story structure. Furthermore, it automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

[0471] "Example 2"

[0472] Furthermore, the system includes an emotion engine that recognizes the user's emotions in real time and adjusts the story development according to the emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

[0473] "Example 3"

[0474] The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine recognizes that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine recognizes that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine recognizes that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

[0475] The processing flow of each embodiment will be described below.

[0476] "Example 1"

[0477] Step 1: Receive story settings such as the story theme, setting, and characters as input from the user.

[0478] Step 2: Based on the received story setting, the system automatically creates character dialogue, scenes, and a story structure.

[0479] Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make the story more interesting.

[0480] Step 4: Automatically create graphics and music that match each scene of the story you've created. Step 5: Enhance the story structure, character dialogue, video, and music to effectively stir the viewer's emotions.

[0481] "Example 2"

[0482] Step 1: Activate the emotion engine and recognize the user's emotions in real time.

[0483] Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will be more cheerful. Conversely, if the user feels sad about the story, the story will be more profound.

[0484] "Example 3"

[0485] Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

[0486] Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy, and if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

[0487] Step 3: If the user further inputs the text "fun," the emotion engine recognizes that the user is feeling fun. Based on this information, the emotion engine adjusts the story development and provides a story experience that corresponds to the user's emotions.

[0488] Example 1

[0489] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0490] In conventional story generation systems, when automatically generating a story based on a story setting entered by a user, consistency in the flow of the story, the dialogue of the characters, and the scenery often lacked. It was also difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it impossible to effectively stir the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. In order to solve these problems, the present invention aims to provide a system that generates a more consistent story and effectively stirs the viewer's emotions.

[0491] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0492] In this invention, the server includes means for inputting story settings such as the theme, setting, and characters of the story, means for analyzing the input story settings, means for searching an existing story database based on the analyzed story settings, means for generating and inputting prompt sentences to a generation AI model, means for automatically creating character lines, scenes, and a story structure using the generation AI model, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user and effectively stir the viewer's emotions.

[0493] A "story theme" is the central subject or theme of a story.

[0494] The "setting" refers to the place and historical background in which the story unfolds.

[0495] "Characters" are the characters or people who appear in a story.

[0496] "Story setting" refers to the setting of a story, including its basic elements such as theme, setting, and characters.

[0497] "Input means" refers to a method or device that allows a user to provide the story setting, such as the theme, setting, and characters of the story, to the system.

[0498] The "analyzing means" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

[0499] The "means for searching a database" refers to a method or device for searching relevant information from an existing story database based on the analyzed story setting.

[0500] A "generative AI model" is a model that uses artificial intelligence technology to generate text or stories.

[0501] A "prompt" is an instruction or question that is input into a generative AI model.

[0502] "Means for generating and inputting prompt sentences" refers to a method or device for creating a prompt sentence suitable for a generative AI model and inputting it into the model.

[0503] "Dialogue" refers to the words or lines spoken by characters.

[0504] "Scene" refers to the scenery or scene depicted in the story.

[0505] "Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

[0506] A "foreshadowing" is an element or hint placed in the first half of a story that will have important meaning later on.

[0507] "Mystery elements" are elements that add mystery and suspense to a story.

[0508] A "moving scene" is one that strongly stirs the emotions of the viewer.

[0509] "Graphics" refers to images and illustrations generated to match each scene in the story.

[0510] "Songs" refer to music and melodies that are generated to match each scene in the story.

[0511] "Production effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

[0512] The present invention relates to a story generation system, and aims to generate a consistent story based on a story setting input by a user, and to effectively stir the emotions of viewers.

[0513] System configuration

[0514] User Input

[0515] The user inputs the story setting, such as the theme, setting, and characters, in text format. For example, the user can provide a specific setting such as "a knight's adventure story set in medieval Europe."

[0516] Server input parsing

[0517] The server analyzes the story settings received from the user. This analysis uses natural language processing techniques, specifically the Python NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

[0518] Searching the database

[0519] The server searches an existing story database based on the extracted keywords and phrases. This database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

[0520] Prompt generation for generative AI models

[0521] The server generates prompts to be input to the generative AI model based on user input and information obtained from the database. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

[0522] Story generation using generative AI models

[0523] The server inputs prompts into a generative AI model (e.g., OpenAI's GPT-4) to generate the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

[0524] Applying the Emotion Engine

[0525] The server applies an emotion engine to the generated story to add emotional elements to each scene of the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson (registered trademark) Tone Analyzer.

[0526] Graphics and Music Generation

[0527] The server automatically creates graphics and music that match each scene in the story, using image generation AI to generate the graphics and music generation AI to generate the music.

[0528] Final output integration

[0529] The server integrates the generated story, graphics, and music and provides them to the user, who can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy the graphics and music.

[0530] Examples of specific examples and prompts

[0531] Specific examples

[0532] If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate the following story:

[0533] The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

[0534] Character Quote: "Arthur: 'I swear on this sword I will slay the dragon!'"

[0535] Scene description: "Under the setting sun, Arthur gripped his sword and renewed his resolve."

[0536] Prompt Sentence Examples

[0537] An example of a prompt to input to the generative AI model is:

[0538] "Create a knightly adventure story set in medieval Europe. The main character is Arthur, a brave knight whose mission is to slay an evil dragon and save the kingdom. Include an introduction, character dialogue, and a description of the setting."

[0539] By inputting this prompt into a generative AI model, each element of the story is automatically generated.

[0540] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0541] Step 1:

[0542] The user inputs the story setting, such as the theme, setting, and characters, in text format. For example, they might input "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The input text is then sent to the server.

[0543] Step 2:

[0544] The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

[0545] Step 3:

[0546] The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

[0547] Step 4:

[0548] The server generates a prompt to be input to the generative AI model based on the user's input and information obtained from the database. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and a description of the scenery." This prompt will be used in the next step.

[0549] Step 5:

[0550] The server inputs prompts into a generative AI model (e.g., OpenAI's GPT-4) to generate the story flow, lines for the characters, and scenery. Based on the user's input, the generative AI model generates text that matches the theme of the story and the personalities of the characters. For example, the generative AI model generates the following story:

[0551] The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

[0552] Character Quote: "Arthur: 'I swear on this sword I will slay the dragon!'"

[0553] Scene description: "Under the setting sun, Arthur gripped his sword and renewed his resolve."

[0554] Step 6:

[0555] The server applies an emotion engine to the generated story and adds emotional elements to each scene of the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

[0556] Step 7:

[0557] The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

[0558] Step 8:

[0559] The server integrates the generated story, graphics, and music and provides them to the user, who can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy the graphics and music.

[0560] (Application example 1)

[0561] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0562] Conventional story generation systems can generate stories based on user-entered settings, but they cannot visualize and sound out the generated story in real time and allow users to experience it in a virtual reality space. It is also difficult to automatically generate graphics and music that match each scene of the story, enhancing the dramatic effects that effectively move the viewer's emotions. This limits the user's experience and makes the story less immersive.

[0563] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0564] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically generating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This enables the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

[0565] A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

[0566] The "setting" refers to the place and historical background in which the story unfolds, and is an element that forms the setting and atmosphere of the story.

[0567] A "character" is someone who acts in a story and plays an important role in the progression and development of the story.

[0568] "Story setting" refers to the setting that includes the basic elements of the story, such as theme, setting, and characters, and forms the foundation of the story.

[0569] A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

[0570] A "foreshadowing" is an element or hint in a story that suggests future developments, and is a technique used to make a story more interesting.

[0571] A "mystery element" is an element in a story that presents a mystery or question, and is intended to attract the interest of readers or viewers.

[0572] A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

[0573] "Graphics" refers to visual images or footage that visually represent scenes and characters in a story.

[0574] "Song" refers to music, and is used to create scenes and atmosphere in a story.

[0575] "Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

[0576] "Audio" refers to expressing the content of a story in audio form, allowing users to enjoy the story by listening to it.

[0577] A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

[0578] A "head-mounted display" is a display device worn on the head, which is used to allow users to experience virtual reality spaces.

[0579] A system for implementing this invention has the following configuration: First, a user inputs story settings such as the theme, setting, and characters of the story using a smartphone or head-mounted display. These inputs are made in text format.

[0580] Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

[0581] Furthermore, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

[0582] Next, the server uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene of the story, enhancing the visual and auditory effects of the story and effectively moving the viewer's emotions.

[0583] Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display, allowing the user to immerse themselves in the story and gain a deeper experience.

[0584] As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model.

[0585] Example prompt sentence:

[0586] Theme: Medieval Europe

[0587] Setting: Knight's Adventure

[0588] Characters: Brave knights, wizards, and dragons

[0589] Story Setting: A brave knight and a wizard embark on an adventure to slay a dragon.

[0590] Based on this prompt, the generative AI model generates a story, the emotion engine adds emotional elements, and the graphics and music engines generate graphics and music. Users can experience this story in real time through a head-mounted display.

[0591] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0592] Step 1:

[0593] The user uses a smartphone or head-mounted display to input story settings such as the theme, setting, and characters in text format, and the input data is sent to the server.

[0594] Input: Story setting such as theme, setting, characters, etc.

[0595] Output: Story setting data sent to the server

[0596] Step 2:

[0597] Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

[0598] Input: Story setting data

[0599] Output: Generated story structure, dialogue, and scenery

[0600] Step 3:

[0601] The server uses an emotion engine to add emotional elements to the generated story, such as foreshadowing, mystery elements, and moving scenes. The emotion engine automatically inserts emotional elements appropriate for each scene in the story.

[0602] Input: Generated story structure, dialogue, and scenery

[0603] Output: A story with added emotional content

[0604] Step 4:

[0605] The server uses a graphics generation engine to automatically create graphics that match each scene in the story, taking into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

[0606] Input: A story with added emotional content

[0607] Output: Graphics that match each scene

[0608] Step 5:

[0609] The server uses a music generation engine to automatically create music that matches each scene in the story, taking into account the atmosphere of the scene and the emotional state of the characters.

[0610] Input: A story with added emotional content

[0611] Output: Songs that match each scene

[0612] Step 6:

[0613] The server integrates the generated story, graphics, and music, and visualizes and sounds them in real time, allowing users to experience the story visually and aurally.

[0614] Input: Generated story, graphics, music

[0615] Output: Real-time visualised and spoken narrative

[0616] Step 7:

[0617] Using a head-mounted display, users experience stories generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

[0618] Input: Real-time visualised and spoken story

[0619] Output: A narrative experience in a virtual reality space

[0620] Example 2

[0621] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0622] Conventional story generation systems can automatically generate a story based on the story's theme and character settings, but they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the ability to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system that can solve these issues is needed.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0624] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character lines, scenery, and story structure based on the input story settings, means for automatically generating hints, mystery elements, and moving scenes to make the created story more interesting, means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions, means for automatically creating graphics and music that match each scene of the created story, and means for enhancing the dramatic effect with the story structure, character conversation, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate hints, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

[0625] A "story theme" indicates the overall subject or direction of a story.

[0626] "Setting" refers to the place or environment in which the story unfolds.

[0627] "Characters" refers to the characters who take action in the story.

[0628] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[0629] "Dialogue" refers to the words or lines spoken by characters.

[0630] "Scene" refers to the scenery or scene depicted in the story.

[0631] "Story structure" refers to the overall flow or structure of a story, including a beginning, middle, and end.

[0632] A "foreshadowing" is an element that is placed in advance in a way that will have important meaning later in the story.

[0633] A "mystery element" refers to an element in a story that attracts the interest of readers or viewers by including mysteries or unknown points in the story.

[0634] A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

[0635] "User emotions" refer to emotions such as joy, sadness, and surprise felt by a user experiencing a story.

[0636] "Real-time recognition" refers to detecting the user's emotions instantly on the spot.

[0637] "Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

[0638] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[0639] "Song" refers to the music that plays along with the scenes in the story.

[0640] "Dramatic effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

[0641] This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates plot twists, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story development accordingly.

[0642] Hardware and software used

[0643] The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progress, the server inputs prompt sentences into the generative AI model and generates story elements.

[0644] The device transmits the user's input information to the server, displays the story elements received from the server to the user, and uses emotion recognition software to collect emotion data from the user's facial expressions and voice and transmit it to the server.

[0645] Users can enjoy the story by inputting information about the story's theme, the personalities of the characters, and the story's progress into the device. The device recognizes the user's emotions in real time and reflects them in the development of the story.

[0646] Data processing and calculation

[0647] The server performs the following data processing and calculations:

[0648] Generate appropriate hints based on the story's theme and the characters' personalities.

[0649] Mystery elements are scattered throughout the story as it progresses.

[0650] Analyze the emotions of the characters and the situation to create a moving scene.

[0651] The device collects emotional data from the user's facial expressions and voice and sends it to the server, which analyzes the data and adjusts the story development accordingly.

[0652] Specific examples

[0653] For example, to generate a plot twist in which a knight meets someone who plays an important role later in the story, the following prompt sentence can be input to the generative AI model:

[0654] Example prompt sentence:

[0655] "Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

[0656] Or, if the user is feeling joyful about the story, the prompt to make the story even more upbeat might look like this:

[0657] Example prompt sentence:

[0658] "Users are enjoying it, so please add a more upbeat twist to the story. For example, add a scene where the knight finds a new companion."

[0659] In this way, the server utilizes a generative AI model and an emotion engine to automatically generate a story development that corresponds to the user's emotions.

[0660] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0661] Step 1:

[0662] The user inputs information about the story's theme, the characters' personalities, and the story's progression.

[0663] Input: Story theme (e.g., adventure), character traits (e.g., brave), and information about the story's progression

[0664] Output: User input information

[0665] Specific operation: The user enters the theme "adventure" into the input form on the device and sets the character's personality to "brave."

[0666] Step 2:

[0667] The terminal transmits the user's input information to the server.

[0668] Input: User input information

[0669] Output: User input sent to the server

[0670] Specific operation: The device sends the information "Theme: Adventure, Character: Brave" to the server.

[0671] Step 3:

[0672] The server generates story elements using generative AI models.

[0673] Input: User input information

[0674] Output: Generated story elements

[0675] Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" to the generation AI model, and generates elements of the story.

[0676] Step 4:

[0677] The server sends the generated story elements to the terminal.

[0678] Input: Generated story elements

[0679] Output: Story elements sent to the device

[0680] Specific operation: The server sends the story elements it generates (e.g., "The mysterious old man the knight meets during his adventure") to the terminal.

[0681] Step 5:

[0682] The terminal displays the story to the user.

[0683] Input: Story elements received from the server

[0684] Output: The story displayed to the user

[0685] Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

[0686] Step 6:

[0687] Recognize user emotions in real time.

[0688] Input: User's facial expressions and voice

[0689] Output: Recognized user emotion data

[0690] Specific operation: When the user smiles, the device captures the user's facial expression with the camera and recognizes that the user is feeling happy.

[0691] Step 7:

[0692] The terminal transmits the user's emotion data to the server.

[0693] Input: Recognized user emotion data

[0694] Output: Emotion data sent to the server

[0695] Specific operation: The device sends emotional data to the server, stating that "the user is feeling happy."

[0696] Step 8:

[0697] The server analyzes the emotional data and adjusts the story development.

[0698] Input: User emotion data

[0699] Output: Coordinated story development

[0700] Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more cheerful."

[0701] Step 9:

[0702] The server sends the adjusted story to the device.

[0703] Input: Coordinated story development

[0704] Output: The adjusted story sent to the terminal.

[0705] Specific operation: The server generates a story with an upbeat plot (e.g., "A scene in which the knight finds a new companion") and sends it to the device.

[0706] Step 10:

[0707] The device displays the adjusted story to the user.

[0708] Input: Adjusted story received from the server

[0709] Output: The adjusted story displayed to the user

[0710] Specific operation: The device displays to the user a scene in which a knight finds a new companion.

[0711] (Application example 2)

[0712] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0713] Conventional story generation systems can automatically generate a story based on the story's theme and character settings, but they are unable to adjust the story's development in real time in response to the user's emotions. Furthermore, they lack the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This makes it difficult to provide an interactive story experience that responds to the user's emotions.

[0714] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0715] In this invention, the server includes: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating character dialogue, scenery, and a story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect of the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for recognizing a user's emotions in real time and adjusting the development of the story in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that corresponds to the user's emotions.

[0716] A "story theme" refers to the central subject or message of a story.

[0717] "Setting" refers to the place or environment in which the story unfolds.

[0718] "Characters" refers to the characters and people who play roles in a story.

[0719] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[0720] "Dialogue" refers to the words or lines spoken by characters.

[0721] "Scene" refers to the scenery or scene depicted in the story.

[0722] "Story structure" refers to the overall flow or structure of a story, including a beginning, middle, and end.

[0723] A "foreshadowing" is an element or hint placed in the first half of a story that will have important meaning later in the story.

[0724] "Mystery elements" refer to elements in a story that raise questions or raise doubts.

[0725] A "moving scene" is one that evokes strong emotions in the viewer or reader.

[0726] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[0727] "Songs" refers to the music or soundtrack that accompanies the story scenes.

[0728] "Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

[0729] "Viewer sentiment" refers to the emotions and feelings of the people watching the story.

[0730] "Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

[0731] "Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

[0732] A "generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

[0733] A "prompt" refers to an instruction or keyword to be input into a generative AI model.

[0734] The system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the theme, setting, characters, etc. By inputting these settings, the user determines the basic elements of the story.

[0735] Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

[0736] Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, etc. to make the story more interesting, thereby adding depth and interest to the story.

[0737] The server also has the means to automatically create graphics and music that match each scene of the story, enhancing the visual and auditory effects of the story. Specifically, the server creates graphics and music taking into account the atmosphere of the scene and the emotional state of the characters.

[0738] Furthermore, the server has the means to effectively stir the viewer's emotions by enhancing the dramatic effects of the story structure, the characters' conversations, images, and music, thereby drawing the user more emotionally into the story.

[0739] What is particularly important is that the server has a means to recognize the user's emotions in real time and adjust the story development accordingly. The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice to understand the user's emotions. Based on this emotional data, the server generates prompt sentences for the story and adjusts the story development accordingly.

[0740] For example, if a camera captures the user's facial expressions while the user is reading a story and recognizes that the user is smiling, the story will develop in a lighthearted and joyful way. Conversely, if the user is sad, the story will develop in a more profound way.

[0741] An example of a prompt is as follows:

[0742] "The protagonist continues his adventure with a happy feeling."

[0743] "The protagonist faces difficulties but does not lose hope."

[0744] The protagonist takes on a new mystery.

[0745] In this way, it is possible to provide an interactive storytelling experience that responds to the user's emotions.

[0746] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0747] Step 1:

[0748] The user inputs the story setting, such as the theme, setting, and characters.

[0749] Input: Setting information such as the story's theme, setting, and characters entered by the user.

[0750] Output: Story setting data.

[0751] How it works: A user uses a smartphone application to input the basic setting of a story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

[0752] Step 2:

[0753] Based on the story settings entered into the server, the server automatically creates the characters' lines, scenes, and story structure.

[0754] Input: Story setting data.

[0755] Output: Story lines, scenes, and composition data.

[0756] How it works: The server uses a generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

[0757] Step 3:

[0758] The server automatically generates hints, mystery elements, moving scenes, and other elements to make the stories it creates more interesting.

[0759] Input: Story dialogue, scenery, and composition data.

[0760] Output: Foreshadowing, mystery elements, and moving scene data.

[0761] How it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story, adding depth and intrigue to the story.

[0762] Step 4:

[0763] The server automatically creates graphics and music that match each scene in the story created by the server.

[0764] Input: Foreshadowing, mystery elements, and moving scene data.

[0765] Output: Graphic data, music data.

[0766] How it works: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

[0767] Step 5:

[0768] The server recognizes the user's emotions in real time and adjusts the story development accordingly.

[0769] Input: User's facial expression data, tone of voice data.

[0770] Output: Tailored narrative prompt.

[0771] Specific operation: The system uses the smartphone's camera and microphone to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates prompt sentences for the story and adjusts the story development.

[0772] Step 6:

[0773] The server uses a generative AI model to generate story prompts and automatically generate the story.

[0774] Input: Tailored narrative prompt sentence.

[0775] Output: The generated story.

[0776] Specific operation: The server uses the generative AI model to generate a story based on the adjusted prompt sentences. The generated story is used to provide an interactive experience that responds to the user's emotions.

[0777] Example 3

[0778] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0779] Conventional story generation systems struggled to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide a consistent emotional experience to the user. This resulted in a decline in the quality of the story experience and lowered user satisfaction.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0781] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character lines, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character conversation, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

[0782] A "story theme" refers to the central subject or theme of a story.

[0783] "Setting" refers to the place or environment in which the story unfolds.

[0784] "Characters" refers to the characters that appear in the story.

[0785] "Story setting" refers to the setting, including basic elements such as the theme, setting, and characters of the story.

[0786] "Dialogue" refers to the words spoken by the characters.

[0787] "Scene" refers to the scenery or scene depicted in the story.

[0788] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[0789] A "foreshadowing" is an element that provides advance information or events that will be important later in the story.

[0790] "Mystery elements" refer to elements that add mystery or suspense to a story.

[0791] A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

[0792] "Graphics" refers to images and illustrations generated to match the story scenes.

[0793] "Song" refers to music generated to match the scenes in the story.

[0794] "User emotion" refers to the emotional state such as joy, sadness, surprise, etc., that the user is feeling.

[0795] "Facial expression" refers to emotions expressed by the movement of the user's facial muscles.

[0796] "Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

[0797] "Text input" refers to character information that a user inputs using a keyboard or touch panel.

[0798] "Story development" refers to the process or flow of the story.

[0799] "Staging effects" refers to the impact that the visual and auditory elements of a story have on the viewer.

[0800] "Audience sentiment" refers to the emotions and feelings of the people watching the story.

[0801] This invention is a system that inputs story settings such as the story theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the development of the story according to the user's emotions. This system is implemented using the following hardware and software.

[0802] Hardware and software used

[0803] Hardware: High-performance servers (e.g., servers equipped with GPUs) and user devices (e.g., PCs or smartphones equipped with cameras and microphones)

[0804] Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

[0805] Explanation of program processing

[0806] 1. Narrative input and analysis

[0807] The user uses a device to input the story setting, such as the theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the mood of the scene and the emotional state of the characters.

[0808] 2. Graphics and Music Generation

[0809] Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it generates powerful graphics of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it generates music that heightens the tension of a battle scene.

[0810] 3. User Emotion Recognition

[0811] The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, a voice analysis library) to analyze the user's emotions.

[0812] 4. Adjusting the story development

[0813] The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the server adjusts the story development to a more exciting direction.

[0814] Examples of specific examples and prompts

[0815] Examples:

[0816] The user inputs "a scene in which a hero fights a dragon."

[0817] The server analyzes it as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

[0818] Music generation AI is used to generate music that heightens tension.

[0819] If the user is smiling, the emotion engine will recognize that the user is having fun and adjust the story development in a more exciting direction.

[0820] Example prompt sentence:

[0821] "Generate graphics of a scene where a hero fights a dragon."

[0822] "Generate music to accompany moving scenes."

[0823] "If the user is smiling, adjust the storyline to be more fun."

[0824] This system automatically creates graphics and music that match each scene of the story, making it possible to provide a story development that matches the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 will be described with reference to FIG. 21.

[0825] Step 1:

[0826] The user enters the text of the story.

[0827] The user enters "A scene in which a hero fights a dragon" into the text input field on the terminal and clicks the send button. The entered text is sent from the terminal to the server.

[0828] Step 2:

[0829] The server parses the story text.

[0830] The server inputs the received text into a generative AI model (e.g., GPT-4) to analyze each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "fight scene" and "tension" are extracted. The analysis results are stored in an internal database.

[0831] Step 3:

[0832] The server generates graphics that match the scene.

[0833] Based on the analysis results, the server uses an image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, you can input "a scene where a dragon and a hero are fighting" as a prompt and obtain the generated graphics. The generated graphics are then sent to the user's device.

[0834] Step 4:

[0835] The server generates music that matches the scene.

[0836] The server uses music generation AI to generate music appropriate for the scene. For example, a user can input "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

[0837] Step 5:

[0838] The device recognizes the user's emotions.

[0839] The device captures the user's facial expressions with a camera and records the tone of voice with a microphone. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

[0840] Step 6:

[0841] The server adjusts the story development according to the user's emotions.

[0842] The server analyzes the received emotion data using emotion recognition software (e.g., OpenCV, speech analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

[0843] (Application example 3)

[0844] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0845] Conventional story generation systems have the problem that they do not adjust the story according to the user's emotions, resulting in a uniform user experience. Also, because the story scenes and music are fixed, they are unable to provide dynamic changes according to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0847] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically producing graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character conversation, images, and music to effectively stir the viewer's emotions, means for recognizing the user's emotions and adjusting the story scenes and music in accordance with the recognized emotions, and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that corresponds to the user's emotions.

[0848] A "story theme" refers to the central idea or message of a story.

[0849] "Setting" refers to the place or environment in which the story unfolds.

[0850] "Characters" refers to the characters that appear in the story.

[0851] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[0852] "Dialogue" refers to the words spoken by the characters.

[0853] "Scene" refers to the scenery or scene depicted in the story.

[0854] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[0855] A "foreshadowing" is an element in the first half of a story that becomes important later in the story.

[0856] "Mystery elements" refer to elements that add mystery or suspense to a story.

[0857] A "moving scene" is one that strongly stirs the emotions of the viewer.

[0858] "Graphics" refers to images and videos that visually represent scenes from a story.

[0859] "Song" refers to the music that plays along with the scenes in the story.

[0860] "Direction effects" refer to effects that influence the viewer's emotions through story scenes and music.

[0861] "User emotion" refers to the emotional state, such as joy, sadness, or surprise, that the user is feeling.

[0862] "Means for recognizing emotions" refers to technology that determines a user's emotions using the user's facial expressions, tone of voice, text input, etc.

[0863] The "means for adjusting the story scenes and music" refers to a technique for changing the story scenes and music according to the user's recognized emotions.

[0864] The system for implementing the present invention can dynamically adjust the story scenes and music according to the user's emotions. A specific embodiment of this system will be described below.

[0865] System Configuration

[0866] The system includes the following major hardware and software components:

[0867] Hardware: Smartphone camera

[0868] Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

[0869] Program processing

[0870] The server first captures the user's facial expression using the smartphone camera, then performs face detection using OpenCV, and then uses Keras' emotion recognition model to recognize the user's emotion from the detected face.

[0871] Based on the recognized emotion, the server adjusts the story scene and music. For example, if the user is smiling, the story scene will change to a bright and cheerful one and the music will also change to a lighter one. Conversely, if the user has a sad expression, the story will transition to a more emotional scene and the music will also change to a more moving one.

[0872] Specific examples

[0873] As a concrete example, if the user smiles while enjoying a story, the server would do the following:

[0874] 1. Capture the user's facial expressions using the smartphone camera.

[0875] 2. Detect faces using OpenCV.

[0876] 3. The user's emotion is recognized as "joy" using the Keras emotion recognition model.

[0877] 4. Based on the emotions identified, change the story scene to something lighter and more cheerful, and play upbeat music.

[0878] Prompt Sentence Examples

[0879] An example of a prompt to input to a generative AI model is as follows:

[0880] If the user smiles, change the story scene to something light and cheerful and play upbeat music.

[0881] In this way, it is possible to provide a dynamic story experience that responds to the user's emotions, allowing the user to be more deeply immersed in the story and experience a more emotional experience.

[0882] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0883] Step 1:

[0884] The server captures the user's facial expressions using the smartphone's camera. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

[0885] Step 2:

[0886] The server uses OpenCV to detect faces from the captured image data. The input is the captured image data, and the output is the location information of the detected faces. Specifically, the OpenCV face detection algorithm identifies the face in the image and obtains its coordinates.

[0887] Step 3:

[0888] The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face location information and image data, and the output is the recognized emotion label. Specifically, the server extracts the face area, performs preprocessing, and then inputs it into the emotion recognition model to classify the emotion.

[0889] Step 4:

[0890] The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. The specific operation is to select scenes and music corresponding to the emotion label and change the progression of the story.

[0891] Step 5:

[0892] The server transmits the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and transmits it to the user's device via the network.

[0893] Step 6:

[0894] The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speaker.

[0895] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0896] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0897] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0898] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0899] [Second embodiment]

[0900] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0901] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0902] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0903] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0904] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0905] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0906] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0907] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0908] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0909] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0910] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0911] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0912] "Example 1"

[0913] The system of the present invention receives input from the user about the story setting, such as the theme, setting, and characters. This input is provided in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenery, and a story structure. This creation is performed using existing story databases and artificial intelligence technology, generating dialogue and scenery that match the story flow, character personalities, and theme.

[0914] "Example 2"

[0915] Furthermore, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done in accordance with the story's theme, the characters' personalities, and the progression of the story, laying hints at the right time, sprinkling mystery elements, and creating moving scenes. For example, it automatically generates hints that people the knight meets during his adventure will play an important role in the story later on, elements that the knight must solve, and moving scenes when the knight overcomes difficulties.

[0916] "Example 3"

[0917] The system then automatically creates graphics and music that match each scene in the story. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story structure, character dialogue, images, and music, effectively stirring the emotions of the viewer.

[0918] The processing flow of each embodiment will be described below.

[0919] "Example 1"

[0920] Step 1: The user provides textual input of the story setting, such as the theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

[0921] Step 2: Based on the input, the system automatically generates character dialogue, scenery, and story structure, using existing story databases and artificial intelligence techniques.

[0922] Step 3: Lines and scenes are generated that fit the story flow, the characters' personalities, and the story's theme.

[0923] "Example 2"

[0924] Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and other elements to make the story more interesting.

[0925] Step 2: Lay out hints at the right time, sprinkle in elements of mystery, and create moving scenes in line with the story's theme, the characters' personalities, and the progression of the story.

[0926] Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing an important role in the story later on, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

[0927] "Example 3"

[0928] Step 1: The system automatically creates graphics and music that match each scene in the story you create.

[0929] Step 2: Graphics and music are created taking into account the mood of the scene and the emotional state of the characters.

[0930] Step 3: For example, powerful graphics and music that heightens the tension are produced for battle scenes, while beautiful landscape graphics and moving music are produced for moving scenes. This enhances the dramatic effect of the story structure, the characters' conversations, images and music, and effectively stirs the viewer's emotions.

[0931] Example 1

[0932] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0933] With conventional story generation systems, when automatically generating a story based on a story setting entered by the user, it is difficult to maintain consistency in the flow of the story, the dialogue of the characters, and the scenery. Furthermore, there is a lack of ingenuity to ensure that the generated story is moving and entertaining for the viewer. Furthermore, there is an issue that it is not possible to automatically create graphics and music that match each scene of the story, making it difficult to effectively move the viewer's emotions.

[0934] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0935] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect of the story structure, character dialogue, and video and music to effectively stir the viewer's emotions; means for receiving input from a user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into a generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

[0936] The "theme of a story" refers to the central subject or theme of a story, and is an element that determines the overall direction and atmosphere of the story.

[0937] The "setting" refers to the place and historical background in which the story unfolds, and is an important element in forming the setting and atmosphere of the story.

[0938] A "character" is someone who acts in a story and moves the story forward, and is an element that has a major influence on the development and theme of the story.

[0939] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and is the element that forms the framework of the story.

[0940] "Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[0941] "Scenes" refer to the landscapes and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

[0942] "Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the development of a story effective.

[0943] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that adds consistency and depth to a story.

[0944] A "mystery element" is an element that presents a mystery or question in a story to attract the interest of readers or viewers.

[0945] A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

[0946] "Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotions of the story.

[0947] "Songs" are melodies and sounds that are musically expressed to match each scene in the story, and are elements that musically complement the atmosphere and emotions of the story.

[0948] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

[0949] A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

[0950] A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content that the model generates.

[0951] "Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read before it is presented to the user.

[0952] This invention is a system that automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[0953] User input

[0954] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe."

[0955] The server receives and analyzes the input data

[0956] The server receives the text data sent by the user. It then analyzes the input data using natural language processing techniques. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

[0957] Generate prompt statement

[0958] The server generates a prompt to be input to the generative AI model based on the extracted information. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

[0959] Generating stories with generative AI models

[0960] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

[0961] Formatting and serving the generated results

[0962] The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

[0963] Specific examples

[0964] Consider the user's input, "A knight's adventure set in medieval Europe." Based on this input, the server generates the following prompt:

[0965] "Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

[0966] The generative AI model generates a story based on this prompt, outputting text such as the following:

[0967] "The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

[0968] In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

[0969] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0970] Step 1:

[0971] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe." This input data is then sent to the server.

[0972] Step 2:

[0973] The server receives the text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

[0974] Step 3:

[0975] The server generates a prompt sentence to be input to the generative AI model based on the analysis results. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt sentence would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt sentence becomes the input for the next step.

[0976] Step 4:

[0977] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it generates text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'" This generated text becomes the input for the next step.

[0978] Step 5:

[0979] The server receives the narrative text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted narrative is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

[0980] Step 6:

[0981] The server provides the formatted story to the user, who can then view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

[0982] (Application example 1)

[0983] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0984] Conventional story generation systems automatically generate stories based on themes and settings entered by users, but the generation of story structure, character dialogue, and scenery is insufficient, making it difficult to effectively move the viewer's emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories results in poor user convenience.

[0985] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0986] In this invention, the server includes: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating character lines, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user and effectively stir the viewer's emotions.

[0987] A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

[0988] The "setting" refers to the place and historical background in which the story unfolds, and is an element that forms the setting and atmosphere of the story.

[0989] "Characters" are characters that appear in a story and play an important role in the progression and development of the story.

[0990] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and forms the framework of the story.

[0991] "Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[0992] "Scenes" are the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

[0993] "Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

[0994] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

[0995] "Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

[0996] A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer.

[0997] "Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

[0998] A "song" is music created to accompany each scene in a story, and is intended to musically express the atmosphere and emotions of the story.

[0999] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

[1000] A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

[1001] "Preview" is a function that allows users to check the generated story in advance and check the content.

[1002] "Preservation" refers to recording the generated stories in a digital format so that they can be reused and shared later.

[1003] "Sharing" refers to sharing the generated story with other users and platforms, in order to promote the spread and appreciation of the story.

[1004] A system for carrying out this invention automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[1005] System Configuration

[1006] This system is mainly composed of the following hardware and software.

[1007] Hardware: Smartphones, servers

[1008] Software: Python, OpenAI API

[1009] Program processing

[1010] User Input

[1011] Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they can enter the theme "A knight's adventure story set in medieval Europe."

[1012] Prompt Generation

[1013] The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input to the generative AI model. As a specific example, the following prompt is generated:

[1014] Theme: A knight's adventure set in medieval Europe

[1015] Setting: Medieval Europe

[1016] Characters: Brave knights, wizards, and dragons

[1017] Generate a story.

[1018] Story Generation

[1019] The server automatically generates a story based on the generated prompt sentence using a generative AI model (e.g., GPT-4). This generative AI model is called through the OpenAI API.

[1020] Preview, save, and share

[1021] The generated story is displayed for users to preview through the smartphone application, and users can save the story and share it on social media.

[1022] Specific examples

[1023] If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

[1024] Theme: A knight's adventure set in medieval Europe

[1025] Setting: Medieval Europe

[1026] Characters: Brave knights, wizards, and dragons

[1027] Generate a story.

[1028] By inputting this prompt into a generative AI model, a story is automatically generated, which users can preview, save, and share.

[1029] Hardware and software used

[1030] Hardware: Smartphones, servers

[1031] Software: Python, OpenAI API

[1032] This makes it possible to automatically generate high-quality stories based on the settings entered by the user, effectively stirring the emotions of the viewer.

[1033] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1034] Step 1:

[1035] The user uses a smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

[1036] Step 2:

[1037] The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt suitable for the generative AI model. The generated prompt looks like this:

[1038] Theme: A knight's adventure set in medieval Europe

[1039] Setting: Medieval Europe

[1040] Characters: Brave knights, wizards, and dragons

[1041] Generate a story.

[1042] This prompt sentence is generated.

[1043] Step 3:

[1044] The server calls a generative AI model (e.g., GPT-4) with the generated prompt sentence. It sends the prompt sentence using the OpenAI API and requests the generation of a story. The input is the prompt sentence, and the output is the generated story text.

[1045] Step 4:

[1046] The generative AI model automatically generates a story based on the prompt sentence. The generated story text is returned to the server. For example, the following story is generated:

[1047] Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight named Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

[1048] Step 5:

[1049] The server sends the generated story text to the smartphone terminal, and the user can preview the generated story through the smartphone terminal. The input is the generated story text, and the output is the preview screen displayed on the user's terminal.

[1050] Step 6:

[1051] Users can save the created story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

[1052] By following the steps above, it is possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the emotions of the viewer.

[1053] Example 2

[1054] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1055] Conventional story generation systems can automatically generate stories based on themes and character settings, but they struggle to properly generate plot twists, mystery elements, and moving scenes to make a story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively utilize the generated story. Furthermore, they lack the means to enhance the dramatic effect of story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1057] In this invention, the server includes means for inputting story settings such as the theme, setting, and characters of the story, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for organizing the generated story elements and providing them to users, and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewers' emotions. This makes it possible to make the story more interesting and enable users to effectively use the created story.

[1058] A "story theme" refers to the central subject or theme of a story.

[1059] "Setting" refers to the place or environment in which the story unfolds.

[1060] "Characters" refers to the characters and people who appear in a story.

[1061] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[1062] "Dialogue" refers to the words or lines spoken by characters.

[1063] "Scene" refers to the scenery or scene depicted in the story.

[1064] "Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

[1065] A "foreshadowing" is an element or event placed in the first half of a story in such a way that it will have an important meaning later on.

[1066] "Mystery elements" refer to elements that add mystery or detective elements to a story, thereby attracting the interest of readers and viewers.

[1067] A "moving scene" is a scene in a story that stirs strong emotions.

[1068] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[1069] "Song" refers to the music or background music that plays along with the story scenes.

[1070] "Staging effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and responses from the viewer.

[1071] "Audience sentiment" refers to the emotions and feelings of the people who watch or read the story.

[1072] This invention is a system that automatically generates plot twists, mystery elements, and moving scenes to make a story more interesting, based on the story's theme, the personalities of the characters, and the story's progression. A specific embodiment of this system is described below.

[1073] System configuration

[1074] Hardware

[1075] The server uses a cloud server equipped with a high-performance GPU.

[1076] software

[1077] The server uses a generative AI model, specifically one such as OpenAI's GPT-4.

[1078] Program processing

[1079] User Input

[1080] The user uses the terminal to input information about the theme of the story, the personalities of the characters, and the progress of the story. For example, the user may input the following information:

[1081] Theme: Medieval Knight's Adventure

[1082] Characters: A brave knight, a mysterious old man, and a princess

[1083] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[1084] Generate prompt statement

[1085] The server receives the information entered by the user and generates a prompt based on that information. The generated prompt looks like this:

[1086] Theme: Medieval Knight's Adventure

[1087] Characters: A brave knight, a mysterious old man, and a princess

[1088] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[1089] Generating story elements

[1090] The server inputs the generated prompt sentence into the generative AI model to generate story elements. The generative AI model generates the following story elements based on the prompt sentence.

[1091] Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

[1092] Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

[1093] Emotional Scene: The knight saves the princess and fights a final battle to protect the kingdom.

[1094] Organizing and presenting generated story elements

[1095] The server organizes the story elements output by the generative AI model and provides them to the user, who can then review the generated story elements via their device and make corrections or additions as needed.

[1096] Specific examples

[1097] Below is a concrete example of how a user can use the system to generate a story.

[1098] Theme: Medieval Knight's Adventure

[1099] Characters: A brave knight, a mysterious old man, and a princess

[1100] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[1101] The server inputs this prompt into a generative AI model, which automatically generates plot twists, mystery elements, and moving scenes based on the story's theme, the characters' personalities, and the story's progression. The generated story elements are provided to the user, who can review them and make corrections or additions as needed.

[1102] In this way, it is possible to make the story more interesting and enable the user to effectively use the generated story.

[1103] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1104] Step 1:

[1105] The user inputs information about the story's theme and characters.

[1106] Input: Information about the story's theme, character personalities, and story progression.

[1107] Specific operation: The user enters the theme "Adventures of a Medieval Knight" and the characters "A Brave Knight, a Mysterious Old Man, and a Princess" into the input form on the terminal. When the input is complete, the user clicks the "Submit" button.

[1108] Output: The entered information is sent to the server.

[1109] Step 2:

[1110] The server receives the input information and generates a prompt.

[1111] Input: User-submitted information about the story's theme, character personalities, and story progression.

[1112] Specific operation: Based on the received information, the server generates a prompt sentence to be input into the generative AI model.

[1113] Output: The generated prompt statement.

[1114] Theme: Medieval Knight's Adventure

[1115] Characters: A brave knight, a mysterious old man, and a princess

[1116] Storyline: The knight meets a mysterious old man during his adventure, receives advice from him, and continues his adventure. In the end, the knight saves the princess and protects the kingdom.

[1117] Step 3:

[1118] The server inputs prompts into the generative AI model to generate story elements.

[1119] Input: The generated prompt statement.

[1120] Specific operation: The server inputs prompt text into the generative AI model to generate plot twists, mystery elements, and moving scenes.

[1121] Output: Generated story elements.

[1122] Foreshadowing: The mysterious old man the knight meets during his adventure actually knows the kingdom's hidden secrets.

[1123] Mystery element: The knight solves the mystery hidden in an old map he received from an old man.

[1124] Emotional Scene: The knight saves the princess and fights a final battle to protect the kingdom.

[1125] Step 4:

[1126] The server organizes the generated story elements and provides them to the user.

[1127] Input: Story elements output from a generative AI model.

[1128] Specific operation: The server organizes the generated story elements and provides them to the user. The user can check the generated story elements through their terminal and make corrections or additions as necessary.

[1129] Output: The organized story elements are presented to the user.

[1130] (Application example 2)

[1131] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1132] While conventional story generation systems can automatically create stories based on themes and character settings, they struggle to properly generate plot twists, mystery elements, and moving scenes to make a story more interesting. They also struggle to automatically create graphics and music that match each scene in the generated story and effectively sway the viewer's emotions. Furthermore, when using generative AI models to generate story elements, there is a lack of a way to properly generate prompts and input them into the model, which poses a challenge in providing a satisfying story to users.

[1133] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1134] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions, means for generating story elements using a generative AI model, means for generating prompt sentences and inputting them into the generative AI model, and means for displaying the generated story elements to the user. This makes it possible to make the story more interesting and effectively stir the viewer's emotions.

[1135] A "story theme" is the central subject or theme of a story.

[1136] The "setting" is the place or environment in which the story unfolds.

[1137] "Characters" are the characters that appear in the story.

[1138] "Story setting" refers to the basic elements of a story, such as its theme, setting, and characters.

[1139] "Dialogue" refers to the words spoken by the characters.

[1140] "Scene" refers to the scenery or scene depicted in the story.

[1141] "Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

[1142] A "foreshadowing" is the advance indication of an important element related to later developments.

[1143] A "mystery element" is the inclusion of a mystery or mysterious event in a story.

[1144] A "moving scene" is one that strongly stirs the emotions of the viewer.

[1145] "Graphics" refers to visual images or illustrations.

[1146] A "song" is a piece of music or melody.

[1147] "Stage effects" are visual and auditory devices used to make a story more appealing.

[1148] The "audience" is the person who watches the story.

[1149] A "generative AI model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[1150] A "prompt sentence" is an instruction sentence to be input into a generative AI model.

[1151] "User" means a person who uses the system.

[1152] A system for implementing this invention includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character dialogue, images, and music, and effectively stirring the emotions of the viewer, means for generating story elements using a generative AI model, means for generating prompt sentences and inputting them into the generative AI model, and means for displaying the generated story elements to the user.

[1153] The server receives information about the story's theme and characters entered by the user and generates a prompt based on that information. The generated prompt is then input into a generative AI model (e.g., GPT-3) to generate story elements. The generated story elements are then displayed to the user.

[1154] Specifically, the server operates in the following steps: First, the user inputs information about the story's theme and characters. Next, the server generates a prompt based on this information. The generated prompt is input into a generative AI model, which generates story elements. The generated story elements are then displayed to the user.

[1155] The hardware used includes servers and user devices, and the software used includes generative AI models (e.g., GPT-3) and prompt generation algorithms.

[1156] As a concrete example, the following prompt sentence is generated:

[1157] Theme: Knight's Adventure

[1158] Characters: Brave knight, mysterious old man, wizard

[1159] Generate plot twists, mystery elements, and moving scenes to make your story more interesting.

[1160] By inputting this prompt into a generative AI model, story elements are automatically generated. These story elements are then displayed to the user, enhancing the story structure, character dialogue, visuals, and music for dramatic effect.

[1161] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1162] Step 1:

[1163] The user inputs information about the story's theme and characters.

[1164] Input: Story theme, character information

[1165] Output: Input theme and character information

[1166] Specific operation: The user enters the story's theme (e.g., "The Knight's Adventure") and characters (e.g., "The Brave Knight, the Mysterious Old Man, and the Wizard") into an input form on a smartphone application.

[1167] Step 2:

[1168] The server receives the input information about the theme and characters and generates a prompt sentence.

[1169] Input: Theme and character information entered

[1170] Output: Generated prompt statement

[1171] Specific operation: Based on the received theme and character information, the server generates a prompt like the following: "Theme: A knight's adventure. Characters: A brave knight, a mysterious old man, a wizard. Please generate hints, mystery elements, and moving scenes to make the story more interesting."

[1172] Step 3:

[1173] The server inputs the generated prompt sentence into the generative AI model.

[1174] Input: Generated prompt statement

[1175] Output: Story elements generated by the generative AI model

[1176] Specific operation: The server sends the generated prompt sentence to a generative AI model such as GPT-3 to generate story elements (e.g., foreshadowing, mystery elements, and moving scenes).

[1177] Step 4:

[1178] The server receives the generated story elements and displays them to the user.

[1179] Input: Story elements generated by a generative AI model

[1180] Output: The story elements displayed to the user

[1181] Specific operation: The server sends the story elements received from the generative AI model to the user's smartphone application and displays them to the user, who can then review the generated story elements.

[1182] Step 5:

[1183] The server automatically creates graphics and music that match each scene in the story.

[1184] Input: Generated story elements

[1185] Output: Graphics and music that match each scene

[1186] Specific operation: Based on the generated story elements, the server automatically creates appropriate graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters.

[1187] Step 6:

[1188] The server enhances the dramatic effect through the story structure, character dialogue, images and music.

[1189] Input: Graphics, music, and story elements that match each scene

[1190] Output: A narrative with enhanced effects

[1191] Specific operation: The server combines the story structure, the characters' dialogue, images and music to enhance the production effect, effectively stirring the viewer's emotions.

[1192] Example 3

[1193] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1194] Conventional story generation systems have difficulty automatically creating graphics and music that match each scene in a story, and have been unable to enhance the dramatic effect of effectively swaying the viewer's emotions. In addition, since graphics and music that take into account the atmosphere of the scene and the emotional state of the characters are generated manually, this is time-consuming and labor-intensive.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1196] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for analyzing scene data and extracting the atmosphere of the scene and the emotional state of the characters; means for generating prompt statements to be input to a generative AI model based on the extracted information; means for inputting the prompt statements to the generative AI model and generating graphics and music that match the scene; means for integrating the generated graphics and music with the scene data to generate a final output; and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and enhance the dramatic effect that effectively stirs the viewer's emotions.

[1197] "Story settings such as the story's theme, setting, and characters" are the basic elements of a story, and refer to the background of the story, the characteristics of the characters, and the settings necessary for the story's progression.

[1198] "Input means" refers to the interface that allows the user to provide the story setting, such as the theme, setting, and characters, to the system.

[1199] "Means for automatically creating character dialogue, scenes, and story structure" refers to a function that automatically generates character dialogue, scene descriptions, and the overall flow of the story based on the input story setting.

[1200] "Means for automatically generating foreshadowing, mystery elements, moving scenes, etc." refers to a function that automatically adds elements that cause surprises or emotions to a pre-set story in order to make the story more interesting.

[1201] "Means for automatically creating graphics and music" refers to the ability to automatically generate visual and auditory elements to match each scene in a story.

[1202] "Means for analyzing scene data" refers to the function of analyzing the content of each scene in a story and extracting the atmosphere of the scene and the emotional state of the characters.

[1203] "Means for generating prompt sentences to be input to the generative AI model" refers to a function that creates text to provide appropriate instructions to the generative AI model based on analyzed scene data.

[1204] "A means for inputting a prompt sentence into a generative AI model and generating graphics and music that match the scene" refers to the function of inputting a prompt sentence into a generative AI model and, as a result, generating graphics and music that are appropriate for the scene.

[1205] "Means for integrating generated graphics and music into scene data and producing final output" refers to functionality that incorporates generated visual and auditory elements into scene data to create final narrative output.

[1206] "Means of effectively stirring the viewer's emotions" refers to the function of having a strong impact on the viewer's emotions through the structure of the story, the dialogue of the characters, images, and music.

[1207] This invention relates to a system that automatically creates graphics and music that match each scene in a story. This system inputs story settings such as the story's theme, setting, and characters, and automatically creates character dialogue, scenery, and story structure based on the input story settings. It also automatically generates plot twists, mystery elements, and moving scenes to make the story more interesting, and automatically creates graphics and music that match each scene.

[1208] Hardware and software used

[1209] The system uses the following hardware and software:

[1210] Server: Analyzes data and runs generative AI models.

[1211] Terminal: Provides an interface for users to input the story setting of a story.

[1212] Generative AI models: For example, OpenAI's GPT-4, DALL-E, and music generation models are used.

[1213] Data processing and calculation

[1214] The server receives the story settings entered by the user through the device. The story settings include the theme, setting, and character traits of the story. The server analyzes this data and automatically creates the characters' lines, scenery, and story structure.

[1215] Next, the server automatically generates plot twists, mystery elements, and moving scenes based on the story, making the story more interesting and moving for viewers.

[1216] The server then analyzes the scene data to automatically create graphics and music that match each scene in the story. The analysis extracts the scene's atmosphere and the characters' emotional states. Based on the extracted information, the server generates prompts to be input into the generative AI model.

[1217] Specific examples

[1218] For example, in the case of an emotional scene, the server might input the following prompt sentence into the generative AI model:

[1219] Example prompt for graphic generation: "Generate a graphic of a beautiful landscape. The scene is emotional and the character is in tears."

[1220] Example prompt for music generation: "Generate moving, poignant music. The scene is emotional and the characters are in tears."

[1221] The generative AI model generates graphics and music based on these prompts. The server integrates the generated graphics and music with the scene data to generate the final output. The terminal displays the integrated results to the user.

[1222] In this way, the user can automatically create graphics and music that match each scene of the story, thereby enhancing the dramatic effect that effectively stirs the emotions of the viewer. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1223] Step 1:

[1224] The user inputs the story setting, such as the theme, setting, and characters, into the terminal.

[1225] Specific operation: The user enters information such as the story theme (e.g., fantasy), setting (e.g., medieval castle), and characters (e.g., hero, wizard) into an input form on the device. The device then sends the entered story settings to the server.

[1226] Input: Story setting such as theme, setting, characters, etc.

[1227] Output: Story setting data sent to the server

[1228] Step 2:

[1229] The server analyzes the received story setting data and automatically creates the characters' lines, scenes, and story structure.

[1230] Specific operation: The server uses natural language processing (NLP) techniques to analyze the story setting data and generate character dialogue, scene descriptions, and the overall flow of the story.

[1231] Input: Story setting data

[1232] Output: Character dialogue, scenery, and story structure data

[1233] Step 3:

[1234] Based on the story you create, the server automatically generates foreshadowing, mystery elements, moving scenes, and more.

[1235] Specific operation: The server analyzes the story's composition data and adds hints, mystery elements, and moving scenes to make the story more interesting.

[1236] Input: Story composition data

[1237] Output: Story data including hints, mystery elements, and moving scenes

[1238] Step 4:

[1239] The server analyzes the scene data to automatically create graphics and music that match each scene in the story.

[1240] Specific operation: The server divides the story data into scenes and extracts the atmosphere of each scene and the emotional state of the characters.

[1241] Input: Narrative data

[1242] Output: Scene atmosphere and characters' emotional state data

[1243] Step 5:

[1244] The server generates prompt sentences to input into the generative AI model based on the extracted information.

[1245] Specific operation: The server creates a prompt sentence according to the atmosphere of the scene and the emotional state of the characters. For example, in the case of a touching scene, the server generates a prompt sentence such as "Generate a graphic of a beautiful landscape. The scene is touching, and the characters are crying."

[1246] Input: Scene atmosphere and character emotional state data

[1247] Output: A prompt to be input to the generative AI model

[1248] Step 6:

[1249] The server inputs prompt text into the generative AI model and generates graphics and music that match the scene.

[1250] Specific operation: The server sends prompts to the generative AI model (e.g., DALL-E) to generate graphics that match the scene. Similarly, it sends prompts to the music generation model to generate music that matches the scene.

[1251] Input: A prompt to be input to the generative AI model

[1252] Output: Generated graphics and music

[1253] Step 7:

[1254] The server integrates the generated graphics and music with the scene data to generate the final output.

[1255] Specific operation: The server integrates the generated graphics and music with each scene data to create the final output of the entire story.

[1256] Input: Generated graphics, music, and scene data

[1257] Output: Final integrated narrative data

[1258] Step 8:

[1259] The terminal displays the final integrated story data to the user.

[1260] Specific operation: The terminal receives the final story data sent from the server and displays it visually and audibly to the user.

[1261] Input: Final consolidated narrative data

[1262] Output: The visual and auditory content of the story that is displayed to the user

[1263] (Application example 3)

[1264] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1265] Conventional story generation systems have difficulty automatically creating graphics and music that match each scene in the story, limiting the visual and auditory effects they can produce. Even when users input their own stories, there is a lack of means to generate graphics and music that are appropriate for that story, which leads to a problem of reduced content quality. Furthermore, the lack of functionality to preview and export the generated content makes it less convenient for users.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1267] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, means for enhancing the story structure, character dialogue, and visuals and music to effectively stir the viewer's emotions, means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters, means for generating graphics based on scene descriptions, means for generating music based on scene descriptions, means for previewing the generated content, and means for exporting the completed content. This makes it possible to automatically generate visually and aurally rich content based on a story input by a user, and preview and export it.

[1268] A "story theme" refers to the central subject or message of a story.

[1269] "Setting" refers to the place or environment in which the story unfolds.

[1270] "Characters" refers to the characters that appear in the story.

[1271] "Story setting" refers to basic elements such as the story's theme, setting, and characters.

[1272] "Dialogue" refers to the words spoken by the characters.

[1273] "Scene" refers to the scenery or scene depicted in the story.

[1274] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[1275] A "foreshadowing" is an element that provides advance information or events that will become important later in the story.

[1276] "Mystery elements" refer to elements that add mystery or suspense to a story.

[1277] A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

[1278] "Graphics" refers to visual images and illustrations generated to accompany the story scenes.

[1279] "Song" refers to music generated to match the scenes in the story.

[1280] "Mood of the scene" refers to the overall feeling or mood of a particular scene in a story.

[1281] "Character emotional state" refers to the emotions that the characters are feeling in the story.

[1282] A "scene description" is a detailed description or explanation of a particular moment in a story.

[1283] "Preview" refers to checking the generated content in advance.

[1284] "Export" refers to saving the generated content as an external file.

[1285] A system for implementing this invention includes means for inputting story settings such as the theme, setting, and characters of the story, means for automatically creating character lines and scenes and a story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically producing graphics and music that match each scene of the created story, means for enhancing the story structure, character conversations, and the dramatic effect of using images and music to effectively stir the emotions of viewers, means for analyzing story scenes and extracting the atmosphere of the scene and the emotional state of the characters, means for generating graphics based on a description of the scene, means for generating music based on a description of the scene, means for previewing the generated content, and means for exporting the completed content.

[1286] Hardware and Software Configuration

[1287] Hardware:

[1288] Smartphone

[1289] software:

[1290] Python

[1291] OpenAI API

[1292] PIL (Python Imaging Library)

[1293] Music Generation Library

[1294] Data processing and calculation

[1295] Ways to enter story settings such as theme, setting, and characters:

[1296] Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

[1297] How to automatically generate character dialogue, scenes, and story structure:

[1298] Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

[1299] How to automatically generate plot twists, mystery elements, and moving scenes:

[1300] The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

[1301] How to automatically create graphics and music that match each scene in the story:

[1302] The server analyzes each scene in the story and extracts the mood of the scene and the emotional state of the characters, based on which it generates graphics using OpenAI's image generation API and music using OpenAI's music generation API.

[1303] To preview the generated content:

[1304] Through a smartphone application, users can preview each scene of the story, including the generated graphics and music.

[1305] To export your finished content:

[1306] Users can export the completed content as a video file through a smartphone application.

[1307] Specific examples

[1308] For example, if a user inputs the story "A hero climbed a mountain to fight a dragon," the system will analyze this scene and generate powerful graphics and music that heightens the tension to match the battle scene between the hero and the dragon.

[1309] Example prompt sentence:

[1310] Analyze the following story scene and print the mood of the scene and the emotional state of the characters:

[1311] The hero climbed the mountain to fight the dragon.

[1312] In this way, users can enjoy their stories as visually and aurally rich content.

[1313] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1314] Step 1:

[1315] Users enter story settings such as the story's theme, setting, and characters in text format through a smartphone application.

[1316] Input: Story setting, including theme, setting, and characters

[1317] Output: Story setting data in text format

[1318] Step 2:

[1319] Based on the input story setting, the server uses a generative AI model (e.g., GPT-3) to automatically create character dialogue, scenes, and story structure.

[1320] Input: Story setting data in text format

[1321] Output: Character dialogue, scenery, and story structure data

[1322] Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

[1323] Step 3:

[1324] The server uses a generative AI model to automatically generate plot twists, mystery elements, and moving scenes to make the story more interesting.

[1325] Input: Character dialogue, scenery, and story structure data

[1326] Output: Foreshadowing, mystery elements, and moving scene data

[1327] Specific operation: The server sends a prompt to the generative AI model and receives the generated text.

[1328] Step 4:

[1329] The server analyzes each scene in the story and extracts the mood of the scene and the emotional state of the characters.

[1330] Input: Foreshadowing, mystery elements, and moving scene data

[1331] Output: Scene mood, characters' emotional states

[1332] Specific operation: The server sends a prompt to the generative AI model and receives the analysis result.

[1333] Step 5:

[1334] The server generates graphics based on the scene description.

[1335] Input: Scene mood, character emotional state data

[1336] Output: Graphic data

[1337] Specific operation: The server sends a prompt to OpenAI's image generation API and receives the generated image.

[1338] Step 6:

[1339] The server generates music based on the scene description.

[1340] Input: Scene mood, character emotional state data

[1341] Output: Music data

[1342] Specific operation: The server sends a prompt to OpenAI's music generation API and receives the generated music.

[1343] Step 7:

[1344] Through a smartphone application, users can preview each scene of the story, including the generated graphics and music.

[1345] Input: Graphic data, music data

[1346] Output: Preview screen

[1347] What it does: The smartphone application displays the generated data and gives the user a preview.

[1348] Step 8:

[1349] Users can export the completed content as a video file through a smartphone application.

[1350] Input: Preview screen

[1351] Output: Video file

[1352] Specific operation: The smartphone application saves the generated data as a video file and provides it to the user.

[1353] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1354] "Example 1"

[1355] One embodiment of the present invention is a story generation system that incorporates an emotion engine. This system receives story settings, such as the story's theme, setting, and characters, as input information from the user. Then, based on the story settings, it automatically creates character dialogue, scenery, and story structure. Furthermore, it automatically generates foreshadowing, mystery elements, and moving scenes to make the story more interesting. It also automatically creates graphics and music that match each scene in the story. The story structure, character dialogue, video, and music enhance the dramatic effect, effectively stirring the viewer's emotions.

[1356] "Example 2"

[1357] Furthermore, the system includes an emotion engine that recognizes the user's emotions in real time and adjusts the story development according to the emotions. For example, if the user feels joyful about the story, the story will develop in a more cheerful manner. Conversely, if the user feels sad about the story, the story will develop in a more profound manner.

[1358] "Example 3"

[1359] The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions. For example, if the user is smiling, the emotion engine recognizes that the user is feeling happy. If the user speaks in a sad tone of voice, the emotion engine recognizes that the user is feeling sad. Furthermore, if the user inputs the text "fun," the emotion engine recognizes that the user is feeling happy. Based on this information, the emotion engine adjusts the development of the story and provides a story experience that matches the user's emotions.

[1360] The processing flow of each embodiment will be described below.

[1361] "Example 1"

[1362] Step 1: Receive story settings such as the story theme, setting, and characters as input from the user.

[1363] Step 2: Based on the received story setting, the system automatically creates character dialogue, scenes, and a story structure.

[1364] Step 3: Automatically generate hints, mystery elements, moving scenes, and more to make the story more interesting.

[1365] Step 4: Automatically create graphics and music that match each scene of the story you've created. Step 5: Enhance the story structure, character dialogue, video, and music to effectively stir the viewer's emotions.

[1366] "Example 2"

[1367] Step 1: Activate the emotion engine and recognize the user's emotions in real time.

[1368] Step 2: Adjust the story development according to the recognized emotion. For example, if the user feels happy about the story, the story will be more cheerful. Conversely, if the user feels sad about the story, the story will be more profound.

[1369] "Example 3"

[1370] Step 1: The emotion engine uses information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions.

[1371] Step 2: For example, if the user is smiling, the emotion engine will recognize that the user is feeling happy, and if the user is speaking in a sad tone of voice, the emotion engine will recognize that the user is feeling sad.

[1372] Step 3: If the user further inputs the text "fun," the emotion engine recognizes that the user is feeling fun. Based on this information, the emotion engine adjusts the story development and provides a story experience that corresponds to the user's emotions.

[1373] Example 1

[1374] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1375] In conventional story generation systems, when automatically generating a story based on a story setting entered by a user, consistency in the flow of the story, the dialogue of the characters, and the scenery often lacked. It was also difficult to generate foreshadowing, mystery elements, and moving scenes to make the story more interesting, making it impossible to effectively stir the viewer's emotions. Furthermore, it was difficult to automatically create graphics and music that matched each scene in the story. In order to solve these problems, the present invention aims to provide a system that generates a more consistent story and effectively stirs the viewer's emotions.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1377] In this invention, the server includes means for inputting story settings such as the theme, setting, and characters of the story, means for analyzing the input story settings, means for searching an existing story database based on the analyzed story settings, means for generating and inputting prompt sentences to a generation AI model, means for automatically creating character lines, scenes, and a story structure using the generation AI model, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically creating graphics and music that match each scene of the created story, and means for enhancing the dramatic effects of the story structure, character dialogue, images, and music to effectively stir the viewer's emotions. This makes it possible to generate a consistent story based on the story settings input by the user and effectively stir the viewer's emotions.

[1378] A "story theme" is the central subject or theme of a story.

[1379] The "setting" refers to the place and historical background in which the story unfolds.

[1380] "Characters" are the characters or people who appear in a story.

[1381] "Story setting" refers to the setting of a story, including its basic elements such as theme, setting, and characters.

[1382] "Input means" refers to a method or device that allows a user to provide the story setting, such as the theme, setting, and characters of the story, to the system.

[1383] The "analyzing means" refers to a method or device for analyzing the input story setting and extracting important keywords and phrases.

[1384] The "means for searching a database" refers to a method or device for searching relevant information from an existing story database based on the analyzed story setting.

[1385] A "generative AI model" is a model that uses artificial intelligence technology to generate text or stories.

[1386] A "prompt" is an instruction or question that is input into a generative AI model.

[1387] "Means for generating and inputting prompt sentences" refers to a method or device for creating a prompt sentence suitable for a generative AI model and inputting it into the model.

[1388] "Dialogue" refers to the words or lines spoken by characters.

[1389] "Scene" refers to the scenery or scene depicted in the story.

[1390] "Narrative structure" refers to the structure and arrangement that determines the flow and development of a story.

[1391] A "foreshadowing" is an element or hint placed in the first half of a story that will have important meaning later on.

[1392] "Mystery elements" are elements that add mystery and suspense to a story.

[1393] A "moving scene" is one that strongly stirs the emotions of the viewer.

[1394] "Graphics" refers to images and illustrations generated to match each scene in the story.

[1395] "Songs" refer to music and melodies that are generated to match each scene in the story.

[1396] "Production effects" refer to effects that use story structure, character dialogue, images, and music to stir the viewer's emotions.

[1397] The present invention relates to a story generation system, and aims to generate a consistent story based on a story setting input by a user, and to effectively stir the emotions of viewers.

[1398] System configuration

[1399] User Input

[1400] The user inputs the story setting, such as the theme, setting, and characters, in text format. For example, the user can provide a specific setting such as "a knight's adventure story set in medieval Europe."

[1401] Server input parsing

[1402] The server analyzes the story settings received from the user. This analysis uses natural language processing techniques, specifically the Python NLTK library and SpaCy. The server tokenizes the input text and extracts important keywords and phrases.

[1403] Searching the database

[1404] The server searches an existing story database based on the extracted keywords and phrases. This database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information.

[1405] Prompt generation for generative AI models

[1406] The server generates prompts to be input to the generative AI model based on user input and information obtained from the database. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and descriptions of the scenery."

[1407] Story generation using generative AI models

[1408] The server inputs prompts into a generative AI model (e.g., OpenAI's GPT-4) to generate the story flow, character dialogue, and scenery. Based on the user's input, the generative AI model generates text that matches the story's theme and the characters' personalities.

[1409] Applying the Emotion Engine

[1410] The server applies an emotion engine to the generated story, adding emotional elements to each scene of the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer.

[1411] Graphics and Music Generation

[1412] The server automatically creates graphics and music that match each scene in the story, using image generation AI to generate the graphics and music generation AI to generate the music.

[1413] Final output integration

[1414] The server integrates the generated story, graphics, and music and provides them to the user, who can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy the graphics and music.

[1415] Examples of specific examples and prompts

[1416] Specific examples

[1417] If the user inputs the setting "A knight's adventure story set in medieval Europe," the server will generate the following story:

[1418] The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

[1419] Character Quote: "Arthur: 'I swear on this sword I will slay the dragon!'"

[1420] Scene description: "Under the setting sun, Arthur gripped his sword and renewed his resolve."

[1421] Prompt Sentence Examples

[1422] An example of a prompt to input to the generative AI model is:

[1423] "Create a knightly adventure story set in medieval Europe. The main character is Arthur, a brave knight whose mission is to slay an evil dragon and save the kingdom. Include an introduction, character dialogue, and a description of the setting."

[1424] By inputting this prompt into a generative AI model, each element of the story is automatically generated.

[1425] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1426] Step 1:

[1427] The user inputs the story setting, such as the theme, setting, and characters, in text format. For example, they might input "A knight's adventure story set in medieval Europe." This input is done through the system's user interface. The input text is then sent to the server.

[1428] Step 2:

[1429] The server analyzes the story setting received from the user. Specifically, it uses Python's NLTK library and SpaCy to tokenize the text and extract important keywords and phrases. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis are used in the next step.

[1430] Step 3:

[1431] The server searches an existing story database based on the extracted keywords and phrases. The database contains information on past stories and characters. The server finds similar stories and character settings and uses them as reference information. For example, it searches for past stories with a "medieval Europe" theme or stories with a "knight" as the main character. The search results are used in the next step.

[1432] Step 4:

[1433] The server generates a prompt to be input to the generative AI model based on the user's input and information obtained from the database. For example, it might create a prompt such as, "Create a knight's adventure story set in medieval Europe. The main character is a brave knight named Arthur, whose mission is to defeat an evil dragon and save the kingdom. Please include the opening of the story, lines from the characters, and a description of the scenery." This prompt will be used in the next step.

[1434] Step 5:

[1435] The server inputs prompts into a generative AI model (e.g., OpenAI's GPT-4) to generate the story flow, lines for the characters, and scenery. Based on the user's input, the generative AI model generates text that matches the theme of the story and the personalities of the characters. For example, the generative AI model generates the following story:

[1436] The story begins: "In the vast plains of medieval Europe stood the brave knight Arthur. His mission was to slay the evil dragon and save the kingdom."

[1437] Character Quote: "Arthur: 'I swear on this sword I will slay the dragon!'"

[1438] Scene description: "Under the setting sun, Arthur gripped his sword and renewed his resolve."

[1439] Step 6:

[1440] The server applies an emotion engine to the generated story and adds emotional elements to each scene of the story. For example, it automatically generates foreshadowing, mystery elements, and moving scenes. This emotion engine uses IBM Watson's Tone Analyzer. The emotion engine analyzes the generated text and adds appropriate emotional elements.

[1441] Step 7:

[1442] The server automatically creates graphics and music that match each scene in the story. Image generation AI is used to generate the graphics, and music generation AI is used to generate the music. For example, to match the opening scene of the story, it generates graphics depicting the scenery of medieval Europe and music that evokes a sense of adventure.

[1443] Step 8:

[1444] The server integrates the generated story, graphics, and music and provides them to the user, who can then view and enjoy the completed story. For example, the user can view the story through a web browser and enjoy the graphics and music.

[1445] (Application example 1)

[1446] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1447] Conventional story generation systems can generate stories based on user-entered settings, but they cannot visualize and sound out the generated story in real time and allow users to experience it in a virtual reality space. It is also difficult to automatically generate graphics and music that match each scene of the story, enhancing the dramatic effects that effectively move the viewer's emotions. This limits the user's experience and makes the story less immersive.

[1448] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1449] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically generating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating a story using a generative AI model based on the story settings input by the user; means for visualizing and audibly converting the generated story in real time; and means for the user to experience the generated story in a virtual reality space using a head-mounted display. This enables the user to visualize and audibly convert the generated story in real time and experience it in a virtual reality space.

[1450] A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

[1451] The "setting" refers to the place and historical background in which the story unfolds, and is an element that forms the setting and atmosphere of the story.

[1452] A "character" is someone who acts in a story and plays an important role in the progression and development of the story.

[1453] "Story setting" refers to the setting that includes the basic elements of the story, such as theme, setting, and characters, and forms the foundation of the story.

[1454] A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically create story structures and dialogue.

[1455] A "foreshadowing" is an element or hint in a story that suggests future developments, and is a technique used to make a story more interesting.

[1456] A "mystery element" is an element in a story that presents a mystery or question, and is intended to attract the interest of readers or viewers.

[1457] A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer's mind.

[1458] "Graphics" refers to visual images or footage that visually represent scenes and characters in a story.

[1459] "Song" refers to music, and is used to create scenes and atmosphere in a story.

[1460] "Visualization" refers to the visual representation of the story content, allowing users to enjoy the story visually.

[1461] "Audio" refers to expressing the content of a story in audio form, allowing users to enjoy the story by listening to it.

[1462] A "virtual reality space" is a virtual space created using computer technology that allows users to experience reality.

[1463] A "head-mounted display" is a display device worn on the head, which is used to allow users to experience virtual reality spaces.

[1464] A system for implementing this invention has the following configuration: First, a user inputs story settings such as the theme, setting, and characters of the story using a smartphone or head-mounted display. These inputs are made in text format.

[1465] Next, the server uses a generative AI model (e.g., GPT-4) to automatically create character dialogue, scenes, and a story structure based on the story setting entered by the user. The generative AI model references an existing story database to generate dialogue and scenes that match the story flow, character personalities, and story theme.

[1466] Furthermore, the server uses an emotion engine to add hints, mystery elements, and moving scenes to the stories created, making them more interesting and moving.

[1467] Next, the server uses a graphics generation engine and a music generation engine to automatically create graphics and music that match each scene of the story, enhancing the visual and auditory effects of the story and effectively moving the viewer's emotions.

[1468] Finally, the generated story is visualized and audible in real time, and the user can experience the story in a virtual reality space using a head-mounted display, allowing the user to immerse themselves in the story and gain a deeper experience.

[1469] As a concrete example, if a user inputs "a knight's adventure story set in medieval Europe," the following prompt sentence will be input to the generative AI model.

[1470] Example prompt sentence:

[1471] Theme: Medieval Europe

[1472] Setting: Knight's Adventure

[1473] Characters: Brave knights, wizards, and dragons

[1474] Story Setting: A brave knight and a wizard embark on an adventure to slay a dragon.

[1475] Based on this prompt, the generative AI model generates a story, the emotion engine adds emotional elements, and the graphics and music engines generate graphics and music. Users can experience this story in real time through a head-mounted display.

[1476] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1477] Step 1:

[1478] The user uses a smartphone or head-mounted display to input story settings such as the theme, setting, and characters in text format, and the input data is sent to the server.

[1479] Input: Story setting such as theme, setting, characters, etc.

[1480] Output: Story setting data sent to the server

[1481] Step 2:

[1482] Based on the received story setting data, the server uses a generative AI model (e.g., GPT-4) to automatically create the story structure, character dialogue, and scenery. The generative AI model references an existing story database to generate text that matches the story flow and character personalities.

[1483] Input: Story setting data

[1484] Output: Generated story structure, dialogue, and scenery

[1485] Step 3:

[1486] The server uses an emotion engine to add emotional elements to the generated story, such as foreshadowing, mystery elements, and moving scenes. The emotion engine automatically inserts emotional elements appropriate for each scene in the story.

[1487] Input: Generated story structure, dialogue, and scenery

[1488] Output: A story with added emotional content

[1489] Step 4:

[1490] The server uses a graphics generation engine to automatically create graphics that match each scene in the story, taking into account the atmosphere of the scene and the emotional state of the characters to generate appropriate visuals.

[1491] Input: A story with added emotional content

[1492] Output: Graphics that match each scene

[1493] Step 5:

[1494] The server uses a music generation engine to automatically create music that matches each scene in the story, taking into account the atmosphere of the scene and the emotional state of the characters.

[1495] Input: A story with added emotional content

[1496] Output: Songs that match each scene

[1497] Step 6:

[1498] The server integrates the generated story, graphics, and music, and visualizes and sounds them in real time, allowing users to experience the story visually and aurally.

[1499] Input: Generated story, graphics, music

[1500] Output: Real-time visualised and spoken narrative

[1501] Step 7:

[1502] Using a head-mounted display, users experience stories generated in a virtual reality space, immersing themselves in the story and providing a deeper experience.

[1503] Input: Real-time visualised and spoken story

[1504] Output: A narrative experience in a virtual reality space

[1505] Example 2

[1506] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1507] Conventional story generation systems can automatically generate a story based on the story's theme and character settings, but they have difficulty automatically generating foreshadowing, mystery elements, and moving scenes to make the story more interesting. Furthermore, they lack the ability to adjust the story's development in real time based on the user's emotions, making it impossible to provide a narrative experience that is in tune with the user's emotions. Furthermore, it is difficult to automatically create graphics and music that match each scene in the story. A new system that can solve these issues is needed.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1509] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character lines, scenery, and story structure based on the input story settings, means for automatically generating hints, mystery elements, and moving scenes to make the created story more interesting, means for recognizing the user's emotions in real time and adjusting the story development in accordance with those emotions, means for automatically creating graphics and music that match each scene of the created story, and means for enhancing the dramatic effect with the story structure, character conversation, images, and music to effectively stir the viewer's emotions. This makes it possible not only to automatically generate a story based on the story theme and character settings, but also to automatically generate hints, mystery elements, and moving scenes, and provide a story development that matches the user's emotions.

[1510] A "story theme" indicates the overall subject or direction of a story.

[1511] "Setting" refers to the place or environment in which the story unfolds.

[1512] "Characters" refers to the characters who take action in the story.

[1513] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[1514] "Dialogue" refers to the words or lines spoken by characters.

[1515] "Scene" refers to the scenery or scene depicted in the story.

[1516] "Story structure" refers to the overall flow or structure of a story, including a beginning, middle, and end.

[1517] A "foreshadowing" is an element that is placed in advance in a way that will have important meaning later in the story.

[1518] A "mystery element" refers to an element in a story that attracts the interest of readers or viewers by including mysteries or unknown points in the story.

[1519] A "moving scene" is one that strongly stirs the emotions of the reader or viewer.

[1520] "User emotions" refer to emotions such as joy, sadness, and surprise felt by a user experiencing a story.

[1521] "Real-time recognition" refers to detecting the user's emotions instantly on the spot.

[1522] "Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

[1523] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[1524] "Song" refers to the music that plays along with the scenes in the story.

[1525] "Dramatic effects" refer to the effects of using the visual and auditory elements of a story to stimulate the emotions and interest of readers or viewers.

[1526] This invention is a system that automatically generates a story based on the story's theme and character settings, and also automatically generates plot twists, mystery elements, and moving scenes. It also has the ability to recognize the user's emotions in real time and adjust the story development accordingly.

[1527] Hardware and software used

[1528] The server generates story elements using a generative AI model (e.g., GPT-4). Based on information received from the user about the story's theme, the characters' personalities, and the story's progress, the server inputs prompt sentences into the generative AI model and generates story elements.

[1529] The device transmits the user's input information to the server, displays the story elements received from the server to the user, and uses emotion recognition software to collect emotion data from the user's facial expressions and voice and transmit it to the server.

[1530] Users can enjoy the story by inputting information about the story's theme, the personalities of the characters, and the story's progress into the device. The device recognizes the user's emotions in real time and reflects them in the development of the story.

[1531] Data processing and calculation

[1532] The server performs the following data processing and calculations:

[1533] Generate appropriate hints based on the story's theme and the characters' personalities.

[1534] Mystery elements are scattered throughout the story as it progresses.

[1535] Analyze the emotions of the characters and the situation to create a moving scene.

[1536] The device collects emotional data from the user's facial expressions and voice and sends it to the server, which analyzes the data and adjusts the story development accordingly.

[1537] Specific examples

[1538] For example, to generate a plot twist in which a knight meets someone who plays an important role later in the story, the following prompt sentence can be input to the generative AI model:

[1539] Example prompt sentence:

[1540] "Please create a plot twist that will lead to the mysterious old man the knight meets during his adventure playing an important role later in the story."

[1541] Or, if the user is feeling joyful about the story, the prompt to make the story even more upbeat might look like this:

[1542] Example prompt sentence:

[1543] "Users are enjoying it, so please add a more upbeat twist to the story. For example, add a scene where the knight finds a new companion."

[1544] In this way, the server utilizes a generative AI model and an emotion engine to automatically generate a story development that corresponds to the user's emotions.

[1545] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1546] Step 1:

[1547] The user inputs information about the story's theme, the characters' personalities, and the story's progression.

[1548] Input: Story theme (e.g., adventure), character traits (e.g., brave), and information about the story's progression

[1549] Output: User input information

[1550] Specific operation: The user enters the theme "adventure" into the input form on the device and sets the character's personality to "brave."

[1551] Step 2:

[1552] The terminal transmits the user's input information to the server.

[1553] Input: User input information

[1554] Output: User input sent to the server

[1555] Specific operation: The device sends the information "Theme: Adventure, Character: Brave" to the server.

[1556] Step 3:

[1557] The server generates story elements using generative AI models.

[1558] Input: User input information

[1559] Output: Generated story elements

[1560] Specific operation: The server inputs the prompt "Generate a story with an adventure theme and a brave knight as the protagonist" to the generation AI model, and generates elements of the story.

[1561] Step 4:

[1562] The server sends the generated story elements to the terminal.

[1563] Input: Generated story elements

[1564] Output: Story elements sent to the device

[1565] Specific operation: The server sends the story elements it generates (e.g., "The mysterious old man the knight meets during his adventure") to the terminal.

[1566] Step 5:

[1567] The terminal displays the story to the user.

[1568] Input: Story elements received from the server

[1569] Output: The story displayed to the user

[1570] Specific operation: The device displays a scene to the user in which a knight meets a mysterious old man during his adventure.

[1571] Step 6:

[1572] Recognize user emotions in real time.

[1573] Input: User's facial expressions and voice

[1574] Output: Recognized user emotion data

[1575] Specific operation: When the user smiles, the device captures the user's facial expression with the camera and recognizes that the user is feeling happy.

[1576] Step 7:

[1577] The terminal transmits the user's emotion data to the server.

[1578] Input: Recognized user emotion data

[1579] Output: Emotion data sent to the server

[1580] Specific operation: The device sends emotional data to the server, stating that "the user is feeling happy."

[1581] Step 8:

[1582] The server analyzes the emotional data and adjusts the story development.

[1583] Input: User emotion data

[1584] Output: Coordinated story development

[1585] Specific operation: The server analyzes the emotional data and inputs prompt sentences into the generative AI model to "make the story more cheerful."

[1586] Step 9:

[1587] The server sends the adjusted story to the device.

[1588] Input: Coordinated story development

[1589] Output: The adjusted story sent to the terminal.

[1590] Specific operation: The server generates a story with an upbeat plot (e.g., "A scene in which the knight finds a new companion") and sends it to the device.

[1591] Step 10:

[1592] The device displays the adjusted story to the user.

[1593] Input: Adjusted story received from the server

[1594] Output: The adjusted story displayed to the user

[1595] Specific operation: The device displays to the user a scene in which a knight finds a new companion.

[1596] (Application example 2)

[1597] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1598] Conventional story generation systems can automatically generate a story based on the story's theme and character settings, but they are unable to adjust the story's development in real time in response to the user's emotions. Furthermore, they lack the functionality to automatically generate plot twists, mystery elements, and moving scenes to make a story more interesting. This makes it difficult to provide an interactive story experience that responds to the user's emotions.

[1599] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1600] In this invention, the server includes: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating character dialogue, scenery, and a story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect of the story structure, character dialogue, and images and music to effectively stir the viewer's emotions; means for recognizing a user's emotions in real time and adjusting the development of the story in response to those emotions; and means for generating story prompts using a generative AI model and automatically generating a story. This makes it possible to provide an interactive story experience that corresponds to the user's emotions.

[1601] A "story theme" refers to the central subject or message of a story.

[1602] "Setting" refers to the place or environment in which the story unfolds.

[1603] "Characters" refers to the characters and people who play roles in a story.

[1604] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[1605] "Dialogue" refers to the words or lines spoken by characters.

[1606] "Scene" refers to the scenery or scene depicted in the story.

[1607] "Story structure" refers to the overall flow or structure of a story, including a beginning, middle, and end.

[1608] A "foreshadowing" is an element or hint placed in the first half of a story that will have important meaning later in the story.

[1609] "Mystery elements" refer to elements in a story that raise questions or raise doubts.

[1610] A "moving scene" is one that evokes strong emotions in the viewer or reader.

[1611] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[1612] "Songs" refers to the music or soundtrack that accompanies the story scenes.

[1613] "Stage effects" refer to visual and auditory techniques used to make story scenes appear more effective.

[1614] "Viewer sentiment" refers to the emotions and feelings of the people watching the story.

[1615] "Recognizing emotions in real time" refers to analyzing the user's facial expressions, tone of voice, etc. to grasp their emotions at that moment.

[1616] "Adjusting the development of the story" refers to changing the progress and content of the story in response to the user's emotions.

[1617] A "generative AI model" refers to a model that uses artificial intelligence to generate text or stories.

[1618] A "prompt" refers to an instruction or keyword to be input into a generative AI model.

[1619] The system for implementing this invention has the following configuration: First, the server provides a means for inputting story settings such as the theme, setting, characters, etc. By inputting these settings, the user determines the basic elements of the story.

[1620] Next, the server has a means to automatically create character dialogue, scenery, and story structure based on the input story setting. This means uses a generative AI model to generate story prompts and automatically generate the story.

[1621] Furthermore, the server has the means to automatically generate hints, mystery elements, moving scenes, etc. to make the story more interesting, thereby adding depth and interest to the story.

[1622] The server also has the means to automatically create graphics and music that match each scene of the story, enhancing the visual and auditory effects of the story. Specifically, the server creates graphics and music taking into account the atmosphere of the scene and the emotional state of the characters.

[1623] Furthermore, the server has the means to effectively stir the viewer's emotions by enhancing the dramatic effects of the story structure, the characters' conversations, images, and music, thereby drawing the user more emotionally into the story.

[1624] What is particularly important is that the server has a means to recognize the user's emotions in real time and adjust the story development accordingly. The smartphone's camera and microphone are used to analyze the user's facial expressions and tone of voice to understand the user's emotions. Based on this emotional data, the server generates prompt sentences for the story and adjusts the story development accordingly.

[1625] For example, if a camera captures the user's facial expressions while the user is reading a story and recognizes that the user is smiling, the story will develop in a lighthearted and joyful way. Conversely, if the user is sad, the story will develop in a more profound way.

[1626] An example of a prompt is as follows:

[1627] "The protagonist continues his adventure with a happy feeling."

[1628] "The protagonist faces difficulties but does not lose hope."

[1629] The protagonist takes on a new mystery.

[1630] In this way, it is possible to provide an interactive storytelling experience that responds to the user's emotions.

[1631] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1632] Step 1:

[1633] The user inputs the story setting, such as the theme, setting, and characters.

[1634] Input: Setting information such as the story's theme, setting, and characters entered by the user.

[1635] Output: Story setting data.

[1636] How it works: A user uses a smartphone application to input the basic setting of a story, including information about the story's theme, location, and main characters. The server receives this setting information and stores it as story setting data.

[1637] Step 2:

[1638] Based on the story settings entered into the server, the server automatically creates the characters' lines, scenes, and story structure.

[1639] Input: Story setting data.

[1640] Output: Story lines, scenes, and composition data.

[1641] How it works: The server uses a generative AI model to generate character dialogue, scenery, and story structure based on story setting data. The generated data is saved as elements necessary for the story's progression.

[1642] Step 3:

[1643] The server automatically generates hints, mystery elements, moving scenes, and other elements to make the stories it creates more interesting.

[1644] Input: Story dialogue, scenery, and composition data.

[1645] Output: Foreshadowing, mystery elements, and moving scene data.

[1646] How it works: The server uses a generative AI model to generate plot twists, mystery elements, and moving scenes based on the dialogue, scenery, and composition data of the story, adding depth and intrigue to the story.

[1647] Step 4:

[1648] The server automatically creates graphics and music that match each scene in the story created by the server.

[1649] Input: Foreshadowing, mystery elements, and moving scene data.

[1650] Output: Graphic data, music data.

[1651] How it works: The server automatically creates graphics and music, taking into account the atmosphere of the scene and the emotional state of the characters. This is done using image generation models and music generation models. The generated graphics and music are then applied to each scene in the story.

[1652] Step 5:

[1653] The server recognizes the user's emotions in real time and adjusts the story development accordingly.

[1654] Input: User's facial expression data, tone of voice data.

[1655] Output: Tailored narrative prompt.

[1656] Specific operation: The system uses the smartphone's camera and microphone to capture the user's facial expressions and tone of voice. The server analyzes this data and recognizes the user's emotions in real time. Based on the recognized emotions, it generates prompt sentences for the story and adjusts the story development.

[1657] Step 6:

[1658] The server uses a generative AI model to generate story prompts and automatically generate the story.

[1659] Input: Tailored narrative prompt sentence.

[1660] Output: The generated story.

[1661] Specific operation: The server uses the generative AI model to generate a story based on the adjusted prompt sentences. The generated story is used to provide an interactive experience that responds to the user's emotions.

[1662] Example 3

[1663] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1664] Conventional story generation systems struggled to automatically create graphics and music that matched each scene in the story, making it difficult to effectively sway the viewer's emotions. Furthermore, they lacked the ability to recognize the user's emotions in real time and adjust the story's development accordingly, making it difficult to provide a consistent emotional experience to the user. This resulted in a decline in the quality of the story experience and lowered user satisfaction.

[1665] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1666] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character lines, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for using information such as the user's facial expressions, tone of voice, and text input to recognize the user's emotions; means for adjusting the story development in accordance with the user's emotions; and means for enhancing the dramatic effects of the story structure, character conversation, images, and music to effectively stir the viewer's emotions. This makes it possible to automatically create graphics and music that match each scene of the story and provide a story development that matches the user's emotions.

[1667] A "story theme" refers to the central subject or theme of a story.

[1668] "Setting" refers to the place or environment in which the story unfolds.

[1669] "Characters" refers to the characters that appear in the story.

[1670] "Story setting" refers to the setting, including basic elements such as the theme, setting, and characters of the story.

[1671] "Dialogue" refers to the words spoken by the characters.

[1672] "Scene" refers to the scenery or scene depicted in the story.

[1673] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[1674] A "foreshadowing" is an element that provides advance information or events that will be important later in the story.

[1675] "Mystery elements" refer to elements that add mystery or suspense to a story.

[1676] A "moving scene" refers to a scene that strongly stirs the viewer's emotions.

[1677] "Graphics" refers to images and illustrations generated to match the story scenes.

[1678] "Song" refers to music generated to match the scenes in the story.

[1679] "User emotion" refers to the emotional state such as joy, sadness, surprise, etc., that the user is feeling.

[1680] "Facial expression" refers to emotions expressed by the movement of the user's facial muscles.

[1681] "Tone of voice" refers to the pitch, strength, and intonation of the user's voice.

[1682] "Text input" refers to character information that a user inputs using a keyboard or touch panel.

[1683] "Story development" refers to the process or flow of the story.

[1684] "Staging effects" refers to the impact that the visual and auditory elements of a story have on the viewer.

[1685] "Audience sentiment" refers to the emotions and feelings of the people watching the story.

[1686] This invention is a system that inputs story settings such as the story theme, setting, and characters, automatically generates a story based on the input story settings, and further adjusts the development of the story according to the user's emotions. This system is implemented using the following hardware and software.

[1687] Hardware and software used

[1688] Hardware: High-performance servers (e.g., servers equipped with GPUs) and user devices (e.g., PCs or smartphones equipped with cameras and microphones)

[1689] Software: Generative AI models (e.g., GPT-4, image generation AI, music generation AI), emotion recognition software (e.g., OpenCV, voice analysis libraries)

[1690] Explanation of program processing

[1691] 1. Narrative input and analysis

[1692] The user uses a device to input the story setting, such as the theme, setting, and characters. For example, they might input "a scene where a hero fights a dragon." The server receives this input and uses a generative AI model (e.g., GPT-4) to analyze each scene in the story and extract the mood of the scene and the emotional state of the characters.

[1693] 2. Graphics and Music Generation

[1694] Based on the analysis results, the server uses image generation AI to generate graphics that match the scene. For example, it generates powerful graphics of a hero fighting a dragon. Similarly, it uses music generation AI to generate music that suits the scene. For example, it generates music that heightens the tension of a battle scene.

[1695] 3. User Emotion Recognition

[1696] The device uses a camera and microphone to capture the user's facial expressions and tone of voice. For example, if the user is smiling, the device sends that information to the server. The server then uses emotion recognition software (e.g., OpenCV, a voice analysis library) to analyze the user's emotions.

[1697] 4. Adjusting the story development

[1698] The server uses an emotion engine to adjust the story development based on the user's emotional information. For example, if the user is smiling, the server adjusts the story development to a more exciting direction.

[1699] Examples of specific examples and prompts

[1700] Examples:

[1701] The user inputs "a scene in which a hero fights a dragon."

[1702] The server analyzes it as a battle scene and uses image generation AI to generate powerful graphics of the dragon and hero.

[1703] Music generation AI is used to generate music that heightens tension.

[1704] If the user is smiling, the emotion engine will recognize that the user is having fun and adjust the story development in a more exciting direction.

[1705] Example prompt sentence:

[1706] "Generate graphics of a scene where a hero fights a dragon."

[1707] "Generate music to accompany moving scenes."

[1708] "If the user is smiling, adjust the storyline to be more fun."

[1709] This system automatically creates graphics and music that match each scene of the story, making it possible to provide a story development that matches the user's emotions. This effectively stirs the viewer's emotions and improves the quality of the story experience. The flow of the specific processing in Example 3 will be described with reference to FIG. 21.

[1710] Step 1:

[1711] The user enters the text of the story.

[1712] The user enters "A scene in which a hero fights a dragon" into the text input field on the terminal and clicks the send button. The entered text is sent from the terminal to the server.

[1713] Step 2:

[1714] The server parses the story text.

[1715] The server inputs the received text into a generative AI model (e.g., GPT-4) to analyze each scene in the story. As a result of the analysis, the atmosphere of the scene and the emotional state of the characters are extracted. For example, "fight scene" and "tension" are extracted. The analysis results are stored in an internal database.

[1716] Step 3:

[1717] The server generates graphics that match the scene.

[1718] Based on the analysis results, the server uses an image generation AI (e.g., DALL-E) to generate graphics that match the scene. For example, you can input "a scene where a dragon and a hero are fighting" as a prompt and obtain the generated graphics. The generated graphics are then sent to the user's device.

[1719] Step 4:

[1720] The server generates music that matches the scene.

[1721] The server uses music generation AI to generate music appropriate for the scene. For example, a user can input "music that will increase tension in a battle scene" as a prompt and obtain the generated music. The generated music is then sent to the user's device.

[1722] Step 5:

[1723] The device recognizes the user's emotions.

[1724] The device captures the user's facial expressions with a camera and records the tone of voice with a microphone. This data is sent to the server in real time. For example, if the user is smiling, that information is sent to the server.

[1725] Step 6:

[1726] The server adjusts the story development according to the user's emotions.

[1727] The server analyzes the received emotion data using emotion recognition software (e.g., OpenCV, speech analysis library) to recognize the user's emotions. For example, if it recognizes that the user is smiling, it uses the emotion engine to adjust the story development in an exciting direction. A new scene is generated and sent to the user's device.

[1728] (Application example 3)

[1729] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1730] Conventional story generation systems have the problem that they do not adjust the story according to the user's emotions, resulting in a uniform user experience. Also, because the story scenes and music are fixed, they are unable to provide dynamic changes according to the user's emotions. This makes it difficult to provide a story experience that deeply appeals to the user's emotions.

[1731] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1732] In this invention, the server includes means for inputting story settings such as the story theme, setting, and characters, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for automatically producing graphics and music that match each scene of the created story, means for enhancing the dramatic effect with the story structure, character conversation, images, and music to effectively stir the viewer's emotions, means for recognizing the user's emotions and adjusting the story scenes and music in accordance with the recognized emotions, and means for recognizing emotions using the user's facial expressions, tone of voice, text input, etc. This enables a dynamic story experience that corresponds to the user's emotions.

[1733] A "story theme" refers to the central idea or message of a story.

[1734] "Setting" refers to the place or environment in which the story unfolds.

[1735] "Characters" refers to the characters that appear in the story.

[1736] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[1737] "Dialogue" refers to the words spoken by the characters.

[1738] "Scene" refers to the scenery or scene depicted in the story.

[1739] "Story structure" refers to the overall flow of a story, including a beginning, middle, and end.

[1740] A "foreshadowing" is an element in the first half of a story that becomes important later in the story.

[1741] "Mystery elements" refer to elements that add mystery or suspense to a story.

[1742] A "moving scene" is one that strongly stirs the emotions of the viewer.

[1743] "Graphics" refers to images and videos that visually represent scenes from a story.

[1744] "Song" refers to the music that plays along with the scenes in the story.

[1745] "Direction effects" refer to effects that influence the viewer's emotions through story scenes and music.

[1746] "User emotion" refers to the emotional state, such as joy, sadness, or surprise, that the user is feeling.

[1747] "Means for recognizing emotions" refers to technology that determines a user's emotions using the user's facial expressions, tone of voice, text input, etc.

[1748] The "means for adjusting the story scenes and music" refers to a technique for changing the story scenes and music according to the user's recognized emotions.

[1749] The system for implementing the present invention can dynamically adjust the story scenes and music according to the user's emotions. A specific embodiment of this system will be described below.

[1750] System Configuration

[1751] The system includes the following major hardware and software components:

[1752] Hardware: Smartphone camera

[1753] Software: OpenCV (face detection), Keras (emotion recognition model), music playback software

[1754] Program processing

[1755] The server first captures the user's facial expression using the smartphone camera, then performs face detection using OpenCV, and then uses Keras' emotion recognition model to recognize the user's emotion from the detected face.

[1756] Based on the recognized emotion, the server adjusts the story scene and music. For example, if the user is smiling, the story scene will change to a bright and cheerful one and the music will also change to a lighter one. Conversely, if the user has a sad expression, the story will transition to a more emotional scene and the music will also change to a more moving one.

[1757] Specific examples

[1758] As a concrete example, if the user smiles while enjoying a story, the server would do the following:

[1759] 1. Capture the user's facial expressions using the smartphone camera.

[1760] 2. Detect faces using OpenCV.

[1761] 3. The user's emotion is recognized as "joy" using the Keras emotion recognition model.

[1762] 4. Based on the emotions identified, change the story scene to something lighter and more cheerful, and play upbeat music.

[1763] Prompt Sentence Examples

[1764] An example of a prompt to input to a generative AI model is as follows:

[1765] If the user smiles, change the story scene to something light and cheerful and play upbeat music.

[1766] In this way, it is possible to provide a dynamic story experience that responds to the user's emotions, allowing the user to be more deeply immersed in the story and experience a more emotional experience.

[1767] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1768] Step 1:

[1769] The server captures the user's facial expressions using the smartphone's camera. The input is the camera image, and the output is the captured image data. Specifically, the camera acquires the image in real time and captures the image frame by frame.

[1770] Step 2:

[1771] The server uses OpenCV to detect faces from the captured image data. The input is the captured image data, and the output is the location information of the detected faces. Specifically, the OpenCV face detection algorithm identifies the face in the image and obtains its coordinates.

[1772] Step 3:

[1773] The server uses a Keras emotion recognition model to recognize the user's emotion from the detected face. The input is face location information and image data, and the output is the recognized emotion label. Specifically, the server extracts the face area, performs preprocessing, and then inputs it into the emotion recognition model to classify the emotion.

[1774] Step 4:

[1775] The server adjusts the story scenes and music based on the recognized emotions. The input is the recognized emotion label, and the output is the adjusted scenes and music. The specific operation is to select scenes and music corresponding to the emotion label and change the progression of the story.

[1776] Step 5:

[1777] The server transmits the adjusted scene and music to the user's device. The input is the adjusted scene and music, and the output is the story scene displayed on the user's device and the music played. Specifically, the server encodes the scene and music data and transmits it to the user's device via the network.

[1778] Step 6:

[1779] The user's device plays the received scenes and music and provides them to the user. The input is the scene and music data received from the server, and the output is the video displayed to the user and the music played. Specifically, the device decodes the data, displays it on the screen, and plays the music from the speaker.

[1780] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1781] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1782] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1783] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1784] [Third embodiment]

[1785] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1786] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1787] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1788] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1789] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1790] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1791] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1792] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1793] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1794] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1795] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1796] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1797] "Example 1"

[1798] The system of the present invention receives input from the user about the story setting, such as the theme, setting, and characters. This input is provided in text format, such as "A knight's adventure story set in medieval Europe." Next, based on this input, the system automatically creates character dialogue, scenery, and a story structure. This creation is performed using existing story databases and artificial intelligence technology, generating dialogue and scenery that match the story flow, character personalities, and theme.

[1799] "Example 2"

[1800] Furthermore, the system automatically generates hints, mystery elements, and moving scenes to make the story more interesting. This is done in accordance with the story's theme, the characters' personalities, and the progression of the story, laying hints at the right time, sprinkling mystery elements, and creating moving scenes. For example, it automatically generates hints that people the knight meets during his adventure will play an important role in the story later on, elements that the knight must solve, and moving scenes when the knight overcomes difficulties.

[1801] "Example 3"

[1802] The system then automatically creates graphics and music that match each scene in the story. The graphics and music are created taking into account the atmosphere of the scene and the emotional state of the characters. For example, powerful graphics and music that heighten tension are created for battle scenes, while beautiful landscape graphics and moving music are created for moving scenes. This makes it possible to enhance the dramatic effect of the story structure, character dialogue, images, and music, effectively stirring the emotions of the viewer.

[1803] The processing flow of each embodiment will be described below.

[1804] "Example 1"

[1805] Step 1: The user provides textual input of the story setting, such as the theme, setting, and characters. For example, it could be provided as "a knight's adventure story set in medieval Europe."

[1806] Step 2: Based on the input, the system automatically generates character dialogue, scenery, and story structure, using existing story databases and artificial intelligence techniques.

[1807] Step 3: Lines and scenes are generated that fit the story flow, the characters' personalities, and the story's theme.

[1808] "Example 2"

[1809] Step 1: The system automatically generates plot twists, mystery elements, moving scenes, and other elements to make the story more interesting.

[1810] Step 2: Lay out hints at the right time, sprinkle in elements of mystery, and create moving scenes in line with the story's theme, the characters' personalities, and the progression of the story.

[1811] Step 3: For example, foreshadowing, such as people the knight meets during his adventure playing an important role in the story later on, puzzles the knight faces, and moving scenes when the knight overcomes difficulties, are automatically generated.

[1812] "Example 3"

[1813] Step 1: The system automatically creates graphics and music that match each scene in the story you create.

[1814] Step 2: Graphics and music are created taking into account the mood of the scene and the emotional state of the characters.

[1815] Step 3: For example, powerful graphics and music that heightens the tension are produced for battle scenes, while beautiful landscape graphics and moving music are produced for moving scenes. This enhances the dramatic effect of the story structure, the characters' conversations, images and music, and effectively stirs the viewer's emotions.

[1816] Example 1

[1817] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1818] With conventional story generation systems, when automatically generating a story based on a story setting entered by the user, it is difficult to maintain consistency in the flow of the story, the dialogue of the characters, and the scenery. Furthermore, there is a lack of ingenuity to ensure that the generated story is moving and entertaining for the viewer. Furthermore, there is an issue that it is not possible to automatically create graphics and music that match each scene of the story, making it difficult to effectively move the viewer's emotions.

[1819] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1820] In this invention, the server includes: means for inputting story settings such as the story theme, setting, and characters; means for automatically creating character dialogue, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect of the story structure, character dialogue, and video and music to effectively stir the viewer's emotions; means for receiving input from a user and analyzing it using natural language processing technology; means for generating prompt sentences to be input into a generative AI model based on the analysis results; means for automatically generating story dialogue, scenery, and structure using the generative AI model; and means for formatting the generated story and providing it to the user. This makes it possible to automatically generate a consistent story based on user input and provide the viewer with a moving and entertaining story.

[1821] The "theme of a story" refers to the central subject or theme of a story, and is an element that determines the overall direction and atmosphere of the story.

[1822] The "setting" refers to the place and historical background in which the story unfolds, and is an important element in forming the setting and atmosphere of the story.

[1823] A "character" is someone who acts in a story and moves the story forward, and is an element that has a major influence on the development and theme of the story.

[1824] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and is the element that forms the framework of the story.

[1825] "Dialogue" refers to the words or lines spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[1826] "Scenes" refer to the landscapes and scenes depicted in a story, and are elements that visually express the atmosphere and emotions of the story.

[1827] "Story structure" refers to the overall flow and structure of a story, including the beginning, middle, and end, and is an element that makes the development of a story effective.

[1828] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that adds consistency and depth to a story.

[1829] A "mystery element" is an element that presents a mystery or question in a story to attract the interest of readers or viewers.

[1830] A "moving scene" is a scene in a story that strongly stirs emotions and is an impressive element that stays in the viewer's mind.

[1831] "Graphics" refers to images and illustrations that visually represent each scene in the story, and are elements that visually complement the atmosphere and emotions of the story.

[1832] "Songs" are melodies and sounds that are musically expressed to match each scene in the story, and are elements that musically complement the atmosphere and emotions of the story.

[1833] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is used to analyze text data and extract information.

[1834] A "generative AI model" is a model that uses artificial intelligence technology to generate text and data, and is used to automatically generate dialogue, scenes, and structure for a story.

[1835] A "prompt" is an instruction to be input into a generative AI model, and is an element that guides the content that the model generates.

[1836] "Formatting" refers to the process of adding paragraph breaks and line breaks to make the generated narrative text easier to read before it is presented to the user.

[1837] This invention is a system that automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[1838] User input

[1839] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe."

[1840] The server receives and analyzes the input data

[1841] The server receives the text data sent by the user. It then analyzes the input data using natural language processing techniques. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story."

[1842] Generate prompt statement

[1843] The server generates a prompt to be input to the generative AI model based on the extracted information. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe."

[1844] Generating stories with generative AI models

[1845] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it might generate text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to slay the dragon.'"

[1846] Formatting and serving the generated results

[1847] The server receives the story text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted story is then converted into a format that is easy for the user to read. Finally, the server provides the formatted story to the user, who can view the results through their device.

[1848] Specific examples

[1849] Consider the user's input, "A knight's adventure set in medieval Europe." Based on this input, the server generates the following prompt:

[1850] "Generate lines and scenes from characters in a knight's adventure story set in medieval Europe."

[1851] The generative AI model generates a story based on this prompt, outputting text such as the following:

[1852] "The brave knight Arthur met a wizard in a dark forest. The wizard told Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'"

[1853] In this way, it is possible to automatically generate a coherent story based on user input, providing an inspiring and entertaining story for the viewer.

[1854] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1855] Step 1:

[1856] The user uses a terminal to enter text information about the story's theme, setting, characters, etc. into the system's input form. For example, they might enter "a knight's adventure story set in medieval Europe." This input data is then sent to the server.

[1857] Step 2:

[1858] The server receives the text data sent by the user. It analyzes the received data using natural language processing technology. Specifically, it uses Python's NLTK library and spaCy to extract information such as theme, setting, and characters from the input text. For example, it extracts keywords such as "medieval Europe," "knights," and "adventure story." The results of this analysis become the input for the next step.

[1859] Step 3:

[1860] The server generates a prompt sentence to be input to the generative AI model based on the analysis results. For example, if the user input is "A knight's adventure story set in medieval Europe," the prompt sentence would be "Please generate lines and scenes for the characters in a knight's adventure story set in medieval Europe." This prompt sentence becomes the input for the next step.

[1861] Step 4:

[1862] The server inputs the generated prompt into a generative AI model (for example, OpenAI's GPT-4). The generative AI model automatically generates the dialogue, scenery, and structure of a story based on the prompt. For example, it generates text such as, "The brave knight Arthur met a wizard in a dark forest. The wizard said to Arthur, 'Deep in this forest lies an ancient dragon. Your mission is to defeat the dragon.'" This generated text becomes the input for the next step.

[1863] Step 5:

[1864] The server receives the narrative text returned by the generative AI model and formats it as needed, for example by adding paragraph breaks and line breaks to make it easier to read. The formatted narrative is then converted into a format that is easier for the user to read. This formatted text becomes the input for the next step.

[1865] Step 6:

[1866] The server provides the formatted story to the user, who can then view the results through their device, for example, on a web page or application screen that displays the generated story. The user can then read and enjoy the generated story.

[1867] (Application example 1)

[1868] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1869] Conventional story generation systems automatically generate stories based on themes and settings entered by users, but the generation of story structure, character dialogue, and scenery is insufficient, making it difficult to effectively move the viewer's emotions. Furthermore, the lack of functionality to preview, save, and share the generated stories results in poor user convenience.

[1870] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1871] In this invention, the server includes: means for inputting story settings such as the theme, setting, and characters of the story; means for automatically creating character lines, scenery, and story structure based on the input story settings; means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting; means for automatically creating graphics and music that match each scene of the created story; means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewer's emotions; means for generating prompt sentences using a generative AI model and automatically generating a story; and means for previewing, saving, and sharing the generated story. This makes it possible to automatically generate high-quality stories based on the settings input by the user and effectively stir the viewer's emotions.

[1872] A "story theme" is the central subject or theme of a story, and determines the overall direction and atmosphere of the story.

[1873] The "setting" refers to the place and historical background in which the story unfolds, and is an element that forms the setting and atmosphere of the story.

[1874] "Characters" are characters that appear in a story and play an important role in the progression and development of the story.

[1875] "Story setting" refers to the setting that includes basic elements such as the theme, setting, and characters of the story, and forms the framework of the story.

[1876] "Dialogue" refers to the words spoken by characters, and is an important element in expressing the progression of the story and the character's personality.

[1877] "Scenes" are the landscapes and scenes depicted in a story, and visually express the atmosphere and emotions of the story.

[1878] "Story structure" refers to the overall flow and structure of a story, including a beginning, middle, and end, and is the framework for effectively developing the story.

[1879] A "foreshadowing" is a hint or omen that relates to later developments in a story, and is an element that makes a story more interesting.

[1880] "Mystery elements" are elements that add mystery and suspense to a story, and are meant to attract the interest of readers and viewers.

[1881] A "moving scene" is a scene in a story that stirs strong emotions and leaves a deep impression on the viewer.

[1882] "Graphics" refers to the visual images and illustrations that accompany each scene in the story, and are intended to visually express the atmosphere and emotions of the story.

[1883] A "song" is music created to accompany each scene in a story, and is intended to musically express the atmosphere and emotions of the story.

[1884] A "generative AI model" is a model that uses artificial intelligence technology to automatically generate text, images, etc., and is used to automatically generate stories.

[1885] A "prompt" is an instruction to be input into a generative AI model, and is generated based on the theme and setting of the story.

[1886] "Preview" is a function that allows users to check the generated story in advance and check the content.

[1887] "Preservation" refers to recording the generated stories in a digital format so that they can be reused and shared later.

[1888] "Sharing" refers to sharing the generated story with other users and platforms, in order to promote the spread and appreciation of the story.

[1889] A system for carrying out this invention automatically generates a story based on a story setting input by a user, such as the theme, setting, characters, etc. A specific embodiment of this system will be described below.

[1890] System Configuration

[1891] This system is mainly composed of the following hardware and software.

[1892] Hardware: Smartphones, servers

[1893] Software: Python, OpenAI API

[1894] Program processing

[1895] User Input

[1896] Users enter the story's theme, setting, and characters in text form through a smartphone application. For example, they can enter the theme "A knight's adventure story set in medieval Europe."

[1897] Prompt Generation

[1898] The server generates a prompt based on the information entered by the user. This prompt is an instruction that is input to the generative AI model. As a specific example, the following prompt is generated:

[1899] Theme: A knight's adventure set in medieval Europe

[1900] Setting: Medieval Europe

[1901] Characters: Brave knights, wizards, and dragons

[1902] Generate a story.

[1903] Story Generation

[1904] The server automatically generates a story based on the generated prompt sentence using a generative AI model (e.g., GPT-4). This generative AI model is called through the OpenAI API.

[1905] Preview, save, and share

[1906] The generated story is displayed for users to preview through the smartphone application, and users can save the story and share it on social media.

[1907] Specific examples

[1908] If a user sets the theme to "a knight's adventure story set in medieval Europe" and enters "a brave knight, a wizard, and a dragon" as the characters, the following story will be generated.

[1909] Theme: A knight's adventure set in medieval Europe

[1910] Setting: Medieval Europe

[1911] Characters: Brave knights, wizards, and dragons

[1912] Generate a story.

[1913] By inputting this prompt into a generative AI model, a story is automatically generated, which users can preview, save, and share.

[1914] Hardware and software used

[1915] Hardware: Smartphones, servers

[1916] Software: Python, OpenAI API

[1917] This makes it possible to automatically generate high-quality stories based on the settings entered by the user, effectively stirring the emotions of the viewer.

[1918] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1919] Step 1:

[1920] The user uses a smartphone to input the story's theme, setting, and characters in text format. The input data is then sent from the device to the server. An example input includes the theme "A knight's adventure story set in medieval Europe."

[1921] Step 2:

[1922] The server generates a prompt based on the received user input data. Specifically, it combines information about the theme, setting, and characters to create a prompt suitable for the generative AI model. The generated prompt looks like this:

[1923] Theme: A knight's adventure set in medieval Europe

[1924] Setting: Medieval Europe

[1925] Characters: Brave knights, wizards, and dragons

[1926] Generate a story.

[1927] This prompt sentence is generated.

[1928] Step 3:

[1929] The server calls a generative AI model (e.g., GPT-4) with the generated prompt sentence. It sends the prompt sentence using the OpenAI API and requests the generation of a story. The input is the prompt sentence, and the output is the generated story text.

[1930] Step 4:

[1931] The generative AI model automatically generates a story based on the prompt sentence. The generated story text is returned to the server. For example, the following story is generated:

[1932] Once upon a time, in a vast kingdom in medieval Europe, there lived a brave knight named Arthur. One day, with the help of a wizard, he embarked on an adventure to battle a fearsome dragon.

[1933] Step 5:

[1934] The server sends the generated story text to the smartphone terminal, and the user can preview the generated story through the smartphone terminal. The input is the generated story text, and the output is the preview screen displayed on the user's terminal.

[1935] Step 6:

[1936] Users can save the created story or share it on social media. Saved stories are recorded in the device's storage, and shared stories are sent to the specified platform. The input is the user's selection, and the output is the saved file or shared content.

[1937] By following the steps above, it is possible to automatically generate a high-quality story based on the settings entered by the user, effectively stirring the emotions of the viewer.

[1938] Example 2

[1939] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1940] Conventional story generation systems can automatically generate stories based on themes and character settings, but they struggle to properly generate plot twists, mystery elements, and moving scenes to make a story more interesting. Furthermore, they lack the functionality to organize the generated story elements and provide them to users, making it difficult for users to effectively utilize the generated story. Furthermore, they lack the means to enhance the dramatic effect of story structure, character dialogue, visuals, and music, and to effectively stir the viewer's emotions.

[1941] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1942] In this invention, the server includes means for inputting story settings such as the theme, setting, and characters of the story, means for automatically creating character dialogue, scenery, and story structure based on the input story settings, means for automatically generating foreshadowing, mystery elements, and moving scenes to make the created story more interesting, means for organizing the generated story elements and providing them to users, and means for enhancing the dramatic effect with the story structure, character dialogue, images, and music to effectively stir the viewers' emotions. This makes it possible to make the story more interesting and enable users to effectively use the created story.

[1943] A "story theme" refers to the central subject or theme of a story.

[1944] "Setting" refers to the place or environment in which the story unfolds.

[1945] "Characters" refers to the characters and people who appear in a story.

[1946] "Story setting" refers to the setting, including basic elements such as theme, setting, and characters of the story.

[1947] "Dialogue" refers to the words or lines spoken by characters.

[1948] "Scene" refers to the scenery or scene depicted in the story.

[1949] "Story structure" refers to the overall flow and structure of a story, including its beginning, middle, and end.

[1950] A "foreshadowing" is an element or event placed in the first half of a story in such a way that it will have an important meaning later on.

[1951] "Mystery elements" refer to elements that add mystery or detective elements to a story, thereby attracting the interest of readers and viewers.

[1952] A "moving scene" is a scene in a story that stirs strong emotions.

[1953] "Graphics" refers to images and illustrations that are displayed in accordance with the story scenes.

[1954] "Song" refers to the music or background music that plays along with the story scenes.

[1955] "Staging effects" refers to techniques that use the visual and auditory elements of a story to elicit emotions and responses from the viewer.

[1956] "Audience sentiment" refers to the emotions and fe...

Claims

1. A means for receiving text data including information on the theme, setting, and characters of a story input by a user and analyzing the data using natural language processing technology; A means for generating a prompt sentence to be input to the generation AI model based on the analysis result; A means for automatically creating a story structure based on the prompt sentence using the generative AI model; A means for automatically generating at least one of foreshadowing, mystery elements, and moving scenes using the generative AI model; means for analyzing a story scene and extracting the mood of said scene and the emotional state of the characters; a means for automatically creating graphics corresponding to the story using a graphics generation engine and automatically creating music corresponding to the story using a music generation engine based on the extracted atmosphere and emotional state; means for recognizing a user's emotion; a means for generating prompt sentences to be input to the generative AI model based on the recognized emotion of the user, thereby adjusting the scenes and music of the story according to the recognized emotion of the user; A system including:

2. the means for automatically creating a story structure creates the story based on a three-act structure consisting of a beginning, a middle, and an end of the story; The system of claim 1 .

Images