System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving information input by a user; means for generating a customized story based on the received information; means for generating an illustration based on the generated story; means for combining the story and the illustration to generate a digital preview of the picture book; means for confirming the digital preview by the user and sending a modification request if necessary; means for generating and providing a final version of the digital picture book to the user; and means for providing printing and delivery options if desired by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to find picture books that are suitable for children, especially when providing stories that match their interests and preferences. As a result, children often lose interest in picture books, resulting in unfulfilling parent-child reading time. Furthermore, many parents lack confidence in their ability to read picture books aloud, which can lead to ineffective reading. Therefore, there is a need for a system that can generate customizable picture books tailored to children's interests and preferences, providing a fun reading experience. [Means for solving the problem]

[0005] The present invention provides a system including: a means for receiving information input by a user; a means for generating a customized story based on the received information; a means for generating illustrations based on the generated story; a means for combining the story and illustrations to generate a digital preview of a picture book; a means for the user to check the digital preview and submit correction requests as needed; a means for generating a final digital picture book and providing it to the user; and a means for providing printing and delivery options if the user desires. Furthermore, by using a generative AI model to generate a customized story and an image generation engine to generate illustrations, an original picture book tailored to a child's interests and preferences can be generated. This allows children to read picture books with interest, making parent-child reading time more enjoyable.

[0006] "User" refers to a person who uses the system to input the necessary information to create a customized picture book and checks and modifies the created picture book.

[0007] "Information to be entered" refers to data required to generate a customized picture book, such as the child's name, age, gender, and topics of interest.

[0008] "Means for receiving" refers to the system functions or modules for receiving information entered by the user.

[0009] "Customized Story" refers to an individual narrative generated by a generative AI model based on information entered by a user.

[0010] "Generating means" refers to the system features or modules used to create stories and illustrations based on received information.

[0011] "Illustrations" refer to pictures or images that are drawn to accompany the generated story.

[0012] "Digital Preview" refers to a preliminary version of a picture book in digital format that combines story and illustrations.

[0013] "Correction Request" means a request made by a User to request changes or additions after reviewing a Digital Preview.

[0014] "Final digital picture book" refers to the final, completed digital picture book that reflects the user's confirmation and correction requests.

[0015] "Means for providing" refers to the system functions and modules for providing the generated picture book to users in a downloadable format.

[0016] "Printing and Delivery Option" refers to a service that physically prints the generated picture book and delivers it to an address specified by the user.

[0017] "Generative AI model" refers to an artificial intelligence model used to create customized stories based on user input.

[0018] "Image generation engine" refers to an image processing system for creating illustrations based on the generated story. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] Specific embodiments for carrying out the present invention will be described below.

[0041] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0042] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[0043] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version, enter correction requests if necessary, and resubmit it.

[0044] The server receives revision requests from the user and recreates the story and illustrations based on those requests. Finally, once the user is satisfied, the final digital book is generated. This final book is then converted to the appropriate format (PDF, EPUB, etc.) in high resolution.

[0045] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0046] Below is an example of this system:

[0047] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[0048] 2. The terminal sends the entered data to the server.

[0049] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[0050] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[0051] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[0052] 6. The user checks the preview and submits a revision request to add more dinosaur scenes.

[0053] 7. The server processes the requested modifications and provides the regenerated preview to the user.

[0054] 8. The user is happy with the final version and selects download, print and delivery options.

[0055] 9. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[0056] 10. The printing service will deliver the printed picture book to the user's address.

[0057] Through this process, users can create a customized picture book that matches their child's interests and preferences, creating a fun reading experience.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[0061] Step 2:

[0062] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[0063] Step 3:

[0064] The server receives the HTTP request and analyzes the input data. For example, it analyzes information such as "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[0065] Step 4:

[0066] The server passes the input data to a generative AI model that generates a customized story. Based on the user's input, the generative AI model creates a story such as "Taro meets a dinosaur and goes on a great adventure."

[0067] Step 5:

[0068] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page, such as the scene where Taro meets a dinosaur.

[0069] Step 6:

[0070] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0071] Step 7:

[0072] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[0073] Step 8:

[0074] The user checks the preview and, if necessary, enters a request for corrections, such as "I want more dinosaur scenes," and resubmits the video.

[0075] Step 9:

[0076] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[0077] Step 10:

[0078] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[0079] Step 11:

[0080] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[0081] Step 12:

[0082] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[0083] Step 13:

[0084] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[0085] Step 14:

[0086] The printing service company will deliver the printed picture book to the address specified by the user.

[0087] Example 1

[0088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0089] Existing picture book generation systems have difficulty efficiently generating customized picture books based on user input. Furthermore, the quality of story and illustration generation is not consistent, and modifications to meet user requests are not easily made. This makes it necessary to solve the problem of being unable to quickly and accurately provide customized picture books that satisfy users.

[0090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0091] In this invention, the server includes means for receiving information input by a user, means for generating prompt sentences for a generative AI model based on the received information, means for generating a customized story using the generated prompt sentences, means for generating illustrations using an image generation engine based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send correction requests as needed, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to efficiently generate high-quality customized picture books based on user input and quickly accommodate corrections requested by the user.

[0092] "User" refers to a person who accesses the system and inputs information.

[0093] "Information to be entered" refers to data such as the child's name, age, gender, and interests.

[0094] "Means of receiving" refers to the mechanism by which the server obtains the information entered by the user.

[0095] "Generative AI model" refers to an artificial intelligence model that generates customized stories based on user input.

[0096] A "prompt" refers to an instruction used to generate a story for a generative AI model.

[0097] "Customized Story" refers to a bespoke narrative generated based on user input.

[0098] "Image generation engine" refers to a system for automatically generating illustrations based on a customized story.

[0099] "Illustration" refers to an image that corresponds to a customized story.

[0100] "Digital Preview" refers to a digital version of a picture book that combines a generated story and illustrations for advance viewing.

[0101] "Correction Request" refers to a request for changes or additions made by a user after reviewing a digital preview.

[0102] A "final digital picture book" refers to a completed digital picture book that has been approved by the user.

[0103] "Print and Delivery Options" means the option to have the final digital book printed as a physical book and delivered to a specified address.

[0104] The "system" refers to the overall mechanism that combines these means to provide users with customized picture books.

[0105] The present invention relates to a system for generating a customized picture book based on information input by a user. Specific embodiments for carrying out the present invention will be described below.

[0106] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0107] The server analyzes the user's input data received from the device and generates a customized story based on it. Specifically, it uses a generative AI model (e.g., GPT-4 (registered trademark)) to convert the user's input data into a prompt. For example, this prompt might be, "Please create a story about a 6-year-old boy named 'Taro' who goes on a great adventure with dinosaurs."

[0108] A customized story is created based on the prompts generated by the generative AI model. The server then uses an image generation engine (e.g., DALL-E) to generate illustrations appropriate for each page of the story. For example, the illustration for "Taro's first encounter with a dinosaur" is generated by this engine.

[0109] The server combines the generated story and illustrations to create a preview version of the digital picture book. This preview version is created in an interactive format (e.g., HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version and, if necessary, enter correction requests and resubmit it.

[0110] The server receives revision requests from the user and regenerates the story and illustrations based on those requests. For example, if a user requests "more dinosaur scenes," the server regenerates new stories and illustrations to reflect that request. Finally, once the user is satisfied, the final digital picture book is generated. The final picture book is then converted to a suitable format (e.g., PDF, EPUB, etc.) at high resolution.

[0111] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0112] As a result, the present invention can efficiently generate a high-quality customized picture book based on user input, and can also quickly accommodate modifications in response to user requests.

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1:

[0115] The user opens a web browser, accesses the system's URL, enters the child's name, age, gender, and interest (e.g., dinosaurs, adventure) in the form that appears, and clicks the submit button.

[0116] Input: Child's name, age, gender, and interests.

[0117] Output: The input data in JSON format.

[0118] Step 2:

[0119] The terminal sends the input data to the server. Specifically, it converts the input data into JSON format using JavaScript (registered trademark) and sends it to the server using an HTTP POST request.

[0120] Input: Input data in JSON format.

[0121] Output: The input data sent to the server.

[0122] Step 3:

[0123] The server analyzes the received input data and generates prompts for the generative AI model. This processing is done using the Python Flask framework.

[0124] Input: The input data sent to the server in JSON format.

[0125] Output: A prompt to the generative AI model.

[0126] Step 4:

[0127] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the prompt. Here, data generation calculations are performed based on the prompt. The generated story is stored in a database (e.g., PostgreSQL).

[0128] Input: A prompt to the generative AI model.

[0129] Output: A customized story.

[0130] Step 5:

[0131] The server analyzes the generated story and generates illustrations appropriate for each scene using an image generation engine (e.g., DALL-E). The generated illustrations are temporarily stored in a database.

[0132] Input: Your customized story.

[0133] Output: The generated illustration.

[0134] Step 6:

[0135] The server combines the generated story and illustrations to create a digital preview in HTML or PDF format, which is stored on the server.

[0136] Input: Generated story and illustrations.

[0137] Output: Digital preview (HTML, PDF formats).

[0138] Step 7:

[0139] The user uses a web browser to check the digital preview, enter correction requests if necessary, and submit it again.

[0140] Input: Digital preview and revision request.

[0141] Output: The correction request.

[0142] Step 8:

[0143] The server receives the modification request and regenerates the story and illustrations based on the request, which are also stored in the database.

[0144] Input: Correction request.

[0145] Output: Regenerated story and illustrations.

[0146] Step 9:

[0147] The server generates the final digital picture book by converting it into a suitable high-resolution format (e.g. PDF, EPUB, etc.) and stores the generated final version on the server.

[0148] Input: The final story and illustrations, as reviewed by the user.

[0149] Output: Final digital picture book (PDF, EPUB format).

[0150] Step 10:

[0151] The user downloads the final digital book and selects printing and delivery options if desired. If selected, the server receives the user's delivery information.

[0152] Input: User downloads and prints; selects shipping options; shipping information.

[0153] Output: Download link and shipping information.

[0154] Step 11:

[0155] The server sends a delivery request to a partner printing service company.

[0156] Input: Shipping information.

[0157] Output: A delivery request to a printing service company.

[0158] Step 12:

[0159] The printing service will deliver the printed picture book to the address specified by the user. The delivery status will also be updated and notified to the server.

[0160] Input: Delivery request.

[0161] Output: Delivery completed to user.

[0162] Through this process, users can create a customized storybook tailored to their child's interests and preferences, which can then be downloaded, printed, and delivered.

[0163] (Application example 1)

[0164] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0165] Conventional picture book creation services have had problems such as difficulty in customizing to individual users and lack of flexibility in previewing and modifying the generated picture books. In particular, it has been felt to be difficult to easily create digital picture books customized according to children's interests and preferences, and to easily view and share them on smart devices. The objective of this invention is to solve these problems and provide a system that enables users to easily create, modify, view, and share customized picture books.

[0166] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0167] In this invention, the server includes means for receiving information entered by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send a request for corrections as needed, means for generating a final digital picture book and providing it to the user, means for providing printing and delivery options if desired by the user, means for making the generated digital picture book viewable on a smart device, means for making the digital picture book downloadable in high resolution, and means for enabling online sharing. This allows users to easily create customized picture books, preview them on the spot, make any necessary corrections, and then view or share them on their smart devices.

[0168] A "user" is a person who accesses the system, inputs information, and uses the generated digital picture book.

[0169] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides to the system.

[0170] A "customized story" is a narrative tailored to the individual user's needs, generated based on information entered by the user.

[0171] "Illustrations" are pictures or images generated based on a customized story.

[0172] "Digital Preview" is a digital picture book that combines story and illustrations and allows users to view it interactively.

[0173] A "request for correction" is a request made by a user to request corrections to content after checking the digital preview.

[0174] The "final digital picture book" is the final, high-resolution digital picture book that incorporates user correction requests.

[0175] "Print and Delivery Option" means an option that allows a user to physically print the final digital book and have it delivered to an address of their choosing.

[0176] "Smart devices" are electronic devices that can connect to the Internet, such as smartphones, smart glasses, and head-mounted displays.

[0177] "High resolution" refers to the quality of visual content, such as images and documents, that can be displayed with clear detail.

[0178] "Sharing online" refers to the ability to share the generated digital picture book with other users via the Internet.

[0179] Specific embodiments for carrying out the present invention will be described below.

[0180] Users access the system using a smartphone, personal computer, etc. The terminal displays an input form and allows the user to enter information such as the child's name, age, gender, and interests. Once the user has completed the input and clicked the submit button, the terminal sends this data to the server.

[0181] The server analyzes the received information and generates a customized story using a generative AI model (e.g., GPT-4). For example, if a user enters information such as "6 years old, boy, named Taro, loves dinosaurs," the server generates "a story about Taro's adventures with dinosaurs." The prompt used in this case is something like: "6-year-old boy, named Taro, loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[0182] Based on the generated story, the server uses an image generation engine (e.g., DALL-E) to generate illustrations. The image generation engine generates illustrations appropriate for each scene in the story, and combines the story and illustrations to create a digital preview.

[0183] Users can review this digital preview and submit revision requests as needed, which are then sent to the server, which then uses the generative AI model and image generation engine to update the preview and serve it back to the user, allowing them to iteratively customize the final version until they are satisfied with it.

[0184] Once the final digital book is complete, the server generates a high-resolution digital book and provides the user with a download link. If the user prefers, the server can also provide a printing and delivery option, allowing an external printing service to deliver the book to the user's specified address.

[0185] Furthermore, the generated digital picture books can be viewed on smart devices (e.g., smartphones, smart glasses, head-mounted displays) and can also be shared with other users online. This allows users to easily share their individually customized picture books with others and provides a fun reading experience for children.

[0186] To illustrate, here's an example prompt:

[0187] "I have a 6-year-old boy named Taro who loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[0188] The system effectively utilizes generative AI models and image generation engines to provide users with flexible customization and high-quality digital content.

[0189] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0190] Step 1:

[0191] A user accesses the system using a terminal and enters information such as the child's name, age, gender, and interests into the input form. After completing the input and clicking the submit button, the terminal sends this data to the server. Input is the various pieces of information provided by the user, and output is the data sent to the server.

[0192] Step 2:

[0193] The server analyzes the information received from the user. Here, it generates a prompt sentence appropriate for the generative AI model based on input data, particularly the child's name, age, gender, and topics of interest. The input is the data provided by the user, and the output is the generated prompt sentence. Specifically, it uses the data entered by the user to generate a prompt sentence such as, "A 6-year-old boy named Taro who loves dinosaurs. Please write a story about Taro's adventures with dinosaurs."

[0194] Step 3:

[0195] The server sends the generated prompt to a generative AI model (e.g., GPT-4) to generate a customized story. The generative AI model generates an optimal story based on the prompt and returns it to the server. The input is the prompt, and the output is the generated story. Specifically, the AI ​​model analyzes the prompt and generates a story such as "A story about Taro meeting a dinosaur and going on a great adventure."

[0196] Step 4:

[0197] The server receives the generated story and uses an image generation engine (e.g., DALL-E) to generate illustrations for each scene of the story as needed. The input is the generated story, and the output is illustrations corresponding to each scene. Specifically, it generates illustrations appropriate for each scene, such as "the scene where Taro meets a dinosaur for the first time."

[0198] Step 5:

[0199] The server combines the generated story and illustrations to create a digital preview. The input is a story-illustration pair, and the output is a digital preview. Specifically, the server combines the story and illustration in HTML or PDF format to create a preview version that users can check interactively.

[0200] Step 6:

[0201] The user checks the digital preview and sends a correction request if necessary. The server receives this correction request and updates the preview content again using the generative AI model and image generation engine. The input is the correction request from the user, and the output is the corrected digital preview. Specifically, based on the user request to "add more dinosaur scenes," the story and illustrations are regenerated.

[0202] Step 7:

[0203] Once the user is satisfied with the final version of the digital book, the server generates the final version and provides it to the user. The input is the revised preview, and the output is the final digital book. Specifically, it converts it to PDF or EPUB format at high resolution and provides a download link to the user.

[0204] Step 8:

[0205] The server provides printing and delivery options to the user if desired. The input is the user's delivery information, and the output is a request to the printing service company. Specifically, the server sends the necessary information to an external printing service company to deliver the picture book to the address specified by the user.

[0206] To ensure a smooth process, the document can also be viewed on smart devices and shared online.

[0207] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0208] Specific embodiments for carrying out the present invention will be described below.

[0209] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0210] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[0211] The server combines the generated story and illustrations to create a preview version of the digital picture book, which is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0212] The server sends an HTTP response containing the URL of the preview version to the device, and the user clicks on the link to view the preview. At this stage, the system is equipped with an emotion engine that recognizes emotions by analyzing the user's facial expressions and voice. The emotion engine analyzes the user's emotions and determines their feelings, such as "They seem to be having fun" or "They seem not to be interested."

[0213] The user checks the preview and, if any particular corrections are needed, enters the correction request in the input field. At this time, the server generates automatic correction suggestions for the story or illustrations, taking into account the user's emotions recognized by the emotion engine. For example, if the user expresses an expression that says "it doesn't look like he's having fun," the system can suggest corrections such as "add more dinosaur scenes." The user can review the correction suggestions and implement them as necessary.

[0214] The server receives the revision request and the revision suggestions from the emotion engine, then re-runs the generative AI model and image generation engine to generate a picture book that reflects the revisions. The regenerated preview version is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[0215] Finally, once the user is satisfied with the final version, the server generates the final high-resolution digital book and converts it into the appropriate format (PDF, EPUB, etc.). It then sends the URL of the final version to the user's device, allowing them to download it using the download link. At the same time, if the user wishes, it provides a print and delivery option, and the server sends a print and delivery request to the printing service provider.

[0216] Below is an example of this system:

[0217] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[0218] 2. The terminal sends the entered data to the server.

[0219] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[0220] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[0221] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[0222] 6. The user checks the preview. The emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0223] 7. The user reviews the sentiment-based revision suggestions and submits a revision request to include more dinosaur scenes.

[0224] 8. The server reflects the modification request and the emotion engine's suggestions and provides the regenerated preview to the user.

[0225] 9. The user is happy with the final version and selects download, print and delivery options.

[0226] 10. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[0227] 11. The printing service will deliver the printed picture book to the user's address.

[0228] Through this process, users can create a customized picture book that matches their child's interests, and by using an emotion engine, they can create a more personalized and enjoyable reading experience.

[0229] The processing flow will be explained below.

[0230] Step 1:

[0231] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[0232] Step 2:

[0233] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[0234] Step 3:

[0235] The server receives the HTTP request and analyzes the input data. The analysis results include the following information: "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[0236] Step 4:

[0237] The server passes the input data to the generative AI model, which then generates a customized story. Based on the user's input, the generative AI model creates a story about Taro meeting a dinosaur and going on a great adventure.

[0238] Step 5:

[0239] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page. For example, it generates illustrations for scenes such as "Taro meets a dinosaur" and "Taro goes on an adventure with a dinosaur."

[0240] Step 6:

[0241] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0242] Step 7:

[0243] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[0244] Step 8:

[0245] The user checks the preview. The device uses the camera and microphone to capture the user's facial expressions and voice, which are then analyzed by the emotion engine. For example, if the user is smiling, it will be determined that they look happy.

[0246] Step 9:

[0247] The device sends the analysis results from the emotion engine to the server, which receives the results and generates automatic suggestions for revising the story or illustrations based on emotional information such as "it looks fun."

[0248] Step 10:

[0249] The user checks the proposed revisions and makes any necessary changes. For example, the user enters a request such as "I want more dinosaur scenes" and clicks the submit button.

[0250] Step 11:

[0251] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[0252] Step 12:

[0253] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[0254] Step 13:

[0255] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[0256] Step 14:

[0257] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[0258] Step 15:

[0259] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[0260] Step 16:

[0261] The printing service company will deliver the printed picture book to the address specified by the user.

[0262] Example 2

[0263] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] While conventional customized picture book creation systems can generate stories based on information entered by users, they lack the ability to modify the story and illustrations taking into account the user's emotional responses. As a result, if a user is not satisfied with the content of the picture book, they must provide detailed correction instructions again, which is time-consuming. The present invention aims to solve these problems and efficiently generate more personalized customized picture books.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0266] In this invention, the server includes means for receiving information input by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the server to provide the user with a link to the generated digital preview, means for the user to check the digital preview and send a correction request if necessary, means for analyzing the user's emotions using an emotion engine and generating correction suggestions based on the analysis results, means for regenerating the story and illustrations by reflecting the correction requests and the emotion analysis results, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to incorporate feedback based on the user's emotions and generate a customized picture book that provides a higher level of satisfaction.

[0267] "User" means an individual or end user who uses the system to create a customized digital picture book.

[0268] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides when using the system.

[0269] "Server" means a central processing unit or server system that receives input from users and generates, modifies, and distributes stories and illustrations based thereon.

[0270] A "customized story" is a narrative that is individually generated based on input information provided by the user.

[0271] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to automatically generate stories from user input.

[0272] An "illustration" is an image created to visually represent a scene from the generated story.

[0273] An "image generation engine" is an image generation algorithm or software for automatically generating illustrations based on a generated story.

[0274] The "Digital Preview" is an early version of the digital picture book that combines the generated story and illustrations, and is in a format that allows users to check and edit it.

[0275] An "emotion engine" is a technology or software that analyzes data such as a user's facial expressions and voice and recognizes their emotional state.

[0276] A "revision request" is a request for changes to the story or illustrations submitted by a user after viewing the digital preview.

[0277] "Revision suggestions" are suggestions for changes to stories or illustrations generated by the server based on the results of an analysis of the user's emotions by the emotion engine.

[0278] The "final digital picture book" is a high-quality digital picture book that is finally generated by reflecting the user's correction requests and suggestions from the emotion engine.

[0279] "Printing and Delivery Option" means an additional service that allows a user to print and deliver the final digital picture book as a physical book if the user so desires.

[0280] The present invention is a system for users to generate customized digital picture books through a process that includes collecting user input data, generating a story using a generative AI model, generating illustrations using an image generation engine, and analyzing user emotions using an emotion engine and making modifications based on the analysis.

[0281] Hardware and software used

[0282] This system uses the following hardware and software:

[0283] Terminal: A device (PC, smartphone, tablet, etc.) that a user uses to access the system using a web browser and enter and confirm information.

[0284] Server: The central processing system that receives user input and generates, modifies, and delivers stories and illustrations.

[0285] Generative AI models: Models that automatically generate stories using natural language processing techniques (e.g., OpenAI® GPT-4).

[0286] Image generation engine: An engine that generates illustrations based on the generated story (e.g., DALL-E).

[0287] Emotion engine: Technology that analyzes the user's facial expressions and voice to recognize emotions (e.g., Affectiva).

[0288] Explanation of program processing

[0289] Collecting user-entered data

[0290] When a user accesses the system through a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interests, and submits it. For example, enter the following:

[0291] Name: Taro

[0292] Age: 6

[0293] Gender: Male

[0294] Interests: Dinosaurs, adventure

[0295] Sending data to the server

[0296] When the user clicks the submit button, the device sends the input data to the server, which receives and analyzes the data.

[0297] Story Generation

[0298] The server uses a generative AI model to generate a customized story based on the received data, for example using the following prompt:

[0299] "Create a story about the great adventure of a 6-year-old boy named Taro, who begins with an encounter with a dinosaur."

[0300] The generative AI model generates a basic story, for example, "Once upon a time, a six-year-old boy named Taro encountered a dinosaur..."

[0301] Illustration generation

[0302] Based on the generated story, the server uses an image generation engine to generate illustrations appropriate for each scene, for example using the following prompt:

[0303] "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time."

[0304] This prompt will cause the image generation engine to generate an illustration.

[0305] Generate and deliver digital previews

[0306] The server combines the generated story and illustrations to create a preview version of the digital picture book (in HTML or PDF format), and then generates a URL for the preview version and provides it to the user.

[0307] Analyzing user sentiment and reflecting correction requests

[0308] When a user checks the preview version, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state. For example, it can distinguish emotions such as "interested" or "uninterested." When the user submits an emotion-based revision request, the server generates and implements revision suggestions based on the request and emotion analysis.

[0309] The server re-serves the regenerated story and illustrations as a digital picture book, and the process is repeated until the user is satisfied.

[0310] Final digital book delivery and printing / shipping options

[0311] Once the user is satisfied with the final version, the server generates the final high-resolution digital book, converts it to PDF or EPUB format, provides the user with a URL, and offers printing and delivery options if desired, and sends the request to a printing service, which will then deliver it to the address specified by the user.

[0312] The system allows users to quickly and efficiently generate customized picture books tailored to a child's interests, and by using an emotion engine, provides a more personalized and satisfying reading experience.

[0313] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0314] Step 1:

[0315] The user accesses the system via a web browser and the terminal displays an input form.

[0316] Input: The URL the user visited

[0317] Output: The input form is displayed on the terminal.

[0318] What happens: The user opens a browser and visits a URL (e.g., https: / / example.com). The device displays a user interface and provides a form for entering the child's information.

[0319] Step 2:

[0320] The user enters the child's name, age, gender, and interests and clicks the submit button.

[0321] Input: Name, age, gender, interests

[0322] Output: User input data is sent to the server

[0323] Specific operation: The user enters information such as "Taro," "6 years old," "male," "dinosaur, adventure" into the form and clicks the submit button, which causes the device to send the input data to the server.

[0324] Step 3:

[0325] The server parses the received input data.

[0326] Input: User input data (Taro, 6 years old, male, dinosaur, adventure)

[0327] Output: Parsed data

[0328] Specific operation: The server receives an HTTP POST request and analyzes the user's input data. The analyzed data is used for further processing.

[0329] Step 4:

[0330] The server uses generative AI models to generate customized stories.

[0331] Input: Parsed user data

[0332] Output: Customized story

[0333] What happens: The server sends the following prompt to a generative AI model (e.g., GPT-4): "Write a story about the great adventure of a 6-year-old boy named Taro, who begins with his encounter with a dinosaur." The generative AI model responds with a story: "Once upon a time, a 6-year-old boy named Taro met a dinosaur..."

[0334] Step 5:

[0335] The server uses an image generation engine to generate illustrations based on the generated story.

[0336] Input: Your customized story

[0337] Output: Illustration

[0338] Specific behavior: The server sends the following prompt to the image generation engine (e.g., DALL-E): "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time." The image generation engine generates the illustration and returns it to the server.

[0339] Step 6:

[0340] The server combines the generated story and illustrations to create a preview version of the digital picture book and generates a URL for it.

[0341] Input: Story, illustrations

[0342] Output: URL of the preview version of the digital picture book

[0343] What it does: The server combines the story and illustrations in HTML or PDF format to create a preview version of the digital picture book, and provides the generated URL to the user.

[0344] Step 7:

[0345] The server sends an HTTP response containing the preview URL to the device.

[0346] Input: Preview URL

[0347] Output: HTTP response

[0348] Specific operation: The server sends an HTTP response to the device and notifies the user of the preview URL (e.g., https: / / example.com / preview / taro_story). The user clicks the URL to view the preview.

[0349] Step 8:

[0350] The user checks the preview version, and the emotion engine analyzes the user's facial expressions and voice.

[0351] Input: Preview, facial expressions, and voice data

[0352] Output: Emotion analysis results

[0353] How it works: The user clicks on the preview URL to check the content. The emotion engine uses the camera and microphone to analyze the user's facial expressions and voice, and determines their emotions, such as "They seem to be having fun" or "They seem to be uninterested."

[0354] Step 9:

[0355] The user submits a correction request, and the server makes correction suggestions based on the results of sentiment analysis.

[0356] Input: Correction request, sentiment analysis results

[0357] Output: suggested fixes

[0358] How it works: A user inputs and submits a revision request, such as "I want more dinosaur scenes." The server, taking into account the results of sentiment analysis, generates a revision suggestion, such as "Add more adventurous scenes."

[0359] Step 10:

[0360] The server regenerates the story and illustrations based on the revision requests and suggestions.

[0361] Input: Correction request, correction proposal

[0362] Output: Regenerated story and illustrations

[0363] What it does: The server re-runs the generative AI model and image generation engine to generate new stories and illustrations that reflect the modifications.

[0364] Step 11:

[0365] The server sends an HTTP response containing the URL of the regenerated preview version to the device and provides it to the user.

[0366] Input: Regenerated story and illustrations

[0367] Output: Regenerated preview URL

[0368] Specific behavior: The server provides the newly generated preview URL (e.g., https: / / example.com / preview / taro_story_v2) to the user. The user checks the preview again.

[0369] Step 12:

[0370] Once the user is satisfied with the final version, the server generates the final digital picture book and provides it in PDF or EPUB format.

[0371] Input: User's final confirmation

[0372] Output: Final digital picture book (PDF or EPUB format)

[0373] What happens: Once the user is satisfied with the final version, the server generates a high-resolution digital storybook (e.g., PDF format) and provides a URL (e.g., https: / / example.com / final / taro_story_final.pdf).

[0374] Step 13:

[0375] If the user so desires, the server sends a print and delivery request to the printing service company.

[0376] Input: User's print and delivery request

[0377] Output: Request to printing service company

[0378] Specific operation: When the user selects the print and delivery option, the server sends the data and delivery information to the printing service company. The printed picture book is then delivered to the specified address.

[0379] (Application example 2)

[0380] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Previous systems were unable to consider user emotions when generating customized digital picture books, which could result in lower user satisfaction. Furthermore, because the books were only available online, there was no way to improve user engagement through in-store experiences. This meant that users lacked the flexibility to experience the book on the spot and make suggestions for revisions immediately.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0383] In this invention, the server includes a means for receiving information entered by the user, a means for generating a customized story based on the received information, a means for collecting and analyzing the user's emotional data using an emotion recognition engine, and a means for generating suggested revisions to the story and illustrations. This makes it possible to generate a customized picture book that takes the user's emotions into consideration, and by providing this service in a physical store, it becomes possible to provide an interactive experience on the spot.

[0384] "Means for receiving information entered by the user" refers to devices or software that collect information entered by the user, such as name, age, gender, and interests, and transmits it to the server.

[0385] "Means for generating a customized story based on received information" refers to a generative AI model or algorithm that creates an individualized customized story based on information received from a user.

[0386] "Means for generating illustrations based on the generated story" refers to an image generation engine or tool for creating appropriate illustrations according to the content of the generated story.

[0387] "Means for combining story and illustrations to generate a digital preview of a picture book" refers to software or a system that combines a customized story with generated illustrations and provides them in a user-viewable digital preview format (e.g., HTML, PDF).

[0388] "Means for users to check the digital preview and send correction requests if necessary" refers to an interface that allows users to check the preview of the generated digital picture book and send correction requests to the server if corrections are necessary.

[0389] "Means for collecting and analyzing user emotional data using an emotion recognition engine" refers to devices and software for automatically collecting and analyzing emotional data from users' facial expressions, voice, etc.

[0390] "Means for generating suggested revisions to stories and illustrations based on emotional data" refers to a system or algorithm that automatically generates more appropriate suggested revisions to stories and illustrations based on collected user emotional data.

[0391] "Means for regenerating the story and illustrations and providing a digital preview again" refers to a system for reflecting the correction request and emotional data and providing the newly generated story and illustrations again in a digital preview format.

[0392] "Means for generating and delivering the final digital book to the user" refers to a system that generates the final customized book in high resolution and delivers it to the user in an appropriate format (e.g., PDF, EPUB).

[0393] "Means for providing printing and delivery options upon user's request" refers to systems or services that provide users with the option to print and deliver physical picture books upon user's request.

[0394] "Means for providing services to users using tablet devices or dedicated kiosk terminals in physical stores" refers to devices and systems that allow users to directly experience creating customized picture books using tablet devices or dedicated kiosk terminals in physical stores.

[0395] Specific embodiments of the present invention will be described below.

[0396] Users access the "Detailed Description of the Invention" using a tablet terminal or dedicated kiosk terminal installed in a physical store. The terminal displays an input form to the user via a web browser. The user enters the child's name, age, gender, and interests and clicks the submit button. This sends the input data from the terminal to the server.

[0397] The server is a high-performance cloud server (e.g., AWS (registered trademark) EC2, Google (registered trademark) Cloud Platform) that performs the following processes: First, a generative AI model (e.g., GPT-4, BERT) is used to generate a customized story based on the user's input data. Next, an image generation engine (e.g., DALL-E, Stable Diffusion) is used to generate illustrations appropriate for each scene in the generated story. The generated story and illustrations are combined to create a preview of the digital picture book. The preview version is created in an interactive format (e.g., HTML, PDF), and a URL is generated. The server sends this URL to the device, and the user clicks the link to view the preview version.

[0398] Using an emotion recognition engine (e.g., Microsoft® Azure® Emotion API), the system analyzes the facial expressions and voice of the user while previewing the digital picture book to recognize their emotions. This emotion data is sent to the server. The server then automatically generates revision suggestions based on the collected emotion data. For example, if the system recognizes an expression indicating "not looking happy," it will suggest "add more dinosaur scenes." The user can review the revision suggestions and, if necessary, enter their revision requests in the input field.

[0399] The server receives the revision request and the emotion-based revision suggestions, re-runs the generative AI model and image generation engine to regenerate the picture book reflecting the revisions. A preview of the regenerated digital picture book is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[0400] Finally, once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g., PDF, EPUB). It then sends a URL for the final version to the user's device, which the user can use to download the digital book. At the same time, it provides a print and delivery option if the user wishes, and the server sends a print and delivery request to the printing service provider.

[0401] For example, if a user inputs "Taro," "6 years old," "male," "dinosaur, adventure" into a tablet device, the server generates a story about Taro meeting a dinosaur and going on a great adventure, and an image generation engine creates illustrations based on the story. A preview of the digital picture book is then generated for the user to review. If the emotion recognition engine determines that the book "doesn't look fun," the system suggests adding more dinosaur scenes. This process is repeated until the user is satisfied, and the final version is generated.

[0402] An example of a prompt for a generative AI model is:

[0403] Child information provided by the user:

[0404] Name: Taro

[0405] Age: 6

[0406] Gender: Male

[0407] Interests: Dinosaurs, adventure

[0408] prompt:

[0409] Create a story about a 6-year-old boy named Taro who meets a dinosaur and goes on a great adventure. Include scene descriptions for each scene to illustrate.

[0410] This allows users to enjoy the experience of creating a customized picture book in real time in a physical store.

[0411] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0412] Step 1:

[0413] The user accesses the input form through a web browser using a tablet device or dedicated kiosk device in a physical store. The user enters the child's name, age, gender, and interests, and clicks the submit button. The input, such as name (e.g., "Taro"), age (e.g., "6 years old"), gender (e.g., "male"), and interests (e.g., "dinosaurs," "adventure"), is sent to the server. This allows the user's information to be collected.

[0414] Step 2:

[0415] The server receives the data sent by the device. The server stores the received user input data. User information is passed to the server as input, and customized data is prepared based on this information.

[0416] Step 3:

[0417] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the input data. The server passes a prompt to the generative AI model, which then generates a story based on it. Specifically, the name, age, gender, and interests entered by the user are input as prompts into the generative AI model, and a customized story is returned.

[0418] Step 4:

[0419] The server uses an image generation engine (e.g., DALL-E) to create illustrations appropriate for each scene in the generated story. The server receives the story scenes as input and generates illustrations corresponding to each scene as output. These illustrations are then combined with the story.

[0420] Step 5:

[0421] The server combines the generated story and illustrations to create a preview of the digital picture book. The preview is generated in an interactive format (e.g., HTML, PDF), and a URL is created. The server then sends this URL to the device, where the preview is provided to the user.

[0422] Step 6:

[0423] The user checks a preview of the digital picture book on their device. An emotion recognition engine (e.g., Microsoft Azure Emotion API) analyzes the user's facial expressions and voice to recognize their emotions. The input is collected by the camera and microphone, and emotional data is generated as output.

[0424] Step 7:

[0425] The server automatically generates revision suggestions based on the emotion data. For example, if the user's facial expression indicates that they are not having fun, the server will suggest revisions such as adding more dinosaur scenes. The revision suggestions are provided to the user, who then confirms them.

[0426] Step 8:

[0427] The user checks the suggested revisions and, if necessary, enters a revision request into the input field on the device. The server receives this revision request and again runs the generative AI model and image generation engine to revise the story and illustration. The revision request is passed as input, and the revised story and illustration are generated as output.

[0428] Step 9:

[0429] The server then provides the user with a regenerated preview of the digital book that reflects the modifications. The user then checks the preview again and repeats this process until satisfied. This results in customization that increases user satisfaction.

[0430] Step 10:

[0431] Once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g. PDF, EPUB). A URL for the final version is sent to the device, where the user can use this link to download the digital book. If desired, the user selects the print and delivery option, and the server sends a print and delivery request to the printing service provider.

[0432] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0433] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0434] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0435] [Second embodiment]

[0436] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0437] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0438] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0439] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0440] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0441] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0442] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0443] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0444] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0445] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0446] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0447] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0448] Specific embodiments for carrying out the present invention will be described below.

[0449] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0450] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[0451] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version, enter correction requests if necessary, and resubmit it.

[0452] The server receives revision requests from the user and recreates the story and illustrations based on those requests. Finally, once the user is satisfied, the final digital book is generated. This final book is then converted to the appropriate format (PDF, EPUB, etc.) in high resolution.

[0453] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0454] Below is an example of this system:

[0455] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[0456] 2. The terminal sends the entered data to the server.

[0457] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[0458] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[0459] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[0460] 6. The user checks the preview and submits a revision request to add more dinosaur scenes.

[0461] 7. The server processes the requested modifications and provides the regenerated preview to the user.

[0462] 8. The user is happy with the final version and selects download, print and delivery options.

[0463] 9. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[0464] 10. The printing service will deliver the printed picture book to the user's address.

[0465] Through this process, users can create a customized picture book that matches their child's interests and preferences, creating a fun reading experience.

[0466] The processing flow will be explained below.

[0467] Step 1:

[0468] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[0469] Step 2:

[0470] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[0471] Step 3:

[0472] The server receives the HTTP request and analyzes the input data. For example, it analyzes information such as "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[0473] Step 4:

[0474] The server passes the input data to a generative AI model that generates a customized story. Based on the user's input, the generative AI model creates a story such as "Taro meets a dinosaur and goes on a great adventure."

[0475] Step 5:

[0476] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page, such as the scene where Taro meets a dinosaur.

[0477] Step 6:

[0478] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0479] Step 7:

[0480] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[0481] Step 8:

[0482] The user checks the preview and, if necessary, enters a request for corrections, such as "I want more dinosaur scenes," and resubmits the video.

[0483] Step 9:

[0484] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[0485] Step 10:

[0486] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[0487] Step 11:

[0488] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[0489] Step 12:

[0490] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[0491] Step 13:

[0492] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[0493] Step 14:

[0494] The printing service company will deliver the printed picture book to the address specified by the user.

[0495] Example 1

[0496] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0497] Existing picture book generation systems have difficulty efficiently generating customized picture books based on user input. Furthermore, the quality of story and illustration generation is not consistent, and modifications to meet user requests are not easily made. This makes it necessary to solve the problem of being unable to quickly and accurately provide customized picture books that satisfy users.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0499] In this invention, the server includes means for receiving information input by a user, means for generating prompt sentences for a generative AI model based on the received information, means for generating a customized story using the generated prompt sentences, means for generating illustrations using an image generation engine based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send correction requests as needed, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to efficiently generate high-quality customized picture books based on user input and quickly accommodate corrections requested by the user.

[0500] "User" refers to a person who accesses the system and inputs information.

[0501] "Information to be entered" refers to data such as the child's name, age, gender, and interests.

[0502] "Means of receiving" refers to the mechanism by which the server obtains the information entered by the user.

[0503] "Generative AI model" refers to an artificial intelligence model that generates customized stories based on user input.

[0504] A "prompt" refers to an instruction used to generate a story for a generative AI model.

[0505] "Customized Story" refers to a bespoke narrative generated based on user input.

[0506] "Image generation engine" refers to a system for automatically generating illustrations based on a customized story.

[0507] "Illustration" refers to an image that corresponds to a customized story.

[0508] "Digital Preview" refers to a digital version of a picture book that combines a generated story and illustrations for advance viewing.

[0509] "Correction Request" refers to a request for changes or additions made by a user after reviewing a digital preview.

[0510] A "final digital picture book" refers to a completed digital picture book that has been approved by the user.

[0511] "Print and Delivery Options" means the option to have the final digital book printed as a physical book and delivered to a specified address.

[0512] The "system" refers to the overall mechanism that combines these means to provide users with customized picture books.

[0513] The present invention relates to a system for generating a customized picture book based on information input by a user. Specific embodiments for carrying out the present invention will be described below.

[0514] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0515] The server analyzes the user's input data received from the device and generates a customized story based on it. Specifically, it uses a generative AI model (e.g., GPT-4) to convert the user's input data into a prompt. For example, this prompt might say, "Please create a story about a 6-year-old boy named Taro who goes on a great adventure with dinosaurs."

[0516] A customized story is created based on the prompts generated by the generative AI model. The server then uses an image generation engine (e.g., DALL-E) to generate illustrations appropriate for each page of the story. For example, the illustration for "Taro's first encounter with a dinosaur" is generated by this engine.

[0517] The server combines the generated story and illustrations to create a preview version of the digital picture book. This preview version is created in an interactive format (e.g., HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version and, if necessary, enter correction requests and resubmit it.

[0518] The server receives revision requests from the user and regenerates the story and illustrations based on those requests. For example, if a user requests "more dinosaur scenes," the server regenerates new stories and illustrations to reflect that request. Finally, once the user is satisfied, the final digital picture book is generated. The final picture book is then converted to a suitable format (e.g., PDF, EPUB, etc.) at high resolution.

[0519] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0520] As a result, the present invention can efficiently generate a high-quality customized picture book based on user input, and can also quickly accommodate modifications in response to user requests.

[0521] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0522] Step 1:

[0523] The user opens a web browser, accesses the system's URL, enters the child's name, age, gender, and interest (e.g., dinosaurs, adventure) in the form that appears, and clicks the submit button.

[0524] Input: Child's name, age, gender, and interests.

[0525] Output: The input data in JSON format.

[0526] Step 2:

[0527] The terminal sends the input data to the server. Specifically, it converts the input data into JSON format using JavaScript and sends it to the server using an HTTP POST request.

[0528] Input: Input data in JSON format.

[0529] Output: The input data sent to the server.

[0530] Step 3:

[0531] The server analyzes the received input data and generates prompts for the generative AI model. This processing is done using the Python Flask framework.

[0532] Input: The input data sent to the server in JSON format.

[0533] Output: A prompt to the generative AI model.

[0534] Step 4:

[0535] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the prompt. Here, data generation calculations are performed based on the prompt. The generated story is stored in a database (e.g., PostgreSQL).

[0536] Input: A prompt to the generative AI model.

[0537] Output: A customized story.

[0538] Step 5:

[0539] The server analyzes the generated story and generates illustrations appropriate for each scene using an image generation engine (e.g., DALL-E). The generated illustrations are temporarily stored in a database.

[0540] Input: Your customized story.

[0541] Output: The generated illustration.

[0542] Step 6:

[0543] The server combines the generated story and illustrations to create a digital preview in HTML or PDF format, which is stored on the server.

[0544] Input: Generated story and illustrations.

[0545] Output: Digital preview (HTML, PDF formats).

[0546] Step 7:

[0547] The user uses a web browser to check the digital preview, enter correction requests if necessary, and submit it again.

[0548] Input: Digital preview and revision request.

[0549] Output: The correction request.

[0550] Step 8:

[0551] The server receives the modification request and regenerates the story and illustrations based on the request, which are also stored in the database.

[0552] Input: Correction request.

[0553] Output: Regenerated story and illustrations.

[0554] Step 9:

[0555] The server generates the final digital picture book by converting it into a suitable high-resolution format (e.g. PDF, EPUB, etc.) and stores the generated final version on the server.

[0556] Input: The final story and illustrations, as reviewed by the user.

[0557] Output: Final digital picture book (PDF, EPUB format).

[0558] Step 10:

[0559] The user downloads the final digital book and selects printing and delivery options if desired. If selected, the server receives the user's delivery information.

[0560] Input: User downloads and prints; selects shipping options; shipping information.

[0561] Output: Download link and shipping information.

[0562] Step 11:

[0563] The server sends a delivery request to a partner printing service company.

[0564] Input: Shipping information.

[0565] Output: A delivery request to a printing service company.

[0566] Step 12:

[0567] The printing service will deliver the printed picture book to the address specified by the user. The delivery status will also be updated and notified to the server.

[0568] Input: Delivery request.

[0569] Output: Delivery completed to user.

[0570] Through this process, users can create a customized storybook tailored to their child's interests and preferences, which can then be downloaded, printed, and delivered.

[0571] (Application example 1)

[0572] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0573] Conventional picture book creation services have had problems such as difficulty in customizing to individual users and lack of flexibility in previewing and modifying the generated picture books. In particular, it has been felt to be difficult to easily create digital picture books customized according to children's interests and preferences, and to easily view and share them on smart devices. The objective of this invention is to solve these problems and provide a system that enables users to easily create, modify, view, and share customized picture books.

[0574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0575] In this invention, the server includes means for receiving information entered by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send a request for corrections as needed, means for generating a final digital picture book and providing it to the user, means for providing printing and delivery options if desired by the user, means for making the generated digital picture book viewable on a smart device, means for making the digital picture book downloadable in high resolution, and means for enabling online sharing. This allows users to easily create customized picture books, preview them on the spot, make any necessary corrections, and then view or share them on their smart devices.

[0576] A "user" is a person who accesses the system, inputs information, and uses the generated digital picture book.

[0577] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides to the system.

[0578] A "customized story" is a narrative tailored to the individual user's needs, generated based on information entered by the user.

[0579] "Illustrations" are pictures or images generated based on a customized story.

[0580] "Digital Preview" is a digital picture book that combines story and illustrations and allows users to view it interactively.

[0581] A "request for correction" is a request made by a user to request corrections to content after checking the digital preview.

[0582] The "final digital picture book" is the final, high-resolution digital picture book that incorporates user correction requests.

[0583] "Print and Delivery Option" means an option that allows a user to physically print the final digital book and have it delivered to an address of their choosing.

[0584] "Smart devices" are electronic devices that can connect to the Internet, such as smartphones, smart glasses, and head-mounted displays.

[0585] "High resolution" refers to the quality of visual content, such as images and documents, that can be displayed with clear detail.

[0586] "Sharing online" refers to the ability to share the generated digital picture book with other users via the Internet.

[0587] Specific embodiments for carrying out the present invention will be described below.

[0588] Users access the system using a smartphone, personal computer, etc. The terminal displays an input form and allows the user to enter information such as the child's name, age, gender, and interests. Once the user has completed the input and clicked the submit button, the terminal sends this data to the server.

[0589] The server analyzes the received information and generates a customized story using a generative AI model (e.g., GPT-4). For example, if a user enters information such as "6 years old, boy, named Taro, loves dinosaurs," the server generates "a story about Taro's adventures with dinosaurs." The prompt used in this case is something like: "6-year-old boy, named Taro, loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[0590] Based on the generated story, the server uses an image generation engine (e.g., DALL-E) to generate illustrations. The image generation engine generates illustrations appropriate for each scene in the story, and combines the story and illustrations to create a digital preview.

[0591] Users can review this digital preview and submit revision requests as needed, which are then sent to the server, which then uses the generative AI model and image generation engine to update the preview and serve it back to the user, allowing them to iteratively customize the final version until they are satisfied with it.

[0592] Once the final digital book is complete, the server generates a high-resolution digital book and provides the user with a download link. If the user prefers, the server can also provide a printing and delivery option, allowing an external printing service to deliver the book to the user's specified address.

[0593] Furthermore, the generated digital picture books can be viewed on smart devices (e.g., smartphones, smart glasses, head-mounted displays) and can also be shared with other users online. This allows users to easily share their individually customized picture books with others and provides a fun reading experience for children.

[0594] To illustrate, here's an example prompt:

[0595] "I have a 6-year-old boy named Taro who loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[0596] The system effectively utilizes generative AI models and image generation engines to provide users with flexible customization and high-quality digital content.

[0597] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0598] Step 1:

[0599] A user accesses the system using a terminal and enters information such as the child's name, age, gender, and interests into the input form. After completing the input and clicking the submit button, the terminal sends this data to the server. Input is the various pieces of information provided by the user, and output is the data sent to the server.

[0600] Step 2:

[0601] The server analyzes the information received from the user. Here, it generates a prompt sentence appropriate for the generative AI model based on input data, particularly the child's name, age, gender, and topics of interest. The input is the data provided by the user, and the output is the generated prompt sentence. Specifically, it uses the data entered by the user to generate a prompt sentence such as, "A 6-year-old boy named Taro who loves dinosaurs. Please write a story about Taro's adventures with dinosaurs."

[0602] Step 3:

[0603] The server sends the generated prompt to a generative AI model (e.g., GPT-4) to generate a customized story. The generative AI model generates an optimal story based on the prompt and returns it to the server. The input is the prompt, and the output is the generated story. Specifically, the AI ​​model analyzes the prompt and generates a story such as "A story about Taro meeting a dinosaur and going on a great adventure."

[0604] Step 4:

[0605] The server receives the generated story and uses an image generation engine (e.g., DALL-E) to generate illustrations for each scene of the story as needed. The input is the generated story, and the output is illustrations corresponding to each scene. Specifically, it generates illustrations appropriate for each scene, such as "the scene where Taro meets a dinosaur for the first time."

[0606] Step 5:

[0607] The server combines the generated story and illustrations to create a digital preview. The input is a story-illustration pair, and the output is a digital preview. Specifically, the server combines the story and illustration in HTML or PDF format to create a preview version that users can check interactively.

[0608] Step 6:

[0609] The user checks the digital preview and sends a correction request if necessary. The server receives this correction request and updates the preview content again using the generative AI model and image generation engine. The input is the correction request from the user, and the output is the corrected digital preview. Specifically, based on the user request to "add more dinosaur scenes," the story and illustrations are regenerated.

[0610] Step 7:

[0611] Once the user is satisfied with the final version of the digital book, the server generates the final version and provides it to the user. The input is the revised preview, and the output is the final digital book. Specifically, it converts it to PDF or EPUB format at high resolution and provides a download link to the user.

[0612] Step 8:

[0613] The server provides printing and delivery options to the user if desired. The input is the user's delivery information, and the output is a request to the printing service company. Specifically, the server sends the necessary information to an external printing service company to deliver the picture book to the address specified by the user.

[0614] To ensure a smooth process, the document can also be viewed on smart devices and shared online.

[0615] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0616] Specific embodiments for carrying out the present invention will be described below.

[0617] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0618] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[0619] The server combines the generated story and illustrations to create a preview version of the digital picture book, which is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0620] The server sends an HTTP response containing the URL of the preview version to the device, and the user clicks on the link to view the preview. At this stage, the system is equipped with an emotion engine that recognizes emotions by analyzing the user's facial expressions and voice. The emotion engine analyzes the user's emotions and determines their feelings, such as "They seem to be having fun" or "They seem not to be interested."

[0621] The user checks the preview and, if any particular corrections are needed, enters the correction request in the input field. At this time, the server generates automatic correction suggestions for the story or illustrations, taking into account the user's emotions recognized by the emotion engine. For example, if the user expresses an expression that says "it doesn't look like he's having fun," the system can suggest corrections such as "add more dinosaur scenes." The user can review the correction suggestions and implement them as necessary.

[0622] The server receives the revision request and the revision suggestions from the emotion engine, then re-runs the generative AI model and image generation engine to generate a picture book that reflects the revisions. The regenerated preview version is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[0623] Finally, once the user is satisfied with the final version, the server generates the final high-resolution digital book and converts it into the appropriate format (PDF, EPUB, etc.). It then sends the URL of the final version to the user's device, allowing them to download it using the download link. At the same time, if the user wishes, it provides a print and delivery option, and the server sends a print and delivery request to the printing service provider.

[0624] Below is an example of this system:

[0625] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[0626] 2. The terminal sends the entered data to the server.

[0627] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[0628] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[0629] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[0630] 6. The user checks the preview. The emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0631] 7. The user reviews the sentiment-based revision suggestions and submits a revision request to include more dinosaur scenes.

[0632] 8. The server reflects the modification request and the emotion engine's suggestions and provides the regenerated preview to the user.

[0633] 9. The user is happy with the final version and selects download, print and delivery options.

[0634] 10. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[0635] 11. The printing service will deliver the printed picture book to the user's address.

[0636] Through this process, users can create a customized picture book that matches their child's interests, and by using an emotion engine, they can create a more personalized and enjoyable reading experience.

[0637] The processing flow will be explained below.

[0638] Step 1:

[0639] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[0640] Step 2:

[0641] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[0642] Step 3:

[0643] The server receives the HTTP request and analyzes the input data. The analysis results include the following information: "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[0644] Step 4:

[0645] The server passes the input data to the generative AI model, which then generates a customized story. Based on the user's input, the generative AI model creates a story about Taro meeting a dinosaur and going on a great adventure.

[0646] Step 5:

[0647] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page. For example, it generates illustrations for scenes such as "Taro meets a dinosaur" and "Taro goes on an adventure with a dinosaur."

[0648] Step 6:

[0649] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0650] Step 7:

[0651] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[0652] Step 8:

[0653] The user checks the preview. The device uses the camera and microphone to capture the user's facial expressions and voice, which are then analyzed by the emotion engine. For example, if the user is smiling, it will be determined that they look happy.

[0654] Step 9:

[0655] The device sends the analysis results from the emotion engine to the server, which receives the results and generates automatic suggestions for revising the story or illustrations based on emotional information such as "it looks fun."

[0656] Step 10:

[0657] The user checks the proposed revisions and makes any necessary changes. For example, the user enters a request such as "I want more dinosaur scenes" and clicks the submit button.

[0658] Step 11:

[0659] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[0660] Step 12:

[0661] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[0662] Step 13:

[0663] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[0664] Step 14:

[0665] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[0666] Step 15:

[0667] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[0668] Step 16:

[0669] The printing service company will deliver the printed picture book to the address specified by the user.

[0670] Example 2

[0671] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0672] While conventional customized picture book creation systems can generate stories based on information entered by users, they lack the ability to modify the story and illustrations taking into account the user's emotional responses. As a result, if a user is not satisfied with the content of the picture book, they must provide detailed correction instructions again, which is time-consuming. The present invention aims to solve these problems and efficiently generate more personalized customized picture books.

[0673] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0674] In this invention, the server includes means for receiving information input by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the server to provide the user with a link to the generated digital preview, means for the user to check the digital preview and send a correction request if necessary, means for analyzing the user's emotions using an emotion engine and generating correction suggestions based on the analysis results, means for regenerating the story and illustrations by reflecting the correction requests and the emotion analysis results, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to incorporate feedback based on the user's emotions and generate a customized picture book that provides a higher level of satisfaction.

[0675] "User" means an individual or end user who uses the system to create a customized digital picture book.

[0676] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides when using the system.

[0677] "Server" means a central processing unit or server system that receives input from users and generates, modifies, and distributes stories and illustrations based thereon.

[0678] A "customized story" is a narrative that is individually generated based on input information provided by the user.

[0679] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to automatically generate stories from user input.

[0680] An "illustration" is an image created to visually represent a scene from the generated story.

[0681] An "image generation engine" is an image generation algorithm or software for automatically generating illustrations based on a generated story.

[0682] The "Digital Preview" is an early version of the digital picture book that combines the generated story and illustrations, and is in a format that allows users to check and edit it.

[0683] An "emotion engine" is a technology or software that analyzes data such as a user's facial expressions and voice and recognizes their emotional state.

[0684] A "revision request" is a request for changes to the story or illustrations submitted by a user after viewing the digital preview.

[0685] "Revision suggestions" are suggestions for changes to stories or illustrations generated by the server based on the results of an analysis of the user's emotions by the emotion engine.

[0686] The "final digital picture book" is a high-quality digital picture book that is finally generated by reflecting the user's correction requests and suggestions from the emotion engine.

[0687] "Printing and Delivery Option" means an additional service that allows a user to print and deliver the final digital picture book as a physical book if the user so desires.

[0688] The present invention is a system for users to generate customized digital picture books through a process that includes collecting user input data, generating a story using a generative AI model, generating illustrations using an image generation engine, and analyzing user emotions using an emotion engine and making modifications based on the analysis.

[0689] Hardware and software used

[0690] This system uses the following hardware and software:

[0691] Terminal: A device (PC, smartphone, tablet, etc.) that a user uses to access the system using a web browser and enter and confirm information.

[0692] Server: The central processing system that receives user input and generates, modifies, and delivers stories and illustrations.

[0693] Generative AI models: Models that use natural language processing techniques to automatically generate stories (e.g., OpenAI GPT-4).

[0694] Image generation engine: An engine that generates illustrations based on the generated story (e.g., DALL-E).

[0695] Emotion engine: Technology that analyzes the user's facial expressions and voice to recognize emotions (e.g., Affectiva).

[0696] Explanation of program processing

[0697] Collecting user-entered data

[0698] When a user accesses the system through a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interests, and submits it. For example, enter the following:

[0699] Name: Taro

[0700] Age: 6

[0701] Gender: Male

[0702] Interests: Dinosaurs, adventure

[0703] Sending data to the server

[0704] When the user clicks the submit button, the device sends the input data to the server, which receives and analyzes the data.

[0705] Story Generation

[0706] The server uses a generative AI model to generate a customized story based on the received data, for example using the following prompt:

[0707] "Create a story about the great adventure of a 6-year-old boy named Taro, who begins with an encounter with a dinosaur."

[0708] The generative AI model generates a basic story, for example, "Once upon a time, a six-year-old boy named Taro encountered a dinosaur..."

[0709] Illustration generation

[0710] Based on the generated story, the server uses an image generation engine to generate illustrations appropriate for each scene, for example using the following prompt:

[0711] "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time."

[0712] This prompt will cause the image generation engine to generate an illustration.

[0713] Generate and deliver digital previews

[0714] The server combines the generated story and illustrations to create a preview version of the digital picture book (in HTML or PDF format), and then generates a URL for the preview version and provides it to the user.

[0715] Analyzing user sentiment and reflecting correction requests

[0716] When a user checks the preview version, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state. For example, it can distinguish emotions such as "interested" or "uninterested." When the user submits an emotion-based revision request, the server generates and implements revision suggestions based on the request and emotion analysis.

[0717] The server re-serves the regenerated story and illustrations as a digital picture book, and the process is repeated until the user is satisfied.

[0718] Final digital book delivery and printing / shipping options

[0719] Once the user is satisfied with the final version, the server generates the final high-resolution digital book, converts it to PDF or EPUB format, provides the user with a URL, and offers printing and delivery options if desired, and sends the request to a printing service, which will then deliver it to the address specified by the user.

[0720] The system allows users to quickly and efficiently generate customized picture books tailored to a child's interests, and by using an emotion engine, provides a more personalized and satisfying reading experience.

[0721] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0722] Step 1:

[0723] The user accesses the system via a web browser and the terminal displays an input form.

[0724] Input: The URL the user visited

[0725] Output: The input form is displayed on the terminal.

[0726] What happens: The user opens a browser and visits a URL (e.g., https: / / example.com). The device displays a user interface and provides a form for entering the child's information.

[0727] Step 2:

[0728] The user enters the child's name, age, gender, and interests and clicks the submit button.

[0729] Input: Name, age, gender, interests

[0730] Output: User input data is sent to the server

[0731] Specific operation: The user enters information such as "Taro," "6 years old," "male," "dinosaur, adventure" into the form and clicks the submit button, which causes the device to send the input data to the server.

[0732] Step 3:

[0733] The server parses the received input data.

[0734] Input: User input data (Taro, 6 years old, male, dinosaur, adventure)

[0735] Output: Parsed data

[0736] Specific operation: The server receives an HTTP POST request and analyzes the user's input data. The analyzed data is used for further processing.

[0737] Step 4:

[0738] The server uses generative AI models to generate customized stories.

[0739] Input: Parsed user data

[0740] Output: Customized story

[0741] What happens: The server sends the following prompt to a generative AI model (e.g., GPT-4): "Write a story about the great adventure of a 6-year-old boy named Taro, who begins with his encounter with a dinosaur." The generative AI model responds with a story: "Once upon a time, a 6-year-old boy named Taro met a dinosaur..."

[0742] Step 5:

[0743] The server uses an image generation engine to generate illustrations based on the generated story.

[0744] Input: Your customized story

[0745] Output: Illustration

[0746] Specific behavior: The server sends the following prompt to the image generation engine (e.g., DALL-E): "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time." The image generation engine generates the illustration and returns it to the server.

[0747] Step 6:

[0748] The server combines the generated story and illustrations to create a preview version of the digital picture book and generates a URL for it.

[0749] Input: Story, illustrations

[0750] Output: URL of the preview version of the digital picture book

[0751] What it does: The server combines the story and illustrations in HTML or PDF format to create a preview version of the digital picture book, and provides the generated URL to the user.

[0752] Step 7:

[0753] The server sends an HTTP response containing the preview URL to the device.

[0754] Input: Preview URL

[0755] Output: HTTP response

[0756] Specific operation: The server sends an HTTP response to the device and notifies the user of the preview URL (e.g., https: / / example.com / preview / taro_story). The user clicks the URL to view the preview.

[0757] Step 8:

[0758] The user checks the preview version, and the emotion engine analyzes the user's facial expressions and voice.

[0759] Input: Preview, facial expressions, and voice data

[0760] Output: Emotion analysis results

[0761] How it works: The user clicks on the preview URL to check the content. The emotion engine uses the camera and microphone to analyze the user's facial expressions and voice, and determines their emotions, such as "They seem to be having fun" or "They seem to be uninterested."

[0762] Step 9:

[0763] The user submits a correction request, and the server makes correction suggestions based on the results of sentiment analysis.

[0764] Input: Correction request, sentiment analysis results

[0765] Output: suggested fixes

[0766] How it works: A user inputs and submits a revision request, such as "I want more dinosaur scenes." The server, taking into account the results of sentiment analysis, generates a revision suggestion, such as "Add more adventurous scenes."

[0767] Step 10:

[0768] The server regenerates the story and illustrations based on the revision requests and suggestions.

[0769] Input: Correction request, correction proposal

[0770] Output: Regenerated story and illustrations

[0771] What it does: The server re-runs the generative AI model and image generation engine to generate new stories and illustrations that reflect the modifications.

[0772] Step 11:

[0773] The server sends an HTTP response containing the URL of the regenerated preview version to the device and provides it to the user.

[0774] Input: Regenerated story and illustrations

[0775] Output: Regenerated preview URL

[0776] Specific behavior: The server provides the newly generated preview URL (e.g., https: / / example.com / preview / taro_story_v2) to the user. The user checks the preview again.

[0777] Step 12:

[0778] Once the user is satisfied with the final version, the server generates the final digital picture book and provides it in PDF or EPUB format.

[0779] Input: User's final confirmation

[0780] Output: Final digital picture book (PDF or EPUB format)

[0781] What happens: Once the user is satisfied with the final version, the server generates a high-resolution digital storybook (e.g., PDF format) and provides a URL (e.g., https: / / example.com / final / taro_story_final.pdf).

[0782] Step 13:

[0783] If the user so desires, the server sends a print and delivery request to the printing service company.

[0784] Input: User's print and delivery request

[0785] Output: Request to printing service company

[0786] Specific operation: When the user selects the print and delivery option, the server sends the data and delivery information to the printing service company. The printed picture book is then delivered to the specified address.

[0787] (Application example 2)

[0788] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0789] Previous systems were unable to consider user emotions when generating customized digital picture books, which could result in lower user satisfaction. Furthermore, because the books were only available online, there was no way to improve user engagement through in-store experiences. This meant that users lacked the flexibility to experience the book on the spot and make suggestions for revisions immediately.

[0790] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0791] In this invention, the server includes a means for receiving information entered by the user, a means for generating a customized story based on the received information, a means for collecting and analyzing the user's emotional data using an emotion recognition engine, and a means for generating suggested revisions to the story and illustrations. This makes it possible to generate a customized picture book that takes the user's emotions into consideration, and by providing this service in a physical store, it becomes possible to provide an interactive experience on the spot.

[0792] "Means for receiving information entered by the user" refers to devices or software that collect information entered by the user, such as name, age, gender, and interests, and transmits it to the server.

[0793] "Means for generating a customized story based on received information" refers to a generative AI model or algorithm that creates an individualized customized story based on information received from a user.

[0794] "Means for generating illustrations based on the generated story" refers to an image generation engine or tool for creating appropriate illustrations according to the content of the generated story.

[0795] "Means for combining story and illustrations to generate a digital preview of a picture book" refers to software or a system that combines a customized story with generated illustrations and provides them in a user-viewable digital preview format (e.g., HTML, PDF).

[0796] "Means for users to check the digital preview and send correction requests if necessary" refers to an interface that allows users to check the preview of the generated digital picture book and send correction requests to the server if corrections are necessary.

[0797] "Means for collecting and analyzing user emotional data using an emotion recognition engine" refers to devices and software for automatically collecting and analyzing emotional data from users' facial expressions, voice, etc.

[0798] "Means for generating suggested revisions to stories and illustrations based on emotional data" refers to a system or algorithm that automatically generates more appropriate suggested revisions to stories and illustrations based on collected user emotional data.

[0799] "Means for regenerating the story and illustrations and providing a digital preview again" refers to a system for reflecting the correction request and emotional data and providing the newly generated story and illustrations again in a digital preview format.

[0800] "Means for generating and delivering the final digital book to the user" refers to a system that generates the final customized book in high resolution and delivers it to the user in an appropriate format (e.g., PDF, EPUB).

[0801] "Means for providing printing and delivery options upon user's request" refers to systems or services that provide users with the option to print and deliver physical picture books upon user's request.

[0802] "Means for providing services to users using tablet devices or dedicated kiosk terminals in physical stores" refers to devices and systems that allow users to directly experience creating customized picture books using tablet devices or dedicated kiosk terminals in physical stores.

[0803] Specific embodiments of the present invention will be described below.

[0804] Users access the "Detailed Description of the Invention" using a tablet terminal or dedicated kiosk terminal installed in a physical store. The terminal displays an input form to the user via a web browser. The user enters the child's name, age, gender, and interests and clicks the submit button. This sends the input data from the terminal to the server.

[0805] The server is a high-performance cloud server (e.g., AWS EC2, Google Cloud Platform) that performs the following processes: First, a generative AI model (e.g., GPT-4, BERT) is used to generate a customized story based on user input data. Next, an image generation engine (e.g., DALL-E, Stable Diffusion) is used to generate illustrations appropriate for each scene in the generated story. The generated story and illustrations are combined to create a preview of the digital picture book. The preview version is created in an interactive format (e.g., HTML, PDF), and a URL is generated. The server sends this URL to the device, and the user clicks the link to view the preview version.

[0806] An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to analyze the facial expressions and voice of the user while previewing the digital picture book to recognize their emotions. This emotional data is sent to the server. The server then automatically generates revision suggestions based on the collected emotional data. For example, if an expression indicating "not looking happy" is recognized, the system will suggest "add more dinosaur scenes." The user can review the revision suggestions and, if necessary, enter their revision requests in the input field.

[0807] The server receives the revision request and the emotion-based revision suggestions, re-runs the generative AI model and image generation engine to regenerate the picture book reflecting the revisions. A preview of the regenerated digital picture book is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[0808] Finally, once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g., PDF, EPUB). It then sends a URL for the final version to the user's device, which the user can use to download the digital book. At the same time, it provides a print and delivery option if the user wishes, and the server sends a print and delivery request to the printing service provider.

[0809] For example, if a user inputs "Taro," "6 years old," "male," "dinosaur, adventure" into a tablet device, the server generates a story about Taro meeting a dinosaur and going on a great adventure, and an image generation engine creates illustrations based on the story. A preview of the digital picture book is then generated for the user to review. If the emotion recognition engine determines that the book "doesn't look fun," the system suggests adding more dinosaur scenes. This process is repeated until the user is satisfied, and the final version is generated.

[0810] An example of a prompt for a generative AI model is:

[0811] Child information provided by the user:

[0812] Name: Taro

[0813] Age: 6

[0814] Gender: Male

[0815] Interests: Dinosaurs, adventure

[0816] prompt:

[0817] Create a story about a 6-year-old boy named Taro who meets a dinosaur and goes on a great adventure. Include scene descriptions for each scene to illustrate.

[0818] This allows users to enjoy the experience of creating a customized picture book in real time in a physical store.

[0819] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0820] Step 1:

[0821] The user accesses the input form through a web browser using a tablet device or dedicated kiosk device in a physical store. The user enters the child's name, age, gender, and interests, and clicks the submit button. The input, such as name (e.g., "Taro"), age (e.g., "6 years old"), gender (e.g., "male"), and interests (e.g., "dinosaurs," "adventure"), is sent to the server. This allows the user's information to be collected.

[0822] Step 2:

[0823] The server receives the data sent by the device. The server stores the received user input data. User information is passed to the server as input, and customized data is prepared based on this information.

[0824] Step 3:

[0825] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the input data. The server passes a prompt to the generative AI model, which then generates a story based on it. Specifically, the name, age, gender, and interests entered by the user are input as prompts into the generative AI model, and a customized story is returned.

[0826] Step 4:

[0827] The server uses an image generation engine (e.g., DALL-E) to create illustrations appropriate for each scene in the generated story. The server receives the story scenes as input and generates illustrations corresponding to each scene as output. These illustrations are then combined with the story.

[0828] Step 5:

[0829] The server combines the generated story and illustrations to create a preview of the digital picture book. The preview is generated in an interactive format (e.g., HTML, PDF), and a URL is created. The server then sends this URL to the device, where the preview is provided to the user.

[0830] Step 6:

[0831] The user checks a preview of the digital picture book on their device. An emotion recognition engine (e.g., Microsoft Azure Emotion API) analyzes the user's facial expressions and voice to recognize their emotions. The input is collected by the camera and microphone, and emotional data is generated as output.

[0832] Step 7:

[0833] The server automatically generates revision suggestions based on the emotion data. For example, if the user's facial expression indicates that they are not having fun, the server will suggest revisions such as adding more dinosaur scenes. The revision suggestions are provided to the user, who then confirms them.

[0834] Step 8:

[0835] The user checks the suggested revisions and, if necessary, enters a revision request into the input field on the device. The server receives this revision request and again runs the generative AI model and image generation engine to revise the story and illustration. The revision request is passed as input, and the revised story and illustration are generated as output.

[0836] Step 9:

[0837] The server then provides the user with a regenerated preview of the digital book that reflects the modifications. The user then checks the preview again and repeats this process until satisfied. This results in customization that increases user satisfaction.

[0838] Step 10:

[0839] Once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g. PDF, EPUB). A URL for the final version is sent to the device, where the user can use this link to download the digital book. If desired, the user selects the print and delivery option, and the server sends a print and delivery request to the printing service provider.

[0840] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0841] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0842] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0843] [Third embodiment]

[0844] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0845] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0846] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0847] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0848] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0849] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0850] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0851] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0852] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0853] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0854] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0855] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0856] Specific embodiments for carrying out the present invention will be described below.

[0857] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0858] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[0859] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version, enter correction requests if necessary, and resubmit it.

[0860] The server receives revision requests from the user and recreates the story and illustrations based on those requests. Finally, once the user is satisfied, the final digital book is generated. This final book is then converted to the appropriate format (PDF, EPUB, etc.) in high resolution.

[0861] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0862] Below is an example of this system:

[0863] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[0864] 2. The terminal sends the entered data to the server.

[0865] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[0866] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[0867] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[0868] 6. The user checks the preview and submits a revision request to add more dinosaur scenes.

[0869] 7. The server processes the requested modifications and provides the regenerated preview to the user.

[0870] 8. The user is happy with the final version and selects download, print and delivery options.

[0871] 9. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[0872] 10. The printing service will deliver the printed picture book to the user's address.

[0873] Through this process, users can create a customized picture book that matches their child's interests and preferences, creating a fun reading experience.

[0874] The processing flow will be explained below.

[0875] Step 1:

[0876] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[0877] Step 2:

[0878] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[0879] Step 3:

[0880] The server receives the HTTP request and analyzes the input data. For example, it analyzes information such as "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[0881] Step 4:

[0882] The server passes the input data to a generative AI model that generates a customized story. Based on the user's input, the generative AI model creates a story such as "Taro meets a dinosaur and goes on a great adventure."

[0883] Step 5:

[0884] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page, such as the scene where Taro meets a dinosaur.

[0885] Step 6:

[0886] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[0887] Step 7:

[0888] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[0889] Step 8:

[0890] The user checks the preview and, if necessary, enters a request for corrections, such as "I want more dinosaur scenes," and resubmits the video.

[0891] Step 9:

[0892] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[0893] Step 10:

[0894] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[0895] Step 11:

[0896] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[0897] Step 12:

[0898] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[0899] Step 13:

[0900] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[0901] Step 14:

[0902] The printing service company will deliver the printed picture book to the address specified by the user.

[0903] Example 1

[0904] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0905] Existing picture book generation systems have difficulty efficiently generating customized picture books based on user input. Furthermore, the quality of story and illustration generation is not consistent, and modifications to meet user requests are not easily made. This makes it necessary to solve the problem of being unable to quickly and accurately provide customized picture books that satisfy users.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0907] In this invention, the server includes means for receiving information input by a user, means for generating prompt sentences for a generative AI model based on the received information, means for generating a customized story using the generated prompt sentences, means for generating illustrations using an image generation engine based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send correction requests as needed, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to efficiently generate high-quality customized picture books based on user input and quickly accommodate corrections requested by the user.

[0908] "User" refers to a person who accesses the system and inputs information.

[0909] "Information to be entered" refers to data such as the child's name, age, gender, and interests.

[0910] "Means of receiving" refers to the mechanism by which the server obtains the information entered by the user.

[0911] "Generative AI model" refers to an artificial intelligence model that generates customized stories based on user input.

[0912] A "prompt" refers to an instruction used to generate a story for a generative AI model.

[0913] "Customized Story" refers to a bespoke narrative generated based on user input.

[0914] "Image generation engine" refers to a system for automatically generating illustrations based on a customized story.

[0915] "Illustration" refers to an image that corresponds to a customized story.

[0916] "Digital Preview" refers to a digital version of a picture book that combines a generated story and illustrations for advance viewing.

[0917] "Correction Request" refers to a request for changes or additions made by a user after reviewing a digital preview.

[0918] A "final digital picture book" refers to a completed digital picture book that has been approved by the user.

[0919] "Print and Delivery Options" means the option to have the final digital book printed as a physical book and delivered to a specified address.

[0920] The "system" refers to the overall mechanism that combines these means to provide users with customized picture books.

[0921] The present invention relates to a system for generating a customized picture book based on information input by a user. Specific embodiments for carrying out the present invention will be described below.

[0922] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[0923] The server analyzes the user's input data received from the device and generates a customized story based on it. Specifically, it uses a generative AI model (e.g., GPT-4) to convert the user's input data into a prompt. For example, this prompt might say, "Please create a story about a 6-year-old boy named Taro who goes on a great adventure with dinosaurs."

[0924] A customized story is created based on the prompts generated by the generative AI model. The server then uses an image generation engine (e.g., DALL-E) to generate illustrations appropriate for each page of the story. For example, the illustration for "Taro's first encounter with a dinosaur" is generated by this engine.

[0925] The server combines the generated story and illustrations to create a preview version of the digital picture book. This preview version is created in an interactive format (e.g., HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version and, if necessary, enter correction requests and resubmit it.

[0926] The server receives revision requests from the user and regenerates the story and illustrations based on those requests. For example, if a user requests "more dinosaur scenes," the server regenerates new stories and illustrations to reflect that request. Finally, once the user is satisfied, the final digital picture book is generated. The final picture book is then converted to a suitable format (e.g., PDF, EPUB, etc.) at high resolution.

[0927] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[0928] As a result, the present invention can efficiently generate a high-quality customized picture book based on user input, and can also quickly accommodate modifications in response to user requests.

[0929] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0930] Step 1:

[0931] The user opens a web browser, accesses the system's URL, enters the child's name, age, gender, and interest (e.g., dinosaurs, adventure) in the form that appears, and clicks the submit button.

[0932] Input: Child's name, age, gender, and interests.

[0933] Output: The input data in JSON format.

[0934] Step 2:

[0935] The terminal sends the input data to the server. Specifically, it converts the input data into JSON format using JavaScript and sends it to the server using an HTTP POST request.

[0936] Input: Input data in JSON format.

[0937] Output: The input data sent to the server.

[0938] Step 3:

[0939] The server analyzes the received input data and generates prompts for the generative AI model. This processing is done using the Python Flask framework.

[0940] Input: The input data sent to the server in JSON format.

[0941] Output: A prompt to the generative AI model.

[0942] Step 4:

[0943] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the prompt. Here, data generation calculations are performed based on the prompt. The generated story is stored in a database (e.g., PostgreSQL).

[0944] Input: A prompt to the generative AI model.

[0945] Output: A customized story.

[0946] Step 5:

[0947] The server analyzes the generated story and generates illustrations appropriate for each scene using an image generation engine (e.g., DALL-E). The generated illustrations are temporarily stored in a database.

[0948] Input: Your customized story.

[0949] Output: The generated illustration.

[0950] Step 6:

[0951] The server combines the generated story and illustrations to create a digital preview in HTML or PDF format, which is stored on the server.

[0952] Input: Generated story and illustrations.

[0953] Output: Digital preview (HTML, PDF formats).

[0954] Step 7:

[0955] The user uses a web browser to check the digital preview, enter correction requests if necessary, and submit it again.

[0956] Input: Digital preview and revision request.

[0957] Output: The correction request.

[0958] Step 8:

[0959] The server receives the modification request and regenerates the story and illustrations based on the request, which are also stored in the database.

[0960] Input: Correction request.

[0961] Output: Regenerated story and illustrations.

[0962] Step 9:

[0963] The server generates the final digital picture book by converting it into a suitable high-resolution format (e.g. PDF, EPUB, etc.) and stores the generated final version on the server.

[0964] Input: The final story and illustrations, as reviewed by the user.

[0965] Output: Final digital picture book (PDF, EPUB format).

[0966] Step 10:

[0967] The user downloads the final digital book and selects printing and delivery options if desired. If selected, the server receives the user's delivery information.

[0968] Input: User downloads and prints; selects shipping options; shipping information.

[0969] Output: Download link and shipping information.

[0970] Step 11:

[0971] The server sends a delivery request to a partner printing service company.

[0972] Input: Shipping information.

[0973] Output: A delivery request to a printing service company.

[0974] Step 12:

[0975] The printing service will deliver the printed picture book to the address specified by the user. The delivery status will also be updated and notified to the server.

[0976] Input: Delivery request.

[0977] Output: Delivery completed to user.

[0978] Through this process, users can create a customized storybook tailored to their child's interests and preferences, which can then be downloaded, printed, and delivered.

[0979] (Application example 1)

[0980] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0981] Conventional picture book creation services have had problems such as difficulty in customizing to individual users and lack of flexibility in previewing and modifying the generated picture books. In particular, it has been felt to be difficult to easily create digital picture books customized according to children's interests and preferences, and to easily view and share them on smart devices. The objective of this invention is to solve these problems and provide a system that enables users to easily create, modify, view, and share customized picture books.

[0982] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0983] In this invention, the server includes means for receiving information entered by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send a request for corrections as needed, means for generating a final digital picture book and providing it to the user, means for providing printing and delivery options if desired by the user, means for making the generated digital picture book viewable on a smart device, means for making the digital picture book downloadable in high resolution, and means for enabling online sharing. This allows users to easily create customized picture books, preview them on the spot, make any necessary corrections, and then view or share them on their smart devices.

[0984] A "user" is a person who accesses the system, inputs information, and uses the generated digital picture book.

[0985] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides to the system.

[0986] A "customized story" is a narrative tailored to the individual user's needs, generated based on information entered by the user.

[0987] "Illustrations" are pictures or images generated based on a customized story.

[0988] "Digital Preview" is a digital picture book that combines story and illustrations and allows users to view it interactively.

[0989] A "request for correction" is a request made by a user to request corrections to content after checking the digital preview.

[0990] The "final digital picture book" is the final, high-resolution digital picture book that incorporates user correction requests.

[0991] "Print and Delivery Option" means an option that allows a user to physically print the final digital book and have it delivered to an address of their choosing.

[0992] "Smart devices" are electronic devices that can connect to the Internet, such as smartphones, smart glasses, and head-mounted displays.

[0993] "High resolution" refers to the quality of visual content, such as images and documents, that can be displayed with clear detail.

[0994] "Sharing online" refers to the ability to share the generated digital picture book with other users via the Internet.

[0995] Specific embodiments for carrying out the present invention will be described below.

[0996] Users access the system using a smartphone, personal computer, etc. The terminal displays an input form and allows the user to enter information such as the child's name, age, gender, and interests. Once the user has completed the input and clicked the submit button, the terminal sends this data to the server.

[0997] The server analyzes the received information and generates a customized story using a generative AI model (e.g., GPT-4). For example, if a user enters information such as "6 years old, boy, named Taro, loves dinosaurs," the server generates "a story about Taro's adventures with dinosaurs." The prompt used in this case is something like: "6-year-old boy, named Taro, loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[0998] Based on the generated story, the server uses an image generation engine (e.g., DALL-E) to generate illustrations. The image generation engine generates illustrations appropriate for each scene in the story, and combines the story and illustrations to create a digital preview.

[0999] Users can review this digital preview and submit revision requests as needed, which are then sent to the server, which then uses the generative AI model and image generation engine to update the preview and serve it back to the user, allowing them to iteratively customize the final version until they are satisfied with it.

[1000] Once the final digital book is complete, the server generates a high-resolution digital book and provides the user with a download link. If the user prefers, the server can also provide a printing and delivery option, allowing an external printing service to deliver the book to the user's specified address.

[1001] Furthermore, the generated digital picture books can be viewed on smart devices (e.g., smartphones, smart glasses, head-mounted displays) and can also be shared with other users online. This allows users to easily share their individually customized picture books with others and provides a fun reading experience for children.

[1002] To illustrate, here's an example prompt:

[1003] "I have a 6-year-old boy named Taro who loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[1004] The system effectively utilizes generative AI models and image generation engines to provide users with flexible customization and high-quality digital content.

[1005] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1006] Step 1:

[1007] A user accesses the system using a terminal and enters information such as the child's name, age, gender, and interests into the input form. After completing the input and clicking the submit button, the terminal sends this data to the server. Input is the various pieces of information provided by the user, and output is the data sent to the server.

[1008] Step 2:

[1009] The server analyzes the information received from the user. Here, it generates a prompt sentence appropriate for the generative AI model based on input data, particularly the child's name, age, gender, and topics of interest. The input is the data provided by the user, and the output is the generated prompt sentence. Specifically, it uses the data entered by the user to generate a prompt sentence such as, "A 6-year-old boy named Taro who loves dinosaurs. Please write a story about Taro's adventures with dinosaurs."

[1010] Step 3:

[1011] The server sends the generated prompt to a generative AI model (e.g., GPT-4) to generate a customized story. The generative AI model generates an optimal story based on the prompt and returns it to the server. The input is the prompt, and the output is the generated story. Specifically, the AI ​​model analyzes the prompt and generates a story such as "A story about Taro meeting a dinosaur and going on a great adventure."

[1012] Step 4:

[1013] The server receives the generated story and uses an image generation engine (e.g., DALL-E) to generate illustrations for each scene of the story as needed. The input is the generated story, and the output is illustrations corresponding to each scene. Specifically, it generates illustrations appropriate for each scene, such as "the scene where Taro meets a dinosaur for the first time."

[1014] Step 5:

[1015] The server combines the generated story and illustrations to create a digital preview. The input is a story-illustration pair, and the output is a digital preview. Specifically, the server combines the story and illustration in HTML or PDF format to create a preview version that users can check interactively.

[1016] Step 6:

[1017] The user checks the digital preview and sends a correction request if necessary. The server receives this correction request and updates the preview content again using the generative AI model and image generation engine. The input is the correction request from the user, and the output is the corrected digital preview. Specifically, based on the user request to "add more dinosaur scenes," the story and illustrations are regenerated.

[1018] Step 7:

[1019] Once the user is satisfied with the final version of the digital book, the server generates the final version and provides it to the user. The input is the revised preview, and the output is the final digital book. Specifically, it converts it to PDF or EPUB format at high resolution and provides a download link to the user.

[1020] Step 8:

[1021] The server provides printing and delivery options to the user if desired. The input is the user's delivery information, and the output is a request to the printing service company. Specifically, the server sends the necessary information to an external printing service company to deliver the picture book to the address specified by the user.

[1022] To ensure a smooth process, the document can also be viewed on smart devices and shared online.

[1023] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1024] Specific embodiments for carrying out the present invention will be described below.

[1025] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[1026] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[1027] The server combines the generated story and illustrations to create a preview version of the digital picture book, which is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[1028] The server sends an HTTP response containing the URL of the preview version to the device, and the user clicks on the link to view the preview. At this stage, the system is equipped with an emotion engine that recognizes emotions by analyzing the user's facial expressions and voice. The emotion engine analyzes the user's emotions and determines their feelings, such as "They seem to be having fun" or "They seem not to be interested."

[1029] The user checks the preview and, if any particular corrections are needed, enters the correction request in the input field. At this time, the server generates automatic correction suggestions for the story or illustrations, taking into account the user's emotions recognized by the emotion engine. For example, if the user expresses an expression that says "it doesn't look like he's having fun," the system can suggest corrections such as "add more dinosaur scenes." The user can review the correction suggestions and implement them as necessary.

[1030] The server receives the revision request and the revision suggestions from the emotion engine, then re-runs the generative AI model and image generation engine to generate a picture book that reflects the revisions. The regenerated preview version is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[1031] Finally, once the user is satisfied with the final version, the server generates the final high-resolution digital book and converts it into the appropriate format (PDF, EPUB, etc.). It then sends the URL of the final version to the user's device, allowing them to download it using the download link. At the same time, if the user wishes, it provides a print and delivery option, and the server sends a print and delivery request to the printing service provider.

[1032] Below is an example of this system:

[1033] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[1034] 2. The terminal sends the entered data to the server.

[1035] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[1036] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[1037] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[1038] 6. The user checks the preview. The emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1039] 7. The user reviews the sentiment-based revision suggestions and submits a revision request to include more dinosaur scenes.

[1040] 8. The server reflects the modification request and the emotion engine's suggestions and provides the regenerated preview to the user.

[1041] 9. The user is happy with the final version and selects download, print and delivery options.

[1042] 10. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[1043] 11. The printing service will deliver the printed picture book to the user's address.

[1044] Through this process, users can create a customized picture book that matches their child's interests, and by using an emotion engine, they can create a more personalized and enjoyable reading experience.

[1045] The processing flow will be explained below.

[1046] Step 1:

[1047] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[1048] Step 2:

[1049] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[1050] Step 3:

[1051] The server receives the HTTP request and analyzes the input data. The analysis results include the following information: "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[1052] Step 4:

[1053] The server passes the input data to the generative AI model, which then generates a customized story. Based on the user's input, the generative AI model creates a story about Taro meeting a dinosaur and going on a great adventure.

[1054] Step 5:

[1055] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page. For example, it generates illustrations for scenes such as "Taro meets a dinosaur" and "Taro goes on an adventure with a dinosaur."

[1056] Step 6:

[1057] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[1058] Step 7:

[1059] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[1060] Step 8:

[1061] The user checks the preview. The device uses the camera and microphone to capture the user's facial expressions and voice, which are then analyzed by the emotion engine. For example, if the user is smiling, it will be determined that they look happy.

[1062] Step 9:

[1063] The device sends the analysis results from the emotion engine to the server, which receives the results and generates automatic suggestions for revising the story or illustrations based on emotional information such as "it looks fun."

[1064] Step 10:

[1065] The user checks the proposed revisions and makes any necessary changes. For example, the user enters a request such as "I want more dinosaur scenes" and clicks the submit button.

[1066] Step 11:

[1067] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[1068] Step 12:

[1069] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[1070] Step 13:

[1071] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[1072] Step 14:

[1073] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[1074] Step 15:

[1075] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[1076] Step 16:

[1077] The printing service company will deliver the printed picture book to the address specified by the user.

[1078] Example 2

[1079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1080] While conventional customized picture book creation systems can generate stories based on information entered by users, they lack the ability to modify the story and illustrations taking into account the user's emotional responses. As a result, if a user is not satisfied with the content of the picture book, they must provide detailed correction instructions again, which is time-consuming. The present invention aims to solve these problems and efficiently generate more personalized customized picture books.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1082] In this invention, the server includes means for receiving information input by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the server to provide the user with a link to the generated digital preview, means for the user to check the digital preview and send a correction request if necessary, means for analyzing the user's emotions using an emotion engine and generating correction suggestions based on the analysis results, means for regenerating the story and illustrations by reflecting the correction requests and the emotion analysis results, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to incorporate feedback based on the user's emotions and generate a customized picture book that provides a higher level of satisfaction.

[1083] "User" means an individual or end user who uses the system to create a customized digital picture book.

[1084] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides when using the system.

[1085] "Server" means a central processing unit or server system that receives input from users and generates, modifies, and distributes stories and illustrations based thereon.

[1086] A "customized story" is a narrative that is individually generated based on input information provided by the user.

[1087] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to automatically generate stories from user input.

[1088] An "illustration" is an image created to visually represent a scene from the generated story.

[1089] An "image generation engine" is an image generation algorithm or software for automatically generating illustrations based on a generated story.

[1090] The "Digital Preview" is an early version of the digital picture book that combines the generated story and illustrations, and is in a format that allows users to check and edit it.

[1091] An "emotion engine" is a technology or software that analyzes data such as a user's facial expressions and voice and recognizes their emotional state.

[1092] A "revision request" is a request for changes to the story or illustrations submitted by a user after viewing the digital preview.

[1093] "Revision suggestions" are suggestions for changes to stories or illustrations generated by the server based on the results of an analysis of the user's emotions by the emotion engine.

[1094] The "final digital picture book" is a high-quality digital picture book that is finally generated by reflecting the user's correction requests and suggestions from the emotion engine.

[1095] "Printing and Delivery Option" means an additional service that allows a user to print and deliver the final digital picture book as a physical book if the user so desires.

[1096] The present invention is a system for users to generate customized digital picture books through a process that includes collecting user input data, generating a story using a generative AI model, generating illustrations using an image generation engine, and analyzing user emotions using an emotion engine and making modifications based on the analysis.

[1097] Hardware and software used

[1098] This system uses the following hardware and software:

[1099] Terminal: A device (PC, smartphone, tablet, etc.) that a user uses to access the system using a web browser and enter and confirm information.

[1100] Server: The central processing system that receives user input and generates, modifies, and delivers stories and illustrations.

[1101] Generative AI models: Models that use natural language processing techniques to automatically generate stories (e.g., OpenAI GPT-4).

[1102] Image generation engine: An engine that generates illustrations based on the generated story (e.g., DALL-E).

[1103] Emotion engine: Technology that analyzes the user's facial expressions and voice to recognize emotions (e.g., Affectiva).

[1104] Explanation of program processing

[1105] Collecting user-entered data

[1106] When a user accesses the system through a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interests, and submits it. For example, enter the following:

[1107] Name: Taro

[1108] Age: 6

[1109] Gender: Male

[1110] Interests: Dinosaurs, adventure

[1111] Sending data to the server

[1112] When the user clicks the submit button, the device sends the input data to the server, which receives and analyzes the data.

[1113] Story Generation

[1114] The server uses a generative AI model to generate a customized story based on the received data, for example using the following prompt:

[1115] "Create a story about the great adventure of a 6-year-old boy named Taro, who begins with an encounter with a dinosaur."

[1116] The generative AI model generates a basic story, for example, "Once upon a time, a six-year-old boy named Taro encountered a dinosaur..."

[1117] Illustration generation

[1118] Based on the generated story, the server uses an image generation engine to generate illustrations appropriate for each scene, for example using the following prompt:

[1119] "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time."

[1120] This prompt will cause the image generation engine to generate an illustration.

[1121] Generate and deliver digital previews

[1122] The server combines the generated story and illustrations to create a preview version of the digital picture book (in HTML or PDF format), and then generates a URL for the preview version and provides it to the user.

[1123] Analyzing user sentiment and reflecting correction requests

[1124] When a user checks the preview version, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state. For example, it can distinguish emotions such as "interested" or "uninterested." When the user submits an emotion-based revision request, the server generates and implements revision suggestions based on the request and emotion analysis.

[1125] The server re-serves the regenerated story and illustrations as a digital picture book, and the process is repeated until the user is satisfied.

[1126] Final digital book delivery and printing / shipping options

[1127] Once the user is satisfied with the final version, the server generates the final high-resolution digital book, converts it to PDF or EPUB format, provides the user with a URL, and offers printing and delivery options if desired, and sends the request to a printing service, which will then deliver it to the address specified by the user.

[1128] The system allows users to quickly and efficiently generate customized picture books tailored to a child's interests, and by using an emotion engine, provides a more personalized and satisfying reading experience.

[1129] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1130] Step 1:

[1131] The user accesses the system via a web browser and the terminal displays an input form.

[1132] Input: The URL the user visited

[1133] Output: The input form is displayed on the terminal.

[1134] What happens: The user opens a browser and visits a URL (e.g., https: / / example.com). The device displays a user interface and provides a form for entering the child's information.

[1135] Step 2:

[1136] The user enters the child's name, age, gender, and interests and clicks the submit button.

[1137] Input: Name, age, gender, interests

[1138] Output: User input data is sent to the server

[1139] Specific operation: The user enters information such as "Taro," "6 years old," "male," "dinosaur, adventure" into the form and clicks the submit button, which causes the device to send the input data to the server.

[1140] Step 3:

[1141] The server parses the received input data.

[1142] Input: User input data (Taro, 6 years old, male, dinosaur, adventure)

[1143] Output: Parsed data

[1144] Specific operation: The server receives an HTTP POST request and analyzes the user's input data. The analyzed data is used for further processing.

[1145] Step 4:

[1146] The server uses generative AI models to generate customized stories.

[1147] Input: Parsed user data

[1148] Output: Customized story

[1149] What happens: The server sends the following prompt to a generative AI model (e.g., GPT-4): "Write a story about the great adventure of a 6-year-old boy named Taro, who begins with his encounter with a dinosaur." The generative AI model responds with a story: "Once upon a time, a 6-year-old boy named Taro met a dinosaur..."

[1150] Step 5:

[1151] The server uses an image generation engine to generate illustrations based on the generated story.

[1152] Input: Your customized story

[1153] Output: Illustration

[1154] Specific behavior: The server sends the following prompt to the image generation engine (e.g., DALL-E): "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time." The image generation engine generates the illustration and returns it to the server.

[1155] Step 6:

[1156] The server combines the generated story and illustrations to create a preview version of the digital picture book and generates a URL for it.

[1157] Input: Story, illustrations

[1158] Output: URL of the preview version of the digital picture book

[1159] What it does: The server combines the story and illustrations in HTML or PDF format to create a preview version of the digital picture book, and provides the generated URL to the user.

[1160] Step 7:

[1161] The server sends an HTTP response containing the preview URL to the device.

[1162] Input: Preview URL

[1163] Output: HTTP response

[1164] Specific operation: The server sends an HTTP response to the device and notifies the user of the preview URL (e.g., https: / / example.com / preview / taro_story). The user clicks the URL to view the preview.

[1165] Step 8:

[1166] The user checks the preview version, and the emotion engine analyzes the user's facial expressions and voice.

[1167] Input: Preview, facial expressions, and voice data

[1168] Output: Emotion analysis results

[1169] How it works: The user clicks on the preview URL to check the content. The emotion engine uses the camera and microphone to analyze the user's facial expressions and voice, and determines their emotions, such as "They seem to be having fun" or "They seem to be uninterested."

[1170] Step 9:

[1171] The user submits a correction request, and the server makes correction suggestions based on the results of sentiment analysis.

[1172] Input: Correction request, sentiment analysis results

[1173] Output: suggested fixes

[1174] How it works: A user inputs and submits a revision request, such as "I want more dinosaur scenes." The server, taking into account the results of sentiment analysis, generates a revision suggestion, such as "Add more adventurous scenes."

[1175] Step 10:

[1176] The server regenerates the story and illustrations based on the revision requests and suggestions.

[1177] Input: Correction request, correction proposal

[1178] Output: Regenerated story and illustrations

[1179] What it does: The server re-runs the generative AI model and image generation engine to generate new stories and illustrations that reflect the modifications.

[1180] Step 11:

[1181] The server sends an HTTP response containing the URL of the regenerated preview version to the device and provides it to the user.

[1182] Input: Regenerated story and illustrations

[1183] Output: Regenerated preview URL

[1184] Specific behavior: The server provides the newly generated preview URL (e.g., https: / / example.com / preview / taro_story_v2) to the user. The user checks the preview again.

[1185] Step 12:

[1186] Once the user is satisfied with the final version, the server generates the final digital picture book and provides it in PDF or EPUB format.

[1187] Input: User's final confirmation

[1188] Output: Final digital picture book (PDF or EPUB format)

[1189] What happens: Once the user is satisfied with the final version, the server generates a high-resolution digital storybook (e.g., PDF format) and provides a URL (e.g., https: / / example.com / final / taro_story_final.pdf).

[1190] Step 13:

[1191] If the user so desires, the server sends a print and delivery request to the printing service company.

[1192] Input: User's print and delivery request

[1193] Output: Request to printing service company

[1194] Specific operation: When the user selects the print and delivery option, the server sends the data and delivery information to the printing service company. The printed picture book is then delivered to the specified address.

[1195] (Application example 2)

[1196] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1197] Previous systems were unable to consider user emotions when generating customized digital picture books, which could result in lower user satisfaction. Furthermore, because the books were only available online, there was no way to improve user engagement through in-store experiences. This meant that users lacked the flexibility to experience the book on the spot and make suggestions for revisions immediately.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1199] In this invention, the server includes a means for receiving information entered by the user, a means for generating a customized story based on the received information, a means for collecting and analyzing the user's emotional data using an emotion recognition engine, and a means for generating suggested revisions to the story and illustrations. This makes it possible to generate a customized picture book that takes the user's emotions into consideration, and by providing this service in a physical store, it becomes possible to provide an interactive experience on the spot.

[1200] "Means for receiving information entered by the user" refers to devices or software that collect information entered by the user, such as name, age, gender, and interests, and transmits it to the server.

[1201] "Means for generating a customized story based on received information" refers to a generative AI model or algorithm that creates an individualized customized story based on information received from a user.

[1202] "Means for generating illustrations based on the generated story" refers to an image generation engine or tool for creating appropriate illustrations according to the content of the generated story.

[1203] "Means for combining story and illustrations to generate a digital preview of a picture book" refers to software or a system that combines a customized story with generated illustrations and provides them in a user-viewable digital preview format (e.g., HTML, PDF).

[1204] "Means for users to check the digital preview and send correction requests if necessary" refers to an interface that allows users to check the preview of the generated digital picture book and send correction requests to the server if corrections are necessary.

[1205] "Means for collecting and analyzing user emotional data using an emotion recognition engine" refers to devices and software for automatically collecting and analyzing emotional data from users' facial expressions, voice, etc.

[1206] "Means for generating suggested revisions to stories and illustrations based on emotional data" refers to a system or algorithm that automatically generates more appropriate suggested revisions to stories and illustrations based on collected user emotional data.

[1207] "Means for regenerating the story and illustrations and providing a digital preview again" refers to a system for reflecting the correction request and emotional data and providing the newly generated story and illustrations again in a digital preview format.

[1208] "Means for generating and delivering the final digital book to the user" refers to a system that generates the final customized book in high resolution and delivers it to the user in an appropriate format (e.g., PDF, EPUB).

[1209] "Means for providing printing and delivery options upon user's request" refers to systems or services that provide users with the option to print and deliver physical picture books upon user's request.

[1210] "Means for providing services to users using tablet devices or dedicated kiosk terminals in physical stores" refers to devices and systems that allow users to directly experience creating customized picture books using tablet devices or dedicated kiosk terminals in physical stores.

[1211] Specific embodiments of the present invention will be described below.

[1212] Users access the "Detailed Description of the Invention" using a tablet terminal or dedicated kiosk terminal installed in a physical store. The terminal displays an input form to the user via a web browser. The user enters the child's name, age, gender, and interests and clicks the submit button. This sends the input data from the terminal to the server.

[1213] The server is a high-performance cloud server (e.g., AWS EC2, Google Cloud Platform) that performs the following processes: First, a generative AI model (e.g., GPT-4, BERT) is used to generate a customized story based on user input data. Next, an image generation engine (e.g., DALL-E, Stable Diffusion) is used to generate illustrations appropriate for each scene in the generated story. The generated story and illustrations are combined to create a preview of the digital picture book. The preview version is created in an interactive format (e.g., HTML, PDF), and a URL is generated. The server sends this URL to the device, and the user clicks the link to view the preview version.

[1214] An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to analyze the facial expressions and voice of the user while previewing the digital picture book to recognize their emotions. This emotional data is sent to the server. The server then automatically generates revision suggestions based on the collected emotional data. For example, if an expression indicating "not looking happy" is recognized, the system will suggest "add more dinosaur scenes." The user can review the revision suggestions and, if necessary, enter their revision requests in the input field.

[1215] The server receives the revision request and the emotion-based revision suggestions, re-runs the generative AI model and image generation engine to regenerate the picture book reflecting the revisions. A preview of the regenerated digital picture book is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[1216] Finally, once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g., PDF, EPUB). It then sends a URL for the final version to the user's device, which the user can use to download the digital book. At the same time, it provides a print and delivery option if the user wishes, and the server sends a print and delivery request to the printing service provider.

[1217] For example, if a user inputs "Taro," "6 years old," "male," "dinosaur, adventure" into a tablet device, the server generates a story about Taro meeting a dinosaur and going on a great adventure, and an image generation engine creates illustrations based on the story. A preview of the digital picture book is then generated for the user to review. If the emotion recognition engine determines that the book "doesn't look fun," the system suggests adding more dinosaur scenes. This process is repeated until the user is satisfied, and the final version is generated.

[1218] An example of a prompt for a generative AI model is:

[1219] Child information provided by the user:

[1220] Name: Taro

[1221] Age: 6

[1222] Gender: Male

[1223] Interests: Dinosaurs, adventure

[1224] prompt:

[1225] Create a story about a 6-year-old boy named Taro who meets a dinosaur and goes on a great adventure. Include scene descriptions for each scene to illustrate.

[1226] This allows users to enjoy the experience of creating a customized picture book in real time in a physical store.

[1227] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1228] Step 1:

[1229] The user accesses the input form through a web browser using a tablet device or dedicated kiosk device in a physical store. The user enters the child's name, age, gender, and interests, and clicks the submit button. The input, such as name (e.g., "Taro"), age (e.g., "6 years old"), gender (e.g., "male"), and interests (e.g., "dinosaurs," "adventure"), is sent to the server. This allows the user's information to be collected.

[1230] Step 2:

[1231] The server receives the data sent by the device. The server stores the received user input data. User information is passed to the server as input, and customized data is prepared based on this information.

[1232] Step 3:

[1233] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the input data. The server passes a prompt to the generative AI model, which then generates a story based on it. Specifically, the name, age, gender, and interests entered by the user are input as prompts into the generative AI model, and a customized story is returned.

[1234] Step 4:

[1235] The server uses an image generation engine (e.g., DALL-E) to create illustrations appropriate for each scene in the generated story. The server receives the story scenes as input and generates illustrations corresponding to each scene as output. These illustrations are then combined with the story.

[1236] Step 5:

[1237] The server combines the generated story and illustrations to create a preview of the digital picture book. The preview is generated in an interactive format (e.g., HTML, PDF), and a URL is created. The server then sends this URL to the device, where the preview is provided to the user.

[1238] Step 6:

[1239] The user checks a preview of the digital picture book on their device. An emotion recognition engine (e.g., Microsoft Azure Emotion API) analyzes the user's facial expressions and voice to recognize their emotions. The input is collected by the camera and microphone, and emotional data is generated as output.

[1240] Step 7:

[1241] The server automatically generates revision suggestions based on the emotion data. For example, if the user's facial expression indicates that they are not having fun, the server will suggest revisions such as adding more dinosaur scenes. The revision suggestions are provided to the user, who then confirms them.

[1242] Step 8:

[1243] The user checks the suggested revisions and, if necessary, enters a revision request into the input field on the device. The server receives this revision request and again runs the generative AI model and image generation engine to revise the story and illustration. The revision request is passed as input, and the revised story and illustration are generated as output.

[1244] Step 9:

[1245] The server then provides the user with a regenerated preview of the digital book that reflects the modifications. The user then checks the preview again and repeats this process until satisfied. This results in customization that increases user satisfaction.

[1246] Step 10:

[1247] Once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g. PDF, EPUB). A URL for the final version is sent to the device, where the user can use this link to download the digital book. If desired, the user selects the print and delivery option, and the server sends a print and delivery request to the printing service provider.

[1248] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1249] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1250] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1251] [Fourth embodiment]

[1252] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1253] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1254] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1255] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1256] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1257] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1258] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1259] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1260] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1261] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1262] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1263] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1264] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1265] Specific embodiments for carrying out the present invention will be described below.

[1266] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[1267] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[1268] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version, enter correction requests if necessary, and resubmit it.

[1269] The server receives revision requests from the user and recreates the story and illustrations based on those requests. Finally, once the user is satisfied, the final digital book is generated. This final book is then converted to the appropriate format (PDF, EPUB, etc.) in high resolution.

[1270] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[1271] Below is an example of this system:

[1272] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[1273] 2. The terminal sends the entered data to the server.

[1274] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[1275] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[1276] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[1277] 6. The user checks the preview and submits a revision request to add more dinosaur scenes.

[1278] 7. The server processes the requested modifications and provides the regenerated preview to the user.

[1279] 8. The user is happy with the final version and selects download, print and delivery options.

[1280] 9. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[1281] 10. The printing service will deliver the printed picture book to the user's address.

[1282] Through this process, users can create a customized picture book that matches their child's interests and preferences, creating a fun reading experience.

[1283] The processing flow will be explained below.

[1284] Step 1:

[1285] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[1286] Step 2:

[1287] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[1288] Step 3:

[1289] The server receives the HTTP request and analyzes the input data. For example, it analyzes information such as "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[1290] Step 4:

[1291] The server passes the input data to a generative AI model that generates a customized story. Based on the user's input, the generative AI model creates a story such as "Taro meets a dinosaur and goes on a great adventure."

[1292] Step 5:

[1293] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page, such as the scene where Taro meets a dinosaur.

[1294] Step 6:

[1295] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[1296] Step 7:

[1297] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[1298] Step 8:

[1299] The user checks the preview and, if necessary, enters a request for corrections, such as "I want more dinosaur scenes," and resubmits the video.

[1300] Step 9:

[1301] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[1302] Step 10:

[1303] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[1304] Step 11:

[1305] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[1306] Step 12:

[1307] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[1308] Step 13:

[1309] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[1310] Step 14:

[1311] The printing service company will deliver the printed picture book to the address specified by the user.

[1312] Example 1

[1313] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] Existing picture book generation systems have difficulty efficiently generating customized picture books based on user input. Furthermore, the quality of story and illustration generation is not consistent, and modifications to meet user requests are not easily made. This makes it necessary to solve the problem of being unable to quickly and accurately provide customized picture books that satisfy users.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1316] In this invention, the server includes means for receiving information input by a user, means for generating prompt sentences for a generative AI model based on the received information, means for generating a customized story using the generated prompt sentences, means for generating illustrations using an image generation engine based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send correction requests as needed, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to efficiently generate high-quality customized picture books based on user input and quickly accommodate corrections requested by the user.

[1317] "User" refers to a person who accesses the system and inputs information.

[1318] "Information to be entered" refers to data such as the child's name, age, gender, and interests.

[1319] "Means of receiving" refers to the mechanism by which the server obtains the information entered by the user.

[1320] "Generative AI model" refers to an artificial intelligence model that generates customized stories based on user input.

[1321] A "prompt" refers to an instruction used to generate a story for a generative AI model.

[1322] "Customized Story" refers to a bespoke narrative generated based on user input.

[1323] "Image generation engine" refers to a system for automatically generating illustrations based on a customized story.

[1324] "Illustration" refers to an image that corresponds to a customized story.

[1325] "Digital Preview" refers to a digital version of a picture book that combines a generated story and illustrations for advance viewing.

[1326] "Correction Request" refers to a request for changes or additions made by a user after reviewing a digital preview.

[1327] A "final digital picture book" refers to a completed digital picture book that has been approved by the user.

[1328] "Print and Delivery Options" means the option to have the final digital book printed as a physical book and delivered to a specified address.

[1329] The "system" refers to the overall mechanism that combines these means to provide users with customized picture books.

[1330] The present invention relates to a system for generating a customized picture book based on information input by a user. Specific embodiments for carrying out the present invention will be described below.

[1331] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[1332] The server analyzes the user's input data received from the device and generates a customized story based on it. Specifically, it uses a generative AI model (e.g., GPT-4) to convert the user's input data into a prompt. For example, this prompt might say, "Please create a story about a 6-year-old boy named Taro who goes on a great adventure with dinosaurs."

[1333] A customized story is created based on the prompts generated by the generative AI model. The server then uses an image generation engine (e.g., DALL-E) to generate illustrations appropriate for each page of the story. For example, the illustration for "Taro's first encounter with a dinosaur" is generated by this engine.

[1334] The server combines the generated story and illustrations to create a preview version of the digital picture book. This preview version is created in an interactive format (e.g., HTML, PDF, etc.) and can be viewed by the user in a web browser. The user can check the preview version and, if necessary, enter correction requests and resubmit it.

[1335] The server receives revision requests from the user and regenerates the story and illustrations based on those requests. For example, if a user requests "more dinosaur scenes," the server regenerates new stories and illustrations to reflect that request. Finally, once the user is satisfied, the final digital picture book is generated. The final picture book is then converted to a suitable format (e.g., PDF, EPUB, etc.) at high resolution.

[1336] Users can download the final digital picture book and, if desired, select the print and delivery option. If the user selects the print and delivery option, the server receives the user's delivery information and sends a request to a partner printing service company based on that information. The printing service company then delivers the printed picture book to the address specified by the user.

[1337] As a result, the present invention can efficiently generate a high-quality customized picture book based on user input, and can also quickly accommodate modifications in response to user requests.

[1338] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1339] Step 1:

[1340] The user opens a web browser, accesses the system's URL, enters the child's name, age, gender, and interest (e.g., dinosaurs, adventure) in the form that appears, and clicks the submit button.

[1341] Input: Child's name, age, gender, and interests.

[1342] Output: The input data in JSON format.

[1343] Step 2:

[1344] The terminal sends the input data to the server. Specifically, it converts the input data into JSON format using JavaScript and sends it to the server using an HTTP POST request.

[1345] Input: Input data in JSON format.

[1346] Output: The input data sent to the server.

[1347] Step 3:

[1348] The server analyzes the received input data and generates prompts for the generative AI model. This processing is done using the Python Flask framework.

[1349] Input: The input data sent to the server in JSON format.

[1350] Output: A prompt to the generative AI model.

[1351] Step 4:

[1352] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the prompt. Here, data generation calculations are performed based on the prompt. The generated story is stored in a database (e.g., PostgreSQL).

[1353] Input: A prompt to the generative AI model.

[1354] Output: A customized story.

[1355] Step 5:

[1356] The server analyzes the generated story and generates illustrations appropriate for each scene using an image generation engine (e.g., DALL-E). The generated illustrations are temporarily stored in a database.

[1357] Input: Your customized story.

[1358] Output: The generated illustration.

[1359] Step 6:

[1360] The server combines the generated story and illustrations to create a digital preview in HTML or PDF format, which is stored on the server.

[1361] Input: Generated story and illustrations.

[1362] Output: Digital preview (HTML, PDF formats).

[1363] Step 7:

[1364] The user uses a web browser to check the digital preview, enter correction requests if necessary, and submit it again.

[1365] Input: Digital preview and revision request.

[1366] Output: The correction request.

[1367] Step 8:

[1368] The server receives the modification request and regenerates the story and illustrations based on the request, which are also stored in the database.

[1369] Input: Correction request.

[1370] Output: Regenerated story and illustrations.

[1371] Step 9:

[1372] The server generates the final digital picture book by converting it into a suitable high-resolution format (e.g. PDF, EPUB, etc.) and stores the generated final version on the server.

[1373] Input: The final story and illustrations, as reviewed by the user.

[1374] Output: Final digital picture book (PDF, EPUB format).

[1375] Step 10:

[1376] The user downloads the final digital book and selects printing and delivery options if desired. If selected, the server receives the user's delivery information.

[1377] Input: User downloads and prints; selects shipping options; shipping information.

[1378] Output: Download link and shipping information.

[1379] Step 11:

[1380] The server sends a delivery request to a partner printing service company.

[1381] Input: Shipping information.

[1382] Output: A delivery request to a printing service company.

[1383] Step 12:

[1384] The printing service will deliver the printed picture book to the address specified by the user. The delivery status will also be updated and notified to the server.

[1385] Input: Delivery request.

[1386] Output: Delivery completed to user.

[1387] Through this process, users can create a customized storybook tailored to their child's interests and preferences, which can then be downloaded, printed, and delivered.

[1388] (Application example 1)

[1389] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] Conventional picture book creation services have had problems such as difficulty in customizing to individual users and lack of flexibility in previewing and modifying the generated picture books. In particular, it has been felt to be difficult to easily create digital picture books customized according to children's interests and preferences, and to easily view and share them on smart devices. The objective of this invention is to solve these problems and provide a system that enables users to easily create, modify, view, and share customized picture books.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1392] In this invention, the server includes means for receiving information entered by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the user to check the digital preview and send a request for corrections as needed, means for generating a final digital picture book and providing it to the user, means for providing printing and delivery options if desired by the user, means for making the generated digital picture book viewable on a smart device, means for making the digital picture book downloadable in high resolution, and means for enabling online sharing. This allows users to easily create customized picture books, preview them on the spot, make any necessary corrections, and then view or share them on their smart devices.

[1393] A "user" is a person who accesses the system, inputs information, and uses the generated digital picture book.

[1394] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides to the system.

[1395] A "customized story" is a narrative tailored to the individual user's needs, generated based on information entered by the user.

[1396] "Illustrations" are pictures or images generated based on a customized story.

[1397] "Digital Preview" is a digital picture book that combines story and illustrations and allows users to view it interactively.

[1398] A "request for correction" is a request made by a user to request corrections to content after checking the digital preview.

[1399] The "final digital picture book" is the final, high-resolution digital picture book that incorporates user correction requests.

[1400] "Print and Delivery Option" means an option that allows a user to physically print the final digital book and have it delivered to an address of their choosing.

[1401] "Smart devices" are electronic devices that can connect to the Internet, such as smartphones, smart glasses, and head-mounted displays.

[1402] "High resolution" refers to the quality of visual content, such as images and documents, that can be displayed with clear detail.

[1403] "Sharing online" refers to the ability to share the generated digital picture book with other users via the Internet.

[1404] Specific embodiments for carrying out the present invention will be described below.

[1405] Users access the system using a smartphone, personal computer, etc. The terminal displays an input form and allows the user to enter information such as the child's name, age, gender, and interests. Once the user has completed the input and clicked the submit button, the terminal sends this data to the server.

[1406] The server analyzes the received information and generates a customized story using a generative AI model (e.g., GPT-4). For example, if a user enters information such as "6 years old, boy, named Taro, loves dinosaurs," the server generates "a story about Taro's adventures with dinosaurs." The prompt used in this case is something like: "6-year-old boy, named Taro, loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[1407] Based on the generated story, the server uses an image generation engine (e.g., DALL-E) to generate illustrations. The image generation engine generates illustrations appropriate for each scene in the story, and combines the story and illustrations to create a digital preview.

[1408] Users can review this digital preview and submit revision requests as needed, which are then sent to the server, which then uses the generative AI model and image generation engine to update the preview and serve it back to the user, allowing them to iteratively customize the final version until they are satisfied with it.

[1409] Once the final digital book is complete, the server generates a high-resolution digital book and provides the user with a download link. If the user prefers, the server can also provide a printing and delivery option, allowing an external printing service to deliver the book to the user's specified address.

[1410] Furthermore, the generated digital picture books can be viewed on smart devices (e.g., smartphones, smart glasses, head-mounted displays) and can also be shared with other users online. This allows users to easily share their individually customized picture books with others and provides a fun reading experience for children.

[1411] To illustrate, here's an example prompt:

[1412] "I have a 6-year-old boy named Taro who loves dinosaurs. Please create a story about Taro's adventures with dinosaurs."

[1413] The system effectively utilizes generative AI models and image generation engines to provide users with flexible customization and high-quality digital content.

[1414] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1415] Step 1:

[1416] A user accesses the system using a terminal and enters information such as the child's name, age, gender, and interests into the input form. After completing the input and clicking the submit button, the terminal sends this data to the server. Input is the various pieces of information provided by the user, and output is the data sent to the server.

[1417] Step 2:

[1418] The server analyzes the information received from the user. Here, it generates a prompt sentence appropriate for the generative AI model based on input data, particularly the child's name, age, gender, and topics of interest. The input is the data provided by the user, and the output is the generated prompt sentence. Specifically, it uses the data entered by the user to generate a prompt sentence such as, "A 6-year-old boy named Taro who loves dinosaurs. Please write a story about Taro's adventures with dinosaurs."

[1419] Step 3:

[1420] The server sends the generated prompt to a generative AI model (e.g., GPT-4) to generate a customized story. The generative AI model generates an optimal story based on the prompt and returns it to the server. The input is the prompt, and the output is the generated story. Specifically, the AI ​​model analyzes the prompt and generates a story such as "A story about Taro meeting a dinosaur and going on a great adventure."

[1421] Step 4:

[1422] The server receives the generated story and uses an image generation engine (e.g., DALL-E) to generate illustrations for each scene of the story as needed. The input is the generated story, and the output is illustrations corresponding to each scene. Specifically, it generates illustrations appropriate for each scene, such as "the scene where Taro meets a dinosaur for the first time."

[1423] Step 5:

[1424] The server combines the generated story and illustrations to create a digital preview. The input is a story-illustration pair, and the output is a digital preview. Specifically, the server combines the story and illustration in HTML or PDF format to create a preview version that users can check interactively.

[1425] Step 6:

[1426] The user checks the digital preview and sends a correction request if necessary. The server receives this correction request and updates the preview content again using the generative AI model and image generation engine. The input is the correction request from the user, and the output is the corrected digital preview. Specifically, based on the user request to "add more dinosaur scenes," the story and illustrations are regenerated.

[1427] Step 7:

[1428] Once the user is satisfied with the final version of the digital book, the server generates the final version and provides it to the user. The input is the revised preview, and the output is the final digital book. Specifically, it converts it to PDF or EPUB format at high resolution and provides a download link to the user.

[1429] Step 8:

[1430] The server provides printing and delivery options to the user if desired. The input is the user's delivery information, and the output is a request to the printing service company. Specifically, the server sends the necessary information to an external printing service company to deliver the picture book to the address specified by the user.

[1431] To ensure a smooth process, the document can also be viewed on smart devices and shared online.

[1432] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1433] Specific embodiments for carrying out the present invention will be described below.

[1434] When a user accesses the system using a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interest (e.g., animals, adventure, princesses, etc.) and clicks the submit button. The terminal then sends the input data to the server.

[1435] The server analyzes the received user input data and generates a customized story based on it. A generative AI model handles this process, creating a personalized story based on the user's input, such as "A story about a country girl who meets a prince and marries him." The server then uses an image generation engine to generate illustrations appropriate for each page based on the generated story. For example, an illustration of "a scene where a girl meets a prince" is generated based on a scene depicted in the story.

[1436] The server combines the generated story and illustrations to create a preview version of the digital picture book, which is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[1437] The server sends an HTTP response containing the URL of the preview version to the device, and the user clicks on the link to view the preview. At this stage, the system is equipped with an emotion engine that recognizes emotions by analyzing the user's facial expressions and voice. The emotion engine analyzes the user's emotions and determines their feelings, such as "They seem to be having fun" or "They seem not to be interested."

[1438] The user checks the preview and, if any particular corrections are needed, enters the correction request in the input field. At this time, the server generates automatic correction suggestions for the story or illustrations, taking into account the user's emotions recognized by the emotion engine. For example, if the user expresses an expression that says "it doesn't look like he's having fun," the system can suggest corrections such as "add more dinosaur scenes." The user can review the correction suggestions and implement them as necessary.

[1439] The server receives the revision request and the revision suggestions from the emotion engine, then re-runs the generative AI model and image generation engine to generate a picture book that reflects the revisions. The regenerated preview version is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[1440] Finally, once the user is satisfied with the final version, the server generates the final high-resolution digital book and converts it into the appropriate format (PDF, EPUB, etc.). It then sends the URL of the final version to the user's device, allowing them to download it using the download link. At the same time, if the user wishes, it provides a print and delivery option, and the server sends a print and delivery request to the printing service provider.

[1441] Below is an example of this system:

[1442] 1. A user enters information about their child into a web form, such as name "Taro," age "6 years old," gender "male," and interests "dinosaurs and adventure."

[1443] 2. The terminal sends the entered data to the server.

[1444] 3. The server generates a story about Taro meeting a dinosaur and going on a great adventure.

[1445] 4. The server generates an illustration such as "Taro's first encounter with a dinosaur."

[1446] 5. The server combines the story and illustrations to create a digital preview and provides it to the user.

[1447] 6. The user checks the preview. The emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1448] 7. The user reviews the sentiment-based revision suggestions and submits a revision request to include more dinosaur scenes.

[1449] 8. The server reflects the modification request and the emotion engine's suggestions and provides the regenerated preview to the user.

[1450] 9. The user is happy with the final version and selects download, print and delivery options.

[1451] 10. The server generates the final version and provides a download link to the user, while also sending a delivery request to the printing service provider.

[1452] 11. The printing service will deliver the printed picture book to the user's address.

[1453] Through this process, users can create a customized picture book that matches their child's interests, and by using an emotion engine, they can create a more personalized and enjoyable reading experience.

[1454] The processing flow will be explained below.

[1455] Step 1:

[1456] The user accesses the system using a web browser, and the device displays a web page with an input form for entering the child's name, age, gender, and interests.

[1457] Step 2:

[1458] The user enters the required information and clicks the submit button. The device converts the input data into JSON format and sends it to the server using an HTTP request.

[1459] Step 3:

[1460] The server receives the HTTP request and analyzes the input data. The analysis results include the following information: "Name: Taro, Age: 6, Gender: Male, Interests: Dinosaurs, Adventure."

[1461] Step 4:

[1462] The server passes the input data to the generative AI model, which then generates a customized story. Based on the user's input, the generative AI model creates a story about Taro meeting a dinosaur and going on a great adventure.

[1463] Step 5:

[1464] The server receives the generated story and passes it to an image generation engine to generate illustrations for each page. For example, it generates illustrations for scenes such as "Taro meets a dinosaur" and "Taro goes on an adventure with a dinosaur."

[1465] Step 6:

[1466] The server combines the generated story and illustrations to create a preview version of the digital picture book. The preview version is created in an interactive format (HTML, PDF, etc.) and a URL is generated.

[1467] Step 7:

[1468] The server sends an HTTP response containing a URL for the preview version to the device, and the user clicks the link to view the preview version.

[1469] Step 8:

[1470] The user checks the preview. The device uses the camera and microphone to capture the user's facial expressions and voice, which are then analyzed by the emotion engine. For example, if the user is smiling, it will be determined that they look happy.

[1471] Step 9:

[1472] The device sends the analysis results from the emotion engine to the server, which receives the results and generates automatic suggestions for revising the story or illustrations based on emotional information such as "it looks fun."

[1473] Step 10:

[1474] The user checks the proposed revisions and makes any necessary changes. For example, the user enters a request such as "I want more dinosaur scenes" and clicks the submit button.

[1475] Step 11:

[1476] The device sends the correction request data to the server, which receives the correction request and re-executes the generative AI model and image generation engine to generate a picture book that reflects the corrections.

[1477] Step 12:

[1478] The server sends the regenerated preview to the device, where the user can review it again and request the final version to be generated.

[1479] Step 13:

[1480] The server generates the final digital book in high resolution and converts it into the appropriate format (PDF, EPUB, etc.), then sends the URL of the final version to the device and provides a download link.

[1481] Step 14:

[1482] The user clicks the download link to download the final digital book. If the user selects the print and delivery option, they enter their delivery information, such as their address, and submit.

[1483] Step 15:

[1484] The terminal sends delivery information to the server, which receives the delivery information and sends a print and delivery request to a partner printing service company.

[1485] Step 16:

[1486] The printing service company will deliver the printed picture book to the address specified by the user.

[1487] Example 2

[1488] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1489] While conventional customized picture book creation systems can generate stories based on information entered by users, they lack the ability to modify the story and illustrations taking into account the user's emotional responses. As a result, if a user is not satisfied with the content of the picture book, they must provide detailed correction instructions again, which is time-consuming. The present invention aims to solve these problems and efficiently generate more personalized customized picture books.

[1490] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1491] In this invention, the server includes means for receiving information input by a user, means for generating a customized story based on the received information, means for generating illustrations based on the generated story, means for generating a digital preview of a picture book by combining the story and illustrations, means for the server to provide the user with a link to the generated digital preview, means for the user to check the digital preview and send a correction request if necessary, means for analyzing the user's emotions using an emotion engine and generating correction suggestions based on the analysis results, means for regenerating the story and illustrations by reflecting the correction requests and the emotion analysis results, means for generating a final digital picture book and providing it to the user, and means for providing printing and delivery options if the user desires. This makes it possible to incorporate feedback based on the user's emotions and generate a customized picture book that provides a higher level of satisfaction.

[1492] "User" means an individual or end user who uses the system to create a customized digital picture book.

[1493] "Input information" refers to data such as the child's name, age, gender, and interests that the user provides when using the system.

[1494] "Server" means a central processing unit or server system that receives input from users and generates, modifies, and distributes stories and illustrations based thereon.

[1495] A "customized story" is a narrative that is individually generated based on input information provided by the user.

[1496] A "generative AI model" is an artificial intelligence model that uses natural language processing technology to automatically generate stories from user input.

[1497] An "illustration" is an image created to visually represent a scene from the generated story.

[1498] An "image generation engine" is an image generation algorithm or software for automatically generating illustrations based on a generated story.

[1499] The "Digital Preview" is an early version of the digital picture book that combines the generated story and illustrations, and is in a format that allows users to check and edit it.

[1500] An "emotion engine" is a technology or software that analyzes data such as a user's facial expressions and voice and recognizes their emotional state.

[1501] A "revision request" is a request for changes to the story or illustrations submitted by a user after viewing the digital preview.

[1502] "Revision suggestions" are suggestions for changes to stories or illustrations generated by the server based on the results of an analysis of the user's emotions by the emotion engine.

[1503] The "final digital picture book" is a high-quality digital picture book that is finally generated by reflecting the user's correction requests and suggestions from the emotion engine.

[1504] "Printing and Delivery Option" means an additional service that allows a user to print and deliver the final digital picture book as a physical book if the user so desires.

[1505] The present invention is a system for users to generate customized digital picture books through a process that includes collecting user input data, generating a story using a generative AI model, generating illustrations using an image generation engine, and analyzing user emotions using an emotion engine and making modifications based on the analysis.

[1506] Hardware and software used

[1507] This system uses the following hardware and software:

[1508] Terminal: A device (PC, smartphone, tablet, etc.) that a user uses to access the system using a web browser and enter and confirm information.

[1509] Server: The central processing system that receives user input and generates, modifies, and delivers stories and illustrations.

[1510] Generative AI models: Models that use natural language processing techniques to automatically generate stories (e.g., OpenAI GPT-4).

[1511] Image generation engine: An engine that generates illustrations based on the generated story (e.g., DALL-E).

[1512] Emotion engine: Technology that analyzes the user's facial expressions and voice to recognize emotions (e.g., Affectiva).

[1513] Explanation of program processing

[1514] Collecting user-entered data

[1515] When a user accesses the system through a web browser, the terminal displays an input form. The user enters the child's name, age, gender, and interests, and submits it. For example, enter the following:

[1516] Name: Taro

[1517] Age: 6

[1518] Gender: Male

[1519] Interests: Dinosaurs, adventure

[1520] Sending data to the server

[1521] When the user clicks the submit button, the device sends the input data to the server, which receives and analyzes the data.

[1522] Story Generation

[1523] The server uses a generative AI model to generate a customized story based on the received data, for example using the following prompt:

[1524] "Create a story about the great adventure of a 6-year-old boy named Taro, who begins with an encounter with a dinosaur."

[1525] The generative AI model generates a basic story, for example, "Once upon a time, a six-year-old boy named Taro encountered a dinosaur..."

[1526] Illustration generation

[1527] Based on the generated story, the server uses an image generation engine to generate illustrations appropriate for each scene, for example using the following prompt:

[1528] "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time."

[1529] This prompt will cause the image generation engine to generate an illustration.

[1530] Generate and deliver digital previews

[1531] The server combines the generated story and illustrations to create a preview version of the digital picture book (in HTML or PDF format), and then generates a URL for the preview version and provides it to the user.

[1532] Analyzing user sentiment and reflecting correction requests

[1533] When a user checks the preview version, the emotion engine analyzes the user's facial expressions and voice to recognize their emotional state. For example, it can distinguish emotions such as "interested" or "uninterested." When the user submits an emotion-based revision request, the server generates and implements revision suggestions based on the request and emotion analysis.

[1534] The server re-serves the regenerated story and illustrations as a digital picture book, and the process is repeated until the user is satisfied.

[1535] Final digital book delivery and printing / shipping options

[1536] Once the user is satisfied with the final version, the server generates the final high-resolution digital book, converts it to PDF or EPUB format, provides the user with a URL, and offers printing and delivery options if desired, and sends the request to a printing service, which will then deliver it to the address specified by the user.

[1537] The system allows users to quickly and efficiently generate customized picture books tailored to a child's interests, and by using an emotion engine, provides a more personalized and satisfying reading experience.

[1538] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1539] Step 1:

[1540] The user accesses the system via a web browser and the terminal displays an input form.

[1541] Input: The URL the user visited

[1542] Output: The input form is displayed on the terminal.

[1543] What happens: The user opens a browser and visits a URL (e.g., https: / / example.com). The device displays a user interface and provides a form for entering the child's information.

[1544] Step 2:

[1545] The user enters the child's name, age, gender, and interests and clicks the submit button.

[1546] Input: Name, age, gender, interests

[1547] Output: User input data is sent to the server

[1548] Specific operation: The user enters information such as "Taro," "6 years old," "male," "dinosaur, adventure" into the form and clicks the submit button, which causes the device to send the input data to the server.

[1549] Step 3:

[1550] The server parses the received input data.

[1551] Input: User input data (Taro, 6 years old, male, dinosaur, adventure)

[1552] Output: Parsed data

[1553] Specific operation: The server receives an HTTP POST request and analyzes the user's input data. The analyzed data is used for further processing.

[1554] Step 4:

[1555] The server uses generative AI models to generate customized stories.

[1556] Input: Parsed user data

[1557] Output: Customized story

[1558] What happens: The server sends the following prompt to a generative AI model (e.g., GPT-4): "Write a story about the great adventure of a 6-year-old boy named Taro, who begins with his encounter with a dinosaur." The generative AI model responds with a story: "Once upon a time, a 6-year-old boy named Taro met a dinosaur..."

[1559] Step 5:

[1560] The server uses an image generation engine to generate illustrations based on the generated story.

[1561] Input: Your customized story

[1562] Output: Illustration

[1563] Specific behavior: The server sends the following prompt to the image generation engine (e.g., DALL-E): "Generate an illustration of a scene in which a 6-year-old boy encounters a dinosaur for the first time." The image generation engine generates the illustration and returns it to the server.

[1564] Step 6:

[1565] The server combines the generated story and illustrations to create a preview version of the digital picture book and generates a URL for it.

[1566] Input: Story, illustrations

[1567] Output: URL of the preview version of the digital picture book

[1568] What it does: The server combines the story and illustrations in HTML or PDF format to create a preview version of the digital picture book, and provides the generated URL to the user.

[1569] Step 7:

[1570] The server sends an HTTP response containing the preview URL to the device.

[1571] Input: Preview URL

[1572] Output: HTTP response

[1573] Specific operation: The server sends an HTTP response to the device and notifies the user of the preview URL (e.g., https: / / example.com / preview / taro_story). The user clicks the URL to view the preview.

[1574] Step 8:

[1575] The user checks the preview version, and the emotion engine analyzes the user's facial expressions and voice.

[1576] Input: Preview, facial expressions, and voice data

[1577] Output: Emotion analysis results

[1578] How it works: The user clicks on the preview URL to check the content. The emotion engine uses the camera and microphone to analyze the user's facial expressions and voice, and determines their emotions, such as "They seem to be having fun" or "They seem to be uninterested."

[1579] Step 9:

[1580] The user submits a correction request, and the server makes correction suggestions based on the results of sentiment analysis.

[1581] Input: Correction request, sentiment analysis results

[1582] Output: suggested fixes

[1583] How it works: A user inputs and submits a revision request, such as "I want more dinosaur scenes." The server, taking into account the results of sentiment analysis, generates a revision suggestion, such as "Add more adventurous scenes."

[1584] Step 10:

[1585] The server regenerates the story and illustrations based on the revision requests and suggestions.

[1586] Input: Correction request, correction proposal

[1587] Output: Regenerated story and illustrations

[1588] What it does: The server re-runs the generative AI model and image generation engine to generate new stories and illustrations that reflect the modifications.

[1589] Step 11:

[1590] The server sends an HTTP response containing the URL of the regenerated preview version to the device and provides it to the user.

[1591] Input: Regenerated story and illustrations

[1592] Output: Regenerated preview URL

[1593] Specific behavior: The server provides the newly generated preview URL (e.g., https: / / example.com / preview / taro_story_v2) to the user. The user checks the preview again.

[1594] Step 12:

[1595] Once the user is satisfied with the final version, the server generates the final digital picture book and provides it in PDF or EPUB format.

[1596] Input: User's final confirmation

[1597] Output: Final digital picture book (PDF or EPUB format)

[1598] What happens: Once the user is satisfied with the final version, the server generates a high-resolution digital storybook (e.g., PDF format) and provides a URL (e.g., https: / / example.com / final / taro_story_final.pdf).

[1599] Step 13:

[1600] If the user so desires, the server sends a print and delivery request to the printing service company.

[1601] Input: User's print and delivery request

[1602] Output: Request to printing service company

[1603] Specific operation: When the user selects the print and delivery option, the server sends the data and delivery information to the printing service company. The printed picture book is then delivered to the specified address.

[1604] (Application example 2)

[1605] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1606] Previous systems were unable to consider user emotions when generating customized digital picture books, which could result in lower user satisfaction. Furthermore, because the books were only available online, there was no way to improve user engagement through in-store experiences. This meant that users lacked the flexibility to experience the book on the spot and make suggestions for revisions immediately.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1608] In this invention, the server includes a means for receiving information entered by the user, a means for generating a customized story based on the received information, a means for collecting and analyzing the user's emotional data using an emotion recognition engine, and a means for generating suggested revisions to the story and illustrations. This makes it possible to generate a customized picture book that takes the user's emotions into consideration, and by providing this service in a physical store, it becomes possible to provide an interactive experience on the spot.

[1609] "Means for receiving information entered by the user" refers to devices or software that collect information entered by the user, such as name, age, gender, and interests, and transmits it to the server.

[1610] "Means for generating a customized story based on received information" refers to a generative AI model or algorithm that creates an individualized customized story based on information received from a user.

[1611] "Means for generating illustrations based on the generated story" refers to an image generation engine or tool for creating appropriate illustrations according to the content of the generated story.

[1612] "Means for combining story and illustrations to generate a digital preview of a picture book" refers to software or a system that combines a customized story with generated illustrations and provides them in a user-viewable digital preview format (e.g., HTML, PDF).

[1613] "Means for users to check the digital preview and send correction requests if necessary" refers to an interface that allows users to check the preview of the generated digital picture book and send correction requests to the server if corrections are necessary.

[1614] "Means for collecting and analyzing user emotional data using an emotion recognition engine" refers to devices and software for automatically collecting and analyzing emotional data from users' facial expressions, voice, etc.

[1615] "Means for generating suggested revisions to stories and illustrations based on emotional data" refers to a system or algorithm that automatically generates more appropriate suggested revisions to stories and illustrations based on collected user emotional data.

[1616] "Means for regenerating the story and illustrations and providing a digital preview again" refers to a system for reflecting the correction request and emotional data and providing the newly generated story and illustrations again in a digital preview format.

[1617] "Means for generating and delivering the final digital book to the user" refers to a system that generates the final customized book in high resolution and delivers it to the user in an appropriate format (e.g., PDF, EPUB).

[1618] "Means for providing printing and delivery options upon user's request" refers to systems or services that provide users with the option to print and deliver physical picture books upon user's request.

[1619] "Means for providing services to users using tablet devices or dedicated kiosk terminals in physical stores" refers to devices and systems that allow users to directly experience creating customized picture books using tablet devices or dedicated kiosk terminals in physical stores.

[1620] Specific embodiments of the present invention will be described below.

[1621] Users access the "Detailed Description of the Invention" using a tablet terminal or dedicated kiosk terminal installed in a physical store. The terminal displays an input form to the user via a web browser. The user enters the child's name, age, gender, and interests and clicks the submit button. This sends the input data from the terminal to the server.

[1622] The server is a high-performance cloud server (e.g., AWS EC2, Google Cloud Platform) that performs the following processes: First, a generative AI model (e.g., GPT-4, BERT) is used to generate a customized story based on user input data. Next, an image generation engine (e.g., DALL-E, Stable Diffusion) is used to generate illustrations appropriate for each scene in the generated story. The generated story and illustrations are combined to create a preview of the digital picture book. The preview version is created in an interactive format (e.g., HTML, PDF), and a URL is generated. The server sends this URL to the device, and the user clicks the link to view the preview version.

[1623] An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to analyze the facial expressions and voice of the user while previewing the digital picture book to recognize their emotions. This emotional data is sent to the server. The server then automatically generates revision suggestions based on the collected emotional data. For example, if an expression indicating "not looking happy" is recognized, the system will suggest "add more dinosaur scenes." The user can review the revision suggestions and, if necessary, enter their revision requests in the input field.

[1624] The server receives the revision request and the emotion-based revision suggestions, re-runs the generative AI model and image generation engine to regenerate the picture book reflecting the revisions. A preview of the regenerated digital picture book is provided to the user for further confirmation. This process is repeated until the user is satisfied.

[1625] Finally, once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g., PDF, EPUB). It then sends a URL for the final version to the user's device, which the user can use to download the digital book. At the same time, it provides a print and delivery option if the user wishes, and the server sends a print and delivery request to the printing service provider.

[1626] For example, if a user inputs "Taro," "6 years old," "male," "dinosaur, adventure" into a tablet device, the server generates a story about Taro meeting a dinosaur and going on a great adventure, and an image generation engine creates illustrations based on the story. A preview of the digital picture book is then generated for the user to review. If the emotion recognition engine determines that the book "doesn't look fun," the system suggests adding more dinosaur scenes. This process is repeated until the user is satisfied, and the final version is generated.

[1627] An example of a prompt for a generative AI model is:

[1628] Child information provided by the user:

[1629] Name: Taro

[1630] Age: 6

[1631] Gender: Male

[1632] Interests: Dinosaurs, adventure

[1633] prompt:

[1634] Create a story about a 6-year-old boy named Taro who meets a dinosaur and goes on a great adventure. Include scene descriptions for each scene to illustrate.

[1635] This allows users to enjoy the experience of creating a customized picture book in real time in a physical store.

[1636] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1637] Step 1:

[1638] The user accesses the input form through a web browser using a tablet device or dedicated kiosk device in a physical store. The user enters the child's name, age, gender, and interests, and clicks the submit button. The input, such as name (e.g., "Taro"), age (e.g., "6 years old"), gender (e.g., "male"), and interests (e.g., "dinosaurs," "adventure"), is sent to the server. This allows the user's information to be collected.

[1639] Step 2:

[1640] The server receives the data sent by the device. The server stores the received user input data. User information is passed to the server as input, and customized data is prepared based on this information.

[1641] Step 3:

[1642] The server uses a generative AI model (e.g., GPT-4) to generate a customized story based on the input data. The server passes a prompt to the generative AI model, which then generates a story based on it. Specifically, the name, age, gender, and interests entered by the user are input as prompts into the generative AI model, and a customized story is returned.

[1643] Step 4:

[1644] The server uses an image generation engine (e.g., DALL-E) to create illustrations appropriate for each scene in the generated story. The server receives the story scenes as input and generates illustrations corresponding to each scene as output. These illustrations are then combined with the story.

[1645] Step 5:

[1646] The server combines the generated story and illustrations to create a preview of the digital picture book. The preview is generated in an interactive format (e.g., HTML, PDF), and a URL is created. The server then sends this URL to the device, where the preview is provided to the user.

[1647] Step 6:

[1648] The user checks a preview of the digital picture book on their device. An emotion recognition engine (e.g., Microsoft Azure Emotion API) analyzes the user's facial expressions and voice to recognize their emotions. The input is collected by the camera and microphone, and emotional data is generated as output.

[1649] Step 7:

[1650] The server automatically generates revision suggestions based on the emotion data. For example, if the user's facial expression indicates that they are not having fun, the server will suggest revisions such as adding more dinosaur scenes. The revision suggestions are provided to the user, who then confirms them.

[1651] Step 8:

[1652] The user checks the suggested revisions and, if necessary, enters a revision request into the input field on the device. The server receives this revision request and again runs the generative AI model and image generation engine to revise the story and illustration. The revision request is passed as input, and the revised story and illustration are generated as output.

[1653] Step 9:

[1654] The server then provides the user with a regenerated preview of the digital book that reflects the modifications. The user then checks the preview again and repeats this process until satisfied. This results in customization that increases user satisfaction.

[1655] Step 10:

[1656] Once the user is satisfied with the final version, the server generates a high-resolution final digital book and converts it into the appropriate format (e.g. PDF, EPUB). A URL for the final version is sent to the device, where the user can use this link to download the digital book. If desired, the user selects the print and delivery option, and the server sends a print and delivery request to the printing service provider.

[1657] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1658] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1659] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1660] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1661] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1662] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1663] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1664] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1665] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1666] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1667] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1668] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1669] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1670] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1671] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1672] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1673] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1674] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1675] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1676] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1677] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1678] The following is further disclosed regarding the above embodiment.

[1679] (Claim 1)

[1680] means for receiving user input information;

[1681] means for generating a customized story based on the received information;

[1682] a means for generating illustrations based on the generated story;

[1683] a means for combining the story and illustrations to generate a digital preview of the picture book;

[1684] A means for users to review the digital preview and submit correction requests if necessary;

[1685] A means for generating and providing the final digital picture book to the user;

[1686] A means to provide printing and shipping options if desired by the user;

[1687] A system including:

[1688] (Claim 2)

[1689] 10. The system of claim 1, wherein the means for generating the customized story generates the customized story using a generative AI model.

[1690] (Claim 3)

[1691] 2. The system of claim 1, wherein the means for generating an illustration uses an image generation engine.

[1692] "Example 1"

[1693] (Claim 1)

[1694] means for receiving user input information;

[1695] means for generating a prompt sentence for the generative AI model based on the received information;

[1696] a means for generating a customized story using the generated prompt sentences;

[1697] a means for generating illustrations using an image generation engine based on the generated story;

[1698] a means for combining the story and illustrations to generate a digital preview of the picture book;

[1699] A means for users to review the digital preview and submit correction requests if necessary;

[1700] A means for generating and providing the final digital picture book to the user;

[1701] A means to provide printing and shipping options if desired by the user;

[1702] A system including:

[1703] (Claim 2)

[1704] 10. The system of claim 1, wherein a generative AI model is used to generate a customized story using the generated prompt sentences.

[1705] (Claim 3)

[1706] 10. The system of claim 1, wherein an image generation engine is used to generate illustrations based on the generated story.

[1707] "Application Example 1"

[1708] (Claim 1)

[1709] means for receiving user input information;

[1710] means for generating a customized story based on the received information;

[1711] a means for generating illustrations based on the generated story;

[1712] a means for combining the story and illustrations to generate a digital preview of the picture book;

[1713] A means for users to review the digital preview and submit correction requests if necessary;

[1714] A means for generating and providing the final digital picture book to the user;

[1715] A means to provide printing and shipping options if desired by the user;

[1716] A means for making the generated digital picture book viewable on a smart device;

[1717] A means to make digital picture books available for download in high resolution,

[1718] A means to enable online sharing,

[1719] A system including:

[1720] (Claim 2)

[1721] 10. The system of claim 1, wherein the means for generating the customized story generates the customized story using a generative AI model.

[1722] (Claim 3)

[1723] 2. The system of claim 1, wherein the means for generating an illustration uses an image generation engine.

[1724] "Example 2: Combining Emotion Engines"

[1725] (Claim 1)

[1726] means for receiving user input information;

[1727] means for generating a customized story based on the received information;

[1728] a means for generating illustrations based on the generated story;

[1729] a means for combining the story and illustrations to generate a digital preview of the picture book;

[1730] a means for the server to provide a link to the generated digital preview to the user;

[1731] A means for users to review the digital preview and submit correction requests if necessary;

[1732] means for analyzing a user's emotions using an emotion engine and generating correction suggestions based on the analysis results;

[1733] A means for regenerating the story and illustrations based on the revision requests and the results of the sentiment analysis;

[1734] A means for generating and providing the final digital picture book to the user;

[1735] A means to provide printing and shipping options if desired by the user;

[1736] A system including:

[1737] (Claim 2)

[1738] 10. The system of claim 1, wherein the system generates a customized story using a generative AI model.

[1739] (Claim 3)

[1740] The system of claim 1, wherein the illustration is generated using an image generation engine.

[1741] "Application example 2 when combining emotion engines"

[1742] (Claim 1)

[1743] means for receiving user input information;

[1744] means for generating a customized story based on the received information;

[1745] a means for generating illustrations based on the generated story;

[1746] a means for combining the story and illustrations to generate a digital preview of the picture book;

[1747] A means for users to review the digital preview and submit correction requests if necessary;

[1748] A means for collecting and analyzing user emotion data using an emotion recognition engine;

[1749] means for generating suggested revisions to the story and illustrations based on the emotion data;

[1750] A means to recreate the story and illustrations and again provide a digital preview;

[1751] A means for generating and providing the final digital picture book to the user;

[1752] A means to provide printing and shipping options if desired by the user;

[1753] A means for providing services to users using tablet terminals or dedicated kiosks in physical stores;

[1754] A system including:

[1755] (Claim 2)

[1756] 10. The system of claim 1, wherein the means for generating the customized story generates the customized story using a generative AI model.

[1757] (Claim 3)

[1758] 2. The system of claim 1, wherein the means for generating an illustration uses an image generation engine. [Explanation of symbols]

[1759] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving user input information; means for generating a customized story based on the received information; a means for generating illustrations based on the generated story; a means for combining the story and illustrations to generate a digital preview of the picture book; A means for users to review the digital preview and submit correction requests if necessary; A means for generating and providing the final digital picture book to the user; A means to provide printing and shipping options if desired by the user; A system including:

2. 10. The system of claim 1, wherein the means for generating the customized story generates the customized story using a generative AI model.

3. 10. The system of claim 1, wherein the means for generating an illustration uses an image generation engine.

Images