Picture book creation system, picture book creation program and picture book creation method

Abstract

To provide a technology capable of creating a picture book having illustrations and a story suited to the illustrations, using pictures photographed by a user and information input by the user.SOLUTION: A picture book creation system for creating a picture book, comprises: reception means, illustration means, storytelling means, and integration means. The reception means receives user input information including at least one of photographs, texts, and themes. The illustration means uses the user input information to send an instruction to create illustrations to the illustration creation means. The storytelling means uses the user input information to send an instruction to create a story to the story creation means. The integration means integrates the illustrations generated by the illustration creation means and the story generated by the story creation means to create picture book data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a picture book creation system, a picture book creation program, and a picture book creation method. [Background technology]

[0002] Conventionally, there exists a technique for creating a book using photographs.

[0003] For example, Patent Document 1 discloses that "When the server device 1 selects images to create photobook data, it can generate and add comments (or titles) to the selected images. The server device 1 generates comments based on objects and text in the images detected by image analysis, the purpose of the photobook, etc. For example, the server device 1 generates and adds comments to images in which distinctive objects are detected" (0039). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7091890 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology in Patent Document 1 allows users to create a photo book by adding comments to photos taken. However, it is difficult to properly illustrate the photos and generate a story suitable for a picture book from the information entered by the user.

[0006] The present invention has been made in consideration of the problems with the conventional technology described above, and its purpose is to provide a technology that uses photographs taken by a user and information input by the user to generate a picture book containing illustrations and a story that suits the illustrations. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a picture book creation system for creating a picture book, comprising: The system includes a reception unit, an illustration unit, a story unit, and an integration unit, the accepting means accepts user input information including at least one of a photo, a text, and a theme; the illustration generating means transmits an instruction to generate an illustration to an illustration generating means using the user input information; the story generation means transmits a story generation instruction to a story generation means using the user input information; The integration means integrates the illustrations generated by the illustration generation means and the story generated by the story generation means to generate picture book data.

[0008] With this configuration, a picture book can be created using photos taken by the user.

[0009] In a preferred embodiment of the present invention, the illustration generating means is a machine learning model, The narrative generator is a large-scale language model.

[0010] With this configuration, illustrations and stories for picture books can be easily generated.

[0011] In a preferred embodiment of the present invention, the picture book production system further comprises a storage unit, the storage unit stores illustration pattern information, which is information for specifying a direction of the illustration, and story plot information, which is information for specifying a direction of the story; The illustration generating means further transmits the illustration pattern information to the illustration generating means; The story telling means further transmits the story plot information to the story generating means.

[0012] With this configuration, illustrations and stories for a picture book that suit the user's preferences can be created.

[0013] In a preferred embodiment of the present invention, the accepting means accepts one short sentence as the text corresponding to one photograph, the illustration generating means transmits an instruction to generate an illustration using the one photograph to the illustration generating means; the story generation means transmits a story generation instruction to the story generation means using the one short sentence; The integration means integrates the one illustration generated by the illustration generation means and the story generated by the story generation means to generate picture book data.

[0014] By using this structure, it is possible to integrate the story corresponding to the illustrations.

[0015] In a preferred embodiment of the present invention, the storage unit further stores at least one of character number control information for controlling the number of characters in the story, hiragana conversion information for specifying the type of characters in the story to be hiragana, or prohibited character information for specifying words that cannot be used in the story, The story generation means further transmits at least one of the character number control information, the hiragana conversion information, and the prohibition information to the story generation means.

[0016] With this configuration, a story suitable for a picture book can be created.

[0017] In a preferred embodiment of the present invention, the photographs include character setting photographs showing candidates for characters appearing in the story; The storage unit further stores the character setting photograph in association with a character name, which is the name of the character.

[0018] With this configuration, the user does not need to input the names of the characters every time.

[0019] In a preferred embodiment of the invention, said integration is performed by a machine learning model.

[0020] With this configuration, the illustrations and the story can be easily integrated.

[0021] In a preferred embodiment of the present invention, the storage section further stores template information relating to discipline, manners, or rules in association with the theme.

[0022] With this configuration, it is possible to create a picture book about discipline, manners, or rules.

[0023] In a preferred embodiment of the present invention, the receiving means receives a photograph and a short sentence as text, the illustration generating means transmits an instruction to generate an illustration using the photograph to the illustration generating means; the story generation means transmits a story generation instruction to the story generation means using the short sentence; The integration means divides the story generated by the story generation means, integrates the divided story with the illustrations generated by the illustration generation means, and generates picture book data.

[0024] With this configuration, a picture book can be easily created. Effect of the Invention

[0025] According to the present invention, a novel technique for a picture book creation system can be provided by executing a predetermined process using photographs taken by a user and information input by the user. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a block diagram of a picture book creation system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing the hardware configuration of the present embodiment. [Diagram 3] 3 shows an example of a data configuration according to the present embodiment. [Figure 4] 3 shows an example of a data configuration according to the present embodiment. [Diagram 5] FIG. 4 is a process sequence diagram of generating a picture book according to the present embodiment. [Figure 6] 13 is an example of an input screen for user input information for generating a picture book according to the present embodiment. [Figure 7] 13 is an example of an input screen for user input information for generating a picture book according to the present embodiment. [Figure 8] FIG. 1 is a block diagram of a picture book creation system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, a picture book creation system according to an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment described below is an example of the present invention, and the present invention is not limited to the embodiment described below, and various configurations can be adopted.

[0028] In this embodiment, the configurations, operations, etc. of a picture book creation system, a picture book creation device, and a picture book creation program are described, but a similarly configured method, a computer program, and a program recording medium on which the program is recorded also provide similar effects. By using a program recording medium, for example, the program can be installed on a computer. A series of processes according to this embodiment described below is provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or even via a communication line.

[0029] The picture book creation system is composed of a computer device. The computer device has an arithmetic device such as a CPU (Central Processing Unit) and a storage device. The computer device can function as a picture book creation device by executing a picture book creation program stored in the storage device with the arithmetic device. The picture book creation method is realized by processing of the computer device including the picture book creation device.

[0030] In this embodiment, the picture book creation system provides a picture book as an output in response to an input by a user. Examples of the input by the user include, but are not limited to, a photograph, a short sentence, or a theme.

[0031] <Embodiment 1> 1 shows a block diagram of a picture book creation system 1 in the embodiment 1. The picture book creation system 1 includes a picture book creation device 2, a user terminal 3, an illustration creation device 4, and a story creation device 5, and these components are communicatively connected via a communication network NW. Furthermore, the picture book creation device 2 is connected to a storage unit 6 by wire or wirelessly.

[0032] The picture book creation device 2 comprises, as functional components, a receiving means 21 that receives user input information from a user, an illustration means 22 that transmits an instruction to create an illustration, a story means 23 that transmits an instruction to create a story, and an integration means 24 that integrates the illustrations and the story. The arrangement of these functional components is one example, and part of the functional configuration of the picture book creation device 2 may be arranged in one or more devices configured to be able to communicate with the picture book creation device 2.

[0033] The picture book creation device 2 receives user input information from the user terminal 3. The picture book creation device 2 uses the received user input information to send an instruction to create an illustration to the illustration creation device 4. The picture book creation device 2 receives the created illustration from the illustration creation device 4. The picture book creation device 2 uses the received user input information to send an instruction to create a story to the story creation device 5. The picture book creation device 2 receives the created story from the story creation device 5. The picture book creation device 2 integrates the received illustrations and story to create picture book data.

[0034] Specifically, the picture book creation device 2 transmits text and a photo as an instruction to create an illustration to the illustration creation device 4. The picture book creation device 2 generates a prompt using, for example, information associated with an option selected by the user and information input by the user, and transmits the prompt and the photo received from the user to the illustration creation device 4.

[0035] Specifically, the picture book generation device 2 transmits the text to the narrative generation device 5. The picture book generation device 2 generates a prompt using, for example, information associated with the option selected by the user and information input by the user, and transmits the prompt to the narrative generation device 5. The narrative generation device 5 is ChatGPT or the like.

[0036] A user can input or transmit user input information including at least one of a photo, text, and theme via the user terminal 3. The user terminal 3 can transmit character setting photo information including the names of characters appearing in the picture book. The user terminal 3 can transmit user basic information including the user's name, address, telephone number, and the like. The user terminal 3 can display on a screen illustrations generated on a display unit such as a display and scenes from the picture book including the story. Although only one user terminal 3 is shown in FIG. 1, there may be multiple user terminals 3.

[0037] The illustration generating device 4 includes an illustration generating means 41 as a functional component. The illustration generating means 41 generates an illustration using user input information included in an instruction to generate an illustration, and transmits the generated illustration to a source of the instruction to generate the illustration. In this embodiment, the source of the instruction to generate the illustration is the picture book generating device 2, but is not limited to this, and a mode in which the source is the user terminal 3 can also be adopted.

[0038] In this embodiment, the illustration generating means 41 is a machine learning model. User input information received from the picture book production device 2 is illustrated by a machine learning model such as a neural network. The illustration generating means 41 may be a large-scale language model (LLM (Large Language Model)).

[0039] The story generation device 5 includes a story generation means 51 as a functional component. The story generation means 51 generates a story using user input information included in the story generation instruction, and transmits the generated story to the source of the story generation instruction. In this embodiment, the source of the story generation instruction is the picture book generation device 2, but is not limited to this, and a mode in which the source is the user terminal 3 can also be adopted.

[0040] In this embodiment, the story generation means 51 is a large-scale language model. When text is input to the story generation means 51 as a prompt, the story generation means 51 generates a story. For example, when the user input information received from the picture book creation device 2 is one short sentence, the story is generated based on the short sentence. The story generation means 51 may be a machine learning model such as a neural network.

[0041] The storage unit 6 stores illustration pattern information, which is information for encouraging the illustration generating means 41 to determine the direction of the illustration to be generated, story synopsis information, which is information for encouraging the story generating means 51 to determine the direction of the story to be generated, character setting photo information, which is information for setting the characters appearing in the picture book, character number control information, which is information for controlling the number of characters to be suitable for a picture book, hiragana conversion information, which is information for making characters suitable for a picture book, and prohibition information, which is information for not using terms and expressions that are not suitable for a picture book. Furthermore, the storage unit 6 stores template information, which is information for generating a picture book regarding discipline, manners, and rules. The information stored in the storage unit 6 is used as user input information. In addition, a part or all of the data stored in the storage unit 6 may be stored in one or more devices configured to be able to communicate with the picture book generation system 1.

[0042] Fig. 2(a) shows a hardware configuration diagram of the picture book creation device 2. The picture book creation device 2 includes, as its hardware configuration, a control unit 201, a storage unit 202, and a communication unit 203. In this embodiment, the picture book creation device 2 may use a computer device such as a server device or a personal computer. Note that the picture book creation device 2 is not limited to the configuration shown in Fig. 2(a) as long as it is made up of multiple computers and can realize the above-mentioned functional components (21-24) as a whole.

[0043] The control unit 201 is composed of one or more processors such as a CPU, and executes a picture book generation program, an OS (Operating System), and other applications to control the overall processing in the picture book generation device 2. The storage unit 202 is a hard disk drive (HDD), solid state drive (SSD), flash memory, RAM (Random Access Memory), etc., and stores the picture book generation program and various data. The communication unit 203 controls communication with the communication network NW, and realizes data communication with the user terminal 3, the illustration generation device 4, and the story generation device 5.

[0044] The illustration generating device 4 and the story generating device 5 may have the same hardware configuration as the picture book generating device 2. Moreover, the illustration generating device 4 and the story generating device 5 may be implemented in a computer device similar to the picture book generating device 2. Furthermore, the illustration generating means 41 and the story generating means 51 may be realized as functional components of the picture book generating device 2.

[0045] 2(b) shows a hardware configuration diagram of the user terminal 3. The user terminal 3 includes, as its hardware configuration, a control unit 301, a storage unit 302, a communication unit 303, an input unit 304, and a display unit 305. In this embodiment, the user terminal 3 may be a smartphone, a personal computer, a tablet terminal, or the like.

[0046] The control unit 301 is composed of one or more processors such as a CPU, and controls the overall processing in the user terminal 3 by executing an OS and other applications. The storage unit 302 is an HDD, SSD, flash memory, RAM, etc., and stores a browser application and various data. The communication unit 303 controls communication with the communication network NW, and realizes data communication at least with the picture book generation device 2. The input unit 304 is an input interface that accepts input operations by the user, and is composed of a touch panel, a mouse, a keyboard, etc. The display unit 305 is composed of a display that displays the processing results by the control unit 301, etc.

[0047] 3 and 4 show an example of the data structure stored in the storage unit 6. The storage unit 6 stores the data shown in FIGS.

[0048] The reception means 21 receives user input information from the user via the user terminal 3. The user input information is information used to generate picture book data, and includes at least one of a photograph, text, and theme. The photograph may be a photograph with or without people in it. The photograph is a character setting photograph or a scene setting photograph, which will be described later. The text is a sentence or a word. The sentence may be short or long, and may be one or more. The word may be one or more. The theme is related to discipline, manners, or rules. The manners or rules may be general or may relate to a specific facility such as a nursery school or kindergarten.

[0049] The illustration generating means 22 transmits an instruction to generate an illustration to the illustration generating means 41 using the user input information received by the receiving means 21 .

[0050] Specifically, the illustration creation means 22 uses information contained in the illustration pattern information or template information to generate a prompt as an instruction to generate an illustration, and transmits the generated prompt and the photo received from the user to the illustration generation means 41.

[0051] The story generation means 23 transmits a story generation instruction to the story generation means 51 using the user input information accepted by the acceptance means 21 .

[0052] Specifically, the narrative creation means 23 generates a prompt as an instruction to generate a story using information contained in the narrative plot information, character count control information, hiragana conversion information, prohibition information, and template information, and transmits the generated prompt to the narrative generation means 51.

[0053] The integration means 24 integrates the illustrations generated by the illustration generating means 41 and the story generated by the story generating means 51 to generate picture book data.

[0054] The receiving means 21 further receives user basic information from the user via the user terminal 3. The user basic information is basic information about the user, such as the user's name, address, and phone number. The storage unit 6 stores the received user basic information in association with the user ID, as shown in FIG. 3(a).

[0055] The receiving means 21 receives a character setting photo in which a character appearing in a picture book is photographed. The storage unit 6 may store character setting photo information including the character setting photo. By storing the character setting photo information in the storage unit 6 as shown in Fig. 3(b), the illustration means 22 and the storytelling means 23 can use the character setting photo information as user input information.

[0056] By storing the character setting photo in association with the user ID as shown in FIG. 3(b), the user does not need to input the character setting photo multiple times. The storage unit 6 further stores the names of the people appearing in the character setting photo as shown in FIG. 3(b). When multiple people appear in the character setting photo, the names of the people may be stored in association with each other. The storage unit 6 may also store the names of the people appearing in the character setting photo as shown in FIG. 3(b). By storing the names or titles of the characters in association with the photos in which the characters appear, the way the characters are called in the story is determined at the same time as the input of the characters appearing in the picture book. This eliminates the need for the user to input the names or titles of the characters every time. The storage unit 6 may also store the roles of the people appearing in the character setting photo as shown in FIG. 3(b). By storing the main character or sub-character as the role, when both appear, a story according to the role is generated. For example, when a person whose role is the main character appears in a story, a story centered on the main character is generated.

[0057] The reception means 21 receives a scene setting photo showing a scene appearing in a picture book. The scene setting photo may or may not show a person. If a person is shown in the scene setting photo, the reception means 21 may use the character setting photo and the scene setting photo to detect the person appearing in the scene setting photo by face recognition technology. By detecting the person appearing in the scene setting photo from the character setting photo, the user does not need to select the character setting photo. In addition, the user does not need to input the name, age, role, and title contained in the character setting photo information.

[0058] Since the user input information may include character setting photo information, the illustration generating means 22 may send an instruction to generate an illustration to the illustration generating means 41 using the character setting photo and / or the scene setting photo.

[0059] Since the user input information may include character setting photo information, the story generation means 23 may send a story generation instruction to the story generation means 51 using the name, age, role, and title included in the character setting photo information.

[0060] The reception means 21 further receives a selection of the direction of the illustration contained in the illustration pattern information as user input information from the user via the user terminal 3. The direction of the illustration may be received by inputting text from the user. The storage unit 6 stores an instruction linked to the direction of the illustration as shown in FIG. 3(c). When the reception means 21 receives an input of the direction of the illustration from the user, the illustration means 22 transmits an instruction to generate the illustration to the illustration generating means 41 using the instruction linked to the direction of the illustration. For example, the illustration means 22 generates a prompt using the instruction linked to the direction of the illustration, and transmits the photo and the prompt received from the user to the illustration generating means 41. The illustration generating means 41 generates an illustration according to the instruction of the direction of the illustration. The instruction of the direction of the illustration in FIG. 3(c) is an example, and any instruction that prompts the illustration generating means 41 to generate an illustration suitable for the direction of the illustration may be used.

[0061] The reception means 21 further receives a selection of a story direction contained in the narrativized synopsis information as user input information from the user via the user terminal 3. The story direction may be received by inputting text from the user. The storage unit 6 stores an instruction linked to the story direction as shown in FIG. 3(d). When the reception means 21 receives an input of a story direction from the user, the narrativization means 23 transmits an instruction to generate a story to the story generation means 51 using the instruction linked to the story direction. For example, the narrativization means 23 generates a prompt using the instruction linked to the story direction and transmits the prompt to the story generation means 51. The story generation means 51 generates a story according to the instruction of the story direction. The instruction of the story direction in FIG. 3(d) is an example, and any instruction that prompts the story generation means 51 to generate a story suitable for the story direction may be used.

[0062] The narrativizing means 23 further transmits character number control information for controlling the number of characters of the story to the story generation means 51. As the character number control information, one instruction may be determined in advance, and the narrativizing means 23 may transmit the instruction. In addition, the user may select the number of characters from several patterns. For example, the storage unit 6 stores an instruction linked to the number of characters as shown in FIG. 3(e), so that the user selects the number of characters, and the narrativizing means 23 transmits an instruction to generate a story to the story generation means 51 using the instruction of the number of characters. The instruction of the number of characters in FIG. 3(e) is an example, and any instruction may be used as long as it prompts the story generation means 51 to generate a story with the specified number of characters. In addition, the user may input any number of characters, and an instruction to generate a story with no more than any number of characters is transmitted to the story generation means 51. For example, the storage unit 6 stores "generate a story with no more than X characters" as an instruction of the number of characters, and the narrativizing means 23 transmits an instruction to the story generation means 51 in which the number of characters input by the user is substituted for X. If the user is allowed to enter any number of characters, a character limit may be included in the instructions.

[0063] The narrativizing means 23 further transmits hiragana conversion information that specifies the type of characters in the story to hiragana to the story generation means 51. As the hiragana conversion information, one instruction such as "generate a story entirely in hiragana" may be determined in advance, and the narrativizing means 23 may transmit the instruction. In addition, the user may select the hiragana conversion information from several patterns. For example, the storage unit 6 stores an instruction linked to the character type as shown in FIG. 3(f), so that the user selects the character type, and the narrativizing means 23 transmits an instruction to generate a story to the story generation means 51 using the instruction of the character type. The instruction of the character type in FIG. 3(f) is an example, and any instruction may be used to prompt the story generation means 51 to generate a story of the specified character type. In addition, the user may select a grade of elementary school, etc., and the narrativizing means 23 may transmit an instruction to generate a story using kanji to be learned up to that grade to the story generation means 51. For example, when the second grade of elementary school is selected, the narrative creating means 23 transmits to the narrative generating means 51 an instruction such as "kanji learned up to the second grade of elementary school can be used."

[0064] The narrativizing means 23 further transmits prohibition information specifying words that cannot be used in the story to the story generation means 51. For example, the storage unit 6 stores instructions linked to the prohibitions as shown in FIG. 3(g), and the narrativizing means 23 transmits an instruction to generate a story to the story generation means 51 using the instructions of the prohibitions. In addition to the instructions of the prohibitions shown in FIG. 3(g), an instruction such as "prohibit the use of expressions that may be problematic in religion" may be transmitted to the story generation means 51. In addition to this, the narrativizing means 23 may receive words that cannot be used in the story from the user, and transmit an instruction to the story generation means 51 urging the user not to use the words in the story.

[0065] 4(a), the storage unit 6 stores illustration and story instructions linked to a theme as template information. When the reception means 21 receives a theme as user input information from the user via the user terminal 3, the illustration creating means 22 and the story creating means 23 transmit instructions linked to the respective themes to the illustration generating means 41 and the story generating means 51.

[0066] 5 shows a processing sequence diagram for generating a picture book. In step S501, the user inputs user input information via the user terminal 3.

[0067] FIG. 6 shows an example of an input screen for user input information for generating a picture book about memories. The user sets a character as in W61. If the storage unit 6 stores a character setting photo, the user can select and input a character setting photo such as P61a or P61b as user input information. If the storage unit 6 does not store a character setting photo, the reception means 21 may receive a new character setting photo. P61a in FIG. 6 corresponds to Taro1.jpg in FIG. 3(b), and P61b in FIG. 6 corresponds to Taro2.jpg in FIG. 3(b). When the user selects P61a as a character to be included in the picture book, the illustration means 22 transmits the corresponding Taro1.jpg to the illustration generation means 41. The file format of the photo may be PNG, GIF, or the like, and the format is not specified. In addition, when P61a is selected, the story generation means 23 transmits information such as a name associated with Taro1.jpg to the story generation means 51. The user may be able to input only one character setting photo, or may be able to input multiple characters.

[0068] The user sets the scene of the picture book as shown in W62. In the scene setting, the reception means 21 may receive one text corresponding to one photograph as shown in P62a and T62a, P62b and T62b, . . . in FIG. 6. In FIG. 6, the text is one short sentence. The illustration means 22 transmits an instruction to generate an illustration to the illustration generating means 41 using the received photograph P62a. Furthermore, the story generating means 23 transmits an instruction to generate a story to the story generating means 51 using the received short sentence T62a. The integration means 24 integrates one illustration generated by the illustration generating means 41 and the story generated by the story generating means 51 to generate picture book data. The story generating means 23 may also transmit an instruction to generate a story to make the story to be generated into one sentence to the story generating means 51, and the integration means 24 may integrate one illustration and one sentence of the story.

[0069] The narrative means 23 may transmit the texts received from the user one by one as the generation instructions to the narrative generation means 51. For example, it is conceivable to transmit T62a, T62b, T62c, ... in Fig. 6 one by one.

[0070] In addition, when the narrativizing means 23 transmits the second text received from the user to the narrativizing means 51 as a generation instruction, the narrativizing means 23 may transmit the second text received from the user, the first text, and a story generated by the narrativizing means 51 from the first text. For example, when transmitting T62b in FIG. 6, the narrativizing means 23 transmits answers generated by the narrativizing means 51 from T62b, T62a, and T62a. That is, when transmitting the Nth text as a generation instruction, the narrativizing means 23 transmits the Nth and N-1th texts and the answer of the N-1th text to the narrativizing means 51. Also, when transmitting the third text as a generation instruction, the narrativizing means 23 may transmit a story generated by the narrativizing means 51 from the 1st to 3rd texts and the 1st and 2nd texts. That is, when transmitting the Nth text as a generation instruction, the narrativizing means 23 transmits the story generated by the narrativizing means 51 from the 1st to Nth texts and the 1st to N-1th texts to the narrativizing means 51. The narrative generating means 23 may transmit to the narrative generating means 51 all of the past text received from the user and the answers from the narrative generating means 51, or may transmit all of either the text or the answers.

[0071] Since the received one photo P62a and one text T62a correspond to each other, the illustration generated from P62a and the story generated from T62a are integrated by the integration means 24 and generated as one page of a picture book. The integration means 24 may integrate one page of the generated picture book in any order to generate picture book data. For example, the picture book generation device 2 generates picture book data in the page order of the input order of the user's photos and text. In FIG. 6, the user inputs scene 01, scene 02, and the integration means 24 generates picture book data in the order of scene 01, scene 02, and so on. The illustration means 22 and the story means 23 transmit scene 01, scene 02, and so on in the order in which they were received, and integrate the illustrations and stories in the order in which they were received. In addition, the photos and text to be transmitted may be linked with information indicating the order and transmitted, and the received illustrations and stories may be integrated based on the information indicating the order. A user can generate a picture book in his / her own preferred order by inputting scene setting photographs and text in his / her own preferred order or inputting information indicating the order of each scene.

[0072] Besides, in setting a scene, the reception means 21 may receive one or more pieces of text for multiple photographs. The illustration means 22 sends an instruction to generate an illustration using multiple photographs to the illustration generation means 41, and the story generation means 23 sends an instruction to generate a story using one or more pieces of text to the story generation means 51. The integration means 24 divides the story generated by the story generation means 51 and integrates it with the illustrations generated by the illustration generation means 41 to generate picture book data. Since there is no one-to-one correspondence between photographs and text, the integration means 24 divides the story and associates it with the illustrations.

[0073] In the scene setting, if a person appears in the scene setting photo accepted by the accepting means 21, the user may not set the characters, and the picture book creation device 2 may create a picture book using the person appearing in the scene setting photo. In this case, the role of the person appearing in the scene setting photo may be set in the scene setting. In addition, in the scene setting, if a person appears in the scene setting photo accepted by the accepting means 21, the picture book creation device 2 may perform face recognition by an image analysis method using a machine learning model or the like, detect the person appearing in the scene setting photo from the character setting photo, and transmit character setting photo information linked to the detected character setting photo to the story creation means 51.

[0074] The user sets the illustration directionality as shown in W63. The user selects the illustration directionality as shown in S63a, S63b, ... as user input information. In Fig. 6, "real" is selected as the illustration directionality in S63a. When the receiving means 21 receives the input of "real" as user input information as shown in Fig. 6, the illustration creating means 22 transmits an instruction to generate an illustration to the illustration generating means 41 using an instruction for the illustration directionality as shown in Fig. 3(c) associated with "real".

[0075] The user sets the story direction as in W64. The user selects the story direction as in S64a, S64b, ... as user input information. In Fig. 6, "adventure" is selected as the story direction in S64b. When the reception means 21 receives the input of "adventure" as user input information as in Fig. 6, the narrative means 23 transmits an instruction to generate an illustration to the story generation means 51 using an instruction for the story direction as in Fig. 3(d) linked to "adventure".

[0076] In step S502, the illustration creating means 22 transmits the user input information received from the user by the reception means 21 in S501 as an instruction to generate an illustration to the illustration generating means 41. Furthermore, in step S502, the story creating means 23 transmits the user input information received from the user by the reception means 21 in S501 to the story generating means 51 as an instruction to generate a story.

[0077] In step S503, the illustration generating means 41 and the story generating means 51 transmit the illustration or story generated by them in S502 to the sender of the instruction to generate the illustration or story.

[0078] In step S504, the integration means 24 integrates the illustrations and the story received in S503 to generate picture book data.

[0079] In step S505, the picture book creation device 2 outputs the created picture book data to the user terminal 3. The user can check the illustrations and story of the created picture book via the user terminal 3, and if the user does not like it, can change the user input information.

[0080] The picture book creation device 2 transmits a binding instruction including at least the picture book data to the factory. The factory may bind the book using the received picture book data and transmit the bound picture book to the address of the user included in the user basic information.

[0081] 7 shows an example of an input screen for user input information for generating a picture book related to discipline, manners, or rules. Note that the same reference numerals are used for the same input screen as in FIG. 6, and the description thereof will be omitted.

[0082] The user sets the theme of the picture book as shown in W72. In the theme setting, the reception means 21 receives one theme related to discipline, manners, or rules as shown in T72a, T72b, . . . in FIG. 7. In addition, when generating a picture book related to discipline, manners, or rules, it is necessary to set the characters appearing in the picture book in the character setting. The illustration means 22 transmits to the illustration generating means 41 the photo received in the character setting and the illustration direction associated with the theme received in the theme setting. Furthermore, the story generating means 23 transmits to the story generating means 51 the character setting photo information associated with the photo received in the character setting and the story direction associated with the theme received in the theme setting. Since the storage unit 6 stores the illustration and story direction associated with the theme as shown in FIG. 4(a), the illustration means 22 and the story generating means 23 can transmit the generation instruction to the illustration generating means 41 and the story generating means 51.

[0083] In the first embodiment, the instruction to generate an illustration is sent to the illustration generating means 41, and the instruction to generate a story is sent to the story generating means 51, but the instruction to generate an illustration and a story may be sent to the same generating means. This same generating means may be a machine learning model such as a neural network, or a large-scale language model.

[0084] <Embodiment 2> In the second embodiment, an integration device 7 integrates illustrations and a story. The same components as those in the first embodiment are given the same reference numerals and the description thereof will be omitted. Fig. 8 shows a block diagram of the picture book creation system 1 in the second embodiment.

[0085] The product sending means 25 sends an integration instruction to the integration means 71 using the illustrations generated by the illustration generating means 41 and the story generated by the story generating means 51. When there is a one-to-one correspondence between photos and text in a scene setting, the integration instruction includes information on the one-to-one correspondence between photos and text. The integration means 71 is a machine learning model or a large-scale language model, and the hardware configuration, etc. of the integration device 7 is similar to that of the illustration generating device 4 or the story generating device 5.

[0086] The integration device 7 includes an integration means 71 as a functional component. The integration means 71 generates picture book data using the illustrations and story included in the integration instruction, and transmits the generated picture book to the source of the integration instruction. In this embodiment, the source of the integration instruction is the picture book creation device 2, but is not limited to this, and a mode in which the source of the integration instruction is the user terminal 3 can also be adopted.

[0087] In the second embodiment, an instruction to generate an illustration is sent to the illustration generating means 41, an instruction to generate a story is sent to the story generating means 51, and an instruction to integrate the illustration and the story is sent to the integrating means 71, but an instruction to generate an illustration, an instruction to generate a story, and an instruction to integrate the illustration and the story may be sent to the same generating means or generating device. This same generating means may be a machine learning model such as a neural network, or a large-scale language model. Furthermore, the same generating device may have a generating means that is a machine learning model such as a neural network, or a large-scale language model.

[0088] A third party may provide part or all of the illustration generating device 4, the story generating device 5, and the integration device 7, and a web API may be used to generate illustrations and stories and integrate the illustrations and stories. [Explanation of symbols]

[0089] 1 Picture book generation system 2 Picture book generation device 21 Reception methods 22 Illustration Methods 23 Narrative means 24 Integration means 25 Product transmission means 3. User terminal 4. Illustration Generation Device 41 Illustration generation method 5. Story Generation Device 51 Narrative Generation Methods 6 Memory section 7 Integrated Equipment 71 Integration Measures NW Communication network

Claims

1. A picture book creation system for creating a picture book, comprising: The system includes a reception unit, an illustration unit, a story unit, and an integration unit, the accepting means accepts user input information including at least one of a photo, a text, and a theme; the illustration generating means transmits an instruction to generate an illustration to an illustration generating means using the user input information; the story generation means transmits a story generation instruction to a story generation means using the user input information; the integration means integrates the illustrations generated by the illustration generation means and the story generated by the story generation means to generate picture book data. Picture book generation system.

2. The illustration generating means is a machine learning model, The narrative generation means is a large-scale language model. The picture book creation system according to claim 1 .

3. The picture book production system further comprises a storage unit, the storage unit stores illustration pattern information, which is information for specifying a direction of the illustration, and story plot information, which is information for specifying a direction of the story; The illustration generating means further transmits the illustration pattern information to the illustration generating means; The storytelling means further transmits the story summary information to the story generation means. The picture book creation system according to claim 2 .

4. the accepting means accepts one short sentence as text corresponding to one photograph; the illustration generating means transmits an instruction to generate an illustration using the one photograph to the illustration generating means; the story generation means transmits a story generation instruction to the story generation means using the one short sentence; the integration means integrates the one illustration generated by the illustration generation means and the story generated by the story generation means to generate picture book data. The picture book creation system according to claim 3 .

5. The storage unit further stores at least one of character number control information for controlling the number of characters in the story, hiragana conversion information for specifying the type of characters in the story to be hiragana, and prohibited character information for specifying words that cannot be used in the story; The story generation means further transmits at least one of the character number control information, the hiragana conversion information, and the prohibition information to the story generation means. The picture book creation system according to claim 3 or 4.

6. The photographs include character setting photographs showing candidates for characters appearing in the story; The storage unit further stores the character setting photograph in association with a character name, which is the name of the character. The picture book creation system according to claim 3 or 4.

7. The integration is performed by a machine learning model. The picture book creation system according to claim 3 or 4.

8. The storage unit further stores template information related to discipline, manners, or rules in association with the theme. The picture book creation system according to claim 3 or 4.

9. The receiving means receives a photograph and a short sentence as text, the illustration generating means transmits an instruction to generate an illustration using the photograph to the illustration generating means; the story generation means transmits a story generation instruction to the story generation means using the short sentence; the integration means divides the story generated by the story generation means, integrates the divided story with the illustrations generated by the illustration generation means, and generates picture book data. The picture book creation system according to claim 2 or 3.

10. A picture book generation program for generating a picture book, A computer is made to function as a reception means, an illustration means, a storytelling means, and an integration means; the accepting means accepts user input information including at least one of a photo, a text, and a theme; the illustration generating means transmits an instruction to generate an illustration to an illustration generating means using the user input information; the story generation means transmits a story generation instruction to a story generation means using the user input information; the integration means integrates the illustrations generated by the illustration generation means and the story generated by the story generation means to generate picture book data. Picture book generation program.

11. A picture book creation method for creating a picture book, comprising the steps of: The computer executes a receiving step, an illustration step, a storytelling step, and an integration step; In the receiving step, user input information including at least one of a photo, a text, and a theme is received; In the illustration generating step, an instruction to generate an illustration is transmitted to an illustration generating means using the user input information; In the story generation step, a story generation instruction is sent to a story generation means using the user input information; In the integration step, the illustrations generated by the illustration generation means and the story generated by the story generation means are integrated to generate picture book data. How to generate a picture book.

Images