system

Abstract

Provide a system. [Solution] a means for a user to input a theme; means for collecting image data based on the theme; A means for generating AI images based on the collected image data; means for placing the generated images in a layout and adding captions; A way to save the completed layout as a PDF file, The system includes a means for sending the saved PDF file to a printing company.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the production of photobooks has required the efficient generation of high-quality, unique, creative images and the easy design and layout. However, conventional methods have required the manual collection and individual editing of large amounts of image data, which is time-consuming and labor-intensive. Furthermore, even with image generation technology, it is difficult to generate new images while preserving the characteristics of the collected images. [Means for solving the problem]

[0005] The present invention provides a system that allows users to easily create a photo book based on a theme. Specifically, it provides a means for users to input a theme and automatically collect image data based on that theme. It then includes a means for generating new images based on the collected image data using AI technology, particularly a generative adversarial network (GAN). It also includes a means for automatically arranging the generated images in a layout and adding captions. Finally, the above-mentioned problems can be solved by providing a system that includes a means for saving the completed layout as a PDF file and easily sending it to a printing company.

[0006] "User" refers to the entity that operates the system and performs tasks such as inputting themes and adjusting layouts.

[0007] A "theme" refers to the concept or idea that the user inputs as the basis for the photo album, and images are collected and generated based on that.

[0008] "Image data" refers to visual content such as photographs and illustrations related to the theme.

[0009] "Means of collection" refers to the technology for automatically searching and obtaining image data related to the theme from the Internet or databases.

[0010] "AI image generation" refers to the process of generating new images from existing image data using artificial intelligence techniques, particularly generative adversarial networks (GANs).

[0011] "Generated Image" refers to visual content newly created by AI image generation means.

[0012] "Means for arranging in layout" refers to the technique of appropriately arranging the generated images in the page design and editing them into a photo book.

[0013] "Caption" refers to text information added as a description or comment on an image.

[0014] "PDF file" is an abbreviation for Portable Document Format, and refers to a document file format that can be stored and shared electronically.

[0015] "Printing company" refers to a company that provides a service that outputs PDF files created by users as physical printed materials.

[0016] "Generative adversarial networks (GANs)" refers to a machine learning algorithm in which generative and discriminative models compete against each other to learn, and is a technology used to generate high-quality images. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] A way for users to input the theme

[0039] The user inputs the theme of the photo book using the terminal. For example, if the user sets the theme as "The Four Seasons of Japan," the user inputs the theme through the terminal interface.

[0040] A means for the server to collect image data based on a theme

[0041] The server automatically collects related image data from the internet and internal databases based on the theme entered by the user. It generates specific search queries, such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan," and retrieves a large number of images. Image data is collected using API calls and scraping technology.

[0042] A means for the server to generate AI images based on collected image data

[0043] The server generates new creative images using a generative adversarial network (GAN) based on the collected image data. The GAN model involves the process of extracting features from the image data and generating new images based on them. For example, it generates more fantastical images of cherry blossoms from collected images of spring cherry blossoms.

[0044] A means for the device to place generated images into a layout and provide captions

[0045] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. For example, a generated image of cherry blossoms could be captioned as "Beautiful spring cherry blossom scenery."

[0046] A means for the device to save the completed layout as a PDF file

[0047] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0048] A means for the device to send saved PDF files to a printing company

[0049] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0050] Specific examples

[0051] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device, and the server will automatically collect related images. This will result in a collection of Japanese landscape images for spring, summer, autumn, and winter, and the server's GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, original, and creative photo books.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0055] Step 2:

[0056] The server receives the theme entered by the user and searches and collects related image data from the Internet and internal databases based on that theme. For example, it generates search keywords such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" and retrieves a large number of images.

[0057] Step 3:

[0058] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model learns features from the input image data and generates new images based on them. In this process, for example, a new image of cherry blossoms is generated from the collected images of cherry blossoms.

[0059] Step 4:

[0060] The server transmits the generated creative image to the terminal.

[0061] Step 5:

[0062] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0063] Step 6:

[0064] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, with an automatically generated file name based on the theme selected.

[0065] Step 7:

[0066] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify the printing settings and number of copies.

[0067] Example 1

[0068] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0069] Conventional photobook creation systems require users to manually collect images and manually set layouts and captions, resulting in a time-consuming and labor-intensive process. Furthermore, generating high-quality, creative images requires specialized knowledge and tools, making it difficult for average users. Therefore, there is a need for a method to efficiently and easily create high-quality photobooks.

[0070] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0071] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for generating new images based on the collected image data using a generative adversarial network (GAN), means for arranging the generated images in a layout and adding captions, means for saving the completed layout as an electronic file, and means for sending the saved electronic file to a printing company, thereby enabling a user to efficiently create a high-quality and creative photo book with minimal operations.

[0072] A "user" is an individual or group that uses this system to create a photo book.

[0073] A "theme" is a keyword or phrase that indicates the content or concept of the photo book set by the user.

[0074] "Image data" refers to visual content such as photographs and illustrations that are collected or generated based on a theme.

[0075] "Means of collection" refers to the process of obtaining image data from the Internet or internal databases based on a theme.

[0076] A generative adversarial network (GAN) is a machine learning model in which two neural networks compete with each other to learn and generate new images.

[0077] A "prompt" is an instruction sentence to be input to a generative adversarial network (GAN) that specifies the characteristics of the image to be generated.

[0078] "Layout" refers to the arrangement of images and text within a photo book.

[0079] A "caption" is an explanatory text or comment added to an image.

[0080] An "electronic file" is a file stored in a digital format, and in the present invention refers to a file in PDF format.

[0081] A "printing company" is a company that provides a service of physically printing a photo book created by a user.

[0082] The present invention is a system that allows users to input a specific theme, automatically collects related images based on that theme, generates new creative images, and creates a photo book. The purpose of this system is to enable users to easily create high-quality, unique photo books. Specific embodiments of the system are described below.

[0083] First, the user inputs the theme of the photo book using their own device (e.g., a PC or smartphone). For example, themes such as "The Four Seasons of Japan" or "City Night Views" can be input through the device's interface. The input theme is then sent from the device to the server.

[0084] The server automatically collects related image data from the internet and internal databases based on the received theme. It generates a specific search query (e.g., "Spring scenery in Japan," "Summer scenery in Japan," etc.) and uses API calls and scraping technology to collect image data. The collected image data is temporarily stored in the server's internal storage.

[0085] Next, the server generates new images using a generative adversarial network (GAN) based on the collected image data. The GAN model extracts features from the collected images and generates new creative images based on them. For example, it generates more attractive cherry blossom images from images of spring cherry blossoms. This process is generally accompanied by a prompt sentence, such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan."

[0086] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photo book creation tool. The user can use this tool to arrange the images in a layout and add captions to each image. For example, a generated image of cherry blossoms can be captioned as "Beautiful spring cherry blossom scenery." The user can freely adjust the order, size, and placement of the images.

[0087] Once the user has finished adjusting the layout, the device's photobook creation tool will save the photo collection as a PDF file with an automatically generated filename based on the theme, for example, "Four Seasons of Japan.pdf."

[0088] The saved PDF file is sent to the printing company via the device interface. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request.

[0089] The present invention enables users to efficiently create high-quality and creative photo books with minimal operations.

[0090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0091] Step 1:

[0092] The user uses the terminal to input the theme of the photo book. For example, they enter a theme such as "Four Seasons of Japan" in the input field and click the "Send" button. This sends the theme to the server. The input data is the text of the theme, and the output data is the theme sent to the server.

[0093] Step 2:

[0094] The server collects related image data based on the received theme. For example, search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" are generated, and image data is retrieved from the Internet or an internal database. Techniques used here include API calls and scraping. The input data is the text of the theme, and the output data is the collected image data.

[0095] Step 3:

[0096] The server generates a prompt based on the collected image data. For example, it generates a prompt such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan." The input data is the collected image data, and the output data is the generated prompt.

[0097] Step 4:

[0098] The server uses the generated prompt sentence to generate a new image using a generative adversarial network (GAN). The GAN model learns features from the collected image data and generates new creative images based on them. For example, it generates a more attractive image of cherry blossoms from an image of cherry blossoms. The input data is the prompt sentence and the collected image data, and the output data is the generated creative image.

[0099] Step 5:

[0100] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photobook creation tool. Specifically, the tool displays a list of images in thumbnail format, allowing the user to freely select and drag images to create a layout. The input data is the generated creative images, and the output data is the photobook creation screen where the user can view and edit them.

[0101] Step 6:

[0102] The user uses the photo book creation tool to determine the layout of the images and add captions. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated cherry blossom image. The input data is the generated creative images and editing information by the user, and the output data is the completed layout.

[0103] Step 7:

[0104] Once the user has finished adjusting the layout, the device will save the photo book as a PDF file. The saved PDF file will automatically be given a file name based on the theme selected. For example, "Four Seasons of Japan.pdf." The input data is the completed layout, and the output data is the PDF file.

[0105] Step 8:

[0106] The device sends the saved PDF file to the printing company. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request. The input data is the PDF file and the user's print settings, and the output data is a confirmation of transmission to the printing company.

[0107] (Application example 1)

[0108] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Conventional photobook creation systems require users to manually collect a large number of images and manually set the layout and captions, making it difficult to efficiently create high-quality photobooks. Furthermore, some of the collected images may be of low quality or may not fit the theme, failing to meet user expectations. Furthermore, the limited real-time preview and editing functions for the generated images prevent a satisfactory user experience.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0111] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for transmitting the saved PDF file to an output device, means for inputting a theme via a user interface and previewing in real time, and means for editing the generated images on a smart device. This enables users to efficiently create high-quality photo books, and high customer satisfaction can be achieved through real-time editing and previewing.

[0112] A "means for user theme input" is an interface mechanism by which a user uses a device to input a particular theme.

[0113] The "means for collecting image data based on the theme" is a means for automatically collecting image data related to the input theme from the Internet or an internal database.

[0114] "Means for AI image generation based on collected image data" refers to means for generating new images using collected image data and an artificial intelligence model such as a generative adversarial network (GAN).

[0115] The "means for arranging generated images in a layout and providing captions" refers to a means for arranging generated images in a particular layout and providing a text description (caption) for each image.

[0116] "Means for saving the completed layout as a PDF file" refers to means for saving the completed layout as a PDF file after it has been completed.

[0117] A "means for transmitting a saved PDF file to an output device" is a means for transmitting a saved PDF file to a printer or other output device.

[0118] The "means for inputting a theme via a user interface and previewing it in real time" is an interface mechanism that allows a user to input a theme and preview an image in real time.

[0119] "Means for editing the generated image on a smart device" refers to means for editing the generated image on a device such as a smartphone or tablet.

[0120] This invention is a system that allows a user to input a theme and automatically create a photo album based on that theme. How this system is implemented will be described in detail below.

[0121] A way for users to input the theme

[0122] First, the user launches a dedicated application on a device such as a smartphone or tablet. This application includes an input interface where the user inputs the theme of the photo book. For example, themes such as "family trip," "wedding," and "pet memories" can be considered.

[0123] A means for the server to collect image data based on a theme

[0124] Next, the server collects related image data from the internet or an internal database based on the entered theme. The collection method is API calls or scraping technology. For example, if the theme is "family trip," the server generates search queries such as "family trip scenery" and "family trip images" to retrieve relevant images.

[0125] A means for the server to generate AI images based on collected image data

[0126] The server uses a generative adversarial network (GAN) based on the collected image data to generate new creative images. The GAN model extracts features from the image data and generates new images based on them. This typically involves using AI models from OpenAI (registered trademark) or Google (registered trademark) Cloud.

[0127] Specific prompt examples:

[0128] "Generate more creative images based on landscape images from your family trip."

[0129] A means for real-time preview via a user interface

[0130] The generated images are immediately sent to the user's device. The on-device interface provides a real-time preview, allowing the user to view the generated images and make live selections or add captions as needed. Using devices like smart glasses makes the real-time preview even more intuitive.

[0131] A means for the device to place generated images into a layout and provide captions

[0132] Users can arrange images generated on their device into a specific layout and add captions to them using a dedicated photo book creation tool, which provides a graphical interface for intuitively changing the position and order of images.

[0133] A means for the device to save the completed layout as a PDF file

[0134] Once the layout is complete, the user saves it to their device as a PDF file, which can automatically include the theme set in the filename.

[0135] A means for the device to send saved PDF files to an output device

[0136] The saved PDF file is sent to a printer or other output device via a dedicated output interface, which allows the user to place a print order and specify the number of copies and print settings as needed.

[0137] For example, if a user wants to create a photo book with the theme of "family trip," they can simply type "family trip" into their device. The server will automatically collect related images, and the GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printer. This allows users to efficiently create high-quality, original, and creative photo books.

[0138] This invention stands out from conventional systems by significantly improving the user experience and utilizing advanced AI technology to easily create high-quality photo books.

[0139] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0140] Step 1:

[0141] The device inputs the theme from the user.

[0142] Input: The user enters a theme, such as "family vacation," into the device's interface.

[0143] Operation: The device sends the text data entered as a theme to the server.

[0144] Output: The theme data sent to the server.

[0145] Step 2:

[0146] The server collects image data based on the theme.

[0147] Input: Theme data sent to the server.

[0148] How it works: The server uses API calls and scraping techniques to gather images related to a theme from the internet and an internal database.

[0149] Example: Generate search queries such as "family vacation" and "holiday family pictures" for the "family vacation" theme.

[0150] Output: A list of collected image data.

[0151] Step 3:

[0152] The server generates AI images based on the collected image data.

[0153] Input: A list of collected image data.

[0154] How it works: The server uses a generative adversarial network (GAN) to generate new creative images from collected image data.

[0155] Software used: AI models from OpenAI and Google Cloud.

[0156] Prompt: "Generate more creative images based on landscape images from your family trip."

[0157] Output: A list of the new creative images generated.

[0158] Step 4:

[0159] Send the generated image to the device for real-time preview

[0160] Input: A list of generated creative images.

[0161] Operation: The server sends the generated image data to the device and displays a real-time preview on the user interface.

[0162] The device displays a preview of the image and allows the user to select and add captions.

[0163] Output: Previewed image and user input.

[0164] Step 5:

[0165] Place user-generated images into a layout and add captions

[0166] Input: Previewed image and user caption input.

[0167] How it works: Using the on-device photo book creation tool, users arrange images in a specific layout and add captions to each image.

[0168] Users can move images by dragging and dropping and enter captions in the text fields.

[0169] Output: Photobook data with completed layout.

[0170] Step 6:

[0171] The device saves the completed layout as a PDF file

[0172] Input: Your completed photobook data.

[0173] What it does: The device saves the layout as a PDF file.

[0174] The file name will automatically include the theme you have set.

[0175] Output: Photo book file in PDF format.

[0176] Step 7:

[0177] The device sends the saved PDF file to the output device.

[0178] Input: Photo book file in PDF format.

[0179] How it works: Using a dedicated interface on the device, the user sends a PDF file to a printer or other output device.

[0180] The user places a printing order and enters the number of copies and printing settings.

[0181] Output: The PDF file sent and printing instructions.

[0182] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0183] A way for users to input the theme

[0184] The user inputs the theme of the photo book using the terminal. For example, the user inputs the theme "Four Seasons of Japan." The input is made via the terminal interface.

[0185] Theme recommendation using an emotion engine

[0186] The emotion engine analyzes the user's facial expressions and voice data to recognize their emotions. For example, it uses a camera and microphone to detect whether the user is smiling or sad. Based on the results of this analysis, it recommends a theme that suits the user. For example, if the user is relaxing, it will suggest "natural scenery" as a theme.

[0187] A means for the server to collect image data based on a theme

[0188] The server automatically collects related image data from the internet and internal databases based on themes entered by users or recommended by the emotion engine. For example, specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" can be generated to retrieve a large number of images. This collection process uses API calls and scraping technology.

[0189] A means for the server to generate AI images based on collected image data

[0190] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the input image data and generates new images based on them. For example, it generates more fantastical images of cherry blossoms from the collected images.

[0191] A means for the device to place generated images into a layout and provide captions

[0192] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. The emotion engine can also automatically suggest appropriate captions based on the user's emotions analyzed. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated image of cherry blossoms.

[0193] A means for the device to save the completed layout as a PDF file

[0194] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0195] A means for the device to send saved PDF files to a printing company

[0196] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0197] Specific examples

[0198] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device. The server will then automatically collect related images. This will result in a collection of landscape images of Japan throughout the four seasons, and the GAN model on the server will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Furthermore, an emotion engine is utilized to recommend appropriate captions based on the user's emotions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, unique, and creative photo books.

[0199] The processing flow will be explained below.

[0200] Step 1:

[0201] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0202] Step 2:

[0203] The device uses a camera and microphone to capture the user's facial expressions and voice data, which is then analyzed by an emotion engine to recognize emotions such as whether the user is relaxed, laughing, or sad.

[0204] Step 3:

[0205] The emotion engine suggests appropriate themes to the user based on the analysis results. For example, if the user is relaxed, it suggests themes such as "natural scenery," and if the user is happy, it suggests themes such as "happy memories."

[0206] Step 4:

[0207] The user can accept the suggested theme or select a different theme, for example, "Four Seasons of Japan."

[0208] Step 5:

[0209] The server automatically collects related image data from the Internet and internal databases based on the theme selected by the user. For example, a user can generate specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" to retrieve a large number of images.

[0210] Step 6:

[0211] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the collected image data and generates new images. For example, it generates new images of cherry blossoms from collected images of cherry blossoms.

[0212] Step 7:

[0213] The server transmits the generated creative image to the terminal.

[0214] Step 8:

[0215] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0216] Step 9:

[0217] The emotion engine automatically suggests appropriate captions based on the user's emotions. For example, if the user is feeling relaxed, it will suggest a caption such as "Beautiful spring cherry blossom scenery."

[0218] Step 10:

[0219] The user reviews the captions and edits them if necessary.

[0220] Step 11:

[0221] Once your device has finished adjusting the layout, it will save your photo book as a PDF file with an automatically generated file name based on the theme you set.

[0222] Step 12:

[0223] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify print settings and the number of copies.

[0224] Example 2

[0225] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] In conventional photo book generation systems, each process, such as collecting image data appropriate for a theme, generating new creative images, and adding captions based on the user's emotions, is often performed manually, placing a heavy burden on the user.Furthermore, there are few systems that use emotion recognition to recommend themes, making it difficult to efficiently create personalized photo books for each user.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0228] In this invention, the server includes a means for a user to input a theme, a means for recognizing the user's emotions using an emotion engine and recommending a theme, a means for collecting image data based on the theme, a means for generating new creative images using a generative adversarial network based on the collected image data, a means for arranging the generated images in a layout and adding captions, a means for saving the completed layout as a PDF file, and a means for sending the saved PDF file to a printing service. This significantly reduces the user's workload and enables efficient creation of high-quality photo books personalized to each user.

[0229] "User" refers to the person who operates the system and inputs themes, lays out images, adds captions, etc.

[0230] "Theme" refers to a keyword or phrase that a user enters as the subject of the photo book.

[0231] An "emotion engine" refers to a system that analyzes a user's facial expressions and voice data to recognize and determine their emotions.

[0232] "Image data" refers to all image files collected by the server from the Internet or internal databases.

[0233] "Generative Adversarial Networks (GANs)" refers to machine learning models that generate new creative images based on collected image data.

[0234] "Layout" refers to an arrangement plan that defines how the generated images are arranged.

[0235] "Caption" refers to explanatory text or comments attached to each image.

[0236] "PDF file" refers to the electronic document format in which user-created photo books are saved.

[0237] "Printing services" refers to companies or services that take your completed PDF files and print and bind them into a physical photo book.

[0238] This system is designed to enable users to easily create a photo album. Detailed embodiments of the system will be described below.

[0239] A way for users to input the theme

[0240] The user uses the device to input the theme of the photo book. The device has an interface using a keyboard or touch screen, making it easy to input the theme. For example, the user can input "Four Seasons of Japan" to set the theme. This information is immediately sent to the server and used for the next process.

[0241] Theme recommendation using an emotion engine

[0242] The emotion engine has the ability to analyze the user's facial expressions and voice data. It captures the user's emotional data using the device's built-in camera and microphone. For example, if the user is in a relaxed state, a theme such as "nature scenery" will be automatically recommended. The emotion engine sends this data to a server and provides feedback to the user.

[0243] A means for the server to collect image data based on a theme

[0244] The server automatically collects related image data from the internet and its internal database based on the entered theme. Specifically, it generates search queries such as "Spring in Japan" and "Summer in Japan" for the theme "Four Seasons in Japan," and retrieves images using multiple APIs. This process also uses scraping technology. The collected image data is stored in a database on the server.

[0245] A means for the server to generate AI images based on collected image data

[0246] The server generates new creative images by inputting the collected image data into a generative adversarial network (GAN) model. For example, if an image of cherry blossoms is input into the GAN model, a more fantastical image of cherry blossoms will be generated. The generated images are stored on the server and used for the next process.

[0247] A means for the device to place generated images into a layout and provide captions

[0248] The generated image data is sent to the device, and the user can use the photo book creation tool to arrange the images in a layout. The user can arrange the images in the desired order and add captions to each one. Captions can be entered manually by the user, or automatically suggested based on data analyzed by the emotion engine. For example, a caption such as "Beautiful spring cherry blossom scenery" is provided.

[0249] A means for the device to save the completed layout as a PDF file

[0250] Once the user has completed the layout and captions, the device clicks the "Save" button, which generates the completed photo book as a PDF file and saves it to the device. The file name is automatically generated based on the theme, such as "Four Seasons of Japan.pdf."

[0251] A means for the device to send saved PDF files to a printing company

[0252] The completed PDF file is sent from the device to the printing service. The user uploads the file using a dedicated sending interface and specifies the number of prints and settings. When the "Send" button is clicked, the PDF file and setting data are sent to the printing service via the Internet, and the print request is completed.

[0253] Specific examples

[0254] For example, if a user wants to create a photo book with the theme "The Four Seasons of Japan," they can ask the system using the following prompt sentence:

[0255] I would like to create a photo book with the theme of "The Four Seasons of Japan." Please collect beautiful images related to the theme and generate new creative images. Please layout the collected images with captions that reflect my emotions. Then, please save the completed photo book as a PDF file and send it to a printing service.

[0256] This allows users to efficiently create high-quality photo albums without much effort.

[0257] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0258] Step 1:

[0259] The user inputs the theme of the photo book. The device receives instructions from the user to input the theme. For example, the user inputs "Four Seasons of Japan" into the device's input field and presses the enter key. This input data is sent from the device to the server and saved as the basic data for the next process.

[0260] Step 2:

[0261] The emotion engine recommends themes. The device uses a camera and microphone to capture the user's facial expressions and voice data. This input data is sent to the server, where the emotion engine analyzes it. For example, if the user is smiling, the system determines that the user is in a "relaxed state." Based on the results of this analysis, the server selects an appropriate theme, such as "natural scenery," and recommends it to the user.

[0262] Step 3:

[0263] The server collects image data based on a theme. The server generates search queries from the internet and its internal database. For example, if the theme is "The Four Seasons of Japan," it automatically generates queries such as "Spring in Japan" and "Summer in Japan" to collect related image data. This is done using API calls and scraping technology. The collected images are stored in the server's data storage.

[0264] Step 4:

[0265] The server generates AI images based on the collected image data. Specifically, the server inputs the collected image data into a generative adversarial network (GAN) model to generate new creative images. For example, by inputting image data of cherry blossoms, the GAN model generates a more fantastical image of cherry blossoms. The generated images are stored on the server.

[0266] Step 5:

[0267] The device arranges the generated images in a layout and adds captions. The generated image data is sent from the server to the device. The user then uses a dedicated photo book creation tool on the device to arrange the images in the layout. For example, an image of spring cherry blossoms could be placed on the first page, followed by an image of summer fireworks. The user then adds a caption to each image. Captions can also be automatically suggested based on data analyzed by the emotion engine.

[0268] Step 6:

[0269] The device saves the completed layout as a PDF file. When the user clicks "Save," the device generates a PDF version of the layout photo album and automatically generates a file name. For example, the file name might be "Four Seasons of Japan.pdf." This PDF file is saved on the device.

[0270] Step 7:

[0271] The device sends the saved PDF file to the printing company. The user uploads the PDF file from the device using a dedicated sending interface. Here, they select the print settings and number of copies to print and click "Send." The file and settings information are sent via the Internet to the printing service, and the print request is completed.

[0272] (Application example 2)

[0273] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0274] Conventional content generation systems only collect and generate images based on themes entered by the user, making it difficult to recommend content that takes into account the user's emotional state or provide content in story format. Furthermore, since they only use collected image data, they lack creativity. Furthermore, there are insufficient means to automatically save and distribute user-generated content in an appropriate format.

[0275] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for sending the saved PDF file to a printing company, means for analyzing the user's facial expressions and voice data to recognize emotions and recommending a theme based on the emotions, means for collecting related image and video data based on the recommended theme and generating new creative content, means for laying out the generated creative content in a story format and automatically adding appropriate captions and narration, and means for saving the completed content in a selectable format such as PDF or MP4 and providing it to the user. This enables theme recommendation based on the user's emotional state, the generation of creative story-format content, and the automatic saving and distribution of content.

[0276] The "means for the user to input a theme" refers to an interface that allows the user to input a theme for content generation using a smartphone, tablet, or other device.

[0277] The "means for collecting image data based on the theme" is a module for automatically collecting related image data from the Internet or a database according to the theme entered by the user.

[0278] The "means for generating AI images based on collected image data" is a system for generating new creative images from collected image data using a generative adversarial network (GAN) model.

[0279] The "means for arranging generated images in a layout and adding captions" is an interface that allows a user to intuitively layout generated images and add captions to each image.

[0280] The "means for saving the completed layout as a PDF file" is a function for saving the layout completed by the user as a PDF file.

[0281] "Means for sending saved PDF files to a printing company" refers to a function for sending saved PDF files online to a printing company and requesting printing and publishing.

[0282] "Means for analyzing a user's facial expressions and voice data to recognize emotions and recommend themes based on that" refers to a system that uses a camera and microphone to analyze a user's emotional state and automatically recommends the most appropriate theme based on the results.

[0283] "Means for collecting related image and video data based on the recommended theme and generating new creative content" refers to a function for collecting image and video data on the recommended theme and generating new creative images and videos using a GAN model.

[0284] "Means for laying out generated creative content in a story format and automatically adding appropriate captions and narration" refers to an interface that lays out generated content in a consistent format as a story and automatically adds captions and narration.

[0285] "Means of saving completed content in a selectable format such as PDF or MP4 and providing it to the user" refers to a function that allows completed content to be saved in a format selected by the user (e.g., PDF or MP4) and provided to the user.

[0286] This invention is an implementation method of a system in which a user inputs a theme, and based on that theme, image and video data is collected, analyzed, generated, and provided as creative content.

[0287] First, the device used by the user is equipped with an interface for entering the theme. This interface is implemented as an application that runs on the device, such as a smartphone or tablet. The user starts the process by entering the theme into this interface.

[0288] Next, the server automatically collects relevant image and video data from the internet and internal databases based on the theme entered by the user. This collection is done using API calls and scraping techniques. The collected data is then input into a generative adversarial network (GAN) model to generate new creative images and videos. A GAN model is an algorithm that extracts features from the input image and video data and generates new content based on them.

[0289] Furthermore, the server is equipped with an emotion engine that analyzes the user's facial expressions and voice data to recognize their emotions. Using a camera and microphone, it detects whether the user is smiling, sad, or relaxed, and recommends appropriate themes to the user based on the analysis results. The prompt at this time is "The emotion engine is analyzing the user's emotions."

[0290] The generated creative content is then sent back to the device. The user uses a dedicated photo book creation tool to lay out the generated images and videos in a story format and add captions and narration. Captions and narration are automatically suggested based on the user's emotions analyzed by the emotion engine. This allows users to create high-quality content intuitively and efficiently. An example prompt is "Laying out the generated creative content in a story format and adding captions."

[0291] Finally, the completed content is saved in the format selected by the user. For example, it can be saved as a PDF or MP4 file. After saving, the user can send these files over the Internet to a printing company for printing and publishing. Examples of prompts are "Saving content..." or "Save as PDF? (y / n)".

[0292] This invention is a system that implements a series of processes, from recommending themes based on the user's emotional state to generating creative content using a generative AI model. Compared to conventional technologies, this system enables individual optimization that is in line with the user's emotions, and can provide more attractive content.

[0293] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0294] Step 1:

[0295] The user inputs a theme into the terminal. In this example, the user inputs a theme into the application using a terminal such as a smartphone or tablet. The input theme might be, for example, "The Four Seasons of Japan."

[0296] Input: Theme (e.g. "Four Seasons of Japan")

[0297] Output: The input theme

[0298] Step 2:

[0299] The server collects image and video data based on the theme entered. The server collects image and video data related to the theme from the internet and internal databases. This process uses API calls and scraping technology.

[0300] Input: Theme (e.g. "Four Seasons of Japan")

[0301] Data processing: Collecting image and video data using API calls and scraping techniques

[0302] Output: Collected image and video data

[0303] Step 3:

[0304] The server performs AI image generation based on the collected image and video data, using a generative adversarial network (GAN) model to generate new creative images and videos based on the collected data.

[0305] Input: Collected image and video data

[0306] Data Computing: Novel Creative Image and Video Generation with GAN Models

[0307] Output: Generated creative images and videos

[0308] Step 4:

[0309] The device analyzes the user's facial expressions and voice data to recognize their emotions, and then uses the device's camera and microphone to analyze the user's emotions and recommend the most appropriate theme based on the analysis results.

[0310] Input: Facial expression data and voice data

[0311] Data Computing: Emotion Recognition with an Emotion Analysis Engine

[0312] Output: Recommended theme (e.g. "Nature scenes for a relaxed state")

[0313] Step 5:

[0314] Based on the recommended theme, the server again collects related image and video data to generate new creative content, which is then fed back into the GAN model to generate more creative content.

[0315] Input: Recommended themes, additional image and video data

[0316] Data processing and data computation: Recollection and new content generation using GAN models

[0317] Output: Additional generated creative content

[0318] Step 6:

[0319] The generated creative content is sent to the device, where the user can lay it out and add captions and narration. On the device, the user can use dedicated tools to arrange the generated images and videos in a story format and add captions and narration.

[0320] Input: Generated creative content

[0321] Data processing: User can create story-style layouts and add captions and narration

[0322] Output: Finished story-formatted content

[0323] Step 7:

[0324] The device will save the completed content in PDF or MP4 format, and the user can save the content in the format of their choice.

[0325] Input: Complete story-formatted content

[0326] Data Processing: Export to your chosen format (PDF or MP4)

[0327] Output: Saved PDF or MP4 file

[0328] Step 8:

[0329] The server sends the saved file to a printing company, and users can submit the file online to have it printed and published.

[0330] Input: Saved PDF or MP4 files

[0331] Data calculation: Send to printing company via online sending interface

[0332] Output: Confirmation of sending to printing company

[0333] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0335] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0336] [Second embodiment]

[0337] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0338] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0340] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0344] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0347] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0348] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0349] A way for users to input the theme

[0350] The user inputs the theme of the photo book using the terminal. For example, if the user sets the theme as "The Four Seasons of Japan," the user inputs the theme through the terminal interface.

[0351] A means for the server to collect image data based on a theme

[0352] The server automatically collects related image data from the internet and internal databases based on the theme entered by the user. It generates specific search queries, such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan," and retrieves a large number of images. Image data is collected using API calls and scraping technology.

[0353] A means for the server to generate AI images based on collected image data

[0354] The server generates new creative images using a generative adversarial network (GAN) based on the collected image data. The GAN model involves the process of extracting features from the image data and generating new images based on them. For example, it generates more fantastical images of cherry blossoms from collected images of spring cherry blossoms.

[0355] A means for the device to place generated images into a layout and provide captions

[0356] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. For example, a generated image of cherry blossoms could be captioned as "Beautiful spring cherry blossom scenery."

[0357] A means for the device to save the completed layout as a PDF file

[0358] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0359] A means for the device to send saved PDF files to a printing company

[0360] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0361] Specific examples

[0362] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device, and the server will automatically collect related images. This will result in a collection of Japanese landscape images for spring, summer, autumn, and winter, and the server's GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, original, and creative photo books.

[0363] The processing flow will be explained below.

[0364] Step 1:

[0365] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0366] Step 2:

[0367] The server receives the theme entered by the user and searches and collects related image data from the Internet and internal databases based on that theme. For example, it generates search keywords such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" and retrieves a large number of images.

[0368] Step 3:

[0369] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model learns features from the input image data and generates new images based on them. In this process, for example, a new image of cherry blossoms is generated from the collected images of cherry blossoms.

[0370] Step 4:

[0371] The server transmits the generated creative image to the terminal.

[0372] Step 5:

[0373] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0374] Step 6:

[0375] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, with an automatically generated file name based on the theme selected.

[0376] Step 7:

[0377] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify the printing settings and number of copies.

[0378] Example 1

[0379] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0380] Conventional photobook creation systems require users to manually collect images and manually set layouts and captions, resulting in a time-consuming and labor-intensive process. Furthermore, generating high-quality, creative images requires specialized knowledge and tools, making it difficult for average users. Therefore, there is a need for a method to efficiently and easily create high-quality photobooks.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0382] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for generating new images based on the collected image data using a generative adversarial network (GAN), means for arranging the generated images in a layout and adding captions, means for saving the completed layout as an electronic file, and means for sending the saved electronic file to a printing company, thereby enabling a user to efficiently create a high-quality and creative photo book with minimal operations.

[0383] A "user" is an individual or group that uses this system to create a photo book.

[0384] A "theme" is a keyword or phrase that indicates the content or concept of the photo book set by the user.

[0385] "Image data" refers to visual content such as photographs and illustrations that are collected or generated based on a theme.

[0386] "Means of collection" refers to the process of obtaining image data from the Internet or internal databases based on a theme.

[0387] A generative adversarial network (GAN) is a machine learning model in which two neural networks compete with each other to learn and generate new images.

[0388] A "prompt" is an instruction sentence to be input to a generative adversarial network (GAN) that specifies the characteristics of the image to be generated.

[0389] "Layout" refers to the arrangement of images and text within a photo book.

[0390] A "caption" is an explanatory text or comment added to an image.

[0391] An "electronic file" is a file stored in a digital format, and in the present invention refers to a file in PDF format.

[0392] A "printing company" is a company that provides a service of physically printing a photo book created by a user.

[0393] The present invention is a system that allows users to input a specific theme, automatically collects related images based on that theme, generates new creative images, and creates a photo book. The purpose of this system is to enable users to easily create high-quality, unique photo books. Specific embodiments of the system are described below.

[0394] First, the user inputs the theme of the photo book using their own device (e.g., a PC or smartphone). For example, themes such as "The Four Seasons of Japan" or "City Night Views" can be input through the device's interface. The input theme is then sent from the device to the server.

[0395] The server automatically collects related image data from the internet and internal databases based on the received theme. It generates a specific search query (e.g., "Spring scenery in Japan," "Summer scenery in Japan," etc.) and uses API calls and scraping technology to collect image data. The collected image data is temporarily stored in the server's internal storage.

[0396] Next, the server generates new images using a generative adversarial network (GAN) based on the collected image data. The GAN model extracts features from the collected images and generates new creative images based on them. For example, it generates more attractive cherry blossom images from images of spring cherry blossoms. This process is generally accompanied by a prompt sentence, such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan."

[0397] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photo book creation tool. The user can use this tool to arrange the images in a layout and add captions to each image. For example, a generated image of cherry blossoms can be captioned as "Beautiful spring cherry blossom scenery." The user can freely adjust the order, size, and placement of the images.

[0398] Once the user has finished adjusting the layout, the device's photobook creation tool will save the photo collection as a PDF file with an automatically generated filename based on the theme, for example, "Four Seasons of Japan.pdf."

[0399] The saved PDF file is sent to the printing company via the device interface. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request.

[0400] The present invention enables users to efficiently create high-quality and creative photo books with minimal operations.

[0401] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0402] Step 1:

[0403] The user uses the terminal to input the theme of the photo book. For example, they enter a theme such as "Four Seasons of Japan" in the input field and click the "Send" button. This sends the theme to the server. The input data is the text of the theme, and the output data is the theme sent to the server.

[0404] Step 2:

[0405] The server collects related image data based on the received theme. For example, search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" are generated, and image data is retrieved from the Internet or an internal database. Techniques used here include API calls and scraping. The input data is the text of the theme, and the output data is the collected image data.

[0406] Step 3:

[0407] The server generates a prompt based on the collected image data. For example, it generates a prompt such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan." The input data is the collected image data, and the output data is the generated prompt.

[0408] Step 4:

[0409] The server uses the generated prompt sentence to generate a new image using a generative adversarial network (GAN). The GAN model learns features from the collected image data and generates new creative images based on them. For example, it generates a more attractive image of cherry blossoms from an image of cherry blossoms. The input data is the prompt sentence and the collected image data, and the output data is the generated creative image.

[0410] Step 5:

[0411] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photobook creation tool. Specifically, the tool displays a list of images in thumbnail format, allowing the user to freely select and drag images to create a layout. The input data is the generated creative images, and the output data is the photobook creation screen where the user can view and edit them.

[0412] Step 6:

[0413] The user uses the photo book creation tool to determine the layout of the images and add captions. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated cherry blossom image. The input data is the generated creative images and editing information by the user, and the output data is the completed layout.

[0414] Step 7:

[0415] Once the user has finished adjusting the layout, the device will save the photo book as a PDF file. The saved PDF file will automatically be given a file name based on the theme selected. For example, "Four Seasons of Japan.pdf." The input data is the completed layout, and the output data is the PDF file.

[0416] Step 8:

[0417] The device sends the saved PDF file to the printing company. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request. The input data is the PDF file and the user's print settings, and the output data is a confirmation of transmission to the printing company.

[0418] (Application example 1)

[0419] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0420] Conventional photobook creation systems require users to manually collect a large number of images and manually set the layout and captions, making it difficult to efficiently create high-quality photobooks. Furthermore, some of the collected images may be of low quality or may not fit the theme, failing to meet user expectations. Furthermore, the limited real-time preview and editing functions for the generated images prevent a satisfactory user experience.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0422] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for transmitting the saved PDF file to an output device, means for inputting a theme via a user interface and previewing in real time, and means for editing the generated images on a smart device. This enables users to efficiently create high-quality photo books, and high customer satisfaction can be achieved through real-time editing and previewing.

[0423] A "means for user theme input" is an interface mechanism by which a user uses a device to input a particular theme.

[0424] The "means for collecting image data based on the theme" is a means for automatically collecting image data related to the input theme from the Internet or an internal database.

[0425] "Means for AI image generation based on collected image data" refers to means for generating new images using collected image data and an artificial intelligence model such as a generative adversarial network (GAN).

[0426] The "means for arranging generated images in a layout and providing captions" refers to a means for arranging generated images in a particular layout and providing a text description (caption) for each image.

[0427] "Means for saving the completed layout as a PDF file" refers to means for saving the completed layout as a PDF file after it has been completed.

[0428] A "means for transmitting a saved PDF file to an output device" is a means for transmitting a saved PDF file to a printer or other output device.

[0429] The "means for inputting a theme via a user interface and previewing it in real time" is an interface mechanism that allows a user to input a theme and preview an image in real time.

[0430] "Means for editing the generated image on a smart device" refers to means for editing the generated image on a device such as a smartphone or tablet.

[0431] This invention is a system that allows a user to input a theme and automatically create a photo album based on that theme. How this system is implemented will be described in detail below.

[0432] A way for users to input the theme

[0433] First, the user launches a dedicated application on a device such as a smartphone or tablet. This application includes an input interface where the user inputs the theme of the photo book. For example, themes such as "family trip," "wedding," and "pet memories" can be considered.

[0434] A means for the server to collect image data based on a theme

[0435] Next, the server collects related image data from the internet or an internal database based on the entered theme. The collection method is API calls or scraping technology. For example, if the theme is "family trip," the server generates search queries such as "family trip scenery" and "family trip images" to retrieve relevant images.

[0436] A means for the server to generate AI images based on collected image data

[0437] The server uses a generative adversarial network (GAN) based on the collected image data to generate new creative images. The GAN model extracts features from the image data and generates new images based on them. This typically uses AI models from OpenAI or Google Cloud.

[0438] Specific prompt examples:

[0439] "Generate more creative images based on landscape images from your family trip."

[0440] A means for real-time preview via a user interface

[0441] The generated images are immediately sent to the user's device. The on-device interface provides a real-time preview, allowing the user to view the generated images and make live selections or add captions as needed. Using devices like smart glasses makes the real-time preview even more intuitive.

[0442] A means for the device to place generated images into a layout and provide captions

[0443] Users can arrange images generated on their device into a specific layout and add captions to them using a dedicated photo book creation tool, which provides a graphical interface for intuitively changing the position and order of images.

[0444] A means for the device to save the completed layout as a PDF file

[0445] Once the layout is complete, the user saves it to their device as a PDF file, which can automatically include the theme set in the filename.

[0446] A means for the device to send saved PDF files to an output device

[0447] The saved PDF file is sent to a printer or other output device via a dedicated output interface, which allows the user to place a print order and specify the number of copies and print settings as needed.

[0448] For example, if a user wants to create a photo book with the theme of "family trip," they can simply type "family trip" into their device. The server will automatically collect related images, and the GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printer. This allows users to efficiently create high-quality, original, and creative photo books.

[0449] This invention stands out from conventional systems by significantly improving the user experience and utilizing advanced AI technology to easily create high-quality photo books.

[0450] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0451] Step 1:

[0452] The device inputs the theme from the user.

[0453] Input: The user enters a theme, such as "family vacation," into the device's interface.

[0454] Operation: The device sends the text data entered as a theme to the server.

[0455] Output: The theme data sent to the server.

[0456] Step 2:

[0457] The server collects image data based on the theme.

[0458] Input: Theme data sent to the server.

[0459] How it works: The server uses API calls and scraping techniques to gather images related to a theme from the internet and an internal database.

[0460] Example: Generate search queries such as "family vacation" and "holiday family pictures" for the "family vacation" theme.

[0461] Output: A list of collected image data.

[0462] Step 3:

[0463] The server generates AI images based on the collected image data.

[0464] Input: A list of collected image data.

[0465] How it works: The server uses a generative adversarial network (GAN) to generate new creative images from collected image data.

[0466] Software used: AI models from OpenAI and Google Cloud.

[0467] Prompt: "Generate more creative images based on landscape images from your family trip."

[0468] Output: A list of the new creative images generated.

[0469] Step 4:

[0470] Send the generated image to the device for real-time preview

[0471] Input: A list of generated creative images.

[0472] Operation: The server sends the generated image data to the device and displays a real-time preview on the user interface.

[0473] The device displays a preview of the image and allows the user to select and add captions.

[0474] Output: Previewed image and user input.

[0475] Step 5:

[0476] Place user-generated images into a layout and add captions

[0477] Input: Previewed image and user caption input.

[0478] How it works: Using the on-device photo book creation tool, users arrange images in a specific layout and add captions to each image.

[0479] Users can move images by dragging and dropping and enter captions in the text fields.

[0480] Output: Photobook data with completed layout.

[0481] Step 6:

[0482] The device saves the completed layout as a PDF file

[0483] Input: Your completed photobook data.

[0484] What it does: The device saves the layout as a PDF file.

[0485] The file name will automatically include the theme you have set.

[0486] Output: Photo book file in PDF format.

[0487] Step 7:

[0488] The device sends the saved PDF file to the output device.

[0489] Input: Photo book file in PDF format.

[0490] How it works: Using a dedicated interface on the device, the user sends a PDF file to a printer or other output device.

[0491] The user places a printing order and enters the number of copies and printing settings.

[0492] Output: The PDF file sent and printing instructions.

[0493] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0494] A way for users to input the theme

[0495] The user inputs the theme of the photo book using the terminal. For example, the user inputs the theme "Four Seasons of Japan." The input is made via the terminal interface.

[0496] Theme recommendation using an emotion engine

[0497] The emotion engine analyzes the user's facial expressions and voice data to recognize their emotions. For example, it uses a camera and microphone to detect whether the user is smiling or sad. Based on the results of this analysis, it recommends a theme that suits the user. For example, if the user is relaxing, it will suggest "natural scenery" as a theme.

[0498] A means for the server to collect image data based on a theme

[0499] The server automatically collects related image data from the internet and internal databases based on themes entered by users or recommended by the emotion engine. For example, specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" can be generated to retrieve a large number of images. This collection process uses API calls and scraping technology.

[0500] A means for the server to generate AI images based on collected image data

[0501] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the input image data and generates new images based on them. For example, it generates more fantastical images of cherry blossoms from the collected images.

[0502] A means for the device to place generated images into a layout and provide captions

[0503] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. The emotion engine can also automatically suggest appropriate captions based on the user's emotions analyzed. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated image of cherry blossoms.

[0504] A means for the device to save the completed layout as a PDF file

[0505] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0506] A means for the device to send saved PDF files to a printing company

[0507] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0508] Specific examples

[0509] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device. The server will then automatically collect related images. This will result in a collection of landscape images of Japan throughout the four seasons, and the GAN model on the server will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Furthermore, an emotion engine is utilized to recommend appropriate captions based on the user's emotions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, unique, and creative photo books.

[0510] The processing flow will be explained below.

[0511] Step 1:

[0512] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0513] Step 2:

[0514] The device uses a camera and microphone to capture the user's facial expressions and voice data, which is then analyzed by an emotion engine to recognize emotions such as whether the user is relaxed, laughing, or sad.

[0515] Step 3:

[0516] The emotion engine suggests appropriate themes to the user based on the analysis results. For example, if the user is relaxed, it suggests themes such as "natural scenery," and if the user is happy, it suggests themes such as "happy memories."

[0517] Step 4:

[0518] The user can accept the suggested theme or select a different theme, for example, "Four Seasons of Japan."

[0519] Step 5:

[0520] The server automatically collects related image data from the Internet and internal databases based on the theme selected by the user. For example, a user can generate specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" to retrieve a large number of images.

[0521] Step 6:

[0522] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the collected image data and generates new images. For example, it generates new images of cherry blossoms from collected images of cherry blossoms.

[0523] Step 7:

[0524] The server transmits the generated creative image to the terminal.

[0525] Step 8:

[0526] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0527] Step 9:

[0528] The emotion engine automatically suggests appropriate captions based on the user's emotions. For example, if the user is feeling relaxed, it will suggest a caption such as "Beautiful spring cherry blossom scenery."

[0529] Step 10:

[0530] The user reviews the captions and edits them if necessary.

[0531] Step 11:

[0532] Once your device has finished adjusting the layout, it will save your photo book as a PDF file with an automatically generated file name based on the theme you set.

[0533] Step 12:

[0534] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify print settings and the number of copies.

[0535] Example 2

[0536] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0537] In conventional photo book generation systems, each process, such as collecting image data appropriate for a theme, generating new creative images, and adding captions based on the user's emotions, is often performed manually, placing a heavy burden on the user.Furthermore, there are few systems that use emotion recognition to recommend themes, making it difficult to efficiently create personalized photo books for each user.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0539] In this invention, the server includes a means for a user to input a theme, a means for recognizing the user's emotions using an emotion engine and recommending a theme, a means for collecting image data based on the theme, a means for generating new creative images using a generative adversarial network based on the collected image data, a means for arranging the generated images in a layout and adding captions, a means for saving the completed layout as a PDF file, and a means for sending the saved PDF file to a printing service. This significantly reduces the user's workload and enables efficient creation of high-quality photo books personalized to each user.

[0540] "User" refers to the person who operates the system and inputs themes, lays out images, adds captions, etc.

[0541] "Theme" refers to a keyword or phrase that a user enters as the subject of the photo book.

[0542] An "emotion engine" refers to a system that analyzes a user's facial expressions and voice data to recognize and determine their emotions.

[0543] "Image data" refers to all image files collected by the server from the Internet or internal databases.

[0544] "Generative Adversarial Networks (GANs)" refers to machine learning models that generate new creative images based on collected image data.

[0545] "Layout" refers to an arrangement plan that defines how the generated images are arranged.

[0546] "Caption" refers to explanatory text or comments attached to each image.

[0547] "PDF file" refers to the electronic document format in which user-created photo books are saved.

[0548] "Printing services" refers to companies or services that take your completed PDF files and print and bind them into a physical photo book.

[0549] This system is designed to enable users to easily create a photo album. Detailed embodiments of the system will be described below.

[0550] A way for users to input the theme

[0551] The user uses the device to input the theme of the photo book. The device has an interface using a keyboard or touch screen, making it easy to input the theme. For example, the user can input "Four Seasons of Japan" to set the theme. This information is immediately sent to the server and used for the next process.

[0552] Theme recommendation using an emotion engine

[0553] The emotion engine has the ability to analyze the user's facial expressions and voice data. It captures the user's emotional data using the device's built-in camera and microphone. For example, if the user is in a relaxed state, a theme such as "nature scenery" will be automatically recommended. The emotion engine sends this data to a server and provides feedback to the user.

[0554] A means for the server to collect image data based on a theme

[0555] The server automatically collects related image data from the internet and its internal database based on the entered theme. Specifically, it generates search queries such as "Spring in Japan" and "Summer in Japan" for the theme "Four Seasons in Japan," and retrieves images using multiple APIs. This process also uses scraping technology. The collected image data is stored in a database on the server.

[0556] A means for the server to generate AI images based on collected image data

[0557] The server generates new creative images by inputting the collected image data into a generative adversarial network (GAN) model. For example, if an image of cherry blossoms is input into the GAN model, a more fantastical image of cherry blossoms will be generated. The generated images are stored on the server and used for the next process.

[0558] A means for the device to place generated images into a layout and provide captions

[0559] The generated image data is sent to the device, and the user can use the photo book creation tool to arrange the images in a layout. The user can arrange the images in the desired order and add captions to each one. Captions can be entered manually by the user, or automatically suggested based on data analyzed by the emotion engine. For example, a caption such as "Beautiful spring cherry blossom scenery" is provided.

[0560] A means for the device to save the completed layout as a PDF file

[0561] Once the user has completed the layout and captions, the device clicks the "Save" button, which generates the completed photo book as a PDF file and saves it to the device. The file name is automatically generated based on the theme, such as "Four Seasons of Japan.pdf."

[0562] A means for the device to send saved PDF files to a printing company

[0563] The completed PDF file is sent from the device to the printing service. The user uploads the file using a dedicated sending interface and specifies the number of prints and settings. When the "Send" button is clicked, the PDF file and setting data are sent to the printing service via the Internet, and the print request is completed.

[0564] Specific examples

[0565] For example, if a user wants to create a photo book with the theme "The Four Seasons of Japan," they can ask the system using the following prompt sentence:

[0566] I would like to create a photo book with the theme of "The Four Seasons of Japan." Please collect beautiful images related to the theme and generate new creative images. Please layout the collected images with captions that reflect my emotions. Then, please save the completed photo book as a PDF file and send it to a printing service.

[0567] This allows users to efficiently create high-quality photo albums without much effort.

[0568] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0569] Step 1:

[0570] The user inputs the theme of the photo book. The device receives instructions from the user to input the theme. For example, the user inputs "Four Seasons of Japan" into the device's input field and presses the enter key. This input data is sent from the device to the server and saved as the basic data for the next process.

[0571] Step 2:

[0572] The emotion engine recommends themes. The device uses a camera and microphone to capture the user's facial expressions and voice data. This input data is sent to the server, where the emotion engine analyzes it. For example, if the user is smiling, the system determines that the user is in a "relaxed state." Based on the results of this analysis, the server selects an appropriate theme, such as "natural scenery," and recommends it to the user.

[0573] Step 3:

[0574] The server collects image data based on a theme. The server generates search queries from the internet and its internal database. For example, if the theme is "The Four Seasons of Japan," it automatically generates queries such as "Spring in Japan" and "Summer in Japan" to collect related image data. This is done using API calls and scraping technology. The collected images are stored in the server's data storage.

[0575] Step 4:

[0576] The server generates AI images based on the collected image data. Specifically, the server inputs the collected image data into a generative adversarial network (GAN) model to generate new creative images. For example, by inputting image data of cherry blossoms, the GAN model generates a more fantastical image of cherry blossoms. The generated images are stored on the server.

[0577] Step 5:

[0578] The device arranges the generated images in a layout and adds captions. The generated image data is sent from the server to the device. The user then uses a dedicated photo book creation tool on the device to arrange the images in the layout. For example, an image of spring cherry blossoms could be placed on the first page, followed by an image of summer fireworks. The user then adds a caption to each image. Captions can also be automatically suggested based on data analyzed by the emotion engine.

[0579] Step 6:

[0580] The device saves the completed layout as a PDF file. When the user clicks "Save," the device generates a PDF version of the layout photo album and automatically generates a file name. For example, the file name might be "Four Seasons of Japan.pdf." This PDF file is saved on the device.

[0581] Step 7:

[0582] The device sends the saved PDF file to the printing company. The user uploads the PDF file from the device using a dedicated sending interface. Here, they select the print settings and number of copies to print and click "Send." The file and settings information are sent via the Internet to the printing service, and the print request is completed.

[0583] (Application example 2)

[0584] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0585] Conventional content generation systems only collect and generate images based on themes entered by the user, making it difficult to recommend content that takes into account the user's emotional state or provide content in story format. Furthermore, since they only use collected image data, they lack creativity. Furthermore, there are insufficient means to automatically save and distribute user-generated content in an appropriate format.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for sending the saved PDF file to a printing company, means for analyzing the user's facial expressions and voice data to recognize emotions and recommending a theme based on the emotions, means for collecting related image and video data based on the recommended theme and generating new creative content, means for laying out the generated creative content in a story format and automatically adding appropriate captions and narration, and means for saving the completed content in a selectable format such as PDF or MP4 and providing it to the user. This enables theme recommendation based on the user's emotional state, the generation of creative story-format content, and the automatic saving and distribution of content.

[0587] The "means for the user to input a theme" refers to an interface that allows the user to input a theme for content generation using a smartphone, tablet, or other device.

[0588] The "means for collecting image data based on the theme" is a module for automatically collecting related image data from the Internet or a database according to the theme entered by the user.

[0589] The "means for generating AI images based on collected image data" is a system for generating new creative images from collected image data using a generative adversarial network (GAN) model.

[0590] The "means for arranging generated images in a layout and adding captions" is an interface that allows a user to intuitively layout generated images and add captions to each image.

[0591] The "means for saving the completed layout as a PDF file" is a function for saving the layout completed by the user as a PDF file.

[0592] "Means for sending saved PDF files to a printing company" refers to a function for sending saved PDF files online to a printing company and requesting printing and publishing.

[0593] "Means for analyzing a user's facial expressions and voice data to recognize emotions and recommend themes based on that" refers to a system that uses a camera and microphone to analyze a user's emotional state and automatically recommends the most appropriate theme based on the results.

[0594] "Means for collecting related image and video data based on the recommended theme and generating new creative content" refers to a function for collecting image and video data on the recommended theme and generating new creative images and videos using a GAN model.

[0595] "Means for laying out generated creative content in a story format and automatically adding appropriate captions and narration" refers to an interface that lays out generated content in a consistent format as a story and automatically adds captions and narration.

[0596] "Means of saving completed content in a selectable format such as PDF or MP4 and providing it to the user" refers to a function that allows completed content to be saved in a format selected by the user (e.g., PDF or MP4) and provided to the user.

[0597] This invention is an implementation method of a system in which a user inputs a theme, and based on that theme, image and video data is collected, analyzed, generated, and provided as creative content.

[0598] First, the device used by the user is equipped with an interface for entering the theme. This interface is implemented as an application that runs on the device, such as a smartphone or tablet. The user starts the process by entering the theme into this interface.

[0599] Next, the server automatically collects relevant image and video data from the internet and internal databases based on the theme entered by the user. This collection is done using API calls and scraping techniques. The collected data is then input into a generative adversarial network (GAN) model to generate new creative images and videos. A GAN model is an algorithm that extracts features from the input image and video data and generates new content based on them.

[0600] Furthermore, the server is equipped with an emotion engine that analyzes the user's facial expressions and voice data to recognize their emotions. Using a camera and microphone, it detects whether the user is smiling, sad, or relaxed, and recommends appropriate themes to the user based on the analysis results. The prompt at this time is "The emotion engine is analyzing the user's emotions."

[0601] The generated creative content is then sent back to the device. The user uses a dedicated photo book creation tool to lay out the generated images and videos in a story format and add captions and narration. Captions and narration are automatically suggested based on the user's emotions analyzed by the emotion engine. This allows users to create high-quality content intuitively and efficiently. An example prompt is "Laying out the generated creative content in a story format and adding captions."

[0602] Finally, the completed content is saved in the format selected by the user. For example, it can be saved as a PDF or MP4 file. After saving, the user can send these files over the Internet to a printing company for printing and publishing. Examples of prompts are "Saving content..." or "Save as PDF? (y / n)".

[0603] This invention is a system that implements a series of processes, from recommending themes based on the user's emotional state to generating creative content using a generative AI model. Compared to conventional technologies, this system enables individual optimization that is in line with the user's emotions, and can provide more attractive content.

[0604] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0605] Step 1:

[0606] The user inputs a theme into the terminal. In this example, the user inputs a theme into the application using a terminal such as a smartphone or tablet. The input theme might be, for example, "The Four Seasons of Japan."

[0607] Input: Theme (e.g. "Four Seasons of Japan")

[0608] Output: The input theme

[0609] Step 2:

[0610] The server collects image and video data based on the theme entered. The server collects image and video data related to the theme from the internet and internal databases. This process uses API calls and scraping technology.

[0611] Input: Theme (e.g. "Four Seasons of Japan")

[0612] Data processing: Collecting image and video data using API calls and scraping techniques

[0613] Output: Collected image and video data

[0614] Step 3:

[0615] The server performs AI image generation based on the collected image and video data, using a generative adversarial network (GAN) model to generate new creative images and videos based on the collected data.

[0616] Input: Collected image and video data

[0617] Data Computing: Novel Creative Image and Video Generation with GAN Models

[0618] Output: Generated creative images and videos

[0619] Step 4:

[0620] The device analyzes the user's facial expressions and voice data to recognize their emotions, and then uses the device's camera and microphone to analyze the user's emotions and recommend the most appropriate theme based on the analysis results.

[0621] Input: Facial expression data and voice data

[0622] Data Computing: Emotion Recognition with an Emotion Analysis Engine

[0623] Output: Recommended theme (e.g. "Nature scenes for a relaxed state")

[0624] Step 5:

[0625] Based on the recommended theme, the server again collects related image and video data to generate new creative content, which is then fed back into the GAN model to generate more creative content.

[0626] Input: Recommended themes, additional image and video data

[0627] Data processing and data computation: Recollection and new content generation using GAN models

[0628] Output: Additional generated creative content

[0629] Step 6:

[0630] The generated creative content is sent to the device, where the user can lay it out and add captions and narration. On the device, the user can use dedicated tools to arrange the generated images and videos in a story format and add captions and narration.

[0631] Input: Generated creative content

[0632] Data processing: User can create story-style layouts and add captions and narration

[0633] Output: Finished story-formatted content

[0634] Step 7:

[0635] The device will save the completed content in PDF or MP4 format, and the user can save the content in the format of their choice.

[0636] Input: Complete story-formatted content

[0637] Data Processing: Export to your chosen format (PDF or MP4)

[0638] Output: Saved PDF or MP4 file

[0639] Step 8:

[0640] The server sends the saved file to a printing company, and users can submit the file online to have it printed and published.

[0641] Input: Saved PDF or MP4 files

[0642] Data calculation: Send to printing company via online sending interface

[0643] Output: Confirmation of sending to printing company

[0644] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0645] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0646] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0647] [Third embodiment]

[0648] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0649] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0650] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0651] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0652] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0653] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0654] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0655] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0656] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0657] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0658] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0659] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0660] A way for users to input the theme

[0661] The user inputs the theme of the photo book using the terminal. For example, if the user sets the theme as "The Four Seasons of Japan," the user inputs the theme through the terminal interface.

[0662] A means for the server to collect image data based on a theme

[0663] The server automatically collects related image data from the internet and internal databases based on the theme entered by the user. It generates specific search queries, such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan," and retrieves a large number of images. Image data is collected using API calls and scraping technology.

[0664] A means for the server to generate AI images based on collected image data

[0665] The server generates new creative images using a generative adversarial network (GAN) based on the collected image data. The GAN model involves the process of extracting features from the image data and generating new images based on them. For example, it generates more fantastical images of cherry blossoms from collected images of spring cherry blossoms.

[0666] A means for the device to place generated images into a layout and provide captions

[0667] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. For example, a generated image of cherry blossoms could be captioned as "Beautiful spring cherry blossom scenery."

[0668] A means for the device to save the completed layout as a PDF file

[0669] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0670] A means for the device to send saved PDF files to a printing company

[0671] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0672] Specific examples

[0673] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device, and the server will automatically collect related images. This will result in a collection of Japanese landscape images for spring, summer, autumn, and winter, and the server's GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, original, and creative photo books.

[0674] The processing flow will be explained below.

[0675] Step 1:

[0676] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0677] Step 2:

[0678] The server receives the theme entered by the user and searches and collects related image data from the Internet and internal databases based on that theme. For example, it generates search keywords such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" and retrieves a large number of images.

[0679] Step 3:

[0680] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model learns features from the input image data and generates new images based on them. In this process, for example, a new image of cherry blossoms is generated from the collected images of cherry blossoms.

[0681] Step 4:

[0682] The server transmits the generated creative image to the terminal.

[0683] Step 5:

[0684] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0685] Step 6:

[0686] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, with an automatically generated file name based on the theme selected.

[0687] Step 7:

[0688] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify the printing settings and number of copies.

[0689] Example 1

[0690] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0691] Conventional photobook creation systems require users to manually collect images and manually set layouts and captions, resulting in a time-consuming and labor-intensive process. Furthermore, generating high-quality, creative images requires specialized knowledge and tools, making it difficult for average users. Therefore, there is a need for a method to efficiently and easily create high-quality photobooks.

[0692] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0693] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for generating new images based on the collected image data using a generative adversarial network (GAN), means for arranging the generated images in a layout and adding captions, means for saving the completed layout as an electronic file, and means for sending the saved electronic file to a printing company, thereby enabling a user to efficiently create a high-quality and creative photo book with minimal operations.

[0694] A "user" is an individual or group that uses this system to create a photo book.

[0695] A "theme" is a keyword or phrase that indicates the content or concept of the photo book set by the user.

[0696] "Image data" refers to visual content such as photographs and illustrations that are collected or generated based on a theme.

[0697] "Means of collection" refers to the process of obtaining image data from the Internet or internal databases based on a theme.

[0698] A generative adversarial network (GAN) is a machine learning model in which two neural networks compete with each other to learn and generate new images.

[0699] A "prompt" is an instruction sentence to be input to a generative adversarial network (GAN) that specifies the characteristics of the image to be generated.

[0700] "Layout" refers to the arrangement of images and text within a photo book.

[0701] A "caption" is an explanatory text or comment added to an image.

[0702] An "electronic file" is a file stored in a digital format, and in the present invention refers to a file in PDF format.

[0703] A "printing company" is a company that provides a service of physically printing a photo book created by a user.

[0704] The present invention is a system that allows users to input a specific theme, automatically collects related images based on that theme, generates new creative images, and creates a photo book. The purpose of this system is to enable users to easily create high-quality, unique photo books. Specific embodiments of the system are described below.

[0705] First, the user inputs the theme of the photo book using their own device (e.g., a PC or smartphone). For example, themes such as "The Four Seasons of Japan" or "City Night Views" can be input through the device's interface. The input theme is then sent from the device to the server.

[0706] The server automatically collects related image data from the internet and internal databases based on the received theme. It generates a specific search query (e.g., "Spring scenery in Japan," "Summer scenery in Japan," etc.) and uses API calls and scraping technology to collect image data. The collected image data is temporarily stored in the server's internal storage.

[0707] Next, the server generates new images using a generative adversarial network (GAN) based on the collected image data. The GAN model extracts features from the collected images and generates new creative images based on them. For example, it generates more attractive cherry blossom images from images of spring cherry blossoms. This process is generally accompanied by a prompt sentence, such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan."

[0708] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photo book creation tool. The user can use this tool to arrange the images in a layout and add captions to each image. For example, a generated image of cherry blossoms can be captioned as "Beautiful spring cherry blossom scenery." The user can freely adjust the order, size, and placement of the images.

[0709] Once the user has finished adjusting the layout, the device's photobook creation tool will save the photo collection as a PDF file with an automatically generated filename based on the theme, for example, "Four Seasons of Japan.pdf."

[0710] The saved PDF file is sent to the printing company via the device interface. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request.

[0711] The present invention enables users to efficiently create high-quality and creative photo books with minimal operations.

[0712] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0713] Step 1:

[0714] The user uses the terminal to input the theme of the photo book. For example, they enter a theme such as "Four Seasons of Japan" in the input field and click the "Send" button. This sends the theme to the server. The input data is the text of the theme, and the output data is the theme sent to the server.

[0715] Step 2:

[0716] The server collects related image data based on the received theme. For example, search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" are generated, and image data is retrieved from the Internet or an internal database. Techniques used here include API calls and scraping. The input data is the text of the theme, and the output data is the collected image data.

[0717] Step 3:

[0718] The server generates a prompt based on the collected image data. For example, it generates a prompt such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan." The input data is the collected image data, and the output data is the generated prompt.

[0719] Step 4:

[0720] The server uses the generated prompt sentence to generate a new image using a generative adversarial network (GAN). The GAN model learns features from the collected image data and generates new creative images based on them. For example, it generates a more attractive image of cherry blossoms from an image of cherry blossoms. The input data is the prompt sentence and the collected image data, and the output data is the generated creative image.

[0721] Step 5:

[0722] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photobook creation tool. Specifically, the tool displays a list of images in thumbnail format, allowing the user to freely select and drag images to create a layout. The input data is the generated creative images, and the output data is the photobook creation screen where the user can view and edit them.

[0723] Step 6:

[0724] The user uses the photo book creation tool to determine the layout of the images and add captions. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated cherry blossom image. The input data is the generated creative images and editing information by the user, and the output data is the completed layout.

[0725] Step 7:

[0726] Once the user has finished adjusting the layout, the device will save the photo book as a PDF file. The saved PDF file will automatically be given a file name based on the theme selected. For example, "Four Seasons of Japan.pdf." The input data is the completed layout, and the output data is the PDF file.

[0727] Step 8:

[0728] The device sends the saved PDF file to the printing company. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request. The input data is the PDF file and the user's print settings, and the output data is a confirmation of transmission to the printing company.

[0729] (Application example 1)

[0730] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0731] Conventional photobook creation systems require users to manually collect a large number of images and manually set the layout and captions, making it difficult to efficiently create high-quality photobooks. Furthermore, some of the collected images may be of low quality or may not fit the theme, failing to meet user expectations. Furthermore, the limited real-time preview and editing functions for the generated images prevent a satisfactory user experience.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0733] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for transmitting the saved PDF file to an output device, means for inputting a theme via a user interface and previewing in real time, and means for editing the generated images on a smart device. This enables users to efficiently create high-quality photo books, and high customer satisfaction can be achieved through real-time editing and previewing.

[0734] A "means for user theme input" is an interface mechanism by which a user uses a device to input a particular theme.

[0735] The "means for collecting image data based on the theme" is a means for automatically collecting image data related to the input theme from the Internet or an internal database.

[0736] "Means for AI image generation based on collected image data" refers to means for generating new images using collected image data and an artificial intelligence model such as a generative adversarial network (GAN).

[0737] The "means for arranging generated images in a layout and providing captions" refers to a means for arranging generated images in a particular layout and providing a text description (caption) for each image.

[0738] "Means for saving the completed layout as a PDF file" refers to means for saving the completed layout as a PDF file after it has been completed.

[0739] A "means for transmitting a saved PDF file to an output device" is a means for transmitting a saved PDF file to a printer or other output device.

[0740] The "means for inputting a theme via a user interface and previewing it in real time" is an interface mechanism that allows a user to input a theme and preview an image in real time.

[0741] "Means for editing the generated image on a smart device" refers to means for editing the generated image on a device such as a smartphone or tablet.

[0742] This invention is a system that allows a user to input a theme and automatically create a photo album based on that theme. How this system is implemented will be described in detail below.

[0743] A way for users to input the theme

[0744] First, the user launches a dedicated application on a device such as a smartphone or tablet. This application includes an input interface where the user inputs the theme of the photo book. For example, themes such as "family trip," "wedding," and "pet memories" can be considered.

[0745] A means for the server to collect image data based on a theme

[0746] Next, the server collects related image data from the internet or an internal database based on the entered theme. The collection method is API calls or scraping technology. For example, if the theme is "family trip," the server generates search queries such as "family trip scenery" and "family trip images" to retrieve relevant images.

[0747] A means for the server to generate AI images based on collected image data

[0748] The server uses a generative adversarial network (GAN) based on the collected image data to generate new creative images. The GAN model extracts features from the image data and generates new images based on them. This typically uses AI models from OpenAI or Google Cloud.

[0749] Specific prompt examples:

[0750] "Generate more creative images based on landscape images from your family trip."

[0751] A means for real-time preview via a user interface

[0752] The generated images are immediately sent to the user's device. The on-device interface provides a real-time preview, allowing the user to view the generated images and make live selections or add captions as needed. Using devices like smart glasses makes the real-time preview even more intuitive.

[0753] A means for the device to place generated images into a layout and provide captions

[0754] Users can arrange images generated on their device into a specific layout and add captions to them using a dedicated photo book creation tool, which provides a graphical interface for intuitively changing the position and order of images.

[0755] A means for the device to save the completed layout as a PDF file

[0756] Once the layout is complete, the user saves it to their device as a PDF file, which can automatically include the theme set in the filename.

[0757] A means for the device to send saved PDF files to an output device

[0758] The saved PDF file is sent to a printer or other output device via a dedicated output interface, which allows the user to place a print order and specify the number of copies and print settings as needed.

[0759] For example, if a user wants to create a photo book with the theme of "family trip," they can simply type "family trip" into their device. The server will automatically collect related images, and the GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printer. This allows users to efficiently create high-quality, original, and creative photo books.

[0760] This invention stands out from conventional systems by significantly improving the user experience and utilizing advanced AI technology to easily create high-quality photo books.

[0761] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0762] Step 1:

[0763] The device inputs the theme from the user.

[0764] Input: The user enters a theme, such as "family vacation," into the device's interface.

[0765] Operation: The device sends the text data entered as a theme to the server.

[0766] Output: The theme data sent to the server.

[0767] Step 2:

[0768] The server collects image data based on the theme.

[0769] Input: Theme data sent to the server.

[0770] How it works: The server uses API calls and scraping techniques to gather images related to a theme from the internet and an internal database.

[0771] Example: Generate search queries such as "family vacation" and "holiday family pictures" for the "family vacation" theme.

[0772] Output: A list of collected image data.

[0773] Step 3:

[0774] The server generates AI images based on the collected image data.

[0775] Input: A list of collected image data.

[0776] How it works: The server uses a generative adversarial network (GAN) to generate new creative images from collected image data.

[0777] Software used: AI models from OpenAI and Google Cloud.

[0778] Prompt: "Generate more creative images based on landscape images from your family trip."

[0779] Output: A list of the new creative images generated.

[0780] Step 4:

[0781] Send the generated image to the device for real-time preview

[0782] Input: A list of generated creative images.

[0783] Operation: The server sends the generated image data to the device and displays a real-time preview on the user interface.

[0784] The device displays a preview of the image and allows the user to select and add captions.

[0785] Output: Previewed image and user input.

[0786] Step 5:

[0787] Place user-generated images into a layout and add captions

[0788] Input: Previewed image and user caption input.

[0789] How it works: Using the on-device photo book creation tool, users arrange images in a specific layout and add captions to each image.

[0790] Users can move images by dragging and dropping and enter captions in the text fields.

[0791] Output: Photobook data with completed layout.

[0792] Step 6:

[0793] The device saves the completed layout as a PDF file

[0794] Input: Your completed photobook data.

[0795] What it does: The device saves the layout as a PDF file.

[0796] The file name will automatically include the theme you have set.

[0797] Output: Photo book file in PDF format.

[0798] Step 7:

[0799] The device sends the saved PDF file to the output device.

[0800] Input: Photo book file in PDF format.

[0801] How it works: Using a dedicated interface on the device, the user sends a PDF file to a printer or other output device.

[0802] The user places a printing order and enters the number of copies and printing settings.

[0803] Output: The PDF file sent and printing instructions.

[0804] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0805] A way for users to input the theme

[0806] The user inputs the theme of the photo book using the terminal. For example, the user inputs the theme "Four Seasons of Japan." The input is made via the terminal interface.

[0807] Theme recommendation using an emotion engine

[0808] The emotion engine analyzes the user's facial expressions and voice data to recognize their emotions. For example, it uses a camera and microphone to detect whether the user is smiling or sad. Based on the results of this analysis, it recommends a theme that suits the user. For example, if the user is relaxing, it will suggest "natural scenery" as a theme.

[0809] A means for the server to collect image data based on a theme

[0810] The server automatically collects related image data from the internet and internal databases based on themes entered by users or recommended by the emotion engine. For example, specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" can be generated to retrieve a large number of images. This collection process uses API calls and scraping technology.

[0811] A means for the server to generate AI images based on collected image data

[0812] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the input image data and generates new images based on them. For example, it generates more fantastical images of cherry blossoms from the collected images.

[0813] A means for the device to place generated images into a layout and provide captions

[0814] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. The emotion engine can also automatically suggest appropriate captions based on the user's emotions analyzed. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated image of cherry blossoms.

[0815] A means for the device to save the completed layout as a PDF file

[0816] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0817] A means for the device to send saved PDF files to a printing company

[0818] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0819] Specific examples

[0820] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device. The server will then automatically collect related images. This will result in a collection of landscape images of Japan throughout the four seasons, and the GAN model on the server will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Furthermore, an emotion engine is utilized to recommend appropriate captions based on the user's emotions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, unique, and creative photo books.

[0821] The processing flow will be explained below.

[0822] Step 1:

[0823] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0824] Step 2:

[0825] The device uses a camera and microphone to capture the user's facial expressions and voice data, which is then analyzed by an emotion engine to recognize emotions such as whether the user is relaxed, laughing, or sad.

[0826] Step 3:

[0827] The emotion engine suggests appropriate themes to the user based on the analysis results. For example, if the user is relaxed, it suggests themes such as "natural scenery," and if the user is happy, it suggests themes such as "happy memories."

[0828] Step 4:

[0829] The user can accept the suggested theme or select a different theme, for example, "Four Seasons of Japan."

[0830] Step 5:

[0831] The server automatically collects related image data from the Internet and internal databases based on the theme selected by the user. For example, a user can generate specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" to retrieve a large number of images.

[0832] Step 6:

[0833] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the collected image data and generates new images. For example, it generates new images of cherry blossoms from collected images of cherry blossoms.

[0834] Step 7:

[0835] The server transmits the generated creative image to the terminal.

[0836] Step 8:

[0837] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0838] Step 9:

[0839] The emotion engine automatically suggests appropriate captions based on the user's emotions. For example, if the user is feeling relaxed, it will suggest a caption such as "Beautiful spring cherry blossom scenery."

[0840] Step 10:

[0841] The user reviews the captions and edits them if necessary.

[0842] Step 11:

[0843] Once your device has finished adjusting the layout, it will save your photo book as a PDF file with an automatically generated file name based on the theme you set.

[0844] Step 12:

[0845] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify print settings and the number of copies.

[0846] Example 2

[0847] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0848] In conventional photo book generation systems, each process, such as collecting image data appropriate for a theme, generating new creative images, and adding captions based on the user's emotions, is often performed manually, placing a heavy burden on the user.Furthermore, there are few systems that use emotion recognition to recommend themes, making it difficult to efficiently create personalized photo books for each user.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0850] In this invention, the server includes a means for a user to input a theme, a means for recognizing the user's emotions using an emotion engine and recommending a theme, a means for collecting image data based on the theme, a means for generating new creative images using a generative adversarial network based on the collected image data, a means for arranging the generated images in a layout and adding captions, a means for saving the completed layout as a PDF file, and a means for sending the saved PDF file to a printing service. This significantly reduces the user's workload and enables efficient creation of high-quality photo books personalized to each user.

[0851] "User" refers to the person who operates the system and inputs themes, lays out images, adds captions, etc.

[0852] "Theme" refers to a keyword or phrase that a user enters as the subject of the photo book.

[0853] An "emotion engine" refers to a system that analyzes a user's facial expressions and voice data to recognize and determine their emotions.

[0854] "Image data" refers to all image files collected by the server from the Internet or internal databases.

[0855] "Generative Adversarial Networks (GANs)" refers to machine learning models that generate new creative images based on collected image data.

[0856] "Layout" refers to an arrangement plan that defines how the generated images are arranged.

[0857] "Caption" refers to explanatory text or comments attached to each image.

[0858] "PDF file" refers to the electronic document format in which user-created photo books are saved.

[0859] "Printing services" refers to companies or services that take your completed PDF files and print and bind them into a physical photo book.

[0860] This system is designed to enable users to easily create a photo album. Detailed embodiments of the system will be described below.

[0861] A way for users to input the theme

[0862] The user uses the device to input the theme of the photo book. The device has an interface using a keyboard or touch screen, making it easy to input the theme. For example, the user can input "Four Seasons of Japan" to set the theme. This information is immediately sent to the server and used for the next process.

[0863] Theme recommendation using an emotion engine

[0864] The emotion engine has the ability to analyze the user's facial expressions and voice data. It captures the user's emotional data using the device's built-in camera and microphone. For example, if the user is in a relaxed state, a theme such as "nature scenery" will be automatically recommended. The emotion engine sends this data to a server and provides feedback to the user.

[0865] A means for the server to collect image data based on a theme

[0866] The server automatically collects related image data from the internet and its internal database based on the entered theme. Specifically, it generates search queries such as "Spring in Japan" and "Summer in Japan" for the theme "Four Seasons in Japan," and retrieves images using multiple APIs. This process also uses scraping technology. The collected image data is stored in a database on the server.

[0867] A means for the server to generate AI images based on collected image data

[0868] The server generates new creative images by inputting the collected image data into a generative adversarial network (GAN) model. For example, if an image of cherry blossoms is input into the GAN model, a more fantastical image of cherry blossoms will be generated. The generated images are stored on the server and used for the next process.

[0869] A means for the device to place generated images into a layout and provide captions

[0870] The generated image data is sent to the device, and the user can use the photo book creation tool to arrange the images in a layout. The user can arrange the images in the desired order and add captions to each one. Captions can be entered manually by the user, or automatically suggested based on data analyzed by the emotion engine. For example, a caption such as "Beautiful spring cherry blossom scenery" is provided.

[0871] A means for the device to save the completed layout as a PDF file

[0872] Once the user has completed the layout and captions, the device clicks the "Save" button, which generates the completed photo book as a PDF file and saves it to the device. The file name is automatically generated based on the theme, such as "Four Seasons of Japan.pdf."

[0873] A means for the device to send saved PDF files to a printing company

[0874] The completed PDF file is sent from the device to the printing service. The user uploads the file using a dedicated sending interface and specifies the number of prints and settings. When the "Send" button is clicked, the PDF file and setting data are sent to the printing service via the Internet, and the print request is completed.

[0875] Specific examples

[0876] For example, if a user wants to create a photo book with the theme "The Four Seasons of Japan," they can ask the system using the following prompt sentence:

[0877] I would like to create a photo book with the theme of "The Four Seasons of Japan." Please collect beautiful images related to the theme and generate new creative images. Please layout the collected images with captions that reflect my emotions. Then, please save the completed photo book as a PDF file and send it to a printing service.

[0878] This allows users to efficiently create high-quality photo albums without much effort.

[0879] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0880] Step 1:

[0881] The user inputs the theme of the photo book. The device receives instructions from the user to input the theme. For example, the user inputs "Four Seasons of Japan" into the device's input field and presses the enter key. This input data is sent from the device to the server and saved as the basic data for the next process.

[0882] Step 2:

[0883] The emotion engine recommends themes. The device uses a camera and microphone to capture the user's facial expressions and voice data. This input data is sent to the server, where the emotion engine analyzes it. For example, if the user is smiling, the system determines that the user is in a "relaxed state." Based on the results of this analysis, the server selects an appropriate theme, such as "natural scenery," and recommends it to the user.

[0884] Step 3:

[0885] The server collects image data based on a theme. The server generates search queries from the internet and its internal database. For example, if the theme is "The Four Seasons of Japan," it automatically generates queries such as "Spring in Japan" and "Summer in Japan" to collect related image data. This is done using API calls and scraping technology. The collected images are stored in the server's data storage.

[0886] Step 4:

[0887] The server generates AI images based on the collected image data. Specifically, the server inputs the collected image data into a generative adversarial network (GAN) model to generate new creative images. For example, by inputting image data of cherry blossoms, the GAN model generates a more fantastical image of cherry blossoms. The generated images are stored on the server.

[0888] Step 5:

[0889] The device arranges the generated images in a layout and adds captions. The generated image data is sent from the server to the device. The user then uses a dedicated photo book creation tool on the device to arrange the images in the layout. For example, an image of spring cherry blossoms could be placed on the first page, followed by an image of summer fireworks. The user then adds a caption to each image. Captions can also be automatically suggested based on data analyzed by the emotion engine.

[0890] Step 6:

[0891] The device saves the completed layout as a PDF file. When the user clicks "Save," the device generates a PDF version of the layout photo album and automatically generates a file name. For example, the file name might be "Four Seasons of Japan.pdf." This PDF file is saved on the device.

[0892] Step 7:

[0893] The device sends the saved PDF file to the printing company. The user uploads the PDF file from the device using a dedicated sending interface. Here, they select the print settings and number of copies to print and click "Send." The file and settings information are sent via the Internet to the printing service, and the print request is completed.

[0894] (Application example 2)

[0895] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0896] Conventional content generation systems only collect and generate images based on themes entered by the user, making it difficult to recommend content that takes into account the user's emotional state or provide content in story format. Furthermore, since they only use collected image data, they lack creativity. Furthermore, there are insufficient means to automatically save and distribute user-generated content in an appropriate format.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for sending the saved PDF file to a printing company, means for analyzing the user's facial expressions and voice data to recognize emotions and recommending a theme based on the emotions, means for collecting related image and video data based on the recommended theme and generating new creative content, means for laying out the generated creative content in a story format and automatically adding appropriate captions and narration, and means for saving the completed content in a selectable format such as PDF or MP4 and providing it to the user. This enables theme recommendation based on the user's emotional state, the generation of creative story-format content, and the automatic saving and distribution of content.

[0898] The "means for the user to input a theme" refers to an interface that allows the user to input a theme for content generation using a smartphone, tablet, or other device.

[0899] The "means for collecting image data based on the theme" is a module for automatically collecting related image data from the Internet or a database according to the theme entered by the user.

[0900] The "means for generating AI images based on collected image data" is a system for generating new creative images from collected image data using a generative adversarial network (GAN) model.

[0901] The "means for arranging generated images in a layout and adding captions" is an interface that allows a user to intuitively layout generated images and add captions to each image.

[0902] The "means for saving the completed layout as a PDF file" is a function for saving the layout completed by the user as a PDF file.

[0903] "Means for sending saved PDF files to a printing company" refers to a function for sending saved PDF files online to a printing company and requesting printing and publishing.

[0904] "Means for analyzing a user's facial expressions and voice data to recognize emotions and recommend themes based on that" refers to a system that uses a camera and microphone to analyze a user's emotional state and automatically recommends the most appropriate theme based on the results.

[0905] "Means for collecting related image and video data based on the recommended theme and generating new creative content" refers to a function for collecting image and video data on the recommended theme and generating new creative images and videos using a GAN model.

[0906] "Means for laying out generated creative content in a story format and automatically adding appropriate captions and narration" refers to an interface that lays out generated content in a consistent format as a story and automatically adds captions and narration.

[0907] "Means of saving completed content in a selectable format such as PDF or MP4 and providing it to the user" refers to a function that allows completed content to be saved in a format selected by the user (e.g., PDF or MP4) and provided to the user.

[0908] This invention is an implementation method of a system in which a user inputs a theme, and based on that theme, image and video data is collected, analyzed, generated, and provided as creative content.

[0909] First, the device used by the user is equipped with an interface for entering the theme. This interface is implemented as an application that runs on the device, such as a smartphone or tablet. The user starts the process by entering the theme into this interface.

[0910] Next, the server automatically collects relevant image and video data from the internet and internal databases based on the theme entered by the user. This collection is done using API calls and scraping techniques. The collected data is then input into a generative adversarial network (GAN) model to generate new creative images and videos. A GAN model is an algorithm that extracts features from the input image and video data and generates new content based on them.

[0911] Furthermore, the server is equipped with an emotion engine that analyzes the user's facial expressions and voice data to recognize their emotions. Using a camera and microphone, it detects whether the user is smiling, sad, or relaxed, and recommends appropriate themes to the user based on the analysis results. The prompt at this time is "The emotion engine is analyzing the user's emotions."

[0912] The generated creative content is then sent back to the device. The user uses a dedicated photo book creation tool to lay out the generated images and videos in a story format and add captions and narration. Captions and narration are automatically suggested based on the user's emotions analyzed by the emotion engine. This allows users to create high-quality content intuitively and efficiently. An example prompt is "Laying out the generated creative content in a story format and adding captions."

[0913] Finally, the completed content is saved in the format selected by the user. For example, it can be saved as a PDF or MP4 file. After saving, the user can send these files over the Internet to a printing company for printing and publishing. Examples of prompts are "Saving content..." or "Save as PDF? (y / n)".

[0914] This invention is a system that implements a series of processes, from recommending themes based on the user's emotional state to generating creative content using a generative AI model. Compared to conventional technologies, this system enables individual optimization that is in line with the user's emotions, and can provide more attractive content.

[0915] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0916] Step 1:

[0917] The user inputs a theme into the terminal. In this example, the user inputs a theme into the application using a terminal such as a smartphone or tablet. The input theme might be, for example, "The Four Seasons of Japan."

[0918] Input: Theme (e.g. "Four Seasons of Japan")

[0919] Output: The input theme

[0920] Step 2:

[0921] The server collects image and video data based on the theme entered. The server collects image and video data related to the theme from the internet and internal databases. This process uses API calls and scraping technology.

[0922] Input: Theme (e.g. "Four Seasons of Japan")

[0923] Data processing: Collecting image and video data using API calls and scraping techniques

[0924] Output: Collected image and video data

[0925] Step 3:

[0926] The server performs AI image generation based on the collected image and video data, using a generative adversarial network (GAN) model to generate new creative images and videos based on the collected data.

[0927] Input: Collected image and video data

[0928] Data Computing: Novel Creative Image and Video Generation with GAN Models

[0929] Output: Generated creative images and videos

[0930] Step 4:

[0931] The device analyzes the user's facial expressions and voice data to recognize their emotions, and then uses the device's camera and microphone to analyze the user's emotions and recommend the most appropriate theme based on the analysis results.

[0932] Input: Facial expression data and voice data

[0933] Data Computing: Emotion Recognition with an Emotion Analysis Engine

[0934] Output: Recommended theme (e.g. "Nature scenes for a relaxed state")

[0935] Step 5:

[0936] Based on the recommended theme, the server again collects related image and video data to generate new creative content, which is then fed back into the GAN model to generate more creative content.

[0937] Input: Recommended themes, additional image and video data

[0938] Data processing and data computation: Recollection and new content generation using GAN models

[0939] Output: Additional generated creative content

[0940] Step 6:

[0941] The generated creative content is sent to the device, where the user can lay it out and add captions and narration. On the device, the user can use dedicated tools to arrange the generated images and videos in a story format and add captions and narration.

[0942] Input: Generated creative content

[0943] Data processing: User can create story-style layouts and add captions and narration

[0944] Output: Finished story-formatted content

[0945] Step 7:

[0946] The device will save the completed content in PDF or MP4 format, and the user can save the content in the format of their choice.

[0947] Input: Complete story-formatted content

[0948] Data Processing: Export to your chosen format (PDF or MP4)

[0949] Output: Saved PDF or MP4 file

[0950] Step 8:

[0951] The server sends the saved file to a printing company, and users can submit the file online to have it printed and published.

[0952] Input: Saved PDF or MP4 files

[0953] Data calculation: Send to printing company via online sending interface

[0954] Output: Confirmation of sending to printing company

[0955] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0956] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0957] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0958] [Fourth embodiment]

[0959] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0960] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0961] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0962] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0963] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0965] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0966] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0967] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0968] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0969] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0970] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0971] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0972] A way for users to input the theme

[0973] The user inputs the theme of the photo book using the terminal. For example, if the user sets the theme as "The Four Seasons of Japan," the user inputs the theme through the terminal interface.

[0974] A means for the server to collect image data based on a theme

[0975] The server automatically collects related image data from the internet and internal databases based on the theme entered by the user. It generates specific search queries, such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan," and retrieves a large number of images. Image data is collected using API calls and scraping technology.

[0976] A means for the server to generate AI images based on collected image data

[0977] The server generates new creative images using a generative adversarial network (GAN) based on the collected image data. The GAN model involves the process of extracting features from the image data and generating new images based on them. For example, it generates more fantastical images of cherry blossoms from collected images of spring cherry blossoms.

[0978] A means for the device to place generated images into a layout and provide captions

[0979] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. For example, a generated image of cherry blossoms could be captioned as "Beautiful spring cherry blossom scenery."

[0980] A means for the device to save the completed layout as a PDF file

[0981] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[0982] A means for the device to send saved PDF files to a printing company

[0983] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[0984] Specific examples

[0985] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device, and the server will automatically collect related images. This will result in a collection of Japanese landscape images for spring, summer, autumn, and winter, and the server's GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, original, and creative photo books.

[0986] The processing flow will be explained below.

[0987] Step 1:

[0988] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[0989] Step 2:

[0990] The server receives the theme entered by the user and searches and collects related image data from the Internet and internal databases based on that theme. For example, it generates search keywords such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" and retrieves a large number of images.

[0991] Step 3:

[0992] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model learns features from the input image data and generates new images based on them. In this process, for example, a new image of cherry blossoms is generated from the collected images of cherry blossoms.

[0993] Step 4:

[0994] The server transmits the generated creative image to the terminal.

[0995] Step 5:

[0996] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[0997] Step 6:

[0998] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, with an automatically generated file name based on the theme selected.

[0999] Step 7:

[1000] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify the printing settings and number of copies.

[1001] Example 1

[1002] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1003] Conventional photobook creation systems require users to manually collect images and manually set layouts and captions, resulting in a time-consuming and labor-intensive process. Furthermore, generating high-quality, creative images requires specialized knowledge and tools, making it difficult for average users. Therefore, there is a need for a method to efficiently and easily create high-quality photobooks.

[1004] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1005] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for generating new images based on the collected image data using a generative adversarial network (GAN), means for arranging the generated images in a layout and adding captions, means for saving the completed layout as an electronic file, and means for sending the saved electronic file to a printing company, thereby enabling a user to efficiently create a high-quality and creative photo book with minimal operations.

[1006] A "user" is an individual or group that uses this system to create a photo book.

[1007] A "theme" is a keyword or phrase that indicates the content or concept of the photo book set by the user.

[1008] "Image data" refers to visual content such as photographs and illustrations that are collected or generated based on a theme.

[1009] "Means of collection" refers to the process of obtaining image data from the Internet or internal databases based on a theme.

[1010] A generative adversarial network (GAN) is a machine learning model in which two neural networks compete with each other to learn and generate new images.

[1011] A "prompt" is an instruction sentence to be input to a generative adversarial network (GAN) that specifies the characteristics of the image to be generated.

[1012] "Layout" refers to the arrangement of images and text within a photo book.

[1013] A "caption" is an explanatory text or comment added to an image.

[1014] An "electronic file" is a file stored in a digital format, and in the present invention refers to a file in PDF format.

[1015] A "printing company" is a company that provides a service of physically printing a photo book created by a user.

[1016] The present invention is a system that allows users to input a specific theme, automatically collects related images based on that theme, generates new creative images, and creates a photo book. The purpose of this system is to enable users to easily create high-quality, unique photo books. Specific embodiments of the system are described below.

[1017] First, the user inputs the theme of the photo book using their own device (e.g., a PC or smartphone). For example, themes such as "The Four Seasons of Japan" or "City Night Views" can be input through the device's interface. The input theme is then sent from the device to the server.

[1018] The server automatically collects related image data from the internet and internal databases based on the received theme. It generates a specific search query (e.g., "Spring scenery in Japan," "Summer scenery in Japan," etc.) and uses API calls and scraping technology to collect image data. The collected image data is temporarily stored in the server's internal storage.

[1019] Next, the server generates new images using a generative adversarial network (GAN) based on the collected image data. The GAN model extracts features from the collected images and generates new creative images based on them. For example, it generates more attractive cherry blossom images from images of spring cherry blossoms. This process is generally accompanied by a prompt sentence, such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan."

[1020] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photo book creation tool. The user can use this tool to arrange the images in a layout and add captions to each image. For example, a generated image of cherry blossoms can be captioned as "Beautiful spring cherry blossom scenery." The user can freely adjust the order, size, and placement of the images.

[1021] Once the user has finished adjusting the layout, the device's photobook creation tool will save the photo collection as a PDF file with an automatically generated filename based on the theme, for example, "Four Seasons of Japan.pdf."

[1022] The saved PDF file is sent to the printing company via the device interface. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request.

[1023] The present invention enables users to efficiently create high-quality and creative photo books with minimal operations.

[1024] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1025] Step 1:

[1026] The user uses the terminal to input the theme of the photo book. For example, they enter a theme such as "Four Seasons of Japan" in the input field and click the "Send" button. This sends the theme to the server. The input data is the text of the theme, and the output data is the theme sent to the server.

[1027] Step 2:

[1028] The server collects related image data based on the received theme. For example, search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" are generated, and image data is retrieved from the Internet or an internal database. Techniques used here include API calls and scraping. The input data is the text of the theme, and the output data is the collected image data.

[1029] Step 3:

[1030] The server generates a prompt based on the collected image data. For example, it generates a prompt such as "Please generate a fantastical image of cherry blossoms based on the theme of the four seasons of Japan." The input data is the collected image data, and the output data is the generated prompt.

[1031] Step 4:

[1032] The server uses the generated prompt sentence to generate a new image using a generative adversarial network (GAN). The GAN model learns features from the collected image data and generates new creative images based on them. For example, it generates a more attractive image of cherry blossoms from an image of cherry blossoms. The input data is the prompt sentence and the collected image data, and the output data is the generated creative image.

[1033] Step 5:

[1034] The generated images are sent from the server to the user's device. The device displays the received images in a dedicated photobook creation tool. Specifically, the tool displays a list of images in thumbnail format, allowing the user to freely select and drag images to create a layout. The input data is the generated creative images, and the output data is the photobook creation screen where the user can view and edit them.

[1035] Step 6:

[1036] The user uses the photo book creation tool to determine the layout of the images and add captions. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated cherry blossom image. The input data is the generated creative images and editing information by the user, and the output data is the completed layout.

[1037] Step 7:

[1038] Once the user has finished adjusting the layout, the device will save the photo book as a PDF file. The saved PDF file will automatically be given a file name based on the theme selected. For example, "Four Seasons of Japan.pdf." The input data is the completed layout, and the output data is the PDF file.

[1039] Step 8:

[1040] The device sends the saved PDF file to the printing company. The user uses the sending interface to upload the file and set print settings (e.g., number of copies, paper type, binding method, etc.). Finally, the user clicks the "Send" button to complete the print request. The input data is the PDF file and the user's print settings, and the output data is a confirmation of transmission to the printing company.

[1041] (Application example 1)

[1042] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1043] Conventional photobook creation systems require users to manually collect a large number of images and manually set the layout and captions, making it difficult to efficiently create high-quality photobooks. Furthermore, some of the collected images may be of low quality or may not fit the theme, failing to meet user expectations. Furthermore, the limited real-time preview and editing functions for the generated images prevent a satisfactory user experience.

[1044] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1045] In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for transmitting the saved PDF file to an output device, means for inputting a theme via a user interface and previewing in real time, and means for editing the generated images on a smart device. This enables users to efficiently create high-quality photo books, and high customer satisfaction can be achieved through real-time editing and previewing.

[1046] A "means for user theme input" is an interface mechanism by which a user uses a device to input a particular theme.

[1047] The "means for collecting image data based on the theme" is a means for automatically collecting image data related to the input theme from the Internet or an internal database.

[1048] "Means for AI image generation based on collected image data" refers to means for generating new images using collected image data and an artificial intelligence model such as a generative adversarial network (GAN).

[1049] The "means for arranging generated images in a layout and providing captions" refers to a means for arranging generated images in a particular layout and providing a text description (caption) for each image.

[1050] "Means for saving the completed layout as a PDF file" refers to means for saving the completed layout as a PDF file after it has been completed.

[1051] A "means for transmitting a saved PDF file to an output device" is a means for transmitting a saved PDF file to a printer or other output device.

[1052] The "means for inputting a theme via a user interface and previewing it in real time" is an interface mechanism that allows a user to input a theme and preview an image in real time.

[1053] "Means for editing the generated image on a smart device" refers to means for editing the generated image on a device such as a smartphone or tablet.

[1054] This invention is a system that allows a user to input a theme and automatically create a photo album based on that theme. How this system is implemented will be described in detail below.

[1055] A way for users to input the theme

[1056] First, the user launches a dedicated application on a device such as a smartphone or tablet. This application includes an input interface where the user inputs the theme of the photo book. For example, themes such as "family trip," "wedding," and "pet memories" can be considered.

[1057] A means for the server to collect image data based on a theme

[1058] Next, the server collects related image data from the internet or an internal database based on the entered theme. The collection method is API calls or scraping technology. For example, if the theme is "family trip," the server generates search queries such as "family trip scenery" and "family trip images" to retrieve relevant images.

[1059] A means for the server to generate AI images based on collected image data

[1060] The server uses a generative adversarial network (GAN) based on the collected image data to generate new creative images. The GAN model extracts features from the image data and generates new images based on them. This typically uses AI models from OpenAI or Google Cloud.

[1061] Specific prompt examples:

[1062] "Generate more creative images based on landscape images from your family trip."

[1063] A means for real-time preview via a user interface

[1064] The generated images are immediately sent to the user's device. The on-device interface provides a real-time preview, allowing the user to view the generated images and make live selections or add captions as needed. Using devices like smart glasses makes the real-time preview even more intuitive.

[1065] A means for the device to place generated images into a layout and provide captions

[1066] Users can arrange images generated on their device into a specific layout and add captions to them using a dedicated photo book creation tool, which provides a graphical interface for intuitively changing the position and order of images.

[1067] A means for the device to save the completed layout as a PDF file

[1068] Once the layout is complete, the user saves it to their device as a PDF file, which can automatically include the theme set in the filename.

[1069] A means for the device to send saved PDF files to an output device

[1070] The saved PDF file is sent to a printer or other output device via a dedicated output interface, which allows the user to place a print order and specify the number of copies and print settings as needed.

[1071] For example, if a user wants to create a photo book with the theme of "family trip," they can simply type "family trip" into their device. The server will automatically collect related images, and the GAN model will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Finally, the completed photo book is saved as a PDF file and sent to a printer. This allows users to efficiently create high-quality, original, and creative photo books.

[1072] This invention stands out from conventional systems by significantly improving the user experience and utilizing advanced AI technology to easily create high-quality photo books.

[1073] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1074] Step 1:

[1075] The device inputs the theme from the user.

[1076] Input: The user enters a theme, such as "family vacation," into the device's interface.

[1077] Operation: The device sends the text data entered as a theme to the server.

[1078] Output: The theme data sent to the server.

[1079] Step 2:

[1080] The server collects image data based on the theme.

[1081] Input: Theme data sent to the server.

[1082] How it works: The server uses API calls and scraping techniques to gather images related to a theme from the internet and an internal database.

[1083] Example: Generate search queries such as "family vacation" and "holiday family pictures" for the "family vacation" theme.

[1084] Output: A list of collected image data.

[1085] Step 3:

[1086] The server generates AI images based on the collected image data.

[1087] Input: A list of collected image data.

[1088] How it works: The server uses a generative adversarial network (GAN) to generate new creative images from collected image data.

[1089] Software used: AI models from OpenAI and Google Cloud.

[1090] Prompt: "Generate more creative images based on landscape images from your family trip."

[1091] Output: A list of the new creative images generated.

[1092] Step 4:

[1093] Send the generated image to the device for real-time preview

[1094] Input: A list of generated creative images.

[1095] Operation: The server sends the generated image data to the device and displays a real-time preview on the user interface.

[1096] The device displays a preview of the image and allows the user to select and add captions.

[1097] Output: Previewed image and user input.

[1098] Step 5:

[1099] Place user-generated images into a layout and add captions

[1100] Input: Previewed image and user caption input.

[1101] How it works: Using the on-device photo book creation tool, users arrange images in a specific layout and add captions to each image.

[1102] Users can move images by dragging and dropping and enter captions in the text fields.

[1103] Output: Photobook data with completed layout.

[1104] Step 6:

[1105] The device saves the completed layout as a PDF file

[1106] Input: Your completed photobook data.

[1107] What it does: The device saves the layout as a PDF file.

[1108] The file name will automatically include the theme you have set.

[1109] Output: Photo book file in PDF format.

[1110] Step 7:

[1111] The device sends the saved PDF file to the output device.

[1112] Input: Photo book file in PDF format.

[1113] How it works: Using a dedicated interface on the device, the user sends a PDF file to a printer or other output device.

[1114] The user places a printing order and enters the number of copies and printing settings.

[1115] Output: The PDF file sent and printing instructions.

[1116] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1117] A way for users to input the theme

[1118] The user inputs the theme of the photo book using the terminal. For example, the user inputs the theme "Four Seasons of Japan." The input is made via the terminal interface.

[1119] Theme recommendation using an emotion engine

[1120] The emotion engine analyzes the user's facial expressions and voice data to recognize their emotions. For example, it uses a camera and microphone to detect whether the user is smiling or sad. Based on the results of this analysis, it recommends a theme that suits the user. For example, if the user is relaxing, it will suggest "natural scenery" as a theme.

[1121] A means for the server to collect image data based on a theme

[1122] The server automatically collects related image data from the internet and internal databases based on themes entered by users or recommended by the emotion engine. For example, specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" can be generated to retrieve a large number of images. This collection process uses API calls and scraping technology.

[1123] A means for the server to generate AI images based on collected image data

[1124] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the input image data and generates new images based on them. For example, it generates more fantastical images of cherry blossoms from the collected images.

[1125] A means for the device to place generated images into a layout and provide captions

[1126] The generated images are sent to the device, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds a caption to each one. The emotion engine can also automatically suggest appropriate captions based on the user's emotions analyzed. For example, a caption such as "Beautiful spring cherry blossom scenery" can be added to a generated image of cherry blossoms.

[1127] A means for the device to save the completed layout as a PDF file

[1128] Once the user has finished adjusting the layout, the device will save the photo album as a PDF file, allowing the entire photo album to be managed as a single electronic file. The file name will be automatically generated based on the theme selected.

[1129] A means for the device to send saved PDF files to a printing company

[1130] The saved PDF file is sent from the device to the printing company. Users can easily upload the file through a dedicated sending interface and request printing and publishing. At the same time, they can also specify print settings and the number of copies.

[1131] Specific examples

[1132] For example, if a user wants to create a photo book with the theme of "The Four Seasons of Japan," they can simply type "The Four Seasons of Japan" into their device. The server will then automatically collect related images. This will result in a collection of landscape images of Japan throughout the four seasons, and the GAN model on the server will generate new creative images. The generated images are then sent to the device, where the user can arrange them appropriately and add captions. Furthermore, an emotion engine is utilized to recommend appropriate captions based on the user's emotions. Finally, the completed photo book is saved as a PDF file and sent to a printing company. This allows users to efficiently create high-quality, unique, and creative photo books.

[1133] The processing flow will be explained below.

[1134] Step 1:

[1135] The user inputs the theme of the photo book using the terminal, for example, "The Four Seasons of Japan."

[1136] Step 2:

[1137] The device uses a camera and microphone to capture the user's facial expressions and voice data, which is then analyzed by an emotion engine to recognize emotions such as whether the user is relaxed, laughing, or sad.

[1138] Step 3:

[1139] The emotion engine suggests appropriate themes to the user based on the analysis results. For example, if the user is relaxed, it suggests themes such as "natural scenery," and if the user is happy, it suggests themes such as "happy memories."

[1140] Step 4:

[1141] The user can accept the suggested theme or select a different theme, for example, "Four Seasons of Japan."

[1142] Step 5:

[1143] The server automatically collects related image data from the Internet and internal databases based on the theme selected by the user. For example, a user can generate specific search queries such as "Spring in Japan," "Summer in Japan," "Autumn in Japan," and "Winter in Japan" to retrieve a large number of images.

[1144] Step 6:

[1145] The server generates new creative images using a generative adversarial network (GAN) model based on the collected image data. The GAN model extracts features from the collected image data and generates new images. For example, it generates new images of cherry blossoms from collected images of cherry blossoms.

[1146] Step 7:

[1147] The server transmits the generated creative image to the terminal.

[1148] Step 8:

[1149] The device receives the generated images, and the user arranges them in a layout using a dedicated photo book creation tool. The user decides the order of the images and adds captions to each one.

[1150] Step 9:

[1151] The emotion engine automatically suggests appropriate captions based on the user's emotions. For example, if the user is feeling relaxed, it will suggest a caption such as "Beautiful spring cherry blossom scenery."

[1152] Step 10:

[1153] The user reviews the captions and edits them if necessary.

[1154] Step 11:

[1155] Once your device has finished adjusting the layout, it will save your photo book as a PDF file with an automatically generated file name based on the theme you set.

[1156] Step 12:

[1157] The device sends the saved PDF file to the printing company. The user uploads the file through a dedicated sending interface and requests printing and publishing. At this time, the user can also specify print settings and the number of copies.

[1158] Example 2

[1159] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1160] In conventional photo book generation systems, each process, such as collecting image data appropriate for a theme, generating new creative images, and adding captions based on the user's emotions, is often performed manually, placing a heavy burden on the user.Furthermore, there are few systems that use emotion recognition to recommend themes, making it difficult to efficiently create personalized photo books for each user.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1162] In this invention, the server includes a means for a user to input a theme, a means for recognizing the user's emotions using an emotion engine and recommending a theme, a means for collecting image data based on the theme, a means for generating new creative images using a generative adversarial network based on the collected image data, a means for arranging the generated images in a layout and adding captions, a means for saving the completed layout as a PDF file, and a means for sending the saved PDF file to a printing service. This significantly reduces the user's workload and enables efficient creation of high-quality photo books personalized to each user.

[1163] "User" refers to the person who operates the system and inputs themes, lays out images, adds captions, etc.

[1164] "Theme" refers to a keyword or phrase that a user enters as the subject of the photo book.

[1165] An "emotion engine" refers to a system that analyzes a user's facial expressions and voice data to recognize and determine their emotions.

[1166] "Image data" refers to all image files collected by the server from the Internet or internal databases.

[1167] "Generative Adversarial Networks (GANs)" refers to machine learning models that generate new creative images based on collected image data.

[1168] "Layout" refers to an arrangement plan that defines how the generated images are arranged.

[1169] "Caption" refers to explanatory text or comments attached to each image.

[1170] "PDF file" refers to the electronic document format in which user-created photo books are saved.

[1171] "Printing services" refers to companies or services that take your completed PDF files and print and bind them into a physical photo book.

[1172] This system is designed to enable users to easily create a photo album. Detailed embodiments of the system will be described below.

[1173] A way for users to input the theme

[1174] The user uses the device to input the theme of the photo book. The device has an interface using a keyboard or touch screen, making it easy to input the theme. For example, the user can input "Four Seasons of Japan" to set the theme. This information is immediately sent to the server and used for the next process.

[1175] Theme recommendation using an emotion engine

[1176] The emotion engine has the ability to analyze the user's facial expressions and voice data. It captures the user's emotional data using the device's built-in camera and microphone. For example, if the user is in a relaxed state, a theme such as "nature scenery" will be automatically recommended. The emotion engine sends this data to a server and provides feedback to the user.

[1177] A means for the server to collect image data based on a theme

[1178] The server automatically collects related image data from the internet and its internal database based on the entered theme. Specifically, it generates search queries such as "Spring in Japan" and "Summer in Japan" for the theme "Four Seasons in Japan," and retrieves images using multiple APIs. This process also uses scraping technology. The collected image data is stored in a database on the server.

[1179] A means for the server to generate AI images based on collected image data

[1180] The server generates new creative images by inputting the collected image data into a generative adversarial network (GAN) model. For example, if an image of cherry blossoms is input into the GAN model, a more fantastical image of cherry blossoms will be generated. The generated images are stored on the server and used for the next process.

[1181] A means for the device to place generated images into a layout and provide captions

[1182] The generated image data is sent to the device, and the user can use the photo book creation tool to arrange the images in a layout. The user can arrange the images in the desired order and add captions to each one. Captions can be entered manually by the user, or automatically suggested based on data analyzed by the emotion engine. For example, a caption such as "Beautiful spring cherry blossom scenery" is provided.

[1183] A means for the device to save the completed layout as a PDF file

[1184] Once the user has completed the layout and captions, the device clicks the "Save" button, which generates the completed photo book as a PDF file and saves it to the device. The file name is automatically generated based on the theme, such as "Four Seasons of Japan.pdf."

[1185] A means for the device to send saved PDF files to a printing company

[1186] The completed PDF file is sent from the device to the printing service. The user uploads the file using a dedicated sending interface and specifies the number of prints and settings. When the "Send" button is clicked, the PDF file and setting data are sent to the printing service via the Internet, and the print request is completed.

[1187] Specific examples

[1188] For example, if a user wants to create a photo book with the theme "The Four Seasons of Japan," they can ask the system using the following prompt sentence:

[1189] I would like to create a photo book with the theme of "The Four Seasons of Japan." Please collect beautiful images related to the theme and generate new creative images. Please layout the collected images with captions that reflect my emotions. Then, please save the completed photo book as a PDF file and send it to a printing service.

[1190] This allows users to efficiently create high-quality photo albums without much effort.

[1191] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1192] Step 1:

[1193] The user inputs the theme of the photo book. The device receives instructions from the user to input the theme. For example, the user inputs "Four Seasons of Japan" into the device's input field and presses the enter key. This input data is sent from the device to the server and saved as the basic data for the next process.

[1194] Step 2:

[1195] The emotion engine recommends themes. The device uses a camera and microphone to capture the user's facial expressions and voice data. This input data is sent to the server, where the emotion engine analyzes it. For example, if the user is smiling, the system determines that the user is in a "relaxed state." Based on the results of this analysis, the server selects an appropriate theme, such as "natural scenery," and recommends it to the user.

[1196] Step 3:

[1197] The server collects image data based on a theme. The server generates search queries from the internet and its internal database. For example, if the theme is "The Four Seasons of Japan," it automatically generates queries such as "Spring in Japan" and "Summer in Japan" to collect related image data. This is done using API calls and scraping technology. The collected images are stored in the server's data storage.

[1198] Step 4:

[1199] The server generates AI images based on the collected image data. Specifically, the server inputs the collected image data into a generative adversarial network (GAN) model to generate new creative images. For example, by inputting image data of cherry blossoms, the GAN model generates a more fantastical image of cherry blossoms. The generated images are stored on the server.

[1200] Step 5:

[1201] The device arranges the generated images in a layout and adds captions. The generated image data is sent from the server to the device. The user then uses a dedicated photo book creation tool on the device to arrange the images in the layout. For example, an image of spring cherry blossoms could be placed on the first page, followed by an image of summer fireworks. The user then adds a caption to each image. Captions can also be automatically suggested based on data analyzed by the emotion engine.

[1202] Step 6:

[1203] The device saves the completed layout as a PDF file. When the user clicks "Save," the device generates a PDF version of the layout photo album and automatically generates a file name. For example, the file name might be "Four Seasons of Japan.pdf." This PDF file is saved on the device.

[1204] Step 7:

[1205] The device sends the saved PDF file to the printing company. The user uploads the PDF file from the device using a dedicated sending interface. Here, they select the print settings and number of copies to print and click "Send." The file and settings information are sent via the Internet to the printing service, and the print request is completed.

[1206] (Application example 2)

[1207] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1208] Conventional content generation systems only collect and generate images based on themes entered by the user, making it difficult to recommend content that takes into account the user's emotional state or provide content in story format. Furthermore, since they only use collected image data, they lack creativity. Furthermore, there are insufficient means to automatically save and distribute user-generated content in an appropriate format.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for a user to input a theme, means for collecting image data based on the theme, means for performing AI image generation based on the collected image data, means for arranging the generated images in a layout and adding captions, means for saving the completed layout as a PDF file, means for sending the saved PDF file to a printing company, means for analyzing the user's facial expressions and voice data to recognize emotions and recommending a theme based on the emotions, means for collecting related image and video data based on the recommended theme and generating new creative content, means for laying out the generated creative content in a story format and automatically adding appropriate captions and narration, and means for saving the completed content in a selectable format such as PDF or MP4 and providing it to the user. This enables theme recommendation based on the user's emotional state, the generation of creative story-format content, and the automatic saving and distribution of content.

[1210] The "means for the user to input a theme" refers to an interface that allows the user to input a theme for content generation using a smartphone, tablet, or other device.

[1211] The "means for collecting image data based on the theme" is a module for automatically collecting related image data from the Internet or a database according to the theme entered by the user.

[1212] The "means for generating AI images based on collected image data" is a system for generating new creative images from collected image data using a generative adversarial network (GAN) model.

[1213] The "means for arranging generated images in a layout and adding captions" is an interface that allows a user to intuitively layout generated images and add captions to each image.

[1214] The "means for saving the completed layout as a PDF file" is a function for saving the layout completed by the user as a PDF file.

[1215] "Means for sending saved PDF files to a printing company" refers to a function for sending saved PDF files online to a printing company and requesting printing and publishing.

[1216] "Means for analyzing a user's facial expressions and voice data to recognize emotions and recommend themes based on that" refers to a system that uses a camera and microphone to analyze a user's emotional state and automatically recommends the most appropriate theme based on the results.

[1217] "Means for collecting related image and video data based on the recommended theme and generating new creative content" refers to a function for collecting image and video data on the recommended theme and generating new creative images and videos using a GAN model.

[1218] "Means for laying out generated creative content in a story format and automatically adding appropriate captions and narration" refers to an interface that lays out generated content in a consistent format as a story and automatically adds captions and narration.

[1219] "Means of saving completed content in a selectable format such as PDF or MP4 and providing it to the user" refers to a function that allows completed content to be saved in a format selected by the user (e.g., PDF or MP4) and provided to the user.

[1220] This invention is an implementation method of a system in which a user inputs a theme, and based on that theme, image and video data is collected, analyzed, generated, and provided as creative content.

[1221] First, the device used by the user is equipped with an interface for entering the theme. This interface is implemented as an application that runs on the device, such as a smartphone or tablet. The user starts the process by entering the theme into this interface.

[1222] Next, the server automatically collects relevant image and video data from the internet and internal databases based on the theme entered by the user. This collection is done using API calls and scraping techniques. The collected data is then input into a generative adversarial network (GAN) model to generate new creative images and videos. A GAN model is an algorithm that extracts features from the input image and video data and generates new content based on them.

[1223] Furthermore, the server is equipped with an emotion engine that analyzes the user's facial expressions and voice data to recognize their emotions. Using a camera and microphone, it detects whether the user is smiling, sad, or relaxed, and recommends appropriate themes to the user based on the analysis results. The prompt at this time is "The emotion engine is analyzing the user's emotions."

[1224] The generated creative content is then sent back to the device. The user uses a dedicated photo book creation tool to lay out the generated images and videos in a story format and add captions and narration. Captions and narration are automatically suggested based on the user's emotions analyzed by the emotion engine. This allows users to create high-quality content intuitively and efficiently. An example prompt is "Laying out the generated creative content in a story format and adding captions."

[1225] Finally, the completed content is saved in the format selected by the user. For example, it can be saved as a PDF or MP4 file. After saving, the user can send these files over the Internet to a printing company for printing and publishing. Examples of prompts are "Saving content..." or "Save as PDF? (y / n)".

[1226] This invention is a system that implements a series of processes, from recommending themes based on the user's emotional state to generating creative content using a generative AI model. Compared to conventional technologies, this system enables individual optimization that is in line with the user's emotions, and can provide more attractive content.

[1227] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1228] Step 1:

[1229] The user inputs a theme into the terminal. In this example, the user inputs a theme into the application using a terminal such as a smartphone or tablet. The input theme might be, for example, "The Four Seasons of Japan."

[1230] Input: Theme (e.g. "Four Seasons of Japan")

[1231] Output: The input theme

[1232] Step 2:

[1233] The server collects image and video data based on the theme entered. The server collects image and video data related to the theme from the internet and internal databases. This process uses API calls and scraping technology.

[1234] Input: Theme (e.g. "Four Seasons of Japan")

[1235] Data processing: Collecting image and video data using API calls and scraping techniques

[1236] Output: Collected image and video data

[1237] Step 3:

[1238] The server performs AI image generation based on the collected image and video data, using a generative adversarial network (GAN) model to generate new creative images and videos based on the collected data.

[1239] Input: Collected image and video data

[1240] Data Computing: Novel Creative Image and Video Generation with GAN Models

[1241] Output: Generated creative images and videos

[1242] Step 4:

[1243] The device analyzes the user's facial expressions and voice data to recognize their emotions, and then uses the device's camera and microphone to analyze the user's emotions and recommend the most appropriate theme based on the analysis results.

[1244] Input: Facial expression data and voice data

[1245] Data Computing: Emotion Recognition with an Emotion Analysis Engine

[1246] Output: Recommended theme (e.g. "Nature scenes for a relaxed state")

[1247] Step 5:

[1248] Based on the recommended theme, the server again collects related image and video data to generate new creative content, which is then fed back into the GAN model to generate more creative content.

[1249] Input: Recommended themes, additional image and video data

[1250] Data processing and data computation: Recollection and new content generation using GAN models

[1251] Output: Additional generated creative content

[1252] Step 6:

[1253] The generated creative content is sent to the device, where the user can lay it out and add captions and narration. On the device, the user can use dedicated tools to arrange the generated images and videos in a story format and add captions and narration.

[1254] Input: Generated creative content

[1255] Data processing: User can create story-style layouts and add captions and narration

[1256] Output: Finished story-formatted content

[1257] Step 7:

[1258] The device will save the completed content in PDF or MP4 format, and the user can save the content in the format of their choice.

[1259] Input: Complete story-formatted content

[1260] Data Processing: Export to your chosen format (PDF or MP4)

[1261] Output: Saved PDF or MP4 file

[1262] Step 8:

[1263] The server sends the saved file to a printing company, and users can submit the file online to have it printed and published.

[1264] Input: Saved PDF or MP4 files

[1265] Data calculation: Send to printing company via online sending interface

[1266] Output: Confirmation of sending to printing company

[1267] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1268] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1269] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1270] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1271] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1272] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1273] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1274] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1275] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1276] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1277] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1278] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1279] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1280] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1281] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1282] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1283] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1284] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1285] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1286] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1287] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1288] The following is further disclosed regarding the above embodiment.

[1289] (Claim 1)

[1290] a means for a user to input a theme;

[1291] means for collecting image data based on the theme;

[1292] A means for generating AI images based on the collected image data;

[1293] means for placing the generated images in a layout and adding captions;

[1294] A way to save the completed layout as a PDF file,

[1295] The system includes a means for sending the saved PDF file to a printing company.

[1296] (Claim 2)

[1297] The system of claim 1, wherein a new image is generated using a generative adversarial network based on the collected image data.

[1298] (Claim 3)

[1299] The system according to claim 1, wherein a photo book is created based on the generated images and captions are added.

[1300] "Example 1"

[1301] (Claim 1)

[1302] a means for a user to input a theme;

[1303] means for collecting image data based on the theme;

[1304] a means for generating new images using a generative adversarial network based on the collected image data;

[1305] means for placing the generated images in a layout and adding captions;

[1306] A means for saving the completed layout as an electronic file;

[1307] The system includes a means for transmitting the stored electronic file to a printing company.

[1308] (Claim 2)

[1309] 10. The system of claim 1, wherein the system generates prompt sentences based on the collected image data and generates new images using a generative adversarial network.

[1310] (Claim 3)

[1311] The system according to claim 1, wherein a photo book is created based on the generated images and captions are added.

[1312] "Application Example 1"

[1313] (Claim 1)

[1314] a means for a user to input a theme;

[1315] means for collecting image data based on the theme;

[1316] A means for generating AI images based on the collected image data;

[1317] means for placing the generated images in a layout and adding captions;

[1318] A way to save the completed layout as a PDF file,

[1319] means for transmitting the saved PDF file to an output device;

[1320] a means for inputting a theme via a user interface and previewing it in real time;

[1321] A means for editing the generated image on a smart device;

[1322] A system including:

[1323] (Claim 2)

[1324] The system of claim 1, wherein a new image is generated using a generative adversarial network based on the collected image data.

[1325] (Claim 3)

[1326] The system according to claim 1, wherein a photo book is created based on the generated images and captions are added.

[1327] "Example 2: Combining Emotion Engines"

[1328] (Claim 1)

[1329] a means for a user to input a theme;

[1330] a means for recognizing a user's emotions using an emotion engine and recommending themes;

[1331] means for collecting image data based on the theme;

[1332] A means for generating new creative images using a generative adversarial network based on the collected image data;

[1333] means for placing the generated images in a layout and adding captions;

[1334] A way to save the completed layout as a PDF file,

[1335] The system includes a means for sending the saved PDF file to a printing service.

[1336] (Claim 2)

[1337] 10. The system of claim 1, wherein the system generates new images using a generative adversarial network based on the collected image data and an emotion engine.

[1338] (Claim 3)

[1339] The system of claim 1, which creates a photo book based on the generated images and adds captions using an emotion engine.

[1340] "Application example 2 when combining emotion engines"

[1341] (Claim 1)

[1342] a means for a user to input a theme;

[1343] means for collecting image data based on the theme;

[1344] A means for generating AI images based on the collected image data;

[1345] means for placing the generated images in a layout and adding captions;

[1346] A way to save the completed layout as a PDF file,

[1347] A way to send the saved PDF file to a printing company,

[1348] A method for recognizing emotions by analyzing the user's facial expressions and voice data and recommending themes based on those emotions;

[1349] a means for collecting relevant image and video data based on the recommended theme and generating new creative content;

[1350] A means to lay out the generated creative content in a story format and automatically add appropriate captions and narration;

[1351] The system includes a means to save the completed content in a selectable format such as PDF or MP4 and provide it to the user.

[1352] (Claim 2)

[1353] The system of claim 1, further comprising: generating new images and videos using a generative adversarial network based on collected image and video data.

[1354] (Claim 3)

[1355] 10. The system of claim 1, wherein the system creates a photo book or video story based on the generated images and videos, and provides captions and narration. [Explanation of symbols]

[1356] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input a theme; means for collecting image data based on the theme; A means for generating AI images based on the collected image data; means for placing the generated images in a layout and adding captions; A way to save the completed layout as a PDF file, The system includes a means for sending the saved PDF file to a printing company.

2. The system of claim 1 , wherein a new image is generated based on collected image data using a generative adversarial network.

3. The system according to claim 1, wherein a photo book is created based on the generated images and captions are added.

Images