System

Abstract

A system is provided.SOLUTION: A system comprising: means for electronically scanning and storing books as digital data; means for analyzing collected book data and extracting categories, abstracts, and key topics; means for inputting a user's favorite genre and past reading history; means for analyzing information input by a user and creating a profile; means for recommending books based on a user profile; means for displaying recommended books; means for providing books in multiple languages by automatic translation; means for automatically generating illustrations and videos; and means for converting stories of novels into video contents.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, the decline in bookstores and the spread of e-commerce have reduced opportunities for individuals to find books that suit them. Furthermore, online advertisements are designed to promote specific books, and personalized recommendations tailored to users' preferences are lacking. As a result, people are increasingly turning away from reading, narrowing their imagination and knowledge. Therefore, there is a need for a system that can easily help users find books that suit them and improve their reading experience. [Means for solving the problem]

[0005] In this invention, we propose a system that uses AI technology to recommend books suitable for a user in a personalized manner. This system includes the following means:

[0006] 1. A method for electronically scanning books and storing them as digital data

[0007] 2. A means of analyzing collected book data and extracting categories, summaries, and major topics.

[0008] 3. A way for users to input their preferred genres and past reading history

[0009] 4. Means of analyzing information entered by users and creating profiles

[0010] 5. A way to recommend books based on user profiles

[0011] 6. How to display recommended books

[0012] 7. A way to provide books in multiple languages ​​through machine translation

[0013] 8. Means of automatically generating illustrations and videos

[0014] 9. How to turn a novel story into video content

[0015] These methods allow users to easily find books that suit them and enrich their reading experience. In addition, if a book is purchased based on a recommendation, a revenue share will be provided to the rights holder, thereby revitalizing the publishing industry as a whole. By continuously analyzing users' reading habits and dynamically updating their profiles, it is possible to always provide the most optimal recommendations.

[0016] "Electronically scanned" means converting a physical book into digital data using image processing techniques.

[0017] "Digital data" means information that has been converted into a format that can be processed and stored by electronic devices.

[0018] "Analysis" refers to information processing to extract meaning and characteristics from digital data.

[0019] A "category" is a label or grouping for classifying books or data based on similarities or commonalities.

[0020] A summary is a short summary of the main text, extracting the most important parts.

[0021] "Major topics" refers to the central theme or themes in the book or document.

[0022] A "user profile" is an individual information record created based on a user's preferences and behavioral history.

[0023] "Recommendations" are suggestions for books and content based on a user's profile.

[0024] "Machine translation" is the automatic conversion of texts into different languages ​​using AI technology.

[0025] "Illustrations" are images or illustrations inserted to visually assist the content of a book.

[0026] "Video content" means expressing the story and content of a book in video format.

[0027] "Revenue share" refers to the distribution of sales to relevant rights holders and distributors. [Brief explanation of the drawings]

[0028] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0029] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0032] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0033] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0034] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] [First embodiment]

[0037] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0038] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0039] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0040] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0041] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0042] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0043] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0044] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0045] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0046] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0047] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0048] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0049] The present invention is a system for personalized book recommendations to users. This system utilizes AI technology to collect and analyze book data and provide recommendations tailored to the user's preferences. Specific embodiments of this system are described below.

[0050] System Overview

[0051] This system consists of three entities: a server, a terminal, and a user. The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations, while the terminal receives input from the user and displays the recommendations. The user provides their preferred genres and reading history through the terminal, and selects and purchases books displayed as recommended results.

[0052] Program processing flow (natural language explanation)

[0053] Step 1: Electronic scanning and digitization of the book

[0054] Server: Electronically scans books and stores them as digital data, making the physical books available in digital form.

[0055] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[0056] Server: Analyzes the content of digitized books and automatically extracts categories, summaries, and major topics. This information is stored in a database.

[0057] Step 3: Collecting user preferences

[0058] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is sent to the server as a user profile.

[0059] Step 4: Analyzing user profiles

[0060] Server: Analyzes the user information sent and creates a detailed user profile, which allows the user to understand their reading habits and interests.

[0061] Step 5: Generate book recommendations

[0062] Server: Based on the created user profile, the server selects the most suitable books from the database and generates recommendations.

[0063] Step 6: Displaying recommendations

[0064] Terminal: Displays the recommendation results received from the server to the user, who can then select books that interest them.

[0065] Step 7: Automated translation and additional content provision

[0066] Server: If necessary, the recommended books are automatically translated and provided in multiple languages. In addition, illustrations and related videos are automatically generated using AI and provided as additional content.

[0067] Step 8: Turning the story of your novel into video content

[0068] Server: Analyzes the story of the book selected by the user and generates a short video, which the user can enjoy visually.

[0069] Step 9: Buy the book and complete the revenue share

[0070] User: Purchase the recommended book.

[0071] Server: Records purchase information and performs revenue share with rights holders.

[0072] Specific examples

[0073] For example, suppose a user is looking for books in the mystery genre. The user logs into the system through a terminal and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery novel from the database, and generates a recommendation result. The recommendation result is displayed on the terminal, and the user can select the book of interest from among them and purchase it. Once the purchase is completed, the server records the information and distributes the sales revenue appropriately to the distributor and rights holder.

[0074] If a recommended book is written in a foreign language, the server will use an automatic translation function to provide a Japanese version. Furthermore, illustrations and videos can be automatically generated to enhance the reader's visual enjoyment. The story of the selected novel is also provided to the user as a short video, allowing users to enjoy the content in a variety of ways.

[0075] In this way, the system of the present invention recommends optimal books to the user in a consistent manner, providing a rich reading experience.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[0079] Step 2:

[0080] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[0081] Step 3:

[0082] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is necessary to create a user profile.

[0083] Step 4:

[0084] Terminal: Sends information entered by the user to the server.

[0085] Step 5:

[0086] Server: Analyzes the received user information and creates a user profile based on the user's reading habits and interests.

[0087] Step 6:

[0088] Server: Based on the user profile, a recommendation algorithm is used to select the most suitable books and generate a recommendation list.

[0089] Step 7:

[0090] Terminal: Displays the recommendation results received from the server to the user, who can then select books of interest from this list.

[0091] Step 8:

[0092] User: Select a book from the recommendation list via the device and purchase or preview it.

[0093] Step 9:

[0094] Server: If the book selected by the user is in a foreign language, it uses an automatic translation function to generate a Japanese version and sends it to the terminal.

[0095] Step 10:

[0096] Server: Using AI technology, illustrations and videos related to the selected books are automatically generated and provided as additional content.

[0097] Step 11:

[0098] Server: Furthermore, it analyzes the story of the novel and generates a short video, which is also provided to the user.

[0099] Step 12:

[0100] User: Purchases books that interest them. After purchase, the books are available as e-books.

[0101] Step 13:

[0102] Server: Records purchase information and distributes revenue to relevant rights holders and distributors. This process is automated.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Current book recommendation systems lack the ability to provide personalized recommendations that reflect users' preferences and reading history, and they face challenges in providing multilingual and visual content. Furthermore, the process of digitizing and analyzing books is manual and inefficient. Furthermore, the post-purchase revenue sharing mechanism is unclear, making it difficult to fairly distribute rewards to rights holders.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes a means for electronically scanning physical books and storing them as digital data, a means for analyzing the collected book data and extracting categories, summaries, and major topics, and a means for recommending books based on user profiles. This enables highly accurate book recommendations that reflect users' preferences and reading history, and also enables multilingual and visual content provision. A post-purchase revenue sharing mechanism is also established, enabling fair reward distribution.

[0108] "Physical Book" means a book in a non-electronic format made of paper or other material.

[0109] "Electronically scanned" refers to the process of converting a physical book into digital data using a device such as a scanner.

[0110] "Digital data" refers to information stored in an electronically processable form.

[0111] "Analysis" refers to the process of automatically extracting meaningful information from digital data.

[0112] A "category" refers to a group of books classified based on their content or theme.

[0113] A "summary" is a brief summary of the book's contents.

[0114] "Major topics" refers to important themes or topics contained in the book's content.

[0115] "Input means" refers to the method or interface by which a user provides information to a system.

[0116] A "profile" refers to a collection of data that reflects a user's preferences and behavior.

[0117] "Recommendation method" refers to the process of selecting and providing books based on a user's profile.

[0118] The "means for displaying" refers to an interface that visually presents the recommended book information to the user.

[0119] "Machine translation" refers to technology that automatically converts text between multiple languages.

[0120] "Means for automatically generating illustrations and videos" refers to the process of using artificial intelligence technology to automatically create visual content related to the content of a book.

[0121] "Video content creation" refers to the process of visualizing the story of a book and providing it in video format.

[0122] "Revenue share" refers to a mechanism for fairly distributing revenue.

[0123] "Means for dynamically updating reading habits" refers to technology that automatically updates a user's profile based on their new reading history and preferences.

[0124] MODE FOR CARRYING OUT THE INVENTION

[0125] This invention is a system that recommends the most suitable books to users in a personalized manner. Specifically, it analyzes electronically digitized book data and provides recommendations that match the user's preferences. This system is mainly composed of three entities: a server, a terminal, and a user.

[0126] The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations. The terminal receives input from the user and displays the recommendations. The user provides their preferred genres and past reading history through the terminal, and selects and purchases the books displayed as recommended results.

[0127] The server first uses a dedicated scanner (e.g., Fujitsu ScanSnap iX1500) to electronically scan the physical book and store it as digital data. This digital data is then converted into text data using OCR technology (e.g., Tesseract OCR). The converted text data is then saved in the server's storage.

[0128] The server then uses natural language processing tools (e.g., Apache OpenNLP) to analyze the text data from the digitized books, including tokenization, part-of-speech tagging, and named entity extraction, and stores categories, summaries, and major topics in a database (e.g., MySQL).

[0129] Users log in to the system using a terminal (e.g., a browser-based interface) and enter their preferred genres and past reading history, such as "mysteries" or "a list of books read in the past three years." The terminal checks the format of this input data in real time and sends it to the server as well-formed data.

[0130] The server then uses Node.js and Python scripts to analyze the received user data and generate detailed user profiles, which include information that classifies and predicts the user's reading habits using machine learning algorithms (e.g., k-means clustering), and store this information in a database.

[0131] The server then uses an AI model (e.g., TensorFlow) to select the most suitable books from the database based on the generated user profile. An example prompt might be, "Recommend the most suitable mystery novel based on the user's past reading history and preferred genres. Include summaries, main topics, and categories in the recommendation. Examples include 'famous detective series' and 'famous mystery novel series.'"

[0132] The recommendation results sent from the server are then displayed on the device using a web application using React.js, allowing users to check detailed information about the recommended books (summary, category, main topics).

[0133] If the recommended book is in a foreign language, the server will automatically translate it using the Google Translate API. It can also use OpenAI's DALL-E API to automatically generate illustrations and video content, allowing users to enjoy the book's contents in a variety of ways.

[0134] Furthermore, when a purchase is made, the server uses the Stripe API to record the purchase information and distribute the appropriate revenue share to the rights holder. In this way, the system provides users with personalized book recommendations, enhancing their reading experience.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1:

[0137] Electronic scanning and digitization of books

[0138] Server: Electronically scans physical books using a specialized scanner. It accepts the physical book as input and produces scanned image data as output, thereby capturing the contents of the book in image format.

[0139] Server: Converts scanned image data into text data using OCR technology. The input is image data and the output is text data. Specifically, OCR technology performs character recognition and extracts text information from the image.

[0140] Step 2:

[0141] Data analysis of books and extraction of categories, summaries, and main topics

[0142] Server: Analyzes text data using natural language processing tools. The input is text data, and the output is the analysis results: categories, summaries, and main topics. This converts the text data into meaningful information.

[0143] Server: Stores the analyzed data in a database. Specifically, it extracts categories, summaries, and main topics from the text data and inserts them into a MySQL database.

[0144] Step 3:

[0145] Collecting user preference information

[0146] User: Logs into the system through a terminal and inputs his / her preferred genre and past reading history. The input provides his / her preferred genre (e.g., mystery, science fiction) and past reading history. The output is the formatted user input data.

[0147] Terminal: Checks the format of the user's input data in real time and sends it to the server as well-formed data. Specific operations include checking the input of required fields and validating the data format.

[0148] Step 4:

[0149] User profile analysis

[0150] Server: Analyzes the received user data and generates a detailed user profile. The input is the user's preferences, and the output is a detailed user profile. This digitizes the user's reading habits and hobbies. Specifically, it runs a machine learning algorithm using Node.js and Python scripts.

[0151] Step 5:

[0152] Book recommendation generation

[0153] Server: Based on the generated user profile, an AI model is used to select the most suitable books. The input is a detailed user profile, and the output is a list of recommended books. This allows the most suitable books to be selected for the user. Specifically, TensorFlow is used to process the profile data and extract the books that best fit the user profile.

[0154] Step 6:

[0155] Displaying recommendations

[0156] Terminal: Displays the recommendation results received from the server to the user. The input is a list of recommended books, and the output is a visual display of the book information. This allows the user to check the list of recommended books. Specifically, the display is done using a web application using React.js.

[0157] Step 7:

[0158] Automatic translation and additional content provision

[0159] Server: Uses the Google Translate API to automatically translate recommended books. The input is the text data of the recommended book, and the output is the translated text data. This allows users to enjoy the book in their native language. Specifically, the API is called to obtain the translation results.

[0160] Server: Generates illustrations and video content using OpenAI's DALL-E API. The input is the book's content data, and the output is the generated visual content. This allows for a deeper understanding of the book. Specifically, the server calls the API to generate related illustrations and videos.

[0161] Step 8:

[0162] Turning novel stories into video content

[0163] Server: Analyzes the story of a book selected by the user and generates it as a short video. The input is the book's story data, and the output is a short video. This allows users to enjoy the story visually. Specifically, the video is created using Adobe Premiere Pro and an AI video generation tool.

[0164] Step 9:

[0165] Buying books and running revenue shares

[0166] User: Purchases the recommended book using a device. The input is a purchase intention and payment information, and the output is a purchase confirmation and a download link for the book. The book then becomes the user's property. Specific actions include paying with a credit card or electronic money.

[0167] Server: Records purchase information and performs revenue sharing using the Stripe API. The input is purchase information, and the output is confirmation of revenue distribution to rights holders. This ensures fair revenue distribution. Specific operations include storing purchase data and distributing revenue at the appropriate rate.

[0168] (Application example 1)

[0169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] Conventional book recommendation systems are generally intended for online use, and when users visit a bookstore, there are limited ways to obtain book information on the spot. This makes it difficult for users to find books that suit their preferences, and the purchasing experience is not sufficiently improved. In addition, differences in reading devices and the lack of dynamic updates to user profiles make it difficult to provide appropriate recommendations in real time.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0172] In this invention, the server includes a means for reading a code installed in the store, a means for displaying in-store book information in real time based on the read code, and a means for recommending books in real time based on a user profile using a generative AI model, thereby enabling a user to receive optimal book recommendations in real time when searching for books in the store.

[0173] "Means for electronically scanning books and storing them as digital data" refers to a function for converting books into digital data using equipment such as an electronic scanner or camera, and storing the data on a server or database.

[0174] "Means of analyzing collected book data and extracting categories, summaries, and key topics" refers to a function that analyzes the contents of a book using natural language processing technology, etc., and automatically extracts the genre to which the book belongs, a summary of the contents, and important topics.

[0175] "Means for users to input their favorite genres and past reading history" refers to an interface or mechanism that allows users to input information about their interests and the books they have read.

[0176] "Means for analyzing information entered by the user and creating a profile" refers to a function that analyzes information on genres and reading history provided by the user and generates a profile that reflects the user's reading tendencies and preferences.

[0177] A "means for recommending books based on a user profile" is an algorithm or system for recommending the most suitable book to a user based on the generated user profile.

[0178] The "means for displaying recommended books" refers to a display or a screen of a mobile device that visually presents information about recommended books to the user.

[0179] "Means for providing books in multiple languages ​​through machine translation" is a function that translates the contents of recommended books into the user's desired language through machine translation, making them available in multiple languages.

[0180] "Means for automatically generating illustrations and videos" refers to a function that uses artificial intelligence and image generation technology to automatically generate related illustrations and videos based on the content of a book.

[0181] "A means of turning a novel story into video content" is a function that analyzes the content of a novel and generates video content such as a short film or animation based on that story.

[0182] "Means for reading codes installed in stores" refers to a function that allows you to read codes installed in stores, such as QR codes and barcodes, using a smartphone or dedicated device.

[0183] "Means for displaying in-store book information in real time based on the scanned code" is a function for displaying detailed information about books in the store in real time on the user's device based on the scanned QR code or barcode.

[0184] "Means of recommending books in real time based on user profile using a generative AI model" is a function that uses an artificial intelligence model to recommend the most suitable book on the spot based on the user's profile information.

[0185] The present invention is a system for recommending the most suitable book to a user in a personalized manner, and is mainly composed of a server, a terminal, and a user. Specific embodiments for implementing this system will be described below.

[0186] 1. Server operation

[0187] The server has the following roles:

[0188] 1. Electronic scanning and digital storage of books:

[0189] The server scans the books using an electronic scanner or camera and stores the content as digital data using cloud storage services such as AWS S3 and Firebase Storage.

[0190] 2. Data Analysis:

[0191] Analyze the content of digitized books and extract categories, summaries, and key topics using natural language processing techniques, leveraging Python libraries (e.g., spaCy and NLTK) and AI models (e.g., BERT).

[0192] 3. Create a user profile:

[0193] The server analyzes the data provided by the user (preferred genres, past reading history) and creates a detailed user profile, which is stored in a cloud database (e.g., Firebase Firestore).

[0194] 4. Recommendation Generation:

[0195] Generate book recommendations based on user profiles using generative AI models (e.g., GPT-3) and integrating multiple recommendation algorithms.

[0196] 5. Automatic translation and additional content generation:

[0197] The content of the book is automatically translated as needed and provided in multiple languages. In addition, illustrations and related videos are automatically generated using a generative AI model and provided to users.

[0198] 6. Turning novel stories into video content:

[0199] The story of the book selected by the user is generated as video content. This step uses generative AI models and machine learning techniques.

[0200] 7. Revenue Share Implementation:

[0201] When a book is purchased based on a recommendation, the purchase information is recorded and the sales revenue is distributed to the rights holder.

[0202] 2. Device operation

[0203] The device has the following features:

[0204] 1. Data input from the user:

[0205] The device provides an interface for users to input their preferred genres and past reading history.

[0206] 2. Displaying recommendation results:

[0207] The recommendation results received from the server are displayed to the user using a mobile device such as a smartphone or tablet.

[0208] 3. QR Code Scanning:

[0209] Read QR codes and barcodes installed in stores to obtain book information. To do this, use a QR code reader library (e.g., Zxing).

[0210] 4. Information display based on the code read:

[0211] Based on the code read, the terminal displays detailed information about the book to the user in real time.

[0212] 3. User operations

[0213] The user performs the following operations:

[0214] 1. Enter the data:

[0215] Users input their preferences and past reading history through the device.

[0216] 2. Check the book recommendations:

[0217] Check the recommendations and choose the books that interest you.

[0218] 3. Scan the QR code:

[0219] Scan the QR code of a book you are interested in in the store to view detailed information.

[0220] 4. Buy a book:

[0221] Purchase recommended books or books that interest you.

[0222] Specific examples

[0223] For example, suppose a user visits a physical store and is looking for a book in the mystery genre. The user logs in using a smartphone app and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery book from the database, and generates a recommendation result. When the user scans the QR code in the store, information about the book is displayed in real time, along with additional content (e.g., illustrations and related videos).

[0224] Examples of prompts for AI models

[0225] "Generate a summary of a mystery novel in which a detective solves a complex case. Include the main plot and key details."

[0226] As a result, this system can recommend optimal books to users in a consistent manner, enriching the purchasing experience in physical stores.

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1:

[0229] The server electronically scans the book and stores it as digital data. The book is scanned using a scanner or camera to generate image data. This is then converted into text data using OCR software and stored in cloud storage. The input to this process is the physical book, and the output is digital text data.

[0230] Step 2:

[0231] The server analyzes the collected digital data of books and extracts categories, summaries, and main topics. It uses Python libraries and natural language processing techniques (e.g., spaCy and NLTK) to analyze the text data and extract the necessary information. The input of this process is the digital text data generated in step 1, and the output is categories, summaries, and main topics.

[0232] Step 3:

[0233] Users input their preferred genres and past reading history using a device. The input interface is a smartphone or tablet, and the user's input data is stored in a cloud database. The input for this process is the genre and reading history data entered by the user, and the output is the user profile data stored in the cloud database.

[0234] Step 4:

[0235] The server analyzes the user's input and creates a detailed user profile. Based on the user data stored in the cloud, an analytical algorithm is used to analyze the user's reading habits. The input to this process is the user data stored in the cloud database, and the output is a detailed user profile.

[0236] Step 5:

[0237] The server uses a generative AI model (e.g., GPT-3) to recommend the most suitable books in real time based on the user profile. The server matches the user profile with book information stored in a database and generates recommendations using the AI ​​model. The input to this process is a detailed user profile and a book database, and the output is a recommendation result.

[0238] Step 6:

[0239] The terminal displays the recommendation results to the user. Recommended book information is displayed on the screen of a smartphone or tablet. The input to this process is the recommendation results received from the server, and the output is recommended book information visually presented to the user.

[0240] Step 7:

[0241] A user reads a QR code attached to a book in a store. The user then scans the QR code using the camera on their smartphone to obtain the book information. The input to this process is the QR code attached to the book, and the output is the book information read from the QR code.

[0242] Step 8:

[0243] The terminal displays in-store book information in real time based on the read QR code information. Detailed book information is displayed on the terminal based on the information obtained from the QR code. The input for this process is the data read from the QR code, and the output is book information displayed in real time.

[0244] Step 9:

[0245] The server generates automatic translations and additional content (illustrations and videos) as needed. A generative AI model is used for translation, and image generation AI such as DALL-E is used to generate illustrations and videos. The input for this process is recommendation results and book content data, and the output is the translated text and generated illustrations and video content.

[0246] Step 10:

[0247] When a user purchases a book, the server executes the revenue share. It records the purchase information and distributes the sales appropriately to the rights holder. The input of this process is the user's purchase information, and the output is the result of the revenue share execution.

[0248] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0249] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[0250] System Overview

[0251] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0252] Program processing flow (natural language explanation)

[0253] Step 1: Electronic scanning and digitization of the book

[0254] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[0255] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[0256] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[0257] Step 3: Collecting user preferences

[0258] User: Logs into the system through a terminal and enters his / her preferred genres and past reading history.

[0259] Step 4: Submit user information

[0260] Terminal: Sends information entered by the user to the server.

[0261] Step 5: Create a user profile

[0262] Server: Analyzes the received user information and creates a detailed user profile.

[0263] Step 6: Emotion Recognition with the Emotion Engine

[0264] Server: Analyzes real-time emotion data collected from the user's device using an emotion engine to recognize the user's emotional state.

[0265] Step 7: Emotion-based recommendation optimization

[0266] Server: Optimizes recommendations based on user profile and emotional state, choosing books that fit the user's current emotions.

[0267] Step 8: Generate and display the recommendation list

[0268] Server: Generates an optimized recommendation list.

[0269] Terminal: Displays the recommendation results received from the server to the user.

[0270] Step 9: Select and purchase books

[0271] User: Select a book from the recommended list and purchase or preview it.

[0272] Step 10: Automated translation and additional content provision

[0273] Server: If the selected book is in a foreign language, an automatic translation function is used to generate a Japanese version and send it to the device. In addition, AI technology is used to automatically generate illustrations and videos, which are provided as additional content.

[0274] Step 11: Turn your story into video content

[0275] Server: Generates the story of the novel as a short video and provides it to the user.

[0276] Step 12: Record purchase information and execute revenue share

[0277] Server: Records purchase information and executes revenue share with relevant rights holders and distributors.

[0278] Specific examples

[0279] For example, suppose a user is looking for a romance novel. The user logs into the system through their device and enters their desired romance and past reading history. Furthermore, the device's camera and sensors are used to transmit the user's emotional data in real time to the server. The server then creates a user profile based on this information and uses an emotion engine to analyze the user's current emotional state.

[0280] Because the user is feeling stressed, the emotion engine prioritizes relaxing romance novels and adds them to the recommendation list. From the recommendation list displayed on the device, the user can select a book that interests them and purchase it. If the selected book is in a foreign language, the server uses an automatic translation function to provide a Japanese version. Users can also view illustrations and short videos generated by AI.

[0281] In this way, the system of the present invention, combined with the emotion engine, provides advanced recommendations that take into account the user's emotional state, enabling a rich reading experience.

[0282] The processing flow will be explained below.

[0283] Step 1:

[0284] Server: The book is scanned electronically and stored as digital data in a database on the server. In this step, OCR technology is used to extract text information for each page and store it as metadata.

[0285] Step 2:

[0286] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Using natural language processing technology, it extracts themes and keywords from the content and organizes the database based on these.

[0287] Step 3:

[0288] User: Logs in to the system using a terminal and enters information such as preferred genres and past reading history, which allows the collection of basic information for a user profile.

[0289] Step 4:

[0290] Terminal: Sends information entered by the user to the server, which uses it to create and update the user profile.

[0291] Step 5:

[0292] Server: Analyzes the received user information and creates a detailed user profile, including data about the user's reading habits and interests.

[0293] Step 6:

[0294] On-device: Using the device's camera and sensors, we collect real-time emotional data from the user. This data is obtained through facial recognition and biometric sensors.

[0295] Step 7:

[0296] Terminal: Sends collected emotional data to the server, which uses it to analyze the user's emotional state.

[0297] Step 8:

[0298] Server: The emotion engine analyzes the transmitted emotion data and recognizes the user's current emotional state. For example, it identifies whether the user is happy, sad, or stressed.

[0299] Step 9:

[0300] Server: Based on the user profile and emotional state, the recommendation algorithm selects the most suitable book and optimizes the recommendation. The selection criteria are dynamically adjusted taking into account the emotion recognition results.

[0301] Step 10:

[0302] Server: Generates an optimized recommendation list and sends it to the device. The recommendation list includes books that are appropriate for the user's emotional state.

[0303] Step 11:

[0304] Terminal: The terminal displays the recommendation results received from the server to the user. The user can select a book of interest from the recommendation list and view detailed information and samples.

[0305] Step 12:

[0306] User: Can select a book from the recommended list and make a purchase. Once purchased, the book will be available as an e-book.

[0307] Step 13:

[0308] Server: If the selected book is in a foreign language, it will use the automatic translation function to provide the Japanese version. This translation process runs in the background.

[0309] Step 14:

[0310] Server: Using AI technology, it automatically generates illustrations and videos related to the selected book and sends them to the device as additional content, enriching the reader's visual experience.

[0311] Step 15:

[0312] Server: The server then generates a short video of the story and provides it to the user. In this step, the main parts of the story are visualized to provide visual enjoyment.

[0313] Step 16:

[0314] Server: Records purchase information and executes revenue sharing with relevant rights holders and distributors. This process is performed automatically based on sales information.

[0315] Example 2

[0316] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] In today's information overload, it is difficult for users to find the book that best suits them. Advanced filtering technology is required to select books that best suit the user's preferences and emotional state from the vast number of books available. Furthermore, a system that can smoothly provide foreign language books and additional content is also needed.

[0318] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0319] In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres and past reading history, means for analyzing the information input by the user and creating a profile, means for recommending books based on the user profile, means for displaying recommended books, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, means for converting novel stories into video content, means for collecting real-time emotional data, means for analyzing the emotional data and recognizing the user's emotional state, and means for recommending books based on the user's emotional state. This allows the server to provide optimal books based on the user's preferences and emotional state, enabling a rich reading experience.

[0320] "Means of electronically scanning books and storing them as digital data" refers to a method of electronically reading paper books using a scanner and storing the data in a database as text data.

[0321] "Methods for analyzing collected book data and extracting categories, summaries, and main topics" refers to a method of analyzing the contents of digitized books using OCR technology and natural language processing technology, and extracting the categories to which the books belong, summaries, and main points of discussion.

[0322] "Means for users to input their favorite genres and past reading history" refers to an interface that allows users to provide the system with their favorite genres and a list of books they have read so far.

[0323] "Means for analyzing information entered by the user and creating a profile" refers to a method for analyzing information provided by the user, such as genre and reading history, using machine learning models to generate a detailed user profile.

[0324] The "means for recommending books based on a user profile" is a method for providing a list of books that are most suitable for each individual user based on the generated user profile.

[0325] The "means for displaying recommended books" is an interface for displaying the recommendation list sent from the server on the user's terminal.

[0326] "Means for providing books in multiple languages ​​through automatic translation" refers to a method in which, if the selected book is in a foreign language, the book is translated into another language using automatic translation technology and provided to the user.

[0327] "Means for automatically generating illustrations and videos" refers to a method of using AI technology to generate illustrations and short videos based on the content of a book and provide them to users as additional content.

[0328] "Means for turning a novel story into video content" refers to a method of using a video generation algorithm to generate and provide a short video using the contents of a novel as input data.

[0329] "Means for collecting real-time emotional data" refers to an interface that uses cameras and sensors to obtain emotional data in real time from the user's facial expressions, voice, etc.

[0330] The "means for analyzing emotion data and recognizing the user's emotional state" is an emotion recognition algorithm for analyzing the collected emotion data and recognizing the user's current emotional state.

[0331] The "means for recommending books based on the user's emotional state" is a method for combining a user profile and an emotional state to recommend books that suit the user's current emotions.

[0332] The present invention is an advanced system that recommends optimal books to a user, recognizes the user's emotional state, and optimizes the recommendations. Hereinafter, a detailed description will be given of an embodiment of the present invention.

[0333] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0334] First, the server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap), extracts the text information using OCR technology (e.g., Tesseract), and stores it in a database. This step generates digital data.

[0335] The server then analyzes the collected digital data using algorithms based on natural language processing technology (e.g., SpaCy or NLTK) to extract book categories, summaries, and main topics.

[0336] Users log in to the system through their devices and enter their favorite genres and past reading history. iOS or Android applications are used on the devices. The information entered by the user is sent to the server using the HTTPS protocol. The server analyzes the information received and creates a detailed user profile using machine learning models (e.g., Scikit-learn or TensorFlow).

[0337] Furthermore, the user's real-time emotional data is collected by the emotion engine. The emotional data acquired using the device's camera and sensors is analyzed by an emotion recognition API (e.g., AWS Rekognition or Microsoft Azure Emotion API). The server recognizes the user's emotional state based on the analysis results.

[0338] The server then optimizes the recommendations based on the user's profile and current emotional state. A reinforcement learning algorithm is used to select books that fit the emotional state. An optimized recommendation list is then generated and sent to the device, which displays the list in the user interface.

[0339] Users can select books of interest from the recommendation list and purchase or preview them. The purchase process is carried out through an e-commerce system (e.g., Amazon or Rakuten). If the selected book is in a foreign language, an automatic translation function (e.g., Google Translate API) is used to generate a Japanese version. Furthermore, illustrations and videos are automatically generated using AI technology (e.g., DALL-E or GAN) and provided as additional content.

[0340] As a concrete example, let's say a user is looking for a romance novel. The user logs into the system through their device and enters their romance and past reading history. Emotional data is sent to the server in real time via the device's camera and sensors. The server creates a user profile based on this information, and an emotion engine analyzes the user's current emotional state. If the emotion engine determines that the user is feeling "stressed," it will prioritize recommendations of relaxing romance novels.

[0341] In this way, advanced recommendations based on the user's emotional state and preferences are realized.

[0342] Example prompt sentence:

[0343] "Recommend positive, relaxing romance novels based on the user's reading history and real-time emotional data. Users are stressed and want content that will help them feel better."

[0344] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0345] Step 1:

[0346] The server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap). The input is the paper book, and the output is the scanned image data. This image data is converted into text information using OCR technology (e.g., Tesseract) and stored in a database. The specific operation is "start the scanner, place the book, and press the scan button."

[0347] Step 2:

[0348] The server analyzes the collected text data. The input is the text data stored in step 1, and the output is categories, summaries, and main topics. Natural language processing techniques (e.g., SpaCy or NLTK) are used to analyze the text data and extract categories, summaries, and main topics. The specific operation is to "input the text data into the algorithm and extract the analysis results."

[0349] Step 3:

[0350] A user logs into the system through a terminal and enters their favorite genres and past reading history. The input is the user's preferences and past reading history, and the output is the user's input data. The specific operation is "entering information into a form displayed on the terminal screen and clicking the submit button."

[0351] Step 4:

[0352] The terminal sends information entered by the user to the server using the HTTPS protocol. The input is the data entered by the user, and the output is the data sent to the server. The specific operation is "clicking the send button to send the entered information to the server."

[0353] Step 5:

[0354] The server analyzes the received user information and creates a detailed user profile using a machine learning model (e.g., Scikit-learn or TensorFlow). The input is user information and the output is a user profile. The specific operation is to "input the received user information into the machine learning model and generate a profile."

[0355] Step 6:

[0356] The server collects real-time emotional data. The input is the user's facial expression and voice data obtained from the device's camera and sensors, and the output is emotional data. Specifically, the system works by "collecting the user's facial expression data through the camera and acquiring voice data from the sensors."

[0357] Step 7:

[0358] The server analyzes the emotion data and recognizes the user's emotional state. The input is the collected emotion data, and the output is the recognition result of the emotional state. The emotion data is analyzed using an emotion recognition API (for example, AWS Rekognition or Microsoft Azure Emotion API). The specific operation is to "input the emotion data into the emotion recognition API and obtain the emotional state."

[0359] Step 8:

[0360] The server optimizes recommendations using a reinforcement learning algorithm based on the user profile and emotional state. The input is the user profile and the recognition result of the emotional state, and the output is an optimized recommendation list. The specific operation is "input the user profile and emotional state into the algorithm and obtain the recommendation result."

[0361] Step 9:

[0362] The server generates an optimized recommendation list and sends it to the terminal. The input is the optimized recommendation results, and the output is the recommendation list sent to the terminal. The terminal displays the received recommendation results on the user interface. The specific operation is "generate a recommendation list and send it to the terminal."

[0363] Step 10:

[0364] The user selects a book from the recommendation list and purchases or previews it. The input is the recommendation list, and the output is the selected book. The specific operation is "select a book from the recommendation list and click the purchase button."

[0365] Step 11:

[0366] If the selected book is in a foreign language, the server uses an automatic translation function to generate a Japanese version and sends it to the terminal. The input is the foreign language book data, and the output is the translated book data. The specific operation is to "input the foreign language text into the translation API and obtain the Japanese text."

[0367] Step 12:

[0368] The server automatically generates illustrations and videos using AI technology (for example, DALL-E or GAN) and provides them as additional content. The input is the book content data, and the output is the generated illustrations and videos. Specifically, the operation is to "use the book content as input data and generate illustrations and videos using AI technology."

[0369] Step 13:

[0370] The server generates a short video from the story of the novel and sends it to the terminal. The input is the story data of the novel, and the output is the generated short video. The specific operation is to "input the story data of the novel into the video generation algorithm and generate a video file."

[0371] Step 14:

[0372] The server records purchase information and executes revenue sharing with the relevant rights holders and distributors. The input is purchase data, and the output is revenue share distribution information. This process is carried out using blockchain technology (e.g., Ethereum Smart Contracts). The specific operation is to "send purchase information to the blockchain network and execute the smart contract."

[0373] (Application example 2)

[0374] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional book recommendation systems only recommend books based on a user's reading history and genre preferences, and are unable to provide personalized recommendations that take into account the user's real-time emotional state. This makes it difficult to provide timely recommendations that match each user's emotions and mood, resulting in limited improvement in user satisfaction. Furthermore, they are unable to provide sufficient multilingual book services or interactive experiences in virtual stores.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres, past reading history, and emotional state, means for analyzing the information and emotional data input by the user to create a profile and perform real-time emotion recognition, means for recommending books based on the user profile and emotional state, means for displaying recommended books and providing virtual store navigation, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, and means for converting novel stories into video content. This enables timely recommendations that take the user's emotional state into consideration and an interactive virtual store experience.

[0377] "Electronically scanning a book" is the process of optically scanning a paper book and storing it in a digital format.

[0378] "Storing as digital data" means storing electronically read information in a database or storage system.

[0379] "Analyzing collected book data" means analyzing electronically acquired book data using algorithms and software to extract information.

[0380] "Extract categories, summaries, and major topics" means categorizing the contents of a book and extracting key points and summaries.

[0381] "User inputs preferred genres, past reading history, and emotional state" means that the user provides the system with their own reading preferences, books they have read in the past, and their current emotional state via their terminal.

[0382] "Analyzing information and emotional data entered by the user to create a profile and perform real-time emotion recognition" means generating a detailed profile of the user based on data provided by the user and emotional information collected in real time.

[0383] "Recommending books based on user profile and emotional state" means recommending appropriate books based on the created user profile and real-time emotional information.

[0384] "Displaying recommended books and providing virtual store navigation" means displaying recommended books on the user's device and providing the experience of exploring a bookstore in a virtual space.

[0385] "Providing books in multiple languages ​​through automatic translation" is a function that automatically translates selected books into different languages ​​and provides them to users.

[0386] "Automatically generating illustrations and videos" means using AI technology to automatically create illustrations and short videos based on the content of a book.

[0387] "Turning a novel story into video content" means creating a short video based on the content of the novel and providing it to users as visual content.

[0388] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[0389] System Overview

[0390] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0391] Program Description

[0392] Hardware and software used

[0393] 1. Hardware

[0394] Smartphones: Equipped with cameras and processors, they collect user input and emotional data and support the overall operation of apps.

[0395] Server: Manages user data and book data, and performs large-scale data analysis and emotion recognition.

[0396] 2. Software

[0397] OpenCV: Used as an image processing library to perform preprocessing for face detection and emotion recognition.

[0398] TensorFlow / Keras: Build an emotion recognition model and analyze users' real-time emotions.

[0399] Requests: Send an HTTP request to the recommendation API to get the recommendation results.

[0400] Specific examples

[0401] For example, suppose a user is looking for a romance novel. The user logs into the smartphone application "Virtual Bookstore Emo" and inputs their past reading history and preferred genres. Furthermore, the smartphone's camera is used to read their facial expressions, and the data is sent to the server in real time.

[0402] The server creates a user profile based on this data and uses an emotion engine to analyze the user's current emotional state. If the emotion engine determines that the user is feeling stressed, it will prioritize relaxing romance novels and add them to the recommendation list. This allows for more personalized book recommendations for the user.

[0403] Recommended books are displayed through a virtual bookstore interface on the application. Users can virtually navigate the bookstore to select a book, and if the book they choose is in a foreign language, an automatic translation function will provide a Japanese version. Users can also view AI-generated illustrations and short videos, further enriching their reading experience.

[0404] Prompt Sentence Examples

[0405] "Recommend relaxing fantasy novels based on the user's real-time emotions and past reading history."

[0406] In this way, the system of the present invention can provide advanced recommendations that take into account the user's emotional state, providing a rich reading experience.

[0407] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0408] Step 1:

[0409] The user logs in to the application through the device and inputs their preferred genres and past reading history. At this point, the device collects the user's input data (genre, reading history).

[0410] Input: Favorite genres, past reading history

[0411] Output: User input data

[0412] Step 2:

[0413] The device uses the smartphone camera to capture the user's facial expressions and collect emotional data in real time.The device then uses an emotion recognition model (TensorFlow / Keras) to analyze emotions from the collected facial data.

[0414] Input: User's face image

[0415] Output: User emotion data

[0416] Step 3:

[0417] The device sends the user's input data (genre, reading history) and emotion data to the server, which stores the data for further processing.

[0418] Input: User input data, emotion data

[0419] Output: User data sent to the server

[0420] Step 4:

[0421] The server creates a user profile based on the received user data, which includes analyzing input data and emotional data. The profile is then used in the next recommendation stage.

[0422] Input: User data (input data, emotion data)

[0423] Output: User profile

[0424] Step 5:

[0425] The server optimizes book recommendations based on the user's profile and emotional state, using AI technology to select the most suitable books for each user.

[0426] Input: User profile, emotional state

[0427] Output: Recommended book list

[0428] Step 6:

[0429] The server generates a recommended book list and sends it to the terminal, allowing the user to receive a list of the most suitable books in real time.

[0430] Input: Recommended book list

[0431] Output: Recommended book list sent to the device

[0432] Step 7:

[0433] The terminal displays the received recommended book list on the interface of the virtual bookstore, and the user can select books while navigating the virtual bookstore.

[0434] Input: Recommended book list

[0435] Output: Display content in the virtual store

[0436] Step 8:

[0437] If a user selects a recommended book, the server uses machine translation to provide the book in multiple languages, and automatically generates illustrations and short videos if necessary.

[0438] Input: Selected book, trigger data for translation function

[0439] Output: translated book, generated content

[0440] Step 9:

[0441] Users can browse translated books and generated content on their devices and enjoy a reading experience, thereby providing users with a diverse reading and digital content experience.

[0442] Input: translated book, generated content

[0443] Output: User's reading and digital content experience

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0447] [Second embodiment]

[0448] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0449] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0454] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0455] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0456] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0459] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0460] The present invention is a system for personalized book recommendations to users. This system utilizes AI technology to collect and analyze book data and provide recommendations tailored to the user's preferences. Specific embodiments of this system are described below.

[0461] System Overview

[0462] This system consists of three entities: a server, a terminal, and a user. The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations, while the terminal receives input from the user and displays the recommendations. The user provides their preferred genres and reading history through the terminal, and selects and purchases books displayed as recommended results.

[0463] Program processing flow (natural language explanation)

[0464] Step 1: Electronic scanning and digitization of the book

[0465] Server: Electronically scans books and stores them as digital data, making the physical books available in digital form.

[0466] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[0467] Server: Analyzes the content of digitized books and automatically extracts categories, summaries, and major topics. This information is stored in a database.

[0468] Step 3: Collecting user preferences

[0469] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is sent to the server as a user profile.

[0470] Step 4: Analyzing user profiles

[0471] Server: Analyzes the user information sent and creates a detailed user profile, which allows the user to understand their reading habits and interests.

[0472] Step 5: Generate book recommendations

[0473] Server: Based on the created user profile, the server selects the most suitable books from the database and generates recommendations.

[0474] Step 6: Displaying recommendations

[0475] Terminal: Displays the recommendation results received from the server to the user, who can then select books that interest them.

[0476] Step 7: Automated translation and additional content provision

[0477] Server: If necessary, the recommended books are automatically translated and provided in multiple languages. In addition, illustrations and related videos are automatically generated using AI and provided as additional content.

[0478] Step 8: Turning the story of your novel into video content

[0479] Server: Analyzes the story of the book selected by the user and generates a short video, which the user can enjoy visually.

[0480] Step 9: Buy the book and complete the revenue share

[0481] User: Purchase the recommended book.

[0482] Server: Records purchase information and performs revenue share with rights holders.

[0483] Specific examples

[0484] For example, suppose a user is looking for books in the mystery genre. The user logs into the system through a terminal and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery novel from the database, and generates a recommendation result. The recommendation result is displayed on the terminal, and the user can select the book of interest from among them and purchase it. Once the purchase is completed, the server records the information and distributes the sales revenue appropriately to the distributor and rights holder.

[0485] If a recommended book is written in a foreign language, the server will use an automatic translation function to provide a Japanese version. Furthermore, illustrations and videos can be automatically generated to enhance the reader's visual enjoyment. The story of the selected novel is also provided to the user as a short video, allowing users to enjoy the content in a variety of ways.

[0486] In this way, the system of the present invention recommends optimal books to the user in a consistent manner, providing a rich reading experience.

[0487] The processing flow will be explained below.

[0488] Step 1:

[0489] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[0490] Step 2:

[0491] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[0492] Step 3:

[0493] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is necessary to create a user profile.

[0494] Step 4:

[0495] Terminal: Sends information entered by the user to the server.

[0496] Step 5:

[0497] Server: Analyzes the received user information and creates a user profile based on the user's reading habits and interests.

[0498] Step 6:

[0499] Server: Based on the user profile, a recommendation algorithm is used to select the most suitable books and generate a recommendation list.

[0500] Step 7:

[0501] Terminal: Displays the recommendation results received from the server to the user, who can then select books of interest from this list.

[0502] Step 8:

[0503] User: Select a book from the recommendation list via the device and purchase or preview it.

[0504] Step 9:

[0505] Server: If the book selected by the user is in a foreign language, it uses an automatic translation function to generate a Japanese version and sends it to the terminal.

[0506] Step 10:

[0507] Server: Using AI technology, illustrations and videos related to the selected books are automatically generated and provided as additional content.

[0508] Step 11:

[0509] Server: Furthermore, it analyzes the story of the novel and generates a short video, which is also provided to the user.

[0510] Step 12:

[0511] User: Purchases books that interest them. After purchase, the books are available as e-books.

[0512] Step 13:

[0513] Server: Records purchase information and distributes revenue to relevant rights holders and distributors. This process is automated.

[0514] Example 1

[0515] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0516] Current book recommendation systems lack the ability to provide personalized recommendations that reflect users' preferences and reading history, and they face challenges in providing multilingual and visual content. Furthermore, the process of digitizing and analyzing books is manual and inefficient. Furthermore, the post-purchase revenue sharing mechanism is unclear, making it difficult to fairly distribute rewards to rights holders.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0518] In this invention, the server includes a means for electronically scanning physical books and storing them as digital data, a means for analyzing the collected book data and extracting categories, summaries, and major topics, and a means for recommending books based on user profiles. This enables highly accurate book recommendations that reflect users' preferences and reading history, and also enables multilingual and visual content provision. A post-purchase revenue sharing mechanism is also established, enabling fair reward distribution.

[0519] "Physical Book" means a book in a non-electronic format made of paper or other material.

[0520] "Electronically scanned" refers to the process of converting a physical book into digital data using a device such as a scanner.

[0521] "Digital data" refers to information stored in an electronically processable form.

[0522] "Analysis" refers to the process of automatically extracting meaningful information from digital data.

[0523] A "category" refers to a group of books classified based on their content or theme.

[0524] A "summary" is a brief summary of the book's contents.

[0525] "Major topics" refers to important themes or topics contained in the book's content.

[0526] "Input means" refers to the method or interface by which a user provides information to a system.

[0527] A "profile" refers to a collection of data that reflects a user's preferences and behavior.

[0528] "Recommendation method" refers to the process of selecting and providing books based on a user's profile.

[0529] The "means for displaying" refers to an interface that visually presents the recommended book information to the user.

[0530] "Machine translation" refers to technology that automatically converts text between multiple languages.

[0531] "Means for automatically generating illustrations and videos" refers to the process of using artificial intelligence technology to automatically create visual content related to the content of a book.

[0532] "Video content creation" refers to the process of visualizing the story of a book and providing it in video format.

[0533] "Revenue share" refers to a mechanism for fairly distributing revenue.

[0534] "Means for dynamically updating reading habits" refers to technology that automatically updates a user's profile based on their new reading history and preferences.

[0535] MODE FOR CARRYING OUT THE INVENTION

[0536] This invention is a system that recommends the most suitable books to users in a personalized manner. Specifically, it analyzes electronically digitized book data and provides recommendations that match the user's preferences. This system is mainly composed of three entities: a server, a terminal, and a user.

[0537] The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations. The terminal receives input from the user and displays the recommendations. The user provides their preferred genres and past reading history through the terminal, and selects and purchases the books displayed as recommended results.

[0538] The server first uses a dedicated scanner (e.g., Fujitsu ScanSnap iX1500) to electronically scan the physical book and store it as digital data. This digital data is then converted into text data using OCR technology (e.g., Tesseract OCR). The converted text data is then saved in the server's storage.

[0539] The server then uses natural language processing tools (e.g., Apache OpenNLP) to analyze the text data from the digitized books, including tokenization, part-of-speech tagging, and named entity extraction, and stores categories, summaries, and major topics in a database (e.g., MySQL).

[0540] Users log in to the system using a terminal (e.g., a browser-based interface) and enter their preferred genres and past reading history, such as "mysteries" or "a list of books read in the past three years." The terminal checks the format of this input data in real time and sends it to the server as well-formed data.

[0541] The server then uses Node.js and Python scripts to analyze the received user data and generate detailed user profiles, which include information that classifies and predicts the user's reading habits using machine learning algorithms (e.g., k-means clustering), and store this information in a database.

[0542] The server then uses an AI model (e.g., TensorFlow) to select the most suitable books from the database based on the generated user profile. An example prompt might be, "Recommend the most suitable mystery novel based on the user's past reading history and preferred genres. Include summaries, main topics, and categories in the recommendation. Examples include 'famous detective series' and 'famous mystery novel series.'"

[0543] The recommendation results sent from the server are then displayed on the device using a web application using React.js, allowing users to check detailed information about the recommended books (summary, category, main topics).

[0544] If the recommended book is in a foreign language, the server will automatically translate it using the Google Translate API. It can also use OpenAI's DALL-E API to automatically generate illustrations and video content, allowing users to enjoy the book's contents in a variety of ways.

[0545] Furthermore, when a purchase is made, the server uses the Stripe API to record the purchase information and distribute the appropriate revenue share to the rights holder. In this way, the system provides users with personalized book recommendations, enhancing their reading experience.

[0546] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0547] Step 1:

[0548] Electronic scanning and digitization of books

[0549] Server: Electronically scans physical books using a specialized scanner. It accepts the physical book as input and produces scanned image data as output, thereby capturing the contents of the book in image format.

[0550] Server: Converts scanned image data into text data using OCR technology. The input is image data and the output is text data. Specifically, OCR technology performs character recognition and extracts text information from the image.

[0551] Step 2:

[0552] Data analysis of books and extraction of categories, summaries, and main topics

[0553] Server: Analyzes text data using natural language processing tools. The input is text data, and the output is the analysis results: categories, summaries, and main topics. This converts the text data into meaningful information.

[0554] Server: Stores the analyzed data in a database. Specifically, it extracts categories, summaries, and main topics from the text data and inserts them into a MySQL database.

[0555] Step 3:

[0556] Collecting user preference information

[0557] User: Logs into the system through a terminal and inputs his / her preferred genre and past reading history. The input provides his / her preferred genre (e.g., mystery, science fiction) and past reading history. The output is the formatted user input data.

[0558] Terminal: Checks the format of the user's input data in real time and sends it to the server as well-formed data. Specific operations include checking the input of required fields and validating the data format.

[0559] Step 4:

[0560] User profile analysis

[0561] Server: Analyzes the received user data and generates a detailed user profile. The input is the user's preferences, and the output is a detailed user profile. This digitizes the user's reading habits and hobbies. Specifically, it runs a machine learning algorithm using Node.js and Python scripts.

[0562] Step 5:

[0563] Book recommendation generation

[0564] Server: Based on the generated user profile, an AI model is used to select the most suitable books. The input is a detailed user profile, and the output is a list of recommended books. This allows the most suitable books to be selected for the user. Specifically, TensorFlow is used to process the profile data and extract the books that best fit the user profile.

[0565] Step 6:

[0566] Displaying recommendations

[0567] Terminal: Displays the recommendation results received from the server to the user. The input is a list of recommended books, and the output is a visual display of the book information. This allows the user to check the list of recommended books. Specifically, the display is done using a web application using React.js.

[0568] Step 7:

[0569] Automatic translation and additional content provision

[0570] Server: Uses the Google Translate API to automatically translate recommended books. The input is the text data of the recommended book, and the output is the translated text data. This allows users to enjoy the book in their native language. Specifically, the API is called to obtain the translation results.

[0571] Server: Generates illustrations and video content using OpenAI's DALL-E API. The input is the book's content data, and the output is the generated visual content. This allows for a deeper understanding of the book. Specifically, the server calls the API to generate related illustrations and videos.

[0572] Step 8:

[0573] Turning novel stories into video content

[0574] Server: Analyzes the story of a book selected by the user and generates it as a short video. The input is the book's story data, and the output is a short video. This allows users to enjoy the story visually. Specifically, the video is created using Adobe Premiere Pro and an AI video generation tool.

[0575] Step 9:

[0576] Buying books and running revenue shares

[0577] User: Purchases the recommended book using a device. The input is a purchase intention and payment information, and the output is a purchase confirmation and a download link for the book. The book then becomes the user's property. Specific actions include paying with a credit card or electronic money.

[0578] Server: Records purchase information and performs revenue sharing using the Stripe API. The input is purchase information, and the output is confirmation of revenue distribution to rights holders. This ensures fair revenue distribution. Specific operations include storing purchase data and distributing revenue at the appropriate rate.

[0579] (Application example 1)

[0580] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0581] Conventional book recommendation systems are generally intended for online use, and when users visit a bookstore, there are limited ways to obtain book information on the spot. This makes it difficult for users to find books that suit their preferences, and the purchasing experience is not sufficiently improved. In addition, differences in reading devices and the lack of dynamic updates to user profiles make it difficult to provide appropriate recommendations in real time.

[0582] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0583] In this invention, the server includes a means for reading a code installed in the store, a means for displaying in-store book information in real time based on the read code, and a means for recommending books in real time based on a user profile using a generative AI model, thereby enabling a user to receive optimal book recommendations in real time when searching for books in the store.

[0584] "Means for electronically scanning books and storing them as digital data" refers to a function for converting books into digital data using equipment such as an electronic scanner or camera, and storing the data on a server or database.

[0585] "Means of analyzing collected book data and extracting categories, summaries, and key topics" refers to a function that analyzes the contents of a book using natural language processing technology, etc., and automatically extracts the genre to which the book belongs, a summary of the contents, and important topics.

[0586] "Means for users to input their favorite genres and past reading history" refers to an interface or mechanism that allows users to input information about their interests and the books they have read.

[0587] "Means for analyzing information entered by the user and creating a profile" refers to a function that analyzes information on genres and reading history provided by the user and generates a profile that reflects the user's reading tendencies and preferences.

[0588] A "means for recommending books based on a user profile" is an algorithm or system for recommending the most suitable book to a user based on the generated user profile.

[0589] The "means for displaying recommended books" refers to a display or a screen of a mobile device that visually presents information about recommended books to the user.

[0590] "Means for providing books in multiple languages ​​through machine translation" is a function that translates the contents of recommended books into the user's desired language through machine translation, making them available in multiple languages.

[0591] "Means for automatically generating illustrations and videos" refers to a function that uses artificial intelligence and image generation technology to automatically generate related illustrations and videos based on the content of a book.

[0592] "A means of turning a novel story into video content" is a function that analyzes the content of a novel and generates video content such as a short film or animation based on that story.

[0593] "Means for reading codes installed in stores" refers to a function that allows you to read codes installed in stores, such as QR codes and barcodes, using a smartphone or dedicated device.

[0594] "Means for displaying in-store book information in real time based on the scanned code" is a function for displaying detailed information about books in the store in real time on the user's device based on the scanned QR code or barcode.

[0595] "Means of recommending books in real time based on user profile using a generative AI model" is a function that uses an artificial intelligence model to recommend the most suitable book on the spot based on the user's profile information.

[0596] The present invention is a system for recommending the most suitable book to a user in a personalized manner, and is mainly composed of a server, a terminal, and a user. Specific embodiments for implementing this system will be described below.

[0597] 1. Server operation

[0598] The server has the following roles:

[0599] 1. Electronic scanning and digital storage of books:

[0600] The server scans the books using an electronic scanner or camera and stores the content as digital data using cloud storage services such as AWS S3 and Firebase Storage.

[0601] 2. Data Analysis:

[0602] Analyze the content of digitized books and extract categories, summaries, and key topics using natural language processing techniques, leveraging Python libraries (e.g., spaCy and NLTK) and AI models (e.g., BERT).

[0603] 3. Create a user profile:

[0604] The server analyzes the data provided by the user (preferred genres, past reading history) and creates a detailed user profile, which is stored in a cloud database (e.g., Firebase Firestore).

[0605] 4. Recommendation Generation:

[0606] Generate book recommendations based on user profiles using generative AI models (e.g., GPT-3) and integrating multiple recommendation algorithms.

[0607] 5. Automatic translation and additional content generation:

[0608] The content of the book is automatically translated as needed and provided in multiple languages. In addition, illustrations and related videos are automatically generated using a generative AI model and provided to users.

[0609] 6. Turning novel stories into video content:

[0610] The story of the book selected by the user is generated as video content. This step uses generative AI models and machine learning techniques.

[0611] 7. Revenue Share Implementation:

[0612] When a book is purchased based on a recommendation, the purchase information is recorded and the sales revenue is distributed to the rights holder.

[0613] 2. Device operation

[0614] The device has the following features:

[0615] 1. Data input from the user:

[0616] The device provides an interface for users to input their preferred genres and past reading history.

[0617] 2. Displaying recommendation results:

[0618] The recommendation results received from the server are displayed to the user using a mobile device such as a smartphone or tablet.

[0619] 3. QR Code Scanning:

[0620] Read QR codes and barcodes installed in stores to obtain book information. To do this, use a QR code reader library (e.g., Zxing).

[0621] 4. Information display based on the code read:

[0622] Based on the code read, the terminal displays detailed information about the book to the user in real time.

[0623] 3. User operations

[0624] The user performs the following operations:

[0625] 1. Enter the data:

[0626] Users input their preferences and past reading history through the device.

[0627] 2. Check the book recommendations:

[0628] Check the recommendations and choose the books that interest you.

[0629] 3. Scan the QR code:

[0630] Scan the QR code of a book you are interested in in the store to view detailed information.

[0631] 4. Buy a book:

[0632] Purchase recommended books or books that interest you.

[0633] Specific examples

[0634] For example, suppose a user visits a physical store and is looking for a book in the mystery genre. The user logs in using a smartphone app and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery book from the database, and generates a recommendation result. When the user scans the QR code in the store, information about the book is displayed in real time, along with additional content (e.g., illustrations and related videos).

[0635] Examples of prompts for AI models

[0636] "Generate a summary of a mystery novel in which a detective solves a complex case. Include the main plot and key details."

[0637] As a result, this system can recommend optimal books to users in a consistent manner, enriching the purchasing experience in physical stores.

[0638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0639] Step 1:

[0640] The server electronically scans the book and stores it as digital data. The book is scanned using a scanner or camera to generate image data. This is then converted into text data using OCR software and stored in cloud storage. The input to this process is the physical book, and the output is digital text data.

[0641] Step 2:

[0642] The server analyzes the collected digital data of books and extracts categories, summaries, and main topics. It uses Python libraries and natural language processing techniques (e.g., spaCy and NLTK) to analyze the text data and extract the necessary information. The input of this process is the digital text data generated in step 1, and the output is categories, summaries, and main topics.

[0643] Step 3:

[0644] Users input their preferred genres and past reading history using a device. The input interface is a smartphone or tablet, and the user's input data is stored in a cloud database. The input for this process is the genre and reading history data entered by the user, and the output is the user profile data stored in the cloud database.

[0645] Step 4:

[0646] The server analyzes the user's input and creates a detailed user profile. Based on the user data stored in the cloud, an analytical algorithm is used to analyze the user's reading habits. The input to this process is the user data stored in the cloud database, and the output is a detailed user profile.

[0647] Step 5:

[0648] The server uses a generative AI model (e.g., GPT-3) to recommend the most suitable books in real time based on the user profile. The server matches the user profile with book information stored in a database and generates recommendations using the AI ​​model. The input to this process is a detailed user profile and a book database, and the output is a recommendation result.

[0649] Step 6:

[0650] The terminal displays the recommendation results to the user. Recommended book information is displayed on the screen of a smartphone or tablet. The input to this process is the recommendation results received from the server, and the output is recommended book information visually presented to the user.

[0651] Step 7:

[0652] A user reads a QR code attached to a book in a store. The user then scans the QR code using the camera on their smartphone to obtain the book information. The input to this process is the QR code attached to the book, and the output is the book information read from the QR code.

[0653] Step 8:

[0654] The terminal displays in-store book information in real time based on the read QR code information. Detailed book information is displayed on the terminal based on the information obtained from the QR code. The input for this process is the data read from the QR code, and the output is book information displayed in real time.

[0655] Step 9:

[0656] The server generates automatic translations and additional content (illustrations and videos) as needed. A generative AI model is used for translation, and image generation AI such as DALL-E is used to generate illustrations and videos. The input for this process is recommendation results and book content data, and the output is the translated text and generated illustrations and video content.

[0657] Step 10:

[0658] When a user purchases a book, the server executes the revenue share. It records the purchase information and distributes the sales appropriately to the rights holder. The input of this process is the user's purchase information, and the output is the result of the revenue share execution.

[0659] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0660] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[0661] System Overview

[0662] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0663] Program processing flow (natural language explanation)

[0664] Step 1: Electronic scanning and digitization of the book

[0665] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[0666] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[0667] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[0668] Step 3: Collecting user preferences

[0669] User: Logs into the system through a terminal and enters his / her preferred genres and past reading history.

[0670] Step 4: Submit user information

[0671] Terminal: Sends information entered by the user to the server.

[0672] Step 5: Create a user profile

[0673] Server: Analyzes the received user information and creates a detailed user profile.

[0674] Step 6: Emotion Recognition with the Emotion Engine

[0675] Server: Analyzes real-time emotion data collected from the user's device using an emotion engine to recognize the user's emotional state.

[0676] Step 7: Emotion-based recommendation optimization

[0677] Server: Optimizes recommendations based on user profile and emotional state, choosing books that fit the user's current emotions.

[0678] Step 8: Generate and display the recommendation list

[0679] Server: Generates an optimized recommendation list.

[0680] Terminal: Displays the recommendation results received from the server to the user.

[0681] Step 9: Select and purchase books

[0682] User: Select a book from the recommended list and purchase or preview it.

[0683] Step 10: Automated translation and additional content provision

[0684] Server: If the selected book is in a foreign language, an automatic translation function is used to generate a Japanese version and send it to the device. In addition, AI technology is used to automatically generate illustrations and videos, which are provided as additional content.

[0685] Step 11: Turn your story into video content

[0686] Server: Generates the story of the novel as a short video and provides it to the user.

[0687] Step 12: Record purchase information and execute revenue share

[0688] Server: Records purchase information and executes revenue share with relevant rights holders and distributors.

[0689] Specific examples

[0690] For example, suppose a user is looking for a romance novel. The user logs into the system through their device and enters their desired romance and past reading history. Furthermore, the device's camera and sensors are used to transmit the user's emotional data in real time to the server. The server then creates a user profile based on this information and uses an emotion engine to analyze the user's current emotional state.

[0691] Because the user is feeling stressed, the emotion engine prioritizes relaxing romance novels and adds them to the recommendation list. From the recommendation list displayed on the device, the user can select a book that interests them and purchase it. If the selected book is in a foreign language, the server uses an automatic translation function to provide a Japanese version. Users can also view illustrations and short videos generated by AI.

[0692] In this way, the system of the present invention, combined with the emotion engine, provides advanced recommendations that take into account the user's emotional state, enabling a rich reading experience.

[0693] The processing flow will be explained below.

[0694] Step 1:

[0695] Server: The book is scanned electronically and stored as digital data in a database on the server. In this step, OCR technology is used to extract text information for each page and store it as metadata.

[0696] Step 2:

[0697] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Using natural language processing technology, it extracts themes and keywords from the content and organizes the database based on these.

[0698] Step 3:

[0699] User: Logs in to the system using a terminal and enters information such as preferred genres and past reading history, which allows the collection of basic information for a user profile.

[0700] Step 4:

[0701] Terminal: Sends information entered by the user to the server, which uses it to create and update the user profile.

[0702] Step 5:

[0703] Server: Analyzes the received user information and creates a detailed user profile, including data about the user's reading habits and interests.

[0704] Step 6:

[0705] On-device: Using the device's camera and sensors, we collect real-time emotional data from the user. This data is obtained through facial recognition and biometric sensors.

[0706] Step 7:

[0707] Terminal: Sends collected emotional data to the server, which uses it to analyze the user's emotional state.

[0708] Step 8:

[0709] Server: The emotion engine analyzes the transmitted emotion data and recognizes the user's current emotional state. For example, it identifies whether the user is happy, sad, or stressed.

[0710] Step 9:

[0711] Server: Based on the user profile and emotional state, the recommendation algorithm selects the most suitable book and optimizes the recommendation. The selection criteria are dynamically adjusted taking into account the emotion recognition results.

[0712] Step 10:

[0713] Server: Generates an optimized recommendation list and sends it to the device. The recommendation list includes books that are appropriate for the user's emotional state.

[0714] Step 11:

[0715] Terminal: The terminal displays the recommendation results received from the server to the user. The user can select a book of interest from the recommendation list and view detailed information and samples.

[0716] Step 12:

[0717] User: Can select a book from the recommended list and make a purchase. Once purchased, the book will be available as an e-book.

[0718] Step 13:

[0719] Server: If the selected book is in a foreign language, it will use the automatic translation function to provide the Japanese version. This translation process runs in the background.

[0720] Step 14:

[0721] Server: Using AI technology, it automatically generates illustrations and videos related to the selected book and sends them to the device as additional content, enriching the reader's visual experience.

[0722] Step 15:

[0723] Server: The server then generates a short video of the story and provides it to the user. In this step, the main parts of the story are visualized to provide visual enjoyment.

[0724] Step 16:

[0725] Server: Records purchase information and executes revenue sharing with relevant rights holders and distributors. This process is performed automatically based on sales information.

[0726] Example 2

[0727] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] In today's information overload, it is difficult for users to find the book that best suits them. Advanced filtering technology is required to select books that best suit the user's preferences and emotional state from the vast number of books available. Furthermore, a system that can smoothly provide foreign language books and additional content is also needed.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0730] In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres and past reading history, means for analyzing the information input by the user and creating a profile, means for recommending books based on the user profile, means for displaying recommended books, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, means for converting novel stories into video content, means for collecting real-time emotional data, means for analyzing the emotional data and recognizing the user's emotional state, and means for recommending books based on the user's emotional state. This allows the server to provide optimal books based on the user's preferences and emotional state, enabling a rich reading experience.

[0731] "Means of electronically scanning books and storing them as digital data" refers to a method of electronically reading paper books using a scanner and storing the data in a database as text data.

[0732] "Methods for analyzing collected book data and extracting categories, summaries, and main topics" refers to a method of analyzing the contents of digitized books using OCR technology and natural language processing technology, and extracting the categories to which the books belong, summaries, and main points of discussion.

[0733] "Means for users to input their favorite genres and past reading history" refers to an interface that allows users to provide the system with their favorite genres and a list of books they have read so far.

[0734] "Means for analyzing information entered by the user and creating a profile" refers to a method for analyzing information provided by the user, such as genre and reading history, using machine learning models to generate a detailed user profile.

[0735] The "means for recommending books based on a user profile" is a method for providing a list of books that are most suitable for each individual user based on the generated user profile.

[0736] The "means for displaying recommended books" is an interface for displaying the recommendation list sent from the server on the user's terminal.

[0737] "Means for providing books in multiple languages ​​through automatic translation" refers to a method in which, if the selected book is in a foreign language, the book is translated into another language using automatic translation technology and provided to the user.

[0738] "Means for automatically generating illustrations and videos" refers to a method of using AI technology to generate illustrations and short videos based on the content of a book and provide them to users as additional content.

[0739] "Means for turning a novel story into video content" refers to a method of using a video generation algorithm to generate and provide a short video using the contents of a novel as input data.

[0740] "Means for collecting real-time emotional data" refers to an interface that uses cameras and sensors to obtain emotional data in real time from the user's facial expressions, voice, etc.

[0741] The "means for analyzing emotion data and recognizing the user's emotional state" is an emotion recognition algorithm for analyzing the collected emotion data and recognizing the user's current emotional state.

[0742] The "means for recommending books based on the user's emotional state" is a method for combining a user profile and an emotional state to recommend books that suit the user's current emotions.

[0743] The present invention is an advanced system that recommends optimal books to a user, recognizes the user's emotional state, and optimizes the recommendations. Hereinafter, a detailed description will be given of an embodiment of the present invention.

[0744] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0745] First, the server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap), extracts the text information using OCR technology (e.g., Tesseract), and stores it in a database. This step generates digital data.

[0746] The server then analyzes the collected digital data using algorithms based on natural language processing technology (e.g., SpaCy or NLTK) to extract book categories, summaries, and main topics.

[0747] Users log in to the system through their devices and enter their favorite genres and past reading history. iOS or Android applications are used on the devices. The information entered by the user is sent to the server using the HTTPS protocol. The server analyzes the information received and creates a detailed user profile using machine learning models (e.g., Scikit-learn or TensorFlow).

[0748] Furthermore, the user's real-time emotional data is collected by the emotion engine. The emotional data acquired using the device's camera and sensors is analyzed by an emotion recognition API (e.g., AWS Rekognition or Microsoft Azure Emotion API). The server recognizes the user's emotional state based on the analysis results.

[0749] The server then optimizes the recommendations based on the user's profile and current emotional state. A reinforcement learning algorithm is used to select books that fit the emotional state. An optimized recommendation list is then generated and sent to the device, which displays the list in the user interface.

[0750] Users can select books of interest from the recommendation list and purchase or preview them. The purchase process is carried out through an e-commerce system (e.g., Amazon or Rakuten). If the selected book is in a foreign language, an automatic translation function (e.g., Google Translate API) is used to generate a Japanese version. Furthermore, illustrations and videos are automatically generated using AI technology (e.g., DALL-E or GAN) and provided as additional content.

[0751] As a concrete example, let's say a user is looking for a romance novel. The user logs into the system through their device and enters their romance and past reading history. Emotional data is sent to the server in real time via the device's camera and sensors. The server creates a user profile based on this information, and an emotion engine analyzes the user's current emotional state. If the emotion engine determines that the user is feeling "stressed," it will prioritize recommendations of relaxing romance novels.

[0752] In this way, advanced recommendations based on the user's emotional state and preferences are realized.

[0753] Example prompt sentence:

[0754] "Recommend positive, relaxing romance novels based on the user's reading history and real-time emotional data. Users are stressed and want content that will help them feel better."

[0755] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0756] Step 1:

[0757] The server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap). The input is the paper book, and the output is the scanned image data. This image data is converted into text information using OCR technology (e.g., Tesseract) and stored in a database. The specific operation is "start the scanner, place the book, and press the scan button."

[0758] Step 2:

[0759] The server analyzes the collected text data. The input is the text data stored in step 1, and the output is categories, summaries, and main topics. Natural language processing techniques (e.g., SpaCy or NLTK) are used to analyze the text data and extract categories, summaries, and main topics. The specific operation is to "input the text data into the algorithm and extract the analysis results."

[0760] Step 3:

[0761] A user logs into the system through a terminal and enters their favorite genres and past reading history. The input is the user's preferences and past reading history, and the output is the user's input data. The specific operation is "entering information into a form displayed on the terminal screen and clicking the submit button."

[0762] Step 4:

[0763] The terminal sends information entered by the user to the server using the HTTPS protocol. The input is the data entered by the user, and the output is the data sent to the server. The specific operation is "clicking the send button to send the entered information to the server."

[0764] Step 5:

[0765] The server analyzes the received user information and creates a detailed user profile using a machine learning model (e.g., Scikit-learn or TensorFlow). The input is user information and the output is a user profile. The specific operation is to "input the received user information into the machine learning model and generate a profile."

[0766] Step 6:

[0767] The server collects real-time emotional data. The input is the user's facial expression and voice data obtained from the device's camera and sensors, and the output is emotional data. Specifically, the system works by "collecting the user's facial expression data through the camera and acquiring voice data from the sensors."

[0768] Step 7:

[0769] The server analyzes the emotion data and recognizes the user's emotional state. The input is the collected emotion data, and the output is the recognition result of the emotional state. The emotion data is analyzed using an emotion recognition API (for example, AWS Rekognition or Microsoft Azure Emotion API). The specific operation is to "input the emotion data into the emotion recognition API and obtain the emotional state."

[0770] Step 8:

[0771] The server optimizes recommendations using a reinforcement learning algorithm based on the user profile and emotional state. The input is the user profile and the recognition result of the emotional state, and the output is an optimized recommendation list. The specific operation is "input the user profile and emotional state into the algorithm and obtain the recommendation result."

[0772] Step 9:

[0773] The server generates an optimized recommendation list and sends it to the terminal. The input is the optimized recommendation results, and the output is the recommendation list sent to the terminal. The terminal displays the received recommendation results on the user interface. The specific operation is "generate a recommendation list and send it to the terminal."

[0774] Step 10:

[0775] The user selects a book from the recommendation list and purchases or previews it. The input is the recommendation list, and the output is the selected book. The specific operation is "select a book from the recommendation list and click the purchase button."

[0776] Step 11:

[0777] If the selected book is in a foreign language, the server uses an automatic translation function to generate a Japanese version and sends it to the terminal. The input is the foreign language book data, and the output is the translated book data. The specific operation is to "input the foreign language text into the translation API and obtain the Japanese text."

[0778] Step 12:

[0779] The server automatically generates illustrations and videos using AI technology (for example, DALL-E or GAN) and provides them as additional content. The input is the book content data, and the output is the generated illustrations and videos. Specifically, the operation is to "use the book content as input data and generate illustrations and videos using AI technology."

[0780] Step 13:

[0781] The server generates a short video from the story of the novel and sends it to the terminal. The input is the story data of the novel, and the output is the generated short video. The specific operation is to "input the story data of the novel into the video generation algorithm and generate a video file."

[0782] Step 14:

[0783] The server records purchase information and executes revenue sharing with the relevant rights holders and distributors. The input is purchase data, and the output is revenue share distribution information. This process is carried out using blockchain technology (e.g., Ethereum Smart Contracts). The specific operation is to "send purchase information to the blockchain network and execute the smart contract."

[0784] (Application example 2)

[0785] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0786] Conventional book recommendation systems only recommend books based on a user's reading history and genre preferences, and are unable to provide personalized recommendations that take into account the user's real-time emotional state. This makes it difficult to provide timely recommendations that match each user's emotions and mood, resulting in limited improvement in user satisfaction. Furthermore, they are unable to provide sufficient multilingual book services or interactive experiences in virtual stores.

[0787] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres, past reading history, and emotional state, means for analyzing the information and emotional data input by the user to create a profile and perform real-time emotion recognition, means for recommending books based on the user profile and emotional state, means for displaying recommended books and providing virtual store navigation, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, and means for converting novel stories into video content. This enables timely recommendations that take the user's emotional state into consideration and an interactive virtual store experience.

[0788] "Electronically scanning a book" is the process of optically scanning a paper book and storing it in a digital format.

[0789] "Storing as digital data" means storing electronically read information in a database or storage system.

[0790] "Analyzing collected book data" means analyzing electronically acquired book data using algorithms and software to extract information.

[0791] "Extract categories, summaries, and major topics" means categorizing the contents of a book and extracting key points and summaries.

[0792] "User inputs preferred genres, past reading history, and emotional state" means that the user provides the system with their own reading preferences, books they have read in the past, and their current emotional state via their terminal.

[0793] "Analyzing information and emotional data entered by the user to create a profile and perform real-time emotion recognition" means generating a detailed profile of the user based on data provided by the user and emotional information collected in real time.

[0794] "Recommending books based on user profile and emotional state" means recommending appropriate books based on the created user profile and real-time emotional information.

[0795] "Displaying recommended books and providing virtual store navigation" means displaying recommended books on the user's device and providing the experience of exploring a bookstore in a virtual space.

[0796] "Providing books in multiple languages ​​through automatic translation" is a function that automatically translates selected books into different languages ​​and provides them to users.

[0797] "Automatically generating illustrations and videos" means using AI technology to automatically create illustrations and short videos based on the content of a book.

[0798] "Turning a novel story into video content" means creating a short video based on the content of the novel and providing it to users as visual content.

[0799] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[0800] System Overview

[0801] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[0802] Program Description

[0803] Hardware and software used

[0804] 1. Hardware

[0805] Smartphones: Equipped with cameras and processors, they collect user input and emotional data and support the overall operation of apps.

[0806] Server: Manages user data and book data, and performs large-scale data analysis and emotion recognition.

[0807] 2. Software

[0808] OpenCV: Used as an image processing library to perform preprocessing for face detection and emotion recognition.

[0809] TensorFlow / Keras: Build an emotion recognition model and analyze users' real-time emotions.

[0810] Requests: Send an HTTP request to the recommendation API to get the recommendation results.

[0811] Specific examples

[0812] For example, suppose a user is looking for a romance novel. The user logs into the smartphone application "Virtual Bookstore Emo" and inputs their past reading history and preferred genres. Furthermore, the smartphone's camera is used to read their facial expressions, and the data is sent to the server in real time.

[0813] The server creates a user profile based on this data and uses an emotion engine to analyze the user's current emotional state. If the emotion engine determines that the user is feeling stressed, it will prioritize relaxing romance novels and add them to the recommendation list. This allows for more personalized book recommendations for the user.

[0814] Recommended books are displayed through a virtual bookstore interface on the application. Users can virtually navigate the bookstore to select a book, and if the book they choose is in a foreign language, an automatic translation function will provide a Japanese version. Users can also view AI-generated illustrations and short videos, further enriching their reading experience.

[0815] Prompt Sentence Examples

[0816] "Recommend relaxing fantasy novels based on the user's real-time emotions and past reading history."

[0817] In this way, the system of the present invention can provide advanced recommendations that take into account the user's emotional state, providing a rich reading experience.

[0818] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0819] Step 1:

[0820] The user logs in to the application through the device and inputs their preferred genres and past reading history. At this point, the device collects the user's input data (genre, reading history).

[0821] Input: Favorite genres, past reading history

[0822] Output: User input data

[0823] Step 2:

[0824] The device uses the smartphone camera to capture the user's facial expressions and collect emotional data in real time.The device then uses an emotion recognition model (TensorFlow / Keras) to analyze emotions from the collected facial data.

[0825] Input: User's face image

[0826] Output: User emotion data

[0827] Step 3:

[0828] The device sends the user's input data (genre, reading history) and emotion data to the server, which stores the data for further processing.

[0829] Input: User input data, emotion data

[0830] Output: User data sent to the server

[0831] Step 4:

[0832] The server creates a user profile based on the received user data, which includes analyzing input data and emotional data. The profile is then used in the next recommendation stage.

[0833] Input: User data (input data, emotion data)

[0834] Output: User profile

[0835] Step 5:

[0836] The server optimizes book recommendations based on the user's profile and emotional state, using AI technology to select the most suitable books for each user.

[0837] Input: User profile, emotional state

[0838] Output: Recommended book list

[0839] Step 6:

[0840] The server generates a recommended book list and sends it to the terminal, allowing the user to receive a list of the most suitable books in real time.

[0841] Input: Recommended book list

[0842] Output: Recommended book list sent to the device

[0843] Step 7:

[0844] The terminal displays the received recommended book list on the interface of the virtual bookstore, and the user can select books while navigating the virtual bookstore.

[0845] Input: Recommended book list

[0846] Output: Display content in the virtual store

[0847] Step 8:

[0848] If a user selects a recommended book, the server uses machine translation to provide the book in multiple languages, and automatically generates illustrations and short videos if necessary.

[0849] Input: Selected book, trigger data for translation function

[0850] Output: translated book, generated content

[0851] Step 9:

[0852] Users can browse translated books and generated content on their devices and enjoy a reading experience, thereby providing users with a diverse reading and digital content experience.

[0853] Input: translated book, generated content

[0854] Output: User's reading and digital content experience

[0855] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0856] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0857] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0858] [Third embodiment]

[0859] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0860] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0861] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0862] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0863] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0864] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0865] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0866] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0867] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0868] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0869] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0870] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0871] The present invention is a system for personalized book recommendations to users. This system utilizes AI technology to collect and analyze book data and provide recommendations tailored to the user's preferences. Specific embodiments of this system are described below.

[0872] System Overview

[0873] This system consists of three entities: a server, a terminal, and a user. The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations, while the terminal receives input from the user and displays the recommendations. The user provides their preferred genres and reading history through the terminal, and selects and purchases books displayed as recommended results.

[0874] Program processing flow (natural language explanation)

[0875] Step 1: Electronic scanning and digitization of the book

[0876] Server: Electronically scans books and stores them as digital data, making the physical books available in digital form.

[0877] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[0878] Server: Analyzes the content of digitized books and automatically extracts categories, summaries, and major topics. This information is stored in a database.

[0879] Step 3: Collecting user preferences

[0880] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is sent to the server as a user profile.

[0881] Step 4: Analyzing user profiles

[0882] Server: Analyzes the user information sent and creates a detailed user profile, which allows the user to understand their reading habits and interests.

[0883] Step 5: Generate book recommendations

[0884] Server: Based on the created user profile, the server selects the most suitable books from the database and generates recommendations.

[0885] Step 6: Displaying recommendations

[0886] Terminal: Displays the recommendation results received from the server to the user, who can then select books that interest them.

[0887] Step 7: Automated translation and additional content provision

[0888] Server: If necessary, the recommended books are automatically translated and provided in multiple languages. In addition, illustrations and related videos are automatically generated using AI and provided as additional content.

[0889] Step 8: Turning the story of your novel into video content

[0890] Server: Analyzes the story of the book selected by the user and generates a short video, which the user can enjoy visually.

[0891] Step 9: Buy the book and complete the revenue share

[0892] User: Purchase the recommended book.

[0893] Server: Records purchase information and performs revenue share with rights holders.

[0894] Specific examples

[0895] For example, suppose a user is looking for books in the mystery genre. The user logs into the system through a terminal and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery novel from the database, and generates a recommendation result. The recommendation result is displayed on the terminal, and the user can select the book of interest from among them and purchase it. Once the purchase is completed, the server records the information and distributes the sales revenue appropriately to the distributor and rights holder.

[0896] If a recommended book is written in a foreign language, the server will use an automatic translation function to provide a Japanese version. Furthermore, illustrations and videos can be automatically generated to enhance the reader's visual enjoyment. The story of the selected novel is also provided to the user as a short video, allowing users to enjoy the content in a variety of ways.

[0897] In this way, the system of the present invention recommends optimal books to the user in a consistent manner, providing a rich reading experience.

[0898] The processing flow will be explained below.

[0899] Step 1:

[0900] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[0901] Step 2:

[0902] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[0903] Step 3:

[0904] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is necessary to create a user profile.

[0905] Step 4:

[0906] Terminal: Sends information entered by the user to the server.

[0907] Step 5:

[0908] Server: Analyzes the received user information and creates a user profile based on the user's reading habits and interests.

[0909] Step 6:

[0910] Server: Based on the user profile, a recommendation algorithm is used to select the most suitable books and generate a recommendation list.

[0911] Step 7:

[0912] Terminal: Displays the recommendation results received from the server to the user, who can then select books of interest from this list.

[0913] Step 8:

[0914] User: Select a book from the recommendation list via the device and purchase or preview it.

[0915] Step 9:

[0916] Server: If the book selected by the user is in a foreign language, it uses an automatic translation function to generate a Japanese version and sends it to the terminal.

[0917] Step 10:

[0918] Server: Using AI technology, illustrations and videos related to the selected books are automatically generated and provided as additional content.

[0919] Step 11:

[0920] Server: Furthermore, it analyzes the story of the novel and generates a short video, which is also provided to the user.

[0921] Step 12:

[0922] User: Purchases books that interest them. After purchase, the books are available as e-books.

[0923] Step 13:

[0924] Server: Records purchase information and distributes revenue to relevant rights holders and distributors. This process is automated.

[0925] Example 1

[0926] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0927] Current book recommendation systems lack the ability to provide personalized recommendations that reflect users' preferences and reading history, and they face challenges in providing multilingual and visual content. Furthermore, the process of digitizing and analyzing books is manual and inefficient. Furthermore, the post-purchase revenue sharing mechanism is unclear, making it difficult to fairly distribute rewards to rights holders.

[0928] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0929] In this invention, the server includes a means for electronically scanning physical books and storing them as digital data, a means for analyzing the collected book data and extracting categories, summaries, and major topics, and a means for recommending books based on user profiles. This enables highly accurate book recommendations that reflect users' preferences and reading history, and also enables multilingual and visual content provision. A post-purchase revenue sharing mechanism is also established, enabling fair reward distribution.

[0930] "Physical Book" means a book in a non-electronic format made of paper or other material.

[0931] "Electronically scanned" refers to the process of converting a physical book into digital data using a device such as a scanner.

[0932] "Digital data" refers to information stored in an electronically processable form.

[0933] "Analysis" refers to the process of automatically extracting meaningful information from digital data.

[0934] A "category" refers to a group of books classified based on their content or theme.

[0935] A "summary" is a brief summary of the book's contents.

[0936] "Major topics" refers to important themes or topics contained in the book's content.

[0937] "Input means" refers to the method or interface by which a user provides information to a system.

[0938] A "profile" refers to a collection of data that reflects a user's preferences and behavior.

[0939] "Recommendation method" refers to the process of selecting and providing books based on a user's profile.

[0940] The "means for displaying" refers to an interface that visually presents the recommended book information to the user.

[0941] "Machine translation" refers to technology that automatically converts text between multiple languages.

[0942] "Means for automatically generating illustrations and videos" refers to the process of using artificial intelligence technology to automatically create visual content related to the content of a book.

[0943] "Video content creation" refers to the process of visualizing the story of a book and providing it in video format.

[0944] "Revenue share" refers to a mechanism for fairly distributing revenue.

[0945] "Means for dynamically updating reading habits" refers to technology that automatically updates a user's profile based on their new reading history and preferences.

[0946] MODE FOR CARRYING OUT THE INVENTION

[0947] This invention is a system that recommends the most suitable books to users in a personalized manner. Specifically, it analyzes electronically digitized book data and provides recommendations that match the user's preferences. This system is mainly composed of three entities: a server, a terminal, and a user.

[0948] The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations. The terminal receives input from the user and displays the recommendations. The user provides their preferred genres and past reading history through the terminal, and selects and purchases the books displayed as recommended results.

[0949] The server first uses a dedicated scanner (e.g., Fujitsu ScanSnap iX1500) to electronically scan the physical book and store it as digital data. This digital data is then converted into text data using OCR technology (e.g., Tesseract OCR). The converted text data is then saved in the server's storage.

[0950] The server then uses natural language processing tools (e.g., Apache OpenNLP) to analyze the text data from the digitized books, including tokenization, part-of-speech tagging, and named entity extraction, and stores categories, summaries, and major topics in a database (e.g., MySQL).

[0951] Users log in to the system using a terminal (e.g., a browser-based interface) and enter their preferred genres and past reading history, such as "mysteries" or "a list of books read in the past three years." The terminal checks the format of this input data in real time and sends it to the server as well-formed data.

[0952] The server then uses Node.js and Python scripts to analyze the received user data and generate detailed user profiles, which include information that classifies and predicts the user's reading habits using machine learning algorithms (e.g., k-means clustering), and store this information in a database.

[0953] The server then uses an AI model (e.g., TensorFlow) to select the most suitable books from the database based on the generated user profile. An example prompt might be, "Recommend the most suitable mystery novel based on the user's past reading history and preferred genres. Include summaries, main topics, and categories in the recommendation. Examples include 'famous detective series' and 'famous mystery novel series.'"

[0954] The recommendation results sent from the server are then displayed on the device using a web application using React.js, allowing users to check detailed information about the recommended books (summary, category, main topics).

[0955] If the recommended book is in a foreign language, the server will automatically translate it using the Google Translate API. It can also use OpenAI's DALL-E API to automatically generate illustrations and video content, allowing users to enjoy the book's contents in a variety of ways.

[0956] Furthermore, when a purchase is made, the server uses the Stripe API to record the purchase information and distribute the appropriate revenue share to the rights holder. In this way, the system provides users with personalized book recommendations, enhancing their reading experience.

[0957] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0958] Step 1:

[0959] Electronic scanning and digitization of books

[0960] Server: Electronically scans physical books using a specialized scanner. It accepts the physical book as input and produces scanned image data as output, thereby capturing the contents of the book in image format.

[0961] Server: Converts scanned image data into text data using OCR technology. The input is image data and the output is text data. Specifically, OCR technology performs character recognition and extracts text information from the image.

[0962] Step 2:

[0963] Data analysis of books and extraction of categories, summaries, and main topics

[0964] Server: Analyzes text data using natural language processing tools. The input is text data, and the output is the analysis results: categories, summaries, and main topics. This converts the text data into meaningful information.

[0965] Server: Stores the analyzed data in a database. Specifically, it extracts categories, summaries, and main topics from the text data and inserts them into a MySQL database.

[0966] Step 3:

[0967] Collecting user preference information

[0968] User: Logs into the system through a terminal and inputs his / her preferred genre and past reading history. The input provides his / her preferred genre (e.g., mystery, science fiction) and past reading history. The output is the formatted user input data.

[0969] Terminal: Checks the format of the user's input data in real time and sends it to the server as well-formed data. Specific operations include checking the input of required fields and validating the data format.

[0970] Step 4:

[0971] User profile analysis

[0972] Server: Analyzes the received user data and generates a detailed user profile. The input is the user's preferences, and the output is a detailed user profile. This digitizes the user's reading habits and hobbies. Specifically, it runs a machine learning algorithm using Node.js and Python scripts.

[0973] Step 5:

[0974] Book recommendation generation

[0975] Server: Based on the generated user profile, an AI model is used to select the most suitable books. The input is a detailed user profile, and the output is a list of recommended books. This allows the most suitable books to be selected for the user. Specifically, TensorFlow is used to process the profile data and extract the books that best fit the user profile.

[0976] Step 6:

[0977] Displaying recommendations

[0978] Terminal: Displays the recommendation results received from the server to the user. The input is a list of recommended books, and the output is a visual display of the book information. This allows the user to check the list of recommended books. Specifically, the display is done using a web application using React.js.

[0979] Step 7:

[0980] Automatic translation and additional content provision

[0981] Server: Uses the Google Translate API to automatically translate recommended books. The input is the text data of the recommended book, and the output is the translated text data. This allows users to enjoy the book in their native language. Specifically, the API is called to obtain the translation results.

[0982] Server: Generates illustrations and video content using OpenAI's DALL-E API. The input is the book's content data, and the output is the generated visual content. This allows for a deeper understanding of the book. Specifically, the server calls the API to generate related illustrations and videos.

[0983] Step 8:

[0984] Turning novel stories into video content

[0985] Server: Analyzes the story of a book selected by the user and generates it as a short video. The input is the book's story data, and the output is a short video. This allows users to enjoy the story visually. Specifically, the video is created using Adobe Premiere Pro and an AI video generation tool.

[0986] Step 9:

[0987] Buying books and running revenue shares

[0988] User: Purchases the recommended book using a device. The input is a purchase intention and payment information, and the output is a purchase confirmation and a download link for the book. The book then becomes the user's property. Specific actions include paying with a credit card or electronic money.

[0989] Server: Records purchase information and performs revenue sharing using the Stripe API. The input is purchase information, and the output is confirmation of revenue distribution to rights holders. This ensures fair revenue distribution. Specific operations include storing purchase data and distributing revenue at the appropriate rate.

[0990] (Application example 1)

[0991] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0992] Conventional book recommendation systems are generally intended for online use, and when users visit a bookstore, there are limited ways to obtain book information on the spot. This makes it difficult for users to find books that suit their preferences, and the purchasing experience is not sufficiently improved. In addition, differences in reading devices and the lack of dynamic updates to user profiles make it difficult to provide appropriate recommendations in real time.

[0993] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0994] In this invention, the server includes a means for reading a code installed in the store, a means for displaying in-store book information in real time based on the read code, and a means for recommending books in real time based on a user profile using a generative AI model, thereby enabling a user to receive optimal book recommendations in real time when searching for books in the store.

[0995] "Means for electronically scanning books and storing them as digital data" refers to a function for converting books into digital data using equipment such as an electronic scanner or camera, and storing the data on a server or database.

[0996] "Means of analyzing collected book data and extracting categories, summaries, and key topics" refers to a function that analyzes the contents of a book using natural language processing technology, etc., and automatically extracts the genre to which the book belongs, a summary of the contents, and important topics.

[0997] "Means for users to input their favorite genres and past reading history" refers to an interface or mechanism that allows users to input information about their interests and the books they have read.

[0998] "Means for analyzing information entered by the user and creating a profile" refers to a function that analyzes information on genres and reading history provided by the user and generates a profile that reflects the user's reading tendencies and preferences.

[0999] A "means for recommending books based on a user profile" is an algorithm or system for recommending the most suitable book to a user based on the generated user profile.

[1000] The "means for displaying recommended books" refers to a display or a screen of a mobile device that visually presents information about recommended books to the user.

[1001] "Means for providing books in multiple languages ​​through machine translation" is a function that translates the contents of recommended books into the user's desired language through machine translation, making them available in multiple languages.

[1002] "Means for automatically generating illustrations and videos" refers to a function that uses artificial intelligence and image generation technology to automatically generate related illustrations and videos based on the content of a book.

[1003] "A means of turning a novel story into video content" is a function that analyzes the content of a novel and generates video content such as a short film or animation based on that story.

[1004] "Means for reading codes installed in stores" refers to a function that allows you to read codes installed in stores, such as QR codes and barcodes, using a smartphone or dedicated device.

[1005] "Means for displaying in-store book information in real time based on the scanned code" is a function for displaying detailed information about books in the store in real time on the user's device based on the scanned QR code or barcode.

[1006] "Means of recommending books in real time based on user profile using a generative AI model" is a function that uses an artificial intelligence model to recommend the most suitable book on the spot based on the user's profile information.

[1007] The present invention is a system for recommending the most suitable book to a user in a personalized manner, and is mainly composed of a server, a terminal, and a user. Specific embodiments for implementing this system will be described below.

[1008] 1. Server operation

[1009] The server has the following roles:

[1010] 1. Electronic scanning and digital storage of books:

[1011] The server scans the books using an electronic scanner or camera and stores the content as digital data using cloud storage services such as AWS S3 and Firebase Storage.

[1012] 2. Data Analysis:

[1013] Analyze the content of digitized books and extract categories, summaries, and key topics using natural language processing techniques, leveraging Python libraries (e.g., spaCy and NLTK) and AI models (e.g., BERT).

[1014] 3. Create a user profile:

[1015] The server analyzes the data provided by the user (preferred genres, past reading history) and creates a detailed user profile, which is stored in a cloud database (e.g., Firebase Firestore).

[1016] 4. Recommendation Generation:

[1017] Generate book recommendations based on user profiles using generative AI models (e.g., GPT-3) and integrating multiple recommendation algorithms.

[1018] 5. Automatic translation and additional content generation:

[1019] The content of the book is automatically translated as needed and provided in multiple languages. In addition, illustrations and related videos are automatically generated using a generative AI model and provided to users.

[1020] 6. Turning novel stories into video content:

[1021] The story of the book selected by the user is generated as video content. This step uses generative AI models and machine learning techniques.

[1022] 7. Revenue Share Implementation:

[1023] When a book is purchased based on a recommendation, the purchase information is recorded and the sales revenue is distributed to the rights holder.

[1024] 2. Device operation

[1025] The device has the following features:

[1026] 1. Data input from the user:

[1027] The device provides an interface for users to input their preferred genres and past reading history.

[1028] 2. Displaying recommendation results:

[1029] The recommendation results received from the server are displayed to the user using a mobile device such as a smartphone or tablet.

[1030] 3. QR Code Scanning:

[1031] Read QR codes and barcodes installed in stores to obtain book information. To do this, use a QR code reader library (e.g., Zxing).

[1032] 4. Information display based on the code read:

[1033] Based on the code read, the terminal displays detailed information about the book to the user in real time.

[1034] 3. User operations

[1035] The user performs the following operations:

[1036] 1. Enter the data:

[1037] Users input their preferences and past reading history through the device.

[1038] 2. Check the book recommendations:

[1039] Check the recommendations and choose the books that interest you.

[1040] 3. Scan the QR code:

[1041] Scan the QR code of a book you are interested in in the store to view detailed information.

[1042] 4. Buy a book:

[1043] Purchase recommended books or books that interest you.

[1044] Specific examples

[1045] For example, suppose a user visits a physical store and is looking for a book in the mystery genre. The user logs in using a smartphone app and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery book from the database, and generates a recommendation result. When the user scans the QR code in the store, information about the book is displayed in real time, along with additional content (e.g., illustrations and related videos).

[1046] Examples of prompts for AI models

[1047] "Generate a summary of a mystery novel in which a detective solves a complex case. Include the main plot and key details."

[1048] As a result, this system can recommend optimal books to users in a consistent manner, enriching the purchasing experience in physical stores.

[1049] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1050] Step 1:

[1051] The server electronically scans the book and stores it as digital data. The book is scanned using a scanner or camera to generate image data. This is then converted into text data using OCR software and stored in cloud storage. The input to this process is the physical book, and the output is digital text data.

[1052] Step 2:

[1053] The server analyzes the collected digital data of books and extracts categories, summaries, and main topics. It uses Python libraries and natural language processing techniques (e.g., spaCy and NLTK) to analyze the text data and extract the necessary information. The input of this process is the digital text data generated in step 1, and the output is categories, summaries, and main topics.

[1054] Step 3:

[1055] Users input their preferred genres and past reading history using a device. The input interface is a smartphone or tablet, and the user's input data is stored in a cloud database. The input for this process is the genre and reading history data entered by the user, and the output is the user profile data stored in the cloud database.

[1056] Step 4:

[1057] The server analyzes the user's input and creates a detailed user profile. Based on the user data stored in the cloud, an analytical algorithm is used to analyze the user's reading habits. The input to this process is the user data stored in the cloud database, and the output is a detailed user profile.

[1058] Step 5:

[1059] The server uses a generative AI model (e.g., GPT-3) to recommend the most suitable books in real time based on the user profile. The server matches the user profile with book information stored in a database and generates recommendations using the AI ​​model. The input to this process is a detailed user profile and a book database, and the output is a recommendation result.

[1060] Step 6:

[1061] The terminal displays the recommendation results to the user. Recommended book information is displayed on the screen of a smartphone or tablet. The input to this process is the recommendation results received from the server, and the output is recommended book information visually presented to the user.

[1062] Step 7:

[1063] A user reads a QR code attached to a book in a store. The user then scans the QR code using the camera on their smartphone to obtain the book information. The input to this process is the QR code attached to the book, and the output is the book information read from the QR code.

[1064] Step 8:

[1065] The terminal displays in-store book information in real time based on the read QR code information. Detailed book information is displayed on the terminal based on the information obtained from the QR code. The input for this process is the data read from the QR code, and the output is book information displayed in real time.

[1066] Step 9:

[1067] The server generates automatic translations and additional content (illustrations and videos) as needed. A generative AI model is used for translation, and image generation AI such as DALL-E is used to generate illustrations and videos. The input for this process is recommendation results and book content data, and the output is the translated text and generated illustrations and video content.

[1068] Step 10:

[1069] When a user purchases a book, the server executes the revenue share. It records the purchase information and distributes the sales appropriately to the rights holder. The input of this process is the user's purchase information, and the output is the result of the revenue share execution.

[1070] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1071] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[1072] System Overview

[1073] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1074] Program processing flow (natural language explanation)

[1075] Step 1: Electronic scanning and digitization of the book

[1076] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[1077] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[1078] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[1079] Step 3: Collecting user preferences

[1080] User: Logs into the system through a terminal and enters his / her preferred genres and past reading history.

[1081] Step 4: Submit user information

[1082] Terminal: Sends information entered by the user to the server.

[1083] Step 5: Create a user profile

[1084] Server: Analyzes the received user information and creates a detailed user profile.

[1085] Step 6: Emotion Recognition with the Emotion Engine

[1086] Server: Analyzes real-time emotion data collected from the user's device using an emotion engine to recognize the user's emotional state.

[1087] Step 7: Emotion-based recommendation optimization

[1088] Server: Optimizes recommendations based on user profile and emotional state, choosing books that fit the user's current emotions.

[1089] Step 8: Generate and display the recommendation list

[1090] Server: Generates an optimized recommendation list.

[1091] Terminal: Displays the recommendation results received from the server to the user.

[1092] Step 9: Select and purchase books

[1093] User: Select a book from the recommended list and purchase or preview it.

[1094] Step 10: Automated translation and additional content provision

[1095] Server: If the selected book is in a foreign language, an automatic translation function is used to generate a Japanese version and send it to the device. In addition, AI technology is used to automatically generate illustrations and videos, which are provided as additional content.

[1096] Step 11: Turn your story into video content

[1097] Server: Generates the story of the novel as a short video and provides it to the user.

[1098] Step 12: Record purchase information and execute revenue share

[1099] Server: Records purchase information and executes revenue share with relevant rights holders and distributors.

[1100] Specific examples

[1101] For example, suppose a user is looking for a romance novel. The user logs into the system through their device and enters their desired romance and past reading history. Furthermore, the device's camera and sensors are used to transmit the user's emotional data in real time to the server. The server then creates a user profile based on this information and uses an emotion engine to analyze the user's current emotional state.

[1102] Because the user is feeling stressed, the emotion engine prioritizes relaxing romance novels and adds them to the recommendation list. From the recommendation list displayed on the device, the user can select a book that interests them and purchase it. If the selected book is in a foreign language, the server uses an automatic translation function to provide a Japanese version. Users can also view illustrations and short videos generated by AI.

[1103] In this way, the system of the present invention, combined with the emotion engine, provides advanced recommendations that take into account the user's emotional state, enabling a rich reading experience.

[1104] The processing flow will be explained below.

[1105] Step 1:

[1106] Server: The book is scanned electronically and stored as digital data in a database on the server. In this step, OCR technology is used to extract text information for each page and store it as metadata.

[1107] Step 2:

[1108] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Using natural language processing technology, it extracts themes and keywords from the content and organizes the database based on these.

[1109] Step 3:

[1110] User: Logs in to the system using a terminal and enters information such as preferred genres and past reading history, which allows the collection of basic information for a user profile.

[1111] Step 4:

[1112] Terminal: Sends information entered by the user to the server, which uses it to create and update the user profile.

[1113] Step 5:

[1114] Server: Analyzes the received user information and creates a detailed user profile, including data about the user's reading habits and interests.

[1115] Step 6:

[1116] On-device: Using the device's camera and sensors, we collect real-time emotional data from the user. This data is obtained through facial recognition and biometric sensors.

[1117] Step 7:

[1118] Terminal: Sends collected emotional data to the server, which uses it to analyze the user's emotional state.

[1119] Step 8:

[1120] Server: The emotion engine analyzes the transmitted emotion data and recognizes the user's current emotional state. For example, it identifies whether the user is happy, sad, or stressed.

[1121] Step 9:

[1122] Server: Based on the user profile and emotional state, the recommendation algorithm selects the most suitable book and optimizes the recommendation. The selection criteria are dynamically adjusted taking into account the emotion recognition results.

[1123] Step 10:

[1124] Server: Generates an optimized recommendation list and sends it to the device. The recommendation list includes books that are appropriate for the user's emotional state.

[1125] Step 11:

[1126] Terminal: The terminal displays the recommendation results received from the server to the user. The user can select a book of interest from the recommendation list and view detailed information and samples.

[1127] Step 12:

[1128] User: Can select a book from the recommended list and make a purchase. Once purchased, the book will be available as an e-book.

[1129] Step 13:

[1130] Server: If the selected book is in a foreign language, it will use the automatic translation function to provide the Japanese version. This translation process runs in the background.

[1131] Step 14:

[1132] Server: Using AI technology, it automatically generates illustrations and videos related to the selected book and sends them to the device as additional content, enriching the reader's visual experience.

[1133] Step 15:

[1134] Server: The server then generates a short video of the story and provides it to the user. In this step, the main parts of the story are visualized to provide visual enjoyment.

[1135] Step 16:

[1136] Server: Records purchase information and executes revenue sharing with relevant rights holders and distributors. This process is performed automatically based on sales information.

[1137] Example 2

[1138] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1139] In today's information overload, it is difficult for users to find the book that best suits them. Advanced filtering technology is required to select books that best suit the user's preferences and emotional state from the vast number of books available. Furthermore, a system that can smoothly provide foreign language books and additional content is also needed.

[1140] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1141] In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres and past reading history, means for analyzing the information input by the user and creating a profile, means for recommending books based on the user profile, means for displaying recommended books, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, means for converting novel stories into video content, means for collecting real-time emotional data, means for analyzing the emotional data and recognizing the user's emotional state, and means for recommending books based on the user's emotional state. This allows the server to provide optimal books based on the user's preferences and emotional state, enabling a rich reading experience.

[1142] "Means of electronically scanning books and storing them as digital data" refers to a method of electronically reading paper books using a scanner and storing the data in a database as text data.

[1143] "Methods for analyzing collected book data and extracting categories, summaries, and main topics" refers to a method of analyzing the contents of digitized books using OCR technology and natural language processing technology, and extracting the categories to which the books belong, summaries, and main points of discussion.

[1144] "Means for users to input their favorite genres and past reading history" refers to an interface that allows users to provide the system with their favorite genres and a list of books they have read so far.

[1145] "Means for analyzing information entered by the user and creating a profile" refers to a method for analyzing information provided by the user, such as genre and reading history, using machine learning models to generate a detailed user profile.

[1146] The "means for recommending books based on a user profile" is a method for providing a list of books that are most suitable for each individual user based on the generated user profile.

[1147] The "means for displaying recommended books" is an interface for displaying the recommendation list sent from the server on the user's terminal.

[1148] "Means for providing books in multiple languages ​​through automatic translation" refers to a method in which, if the selected book is in a foreign language, the book is translated into another language using automatic translation technology and provided to the user.

[1149] "Means for automatically generating illustrations and videos" refers to a method of using AI technology to generate illustrations and short videos based on the content of a book and provide them to users as additional content.

[1150] "Means for turning a novel story into video content" refers to a method of using a video generation algorithm to generate and provide a short video using the contents of a novel as input data.

[1151] "Means for collecting real-time emotional data" refers to an interface that uses cameras and sensors to obtain emotional data in real time from the user's facial expressions, voice, etc.

[1152] The "means for analyzing emotion data and recognizing the user's emotional state" is an emotion recognition algorithm for analyzing the collected emotion data and recognizing the user's current emotional state.

[1153] The "means for recommending books based on the user's emotional state" is a method for combining a user profile and an emotional state to recommend books that suit the user's current emotions.

[1154] The present invention is an advanced system that recommends optimal books to a user, recognizes the user's emotional state, and optimizes the recommendations. Hereinafter, a detailed description will be given of an embodiment of the present invention.

[1155] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1156] First, the server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap), extracts the text information using OCR technology (e.g., Tesseract), and stores it in a database. This step generates digital data.

[1157] The server then analyzes the collected digital data using algorithms based on natural language processing technology (e.g., SpaCy or NLTK) to extract book categories, summaries, and main topics.

[1158] Users log in to the system through their devices and enter their favorite genres and past reading history. iOS or Android applications are used on the devices. The information entered by the user is sent to the server using the HTTPS protocol. The server analyzes the information received and creates a detailed user profile using machine learning models (e.g., Scikit-learn or TensorFlow).

[1159] Furthermore, the user's real-time emotional data is collected by the emotion engine. The emotional data acquired using the device's camera and sensors is analyzed by an emotion recognition API (e.g., AWS Rekognition or Microsoft Azure Emotion API). The server recognizes the user's emotional state based on the analysis results.

[1160] The server then optimizes the recommendations based on the user's profile and current emotional state. A reinforcement learning algorithm is used to select books that fit the emotional state. An optimized recommendation list is then generated and sent to the device, which displays the list in the user interface.

[1161] Users can select books of interest from the recommendation list and purchase or preview them. The purchase process is carried out through an e-commerce system (e.g., Amazon or Rakuten). If the selected book is in a foreign language, an automatic translation function (e.g., Google Translate API) is used to generate a Japanese version. Furthermore, illustrations and videos are automatically generated using AI technology (e.g., DALL-E or GAN) and provided as additional content.

[1162] As a concrete example, let's say a user is looking for a romance novel. The user logs into the system through their device and enters their romance and past reading history. Emotional data is sent to the server in real time via the device's camera and sensors. The server creates a user profile based on this information, and an emotion engine analyzes the user's current emotional state. If the emotion engine determines that the user is feeling "stressed," it will prioritize recommendations of relaxing romance novels.

[1163] In this way, advanced recommendations based on the user's emotional state and preferences are realized.

[1164] Example prompt sentence:

[1165] "Recommend positive, relaxing romance novels based on the user's reading history and real-time emotional data. Users are stressed and want content that will help them feel better."

[1166] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1167] Step 1:

[1168] The server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap). The input is the paper book, and the output is the scanned image data. This image data is converted into text information using OCR technology (e.g., Tesseract) and stored in a database. The specific operation is "start the scanner, place the book, and press the scan button."

[1169] Step 2:

[1170] The server analyzes the collected text data. The input is the text data stored in step 1, and the output is categories, summaries, and main topics. Natural language processing techniques (e.g., SpaCy or NLTK) are used to analyze the text data and extract categories, summaries, and main topics. The specific operation is to "input the text data into the algorithm and extract the analysis results."

[1171] Step 3:

[1172] A user logs into the system through a terminal and enters their favorite genres and past reading history. The input is the user's preferences and past reading history, and the output is the user's input data. The specific operation is "entering information into a form displayed on the terminal screen and clicking the submit button."

[1173] Step 4:

[1174] The terminal sends information entered by the user to the server using the HTTPS protocol. The input is the data entered by the user, and the output is the data sent to the server. The specific operation is "clicking the send button to send the entered information to the server."

[1175] Step 5:

[1176] The server analyzes the received user information and creates a detailed user profile using a machine learning model (e.g., Scikit-learn or TensorFlow). The input is user information and the output is a user profile. The specific operation is to "input the received user information into the machine learning model and generate a profile."

[1177] Step 6:

[1178] The server collects real-time emotional data. The input is the user's facial expression and voice data obtained from the device's camera and sensors, and the output is emotional data. Specifically, the system works by "collecting the user's facial expression data through the camera and acquiring voice data from the sensors."

[1179] Step 7:

[1180] The server analyzes the emotion data and recognizes the user's emotional state. The input is the collected emotion data, and the output is the recognition result of the emotional state. The emotion data is analyzed using an emotion recognition API (for example, AWS Rekognition or Microsoft Azure Emotion API). The specific operation is to "input the emotion data into the emotion recognition API and obtain the emotional state."

[1181] Step 8:

[1182] The server optimizes recommendations using a reinforcement learning algorithm based on the user profile and emotional state. The input is the user profile and the recognition result of the emotional state, and the output is an optimized recommendation list. The specific operation is "input the user profile and emotional state into the algorithm and obtain the recommendation result."

[1183] Step 9:

[1184] The server generates an optimized recommendation list and sends it to the terminal. The input is the optimized recommendation results, and the output is the recommendation list sent to the terminal. The terminal displays the received recommendation results on the user interface. The specific operation is "generate a recommendation list and send it to the terminal."

[1185] Step 10:

[1186] The user selects a book from the recommendation list and purchases or previews it. The input is the recommendation list, and the output is the selected book. The specific operation is "select a book from the recommendation list and click the purchase button."

[1187] Step 11:

[1188] If the selected book is in a foreign language, the server uses an automatic translation function to generate a Japanese version and sends it to the terminal. The input is the foreign language book data, and the output is the translated book data. The specific operation is to "input the foreign language text into the translation API and obtain the Japanese text."

[1189] Step 12:

[1190] The server automatically generates illustrations and videos using AI technology (for example, DALL-E or GAN) and provides them as additional content. The input is the book content data, and the output is the generated illustrations and videos. Specifically, the operation is to "use the book content as input data and generate illustrations and videos using AI technology."

[1191] Step 13:

[1192] The server generates a short video from the story of the novel and sends it to the terminal. The input is the story data of the novel, and the output is the generated short video. The specific operation is to "input the story data of the novel into the video generation algorithm and generate a video file."

[1193] Step 14:

[1194] The server records purchase information and executes revenue sharing with the relevant rights holders and distributors. The input is purchase data, and the output is revenue share distribution information. This process is carried out using blockchain technology (e.g., Ethereum Smart Contracts). The specific operation is to "send purchase information to the blockchain network and execute the smart contract."

[1195] (Application example 2)

[1196] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1197] Conventional book recommendation systems only recommend books based on a user's reading history and genre preferences, and are unable to provide personalized recommendations that take into account the user's real-time emotional state. This makes it difficult to provide timely recommendations that match each user's emotions and mood, resulting in limited improvement in user satisfaction. Furthermore, they are unable to provide sufficient multilingual book services or interactive experiences in virtual stores.

[1198] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres, past reading history, and emotional state, means for analyzing the information and emotional data input by the user to create a profile and perform real-time emotion recognition, means for recommending books based on the user profile and emotional state, means for displaying recommended books and providing virtual store navigation, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, and means for converting novel stories into video content. This enables timely recommendations that take the user's emotional state into consideration and an interactive virtual store experience.

[1199] "Electronically scanning a book" is the process of optically scanning a paper book and storing it in a digital format.

[1200] "Storing as digital data" means storing electronically read information in a database or storage system.

[1201] "Analyzing collected book data" means analyzing electronically acquired book data using algorithms and software to extract information.

[1202] "Extract categories, summaries, and major topics" means categorizing the contents of a book and extracting key points and summaries.

[1203] "User inputs preferred genres, past reading history, and emotional state" means that the user provides the system with their own reading preferences, books they have read in the past, and their current emotional state via their terminal.

[1204] "Analyzing information and emotional data entered by the user to create a profile and perform real-time emotion recognition" means generating a detailed profile of the user based on data provided by the user and emotional information collected in real time.

[1205] "Recommending books based on user profile and emotional state" means recommending appropriate books based on the created user profile and real-time emotional information.

[1206] "Displaying recommended books and providing virtual store navigation" means displaying recommended books on the user's device and providing the experience of exploring a bookstore in a virtual space.

[1207] "Providing books in multiple languages ​​through automatic translation" is a function that automatically translates selected books into different languages ​​and provides them to users.

[1208] "Automatically generating illustrations and videos" means using AI technology to automatically create illustrations and short videos based on the content of a book.

[1209] "Turning a novel story into video content" means creating a short video based on the content of the novel and providing it to users as visual content.

[1210] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[1211] System Overview

[1212] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1213] Program Description

[1214] Hardware and software used

[1215] 1. Hardware

[1216] Smartphones: Equipped with cameras and processors, they collect user input and emotional data and support the overall operation of apps.

[1217] Server: Manages user data and book data, and performs large-scale data analysis and emotion recognition.

[1218] 2. Software

[1219] OpenCV: Used as an image processing library to perform preprocessing for face detection and emotion recognition.

[1220] TensorFlow / Keras: Build an emotion recognition model and analyze users' real-time emotions.

[1221] Requests: Send an HTTP request to the recommendation API to get the recommendation results.

[1222] Specific examples

[1223] For example, suppose a user is looking for a romance novel. The user logs into the smartphone application "Virtual Bookstore Emo" and inputs their past reading history and preferred genres. Furthermore, the smartphone's camera is used to read their facial expressions, and the data is sent to the server in real time.

[1224] The server creates a user profile based on this data and uses an emotion engine to analyze the user's current emotional state. If the emotion engine determines that the user is feeling stressed, it will prioritize relaxing romance novels and add them to the recommendation list. This allows for more personalized book recommendations for the user.

[1225] Recommended books are displayed through a virtual bookstore interface on the application. Users can virtually navigate the bookstore to select a book, and if the book they choose is in a foreign language, an automatic translation function will provide a Japanese version. Users can also view AI-generated illustrations and short videos, further enriching their reading experience.

[1226] Prompt Sentence Examples

[1227] "Recommend relaxing fantasy novels based on the user's real-time emotions and past reading history."

[1228] In this way, the system of the present invention can provide advanced recommendations that take into account the user's emotional state, providing a rich reading experience.

[1229] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1230] Step 1:

[1231] The user logs in to the application through the device and inputs their preferred genres and past reading history. At this point, the device collects the user's input data (genre, reading history).

[1232] Input: Favorite genres, past reading history

[1233] Output: User input data

[1234] Step 2:

[1235] The device uses the smartphone camera to capture the user's facial expressions and collect emotional data in real time.The device then uses an emotion recognition model (TensorFlow / Keras) to analyze emotions from the collected facial data.

[1236] Input: User's face image

[1237] Output: User emotion data

[1238] Step 3:

[1239] The device sends the user's input data (genre, reading history) and emotion data to the server, which stores the data for further processing.

[1240] Input: User input data, emotion data

[1241] Output: User data sent to the server

[1242] Step 4:

[1243] The server creates a user profile based on the received user data, which includes analyzing input data and emotional data. The profile is then used in the next recommendation stage.

[1244] Input: User data (input data, emotion data)

[1245] Output: User profile

[1246] Step 5:

[1247] The server optimizes book recommendations based on the user's profile and emotional state, using AI technology to select the most suitable books for each user.

[1248] Input: User profile, emotional state

[1249] Output: Recommended book list

[1250] Step 6:

[1251] The server generates a recommended book list and sends it to the terminal, allowing the user to receive a list of the most suitable books in real time.

[1252] Input: Recommended book list

[1253] Output: Recommended book list sent to the device

[1254] Step 7:

[1255] The terminal displays the received recommended book list on the interface of the virtual bookstore, and the user can select books while navigating the virtual bookstore.

[1256] Input: Recommended book list

[1257] Output: Display content in the virtual store

[1258] Step 8:

[1259] If a user selects a recommended book, the server uses machine translation to provide the book in multiple languages, and automatically generates illustrations and short videos if necessary.

[1260] Input: Selected book, trigger data for translation function

[1261] Output: translated book, generated content

[1262] Step 9:

[1263] Users can browse translated books and generated content on their devices and enjoy a reading experience, thereby providing users with a diverse reading and digital content experience.

[1264] Input: translated book, generated content

[1265] Output: User's reading and digital content experience

[1266] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1267] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1268] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1269] [Fourth embodiment]

[1270] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1271] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1272] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1273] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1274] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1275] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1276] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1277] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1278] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1279] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1280] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1281] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1282] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1283] The present invention is a system for personalized book recommendations to users. This system utilizes AI technology to collect and analyze book data and provide recommendations tailored to the user's preferences. Specific embodiments of this system are described below.

[1284] System Overview

[1285] This system consists of three entities: a server, a terminal, and a user. The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations, while the terminal receives input from the user and displays the recommendations. The user provides their preferred genres and reading history through the terminal, and selects and purchases books displayed as recommended results.

[1286] Program processing flow (natural language explanation)

[1287] Step 1: Electronic scanning and digitization of the book

[1288] Server: Electronically scans books and stores them as digital data, making the physical books available in digital form.

[1289] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[1290] Server: Analyzes the content of digitized books and automatically extracts categories, summaries, and major topics. This information is stored in a database.

[1291] Step 3: Collecting user preferences

[1292] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is sent to the server as a user profile.

[1293] Step 4: Analyzing user profiles

[1294] Server: Analyzes the user information sent and creates a detailed user profile, which allows the user to understand their reading habits and interests.

[1295] Step 5: Generate book recommendations

[1296] Server: Based on the created user profile, the server selects the most suitable books from the database and generates recommendations.

[1297] Step 6: Displaying recommendations

[1298] Terminal: Displays the recommendation results received from the server to the user, who can then select books that interest them.

[1299] Step 7: Automated translation and additional content provision

[1300] Server: If necessary, the recommended books are automatically translated and provided in multiple languages. In addition, illustrations and related videos are automatically generated using AI and provided as additional content.

[1301] Step 8: Turning the story of your novel into video content

[1302] Server: Analyzes the story of the book selected by the user and generates a short video, which the user can enjoy visually.

[1303] Step 9: Buy the book and complete the revenue share

[1304] User: Purchase the recommended book.

[1305] Server: Records purchase information and performs revenue share with rights holders.

[1306] Specific examples

[1307] For example, suppose a user is looking for books in the mystery genre. The user logs into the system through a terminal and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery novel from the database, and generates a recommendation result. The recommendation result is displayed on the terminal, and the user can select the book of interest from among them and purchase it. Once the purchase is completed, the server records the information and distributes the sales revenue appropriately to the distributor and rights holder.

[1308] If a recommended book is written in a foreign language, the server will use an automatic translation function to provide a Japanese version. Furthermore, illustrations and videos can be automatically generated to enhance the reader's visual enjoyment. The story of the selected novel is also provided to the user as a short video, allowing users to enjoy the content in a variety of ways.

[1309] In this way, the system of the present invention recommends optimal books to the user in a consistent manner, providing a rich reading experience.

[1310] The processing flow will be explained below.

[1311] Step 1:

[1312] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[1313] Step 2:

[1314] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[1315] Step 3:

[1316] User: Logs in to the system through a terminal and inputs his / her preferred genres and past reading history. This information is necessary to create a user profile.

[1317] Step 4:

[1318] Terminal: Sends information entered by the user to the server.

[1319] Step 5:

[1320] Server: Analyzes the received user information and creates a user profile based on the user's reading habits and interests.

[1321] Step 6:

[1322] Server: Based on the user profile, a recommendation algorithm is used to select the most suitable books and generate a recommendation list.

[1323] Step 7:

[1324] Terminal: Displays the recommendation results received from the server to the user, who can then select books of interest from this list.

[1325] Step 8:

[1326] User: Select a book from the recommendation list via the device and purchase or preview it.

[1327] Step 9:

[1328] Server: If the book selected by the user is in a foreign language, it uses an automatic translation function to generate a Japanese version and sends it to the terminal.

[1329] Step 10:

[1330] Server: Using AI technology, illustrations and videos related to the selected books are automatically generated and provided as additional content.

[1331] Step 11:

[1332] Server: Furthermore, it analyzes the story of the novel and generates a short video, which is also provided to the user.

[1333] Step 12:

[1334] User: Purchases books that interest them. After purchase, the books are available as e-books.

[1335] Step 13:

[1336] Server: Records purchase information and distributes revenue to relevant rights holders and distributors. This process is automated.

[1337] Example 1

[1338] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] Current book recommendation systems lack the ability to provide personalized recommendations that reflect users' preferences and reading history, and they face challenges in providing multilingual and visual content. Furthermore, the process of digitizing and analyzing books is manual and inefficient. Furthermore, the post-purchase revenue sharing mechanism is unclear, making it difficult to fairly distribute rewards to rights holders.

[1340] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1341] In this invention, the server includes a means for electronically scanning physical books and storing them as digital data, a means for analyzing the collected book data and extracting categories, summaries, and major topics, and a means for recommending books based on user profiles. This enables highly accurate book recommendations that reflect users' preferences and reading history, and also enables multilingual and visual content provision. A post-purchase revenue sharing mechanism is also established, enabling fair reward distribution.

[1342] "Physical Book" means a book in a non-electronic format made of paper or other material.

[1343] "Electronically scanned" refers to the process of converting a physical book into digital data using a device such as a scanner.

[1344] "Digital data" refers to information stored in an electronically processable form.

[1345] "Analysis" refers to the process of automatically extracting meaningful information from digital data.

[1346] A "category" refers to a group of books classified based on their content or theme.

[1347] A "summary" is a brief summary of the book's contents.

[1348] "Major topics" refers to important themes or topics contained in the book's content.

[1349] "Input means" refers to the method or interface by which a user provides information to a system.

[1350] A "profile" refers to a collection of data that reflects a user's preferences and behavior.

[1351] "Recommendation method" refers to the process of selecting and providing books based on a user's profile.

[1352] The "means for displaying" refers to an interface that visually presents the recommended book information to the user.

[1353] "Machine translation" refers to technology that automatically converts text between multiple languages.

[1354] "Means for automatically generating illustrations and videos" refers to the process of using artificial intelligence technology to automatically create visual content related to the content of a book.

[1355] "Video content creation" refers to the process of visualizing the story of a book and providing it in video format.

[1356] "Revenue share" refers to a mechanism for fairly distributing revenue.

[1357] "Means for dynamically updating reading habits" refers to technology that automatically updates a user's profile based on their new reading history and preferences.

[1358] MODE FOR CARRYING OUT THE INVENTION

[1359] This invention is a system that recommends the most suitable books to users in a personalized manner. Specifically, it analyzes electronically digitized book data and provides recommendations that match the user's preferences. This system is mainly composed of three entities: a server, a terminal, and a user.

[1360] The server is responsible for collecting, analyzing, and digitizing data, and generating recommendations. The terminal receives input from the user and displays the recommendations. The user provides their preferred genres and past reading history through the terminal, and selects and purchases the books displayed as recommended results.

[1361] The server first uses a dedicated scanner (e.g., Fujitsu ScanSnap iX1500) to electronically scan the physical book and store it as digital data. This digital data is then converted into text data using OCR technology (e.g., Tesseract OCR). The converted text data is then saved in the server's storage.

[1362] The server then uses natural language processing tools (e.g., Apache OpenNLP) to analyze the text data from the digitized books, including tokenization, part-of-speech tagging, and named entity extraction, and stores categories, summaries, and major topics in a database (e.g., MySQL).

[1363] Users log in to the system using a terminal (e.g., a browser-based interface) and enter their preferred genres and past reading history, such as "mysteries" or "a list of books read in the past three years." The terminal checks the format of this input data in real time and sends it to the server as well-formed data.

[1364] The server then uses Node.js and Python scripts to analyze the received user data and generate detailed user profiles, which include information that classifies and predicts the user's reading habits using machine learning algorithms (e.g., k-means clustering), and store this information in a database.

[1365] The server then uses an AI model (e.g., TensorFlow) to select the most suitable books from the database based on the generated user profile. An example prompt might be, "Recommend the most suitable mystery novel based on the user's past reading history and preferred genres. Include summaries, main topics, and categories in the recommendation. Examples include 'famous detective series' and 'famous mystery novel series.'"

[1366] The recommendation results sent from the server are then displayed on the device using a web application using React.js, allowing users to check detailed information about the recommended books (summary, category, main topics).

[1367] If the recommended book is in a foreign language, the server will automatically translate it using the Google Translate API. It can also use OpenAI's DALL-E API to automatically generate illustrations and video content, allowing users to enjoy the book's contents in a variety of ways.

[1368] Furthermore, when a purchase is made, the server uses the Stripe API to record the purchase information and distribute the appropriate revenue share to the rights holder. In this way, the system provides users with personalized book recommendations, enhancing their reading experience.

[1369] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1370] Step 1:

[1371] Electronic scanning and digitization of books

[1372] Server: Electronically scans physical books using a specialized scanner. It accepts the physical book as input and produces scanned image data as output, thereby capturing the contents of the book in image format.

[1373] Server: Converts scanned image data into text data using OCR technology. The input is image data and the output is text data. Specifically, OCR technology performs character recognition and extracts text information from the image.

[1374] Step 2:

[1375] Data analysis of books and extraction of categories, summaries, and main topics

[1376] Server: Analyzes text data using natural language processing tools. The input is text data, and the output is the analysis results: categories, summaries, and main topics. This converts the text data into meaningful information.

[1377] Server: Stores the analyzed data in a database. Specifically, it extracts categories, summaries, and main topics from the text data and inserts them into a MySQL database.

[1378] Step 3:

[1379] Collecting user preference information

[1380] User: Logs into the system through a terminal and inputs his / her preferred genre and past reading history. The input provides his / her preferred genre (e.g., mystery, science fiction) and past reading history. The output is the formatted user input data.

[1381] Terminal: Checks the format of the user's input data in real time and sends it to the server as well-formed data. Specific operations include checking the input of required fields and validating the data format.

[1382] Step 4:

[1383] User profile analysis

[1384] Server: Analyzes the received user data and generates a detailed user profile. The input is the user's preferences, and the output is a detailed user profile. This digitizes the user's reading habits and hobbies. Specifically, it runs a machine learning algorithm using Node.js and Python scripts.

[1385] Step 5:

[1386] Book recommendation generation

[1387] Server: Based on the generated user profile, an AI model is used to select the most suitable books. The input is a detailed user profile, and the output is a list of recommended books. This allows the most suitable books to be selected for the user. Specifically, TensorFlow is used to process the profile data and extract the books that best fit the user profile.

[1388] Step 6:

[1389] Displaying recommendations

[1390] Terminal: Displays the recommendation results received from the server to the user. The input is a list of recommended books, and the output is a visual display of the book information. This allows the user to check the list of recommended books. Specifically, the display is done using a web application using React.js.

[1391] Step 7:

[1392] Automatic translation and additional content provision

[1393] Server: Uses the Google Translate API to automatically translate recommended books. The input is the text data of the recommended book, and the output is the translated text data. This allows users to enjoy the book in their native language. Specifically, the API is called to obtain the translation results.

[1394] Server: Generates illustrations and video content using OpenAI's DALL-E API. The input is the book's content data, and the output is the generated visual content. This allows for a deeper understanding of the book. Specifically, the server calls the API to generate related illustrations and videos.

[1395] Step 8:

[1396] Turning novel stories into video content

[1397] Server: Analyzes the story of a book selected by the user and generates it as a short video. The input is the book's story data, and the output is a short video. This allows users to enjoy the story visually. Specifically, the video is created using Adobe Premiere Pro and an AI video generation tool.

[1398] Step 9:

[1399] Buying books and running revenue shares

[1400] User: Purchases the recommended book using a device. The input is a purchase intention and payment information, and the output is a purchase confirmation and a download link for the book. The book then becomes the user's property. Specific actions include paying with a credit card or electronic money.

[1401] Server: Records purchase information and performs revenue sharing using the Stripe API. The input is purchase information, and the output is confirmation of revenue distribution to rights holders. This ensures fair revenue distribution. Specific operations include storing purchase data and distributing revenue at the appropriate rate.

[1402] (Application example 1)

[1403] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] Conventional book recommendation systems are generally intended for online use, and when users visit a bookstore, there are limited ways to obtain book information on the spot. This makes it difficult for users to find books that suit their preferences, and the purchasing experience is not sufficiently improved. In addition, differences in reading devices and the lack of dynamic updates to user profiles make it difficult to provide appropriate recommendations in real time.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1406] In this invention, the server includes a means for reading a code installed in the store, a means for displaying in-store book information in real time based on the read code, and a means for recommending books in real time based on a user profile using a generative AI model, thereby enabling a user to receive optimal book recommendations in real time when searching for books in the store.

[1407] "Means for electronically scanning books and storing them as digital data" refers to a function for converting books into digital data using equipment such as an electronic scanner or camera, and storing the data on a server or database.

[1408] "Means of analyzing collected book data and extracting categories, summaries, and key topics" refers to a function that analyzes the contents of a book using natural language processing technology, etc., and automatically extracts the genre to which the book belongs, a summary of the contents, and important topics.

[1409] "Means for users to input their favorite genres and past reading history" refers to an interface or mechanism that allows users to input information about their interests and the books they have read.

[1410] "Means for analyzing information entered by the user and creating a profile" refers to a function that analyzes information on genres and reading history provided by the user and generates a profile that reflects the user's reading tendencies and preferences.

[1411] A "means for recommending books based on a user profile" is an algorithm or system for recommending the most suitable book to a user based on the generated user profile.

[1412] The "means for displaying recommended books" refers to a display or a screen of a mobile device that visually presents information about recommended books to the user.

[1413] "Means for providing books in multiple languages ​​through machine translation" is a function that translates the contents of recommended books into the user's desired language through machine translation, making them available in multiple languages.

[1414] "Means for automatically generating illustrations and videos" refers to a function that uses artificial intelligence and image generation technology to automatically generate related illustrations and videos based on the content of a book.

[1415] "A means of turning a novel story into video content" is a function that analyzes the content of a novel and generates video content such as a short film or animation based on that story.

[1416] "Means for reading codes installed in stores" refers to a function that allows you to read codes installed in stores, such as QR codes and barcodes, using a smartphone or dedicated device.

[1417] "Means for displaying in-store book information in real time based on the scanned code" is a function for displaying detailed information about books in the store in real time on the user's device based on the scanned QR code or barcode.

[1418] "Means of recommending books in real time based on user profile using a generative AI model" is a function that uses an artificial intelligence model to recommend the most suitable book on the spot based on the user's profile information.

[1419] The present invention is a system for recommending the most suitable book to a user in a personalized manner, and is mainly composed of a server, a terminal, and a user. Specific embodiments for implementing this system will be described below.

[1420] 1. Server operation

[1421] The server has the following roles:

[1422] 1. Electronic scanning and digital storage of books:

[1423] The server scans the books using an electronic scanner or camera and stores the content as digital data using cloud storage services such as AWS S3 and Firebase Storage.

[1424] 2. Data Analysis:

[1425] Analyze the content of digitized books and extract categories, summaries, and key topics using natural language processing techniques, leveraging Python libraries (e.g., spaCy and NLTK) and AI models (e.g., BERT).

[1426] 3. Create a user profile:

[1427] The server analyzes the data provided by the user (preferred genres, past reading history) and creates a detailed user profile, which is stored in a cloud database (e.g., Firebase Firestore).

[1428] 4. Recommendation Generation:

[1429] Generate book recommendations based on user profiles using generative AI models (e.g., GPT-3) and integrating multiple recommendation algorithms.

[1430] 5. Automatic translation and additional content generation:

[1431] The content of the book is automatically translated as needed and provided in multiple languages. In addition, illustrations and related videos are automatically generated using a generative AI model and provided to users.

[1432] 6. Turning novel stories into video content:

[1433] The story of the book selected by the user is generated as video content. This step uses generative AI models and machine learning techniques.

[1434] 7. Revenue Share Implementation:

[1435] When a book is purchased based on a recommendation, the purchase information is recorded and the sales revenue is distributed to the rights holder.

[1436] 2. Device operation

[1437] The device has the following features:

[1438] 1. Data input from the user:

[1439] The device provides an interface for users to input their preferred genres and past reading history.

[1440] 2. Displaying recommendation results:

[1441] The recommendation results received from the server are displayed to the user using a mobile device such as a smartphone or tablet.

[1442] 3. QR Code Scanning:

[1443] Read QR codes and barcodes installed in stores to obtain book information. To do this, use a QR code reader library (e.g., Zxing).

[1444] 4. Information display based on the code read:

[1445] Based on the code read, the terminal displays detailed information about the book to the user in real time.

[1446] 3. User operations

[1447] The user performs the following operations:

[1448] 1. Enter the data:

[1449] Users input their preferences and past reading history through the device.

[1450] 2. Check the book recommendations:

[1451] Check the recommendations and choose the books that interest you.

[1452] 3. Scan the QR code:

[1453] Scan the QR code of a book you are interested in in the store to view detailed information.

[1454] 4. Buy a book:

[1455] Purchase recommended books or books that interest you.

[1456] Specific examples

[1457] For example, suppose a user visits a physical store and is looking for a book in the mystery genre. The user logs in using a smartphone app and enters the mystery and their past reading history. The server creates a user profile based on that information, selects the most suitable mystery book from the database, and generates a recommendation result. When the user scans the QR code in the store, information about the book is displayed in real time, along with additional content (e.g., illustrations and related videos).

[1458] Examples of prompts for AI models

[1459] "Generate a summary of a mystery novel in which a detective solves a complex case. Include the main plot and key details."

[1460] As a result, this system can recommend optimal books to users in a consistent manner, enriching the purchasing experience in physical stores.

[1461] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1462] Step 1:

[1463] The server electronically scans the book and stores it as digital data. The book is scanned using a scanner or camera to generate image data. This is then converted into text data using OCR software and stored in cloud storage. The input to this process is the physical book, and the output is digital text data.

[1464] Step 2:

[1465] The server analyzes the collected digital data of books and extracts categories, summaries, and main topics. It uses Python libraries and natural language processing techniques (e.g., spaCy and NLTK) to analyze the text data and extract the necessary information. The input of this process is the digital text data generated in step 1, and the output is categories, summaries, and main topics.

[1466] Step 3:

[1467] Users input their preferred genres and past reading history using a device. The input interface is a smartphone or tablet, and the user's input data is stored in a cloud database. The input for this process is the genre and reading history data entered by the user, and the output is the user profile data stored in the cloud database.

[1468] Step 4:

[1469] The server analyzes the user's input and creates a detailed user profile. Based on the user data stored in the cloud, an analytical algorithm is used to analyze the user's reading habits. The input to this process is the user data stored in the cloud database, and the output is a detailed user profile.

[1470] Step 5:

[1471] The server uses a generative AI model (e.g., GPT-3) to recommend the most suitable books in real time based on the user profile. The server matches the user profile with book information stored in a database and generates recommendations using the AI ​​model. The input to this process is a detailed user profile and a book database, and the output is a recommendation result.

[1472] Step 6:

[1473] The terminal displays the recommendation results to the user. Recommended book information is displayed on the screen of a smartphone or tablet. The input to this process is the recommendation results received from the server, and the output is recommended book information visually presented to the user.

[1474] Step 7:

[1475] A user reads a QR code attached to a book in a store. The user then scans the QR code using the camera on their smartphone to obtain the book information. The input to this process is the QR code attached to the book, and the output is the book information read from the QR code.

[1476] Step 8:

[1477] The terminal displays in-store book information in real time based on the read QR code information. Detailed book information is displayed on the terminal based on the information obtained from the QR code. The input for this process is the data read from the QR code, and the output is book information displayed in real time.

[1478] Step 9:

[1479] The server generates automatic translations and additional content (illustrations and videos) as needed. A generative AI model is used for translation, and image generation AI such as DALL-E is used to generate illustrations and videos. The input for this process is recommendation results and book content data, and the output is the translated text and generated illustrations and video content.

[1480] Step 10:

[1481] When a user purchases a book, the server executes the revenue share. It records the purchase information and distributes the sales appropriately to the rights holder. The input of this process is the user's purchase information, and the output is the result of the revenue share execution.

[1482] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1483] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[1484] System Overview

[1485] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1486] Program processing flow (natural language explanation)

[1487] Step 1: Electronic scanning and digitization of the book

[1488] Server: The book is scanned electronically and stored as digital data in the server's database. In this step, text information is extracted using OCR technology.

[1489] Step 2: Analyze the data and extract categories, summaries, and key topics from the books

[1490] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Natural language processing techniques are used for this analysis.

[1491] Step 3: Collecting user preferences

[1492] User: Logs into the system through a terminal and enters his / her preferred genres and past reading history.

[1493] Step 4: Submit user information

[1494] Terminal: Sends information entered by the user to the server.

[1495] Step 5: Create a user profile

[1496] Server: Analyzes the received user information and creates a detailed user profile.

[1497] Step 6: Emotion Recognition with the Emotion Engine

[1498] Server: Analyzes real-time emotion data collected from the user's device using an emotion engine to recognize the user's emotional state.

[1499] Step 7: Emotion-based recommendation optimization

[1500] Server: Optimizes recommendations based on user profile and emotional state, choosing books that fit the user's current emotions.

[1501] Step 8: Generate and display the recommendation list

[1502] Server: Generates an optimized recommendation list.

[1503] Terminal: Displays the recommendation results received from the server to the user.

[1504] Step 9: Select and purchase books

[1505] User: Select a book from the recommended list and purchase or preview it.

[1506] Step 10: Automated translation and additional content provision

[1507] Server: If the selected book is in a foreign language, an automatic translation function is used to generate a Japanese version and send it to the device. In addition, AI technology is used to automatically generate illustrations and videos, which are provided as additional content.

[1508] Step 11: Turn your story into video content

[1509] Server: Generates the story of the novel as a short video and provides it to the user.

[1510] Step 12: Record purchase information and execute revenue share

[1511] Server: Records purchase information and executes revenue share with relevant rights holders and distributors.

[1512] Specific examples

[1513] For example, suppose a user is looking for a romance novel. The user logs into the system through their device and enters their desired romance and past reading history. Furthermore, the device's camera and sensors are used to transmit the user's emotional data in real time to the server. The server then creates a user profile based on this information and uses an emotion engine to analyze the user's current emotional state.

[1514] Because the user is feeling stressed, the emotion engine prioritizes relaxing romance novels and adds them to the recommendation list. From the recommendation list displayed on the device, the user can select a book that interests them and purchase it. If the selected book is in a foreign language, the server uses an automatic translation function to provide a Japanese version. Users can also view illustrations and short videos generated by AI.

[1515] In this way, the system of the present invention, combined with the emotion engine, provides advanced recommendations that take into account the user's emotional state, enabling a rich reading experience.

[1516] The processing flow will be explained below.

[1517] Step 1:

[1518] Server: The book is scanned electronically and stored as digital data in a database on the server. In this step, OCR technology is used to extract text information for each page and store it as metadata.

[1519] Step 2:

[1520] Server: Analyzes the stored digital data and extracts categories, summaries, and main topics for each book. Using natural language processing technology, it extracts themes and keywords from the content and organizes the database based on these.

[1521] Step 3:

[1522] User: Logs in to the system using a terminal and enters information such as preferred genres and past reading history, which allows the collection of basic information for a user profile.

[1523] Step 4:

[1524] Terminal: Sends information entered by the user to the server, which uses it to create and update the user profile.

[1525] Step 5:

[1526] Server: Analyzes the received user information and creates a detailed user profile, including data about the user's reading habits and interests.

[1527] Step 6:

[1528] On-device: Using the device's camera and sensors, we collect real-time emotional data from the user. This data is obtained through facial recognition and biometric sensors.

[1529] Step 7:

[1530] Terminal: Sends collected emotional data to the server, which uses it to analyze the user's emotional state.

[1531] Step 8:

[1532] Server: The emotion engine analyzes the transmitted emotion data and recognizes the user's current emotional state. For example, it identifies whether the user is happy, sad, or stressed.

[1533] Step 9:

[1534] Server: Based on the user profile and emotional state, the recommendation algorithm selects the most suitable book and optimizes the recommendation. The selection criteria are dynamically adjusted taking into account the emotion recognition results.

[1535] Step 10:

[1536] Server: Generates an optimized recommendation list and sends it to the device. The recommendation list includes books that are appropriate for the user's emotional state.

[1537] Step 11:

[1538] Terminal: The terminal displays the recommendation results received from the server to the user. The user can select a book of interest from the recommendation list and view detailed information and samples.

[1539] Step 12:

[1540] User: Can select a book from the recommended list and make a purchase. Once purchased, the book will be available as an e-book.

[1541] Step 13:

[1542] Server: If the selected book is in a foreign language, it will use the automatic translation function to provide the Japanese version. This translation process runs in the background.

[1543] Step 14:

[1544] Server: Using AI technology, it automatically generates illustrations and videos related to the selected book and sends them to the device as additional content, enriching the reader's visual experience.

[1545] Step 15:

[1546] Server: The server then generates a short video of the story and provides it to the user. In this step, the main parts of the story are visualized to provide visual enjoyment.

[1547] Step 16:

[1548] Server: Records purchase information and executes revenue sharing with relevant rights holders and distributors. This process is performed automatically based on sales information.

[1549] Example 2

[1550] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1551] In today's information overload, it is difficult for users to find the book that best suits them. Advanced filtering technology is required to select books that best suit the user's preferences and emotional state from the vast number of books available. Furthermore, a system that can smoothly provide foreign language books and additional content is also needed.

[1552] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1553] In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres and past reading history, means for analyzing the information input by the user and creating a profile, means for recommending books based on the user profile, means for displaying recommended books, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, means for converting novel stories into video content, means for collecting real-time emotional data, means for analyzing the emotional data and recognizing the user's emotional state, and means for recommending books based on the user's emotional state. This allows the server to provide optimal books based on the user's preferences and emotional state, enabling a rich reading experience.

[1554] "Means of electronically scanning books and storing them as digital data" refers to a method of electronically reading paper books using a scanner and storing the data in a database as text data.

[1555] "Methods for analyzing collected book data and extracting categories, summaries, and main topics" refers to a method of analyzing the contents of digitized books using OCR technology and natural language processing technology, and extracting the categories to which the books belong, summaries, and main points of discussion.

[1556] "Means for users to input their favorite genres and past reading history" refers to an interface that allows users to provide the system with their favorite genres and a list of books they have read so far.

[1557] "Means for analyzing information entered by the user and creating a profile" refers to a method for analyzing information provided by the user, such as genre and reading history, using machine learning models to generate a detailed user profile.

[1558] The "means for recommending books based on a user profile" is a method for providing a list of books that are most suitable for each individual user based on the generated user profile.

[1559] The "means for displaying recommended books" is an interface for displaying the recommendation list sent from the server on the user's terminal.

[1560] "Means for providing books in multiple languages ​​through automatic translation" refers to a method in which, if the selected book is in a foreign language, the book is translated into another language using automatic translation technology and provided to the user.

[1561] "Means for automatically generating illustrations and videos" refers to a method of using AI technology to generate illustrations and short videos based on the content of a book and provide them to users as additional content.

[1562] "Means for turning a novel story into video content" refers to a method of using a video generation algorithm to generate and provide a short video using the contents of a novel as input data.

[1563] "Means for collecting real-time emotional data" refers to an interface that uses cameras and sensors to obtain emotional data in real time from the user's facial expressions, voice, etc.

[1564] The "means for analyzing emotion data and recognizing the user's emotional state" is an emotion recognition algorithm for analyzing the collected emotion data and recognizing the user's current emotional state.

[1565] The "means for recommending books based on the user's emotional state" is a method for combining a user profile and an emotional state to recommend books that suit the user's current emotions.

[1566] The present invention is an advanced system that recommends optimal books to a user, recognizes the user's emotional state, and optimizes the recommendations. Hereinafter, a detailed description will be given of an embodiment of the present invention.

[1567] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1568] First, the server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap), extracts the text information using OCR technology (e.g., Tesseract), and stores it in a database. This step generates digital data.

[1569] The server then analyzes the collected digital data using algorithms based on natural language processing technology (e.g., SpaCy or NLTK) to extract book categories, summaries, and main topics.

[1570] Users log in to the system through their devices and enter their favorite genres and past reading history. iOS or Android applications are used on the devices. The information entered by the user is sent to the server using the HTTPS protocol. The server analyzes the information received and creates a detailed user profile using machine learning models (e.g., Scikit-learn or TensorFlow).

[1571] Furthermore, the user's real-time emotional data is collected by the emotion engine. The emotional data acquired using the device's camera and sensors is analyzed by an emotion recognition API (e.g., AWS Rekognition or Microsoft Azure Emotion API). The server recognizes the user's emotional state based on the analysis results.

[1572] The server then optimizes the recommendations based on the user's profile and current emotional state. A reinforcement learning algorithm is used to select books that fit the emotional state. An optimized recommendation list is then generated and sent to the device, which displays the list in the user interface.

[1573] Users can select books of interest from the recommendation list and purchase or preview them. The purchase process is carried out through an e-commerce system (e.g., Amazon or Rakuten). If the selected book is in a foreign language, an automatic translation function (e.g., Google Translate API) is used to generate a Japanese version. Furthermore, illustrations and videos are automatically generated using AI technology (e.g., DALL-E or GAN) and provided as additional content.

[1574] As a concrete example, let's say a user is looking for a romance novel. The user logs into the system through their device and enters their romance and past reading history. Emotional data is sent to the server in real time via the device's camera and sensors. The server creates a user profile based on this information, and an emotion engine analyzes the user's current emotional state. If the emotion engine determines that the user is feeling "stressed," it will prioritize recommendations of relaxing romance novels.

[1575] In this way, advanced recommendations based on the user's emotional state and preferences are realized.

[1576] Example prompt sentence:

[1577] "Recommend positive, relaxing romance novels based on the user's reading history and real-time emotional data. Users are stressed and want content that will help them feel better."

[1578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1579] Step 1:

[1580] The server electronically scans the book using a scanner (e.g., Fujitsu ScanSnap). The input is the paper book, and the output is the scanned image data. This image data is converted into text information using OCR technology (e.g., Tesseract) and stored in a database. The specific operation is "start the scanner, place the book, and press the scan button."

[1581] Step 2:

[1582] The server analyzes the collected text data. The input is the text data stored in step 1, and the output is categories, summaries, and main topics. Natural language processing techniques (e.g., SpaCy or NLTK) are used to analyze the text data and extract categories, summaries, and main topics. The specific operation is to "input the text data into the algorithm and extract the analysis results."

[1583] Step 3:

[1584] A user logs into the system through a terminal and enters their favorite genres and past reading history. The input is the user's preferences and past reading history, and the output is the user's input data. The specific operation is "entering information into a form displayed on the terminal screen and clicking the submit button."

[1585] Step 4:

[1586] The terminal sends information entered by the user to the server using the HTTPS protocol. The input is the data entered by the user, and the output is the data sent to the server. The specific operation is "clicking the send button to send the entered information to the server."

[1587] Step 5:

[1588] The server analyzes the received user information and creates a detailed user profile using a machine learning model (e.g., Scikit-learn or TensorFlow). The input is user information and the output is a user profile. The specific operation is to "input the received user information into the machine learning model and generate a profile."

[1589] Step 6:

[1590] The server collects real-time emotional data. The input is the user's facial expression and voice data obtained from the device's camera and sensors, and the output is emotional data. Specifically, the system works by "collecting the user's facial expression data through the camera and acquiring voice data from the sensors."

[1591] Step 7:

[1592] The server analyzes the emotion data and recognizes the user's emotional state. The input is the collected emotion data, and the output is the recognition result of the emotional state. The emotion data is analyzed using an emotion recognition API (for example, AWS Rekognition or Microsoft Azure Emotion API). The specific operation is to "input the emotion data into the emotion recognition API and obtain the emotional state."

[1593] Step 8:

[1594] The server optimizes recommendations using a reinforcement learning algorithm based on the user profile and emotional state. The input is the user profile and the recognition result of the emotional state, and the output is an optimized recommendation list. The specific operation is "input the user profile and emotional state into the algorithm and obtain the recommendation result."

[1595] Step 9:

[1596] The server generates an optimized recommendation list and sends it to the terminal. The input is the optimized recommendation results, and the output is the recommendation list sent to the terminal. The terminal displays the received recommendation results on the user interface. The specific operation is "generate a recommendation list and send it to the terminal."

[1597] Step 10:

[1598] The user selects a book from the recommendation list and purchases or previews it. The input is the recommendation list, and the output is the selected book. The specific operation is "select a book from the recommendation list and click the purchase button."

[1599] Step 11:

[1600] If the selected book is in a foreign language, the server uses an automatic translation function to generate a Japanese version and sends it to the terminal. The input is the foreign language book data, and the output is the translated book data. The specific operation is to "input the foreign language text into the translation API and obtain the Japanese text."

[1601] Step 12:

[1602] The server automatically generates illustrations and videos using AI technology (for example, DALL-E or GAN) and provides them as additional content. The input is the book content data, and the output is the generated illustrations and videos. Specifically, the operation is to "use the book content as input data and generate illustrations and videos using AI technology."

[1603] Step 13:

[1604] The server generates a short video from the story of the novel and sends it to the terminal. The input is the story data of the novel, and the output is the generated short video. The specific operation is to "input the story data of the novel into the video generation algorithm and generate a video file."

[1605] Step 14:

[1606] The server records purchase information and executes revenue sharing with the relevant rights holders and distributors. The input is purchase data, and the output is revenue share distribution information. This process is carried out using blockchain technology (e.g., Ethereum Smart Contracts). The specific operation is to "send purchase information to the blockchain network and execute the smart contract."

[1607] (Application example 2)

[1608] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1609] Conventional book recommendation systems only recommend books based on a user's reading history and genre preferences, and are unable to provide personalized recommendations that take into account the user's real-time emotional state. This makes it difficult to provide timely recommendations that match each user's emotions and mood, resulting in limited improvement in user satisfaction. Furthermore, they are unable to provide sufficient multilingual book services or interactive experiences in virtual stores.

[1610] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for electronically scanning books and storing them as digital data, means for analyzing the collected book data and extracting categories, summaries, and main topics, means for the user to input their preferred genres, past reading history, and emotional state, means for analyzing the information and emotional data input by the user to create a profile and perform real-time emotion recognition, means for recommending books based on the user profile and emotional state, means for displaying recommended books and providing virtual store navigation, means for providing books in multiple languages ​​through automatic translation, means for automatically generating illustrations and videos, and means for converting novel stories into video content. This enables timely recommendations that take the user's emotional state into consideration and an interactive virtual store experience.

[1611] "Electronically scanning a book" is the process of optically scanning a paper book and storing it in a digital format.

[1612] "Storing as digital data" means storing electronically read information in a database or storage system.

[1613] "Analyzing collected book data" means analyzing electronically acquired book data using algorithms and software to extract information.

[1614] "Extract categories, summaries, and major topics" means categorizing the contents of a book and extracting key points and summaries.

[1615] "User inputs preferred genres, past reading history, and emotional state" means that the user provides the system with their own reading preferences, books they have read in the past, and their current emotional state via their terminal.

[1616] "Analyzing information and emotional data entered by the user to create a profile and perform real-time emotion recognition" means generating a detailed profile of the user based on data provided by the user and emotional information collected in real time.

[1617] "Recommending books based on user profile and emotional state" means recommending appropriate books based on the created user profile and real-time emotional information.

[1618] "Displaying recommended books and providing virtual store navigation" means displaying recommended books on the user's device and providing the experience of exploring a bookstore in a virtual space.

[1619] "Providing books in multiple languages ​​through automatic translation" is a function that automatically translates selected books into different languages ​​and provides them to users.

[1620] "Automatically generating illustrations and videos" means using AI technology to automatically create illustrations and short videos based on the content of a book.

[1621] "Turning a novel story into video content" means creating a short video based on the content of the novel and providing it to users as visual content.

[1622] This invention combines a system that uses AI technology to recommend the most suitable books to users in a personalized manner with an emotion engine that recognizes the user's emotions. This enables advanced recommendations that take into account the user's emotional state, enabling a more diverse reading experience and content offering.

[1623] System Overview

[1624] This system is mainly composed of three entities: a server, a terminal, and a user. The server is responsible for data collection, analysis, digitization, recommendation generation, emotion recognition, and optimization. The terminal receives input from the user, collects emotion data, and displays recommendations. The user provides their preferred genres, reading history, and real-time emotional state through the terminal.

[1625] Program Description

[1626] Hardware and software used

[1627] 1. Hardware

[1628] Smartphones: Equipped with cameras and processors, they collect user input and emotional data and support the overall operation of apps.

[1629] Server: Manages user data and book data, and performs large-scale data analysis and emotion recognition.

[1630] 2. Software

[1631] OpenCV: Used as an image processing library to perform preprocessing for face detection and emotion recognition.

[1632] TensorFlow / Keras: Build an emotion recognition model and analyze users' real-time emotions.

[1633] Requests: Send an HTTP request to the recommendation API to get the recommendation results.

[1634] Specific examples

[1635] For example, suppose a user is looking for a romance novel. The user logs into the smartphone application "Virtual Bookstore Emo" and inputs their past reading history and preferred genres. Furthermore, the smartphone's camera is used to read their facial expressions, and the data is sent to the server in real time.

[1636] The server creates a user profile based on this data and uses an emotion engine to analyze the user's current emotional state. If the emotion engine determines that the user is feeling stressed, it will prioritize relaxing romance novels and add them to the recommendation list. This allows for more personalized book recommendations for the user.

[1637] Recommended books are displayed through a virtual bookstore interface on the application. Users can virtually navigate the bookstore to select a book, and if the book they choose is in a foreign language, an automatic translation function will provide a Japanese version. Users can also view AI-generated illustrations and short videos, further enriching their reading experience.

[1638] Prompt Sentence Examples

[1639] "Recommend relaxing fantasy novels based on the user's real-time emotions and past reading history."

[1640] In this way, the system of the present invention can provide advanced recommendations that take into account the user's emotional state, providing a rich reading experience.

[1641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1642] Step 1:

[1643] The user logs in to the application through the device and inputs their preferred genres and past reading history. At this point, the device collects the user's input data (genre, reading history).

[1644] Input: Favorite genres, past reading history

[1645] Output: User input data

[1646] Step 2:

[1647] The device uses the smartphone camera to capture the user's facial expressions and collect emotional data in real time.The device then uses an emotion recognition model (TensorFlow / Keras) to analyze emotions from the collected facial data.

[1648] Input: User's face image

[1649] Output: User emotion data

[1650] Step 3:

[1651] The device sends the user's input data (genre, reading history) and emotion data to the server, which stores the data for further processing.

[1652] Input: User input data, emotion data

[1653] Output: User data sent to the server

[1654] Step 4:

[1655] The server creates a user profile based on the received user data, which includes analyzing input data and emotional data. The profile is then used in the next recommendation stage.

[1656] Input: User data (input data, emotion data)

[1657] Output: User profile

[1658] Step 5:

[1659] The server optimizes book recommendations based on the user's profile and emotional state, using AI technology to select the most suitable books for each user.

[1660] Input: User profile, emotional state

[1661] Output: Recommended book list

[1662] Step 6:

[1663] The server generates a recommended book list and sends it to the terminal, allowing the user to receive a list of the most suitable books in real time.

[1664] Input: Recommended book list

[1665] Output: Recommended book list sent to the device

[1666] Step 7:

[1667] The terminal displays the received recommended book list on the interface of the virtual bookstore, and the user can select books while navigating the virtual bookstore.

[1668] Input: Recommended book list

[1669] Output: Display content in the virtual store

[1670] Step 8:

[1671] If a user selects a recommended book, the server uses machine translation to provide the book in multiple languages, and automatically generates illustrations and short videos if necessary.

[1672] Input: Selected book, trigger data for translation function

[1673] Output: translated book, generated content

[1674] Step 9:

[1675] Users can browse translated books and generated content on their devices and enjoy a reading experience, thereby providing users with a diverse reading and digital content experience.

[1676] Input: translated book, generated content

[1677] Output: User's reading and digital content experience

[1678] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1679] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1680] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1681] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1682] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1683] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1684] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1685] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1686] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1687] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1688] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1689] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1690] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1691] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1692] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1693] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1694] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1695] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1696] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1697] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1698] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1699] The following is further disclosed regarding the above embodiment.

[1700] (Claim 1)

[1701] A means for electronically scanning and storing books as digital data;

[1702] A means of analyzing the collected book data and extracting categories, summaries, and major topics;

[1703] A means for users to input their preferred genres and past reading history;

[1704] means for analyzing information entered by a user and creating a profile;

[1705] A means of recommending books based on user profiles;

[1706] a means for displaying the recommended books;

[1707] A means to provide books in multiple languages ​​through machine translation,

[1708] A means of automatically generating illustrations and videos,

[1709] A system that includes a means for turning a novel story into video content.

[1710] (Claim 2)

[1711] 10. The system of claim 1, further comprising means for providing a revenue share to rights holders for each book recommended and purchased.

[1712] (Claim 3)

[1713] 10. The system of claim 1, further comprising means for analyzing a user's reading habits and dynamically updating the profile.

[1714] "Example 1"

[1715] (Claim 1)

[1716] A means for electronically scanning physical books and storing them as digital data;

[1717] A means of analyzing the collected book data and extracting categories, summaries, and major topics;

[1718] A way for users to input their preferred genres and past reading history,

[1719] A means for analyzing the information entered by the user and creating a profile;

[1720] A means of recommending books based on user profiles;

[1721] a means for displaying recommended books;

[1722] A means of providing books in multiple languages ​​through machine translation,

[1723] A means of automatically generating illustrations and videos,

[1724] A system that includes a means for turning book stories into video content.

[1725] (Claim 2)

[1726] 10. The system of claim 1, further comprising means for performing a revenue share with a rights holder for each book recommendation-based purchase.

[1727] (Claim 3)

[1728] 10. The system of claim 1, further comprising means for analyzing a user's reading habits and dynamically updating the profile.

[1729] "Application Example 1"

[1730] (Claim 1)

[1731] A means for electronically scanning and storing books as digital data;

[1732] A means of analyzing the collected book data and extracting categories, summaries, and major topics;

[1733] A means for users to input their preferred genres and past reading history;

[1734] means for analyzing information entered by a user and creating a profile;

[1735] A means of recommending books based on user profiles;

[1736] a means for displaying the recommended books;

[1737] A means to provide books in multiple languages ​​through machine translation,

[1738] A means of automatically generating illustrations and videos,

[1739] A way to turn a novel's story into video content,

[1740] A means for reading a code installed in the store;

[1741] A means for displaying in-store book information in real time based on the read code;

[1742] A means to recommend books in real time based on user profiles using generative AI models; and

[1743] A system including:

[1744] (Claim 2)

[1745] 10. The system of claim 1, further comprising means for providing a revenue share to rights holders for each book recommended and purchased.

[1746] (Claim 3)

[1747] 10. The system of claim 1, further comprising means for analyzing a user's reading habits and dynamically updating the profile.

[1748] "Example 2: Combining Emotion Engines"

[1749] (Claim 1)

[1750] A means for electronically scanning and storing books as digital data;

[1751] A means of analyzing the collected book data and extracting categories, summaries, and major topics;

[1752] A means for users to input their preferred genres and past reading history;

[1753] means for analyzing information entered by a user and creating a profile;

[1754] A means of recommending books based on user profiles;

[1755] a means for displaying the recommended books;

[1756] A means to provide books in multiple languages ​​through machine translation,

[1757] A means of automatically generating illustrations and videos,

[1758] A way to turn a novel's story into video content,

[1759] a means for collecting real-time sentiment data;

[1760] means for analyzing the emotion data and recognizing the user's emotional state;

[1761] A system including a means for recommending books based on a user's emotional state.

[1762] (Claim 2)

[1763] 10. The system of claim 1, further comprising means for providing a revenue share to rights holders for each book recommended and purchased.

[1764] (Claim 3)

[1765] 10. The system of claim 1, further comprising means for analyzing a user's reading habits and dynamically updating the profile.

[1766] "Application example 2 when combining emotion engines"

[1767] (Claim 1)

[1768] A means for electronically scanning and storing books as digital data;

[1769] A means of analyzing the collected book data and extracting categories, summaries, and major topics;

[1770] A means for the user to input their preferred genres, past reading history, and emotional state;

[1771] A means for analyzing information and emotional data input by a user to create a profile and perform real-time emotion recognition;

[1772] a means for recommending books based on a user profile and emotional state;

[1773] a means for displaying recommended books and providing virtual store navigation;

[1774] A means to provide books in multiple languages ​​through machine translation,

[1775] A means of automatically generating illustrations and videos,

[1776] A system that includes a means for turning a novel story into video content.

[1777] (Claim 2)

[1778] 10. The system of claim 1, further comprising means for providing a revenue share to rights holders for each book recommended and purchased.

[1779] (Claim 3)

[1780] 10. The system of claim 1, further comprising means for analyzing a user's reading habits and emotional state to dynamically update the profile. [Explanation of symbols]

[1781] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for electronically scanning and storing books as digital data; A means of analyzing the collected book data and extracting categories, summaries, and major topics; A means for users to input their preferred genres and past reading history; means for analyzing information entered by a user and creating a profile; A means of recommending books based on user profiles; a means for displaying the recommended books; A means to provide books in multiple languages ​​through machine translation, A means of automatically generating illustrations and videos, A system that includes a means for turning a novel story into video content.

2. 10. The system of claim 1, further comprising means for providing a revenue share to rights holders for each book recommended and purchased.

3. 10. The system of claim 1, further comprising means for analyzing a user's reading habits and dynamically updating the profile.

Images