System

Abstract

A system is provided.SOLUTION: A system includes means for digitizing analog information in Japanese, means for training a generative AI model based on the digitized text data, means for providing a plurality of generatively AI modeled books online, means for providing an interface for a user to input a question to the generative AI model, means for generating an answer to the question of the user by the generative AI model, and means for displaying the generated answer to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, much useful analog information in Japanese (paper books and handwritten documents) has not been digitized, making it difficult to access and effectively utilize this information. Furthermore, the traditional reading experience poses a significant time challenge for users: purchasing a book, reading, comprehending, and applying it to their own problems. Furthermore, there is no way to quickly obtain specific information or get immediate answers to specific questions about the book's contents. This situation hinders the efficient use and rapid understanding of information. This delays users' ability to resolve their problems, especially in situations where practical information is required, such as business books and how-to guides. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides a system that digitizes Japanese analog information and uses it to train a generative AI model. This system includes a means for cleaning and structuring digitized text data. Furthermore, it provides multiple books that have been modeled by generative AI online and includes a means for users to search for and select specific books. Users use an interface to directly input questions to the generative AI model, which then quickly generates an answer. The generated answer is instantly displayed to the user, allowing the user to efficiently obtain the information they need. This system enables users to quickly and deeply understand the contents of books and put them to practical use. It also effectively utilizes valuable Japanese analog information that has not yet been digitized, maximizing the value of the information.

[0006] "Analog information in Japanese" refers to non-digital information such as paper books and handwritten documents written in Japanese.

[0007] "Digitalization" is the process of converting analog information, such as paper books or handwritten documents, into electronic form using scanning and OCR technology.

[0008] "Text data" means digitized textual data that has been extracted from original analog information and structured as a string of characters that can be stored and processed electronically.

[0009] A "generative AI model" is an artificial intelligence model that is trained based on large amounts of text data and is capable of generating natural language and answering questions.

[0010] "Training" is the learning process that generative AI models go through to improve their performance based on input data.

[0011] "Online provision" means providing services or content to users via the Internet.

[0012] A "user interface" refers to the screen or input means that a user uses to access and operate a system, and to enter questions and receive results.

[0013] "Cleaning" is the process of removing misrecognized characters, unnecessary line breaks, and spaces from digitized text data to improve the quality of the data.

[0014] "Structuring" means organizing raw text data logically and systematically and storing it in a format such as a database.

[0015] "Search" is the process of searching a database or storage to find specific information or content.

[0016] A "question input interface" is a screen that provides text boxes or input fields for users to enter questions directly into the generative AI model.

[0017] "Generating an answer" is the process by which a generative AI model creates an appropriate response to a question entered by a user.

[0018] "Display" means presenting the generated answers on the screen in a format that is easy for the user to view. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Specific embodiments of this system are described below.

[0041] Building the dataset

[0042] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[0043] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is structured and stored in a database.

[0044] Creating generative AI models

[0045] Server: Builds a generative AI model that learns the content of specific books based on the cleaned text data. Trains the generative AI model based on the text taken as training data and adjusts the parameters accordingly.

[0046] Server: Deploys the trained generative AI model and makes it available to users online.

[0047] Building an online platform

[0048] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[0049] User Interface and Operation

[0050] Book Selection and Questions

[0051] User device: The user opens a web browser, accesses the provided platform URL, enters keywords in the search bar, and searches for the book they want to read.

[0052] Server: Receives the search query, generates a list of relevant books, and displays them to the user.

[0053] User device: The user selects a particular book from the displayed list to access its detail page.

[0054] Ask a question and get an answer

[0055] User device: On the details page of the selected book, the user enters a question into the text box that asks the generative AI model a question.

[0056] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[0057] Server: The generative AI model analyzes the question and generates an appropriate answer.

[0058] Server: Sends the generated answer to the user's device.

[0059] User device: The user checks the answers displayed on the screen and asks further questions if necessary.

[0060] Specific examples

[0061] Example 1: Using business books

[0062] scenario:

[0063] Let's say a user wants to refer to a specific business book on "strategies for new business development."

[0064] User device: Access the platform and search for "New Business Development".

[0065] Server: Displays a list of relevant business books as search results.

[0066] User Device: Select a specific business book and go to its details page.

[0067] User device: Type the question, "What is the most important step in new business development?"

[0068] Server: Sends questions to the generative AI model and generates answers.

[0069] User device: Receive specific answers such as, "Imaginative market analysis and targeting are the most important."

[0070] Example 2: Using cooking recipes

[0071] scenario:

[0072] Let's say a user wants to consult a cookbook on how to make authentic French toast.

[0073] User device: Accesses the platform and searches for "french toast."

[0074] Server: Displays a list of relevant cookbooks as search results.

[0075] User Device: Select a specific cookbook and go to its detail page.

[0076] User device: Type in the question, "What's the secret to making authentic French toast?"

[0077] Server: Sends questions to the generative AI model and generates answers.

[0078] User device: Receives a response such as "Adding a little vanilla extract will give it an even richer flavor."

[0079] The above is a specific embodiment for carrying out the present invention. This system enables users to effectively obtain information and put it to practical use.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[0083] Step 2:

[0084] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[0085] Step 3:

[0086] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0087] Step 4:

[0088] Server: Starts training a generative AI model (e.g., GPT-3®) based on the cleaned text data. Uses the ingested text as training data to adjust the model's parameters. The training process runs over multiple epochs (iterations).

[0089] Step 5:

[0090] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0091] Step 6:

[0092] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0093] Step 7:

[0094] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0095] Step 8:

[0096] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0097] Step 9:

[0098] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0099] Step 10:

[0100] User device: The user selects a particular book from the displayed list to access its detail page.

[0101] Step 11:

[0102] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0103] Step 12:

[0104] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[0105] Step 13:

[0106] Server: The generative AI model analyzes the question and generates an appropriate answer.

[0107] Step 14:

[0108] Server: Sends the generated answer to the user's device.

[0109] Step 15:

[0110] User device: The user reviews the answers displayed on the screen and asks further questions if necessary.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Previously, digitizing and utilizing analog Japanese information has faced challenges such as low-accuracy OCR technology, inefficient data cleaning methods, and a lack of user-friendly interfaces. Furthermore, building an entire system to train generative AI models and provide appropriate information to users has been complex. There is a need to resolve these issues and provide high-quality digital information while enabling users to access it efficiently.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI models online, means for providing an interface for a user to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for cleaning the digitized text data, means for structuring the cleaned text data, means for a user to search for specific data, and means for displaying search results to the user. This makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and effectively access the information they need.

[0116] "Analog information in Japanese" refers to information written in Japanese that is not stored as electronic data, such as paper books and handwritten documents.

[0117] "Digitizing" means converting analog information into an electronic data format that can be processed by a computer.

[0118] "Text data" refers to data that electronically records information such as letters, sentences, and symbols.

[0119] A "generative AI model" refers to an artificial intelligence model that can learn from text data and perform specific tasks.

[0120] "Training" means using large amounts of data to learn from an AI model and improve its performance.

[0121] "Providing online" means making a service or function available to users via the Internet.

[0122] "Interface" refers to the screens and input means through which users interact with a system.

[0123] "Cleaning" refers to the process of correcting misrecognized parts from text data and removing unnecessary spaces and line breaks.

[0124] "Structuring" means organizing data into a certain format or structure so that it can be easily searched and used.

[0125] "Deploying" means placing a trained AI model in an actual production environment and making it available for use.

[0126] "Database" refers to an electronic system for efficiently storing, managing, and retrieving data.

[0127] "Searching" means finding data based on specific keywords or conditions.

[0128] "Results" refers to the final information or data obtained through operations such as search or calculation.

[0129] An "API request" is a request made by an application to another system to use a function or obtain information.

[0130] "REST API" refers to an API based on uniform design principles for exchanging data between web services.

[0131] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model, and provides it to users. Detailed embodiments for implementing this system are described below.

[0132] Building the dataset

[0133] server:

[0134] To digitize analog information, first, paper books or handwritten documents are scanned to obtain image data. The hardware used is, for example, the common scanner "Epson Perfection V600." This scanned image is then input into the OCR engine "Tesseract OCR," and text data is extracted.

[0135] server:

[0136] Next, the extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. A Python script is used for cleaning. The cleaned text data is stored in a MySQL (registered trademark) database. The stored data is structured so that it can be used for later training and searches.

[0137] Creating generative AI models

[0138] server:

[0139] The generative AI model is trained on the cleaned text data. This process uses machine learning frameworks such as Tensorflow (registered trademark) and PyTorch. Training data is input and specific parameters are adjusted to improve the model's accuracy.

[0140] server:

[0141] Once trained, the generative AI model is deployed in a Docker container, making it operational and enabling it to quickly provide answers to user queries.

[0142] Building an online platform

[0143] server:

[0144] The user interface will be developed using ReactJS and Node.js. This interface allows users to operate it intuitively. The interface will include a book search function, an access interface to the generative AI model, user profile management, and the ability to add book purchase links. The search function will use ElasticSearch (registered trademark) to quickly extract relevant information from large amounts of data.

[0145] User interface and operation examples

[0146] Book Selection and Questions

[0147] User device:

[0148] A user opens a web browser, visits the provided platform URL (e.g., www.examplebooks.com), and enters keywords into the search bar to search for a specific book (e.g., "new business development" or "French toast").

[0149] server:

[0150] It receives a search query, uses Elasticsearch to generate a list of relevant books, and displays them to the user.

[0151] User device:

[0152] The user selects a particular book from the displayed list to access its detail page.

[0153] Ask a question and get an answer

[0154] User device:

[0155] Go to the details page and type your question into the text box (e.g., "What is the most important step in new business development?").

[0156] server:

[0157] The entered question is sent to the generative AI model using the REST API.

[0158] server:

[0159] A generative AI model analyzes the question and generates an appropriate answer, which is composed in natural language based on the model's training data.

[0160] server:

[0161] The generated answer is sent back to the user's device and displayed in real time.

[0162] User device:

[0163] Users review the answers displayed on the screen (e.g., "Imaginative market analysis and targeting are paramount") and ask further questions if necessary.

[0164] This invention makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and efficiently access the information they need.

[0165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0166] Specific explanation of program processing

[0167] Step 1: Digitizing analog information

[0168] server:

[0169] Input: Analog information such as paper books and handwritten documents.

[0170] Specific operation: Analog information is scanned using a scanner. The hardware used is the "Epson Perfection V600."

[0171] Output: Obtain a high-resolution scanned image.

[0172] Step 2: Extract text using OCR

[0173] server:

[0174] Input: Scanned image data.

[0175] Specific operation: The acquired scanned image is input into the OCR engine "Tesseract OCR" and text data is extracted from the image.

[0176] Call the Tesseract API specifying the path to the image data provided by the client or the database.

[0177] Output: The extracted text data.

[0178] Step 3: Cleaning the text data

[0179] server:

[0180] Input: Text data extracted by OCR.

[0181] What it does: Uses a Python script to clean text, including correcting misrecognized characters and removing unnecessary line breaks and spaces.

[0182] Output: Cleaned text data.

[0183] Step 4: Structuring and storing text data

[0184] server:

[0185] Input: Cleaned text data.

[0186] What it does: It uses a Python script to structure text data into the required format, then stores it in a MySQL database, which also contains metadata such as book title, author, and content.

[0187] Output: Structured text data stored in a database.

[0188] Step 5: Training the generative AI model

[0189] server:

[0190] Input: Cleaned text data stored in a database.

[0191] How it works: Use TensorFlow or PyTorch to train a generative AI model. Input training data into the AI ​​model and adjust parameters such as the number of epochs, batch size, and learning rate.

[0192] Output: A fully trained generative AI model.

[0193] Step 6: Deploying the generative AI model

[0194] server:

[0195] Input: A fully trained generative AI model.

[0196] What it does: Deploy an AI model in a Docker container. The containerized model runs on a server and is accessible externally through an API.

[0197] Output: A usable generative AI model is deployed in a production environment.

[0198] Step 7: Developing the User Interface

[0199] server:

[0200] Input: None (early development stage).

[0201] Specific operation: Using "ReactJS" and "Node.js", a user interface will be developed, which will include a book search function, an access interface to the generative AI model, user profile management, and a link to purchase the book.

[0202] Output: A web platform that users can access.

[0203] Step 8: Book Search

[0204] User device:

[0205] Type: Enter a keyword into the search bar (e.g., "new business development" or "french toast").

[0206] Specific actions: Open a web browser, access the provided platform URL, and enter keywords in the search bar.

[0207] server:

[0208] Input: The user's search query.

[0209] Specific operation: Uses Elasticsearch to generate a list of relevant books based on the search query and displays it to the user.

[0210] Output: Shows a list of books relevant to the user.

[0211] Step 9: Select a book and enter your question

[0212] User device:

[0213] Input: Select a specific book from the displayed book list and go to its details page.

[0214] What happens: Visit a book's detail page and enter a question (e.g., "What is the most important step in new business development?").

[0215] Output: The question entered.

[0216] Step 10: Submitting questions and generating answers

[0217] server:

[0218] Input: The user's question.

[0219] How it works: Send a question to the generative AI model via the REST API. The generative AI model analyzes the question and generates an appropriate answer.

[0220] Output: The generated answer.

[0221] Step 11: View your answers

[0222] server:

[0223] Input: The generated answer.

[0224] Specific operation: The generated answer is sent back to the user's device and displayed in real time.

[0225] User device:

[0226] Input: None (display stage).

[0227] Output: The user sees the answer displayed on the screen (e.g., "Imaginative market analysis and targeting are most important.").

[0228] Step 12: Additional Questions

[0229] User device:

[0230] Input: Additional questions (if needed).

[0231] What it does: Review the answers displayed on the screen, enter additional questions if necessary, and submit.

[0232] Output: Additional questions are sent to the generative AI model.

[0233] The above are the specific processing steps of the program of this system.

[0234] (Application example 1)

[0235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] Conventional search systems and AI models have the drawback of making it difficult for users to freely ask questions about books and materials and obtain appropriate answers. It is particularly difficult to ask detailed questions about the contents of each book and obtain immediate answers for books and materials that are widely available in bookstores and libraries. Furthermore, there is a lack of systems that can efficiently digitize vast amounts of analog information, build AI models based on that information, and make them useful to users, making this initiative necessary.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0238] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI modeled materials online, means for providing an interface for users to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for searching for books and selecting specific materials, and means for the generative AI model to analyze questions related to the materials and generate answers. This allows users to freely ask questions about books and materials available in bookstores and libraries and receive appropriate answers in real time.

[0239] "Analog information in Japanese" refers to materials written in Japanese that have not been digitized, such as paper books and handwritten documents.

[0240] "Digital text data" refers to data that has been converted from analog Japanese information into digital text using scanning or optical character recognition (OCR) technology.

[0241] A "generative AI model" is an artificial intelligence model that uses digitized text data as training data to generate appropriate answers to questions.

[0242] "Means for providing online" means means for placing generative AI-modeled materials on a web server or cloud service so that they can be accessed by users via the internet.

[0243] The "interface for inputting questions" refers to a user interface that allows users to input questions in text format to the generative AI model.

[0244] The "means of generating an answer" refers to the process by which the generative AI model analyzes the user's question and generates an appropriate answer.

[0245] "Means for searching and selecting specific materials" means the ability for users to search for materials using specific keywords and select relevant materials.

[0246] "Means for the generative AI model to analyze questions related to the materials and generate answers" refers to the process in which the generative AI model analyzes questions asked by users about selected materials and generates answers based on those materials.

[0247] "Smartphone Application" means a specific software application for installation and use on a smartphone device.

[0248] This invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Detailed embodiments for implementing this system will be described.

[0249] System Program

[0250] 1. Hardware and software configuration

[0251] Hardware: Smartphones (iOS devices, ANDROID (registered trademark) devices), servers

[0252] Software: Python, Flask (for API server), TensorFlow (generative AI model), Tesseract (OCR engine), SQLite (database), React Native (smartphone app development)

[0253] 2. Digitizing data and training generated AI models

[0254] The server scans paper books and handwritten documents available at bookstores and libraries and extracts Japanese text data using the Tesseract OCR engine. The extracted text data is then cleaned, correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is then structured and stored in an SQLite database.

[0255] The server trains a generative AI model using TensorFlow based on the stored cleaned text data, and once trained, the generative AI model is deployed on the Flask API server.

[0256] 3. User interface development and operation

[0257] We developed a smartphone application using React Native, implementing a book search function and a question and answer interface. Through the app, users can find books by keyword search, ask questions about the books, and receive answers in real time.

[0258] Program processing

[0259] 1. Scanning the book and OCR processing

[0260] The server takes scanned images of books provided by bookstores and libraries and extracts the text data using the Tesseract OCR engine.

[0261] The extracted text data is cleaned by the server and stored in an SQLite database.

[0262] 2. Training the generative AI model

[0263] The server uses the cleaned text data as training data to train a generative AI model using TensorFlow.

[0264] Once trained, the generative AI model is deployed on a Flask API server and made accessible online.

[0265] 3. User Interface Operation

[0266] The smartphone application allows users to search for books by entering keywords and select specific books from the search results.

[0267] Users can enter questions into the generative AI model on the book details page and receive real-time answers from the server.

[0268] Examples and prompts

[0269] Examples:

[0270] A user types a question about a history book: "What accomplishments did a particular person achieve?"

[0271] The generative AI model generates and displays the answer, "A specific person achieved numerous victories during the Warring States period, promoted centralized policies, and unified the country."

[0272] Example prompt sentence:

[0273] "Enter a question about a specific person (e.g., 'What accomplishments did a specific person have?')"

[0274] This system allows users to ask any questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[0275] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0276] Step 1:

[0277] The server receives scanned images of books provided by bookstores and libraries. The scanned images are input and converted into text data using Tesseract, an optical character recognition (OCR) engine. The output is OCR-processed text data.

[0278] Step 2:

[0279] The server cleans the OCR-processed text data. Specifically, it corrects misrecognized characters and deletes unnecessary line breaks and spaces. The input is the OCR-processed text data, and the output is the cleaned text data.

[0280] Step 3:

[0281] The server structures the cleaned text data and stores it in an SQLite database. The input is the cleaned text data, and the output is the structured text data stored in the database.

[0282] Step 4:

[0283] The server trains a generative AI model using the stored structured text data. It uses TensorFlow to build the AI ​​model and adjusts its parameters based on the training data. The input is the structured text data, and the output is a trained generative AI model.

[0284] Step 5:

[0285] The server deploys the trained generative AI model on the Flask API server, making the model accessible online. The input is the trained generative AI model, and the output is the deployed generative AI model.

[0286] Step 6:

[0287] Users open the smartphone application and search for books by entering keywords through an interface built with React Native. The input is the search query, and the output is a search result list of matching books.

[0288] Step 7:

[0289] The user selects a particular book from the search results and navigates to that book's detail page. The input is the selection of a particular book, and the output is the display of the book's detail page.

[0290] Step 8:

[0291] The user enters a question on the details page and sends it to the generative AI model. The input is the user's question, and the output is an API request to the generative AI model.

[0292] Step 9:

[0293] The server analyzes the user's question against the generative AI model and generates an appropriate answer. The input is the user's question, and the output is the generated answer.

[0294] Step 10:

[0295] The server sends the generated answer to the user's smartphone, where it is displayed on the user's device. The input is the generated answer, and the output is the answer displayed on the user's device.

[0296] This processing flow allows users to freely ask questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] This invention combines a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0299] Building the dataset

[0300] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[0301] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters, removing unnecessary line breaks and spaces, and standardizing the format. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0302] Creating generative AI models

[0303] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[0304] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0305] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0306] Building an online platform

[0307] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[0308] User Interface and Emotion Engine

[0309] Book Selection and Emotion Recognition

[0310] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0311] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0312] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0313] User device: The user selects a particular book from the displayed list to access its detail page.

[0314] Ask questions, recognize emotions and generate answers

[0315] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0316] Server: Receives the input question and uses the emotion engine to analyze the user's current emotional state. For example, if the user is feeling stressed or anxious, it recognizes that emotion.

[0317] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[0318] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[0319] Server: Sends the generated answer to the user's device.

[0320] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[0321] Specific examples

[0322] Example 1: Using business books

[0323] scenario:

[0324] Let's say the user wants to refer to a specific business book about "strategies for new business development." The user is in a stressful state.

[0325] User device: Access the platform and search for "New Business Development".

[0326] Server: Displays a list of relevant business books as search results.

[0327] User Device: Select a specific business book and go to its details page.

[0328] User device: Type the question, "What is the most important step in new business development?"

[0329] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[0330] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[0331] User device: The user receives the answer and feels relief along with concrete steps.

[0332] Example 2: Using cooking recipes

[0333] scenario:

[0334] Let's say the user wants to consult a cookbook on "how to make authentic French toast." The user is relaxed.

[0335] User device: Accesses the platform and searches for "french toast."

[0336] Server: Displays a list of relevant cookbooks as search results.

[0337] User Device: Select a specific cookbook and go to its detail page.

[0338] User device: Type in the question, "What's the secret to making authentic French toast?"

[0339] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[0340] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[0341] User terminal: Users can receive answers comfortably and enjoy cooking.

[0342] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[0343] The processing flow will be explained below.

[0344] Step 1:

[0345] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[0346] Step 2:

[0347] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[0348] Step 3:

[0349] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0350] Step 4:

[0351] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[0352] Step 5:

[0353] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0354] Step 6:

[0355] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0356] Step 7:

[0357] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0358] Step 8:

[0359] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0360] Step 9:

[0361] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0362] Step 10:

[0363] User device: The user selects a particular book from the displayed list to access its detail page.

[0364] Step 11:

[0365] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0366] Step 12:

[0367] Server: Receives the input question and uses an emotion engine to analyze the user's current emotional state, for example, by inferring emotions from the context of the input text and the user's past behavior.

[0368] Step 13:

[0369] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[0370] Step 14:

[0371] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[0372] Step 15:

[0373] Server: Sends the generated answer to the user's device.

[0374] Step 16:

[0375] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[0376] Specific examples

[0377] Example 1: Using business books

[0378] Step 1:

[0379] User device: The user wants to consult a specific business book on "New Business Development Strategies." The user is in a stressful state.

[0380] Step 2:

[0381] User device: Access the platform and search for "New Business Development".

[0382] Step 3:

[0383] Server: Displays a list of relevant business books as search results.

[0384] Step 4:

[0385] User Device: Select a specific business book and go to its details page.

[0386] Step 5:

[0387] User device: Type the question, "What is the most important step in new business development?"

[0388] Step 6:

[0389] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[0390] Step 7:

[0391] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[0392] Step 8:

[0393] User device: The user receives the answer and feels relief along with concrete steps.

[0394] Example 2: Using cooking recipes

[0395] Step 1:

[0396] User device: The user wants to consult a cookbook on "how to make authentic French toast." The user is in a relaxed state.

[0397] Step 2:

[0398] User device: Accesses the platform and searches for "french toast."

[0399] Step 3:

[0400] Server: Displays a list of relevant cookbooks as search results.

[0401] Step 4:

[0402] User Device: Select a specific cookbook and go to its detail page.

[0403] Step 5:

[0404] User device: Type in the question, "What's the secret to making authentic French toast?"

[0405] Step 6:

[0406] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[0407] Step 7:

[0408] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[0409] Step 8:

[0410] User terminal: Users can receive answers comfortably and enjoy cooking.

[0411] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[0412] Example 2

[0413] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0414] Systems that use generative AI models to help users efficiently acquire information in Japanese need to provide appropriate answers based on the user's emotional state. However, conventional systems generate uniform answers without considering the user's emotional state, making it difficult to fully meet the user's needs. Furthermore, if the digitized text data is not effectively cleaned and structured, and the parameters of the generative AI model are not adjusted effectively, the accuracy of the model decreases.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0416] In this invention, the server includes a means for digitizing Japanese analog information, a means for training a generative AI model based on the digitized text data, and a means for analyzing the user's emotional state and reflecting the analysis results in the generative AI model. This makes it possible to provide answers optimized according to the user's emotional state. Furthermore, high accuracy of answers can be maintained through the process of cleaning and structuring the digitized text data and training the generative AI model.

[0417] "Analog information in Japanese" refers to Japanese information that exists in physical form, rather than digital form, such as paper books or handwritten documents.

[0418] "Digitalization" refers to the conversion of analog information into electronic data in a format that can be processed by a computer.

[0419] "Generative AI model" refers to an artificial intelligence model that has been trained using machine learning techniques to perform a specific task.

[0420] "Training" is the process of providing learning data to a generative AI model to optimize the model's parameters and improve its performance for a specific task.

[0421] "Provided online" means that the service is provided in a form that allows users to access it via the Internet.

[0422] "Interface" refers to the means or operating screen through which a user interacts with a system.

[0423] "Emotional state" refers to the user's current emotional state, such as stressed, relaxed, or anxious.

[0424] "Cleaning" is the process of correcting misrecognized characters in digitized text data and deleting unnecessary data to improve the quality of the data.

[0425] "Structuring" refers to organizing and arranging cleaned text data into a specific format or structure.

[0426] "Parameter tuning" is the process of setting and updating the learning parameters within a generative AI model so that it performs optimally.

[0427] "Search" is the act of locating relevant information or data from a database based on keywords entered by the user.

[0428] "Personalized information provision" refers to providing users with the most appropriate information based on their profile and search history.

[0429] This invention is a system that digitizes analog information in Japanese and trains a generative AI model based on that information, recognizing the user's emotional state and providing the optimal response according to that state. Specific embodiments are described below.

[0430] Building the dataset

[0431] Server: Scan analog books and handwritten documents using a scanner and save the scanned images on the server. We recommend using a high-resolution scanner.

[0432] Server: Input the scanned image into an optical character recognition (OCR) engine to extract the text data. Typically, an OCR engine such as Tesseract or Google® Cloud Vision is used.

[0433] Server: Cleans the extracted text data, corrects misrecognized characters, and deletes unnecessary line breaks and spaces. A natural language processing library (e.g., nltk) is used as text analysis software.

[0434] Server: The clean text data is structured and stored in a database or CSV format for easy subsequent processing.

[0435] Creating generative AI models

[0436] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data. The training process uses machine learning frameworks such as TensorFlow and PyTorch.

[0437] Server: Once training is complete, evaluate the model's performance using the test dataset and tune the model's parameters for optimal performance.

[0438] Server: Deploys the completed generative AI model and makes it accessible to users on an online platform. Uploads the model to a cloud server (e.g., AWS (registered trademark)) and exposes the API.

[0439] Building the user interface

[0440] Server: Develop the user interface using web development tools (e.g., React, Angular), implement book search functionality and a question input form, and integrate user profile management and book purchase links.

[0441] Book Selection and Emotion Recognition

[0442] User terminal: The user opens a web browser and accesses the provided platform URL. The user registers and logs in to the platform.

[0443] User device: Users enter keywords or book titles in the search bar to search for the books they want to read.

[0444] Server: The server receives the search query, generates a list of relevant books from the database, and displays it in the browser.

[0445] User device: The user selects a particular book from the displayed list to access its detail page.

[0446] Ask questions, recognize emotions and generate answers

[0447] User device: On the book details page, the user enters a question into the text box provided to the generative AI model.

[0448] Server: The server receives the input question and uses an emotion engine to analyze the user's emotional state. For example, it identifies if the user is feeling stressed or anxious.

[0449] Server: Sends questions to the generative AI model based on the recognized emotional state and provides response parameters according to the emotion to the model.

[0450] Server: A generative AI model analyzes the question and generates the best answer, including adjustments based on an emotion engine to ensure the answer takes into account the user's emotional state.

[0451] Server: Sends the generated answer to the user's device.

[0452] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[0453] Specific examples

[0454] Examples of using business books

[0455] scenario:

[0456] If a user wants to refer to a business book on "strategies for new business development," the user is in a stressful state.

[0457] Example prompt:

[0458] "What is the most important step in new business development?"

[0459] Operation:

[0460] Users access the platform and receive specific answers along with encouraging comments to help reduce stress.

[0461] Cooking recipe usage examples

[0462] scenario:

[0463] If a user wants to consult a cookbook on "how to make authentic French toast," they're in a relaxed state.

[0464] Example prompt:

[0465] "What's the secret to making authentic French toast?"

[0466] Operation:

[0467] It recognizes when the user is relaxed and adds comments highlighting the good time along with specific answers to enrich the flavor.

[0468] This system allows users to efficiently obtain information that corresponds to the appropriate emotion and utilize it in a more practical way.

[0469] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0470] Step 1:

[0471] Server: Scan analog books and handwritten documents with a scanner to obtain scanned images.

[0472] Specific operation: Place a book page into the scanner and press the scan button to obtain a high-resolution image.

[0473] Input: Analog books and handwritten documents.

[0474] Output: High resolution scanned image.

[0475] Step 2:

[0476] Server: Input the scanned image into an OCR engine to extract text data. For example, we use Tesseract or Google Cloud Vision.

[0477] What it does: The server feeds the scanned image to the OCR software, and the OCR engine extracts the characters in the image as text.

[0478] Input: High resolution scanned image.

[0479] Output: The extracted raw text data.

[0480] Step 3:

[0481] Server: Cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces.

[0482] Specific behavior: The server analyzes the text data using text analysis software (e.g., nltk) and makes corrections to ensure data consistency.

[0483] Input: Extracted raw text data.

[0484] Output: Cleaned text data.

[0485] Step 4:

[0486] Server: The cleaned text data is structured and stored in a database or CSV format.

[0487] What it does: Use the data format conversion tool to convert and save the cleaned text into CSV or database format.

[0488] Input: Cleaned text data.

[0489] Output: Structured data.

[0490] Step 5:

[0491] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data.

[0492] What it does: The server feeds training data to an AI model framework (e.g., TensorFlow or PyTorch) and updates the model parameters every epoch.

[0493] Input: Structured data.

[0494] Output: A trained generative AI model.

[0495] Step 6:

[0496] Server: Evaluates the performance of the trained model using the test dataset, and retunes model parameters if necessary.

[0497] How it works: The server feeds test data to the generative AI model, compares the generated output with the original output, and evaluates the error rate and accuracy. It adjusts hyperparameters as needed.

[0498] Input: A trained generative AI model and a test dataset.

[0499] Output: An evaluated and tuned generative AI model.

[0500] Step 7:

[0501] Server: The completed generative AI model is deployed to a cloud server and made accessible on an online platform.

[0502] Specific operation: The server uploads the generated AI model to a cloud service (e.g., AWS) and exposes an API, allowing users to access the generated AI model.

[0503] Input: An evaluated and tuned generative AI model.

[0504] Output: A generative AI model accessible on the cloud.

[0505] Step 8:

[0506] Server: Develop the user interface and implement the book search function and question input form. Use web development tools (e.g. React, Angular).

[0507] What it does: Front-end developers design the user interface, and back-end developers work with APIs to implement the functionality.

[0508] Input: Design specifications and design requirements.

[0509] Output: User interface.

[0510] Step 9:

[0511] User device: The user opens a web browser, accesses the platform, and logs in.

[0512] Specific actions: Enter a URL in your browser, enter the required information in the login form that appears, and click the login button.

[0513] Input: User login information.

[0514] Output: Access to the platform.

[0515] Step 10:

[0516] User device: To find the book they want to read, users enter keywords or titles into the search bar.

[0517] What happens: A user enters keywords into the search bar and clicks the search button, which sends a search query to the server.

[0518] Input: Search keyword or title.

[0519] Output: The search query.

[0520] Step 11:

[0521] Server: Based on the search query, the server generates a list of relevant books from its database and displays it in the user's browser.

[0522] What happens: The server executes a database query to retrieve related book information and returns it to the browser.

[0523] Input: Your search query.

[0524] Output: A list of related books.

[0525] Step 12:

[0526] User device: The user selects a particular book from the displayed list to access its detail page.

[0527] What happens: The user clicks on the book they want to read and sees more information about it.

[0528] Input: Book selection.

[0529] Output: Book detail page.

[0530] Step 13:

[0531] User device: On the book detail page, the user enters a question for the generative AI model into a text box.

[0532] What happens: The user types a question into the text box and clicks the submit button.

[0533] Input: The user's question.

[0534] Output: The question query.

[0535] Step 14:

[0536] Server: The server receives the question and analyzes the user's emotional state using an emotion engine.

[0537] Specific operation: The server passes the question data to the emotion engine and receives the analysis results.

[0538] Input: The user's question.

[0539] Output: Emotional state information.

[0540] Step 15:

[0541] Server: Sends questions to the generative AI model based on emotion information and reflects response parameters according to the emotion.

[0542] How it works: The server provides the question and emotion parameters to the generative AI model, which then generates a response.

[0543] Input: Question query and emotional state information.

[0544] Output: The answer according to the user.

[0545] Step 16:

[0546] Server: Sends the generated answer to the user's device.

[0547] What happens: The server generates the answer and returns it to the user's browser.

[0548] Input: The answer given to the user.

[0549] Output: The answer text.

[0550] Step 17:

[0551] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[0552] What it does: The user reads the displayed answer and decides whether to enter a follow-up question.

[0553] Input: The displayed answer.

[0554] Output: Change in the user's emotional state and follow-up questions.

[0555] (Application example 2)

[0556] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0557] Conventional digital content delivery systems provide uniform information without considering the user's emotional state, resulting in a limited user experience and a lack of means to provide information optimally tailored to the user's current emotional state. As a result, user satisfaction and engagement may be low. To solve these issues, a system is needed that recognizes user emotions in real time and provides personalized content using generative AI models accordingly.

[0558] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0559] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple books that have been modeled by the generative AI online, means for providing an interface through which a user inputs questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, emotion analysis means for recognizing the user's emotions, and means for adjusting the content of the generative AI model's answers based on the recognized emotion information. This makes it possible to provide optimal information taking into account the user's emotional state.

[0560] "Means for digitizing analog information in Japanese" refers to scanning analog information such as paper books and handwritten documents and extracting text data using optical character recognition.

[0561] "Means for training a generative AI model based on digitized text data" refers to means for using extracted text data to adjust the parameters of a generative AI model and train the generative AI model.

[0562] "Means for providing multiple generative AI modeled books online" refers to means for distributing text data and book data generated using generative AI models through an online platform.

[0563] "Means for providing an interface for users to input questions to a generative AI model" refers to means for providing an interface, such as a web browser or application screen, for users to input questions to a generative AI model.

[0564] "Means by which a generative AI model generates an answer to a user's question" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[0565] "Means for displaying the generated answer to the user" refers to means for displaying the answer generated by the generative AI model on the user's screen.

[0566] "Emotion analysis means for recognizing user emotions" refers to a means for analyzing a user's emotional state from their facial expressions and voice using sensors such as a camera and microphone.

[0567] "Means for adjusting the content of the response of the generative AI model based on recognized emotional information" refers to means for adjusting the content and tone of the response generated by the generative AI model according to the emotional state of the user.

[0568] "Means for cleaning digitized text data" refers to means for generating clean text data by correcting misrecognized characters, deleting unnecessary line breaks and spaces, etc.

[0569] The "means for structuring the cleaned text data" refers to a means for storing the cleaned text data in a structured format such as a database or CSV format.

[0570] "Means for recommending personalized video content based on user emotions" refers to means for analyzing a user's emotional state and recommending and providing appropriate video content based on that state.

[0571] A "means for users to search for specific books" is a means that provides a search bar or filter functionality that allows users to find specific books.

[0572] A "means for displaying search results to a user" is a means for displaying on a user's screen a list of books found based on the search query.

[0573] "Means for monitoring changes in the user's emotions" refers to means for tracking and recording changes in the user's emotional state in real time using a camera or microphone.

[0574] "Means for adaptively changing content recommendations based on monitoring results" refers to means for detecting changes in a user's emotions and dynamically changing the content provided accordingly.

[0575] System construction

[0576] To implement this invention, a server and a user terminal are required. The server digitizes analog information, trains the generative AI model, and provides books online. The user terminal provides an interface for users to input questions and perform searches, and displays generated answers.

[0577] Building the dataset

[0578] server:

[0579] The server first obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are then fed into an optical character recognition (OCR) engine to extract text data. Specific software used at this stage is, for example, Tesseract OCR.

[0580] server:

[0581] The extracted text data is cleaned, misrecognized characters are corrected, and unnecessary line breaks and spaces are deleted. The cleaned text data is then structured and saved in a database or CSV format, for example.

[0582] Creating generative AI models

[0583] server:

[0584] A generative AI model (e.g., GPT-3) is trained on the cleaned text data. The training process takes place over multiple epochs (iterations).

[0585] server:

[0586] Once the model has been trained and its performance evaluated, the generative AI model is deployed and made accessible to users.

[0587] User Interface and Emotion Engine

[0588] User device:

[0589] Users access the provided platform URL using a smartphone or head-mounted display, and search for the book they want to read by entering keywords or the title of the book in the search bar.

[0590] server:

[0591] It receives a search query, generates a list of relevant books from its database, and displays them to the user.

[0592] User device:

[0593] A user visits the detail page of a particular book and enters a question into the generative AI model.

[0594] Ask questions, recognize emotions and generate answers

[0595] User device:

[0596] When a user types a question, the device's camera recognizes the user's facial expression, and the emotion engine analyzes the user's emotional state. For example, if the user is feeling stressed, it will recognize that emotion.

[0597] server:

[0598] Based on the analysis results of the emotion engine, emotional information is sent to the generative AI model. Parameters for adjusting the response content according to the user's emotions are also sent. Based on this, the generative AI model generates a response that takes the user's emotions into consideration.

[0599] server:

[0600] The generated answers are sent to the user's device, where the user can check the answers displayed on the screen. Changes in the user's emotions are continuously monitored, and the answers are updated as needed.

[0601] Specific examples

[0602] scenario:

[0603] Suppose a student wants to "relax" and wants to watch a relaxing music video. If the emotion engine recognizes the user's sad emotion, the generative AI model will recommend a video.

[0604] Example prompt sentence:

[0605] "Please recommend video content that is sad and suitable for relaxation."

[0606] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0607] Step 1:

[0608] The server digitizes analog information

[0609] Input: Scanned image of an analog book

[0610] The server acquires scanned images of analog books and extracts text data using an optical character recognition (OCR) engine. Specifically, the scanned images are input into an OCR engine (e.g., Tesseract OCR) for character recognition. The output is text data.

[0611] Step 2:

[0612] The server cleans the digitized text data.

[0613] Input: Text data extracted by OCR

[0614] The server cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces. Specifically, the script analyzes the text data and automatically corrects misrecognized characters and unnecessary spaces. The output is clean text data.

[0615] Step 3:

[0616] The server trains the generative AI model

[0617] Input: clean text data

[0618] The server trains a generative AI model (e.g., GPT-3) based on clean text data. Specifically, it takes the text data as training data for the model and adjusts the model's parameters. Training is performed over multiple epochs (iterations). The output is a trained generative AI model.

[0619] Step 4:

[0620] The server deploys the generative AI model

[0621] Input: A trained generative AI model

[0622] The server evaluates the trained generative AI model, confirms its performance, and then deploys it to the online platform. Specifically, it uses an evaluation dataset to verify the model's performance and adjusts it for optimal performance. The output is a generative AI model that can be used on the online platform.

[0623] Step 5:

[0624] The user terminal provides the interface

[0625] Input: User access request

[0626] The user device accesses the provided platform URL via a smartphone or head-mounted display. Specifically, the user enters keywords or book titles in the search bar, and an interface for searching for the book they want to read is displayed. The output is the interface displayed to the user.

[0627] Step 6:

[0628] The server searches for books and displays the results

[0629] Input: User's search query

[0630] The server receives a search query, generates a list of relevant books from the database, and displays it to the user. Specifically, it performs a database search based on the query and sends the results to the user's device. The output is a list of search results.

[0631] Step 7:

[0632] The user's device inputs questions and analyzes emotions.

[0633] Input: User's question and facial expression

[0634] A user accesses the details page of a specific book and enters a question. At the same time, the device's camera recognizes the user's facial expressions, and the emotion engine analyzes their emotional state. Specifically, the camera captures the user's facial expression data, and an emotion analysis algorithm is used to identify their emotional state. The output is the user's question and emotional state.

[0635] Step 8:

[0636] The server generates an answer based on the sentiment

[0637] Input: User's question and emotional state

[0638] The server sends emotional information to the generative AI model based on the analysis results of the emotion engine. It also sends parameters to adjust the answer content according to the user's emotions. Specifically, the generative AI model generates an answer based on the question and emotional information. The output is an answer that takes emotions into consideration.

[0639] Step 9:

[0640] The server sends the answer to the user's device.

[0641] Input: Generated Answer

[0642] The server sends the generated answer to the user terminal. Specifically, the generated answer data is sent to the user terminal and displayed on the user's screen. The output is the answer displayed to the user.

[0643] Step 10:

[0644] User devices monitor emotional changes

[0645] Input: Continuous facial expression data of the user

[0646] While the user is checking the answers displayed on the screen, the emotion engine continuously monitors the user's emotional changes. Specifically, the device's camera continuously captures facial expression data and performs emotion analysis in real time. The output is the analysis results of emotional changes.

[0647] Step 11:

[0648] The server adaptively updates content recommendations

[0649] Input: Emotion change analysis results

[0650] The server dynamically adjusts the content provided based on the monitoring results. Specifically, it retrieves new content from the generative AI model that reflects the analysis results of emotion changes and provides it to the user. The output is the updated content.

[0651] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0652] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0653] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0654] [Second embodiment]

[0655] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0656] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0657] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0658] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0659] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0660] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0661] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0662] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0663] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0664] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0665] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0666] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0667] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Specific embodiments of this system are described below.

[0668] Building the dataset

[0669] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[0670] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is structured and stored in a database.

[0671] Creating generative AI models

[0672] Server: Builds a generative AI model that learns the content of specific books based on the cleaned text data. Trains the generative AI model based on the text taken as training data and adjusts the parameters accordingly.

[0673] Server: Deploys the trained generative AI model and makes it available to users online.

[0674] Building an online platform

[0675] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[0676] User Interface and Operation

[0677] Book Selection and Questions

[0678] User device: The user opens a web browser, accesses the provided platform URL, enters keywords in the search bar, and searches for the book they want to read.

[0679] Server: Receives the search query, generates a list of relevant books, and displays them to the user.

[0680] User device: The user selects a particular book from the displayed list to access its detail page.

[0681] Ask a question and get an answer

[0682] User device: On the details page of the selected book, the user enters a question into the text box that asks the generative AI model a question.

[0683] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[0684] Server: The generative AI model analyzes the question and generates an appropriate answer.

[0685] Server: Sends the generated answer to the user's device.

[0686] User device: The user checks the answers displayed on the screen and asks further questions if necessary.

[0687] Specific examples

[0688] Example 1: Using business books

[0689] scenario:

[0690] Let's say a user wants to refer to a specific business book on "strategies for new business development."

[0691] User device: Access the platform and search for "New Business Development".

[0692] Server: Displays a list of relevant business books as search results.

[0693] User Device: Select a specific business book and go to its details page.

[0694] User device: Type the question, "What is the most important step in new business development?"

[0695] Server: Sends questions to the generative AI model and generates answers.

[0696] User device: Receive specific answers such as, "Imaginative market analysis and targeting are the most important."

[0697] Example 2: Using cooking recipes

[0698] scenario:

[0699] Let's say a user wants to consult a cookbook on how to make authentic French toast.

[0700] User device: Accesses the platform and searches for "french toast."

[0701] Server: Displays a list of relevant cookbooks as search results.

[0702] User Device: Select a specific cookbook and go to its detail page.

[0703] User device: Type in the question, "What's the secret to making authentic French toast?"

[0704] Server: Sends questions to the generative AI model and generates answers.

[0705] User device: Receives a response such as "Adding a little vanilla extract will give it an even richer flavor."

[0706] The above is a specific embodiment for carrying out the present invention. This system enables users to effectively obtain information and put it to practical use.

[0707] The processing flow will be explained below.

[0708] Step 1:

[0709] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[0710] Step 2:

[0711] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[0712] Step 3:

[0713] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0714] Step 4:

[0715] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[0716] Step 5:

[0717] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0718] Step 6:

[0719] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0720] Step 7:

[0721] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0722] Step 8:

[0723] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0724] Step 9:

[0725] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0726] Step 10:

[0727] User device: The user selects a particular book from the displayed list to access its detail page.

[0728] Step 11:

[0729] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0730] Step 12:

[0731] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[0732] Step 13:

[0733] Server: The generative AI model analyzes the question and generates an appropriate answer.

[0734] Step 14:

[0735] Server: Sends the generated answer to the user's device.

[0736] Step 15:

[0737] User device: The user reviews the answers displayed on the screen and asks further questions if necessary.

[0738] Example 1

[0739] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0740] Previously, digitizing and utilizing analog Japanese information has faced challenges such as low-accuracy OCR technology, inefficient data cleaning methods, and a lack of user-friendly interfaces. Furthermore, building an entire system to train generative AI models and provide appropriate information to users has been complex. There is a need to resolve these issues and provide high-quality digital information while enabling users to access it efficiently.

[0741] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0742] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI models online, means for providing an interface for a user to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for cleaning the digitized text data, means for structuring the cleaned text data, means for a user to search for specific data, and means for displaying search results to the user. This makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and effectively access the information they need.

[0743] "Analog information in Japanese" refers to information written in Japanese that is not stored as electronic data, such as paper books and handwritten documents.

[0744] "Digitizing" means converting analog information into an electronic data format that can be processed by a computer.

[0745] "Text data" refers to data that electronically records information such as letters, sentences, and symbols.

[0746] A "generative AI model" refers to an artificial intelligence model that can learn from text data and perform specific tasks.

[0747] "Training" means using large amounts of data to learn from an AI model and improve its performance.

[0748] "Providing online" means making a service or function available to users via the Internet.

[0749] "Interface" refers to the screens and input means through which users interact with a system.

[0750] "Cleaning" refers to the process of correcting misrecognized parts from text data and removing unnecessary spaces and line breaks.

[0751] "Structuring" means organizing data into a certain format or structure so that it can be easily searched and used.

[0752] "Deploying" means placing a trained AI model in an actual production environment and making it available for use.

[0753] "Database" refers to an electronic system for efficiently storing, managing, and retrieving data.

[0754] "Searching" means finding data based on specific keywords or conditions.

[0755] "Results" refers to the final information or data obtained through operations such as search or calculation.

[0756] An "API request" is a request made by an application to another system to use a function or obtain information.

[0757] "REST API" refers to an API based on uniform design principles for exchanging data between web services.

[0758] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model, and provides it to users. Detailed embodiments for implementing this system are described below.

[0759] Building the dataset

[0760] server:

[0761] To digitize analog information, first, paper books or handwritten documents are scanned to obtain image data. The hardware used is, for example, the common scanner "Epson Perfection V600." This scanned image is then input into the OCR engine "Tesseract OCR," and text data is extracted.

[0762] server:

[0763] Next, the extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. A Python script is used for cleaning. The cleaned text data is then stored in a MySQL database. The stored data is then structured so that it can be used for later training and search.

[0764] Creating generative AI models

[0765] server:

[0766] A generative AI model is trained on the cleaned text data using a machine learning framework such as TensorFlow or PyTorch, which inputs training data and adjusts certain parameters to improve the model's accuracy.

[0767] server:

[0768] Once trained, the generative AI model is deployed in a Docker container, making it operational and enabling it to quickly provide answers to user queries.

[0769] Building an online platform

[0770] server:

[0771] The user interface will be developed using ReactJS and Node.js. This interface allows users to operate it intuitively. The interface will include a book search function, an access interface to the generative AI model, user profile management, and the ability to add links to purchase books. The search function will use Elasticsearch to quickly extract relevant information from large amounts of data.

[0772] User interface and operation examples

[0773] Book Selection and Questions

[0774] User device:

[0775] A user opens a web browser, visits the provided platform URL (e.g., www.examplebooks.com), and enters keywords into the search bar to search for a specific book (e.g., "new business development" or "French toast").

[0776] server:

[0777] It receives a search query, uses Elasticsearch to generate a list of relevant books, and displays them to the user.

[0778] User device:

[0779] The user selects a particular book from the displayed list to access its detail page.

[0780] Ask a question and get an answer

[0781] User device:

[0782] Go to the details page and type your question into the text box (e.g., "What is the most important step in new business development?").

[0783] server:

[0784] The entered question is sent to the generative AI model using the REST API.

[0785] server:

[0786] A generative AI model analyzes the question and generates an appropriate answer, which is composed in natural language based on the model's training data.

[0787] server:

[0788] The generated answer is sent back to the user's device and displayed in real time.

[0789] User device:

[0790] Users review the answers displayed on the screen (e.g., "Imaginative market analysis and targeting are paramount") and ask further questions if necessary.

[0791] This invention makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and efficiently access the information they need.

[0792] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0793] Specific explanation of program processing

[0794] Step 1: Digitizing analog information

[0795] server:

[0796] Input: Analog information such as paper books and handwritten documents.

[0797] Specific operation: Analog information is scanned using a scanner. The hardware used is the "Epson Perfection V600."

[0798] Output: Obtain a high-resolution scanned image.

[0799] Step 2: Extract text using OCR

[0800] server:

[0801] Input: Scanned image data.

[0802] Specific operation: The acquired scanned image is input into the OCR engine "Tesseract OCR" and text data is extracted from the image.

[0803] Call the Tesseract API specifying the path to the image data provided by the client or the database.

[0804] Output: The extracted text data.

[0805] Step 3: Cleaning the text data

[0806] server:

[0807] Input: Text data extracted by OCR.

[0808] What it does: Uses a Python script to clean text, including correcting misrecognized characters and removing unnecessary line breaks and spaces.

[0809] Output: Cleaned text data.

[0810] Step 4: Structuring and storing text data

[0811] server:

[0812] Input: Cleaned text data.

[0813] What it does: It uses a Python script to structure text data into the required format, then stores it in a MySQL database, which also contains metadata such as book title, author, and content.

[0814] Output: Structured text data stored in a database.

[0815] Step 5: Training the generative AI model

[0816] server:

[0817] Input: Cleaned text data stored in a database.

[0818] How it works: Use TensorFlow or PyTorch to train a generative AI model. Input training data into the AI ​​model and adjust parameters such as the number of epochs, batch size, and learning rate.

[0819] Output: A fully trained generative AI model.

[0820] Step 6: Deploying the generative AI model

[0821] server:

[0822] Input: A fully trained generative AI model.

[0823] What it does: Deploy an AI model in a Docker container. The containerized model runs on a server and is accessible externally through an API.

[0824] Output: A usable generative AI model is deployed in a production environment.

[0825] Step 7: Developing the User Interface

[0826] server:

[0827] Input: None (early development stage).

[0828] Specific operation: Using "ReactJS" and "Node.js", a user interface will be developed, which will include a book search function, an access interface to the generative AI model, user profile management, and a link to purchase the book.

[0829] Output: A web platform that users can access.

[0830] Step 8: Book Search

[0831] User device:

[0832] Type: Enter a keyword into the search bar (e.g., "new business development" or "french toast").

[0833] Specific actions: Open a web browser, access the provided platform URL, and enter keywords in the search bar.

[0834] server:

[0835] Input: The user's search query.

[0836] Specific operation: Uses Elasticsearch to generate a list of relevant books based on the search query and displays it to the user.

[0837] Output: Shows a list of books relevant to the user.

[0838] Step 9: Select a book and enter your question

[0839] User device:

[0840] Input: Select a specific book from the displayed book list and go to its details page.

[0841] What happens: Visit a book's detail page and enter a question (e.g., "What is the most important step in new business development?").

[0842] Output: The question entered.

[0843] Step 10: Submitting questions and generating answers

[0844] server:

[0845] Input: The user's question.

[0846] How it works: Send a question to the generative AI model via the REST API. The generative AI model analyzes the question and generates an appropriate answer.

[0847] Output: The generated answer.

[0848] Step 11: View your answers

[0849] server:

[0850] Input: The generated answer.

[0851] Specific operation: The generated answer is sent back to the user's device and displayed in real time.

[0852] User device:

[0853] Input: None (display stage).

[0854] Output: The user sees the answer displayed on the screen (e.g., "Imaginative market analysis and targeting are most important.").

[0855] Step 12: Additional Questions

[0856] User device:

[0857] Input: Additional questions (if needed).

[0858] What it does: Review the answers displayed on the screen, enter additional questions if necessary, and submit.

[0859] Output: Additional questions are sent to the generative AI model.

[0860] The above are the specific processing steps of the program of this system.

[0861] (Application example 1)

[0862] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] Conventional search systems and AI models have the drawback of making it difficult for users to freely ask questions about books and materials and obtain appropriate answers. It is particularly difficult to ask detailed questions about the contents of each book and obtain immediate answers for books and materials that are widely available in bookstores and libraries. Furthermore, there is a lack of systems that can efficiently digitize vast amounts of analog information, build AI models based on that information, and make them useful to users, making this initiative necessary.

[0864] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0865] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI modeled materials online, means for providing an interface for users to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for searching for books and selecting specific materials, and means for the generative AI model to analyze questions related to the materials and generate answers. This allows users to freely ask questions about books and materials available in bookstores and libraries and receive appropriate answers in real time.

[0866] "Analog information in Japanese" refers to materials written in Japanese that have not been digitized, such as paper books and handwritten documents.

[0867] "Digital text data" refers to data that has been converted from analog Japanese information into digital text using scanning or optical character recognition (OCR) technology.

[0868] A "generative AI model" is an artificial intelligence model that uses digitized text data as training data to generate appropriate answers to questions.

[0869] "Means for providing online" means means for placing generative AI-modeled materials on a web server or cloud service so that they can be accessed by users via the internet.

[0870] The "interface for inputting questions" refers to a user interface that allows users to input questions in text format to the generative AI model.

[0871] The "means of generating an answer" refers to the process by which the generative AI model analyzes the user's question and generates an appropriate answer.

[0872] "Means for searching and selecting specific materials" means the ability for users to search for materials using specific keywords and select relevant materials.

[0873] "Means for the generative AI model to analyze questions related to the materials and generate answers" refers to the process in which the generative AI model analyzes questions asked by users about selected materials and generates answers based on those materials.

[0874] "Smartphone Application" means a specific software application for installation and use on a smartphone device.

[0875] This invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Detailed embodiments for implementing this system will be described.

[0876] System Program

[0877] 1. Hardware and software configuration

[0878] Hardware: Smartphones (iOS devices, Android devices), servers

[0879] Software: Python, Flask (for API server), TensorFlow (generative AI model), Tesseract (OCR engine), SQLite (database), React Native (smartphone app development)

[0880] 2. Digitizing data and training generated AI models

[0881] The server scans paper books and handwritten documents available at bookstores and libraries and extracts Japanese text data using the Tesseract OCR engine. The extracted text data is then cleaned, correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is then structured and stored in an SQLite database.

[0882] The server trains a generative AI model using TensorFlow based on the stored cleaned text data, and once trained, the generative AI model is deployed on the Flask API server.

[0883] 3. User interface development and operation

[0884] We developed a smartphone application using React Native, implementing a book search function and a question and answer interface. Through the app, users can find books by keyword search, ask questions about the books, and receive answers in real time.

[0885] Program processing

[0886] 1. Scanning the book and OCR processing

[0887] The server takes scanned images of books provided by bookstores and libraries and extracts the text data using the Tesseract OCR engine.

[0888] The extracted text data is cleaned by the server and stored in an SQLite database.

[0889] 2. Training the generative AI model

[0890] The server uses the cleaned text data as training data to train a generative AI model using TensorFlow.

[0891] Once trained, the generative AI model is deployed on a Flask API server and made accessible online.

[0892] 3. User Interface Operation

[0893] The smartphone application allows users to search for books by entering keywords and select specific books from the search results.

[0894] Users can enter questions into the generative AI model on the book details page and receive real-time answers from the server.

[0895] Examples and prompts

[0896] Examples:

[0897] A user types a question about a history book: "What accomplishments did a particular person achieve?"

[0898] The generative AI model generates and displays the answer, "A specific person achieved numerous victories during the Warring States period, promoted centralized policies, and unified the country."

[0899] Example prompt sentence:

[0900] "Enter a question about a specific person (e.g., 'What accomplishments did a specific person have?')"

[0901] This system allows users to ask any questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[0902] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0903] Step 1:

[0904] The server receives scanned images of books provided by bookstores and libraries. The scanned images are input and converted into text data using Tesseract, an optical character recognition (OCR) engine. The output is OCR-processed text data.

[0905] Step 2:

[0906] The server cleans the OCR-processed text data. Specifically, it corrects misrecognized characters and deletes unnecessary line breaks and spaces. The input is the OCR-processed text data, and the output is the cleaned text data.

[0907] Step 3:

[0908] The server structures the cleaned text data and stores it in an SQLite database. The input is the cleaned text data, and the output is the structured text data stored in the database.

[0909] Step 4:

[0910] The server trains a generative AI model using the stored structured text data. It uses TensorFlow to build the AI ​​model and adjusts its parameters based on the training data. The input is the structured text data, and the output is a trained generative AI model.

[0911] Step 5:

[0912] The server deploys the trained generative AI model on the Flask API server, making the model accessible online. The input is the trained generative AI model, and the output is the deployed generative AI model.

[0913] Step 6:

[0914] Users open the smartphone application and search for books by entering keywords through an interface built with React Native. The input is the search query, and the output is a search result list of matching books.

[0915] Step 7:

[0916] The user selects a particular book from the search results and navigates to that book's detail page. The input is the selection of a particular book, and the output is the display of the book's detail page.

[0917] Step 8:

[0918] The user enters a question on the details page and sends it to the generative AI model. The input is the user's question, and the output is an API request to the generative AI model.

[0919] Step 9:

[0920] The server analyzes the user's question against the generative AI model and generates an appropriate answer. The input is the user's question, and the output is the generated answer.

[0921] Step 10:

[0922] The server sends the generated answer to the user's smartphone, where it is displayed on the user's device. The input is the generated answer, and the output is the answer displayed on the user's device.

[0923] This processing flow allows users to freely ask questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[0924] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0925] This invention combines a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0926] Building the dataset

[0927] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[0928] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters, removing unnecessary line breaks and spaces, and standardizing the format. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0929] Creating generative AI models

[0930] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[0931] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0932] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0933] Building an online platform

[0934] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[0935] User Interface and Emotion Engine

[0936] Book Selection and Emotion Recognition

[0937] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0938] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0939] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0940] User device: The user selects a particular book from the displayed list to access its detail page.

[0941] Ask questions, recognize emotions and generate answers

[0942] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0943] Server: Receives the input question and uses the emotion engine to analyze the user's current emotional state. For example, if the user is feeling stressed or anxious, it recognizes that emotion.

[0944] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[0945] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[0946] Server: Sends the generated answer to the user's device.

[0947] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[0948] Specific examples

[0949] Example 1: Using business books

[0950] scenario:

[0951] Let's say the user wants to refer to a specific business book about "strategies for new business development." The user is in a stressful state.

[0952] User device: Access the platform and search for "New Business Development".

[0953] Server: Displays a list of relevant business books as search results.

[0954] User Device: Select a specific business book and go to its details page.

[0955] User device: Type the question, "What is the most important step in new business development?"

[0956] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[0957] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[0958] User device: The user receives the answer and feels relief along with concrete steps.

[0959] Example 2: Using cooking recipes

[0960] scenario:

[0961] Let's say the user wants to consult a cookbook on "how to make authentic French toast." The user is relaxed.

[0962] User device: Accesses the platform and searches for "french toast."

[0963] Server: Displays a list of relevant cookbooks as search results.

[0964] User Device: Select a specific cookbook and go to its detail page.

[0965] User device: Type in the question, "What's the secret to making authentic French toast?"

[0966] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[0967] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[0968] User terminal: Users can receive answers comfortably and enjoy cooking.

[0969] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize the information in a more practical manner.

[0970] The processing flow will be explained below.

[0971] Step 1:

[0972] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[0973] Step 2:

[0974] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[0975] Step 3:

[0976] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[0977] Step 4:

[0978] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[0979] Step 5:

[0980] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[0981] Step 6:

[0982] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[0983] Step 7:

[0984] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[0985] Step 8:

[0986] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[0987] Step 9:

[0988] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[0989] Step 10:

[0990] User device: The user selects a particular book from the displayed list to access its detail page.

[0991] Step 11:

[0992] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[0993] Step 12:

[0994] Server: Receives the input question and uses an emotion engine to analyze the user's current emotional state, for example, by inferring emotions from the context of the input text and the user's past behavior.

[0995] Step 13:

[0996] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[0997] Step 14:

[0998] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[0999] Step 15:

[1000] Server: Sends the generated answer to the user's device.

[1001] Step 16:

[1002] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[1003] Specific examples

[1004] Example 1: Using business books

[1005] Step 1:

[1006] User device: The user wants to consult a specific business book on "New Business Development Strategies." The user is in a stressful state.

[1007] Step 2:

[1008] User device: Access the platform and search for "New Business Development".

[1009] Step 3:

[1010] Server: Displays a list of relevant business books as search results.

[1011] Step 4:

[1012] User Device: Select a specific business book and go to its details page.

[1013] Step 5:

[1014] User device: Type the question, "What is the most important step in new business development?"

[1015] Step 6:

[1016] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[1017] Step 7:

[1018] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[1019] Step 8:

[1020] User device: The user receives the answer and feels relief along with concrete steps.

[1021] Example 2: Using cooking recipes

[1022] Step 1:

[1023] User device: The user wants to consult a cookbook on "how to make authentic French toast." The user is in a relaxed state.

[1024] Step 2:

[1025] User device: Accesses the platform and searches for "french toast."

[1026] Step 3:

[1027] Server: Displays a list of relevant cookbooks as search results.

[1028] Step 4:

[1029] User Device: Select a specific cookbook and go to its detail page.

[1030] Step 5:

[1031] User device: Type in the question, "What's the secret to making authentic French toast?"

[1032] Step 6:

[1033] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[1034] Step 7:

[1035] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[1036] Step 8:

[1037] User terminal: Users can receive answers comfortably and enjoy cooking.

[1038] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[1039] Example 2

[1040] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1041] Systems that use generative AI models to help users efficiently acquire information in Japanese need to provide appropriate answers based on the user's emotional state. However, conventional systems generate uniform answers without considering the user's emotional state, making it difficult to fully meet the user's needs. Furthermore, if the digitized text data is not effectively cleaned and structured, and the parameters of the generative AI model are not adjusted effectively, the accuracy of the model decreases.

[1042] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1043] In this invention, the server includes a means for digitizing Japanese analog information, a means for training a generative AI model based on the digitized text data, and a means for analyzing the user's emotional state and reflecting the analysis results in the generative AI model. This makes it possible to provide answers optimized according to the user's emotional state. Furthermore, high accuracy of answers can be maintained through the process of cleaning and structuring the digitized text data and training the generative AI model.

[1044] "Analog information in Japanese" refers to Japanese information that exists in physical form, rather than digital form, such as paper books or handwritten documents.

[1045] "Digitalization" refers to the conversion of analog information into electronic data in a format that can be processed by a computer.

[1046] "Generative AI model" refers to an artificial intelligence model that has been trained using machine learning techniques to perform a specific task.

[1047] "Training" is the process of providing learning data to a generative AI model to optimize the model's parameters and improve its performance for a specific task.

[1048] "Provided online" means that the service is provided in a form that allows users to access it via the Internet.

[1049] "Interface" refers to the means or operating screen through which a user interacts with a system.

[1050] "Emotional state" refers to the user's current emotional state, such as stressed, relaxed, or anxious.

[1051] "Cleaning" is the process of correcting misrecognized characters in digitized text data and deleting unnecessary data to improve the quality of the data.

[1052] "Structuring" refers to organizing and arranging cleaned text data into a specific format or structure.

[1053] "Parameter tuning" is the process of setting and updating the learning parameters within a generative AI model so that it performs optimally.

[1054] "Search" is the act of locating relevant information or data from a database based on keywords entered by the user.

[1055] "Personalized information provision" refers to providing users with the most appropriate information based on their profile and search history.

[1056] This invention is a system that digitizes analog information in Japanese and trains a generative AI model based on that information, recognizing the user's emotional state and providing the optimal response according to that state. Specific embodiments are described below.

[1057] Building the dataset

[1058] Server: Scan analog books and handwritten documents using a scanner and save the scanned images on the server. We recommend using a high-resolution scanner.

[1059] Server: Input the scanned image into an optical character recognition (OCR) engine to extract the text data. Typically, an OCR engine such as Tesseract or Google Cloud Vision is used.

[1060] Server: Cleans the extracted text data, corrects misrecognized characters, and deletes unnecessary line breaks and spaces. A natural language processing library (e.g., nltk) is used as text analysis software.

[1061] Server: The clean text data is structured and stored in a database or CSV format for easy subsequent processing.

[1062] Creating generative AI models

[1063] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data. The training process uses machine learning frameworks such as TensorFlow and PyTorch.

[1064] Server: Once training is complete, evaluate the model's performance using the test dataset and tune the model's parameters for optimal performance.

[1065] Server: Deploy the completed generative AI model and make it accessible to users on an online platform. Upload the model to a cloud server (e.g., AWS) and expose the API.

[1066] Building the user interface

[1067] Server: Develop the user interface using web development tools (e.g., React, Angular), implement book search functionality and a question input form, and integrate user profile management and book purchase links.

[1068] Book Selection and Emotion Recognition

[1069] User terminal: The user opens a web browser and accesses the provided platform URL. The user registers and logs in to the platform.

[1070] User device: Users enter keywords or book titles in the search bar to search for the books they want to read.

[1071] Server: The server receives the search query, generates a list of relevant books from the database, and displays it in the browser.

[1072] User device: The user selects a particular book from the displayed list to access its detail page.

[1073] Ask questions, recognize emotions and generate answers

[1074] User device: On the book details page, the user enters a question into the text box provided to the generative AI model.

[1075] Server: The server receives the input question and uses an emotion engine to analyze the user's emotional state. For example, it identifies if the user is feeling stressed or anxious.

[1076] Server: Sends questions to the generative AI model based on the recognized emotional state and provides response parameters according to the emotion to the model.

[1077] Server: A generative AI model analyzes the question and generates the best answer, including adjustments based on an emotion engine to ensure the answer takes into account the user's emotional state.

[1078] Server: Sends the generated answer to the user's device.

[1079] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[1080] Specific examples

[1081] Examples of using business books

[1082] scenario:

[1083] If a user wants to refer to a business book on "strategies for new business development," the user is in a stressful state.

[1084] Example prompt:

[1085] "What is the most important step in new business development?"

[1086] Operation:

[1087] Users access the platform and receive specific answers along with encouraging comments to help reduce stress.

[1088] Cooking recipe usage examples

[1089] scenario:

[1090] If a user wants to consult a cookbook on "how to make authentic French toast," they're in a relaxed state.

[1091] Example prompt:

[1092] "What's the secret to making authentic French toast?"

[1093] Operation:

[1094] It recognizes when the user is relaxed and adds comments highlighting the good time along with specific answers to enrich the flavor.

[1095] This system allows users to efficiently obtain information that corresponds to the appropriate emotion and utilize it in a more practical way.

[1096] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1097] Step 1:

[1098] Server: Scan analog books and handwritten documents with a scanner to obtain scanned images.

[1099] Specific operation: Place a book page into the scanner and press the scan button to obtain a high-resolution image.

[1100] Input: Analog books and handwritten documents.

[1101] Output: High resolution scanned image.

[1102] Step 2:

[1103] Server: Input the scanned image into an OCR engine to extract text data. For example, we use Tesseract or Google Cloud Vision.

[1104] What it does: The server feeds the scanned image to the OCR software, and the OCR engine extracts the characters in the image as text.

[1105] Input: High resolution scanned image.

[1106] Output: The extracted raw text data.

[1107] Step 3:

[1108] Server: Cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces.

[1109] Specific behavior: The server analyzes the text data using text analysis software (e.g., nltk) and makes corrections to ensure data consistency.

[1110] Input: Extracted raw text data.

[1111] Output: Cleaned text data.

[1112] Step 4:

[1113] Server: The cleaned text data is structured and stored in a database or CSV format.

[1114] What it does: Use the data format conversion tool to convert and save the cleaned text into CSV or database format.

[1115] Input: Cleaned text data.

[1116] Output: Structured data.

[1117] Step 5:

[1118] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data.

[1119] What it does: The server feeds training data to an AI model framework (e.g., TensorFlow or PyTorch) and updates the model parameters every epoch.

[1120] Input: Structured data.

[1121] Output: A trained generative AI model.

[1122] Step 6:

[1123] Server: Evaluates the performance of the trained model using the test dataset, and retunes model parameters if necessary.

[1124] How it works: The server feeds test data to the generative AI model, compares the generated output with the original output, and evaluates the error rate and accuracy. It adjusts hyperparameters as needed.

[1125] Input: A trained generative AI model and a test dataset.

[1126] Output: An evaluated and tuned generative AI model.

[1127] Step 7:

[1128] Server: The completed generative AI model is deployed to a cloud server and made accessible on an online platform.

[1129] Specific operation: The server uploads the generated AI model to a cloud service (e.g., AWS) and exposes an API, allowing users to access the generated AI model.

[1130] Input: An evaluated and tuned generative AI model.

[1131] Output: A generative AI model accessible on the cloud.

[1132] Step 8:

[1133] Server: Develop the user interface and implement the book search function and question input form. Use web development tools (e.g. React, Angular).

[1134] What it does: Front-end developers design the user interface, and back-end developers work with APIs to implement the functionality.

[1135] Input: Design specifications and design requirements.

[1136] Output: User interface.

[1137] Step 9:

[1138] User device: The user opens a web browser, accesses the platform, and logs in.

[1139] Specific actions: Enter a URL in your browser, enter the required information in the login form that appears, and click the login button.

[1140] Input: User login information.

[1141] Output: Access to the platform.

[1142] Step 10:

[1143] User device: To find the book they want to read, users enter keywords or titles into the search bar.

[1144] What happens: A user enters keywords into the search bar and clicks the search button, which sends a search query to the server.

[1145] Input: Search keyword or title.

[1146] Output: The search query.

[1147] Step 11:

[1148] Server: Based on the search query, the server generates a list of relevant books from its database and displays it in the user's browser.

[1149] What happens: The server executes a database query to retrieve related book information and returns it to the browser.

[1150] Input: Your search query.

[1151] Output: A list of related books.

[1152] Step 12:

[1153] User device: The user selects a particular book from the displayed list to access its detail page.

[1154] What happens: The user clicks on the book they want to read and sees more information about it.

[1155] Input: Book selection.

[1156] Output: Book detail page.

[1157] Step 13:

[1158] User device: On the book detail page, the user enters a question for the generative AI model into a text box.

[1159] What happens: The user types a question into the text box and clicks the submit button.

[1160] Input: The user's question.

[1161] Output: The question query.

[1162] Step 14:

[1163] Server: The server receives the question and analyzes the user's emotional state using an emotion engine.

[1164] Specific operation: The server passes the question data to the emotion engine and receives the analysis results.

[1165] Input: The user's question.

[1166] Output: Emotional state information.

[1167] Step 15:

[1168] Server: Sends questions to the generative AI model based on emotion information and reflects response parameters according to the emotion.

[1169] How it works: The server provides the question and emotion parameters to the generative AI model, which then generates a response.

[1170] Input: Question query and emotional state information.

[1171] Output: The answer according to the user.

[1172] Step 16:

[1173] Server: Sends the generated answer to the user's device.

[1174] What happens: The server generates the answer and returns it to the user's browser.

[1175] Input: The answer given to the user.

[1176] Output: The answer text.

[1177] Step 17:

[1178] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[1179] What it does: The user reads the displayed answer and decides whether to enter a follow-up question.

[1180] Input: The displayed answer.

[1181] Output: Change in the user's emotional state and follow-up questions.

[1182] (Application example 2)

[1183] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1184] Conventional digital content delivery systems provide uniform information without considering the user's emotional state, resulting in a limited user experience and a lack of means to provide information optimally tailored to the user's current emotional state. As a result, user satisfaction and engagement may be low. To solve these issues, a system is needed that recognizes user emotions in real time and provides personalized content using generative AI models accordingly.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1186] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple books that have been modeled by the generative AI online, means for providing an interface through which a user inputs questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, emotion analysis means for recognizing the user's emotions, and means for adjusting the content of the generative AI model's answers based on the recognized emotion information. This makes it possible to provide optimal information taking into account the user's emotional state.

[1187] "Means for digitizing analog information in Japanese" refers to scanning analog information such as paper books and handwritten documents and extracting text data using optical character recognition.

[1188] "Means for training a generative AI model based on digitized text data" refers to means for using extracted text data to adjust the parameters of a generative AI model and train the generative AI model.

[1189] "Means for providing multiple generative AI modeled books online" refers to means for distributing text data and book data generated using generative AI models through an online platform.

[1190] "Means for providing an interface for users to input questions to a generative AI model" refers to means for providing an interface, such as a web browser or application screen, for users to input questions to a generative AI model.

[1191] "Means by which a generative AI model generates an answer to a user's question" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[1192] "Means for displaying the generated answer to the user" refers to means for displaying the answer generated by the generative AI model on the user's screen.

[1193] "Emotion analysis means for recognizing user emotions" refers to a means for analyzing a user's emotional state from their facial expressions and voice using sensors such as a camera and microphone.

[1194] "Means for adjusting the content of the response of the generative AI model based on recognized emotional information" refers to means for adjusting the content and tone of the response generated by the generative AI model according to the emotional state of the user.

[1195] "Means for cleaning digitized text data" refers to means for generating clean text data by correcting misrecognized characters, deleting unnecessary line breaks and spaces, etc.

[1196] The "means for structuring the cleaned text data" refers to a means for storing the cleaned text data in a structured format such as a database or CSV format.

[1197] "Means for recommending personalized video content based on user emotions" refers to means for analyzing a user's emotional state and recommending and providing appropriate video content based on that state.

[1198] A "means for users to search for specific books" is a means that provides a search bar or filter functionality that allows users to find specific books.

[1199] A "means for displaying search results to a user" is a means for displaying on a user's screen a list of books found based on the search query.

[1200] "Means for monitoring changes in the user's emotions" refers to means for tracking and recording changes in the user's emotional state in real time using a camera or microphone.

[1201] "Means for adaptively changing content recommendations based on monitoring results" refers to means for detecting changes in a user's emotions and dynamically changing the content provided accordingly.

[1202] System construction

[1203] To implement this invention, a server and a user terminal are required. The server digitizes analog information, trains the generative AI model, and provides books online. The user terminal provides an interface for users to input questions and perform searches, and displays generated answers.

[1204] Building the dataset

[1205] server:

[1206] The server first obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are then fed into an optical character recognition (OCR) engine to extract text data. Specific software used at this stage is, for example, Tesseract OCR.

[1207] server:

[1208] The extracted text data is cleaned, misrecognized characters are corrected, and unnecessary line breaks and spaces are deleted. The cleaned text data is then structured and saved in a database or CSV format, for example.

[1209] Creating generative AI models

[1210] server:

[1211] A generative AI model (e.g., GPT-3) is trained on the cleaned text data. The training process takes place over multiple epochs (iterations).

[1212] server:

[1213] Once the model has been trained and its performance evaluated, the generative AI model is deployed and made accessible to users.

[1214] User Interface and Emotion Engine

[1215] User device:

[1216] Users access the provided platform URL using a smartphone or head-mounted display, and search for the book they want to read by entering keywords or the title of the book in the search bar.

[1217] server:

[1218] It receives a search query, generates a list of relevant books from its database, and displays them to the user.

[1219] User device:

[1220] A user visits the detail page of a particular book and enters a question into the generative AI model.

[1221] Ask questions, recognize emotions and generate answers

[1222] User device:

[1223] When a user types a question, the device's camera recognizes the user's facial expression, and the emotion engine analyzes the user's emotional state. For example, if the user is feeling stressed, it will recognize that emotion.

[1224] server:

[1225] Based on the analysis results of the emotion engine, emotional information is sent to the generative AI model. Parameters for adjusting the response content according to the user's emotions are also sent. Based on this, the generative AI model generates a response that takes the user's emotions into consideration.

[1226] server:

[1227] The generated answers are sent to the user's device, where the user can check the answers displayed on the screen. Changes in the user's emotions are continuously monitored, and the answers are updated as needed.

[1228] Specific examples

[1229] scenario:

[1230] Suppose a student wants to "relax" and wants to watch a relaxing music video. If the emotion engine recognizes the user's sad emotion, the generative AI model will recommend a video.

[1231] Example prompt sentence:

[1232] "Please recommend video content that is sad and suitable for relaxation."

[1233] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1234] Step 1:

[1235] The server digitizes analog information

[1236] Input: Scanned image of an analog book

[1237] The server acquires scanned images of analog books and extracts text data using an optical character recognition (OCR) engine. Specifically, the scanned images are input into an OCR engine (e.g., Tesseract OCR) for character recognition. The output is text data.

[1238] Step 2:

[1239] The server cleans the digitized text data.

[1240] Input: Text data extracted by OCR

[1241] The server cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces. Specifically, the script analyzes the text data and automatically corrects misrecognized characters and unnecessary spaces. The output is clean text data.

[1242] Step 3:

[1243] The server trains the generative AI model

[1244] Input: clean text data

[1245] The server trains a generative AI model (e.g., GPT-3) based on clean text data. Specifically, it takes the text data as training data for the model and adjusts the model's parameters. Training is performed over multiple epochs (iterations). The output is a trained generative AI model.

[1246] Step 4:

[1247] The server deploys the generative AI model

[1248] Input: A trained generative AI model

[1249] The server evaluates the trained generative AI model, confirms its performance, and then deploys it to the online platform. Specifically, it uses an evaluation dataset to verify the model's performance and adjusts it for optimal performance. The output is a generative AI model that can be used on the online platform.

[1250] Step 5:

[1251] The user terminal provides the interface

[1252] Input: User access request

[1253] The user device accesses the provided platform URL via a smartphone or head-mounted display. Specifically, the user enters keywords or book titles in the search bar, and an interface for searching for the book they want to read is displayed. The output is the interface displayed to the user.

[1254] Step 6:

[1255] The server searches for books and displays the results

[1256] Input: User's search query

[1257] The server receives a search query, generates a list of relevant books from the database, and displays it to the user. Specifically, it performs a database search based on the query and sends the results to the user's device. The output is a list of search results.

[1258] Step 7:

[1259] The user's device inputs questions and analyzes emotions.

[1260] Input: User's question and facial expression

[1261] A user accesses the details page of a specific book and enters a question. At the same time, the device's camera recognizes the user's facial expressions, and the emotion engine analyzes their emotional state. Specifically, the camera captures the user's facial expression data, and an emotion analysis algorithm is used to identify their emotional state. The output is the user's question and emotional state.

[1262] Step 8:

[1263] The server generates an answer based on the sentiment

[1264] Input: User's question and emotional state

[1265] The server sends emotional information to the generative AI model based on the analysis results of the emotion engine. It also sends parameters to adjust the answer content according to the user's emotions. Specifically, the generative AI model generates an answer based on the question and emotional information. The output is an answer that takes emotions into consideration.

[1266] Step 9:

[1267] The server sends the answer to the user's device.

[1268] Input: Generated Answer

[1269] The server sends the generated answer to the user terminal. Specifically, the generated answer data is sent to the user terminal and displayed on the user's screen. The output is the answer displayed to the user.

[1270] Step 10:

[1271] User devices monitor emotional changes

[1272] Input: Continuous facial expression data of the user

[1273] While the user is checking the answers displayed on the screen, the emotion engine continuously monitors the user's emotional changes. Specifically, the device's camera continuously captures facial expression data and performs emotion analysis in real time. The output is the analysis results of emotional changes.

[1274] Step 11:

[1275] The server adaptively updates content recommendations

[1276] Input: Emotion change analysis results

[1277] The server dynamically adjusts the content provided based on the monitoring results. Specifically, it retrieves new content from the generative AI model that reflects the analysis results of emotion changes and provides it to the user. The output is the updated content.

[1278] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1279] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1280] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1281] [Third embodiment]

[1282] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1283] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1284] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1285] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1286] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1287] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1288] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1289] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1290] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1291] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1292] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1293] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1294] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Specific embodiments of this system are described below.

[1295] Building the dataset

[1296] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[1297] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is structured and stored in a database.

[1298] Creating generative AI models

[1299] Server: Builds a generative AI model that learns the content of specific books based on the cleaned text data. Trains the generative AI model based on the text taken as training data and adjusts the parameters accordingly.

[1300] Server: Deploys the trained generative AI model and makes it available to users online.

[1301] Building an online platform

[1302] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[1303] User Interface and Operation

[1304] Book Selection and Questions

[1305] User device: The user opens a web browser, accesses the provided platform URL, enters keywords in the search bar, and searches for the book they want to read.

[1306] Server: Receives the search query, generates a list of relevant books, and displays them to the user.

[1307] User device: The user selects a particular book from the displayed list to access its detail page.

[1308] Ask a question and get an answer

[1309] User device: On the details page of the selected book, the user enters a question into the text box that asks the generative AI model a question.

[1310] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[1311] Server: The generative AI model analyzes the question and generates an appropriate answer.

[1312] Server: Sends the generated answer to the user's device.

[1313] User device: The user checks the answers displayed on the screen and asks further questions if necessary.

[1314] Specific examples

[1315] Example 1: Using business books

[1316] scenario:

[1317] Let's say a user wants to refer to a specific business book on "strategies for new business development."

[1318] User device: Access the platform and search for "New Business Development".

[1319] Server: Displays a list of relevant business books as search results.

[1320] User Device: Select a specific business book and go to its details page.

[1321] User device: Type the question, "What is the most important step in new business development?"

[1322] Server: Sends questions to the generative AI model and generates answers.

[1323] User device: Receive specific answers such as, "Imaginative market analysis and targeting are the most important."

[1324] Example 2: Using cooking recipes

[1325] scenario:

[1326] Let's say a user wants to consult a cookbook on how to make authentic French toast.

[1327] User device: Accesses the platform and searches for "french toast."

[1328] Server: Displays a list of relevant cookbooks as search results.

[1329] User Device: Select a specific cookbook and go to its detail page.

[1330] User device: Type in the question, "What's the secret to making authentic French toast?"

[1331] Server: Sends questions to the generative AI model and generates answers.

[1332] User device: Receives a response such as "Adding a little vanilla extract will give it an even richer flavor."

[1333] The above is a specific embodiment for carrying out the present invention. This system enables users to effectively obtain information and put it to practical use.

[1334] The processing flow will be explained below.

[1335] Step 1:

[1336] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[1337] Step 2:

[1338] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[1339] Step 3:

[1340] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[1341] Step 4:

[1342] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[1343] Step 5:

[1344] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[1345] Step 6:

[1346] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[1347] Step 7:

[1348] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[1349] Step 8:

[1350] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[1351] Step 9:

[1352] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[1353] Step 10:

[1354] User device: The user selects a particular book from the displayed list to access its detail page.

[1355] Step 11:

[1356] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[1357] Step 12:

[1358] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[1359] Step 13:

[1360] Server: The generative AI model analyzes the question and generates an appropriate answer.

[1361] Step 14:

[1362] Server: Sends the generated answer to the user's device.

[1363] Step 15:

[1364] User device: The user reviews the answers displayed on the screen and asks further questions if necessary.

[1365] Example 1

[1366] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1367] Previously, digitizing and utilizing analog Japanese information has faced challenges such as low-accuracy OCR technology, inefficient data cleaning methods, and a lack of user-friendly interfaces. Furthermore, building an entire system to train generative AI models and provide appropriate information to users has been complex. There is a need to resolve these issues and provide high-quality digital information while enabling users to access it efficiently.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1369] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI models online, means for providing an interface for a user to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for cleaning the digitized text data, means for structuring the cleaned text data, means for a user to search for specific data, and means for displaying search results to the user. This makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and effectively access the information they need.

[1370] "Analog information in Japanese" refers to information written in Japanese that is not stored as electronic data, such as paper books and handwritten documents.

[1371] "Digitizing" means converting analog information into an electronic data format that can be processed by a computer.

[1372] "Text data" refers to data that electronically records information such as letters, sentences, and symbols.

[1373] A "generative AI model" refers to an artificial intelligence model that can learn from text data and perform specific tasks.

[1374] "Training" means using large amounts of data to learn from an AI model and improve its performance.

[1375] "Providing online" means making a service or function available to users via the Internet.

[1376] "Interface" refers to the screens and input means through which users interact with a system.

[1377] "Cleaning" refers to the process of correcting misrecognized parts from text data and removing unnecessary spaces and line breaks.

[1378] "Structuring" means organizing data into a certain format or structure so that it can be easily searched and used.

[1379] "Deploying" means placing a trained AI model in an actual production environment and making it available for use.

[1380] "Database" refers to an electronic system for efficiently storing, managing, and retrieving data.

[1381] "Searching" means finding data based on specific keywords or conditions.

[1382] "Results" refers to the final information or data obtained through operations such as search or calculation.

[1383] An "API request" is a request made by an application to another system to use a function or obtain information.

[1384] "REST API" refers to an API based on uniform design principles for exchanging data between web services.

[1385] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model, and provides it to users. Detailed embodiments for implementing this system are described below.

[1386] Building the dataset

[1387] server:

[1388] To digitize analog information, first, paper books or handwritten documents are scanned to obtain image data. The hardware used is, for example, the common scanner "Epson Perfection V600." This scanned image is then input into the OCR engine "Tesseract OCR," and text data is extracted.

[1389] server:

[1390] Next, the extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. A Python script is used for cleaning. The cleaned text data is then stored in a MySQL database. The stored data is then structured so that it can be used for later training and search.

[1391] Creating generative AI models

[1392] server:

[1393] A generative AI model is trained on the cleaned text data using a machine learning framework such as TensorFlow or PyTorch, which inputs training data and adjusts certain parameters to improve the model's accuracy.

[1394] server:

[1395] Once trained, the generative AI model is deployed in a Docker container, making it operational and enabling it to quickly provide answers to user queries.

[1396] Building an online platform

[1397] server:

[1398] The user interface will be developed using ReactJS and Node.js. This interface allows users to operate it intuitively. The interface will include a book search function, an access interface to the generative AI model, user profile management, and the ability to add links to purchase books. The search function will use Elasticsearch to quickly extract relevant information from large amounts of data.

[1399] User interface and operation examples

[1400] Book Selection and Questions

[1401] User device:

[1402] A user opens a web browser, visits the provided platform URL (e.g., www.examplebooks.com), and enters keywords into the search bar to search for a specific book (e.g., "new business development" or "French toast").

[1403] server:

[1404] It receives a search query, uses Elasticsearch to generate a list of relevant books, and displays them to the user.

[1405] User device:

[1406] The user selects a particular book from the displayed list to access its detail page.

[1407] Ask a question and get an answer

[1408] User device:

[1409] Go to the details page and type your question into the text box (e.g., "What is the most important step in new business development?").

[1410] server:

[1411] The entered question is sent to the generative AI model using the REST API.

[1412] server:

[1413] A generative AI model analyzes the question and generates an appropriate answer, which is composed in natural language based on the model's training data.

[1414] server:

[1415] The generated answer is sent back to the user's device and displayed in real time.

[1416] User device:

[1417] Users review the answers displayed on the screen (e.g., "Imaginative market analysis and targeting are paramount") and ask further questions if necessary.

[1418] This invention makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and efficiently access the information they need.

[1419] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1420] Specific explanation of program processing

[1421] Step 1: Digitizing analog information

[1422] server:

[1423] Input: Analog information such as paper books and handwritten documents.

[1424] Specific operation: Analog information is scanned using a scanner. The hardware used is the "Epson Perfection V600."

[1425] Output: Obtain a high-resolution scanned image.

[1426] Step 2: Extract text using OCR

[1427] server:

[1428] Input: Scanned image data.

[1429] Specific operation: The acquired scanned image is input into the OCR engine "Tesseract OCR" and text data is extracted from the image.

[1430] Call the Tesseract API specifying the path to the image data provided by the client or the database.

[1431] Output: The extracted text data.

[1432] Step 3: Cleaning the text data

[1433] server:

[1434] Input: Text data extracted by OCR.

[1435] What it does: Uses a Python script to clean text, including correcting misrecognized characters and removing unnecessary line breaks and spaces.

[1436] Output: Cleaned text data.

[1437] Step 4: Structuring and storing text data

[1438] server:

[1439] Input: Cleaned text data.

[1440] What it does: It uses a Python script to structure text data into the required format, then stores it in a MySQL database, which also contains metadata such as book title, author, and content.

[1441] Output: Structured text data stored in a database.

[1442] Step 5: Training the generative AI model

[1443] server:

[1444] Input: Cleaned text data stored in a database.

[1445] How it works: Use TensorFlow or PyTorch to train a generative AI model. Input training data into the AI ​​model and adjust parameters such as the number of epochs, batch size, and learning rate.

[1446] Output: A fully trained generative AI model.

[1447] Step 6: Deploying the generative AI model

[1448] server:

[1449] Input: A fully trained generative AI model.

[1450] What it does: Deploy an AI model in a Docker container. The containerized model runs on a server and is accessible externally through an API.

[1451] Output: A usable generative AI model is deployed in a production environment.

[1452] Step 7: Developing the User Interface

[1453] server:

[1454] Input: None (early development stage).

[1455] Specific operation: Using "ReactJS" and "Node.js", a user interface will be developed, which will include a book search function, an access interface to the generative AI model, user profile management, and a link to purchase the book.

[1456] Output: A web platform that users can access.

[1457] Step 8: Book Search

[1458] User device:

[1459] Type: Enter a keyword into the search bar (e.g., "new business development" or "french toast").

[1460] Specific actions: Open a web browser, access the provided platform URL, and enter keywords in the search bar.

[1461] server:

[1462] Input: The user's search query.

[1463] Specific operation: Uses Elasticsearch to generate a list of relevant books based on the search query and displays it to the user.

[1464] Output: Shows a list of books relevant to the user.

[1465] Step 9: Select a book and enter your question

[1466] User device:

[1467] Input: Select a specific book from the displayed book list and go to its details page.

[1468] What happens: Visit a book's detail page and enter a question (e.g., "What is the most important step in new business development?").

[1469] Output: The question entered.

[1470] Step 10: Submitting questions and generating answers

[1471] server:

[1472] Input: The user's question.

[1473] How it works: Send a question to the generative AI model via the REST API. The generative AI model analyzes the question and generates an appropriate answer.

[1474] Output: The generated answer.

[1475] Step 11: View your answers

[1476] server:

[1477] Input: The generated answer.

[1478] Specific operation: The generated answer is sent back to the user's device and displayed in real time.

[1479] User device:

[1480] Input: None (display stage).

[1481] Output: The user sees the answer displayed on the screen (e.g., "Imaginative market analysis and targeting are most important.").

[1482] Step 12: Additional Questions

[1483] User device:

[1484] Input: Additional questions (if needed).

[1485] What it does: Review the answers displayed on the screen, enter additional questions if necessary, and submit.

[1486] Output: Additional questions are sent to the generative AI model.

[1487] The above are the specific processing steps of the program of this system.

[1488] (Application example 1)

[1489] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1490] Conventional search systems and AI models have the drawback of making it difficult for users to freely ask questions about books and materials and obtain appropriate answers. It is particularly difficult to ask detailed questions about the contents of each book and obtain immediate answers for books and materials that are widely available in bookstores and libraries. Furthermore, there is a lack of systems that can efficiently digitize vast amounts of analog information, build AI models based on that information, and make them useful to users, making this initiative necessary.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1492] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI modeled materials online, means for providing an interface for users to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for searching for books and selecting specific materials, and means for the generative AI model to analyze questions related to the materials and generate answers. This allows users to freely ask questions about books and materials available in bookstores and libraries and receive appropriate answers in real time.

[1493] "Analog information in Japanese" refers to materials written in Japanese that have not been digitized, such as paper books and handwritten documents.

[1494] "Digital text data" refers to data that has been converted from analog Japanese information into digital text using scanning or optical character recognition (OCR) technology.

[1495] A "generative AI model" is an artificial intelligence model that uses digitized text data as training data to generate appropriate answers to questions.

[1496] "Means for providing online" means means for placing generative AI-modeled materials on a web server or cloud service so that they can be accessed by users via the internet.

[1497] The "interface for inputting questions" refers to a user interface that allows users to input questions in text format to the generative AI model.

[1498] The "means of generating an answer" refers to the process by which the generative AI model analyzes the user's question and generates an appropriate answer.

[1499] "Means for searching and selecting specific materials" means the ability for users to search for materials using specific keywords and select relevant materials.

[1500] "Means for the generative AI model to analyze questions related to the materials and generate answers" refers to the process in which the generative AI model analyzes questions asked by users about selected materials and generates answers based on those materials.

[1501] "Smartphone Application" means a specific software application for installation and use on a smartphone device.

[1502] This invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Detailed embodiments for implementing this system will be described.

[1503] System Program

[1504] 1. Hardware and software configuration

[1505] Hardware: Smartphones (iOS devices, Android devices), servers

[1506] Software: Python, Flask (for API server), TensorFlow (generative AI model), Tesseract (OCR engine), SQLite (database), React Native (smartphone app development)

[1507] 2. Digitizing data and training generated AI models

[1508] The server scans paper books and handwritten documents available at bookstores and libraries and extracts Japanese text data using the Tesseract OCR engine. The extracted text data is then cleaned, correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is then structured and stored in an SQLite database.

[1509] The server trains a generative AI model using TensorFlow based on the stored cleaned text data, and once trained, the generative AI model is deployed on the Flask API server.

[1510] 3. User interface development and operation

[1511] We developed a smartphone application using React Native, implementing a book search function and a question and answer interface. Through the app, users can find books by keyword search, ask questions about the books, and receive answers in real time.

[1512] Program processing

[1513] 1. Scanning the book and OCR processing

[1514] The server takes scanned images of books provided by bookstores and libraries and extracts the text data using the Tesseract OCR engine.

[1515] The extracted text data is cleaned by the server and stored in an SQLite database.

[1516] 2. Training the generative AI model

[1517] The server uses the cleaned text data as training data to train a generative AI model using TensorFlow.

[1518] Once trained, the generative AI model is deployed on a Flask API server and made accessible online.

[1519] 3. User Interface Operation

[1520] The smartphone application allows users to search for books by entering keywords and select specific books from the search results.

[1521] Users can enter questions into the generative AI model on the book details page and receive real-time answers from the server.

[1522] Examples and prompts

[1523] Examples:

[1524] A user types a question about a history book: "What accomplishments did a particular person achieve?"

[1525] The generative AI model generates and displays the answer, "A specific person achieved numerous victories during the Warring States period, promoted centralized policies, and unified the country."

[1526] Example prompt sentence:

[1527] "Enter a question about a specific person (e.g., 'What accomplishments did a specific person have?')"

[1528] This system allows users to ask any questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[1529] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1530] Step 1:

[1531] The server receives scanned images of books provided by bookstores and libraries. The scanned images are input and converted into text data using Tesseract, an optical character recognition (OCR) engine. The output is OCR-processed text data.

[1532] Step 2:

[1533] The server cleans the OCR-processed text data. Specifically, it corrects misrecognized characters and deletes unnecessary line breaks and spaces. The input is the OCR-processed text data, and the output is the cleaned text data.

[1534] Step 3:

[1535] The server structures the cleaned text data and stores it in an SQLite database. The input is the cleaned text data, and the output is the structured text data stored in the database.

[1536] Step 4:

[1537] The server trains a generative AI model using the stored structured text data. It uses TensorFlow to build the AI ​​model and adjusts its parameters based on the training data. The input is the structured text data, and the output is a trained generative AI model.

[1538] Step 5:

[1539] The server deploys the trained generative AI model on the Flask API server, making the model accessible online. The input is the trained generative AI model, and the output is the deployed generative AI model.

[1540] Step 6:

[1541] Users open the smartphone application and search for books by entering keywords through an interface built with React Native. The input is the search query, and the output is a search result list of matching books.

[1542] Step 7:

[1543] The user selects a particular book from the search results and navigates to that book's detail page. The input is the selection of a particular book, and the output is the display of the book's detail page.

[1544] Step 8:

[1545] The user enters a question on the details page and sends it to the generative AI model. The input is the user's question, and the output is an API request to the generative AI model.

[1546] Step 9:

[1547] The server analyzes the user's question against the generative AI model and generates an appropriate answer. The input is the user's question, and the output is the generated answer.

[1548] Step 10:

[1549] The server sends the generated answer to the user's smartphone, where it is displayed on the user's device. The input is the generated answer, and the output is the answer displayed on the user's device.

[1550] This processing flow allows users to freely ask questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[1551] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1552] This invention combines a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[1553] Building the dataset

[1554] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[1555] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters, removing unnecessary line breaks and spaces, and standardizing the format. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[1556] Creating generative AI models

[1557] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[1558] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[1559] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[1560] Building an online platform

[1561] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[1562] User Interface and Emotion Engine

[1563] Book Selection and Emotion Recognition

[1564] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[1565] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[1566] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[1567] User device: The user selects a particular book from the displayed list to access its detail page.

[1568] Ask questions, recognize emotions and generate answers

[1569] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[1570] Server: Receives the input question and uses the emotion engine to analyze the user's current emotional state. For example, if the user is feeling stressed or anxious, it recognizes that emotion.

[1571] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[1572] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[1573] Server: Sends the generated answer to the user's device.

[1574] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[1575] Specific examples

[1576] Example 1: Using business books

[1577] scenario:

[1578] Let's say the user wants to refer to a specific business book about "strategies for new business development." The user is in a stressful state.

[1579] User device: Access the platform and search for "New Business Development".

[1580] Server: Displays a list of relevant business books as search results.

[1581] User Device: Select a specific business book and go to its details page.

[1582] User device: Type the question, "What is the most important step in new business development?"

[1583] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[1584] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[1585] User device: The user receives the answer and feels relief along with concrete steps.

[1586] Example 2: Using cooking recipes

[1587] scenario:

[1588] Let's say the user wants to consult a cookbook on "how to make authentic French toast." The user is relaxed.

[1589] User device: Accesses the platform and searches for "french toast."

[1590] Server: Displays a list of relevant cookbooks as search results.

[1591] User Device: Select a specific cookbook and go to its detail page.

[1592] User device: Type in the question, "What's the secret to making authentic French toast?"

[1593] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[1594] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[1595] User terminal: Users can receive answers comfortably and enjoy cooking.

[1596] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[1597] The processing flow will be explained below.

[1598] Step 1:

[1599] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[1600] Step 2:

[1601] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[1602] Step 3:

[1603] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[1604] Step 4:

[1605] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[1606] Step 5:

[1607] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[1608] Step 6:

[1609] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[1610] Step 7:

[1611] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[1612] Step 8:

[1613] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[1614] Step 9:

[1615] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[1616] Step 10:

[1617] User device: The user selects a particular book from the displayed list to access its detail page.

[1618] Step 11:

[1619] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[1620] Step 12:

[1621] Server: Receives the input question and uses an emotion engine to analyze the user's current emotional state, for example, by inferring emotions from the context of the input text and the user's past behavior.

[1622] Step 13:

[1623] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[1624] Step 14:

[1625] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[1626] Step 15:

[1627] Server: Sends the generated answer to the user's device.

[1628] Step 16:

[1629] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[1630] Specific examples

[1631] Example 1: Using business books

[1632] Step 1:

[1633] User device: The user wants to consult a specific business book on "New Business Development Strategies." The user is in a stressful state.

[1634] Step 2:

[1635] User device: Access the platform and search for "New Business Development".

[1636] Step 3:

[1637] Server: Displays a list of relevant business books as search results.

[1638] Step 4:

[1639] User Device: Select a specific business book and go to its details page.

[1640] Step 5:

[1641] User device: Type the question, "What is the most important step in new business development?"

[1642] Step 6:

[1643] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[1644] Step 7:

[1645] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[1646] Step 8:

[1647] User device: The user receives the answer and feels relief along with concrete steps.

[1648] Example 2: Using cooking recipes

[1649] Step 1:

[1650] User device: The user wants to consult a cookbook on "how to make authentic French toast." The user is in a relaxed state.

[1651] Step 2:

[1652] User device: Accesses the platform and searches for "french toast."

[1653] Step 3:

[1654] Server: Displays a list of relevant cookbooks as search results.

[1655] Step 4:

[1656] User Device: Select a specific cookbook and go to its detail page.

[1657] Step 5:

[1658] User device: Type in the question, "What's the secret to making authentic French toast?"

[1659] Step 6:

[1660] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[1661] Step 7:

[1662] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[1663] Step 8:

[1664] User terminal: Users can receive answers comfortably and enjoy cooking.

[1665] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[1666] Example 2

[1667] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1668] Systems that use generative AI models to help users efficiently acquire information in Japanese need to provide appropriate answers based on the user's emotional state. However, conventional systems generate uniform answers without considering the user's emotional state, making it difficult to fully meet the user's needs. Furthermore, if the digitized text data is not effectively cleaned and structured, and the parameters of the generative AI model are not adjusted effectively, the accuracy of the model decreases.

[1669] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1670] In this invention, the server includes a means for digitizing Japanese analog information, a means for training a generative AI model based on the digitized text data, and a means for analyzing the user's emotional state and reflecting the analysis results in the generative AI model. This makes it possible to provide answers optimized according to the user's emotional state. Furthermore, high accuracy of answers can be maintained through the process of cleaning and structuring the digitized text data and training the generative AI model.

[1671] "Analog information in Japanese" refers to Japanese information that exists in physical form, rather than digital form, such as paper books or handwritten documents.

[1672] "Digitalization" refers to the conversion of analog information into electronic data in a format that can be processed by a computer.

[1673] "Generative AI model" refers to an artificial intelligence model that has been trained using machine learning techniques to perform a specific task.

[1674] "Training" is the process of providing learning data to a generative AI model to optimize the model's parameters and improve its performance for a specific task.

[1675] "Provided online" means that the service is provided in a form that allows users to access it via the Internet.

[1676] "Interface" refers to the means or operating screen through which a user interacts with a system.

[1677] "Emotional state" refers to the user's current emotional state, such as stressed, relaxed, or anxious.

[1678] "Cleaning" is the process of correcting misrecognized characters in digitized text data and deleting unnecessary data to improve the quality of the data.

[1679] "Structuring" refers to organizing and arranging cleaned text data into a specific format or structure.

[1680] "Parameter tuning" is the process of setting and updating the learning parameters within a generative AI model so that it performs optimally.

[1681] "Search" is the act of locating relevant information or data from a database based on keywords entered by the user.

[1682] "Personalized information provision" refers to providing users with the most appropriate information based on their profile and search history.

[1683] This invention is a system that digitizes analog information in Japanese and trains a generative AI model based on that information, recognizing the user's emotional state and providing the optimal response according to that state. Specific embodiments are described below.

[1684] Building the dataset

[1685] Server: Scan analog books and handwritten documents using a scanner and save the scanned images on the server. We recommend using a high-resolution scanner.

[1686] Server: Input the scanned image into an optical character recognition (OCR) engine to extract the text data. Typically, an OCR engine such as Tesseract or Google Cloud Vision is used.

[1687] Server: Cleans the extracted text data, corrects misrecognized characters, and deletes unnecessary line breaks and spaces. A natural language processing library (e.g., nltk) is used as text analysis software.

[1688] Server: The clean text data is structured and stored in a database or CSV format for easy subsequent processing.

[1689] Creating generative AI models

[1690] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data. The training process uses machine learning frameworks such as TensorFlow and PyTorch.

[1691] Server: Once training is complete, evaluate the model's performance using the test dataset and tune the model's parameters for optimal performance.

[1692] Server: Deploy the completed generative AI model and make it accessible to users on an online platform. Upload the model to a cloud server (e.g., AWS) and expose the API.

[1693] Building the user interface

[1694] Server: Develop the user interface using web development tools (e.g., React, Angular), implement book search functionality and a question input form, and integrate user profile management and book purchase links.

[1695] Book Selection and Emotion Recognition

[1696] User terminal: The user opens a web browser and accesses the provided platform URL. The user registers and logs in to the platform.

[1697] User device: Users enter keywords or book titles in the search bar to search for the books they want to read.

[1698] Server: The server receives the search query, generates a list of relevant books from the database, and displays it in the browser.

[1699] User device: The user selects a particular book from the displayed list to access its detail page.

[1700] Ask questions, recognize emotions and generate answers

[1701] User device: On the book details page, the user enters a question into the text box provided to the generative AI model.

[1702] Server: The server receives the input question and uses an emotion engine to analyze the user's emotional state. For example, it identifies if the user is feeling stressed or anxious.

[1703] Server: Sends questions to the generative AI model based on the recognized emotional state and provides response parameters according to the emotion to the model.

[1704] Server: A generative AI model analyzes the question and generates the best answer, including adjustments based on an emotion engine to ensure the answer takes into account the user's emotional state.

[1705] Server: Sends the generated answer to the user's device.

[1706] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[1707] Specific examples

[1708] Examples of using business books

[1709] scenario:

[1710] If a user wants to refer to a business book on "strategies for new business development," the user is in a stressful state.

[1711] Example prompt:

[1712] "What is the most important step in new business development?"

[1713] Operation:

[1714] Users access the platform and receive specific answers along with encouraging comments to help reduce stress.

[1715] Cooking recipe usage examples

[1716] scenario:

[1717] If a user wants to consult a cookbook on "how to make authentic French toast," they're in a relaxed state.

[1718] Example prompt:

[1719] "What's the secret to making authentic French toast?"

[1720] Operation:

[1721] It recognizes when the user is relaxed and adds comments highlighting the good time along with specific answers to enrich the flavor.

[1722] This system allows users to efficiently obtain information that corresponds to the appropriate emotion and utilize it in a more practical way.

[1723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1724] Step 1:

[1725] Server: Scan analog books and handwritten documents with a scanner to obtain scanned images.

[1726] Specific operation: Place a book page into the scanner and press the scan button to obtain a high-resolution image.

[1727] Input: Analog books and handwritten documents.

[1728] Output: High resolution scanned image.

[1729] Step 2:

[1730] Server: Input the scanned image into an OCR engine to extract text data. For example, we use Tesseract or Google Cloud Vision.

[1731] What it does: The server feeds the scanned image to the OCR software, and the OCR engine extracts the characters in the image as text.

[1732] Input: High resolution scanned image.

[1733] Output: The extracted raw text data.

[1734] Step 3:

[1735] Server: Cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces.

[1736] Specific behavior: The server analyzes the text data using text analysis software (e.g., nltk) and makes corrections to ensure data consistency.

[1737] Input: Extracted raw text data.

[1738] Output: Cleaned text data.

[1739] Step 4:

[1740] Server: The cleaned text data is structured and stored in a database or CSV format.

[1741] What it does: Use the data format conversion tool to convert and save the cleaned text into CSV or database format.

[1742] Input: Cleaned text data.

[1743] Output: Structured data.

[1744] Step 5:

[1745] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data.

[1746] What it does: The server feeds training data to an AI model framework (e.g., TensorFlow or PyTorch) and updates the model parameters every epoch.

[1747] Input: Structured data.

[1748] Output: A trained generative AI model.

[1749] Step 6:

[1750] Server: Evaluates the performance of the trained model using the test dataset, and retunes model parameters if necessary.

[1751] How it works: The server feeds test data to the generative AI model, compares the generated output with the original output, and evaluates the error rate and accuracy. It adjusts hyperparameters as needed.

[1752] Input: A trained generative AI model and a test dataset.

[1753] Output: An evaluated and tuned generative AI model.

[1754] Step 7:

[1755] Server: The completed generative AI model is deployed to a cloud server and made accessible on an online platform.

[1756] Specific operation: The server uploads the generated AI model to a cloud service (e.g., AWS) and exposes an API, allowing users to access the generated AI model.

[1757] Input: An evaluated and tuned generative AI model.

[1758] Output: A generative AI model accessible on the cloud.

[1759] Step 8:

[1760] Server: Develop the user interface and implement the book search function and question input form. Use web development tools (e.g. React, Angular).

[1761] What it does: Front-end developers design the user interface, and back-end developers work with APIs to implement the functionality.

[1762] Input: Design specifications and design requirements.

[1763] Output: User interface.

[1764] Step 9:

[1765] User device: The user opens a web browser, accesses the platform, and logs in.

[1766] Specific actions: Enter a URL in your browser, enter the required information in the login form that appears, and click the login button.

[1767] Input: User login information.

[1768] Output: Access to the platform.

[1769] Step 10:

[1770] User device: To find the book they want to read, users enter keywords or titles into the search bar.

[1771] What happens: A user enters keywords into the search bar and clicks the search button, which sends a search query to the server.

[1772] Input: Search keyword or title.

[1773] Output: The search query.

[1774] Step 11:

[1775] Server: Based on the search query, the server generates a list of relevant books from its database and displays it in the user's browser.

[1776] What happens: The server executes a database query to retrieve related book information and returns it to the browser.

[1777] Input: Your search query.

[1778] Output: A list of related books.

[1779] Step 12:

[1780] User device: The user selects a particular book from the displayed list to access its detail page.

[1781] What happens: The user clicks on the book they want to read and sees more information about it.

[1782] Input: Book selection.

[1783] Output: Book detail page.

[1784] Step 13:

[1785] User device: On the book detail page, the user enters a question for the generative AI model into a text box.

[1786] What happens: The user types a question into the text box and clicks the submit button.

[1787] Input: The user's question.

[1788] Output: The question query.

[1789] Step 14:

[1790] Server: The server receives the question and analyzes the user's emotional state using an emotion engine.

[1791] Specific operation: The server passes the question data to the emotion engine and receives the analysis results.

[1792] Input: The user's question.

[1793] Output: Emotional state information.

[1794] Step 15:

[1795] Server: Sends questions to the generative AI model based on emotion information and reflects response parameters according to the emotion.

[1796] How it works: The server provides the question and emotion parameters to the generative AI model, which then generates a response.

[1797] Input: Question query and emotional state information.

[1798] Output: The answer according to the user.

[1799] Step 16:

[1800] Server: Sends the generated answer to the user's device.

[1801] What happens: The server generates the answer and returns it to the user's browser.

[1802] Input: The answer given to the user.

[1803] Output: The answer text.

[1804] Step 17:

[1805] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[1806] What it does: The user reads the displayed answer and decides whether to enter a follow-up question.

[1807] Input: The displayed answer.

[1808] Output: Change in the user's emotional state and follow-up questions.

[1809] (Application example 2)

[1810] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1811] Conventional digital content delivery systems provide uniform information without considering the user's emotional state, resulting in a limited user experience and a lack of means to provide information optimally tailored to the user's current emotional state. As a result, user satisfaction and engagement may be low. To solve these issues, a system is needed that recognizes user emotions in real time and provides personalized content using generative AI models accordingly.

[1812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1813] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple books that have been modeled by the generative AI online, means for providing an interface through which a user inputs questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, emotion analysis means for recognizing the user's emotions, and means for adjusting the content of the generative AI model's answers based on the recognized emotion information. This makes it possible to provide optimal information taking into account the user's emotional state.

[1814] "Means for digitizing analog information in Japanese" refers to scanning analog information such as paper books and handwritten documents and extracting text data using optical character recognition.

[1815] "Means for training a generative AI model based on digitized text data" refers to means for using extracted text data to adjust the parameters of a generative AI model and train the generative AI model.

[1816] "Means for providing multiple generative AI modeled books online" refers to means for distributing text data and book data generated using generative AI models through an online platform.

[1817] "Means for providing an interface for users to input questions to a generative AI model" refers to means for providing an interface, such as a web browser or application screen, for users to input questions to a generative AI model.

[1818] "Means by which a generative AI model generates an answer to a user's question" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[1819] "Means for displaying the generated answer to the user" refers to means for displaying the answer generated by the generative AI model on the user's screen.

[1820] "Emotion analysis means for recognizing user emotions" refers to a means for analyzing a user's emotional state from their facial expressions and voice using sensors such as a camera and microphone.

[1821] "Means for adjusting the content of the response of the generative AI model based on recognized emotional information" refers to means for adjusting the content and tone of the response generated by the generative AI model according to the emotional state of the user.

[1822] "Means for cleaning digitized text data" refers to means for generating clean text data by correcting misrecognized characters, deleting unnecessary line breaks and spaces, etc.

[1823] The "means for structuring the cleaned text data" refers to a means for storing the cleaned text data in a structured format such as a database or CSV format.

[1824] "Means for recommending personalized video content based on user emotions" refers to means for analyzing a user's emotional state and recommending and providing appropriate video content based on that state.

[1825] A "means for users to search for specific books" is a means that provides a search bar or filter functionality that allows users to find specific books.

[1826] A "means for displaying search results to a user" is a means for displaying on a user's screen a list of books found based on the search query.

[1827] "Means for monitoring changes in the user's emotions" refers to means for tracking and recording changes in the user's emotional state in real time using a camera or microphone.

[1828] "Means for adaptively changing content recommendations based on monitoring results" refers to means for detecting changes in a user's emotions and dynamically changing the content provided accordingly.

[1829] System construction

[1830] To implement this invention, a server and a user terminal are required. The server digitizes analog information, trains the generative AI model, and provides books online. The user terminal provides an interface for users to input questions and perform searches, and displays generated answers.

[1831] Building the dataset

[1832] server:

[1833] The server first obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are then fed into an optical character recognition (OCR) engine to extract text data. Specific software used at this stage is, for example, Tesseract OCR.

[1834] server:

[1835] The extracted text data is cleaned, misrecognized characters are corrected, and unnecessary line breaks and spaces are deleted. The cleaned text data is then structured and saved in a database or CSV format, for example.

[1836] Creating generative AI models

[1837] server:

[1838] A generative AI model (e.g., GPT-3) is trained on the cleaned text data. The training process takes place over multiple epochs (iterations).

[1839] server:

[1840] Once the model has been trained and its performance evaluated, the generative AI model is deployed and made accessible to users.

[1841] User Interface and Emotion Engine

[1842] User device:

[1843] Users access the provided platform URL using a smartphone or head-mounted display, and search for the book they want to read by entering keywords or the title of the book in the search bar.

[1844] server:

[1845] It receives a search query, generates a list of relevant books from its database, and displays them to the user.

[1846] User device:

[1847] A user visits the detail page of a particular book and enters a question into the generative AI model.

[1848] Ask questions, recognize emotions and generate answers

[1849] User device:

[1850] When a user types a question, the device's camera recognizes the user's facial expression, and the emotion engine analyzes the user's emotional state. For example, if the user is feeling stressed, it will recognize that emotion.

[1851] server:

[1852] Based on the analysis results of the emotion engine, emotional information is sent to the generative AI model. Parameters for adjusting the response content according to the user's emotions are also sent. Based on this, the generative AI model generates a response that takes the user's emotions into consideration.

[1853] server:

[1854] The generated answers are sent to the user's device, where the user can check the answers displayed on the screen. Changes in the user's emotions are continuously monitored, and the answers are updated as needed.

[1855] Specific examples

[1856] scenario:

[1857] Suppose a student wants to "relax" and wants to watch a relaxing music video. If the emotion engine recognizes the user's sad emotion, the generative AI model will recommend a video.

[1858] Example prompt sentence:

[1859] "Please recommend video content that is sad and suitable for relaxation."

[1860] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1861] Step 1:

[1862] The server digitizes analog information

[1863] Input: Scanned image of an analog book

[1864] The server acquires scanned images of analog books and extracts text data using an optical character recognition (OCR) engine. Specifically, the scanned images are input into an OCR engine (e.g., Tesseract OCR) for character recognition. The output is text data.

[1865] Step 2:

[1866] The server cleans the digitized text data.

[1867] Input: Text data extracted by OCR

[1868] The server cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces. Specifically, the script analyzes the text data and automatically corrects misrecognized characters and unnecessary spaces. The output is clean text data.

[1869] Step 3:

[1870] The server trains the generative AI model

[1871] Input: clean text data

[1872] The server trains a generative AI model (e.g., GPT-3) based on clean text data. Specifically, it takes the text data as training data for the model and adjusts the model's parameters. Training is performed over multiple epochs (iterations). The output is a trained generative AI model.

[1873] Step 4:

[1874] The server deploys the generative AI model

[1875] Input: A trained generative AI model

[1876] The server evaluates the trained generative AI model, confirms its performance, and then deploys it to the online platform. Specifically, it uses an evaluation dataset to verify the model's performance and adjusts it for optimal performance. The output is a generative AI model that can be used on the online platform.

[1877] Step 5:

[1878] The user terminal provides the interface

[1879] Input: User access request

[1880] The user device accesses the provided platform URL via a smartphone or head-mounted display. Specifically, the user enters keywords or book titles in the search bar, and an interface for searching for the book they want to read is displayed. The output is the interface displayed to the user.

[1881] Step 6:

[1882] The server searches for books and displays the results

[1883] Input: User's search query

[1884] The server receives a search query, generates a list of relevant books from the database, and displays it to the user. Specifically, it performs a database search based on the query and sends the results to the user's device. The output is a list of search results.

[1885] Step 7:

[1886] The user's device inputs questions and analyzes emotions.

[1887] Input: User's question and facial expression

[1888] A user accesses the details page of a specific book and enters a question. At the same time, the device's camera recognizes the user's facial expressions, and the emotion engine analyzes their emotional state. Specifically, the camera captures the user's facial expression data, and an emotion analysis algorithm is used to identify their emotional state. The output is the user's question and emotional state.

[1889] Step 8:

[1890] The server generates an answer based on the sentiment

[1891] Input: User's question and emotional state

[1892] The server sends emotional information to the generative AI model based on the analysis results of the emotion engine. It also sends parameters to adjust the answer content according to the user's emotions. Specifically, the generative AI model generates an answer based on the question and emotional information. The output is an answer that takes emotions into consideration.

[1893] Step 9:

[1894] The server sends the answer to the user's device.

[1895] Input: Generated Answer

[1896] The server sends the generated answer to the user terminal. Specifically, the generated answer data is sent to the user terminal and displayed on the user's screen. The output is the answer displayed to the user.

[1897] Step 10:

[1898] User devices monitor emotional changes

[1899] Input: Continuous facial expression data of the user

[1900] While the user is checking the answers displayed on the screen, the emotion engine continuously monitors the user's emotional changes. Specifically, the device's camera continuously captures facial expression data and performs emotion analysis in real time. The output is the analysis results of emotional changes.

[1901] Step 11:

[1902] The server adaptively updates content recommendations

[1903] Input: Emotion change analysis results

[1904] The server dynamically adjusts the content provided based on the monitoring results. Specifically, it retrieves new content from the generative AI model that reflects the analysis results of emotion changes and provides it to the user. The output is the updated content.

[1905] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1906] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1907] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1908] [Fourth embodiment]

[1909] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1910] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1911] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1912] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1913] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1914] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1915] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1916] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1917] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1918] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1919] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1920] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1921] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1922] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Specific embodiments of this system are described below.

[1923] Building the dataset

[1924] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[1925] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is structured and stored in a database.

[1926] Creating generative AI models

[1927] Server: Builds a generative AI model that learns the content of specific books based on the cleaned text data. Trains the generative AI model based on the text taken as training data and adjusts the parameters accordingly.

[1928] Server: Deploys the trained generative AI model and makes it available to users online.

[1929] Building an online platform

[1930] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[1931] User Interface and Operation

[1932] Book Selection and Questions

[1933] User device: The user opens a web browser, accesses the provided platform URL, enters keywords in the search bar, and searches for the book they want to read.

[1934] Server: Receives the search query, generates a list of relevant books, and displays them to the user.

[1935] User device: The user selects a particular book from the displayed list to access its detail page.

[1936] Ask a question and get an answer

[1937] User device: On the details page of the selected book, the user enters a question into the text box that asks the generative AI model a question.

[1938] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[1939] Server: The generative AI model analyzes the question and generates an appropriate answer.

[1940] Server: Sends the generated answer to the user's device.

[1941] User device: The user checks the answers displayed on the screen and asks further questions if necessary.

[1942] Specific examples

[1943] Example 1: Using business books

[1944] scenario:

[1945] Let's say a user wants to refer to a specific business book on "strategies for new business development."

[1946] User device: Access the platform and search for "New Business Development".

[1947] Server: Displays a list of relevant business books as search results.

[1948] User Device: Select a specific business book and go to its details page.

[1949] User device: Type the question, "What is the most important step in new business development?"

[1950] Server: Sends questions to the generative AI model and generates answers.

[1951] User device: Receive specific answers such as, "Imaginative market analysis and targeting are the most important."

[1952] Example 2: Using cooking recipes

[1953] scenario:

[1954] Let's say a user wants to consult a cookbook on how to make authentic French toast.

[1955] User device: Accesses the platform and searches for "french toast."

[1956] Server: Displays a list of relevant cookbooks as search results.

[1957] User Device: Select a specific cookbook and go to its detail page.

[1958] User device: Type in the question, "What's the secret to making authentic French toast?"

[1959] Server: Sends questions to the generative AI model and generates answers.

[1960] User device: Receives a response such as "Adding a little vanilla extract will give it an even richer flavor."

[1961] The above is a specific embodiment for carrying out the present invention. This system enables users to effectively obtain information and put it to practical use.

[1962] The processing flow will be explained below.

[1963] Step 1:

[1964] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[1965] Step 2:

[1966] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[1967] Step 3:

[1968] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[1969] Step 4:

[1970] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[1971] Step 5:

[1972] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[1973] Step 6:

[1974] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[1975] Step 7:

[1976] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[1977] Step 8:

[1978] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[1979] Step 9:

[1980] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[1981] Step 10:

[1982] User device: The user selects a particular book from the displayed list to access its detail page.

[1983] Step 11:

[1984] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[1985] Step 12:

[1986] Server: Receives the input question and sends an API request to the corresponding generative AI model.

[1987] Step 13:

[1988] Server: The generative AI model analyzes the question and generates an appropriate answer.

[1989] Step 14:

[1990] Server: Sends the generated answer to the user's device.

[1991] Step 15:

[1992] User device: The user reviews the answers displayed on the screen and asks further questions if necessary.

[1993] Example 1

[1994] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1995] Previously, digitizing and utilizing analog Japanese information has faced challenges such as low-accuracy OCR technology, inefficient data cleaning methods, and a lack of user-friendly interfaces. Furthermore, building an entire system to train generative AI models and provide appropriate information to users has been complex. There is a need to resolve these issues and provide high-quality digital information while enabling users to access it efficiently.

[1996] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1997] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI models online, means for providing an interface for a user to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for cleaning the digitized text data, means for structuring the cleaned text data, means for a user to search for specific data, and means for displaying search results to the user. This makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and effectively access the information they need.

[1998] "Analog information in Japanese" refers to information written in Japanese that is not stored as electronic data, such as paper books and handwritten documents.

[1999] "Digitizing" means converting analog information into an electronic data format that can be processed by a computer.

[2000] "Text data" refers to data that electronically records information such as letters, sentences, and symbols.

[2001] A "generative AI model" refers to an artificial intelligence model that can learn from text data and perform specific tasks.

[2002] "Training" means using large amounts of data to learn from an AI model and improve its performance.

[2003] "Providing online" means making a service or function available to users via the Internet.

[2004] "Interface" refers to the screens and input means through which users interact with a system.

[2005] "Cleaning" refers to the process of correcting misrecognized parts from text data and removing unnecessary spaces and line breaks.

[2006] "Structuring" means organizing data into a certain format or structure so that it can be easily searched and used.

[2007] "Deploying" means placing a trained AI model in an actual production environment and making it available for use.

[2008] "Database" refers to an electronic system for efficiently storing, managing, and retrieving data.

[2009] "Searching" means finding data based on specific keywords or conditions.

[2010] "Results" refers to the final information or data obtained through operations such as search or calculation.

[2011] An "API request" is a request made by an application to another system to use a function or obtain information.

[2012] "REST API" refers to an API based on uniform design principles for exchanging data between web services.

[2013] The present invention is a system that digitizes analog information in Japanese, trains a generative AI model, and provides it to users. Detailed embodiments for implementing this system are described below.

[2014] Building the dataset

[2015] server:

[2016] To digitize analog information, first, paper books or handwritten documents are scanned to obtain image data. The hardware used is, for example, the common scanner "Epson Perfection V600." This scanned image is then input into the OCR engine "Tesseract OCR," and text data is extracted.

[2017] server:

[2018] Next, the extracted text data is cleaned. This process involves correcting misrecognized characters and removing unnecessary line breaks and spaces. A Python script is used for cleaning. The cleaned text data is then stored in a MySQL database. The stored data is then structured so that it can be used for later training and search.

[2019] Creating generative AI models

[2020] server:

[2021] A generative AI model is trained on the cleaned text data using a machine learning framework such as TensorFlow or PyTorch, which inputs training data and adjusts certain parameters to improve the model's accuracy.

[2022] server:

[2023] Once trained, the generative AI model is deployed in a Docker container, making it operational and enabling it to quickly provide answers to user queries.

[2024] Building an online platform

[2025] server:

[2026] The user interface will be developed using ReactJS and Node.js. This interface allows users to operate it intuitively. The interface will include a book search function, an access interface to the generative AI model, user profile management, and the ability to add links to purchase books. The search function will use Elasticsearch to quickly extract relevant information from large amounts of data.

[2027] User interface and operation examples

[2028] Book Selection and Questions

[2029] User device:

[2030] A user opens a web browser, visits the provided platform URL (e.g., www.examplebooks.com), and enters keywords into the search bar to search for a specific book (e.g., "new business development" or "French toast").

[2031] server:

[2032] It receives a search query, uses Elasticsearch to generate a list of relevant books, and displays them to the user.

[2033] User device:

[2034] The user selects a particular book from the displayed list to access its detail page.

[2035] Ask a question and get an answer

[2036] User device:

[2037] Go to the details page and type your question into the text box (e.g., "What is the most important step in new business development?").

[2038] server:

[2039] The entered question is sent to the generative AI model using the REST API.

[2040] server:

[2041] A generative AI model analyzes the question and generates an appropriate answer, which is composed in natural language based on the model's training data.

[2042] server:

[2043] The generated answer is sent back to the user's device and displayed in real time.

[2044] User device:

[2045] Users review the answers displayed on the screen (e.g., "Imaginative market analysis and targeting are paramount") and ask further questions if necessary.

[2046] This invention makes it possible to provide highly accurate digital information from analog information, allowing users to intuitively and efficiently access the information they need.

[2047] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2048] Specific explanation of program processing

[2049] Step 1: Digitizing analog information

[2050] server:

[2051] Input: Analog information such as paper books and handwritten documents.

[2052] Specific operation: Analog information is scanned using a scanner. The hardware used is the "Epson Perfection V600."

[2053] Output: Obtain a high-resolution scanned image.

[2054] Step 2: Extract text using OCR

[2055] server:

[2056] Input: Scanned image data.

[2057] Specific operation: The acquired scanned image is input into the OCR engine "Tesseract OCR" and text data is extracted from the image.

[2058] Call the Tesseract API specifying the path to the image data provided by the client or the database.

[2059] Output: The extracted text data.

[2060] Step 3: Cleaning the text data

[2061] server:

[2062] Input: Text data extracted by OCR.

[2063] What it does: Uses a Python script to clean text, including correcting misrecognized characters and removing unnecessary line breaks and spaces.

[2064] Output: Cleaned text data.

[2065] Step 4: Structuring and storing text data

[2066] server:

[2067] Input: Cleaned text data.

[2068] What it does: It uses a Python script to structure text data into the required format, then stores it in a MySQL database, which also contains metadata such as book title, author, and content.

[2069] Output: Structured text data stored in a database.

[2070] Step 5: Training the generative AI model

[2071] server:

[2072] Input: Cleaned text data stored in a database.

[2073] How it works: Use TensorFlow or PyTorch to train a generative AI model. Input training data into the AI ​​model and adjust parameters such as the number of epochs, batch size, and learning rate.

[2074] Output: A fully trained generative AI model.

[2075] Step 6: Deploying the generative AI model

[2076] server:

[2077] Input: A fully trained generative AI model.

[2078] What it does: Deploy an AI model in a Docker container. The containerized model runs on a server and is accessible externally through an API.

[2079] Output: A usable generative AI model is deployed in a production environment.

[2080] Step 7: Developing the User Interface

[2081] server:

[2082] Input: None (early development stage).

[2083] Specific operation: Using "ReactJS" and "Node.js", a user interface will be developed, which will include a book search function, an access interface to the generative AI model, user profile management, and a link to purchase the book.

[2084] Output: A web platform that users can access.

[2085] Step 8: Book Search

[2086] User device:

[2087] Type: Enter a keyword into the search bar (e.g., "new business development" or "french toast").

[2088] Specific actions: Open a web browser, access the provided platform URL, and enter keywords in the search bar.

[2089] server:

[2090] Input: The user's search query.

[2091] Specific operation: Uses Elasticsearch to generate a list of relevant books based on the search query and displays it to the user.

[2092] Output: Shows a list of books relevant to the user.

[2093] Step 9: Select a book and enter your question

[2094] User device:

[2095] Input: Select a specific book from the displayed book list and go to its details page.

[2096] What happens: Visit a book's detail page and enter a question (e.g., "What is the most important step in new business development?").

[2097] Output: The question entered.

[2098] Step 10: Submitting questions and generating answers

[2099] server:

[2100] Input: The user's question.

[2101] How it works: Send a question to the generative AI model via the REST API. The generative AI model analyzes the question and generates an appropriate answer.

[2102] Output: The generated answer.

[2103] Step 11: View your answers

[2104] server:

[2105] Input: The generated answer.

[2106] Specific operation: The generated answer is sent back to the user's device and displayed in real time.

[2107] User device:

[2108] Input: None (display stage).

[2109] Output: The user sees the answer displayed on the screen (e.g., "Imaginative market analysis and targeting are most important.").

[2110] Step 12: Additional Questions

[2111] User device:

[2112] Input: Additional questions (if needed).

[2113] What it does: Review the answers displayed on the screen, enter additional questions if necessary, and submit.

[2114] Output: Additional questions are sent to the generative AI model.

[2115] The above are the specific processing steps of the program of this system.

[2116] (Application example 1)

[2117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2118] Conventional search systems and AI models have the drawback of making it difficult for users to freely ask questions about books and materials and obtain appropriate answers. It is particularly difficult to ask detailed questions about the contents of each book and obtain immediate answers for books and materials that are widely available in bookstores and libraries. Furthermore, there is a lack of systems that can efficiently digitize vast amounts of analog information, build AI models based on that information, and make them useful to users, making this initiative necessary.

[2119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2120] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple generative AI modeled materials online, means for providing an interface for users to input questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, means for searching for books and selecting specific materials, and means for the generative AI model to analyze questions related to the materials and generate answers. This allows users to freely ask questions about books and materials available in bookstores and libraries and receive appropriate answers in real time.

[2121] "Analog information in Japanese" refers to materials written in Japanese that have not been digitized, such as paper books and handwritten documents.

[2122] "Digital text data" refers to data that has been converted from analog Japanese information into digital text using scanning or optical character recognition (OCR) technology.

[2123] A "generative AI model" is an artificial intelligence model that uses digitized text data as training data to generate appropriate answers to questions.

[2124] "Means for providing online" means means for placing generative AI-modeled materials on a web server or cloud service so that they can be accessed by users via the internet.

[2125] The "interface for inputting questions" refers to a user interface that allows users to input questions in text format to the generative AI model.

[2126] The "means of generating an answer" refers to the process by which the generative AI model analyzes the user's question and generates an appropriate answer.

[2127] "Means for searching and selecting specific materials" means the ability for users to search for materials using specific keywords and select relevant materials.

[2128] "Means for the generative AI model to analyze questions related to the materials and generate answers" refers to the process in which the generative AI model analyzes questions asked by users about selected materials and generates answers based on those materials.

[2129] "Smartphone Application" means a specific software application for installation and use on a smartphone device.

[2130] This invention is a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users. Detailed embodiments for implementing this system will be described.

[2131] System Program

[2132] 1. Hardware and software configuration

[2133] Hardware: Smartphones (iOS devices, Android devices), servers

[2134] Software: Python, Flask (for API server), TensorFlow (generative AI model), Tesseract (OCR engine), SQLite (database), React Native (smartphone app development)

[2135] 2. Digitizing data and training generated AI models

[2136] The server scans paper books and handwritten documents available at bookstores and libraries and extracts Japanese text data using the Tesseract OCR engine. The extracted text data is then cleaned, correcting misrecognized characters and removing unnecessary line breaks and spaces. The cleaned text data is then structured and stored in an SQLite database.

[2137] The server trains a generative AI model using TensorFlow based on the stored cleaned text data, and once trained, the generative AI model is deployed on the Flask API server.

[2138] 3. User interface development and operation

[2139] We developed a smartphone application using React Native, implementing a book search function and a question and answer interface. Through the app, users can find books by keyword search, ask questions about the books, and receive answers in real time.

[2140] Program processing

[2141] 1. Scanning the book and OCR processing

[2142] The server takes scanned images of books provided by bookstores and libraries and extracts the text data using the Tesseract OCR engine.

[2143] The extracted text data is cleaned by the server and stored in an SQLite database.

[2144] 2. Training the generative AI model

[2145] The server uses the cleaned text data as training data to train a generative AI model using TensorFlow.

[2146] Once trained, the generative AI model is deployed on a Flask API server and made accessible online.

[2147] 3. User Interface Operation

[2148] The smartphone application allows users to search for books by entering keywords and select specific books from the search results.

[2149] Users can enter questions into the generative AI model on the book details page and receive real-time answers from the server.

[2150] Examples and prompts

[2151] Examples:

[2152] A user types a question about a history book: "What accomplishments did a particular person achieve?"

[2153] The generative AI model generates and displays the answer, "A specific person achieved numerous victories during the Warring States period, promoted centralized policies, and unified the country."

[2154] Example prompt sentence:

[2155] "Enter a question about a specific person (e.g., 'What accomplishments did a specific person have?')"

[2156] This system allows users to ask any questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[2157] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2158] Step 1:

[2159] The server receives scanned images of books provided by bookstores and libraries. The scanned images are input and converted into text data using Tesseract, an optical character recognition (OCR) engine. The output is OCR-processed text data.

[2160] Step 2:

[2161] The server cleans the OCR-processed text data. Specifically, it corrects misrecognized characters and deletes unnecessary line breaks and spaces. The input is the OCR-processed text data, and the output is the cleaned text data.

[2162] Step 3:

[2163] The server structures the cleaned text data and stores it in an SQLite database. The input is the cleaned text data, and the output is the structured text data stored in the database.

[2164] Step 4:

[2165] The server trains a generative AI model using the stored structured text data. It uses TensorFlow to build the AI ​​model and adjusts its parameters based on the training data. The input is the structured text data, and the output is a trained generative AI model.

[2166] Step 5:

[2167] The server deploys the trained generative AI model on the Flask API server, making the model accessible online. The input is the trained generative AI model, and the output is the deployed generative AI model.

[2168] Step 6:

[2169] Users open the smartphone application and search for books by entering keywords through an interface built with React Native. The input is the search query, and the output is a search result list of matching books.

[2170] Step 7:

[2171] The user selects a particular book from the search results and navigates to that book's detail page. The input is the selection of a particular book, and the output is the display of the book's detail page.

[2172] Step 8:

[2173] The user enters a question on the details page and sends it to the generative AI model. The input is the user's question, and the output is an API request to the generative AI model.

[2174] Step 9:

[2175] The server analyzes the user's question against the generative AI model and generates an appropriate answer. The input is the user's question, and the output is the generated answer.

[2176] Step 10:

[2177] The server sends the generated answer to the user's smartphone, where it is displayed on the user's device. The input is the generated answer, and the output is the answer displayed on the user's device.

[2178] This processing flow allows users to freely ask questions about books and materials available at bookstores and libraries and receive appropriate answers in real time.

[2179] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2180] This invention combines a system that digitizes analog information in Japanese, trains a generative AI model based on that data, and provides it to users with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[2181] Building the dataset

[2182] Server: Obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are fed into an optical character recognition (OCR) engine to extract text data.

[2183] Server: The extracted text data is cleaned. This process involves correcting misrecognized characters, removing unnecessary line breaks and spaces, and standardizing the format. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[2184] Creating generative AI models

[2185] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[2186] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[2187] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[2188] Building an online platform

[2189] Server: Develops the user interface and implements the book search function and the access interface to the generative AI model. It also provides functions such as user profile management and book purchase links.

[2190] User Interface and Emotion Engine

[2191] Book Selection and Emotion Recognition

[2192] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[2193] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[2194] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[2195] User device: The user selects a particular book from the displayed list to access its detail page.

[2196] Ask questions, recognize emotions and generate answers

[2197] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[2198] Server: Receives the input question and uses the emotion engine to analyze the user's current emotional state. For example, if the user is feeling stressed or anxious, it recognizes that emotion.

[2199] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[2200] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[2201] Server: Sends the generated answer to the user's device.

[2202] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[2203] Specific examples

[2204] Example 1: Using business books

[2205] scenario:

[2206] Let's say the user wants to refer to a specific business book about "strategies for new business development." The user is in a stressful state.

[2207] User device: Access the platform and search for "New Business Development".

[2208] Server: Displays a list of relevant business books as search results.

[2209] User Device: Select a specific business book and go to its details page.

[2210] User device: Type the question, "What is the most important step in new business development?"

[2211] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[2212] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[2213] User device: The user receives the answer and feels relief along with concrete steps.

[2214] Example 2: Using cooking recipes

[2215] scenario:

[2216] Let's say the user wants to consult a cookbook on "how to make authentic French toast." The user is relaxed.

[2217] User device: Accesses the platform and searches for "french toast."

[2218] Server: Displays a list of relevant cookbooks as search results.

[2219] User Device: Select a specific cookbook and go to its detail page.

[2220] User device: Type in the question, "What's the secret to making authentic French toast?"

[2221] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[2222] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[2223] User terminal: Users can receive answers comfortably and enjoy cooking.

[2224] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[2225] The processing flow will be explained below.

[2226] Step 1:

[2227] Server: Scans the analog books using a dedicated scanner or externally provided PDF data. The image data of each scanned page is collected and stored on the server.

[2228] Step 2:

[2229] Server: Extracts text data from scanned images using an OCR engine. The OCR engine processes the image of each page, recognizes it as text data, and saves it as string data.

[2230] Step 3:

[2231] Server: The extracted text data is cleaned. Specifically, misrecognized characters are corrected, unnecessary line breaks and extra spaces are removed, and the format is standardized. The clean text data is generated and stored in a structured format (for example, a database or CSV format).

[2232] Step 4:

[2233] Server: Starts training a generative AI model (e.g., GPT-3) based on the cleaned text data. It uses the ingested text as training data and adjusts the model's parameters. The training process runs over multiple epochs (iterations).

[2234] Step 5:

[2235] Server: After the model is trained, it uses the test dataset to evaluate its performance and adjusts the model based on the results to achieve optimal performance.

[2236] Step 6:

[2237] Server: Deploys the trained generative AI model and makes it accessible to users on an online platform.

[2238] Step 7:

[2239] User device: The user opens a web browser and accesses the provided platform URL. The user registers and logs in.

[2240] Step 8:

[2241] User device: Users enter keywords or book titles into the search bar to find the books they want to read.

[2242] Step 9:

[2243] Server: Receives the search query, generates a list of relevant books from the database, and displays it to the user.

[2244] Step 10:

[2245] User device: The user selects a particular book from the displayed list to access its detail page.

[2246] Step 11:

[2247] User device: On the book's details page, the user enters a question into the text box provided for inputting questions to the generative AI model.

[2248] Step 12:

[2249] Server: Receives the input question and uses an emotion engine to analyze the user's current emotional state, for example, by inferring emotions from the context of the input text and the user's past behavior.

[2250] Step 13:

[2251] Server: Based on the recognized emotion information, the server sends questions to the generative AI model, along with parameters to adjust the answer depending on the user's emotion.

[2252] Step 14:

[2253] Server: The generative AI model analyzes the question and generates an appropriate answer. The emotion engine adjusts the answer to take into account the user's emotional state.

[2254] Step 15:

[2255] Server: Sends the generated answer to the user's device.

[2256] Step 16:

[2257] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The emotion engine also continuously monitors the user's emotional changes.

[2258] Specific examples

[2259] Example 1: Using business books

[2260] Step 1:

[2261] User device: The user wants to consult a specific business book on "New Business Development Strategies." The user is in a stressful state.

[2262] Step 2:

[2263] User device: Access the platform and search for "New Business Development".

[2264] Step 3:

[2265] Server: Displays a list of relevant business books as search results.

[2266] Step 4:

[2267] User Device: Select a specific business book and go to its details page.

[2268] Step 5:

[2269] User device: Type the question, "What is the most important step in new business development?"

[2270] Step 6:

[2271] Server: The emotion engine recognizes the user's stress and sends that state to the generative AI model.

[2272] Step 7:

[2273] Server: The generative AI model generates specific answers such as "Imaginative market analysis and targeting are the most important thing," while also adding encouraging comments to reduce stress.

[2274] Step 8:

[2275] User device: The user receives the answer and feels relief along with concrete steps.

[2276] Example 2: Using cooking recipes

[2277] Step 1:

[2278] User device: The user wants to consult a cookbook on "how to make authentic French toast." The user is in a relaxed state.

[2279] Step 2:

[2280] User device: Accesses the platform and searches for "french toast."

[2281] Step 3:

[2282] Server: Displays a list of relevant cookbooks as search results.

[2283] Step 4:

[2284] User Device: Select a specific cookbook and go to its detail page.

[2285] Step 5:

[2286] User device: Type in the question, "What's the secret to making authentic French toast?"

[2287] Step 6:

[2288] Server: The emotion engine recognizes the user's relaxed state and sends that state to the generative AI model.

[2289] Step 7:

[2290] Server: The generative AI model generates specific answers such as "Adding a little vanilla extract will give it an even richer flavor," along with comments that emphasize the fun.

[2291] Step 8:

[2292] User terminal: Users can receive answers comfortably and enjoy cooking.

[2293] The above is a specific embodiment for carrying out the present invention. This system allows users to efficiently obtain information according to their emotional state and to utilize this information in a more practical manner.

[2294] Example 2

[2295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2296] Systems that use generative AI models to help users efficiently acquire information in Japanese need to provide appropriate answers based on the user's emotional state. However, conventional systems generate uniform answers without considering the user's emotional state, making it difficult to fully meet the user's needs. Furthermore, if the digitized text data is not effectively cleaned and structured, and the parameters of the generative AI model are not adjusted effectively, the accuracy of the model decreases.

[2297] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2298] In this invention, the server includes a means for digitizing Japanese analog information, a means for training a generative AI model based on the digitized text data, and a means for analyzing the user's emotional state and reflecting the analysis results in the generative AI model. This makes it possible to provide answers optimized according to the user's emotional state. Furthermore, high accuracy of answers can be maintained through the process of cleaning and structuring the digitized text data and training the generative AI model.

[2299] "Analog information in Japanese" refers to Japanese information that exists in physical form, rather than digital form, such as paper books or handwritten documents.

[2300] "Digitalization" refers to the conversion of analog information into electronic data in a format that can be processed by a computer.

[2301] "Generative AI model" refers to an artificial intelligence model that has been trained using machine learning techniques to perform a specific task.

[2302] "Training" is the process of providing learning data to a generative AI model to optimize the model's parameters and improve its performance for a specific task.

[2303] "Provided online" means that the service is provided in a form that allows users to access it via the Internet.

[2304] "Interface" refers to the means or operating screen through which a user interacts with a system.

[2305] "Emotional state" refers to the user's current emotional state, such as stressed, relaxed, or anxious.

[2306] "Cleaning" is the process of correcting misrecognized characters in digitized text data and deleting unnecessary data to improve the quality of the data.

[2307] "Structuring" refers to organizing and arranging cleaned text data into a specific format or structure.

[2308] "Parameter tuning" is the process of setting and updating the learning parameters within a generative AI model so that it performs optimally.

[2309] "Search" is the act of locating relevant information or data from a database based on keywords entered by the user.

[2310] "Personalized information provision" refers to providing users with the most appropriate information based on their profile and search history.

[2311] This invention is a system that digitizes analog information in Japanese and trains a generative AI model based on that information, recognizing the user's emotional state and providing the optimal response according to that state. Specific embodiments are described below.

[2312] Building the dataset

[2313] Server: Scan analog books and handwritten documents using a scanner and save the scanned images on the server. We recommend using a high-resolution scanner.

[2314] Server: Input the scanned image into an optical character recognition (OCR) engine to extract the text data. Typically, an OCR engine such as Tesseract or Google Cloud Vision is used.

[2315] Server: Cleans the extracted text data, corrects misrecognized characters, and deletes unnecessary line breaks and spaces. A natural language processing library (e.g., nltk) is used as text analysis software.

[2316] Server: The clean text data is structured and stored in a database or CSV format for easy subsequent processing.

[2317] Creating generative AI models

[2318] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data. The training process uses machine learning frameworks such as TensorFlow and PyTorch.

[2319] Server: Once training is complete, evaluate the model's performance using the test dataset and tune the model's parameters for optimal performance.

[2320] Server: Deploy the completed generative AI model and make it accessible to users on an online platform. Upload the model to a cloud server (e.g., AWS) and expose the API.

[2321] Building the user interface

[2322] Server: Develop the user interface using web development tools (e.g., React, Angular), implement book search functionality and a question input form, and integrate user profile management and book purchase links.

[2323] Book Selection and Emotion Recognition

[2324] User terminal: The user opens a web browser and accesses the provided platform URL. The user registers and logs in to the platform.

[2325] User device: Users enter keywords or book titles in the search bar to search for the books they want to read.

[2326] Server: The server receives the search query, generates a list of relevant books from the database, and displays it in the browser.

[2327] User device: The user selects a particular book from the displayed list to access its detail page.

[2328] Ask questions, recognize emotions and generate answers

[2329] User device: On the book details page, the user enters a question into the text box provided to the generative AI model.

[2330] Server: The server receives the input question and uses an emotion engine to analyze the user's emotional state. For example, it identifies if the user is feeling stressed or anxious.

[2331] Server: Sends questions to the generative AI model based on the recognized emotional state and provides response parameters according to the emotion to the model.

[2332] Server: A generative AI model analyzes the question and generates the best answer, including adjustments based on an emotion engine to ensure the answer takes into account the user's emotional state.

[2333] Server: Sends the generated answer to the user's device.

[2334] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[2335] Specific examples

[2336] Examples of using business books

[2337] scenario:

[2338] If a user wants to refer to a business book on "strategies for new business development," the user is in a stressful state.

[2339] Example prompt:

[2340] "What is the most important step in new business development?"

[2341] Operation:

[2342] Users access the platform and receive specific answers along with encouraging comments to help reduce stress.

[2343] Cooking recipe usage examples

[2344] scenario:

[2345] If a user wants to consult a cookbook on "how to make authentic French toast," they're in a relaxed state.

[2346] Example prompt:

[2347] "What's the secret to making authentic French toast?"

[2348] Operation:

[2349] It recognizes when the user is relaxed and adds comments highlighting the good time along with specific answers to enrich the flavor.

[2350] This system allows users to efficiently obtain information that corresponds to the appropriate emotion and utilize it in a more practical way.

[2351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2352] Step 1:

[2353] Server: Scan analog books and handwritten documents with a scanner to obtain scanned images.

[2354] Specific operation: Place a book page into the scanner and press the scan button to obtain a high-resolution image.

[2355] Input: Analog books and handwritten documents.

[2356] Output: High resolution scanned image.

[2357] Step 2:

[2358] Server: Input the scanned image into an OCR engine to extract text data. For example, we use Tesseract or Google Cloud Vision.

[2359] What it does: The server feeds the scanned image to the OCR software, and the OCR engine extracts the characters in the image as text.

[2360] Input: High resolution scanned image.

[2361] Output: The extracted raw text data.

[2362] Step 3:

[2363] Server: Cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces.

[2364] Specific behavior: The server analyzes the text data using text analysis software (e.g., nltk) and makes corrections to ensure data consistency.

[2365] Input: Extracted raw text data.

[2366] Output: Cleaned text data.

[2367] Step 4:

[2368] Server: The cleaned text data is structured and stored in a database or CSV format.

[2369] What it does: Use the data format conversion tool to convert and save the cleaned text into CSV or database format.

[2370] Input: Cleaned text data.

[2371] Output: Structured data.

[2372] Step 5:

[2373] Server: Trains a generative AI model (e.g., GPT-3) based on the cleaned text data.

[2374] What it does: The server feeds training data to an AI model framework (e.g., TensorFlow or PyTorch) and updates the model parameters every epoch.

[2375] Input: Structured data.

[2376] Output: A trained generative AI model.

[2377] Step 6:

[2378] Server: Evaluates the performance of the trained model using the test dataset, and retunes model parameters if necessary.

[2379] How it works: The server feeds test data to the generative AI model, compares the generated output with the original output, and evaluates the error rate and accuracy. It adjusts hyperparameters as needed.

[2380] Input: A trained generative AI model and a test dataset.

[2381] Output: An evaluated and tuned generative AI model.

[2382] Step 7:

[2383] Server: The completed generative AI model is deployed to a cloud server and made accessible on an online platform.

[2384] Specific operation: The server uploads the generated AI model to a cloud service (e.g., AWS) and exposes an API, allowing users to access the generated AI model.

[2385] Input: An evaluated and tuned generative AI model.

[2386] Output: A generative AI model accessible on the cloud.

[2387] Step 8:

[2388] Server: Develop the user interface and implement the book search function and question input form. Use web development tools (e.g. React, Angular).

[2389] What it does: Front-end developers design the user interface, and back-end developers work with APIs to implement the functionality.

[2390] Input: Design specifications and design requirements.

[2391] Output: User interface.

[2392] Step 9:

[2393] User device: The user opens a web browser, accesses the platform, and logs in.

[2394] Specific actions: Enter a URL in your browser, enter the required information in the login form that appears, and click the login button.

[2395] Input: User login information.

[2396] Output: Access to the platform.

[2397] Step 10:

[2398] User device: To find the book they want to read, users enter keywords or titles into the search bar.

[2399] What happens: A user enters keywords into the search bar and clicks the search button, which sends a search query to the server.

[2400] Input: Search keyword or title.

[2401] Output: The search query.

[2402] Step 11:

[2403] Server: Based on the search query, the server generates a list of relevant books from its database and displays it in the user's browser.

[2404] What happens: The server executes a database query to retrieve related book information and returns it to the browser.

[2405] Input: Your search query.

[2406] Output: A list of related books.

[2407] Step 12:

[2408] User device: The user selects a particular book from the displayed list to access its detail page.

[2409] What happens: The user clicks on the book they want to read and sees more information about it.

[2410] Input: Book selection.

[2411] Output: Book detail page.

[2412] Step 13:

[2413] User device: On the book detail page, the user enters a question for the generative AI model into a text box.

[2414] What happens: The user types a question into the text box and clicks the submit button.

[2415] Input: The user's question.

[2416] Output: The question query.

[2417] Step 14:

[2418] Server: The server receives the question and analyzes the user's emotional state using an emotion engine.

[2419] Specific operation: The server passes the question data to the emotion engine and receives the analysis results.

[2420] Input: The user's question.

[2421] Output: Emotional state information.

[2422] Step 15:

[2423] Server: Sends questions to the generative AI model based on emotion information and reflects response parameters according to the emotion.

[2424] How it works: The server provides the question and emotion parameters to the generative AI model, which then generates a response.

[2425] Input: Question query and emotional state information.

[2426] Output: The answer according to the user.

[2427] Step 16:

[2428] Server: Sends the generated answer to the user's device.

[2429] What happens: The server generates the answer and returns it to the user's browser.

[2430] Input: The answer given to the user.

[2431] Output: The answer text.

[2432] Step 17:

[2433] User device: The user checks the answers displayed on the screen and asks additional questions if necessary. The server also continuously monitors the user's emotional changes.

[2434] What it does: The user reads the displayed answer and decides whether to enter a follow-up question.

[2435] Input: The displayed answer.

[2436] Output: Change in the user's emotional state and follow-up questions.

[2437] (Application example 2)

[2438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2439] Conventional digital content delivery systems provide uniform information without considering the user's emotional state, resulting in a limited user experience and a lack of means to provide information optimally tailored to the user's current emotional state. As a result, user satisfaction and engagement may be low. To solve these issues, a system is needed that recognizes user emotions in real time and provides personalized content using generative AI models accordingly.

[2440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2441] In this invention, the server includes means for digitizing Japanese analog information, means for training a generative AI model based on the digitized text data, means for providing multiple books that have been modeled by the generative AI online, means for providing an interface through which a user inputs questions to the generative AI model, means for the generative AI model to generate answers to the user's questions, means for displaying the generated answers to the user, emotion analysis means for recognizing the user's emotions, and means for adjusting the content of the generative AI model's answers based on the recognized emotion information. This makes it possible to provide optimal information taking into account the user's emotional state.

[2442] "Means for digitizing analog information in Japanese" refers to scanning analog information such as paper books and handwritten documents and extracting text data using optical character recognition.

[2443] "Means for training a generative AI model based on digitized text data" refers to means for using extracted text data to adjust the parameters of a generative AI model and train the generative AI model.

[2444] "Means for providing multiple generative AI modeled books online" refers to means for distributing text data and book data generated using generative AI models through an online platform.

[2445] "Means for providing an interface for users to input questions to a generative AI model" refers to means for providing an interface, such as a web browser or application screen, for users to input questions to a generative AI model.

[2446] "Means by which a generative AI model generates an answer to a user's question" refers to means by which a generative AI model generates an appropriate answer based on a question entered by a user.

[2447] "Means for displaying the generated answer to the user" refers to means for displaying the answer generated by the generative AI model on the user's screen.

[2448] "Emotion analysis means for recognizing user emotions" refers to a means for analyzing a user's emotional state from their facial expressions and voice using sensors such as a camera and microphone.

[2449] "Means for adjusting the content of the response of the generative AI model based on recognized emotional information" refers to means for adjusting the content and tone of the response generated by the generative AI model according to the emotional state of the user.

[2450] "Means for cleaning digitized text data" refers to means for generating clean text data by correcting misrecognized characters, deleting unnecessary line breaks and spaces, etc.

[2451] The "means for structuring the cleaned text data" refers to a means for storing the cleaned text data in a structured format such as a database or CSV format.

[2452] "Means for recommending personalized video content based on user emotions" refers to means for analyzing a user's emotional state and recommending and providing appropriate video content based on that state.

[2453] A "means for users to search for specific books" is a means that provides a search bar or filter functionality that allows users to find specific books.

[2454] A "means for displaying search results to a user" is a means for displaying on a user's screen a list of books found based on the search query.

[2455] "Means for monitoring changes in the user's emotions" refers to means for tracking and recording changes in the user's emotional state in real time using a camera or microphone.

[2456] "Means for adaptively changing content recommendations based on monitoring results" refers to means for detecting changes in a user's emotions and dynamically changing the content provided accordingly.

[2457] System construction

[2458] To implement this invention, a server and a user terminal are required. The server digitizes analog information, trains the generative AI model, and provides books online. The user terminal provides an interface for users to input questions and perform searches, and displays generated answers.

[2459] Building the dataset

[2460] server:

[2461] The server first obtains scanned images of analog books (e.g., paper books or handwritten documents). These scanned images are then fed into an optical character recognition (OCR) engine to extract text data. Specific software used at this stage is, for example, Tesseract OCR.

[2462] server:

[2463] The extracted text data is cleaned, misrecognized characters are corrected, and unnecessary line breaks and spaces are deleted. The cleaned text data is then structured and saved in a database or CSV format, for example.

[2464] Creating generative AI models

[2465] server:

[2466] A generative AI model (e.g., GPT-3) is trained on the cleaned text data. The training process takes place over multiple epochs (iterations).

[2467] server:

[2468] Once the model has been trained and its performance evaluated, the generative AI model is deployed and made accessible to users.

[2469] User Interface and Emotion Engine

[2470] User device:

[2471] Users access the provided platform URL using a smartphone or head-mounted display, and search for the book they want to read by entering keywords or the title of the book in the search bar.

[2472] server:

[2473] It receives a search query, generates a list of relevant books from its database, and displays them to the user.

[2474] User device:

[2475] A user visits the detail page of a particular book and enters a question into the generative AI model.

[2476] Ask questions, recognize emotions and generate answers

[2477] User device:

[2478] When a user types a question, the device's camera recognizes the user's facial expression, and the emotion engine analyzes the user's emotional state. For example, if the user is feeling stressed, it will recognize that emotion.

[2479] server:

[2480] Based on the analysis results of the emotion engine, emotional information is sent to the generative AI model. Parameters for adjusting the response content according to the user's emotions are also sent. Based on this, the generative AI model generates a response that takes the user's emotions into consideration.

[2481] server:

[2482] The generated answers are sent to the user's device, where the user can check the answers displayed on the screen. Changes in the user's emotions are continuously monitored, and the answers are updated as needed.

[2483] Specific examples

[2484] scenario:

[2485] Suppose a student wants to "relax" and wants to watch a relaxing music video. If the emotion engine recognizes the user's sad emotion, the generative AI model will recommend a video.

[2486] Example prompt sentence:

[2487] "Please recommend video content that is sad and suitable for relaxation."

[2488] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2489] Step 1:

[2490] The server digitizes analog information

[2491] Input: Scanned image of an analog book

[2492] The server acquires scanned images of analog books and extracts text data using an optical character recognition (OCR) engine. Specifically, the scanned images are input into an OCR engine (e.g., Tesseract OCR) for character recognition. The output is text data.

[2493] Step 2:

[2494] The server cleans the digitized text data.

[2495] Input: Text data extracted by OCR

[2496] The server cleans the extracted text data, correcting misrecognized characters and removing unnecessary line breaks and spaces. Specifically, the script analyzes the text data and automatically corrects misrecognized characters and unnecessary spaces. The output is clean text data.

[2497] Step 3:

[2498] The server trains the generative AI model

[2499] Input: clean text data

[2500] The server trains a generative AI model (e.g., GPT-3) based on clean text data. Specifically, it takes the text data as training data for the model and adjusts the model's parameters. Training is performed over multiple epochs (iterations). The output is a trained generative AI model.

[2501] Step 4:

[2502] The server deploys the generative ...

Claims

1. A means of digitizing analog information in Japanese, A means of training a generative AI model based on digitized text data; and a means for providing a plurality of generative AI modeled books online; A means for providing an interface for users to input questions to the generative AI model; A means by which the generative AI model generates answers to user questions; and a means for displaying the generated answers to the user; A system including:

2. a means for cleaning the digitized text data; a means for structuring the cleaned text data; The system of claim 1 further comprising:

3. A way for users to search for specific books; a means of displaying search results to the user; and The system of claim 1 further comprising:

Images