System

Abstract

A system is provided.SOLUTION: This system is provided with a means for receiving ambiguous information or a keyword inputted by a user, a means for automatically retrieving relevant book data based on the ambiguous information or the keyword, and a means for displaying the retrieved result to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current book search systems make it difficult for users to find the appropriate book if they do not know the specific title or author's name. Furthermore, they lack an interface that can be intuitively operated by users of various ages, especially the elderly and young children. Therefore, there is a need for a system that allows users to easily search for books based on vague information or vague keywords. [Means for solving the problem]

[0005] The system of the present invention is equipped with a means for receiving vague information or keywords entered by the user and analyzing them using natural language processing technology. Based on the analyzed information, the generation AI automatically searches a large database for relevant book data, and organizes and displays the results to the user. This system allows users who do not know the specific title or author's name to easily find books with the content they desire. It also provides an interface that is easy to use for the elderly and young children.

[0006] "Ambiguous information" refers to information input by a user that does not specify a specific title or author name, but indicates vague elements such as content or genre.

[0007] "Keywords" are important words and phrases necessary for book searches that the system of the present invention extracts from the user's vague information.

[0008] "Natural language processing technology" is a technology that enables computers to understand, analyze, and process human language.

[0009] "Generative AI" is an artificial intelligence that uses natural language processing technology to automatically search for relevant books based on the user's vague information or keywords.

[0010] A "book database" is a collection of searchable data that stores information about a large number of books.

[0011] "User" refers to an individual or organization that uses the book search system to search for books.

[0012] An "interface" is a screen or operating area that provides the visual and operational means for users to interact with a book search system. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] ---

[0035] The present invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. Below we will explain how this system works.

[0036] System configuration

[0037] The system consists of three main components:

[0038] User terminal

[0039] server

[0040] Book Database

[0041] User terminal

[0042] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[0043] server

[0044] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[0045] 1. Data Reception

[0046] The server receives the ambiguous information sent from the user terminal.

[0047] 2. Data Analysis

[0048] Analyzes received data and extracts important keywords. Natural language processing technology is used to tokenize text data and obtain meaningful keywords.

[0049] 3. Search Query Generation

[0050] Based on the extracted keywords, a search query is generated, which is then converted into a format that can be input to the generation AI.

[0051] 4. Book database search

[0052] Based on the generated query, the generative AI searches a book database and lists highly relevant books based on the information in the database.

[0053] 5. Organizing search results

[0054] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0055] 6. Data Transmission

[0056] The organized search results are sent to the user's terminal.

[0057] Book Database

[0058] The book database contains information about a huge number of books, which the server uses to find the most suitable book based on a search query. This database contains detailed information such as the book title, author, genre, and summary.

[0059] Specific examples

[0060] Example 1: Searching for moving adventure stories

[0061] 1. User Device

[0062] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[0063] 2. Server

[0064] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0065] The generative AI searches a book database based on the extracted keywords and lists books such as "The Alchemist" and "Life of Pi."

[0066] The resulting list is compiled and sent to the user's terminal.

[0067] 3. User Device

[0068] The search results are presented to the user as a list of books, from which the user can select books of interest.

[0069] Example 2: Looking for an easy-to-understand children's book about science

[0070] 1. User Device

[0071] A user types, "I'm looking for an easy-to-understand children's book about science."

[0072] 2. Server

[0073] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[0074] The generative AI searches for the best books based on these keywords, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0075] The organized results are sent to the user terminal.

[0076] 3. User Device

[0077] A list of search results is displayed to the user to help them select the most suitable book.

[0078] In this way, the present invention provides a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve the convenience of searching for books, especially for the elderly and young children.

[0079] The processing flow will be explained below.

[0080] ---

[0081] Step 1:

[0082] The user terminal displays an input interface for book searches to the user, which is intuitive and easy to operate, and is designed so that even the elderly and young children can use it.

[0083] Step 2:

[0084] Users input vague information or keywords about the book they are looking for, such as "a book that is an adventure story but also has an inspiring story."

[0085] Step 3:

[0086] The user terminal formats the ambiguous information entered by the user and sends it to the server, where the data has been converted into the appropriate format.

[0087] Step 4:

[0088] The server receives the ambiguous information sent from the user terminal and temporarily stores the received data.

[0089] Step 5:

[0090] The server then analyzes the received ambiguous information using natural language processing techniques, specifically tokenizing the text and extracting meaningful keywords.

[0091] Step 6:

[0092] The server generates a search query based on the analyzed keywords, which is then converted into a format that the AI ​​can understand.

[0093] Step 7:

[0094] The server uses a generation AI to search the book database, which references a large amount of book data and identifies books that are highly relevant.

[0095] Step 8:

[0096] The server organizes the generated search results and converts them into a user-friendly format, including, for example, book title, author, publication year, and a brief summary.

[0097] Step 9:

[0098] The server then sends the organized search results to the user's device, where the data is properly encrypted and securely transmitted.

[0099] Step 10:

[0100] The user terminal receives the search results sent from the server and displays them to the user in a visually easy-to-understand list format.

[0101] Step 11:

[0102] The user selects the book of interest from the displayed list of books, which may include more detailed information or related books.

[0103] ---

[0104] The above are the specific processing steps of the program.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional book search systems have the problem that it is difficult to search when you do not know the specific title or author name, or when you search based on only vague information. In addition, there is a lack of means to display search results in a format that is easy for users to understand, which makes them particularly difficult to use for the elderly and young children.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for receiving ambiguous information or keywords entered by a user, means for analyzing the ambiguous information or keywords using natural language processing technology, and means for generating a search query based on the ambiguous information or keywords. This allows a user to easily find a desired book by simply entering ambiguous information or keywords, even if they do not know the specific information.

[0110] The server further includes a means for searching for related book data using the generative AI model based on the generated search query, and a means for organizing detailed information about books based on the search results and displaying it to the user. This makes it possible to provide a system that displays search results in an efficient and easy-to-understand format and is easy to use, especially for the elderly and young children.

[0111] "Ambiguous information or keywords entered by the user" refers to unclear information entered when the user does not clearly know specific information, or keywords used to identify the search target.

[0112] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and process human language, and includes technology for tokenizing text data and performing semantic analysis.

[0113] "Search query" means a sentence or phrase that describes a request to a search engine or database to find specific information.

[0114] A "generative AI model" is an artificial intelligence model that makes predictions and generates based on input data, and refers to an AI that generatively outputs text, for example.

[0115] A "prompt format" refers to a text sentence that has been formatted in a specific way to be input into a generative AI model.

[0116] "Searching book data" refers to searching through book information in a database to find relevant books based on a specific search query.

[0117] "Detailed book information" refers to information about a specific book, including the title, author name, publication year, summary, etc.

[0118] "Search Results" refers to the list of relevant books and information that the system finds based on the user's input.

[0119] "Displaying to the user" refers to providing the organized search results to the user through the interface of the user terminal.

[0120] The present invention is a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. How this system works will be explained in detail below.

[0121] System configuration

[0122] The system consists of three main components:

[0123] User terminal

[0124] server

[0125] Book Database

[0126] User terminal

[0127] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[0128] server

[0129] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[0130] 1. Data Reception

[0131] The server receives the ambiguous information sent from the user terminal, using a communication protocol (e.g., HTTPS) over the Internet.

[0132] 2. Data Analysis

[0133] The received data is analyzed and important keywords are extracted. Natural language processing techniques are used to tokenize the text data and obtain meaningful keywords. Specifically, Python's natural language processing libraries "NLTK" and "spaCy" are used.

[0134] 3. Search Query Generation

[0135] A search query is generated based on the extracted keywords. This query is converted into a format that can be input into a generative AI model. For example, "GPT-3" can be used as the generative AI model.

[0136] 4. Book database search

[0137] Based on the generated query, the generative AI model searches a book database and lists relevant books from the information in the database. The book database contains a large amount of book information, including detailed information such as book title, author, genre, and summary.

[0138] 5. Organizing search results

[0139] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0140] 6. Data Transmission

[0141] The organized search results are sent to the user's terminal, which receives the data and displays it to the user.

[0142] Specific examples

[0143] Example 1: Searching for moving adventure stories

[0144] 1. User Device

[0145] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[0146] 2. Server

[0147] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0148] Based on the extracted keywords, the generative AI model searches a book database and lists books such as "The Alchemist" and "Life of Pi."

[0149] The search results are organized and the list is sent to the user's terminal.

[0150] 3. User Device

[0151] The search results are presented to the user as a list of books from which the user can select books of interest.

[0152] Example 2: Looking for an easy-to-understand children's book about science

[0153] 1. User Device

[0154] A user types, "I'm looking for an easy-to-understand children's book about science."

[0155] 2. Server

[0156] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[0157] Based on the extracted keywords, the generative AI model searches for the most suitable books, listing titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0158] The organized results are sent to the user terminal.

[0159] 3. User Device

[0160] A list of search results is displayed to the user to help them select the most suitable book.

[0161] The above is a specific example of the "Mode for Carrying Out the Invention." This system allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve convenience for elderly people and young children when searching for books.

[0162] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0163] Step 1:

[0164] User input:

[0165] The user uses the device interface to input vague information or keywords, for example, "I'm looking for a book that is an adventure story but also has an inspiring plot." The input data are vague keywords or phrases related to the book the user is looking for.

[0166] Step 2:

[0167] Sending data:

[0168] The terminal sends the text data entered by the user to the server using the HTTPS protocol. This data includes the user's search request.

[0169] Step 3:

[0170] Receiving data:

[0171] The server receives the data sent from the terminal, which is an ambiguous keyword or phrase entered by the user.

[0172] Step 4:

[0173] Data analysis:

[0174] The server analyzes the received data. Specifically, it uses the Python natural language processing library "NLTK" or "spaCy" to tokenize the text data and extract meaningful keywords. For example, the keywords "adventure," "inspirational," and "story" are extracted from the text "A book that is both an adventure story and an inspiring one."

[0175] Step 5:

[0176] Search query generation:

[0177] The server generates a search query based on the extracted keywords. This query is converted into a format that can be input to the generative AI model. For example, by combining the keywords "adventure," "emotional," and "story," the prompt sentence "Please tell me a book that is both an adventure story and an emotional story" is generated.

[0178] Step 6:

[0179] Book database search:

[0180] The server searches a book database using a generative AI model (e.g., GPT-3) based on the generated query. The generative AI model lists relevant books based on the input prompt. The database contains detailed information such as book title, author, genre, and summary.

[0181] Step 7:

[0182] Search result organization:

[0183] The server organizes the search results and generates data to display them in a user-friendly format, including information such as the title, author, publication year, and a brief synopsis. For example, the server retrieves book information such as "The Alchemist," "Paulo Coelho," "1988," and "An adventure story that shows the journey of life."

[0184] Step 8:

[0185] Submit search results:

[0186] The server sends the organized search results to the user terminal, which receives the data and displays it to the user.

[0187] Step 9:

[0188] Displaying search results:

[0189] The user terminal displays the search results sent from the server to the user. The user can then select books that interest them based on the displayed search results. For example, a list of books such as "The Alchemist" and "Life of Pi" may be displayed.

[0190] (Application example 1)

[0191] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] When users search for a book, it is difficult to quickly and accurately find the desired book even if they only have vague information. It is also difficult to immediately identify where the book is actually located in the bookstore, which makes it inconvenient for a wide range of users, including the elderly and young children. To solve this problem, a system is needed that can accurately identify a book even with vague information and easily find where that book is located in the bookstore.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0194] In this invention, the server includes means for receiving vague information or keywords entered by a user, means for automatically searching for related book data based on the vague information or keywords, means for displaying search results to the user, means for identifying the location of the book in the bookstore based on the search results, and means for displaying a route to the identified location. This allows the user to accurately identify a book even with vague information, and then easily find where it is located in the bookstore.

[0195] "Ambiguous information" refers to information that is general or vague and not specifically defined by the user.

[0196] "Keywords" are specific words or short phrases that are necessary to conduct a search for a book.

[0197] "Natural language processing technology" is a technology that enables computers to understand, interpret, and generate human language, and is used to analyze ambiguous user input.

[0198] "Book data" is data consisting of information such as the book title, author, genre, and summary.

[0199] "Search results" are a collection of related book data that are automatically searched based on the user's vague information or keywords.

[0200] A "user terminal" is a device used by a user to input information, including a smartphone or tablet.

[0201] "Server" refers to a central processing unit that receives data sent from user terminals and analyzes and processes it.

[0202] The "location of a book in a bookstore" is the specific location where a particular book is located in a bookstore.

[0203] "Displaying a route" means visually showing a customer the route they should take to reach a particular book within the bookstore.

[0204] The present invention provides a system for effectively searching for books in a bookstore. A specific embodiment for implementing this system will be described below.

[0205] System configuration

[0206] The system consists of three main components:

[0207] User terminal

[0208] server

[0209] Book Database

[0210] User terminal

[0211] A user terminal is a device that allows a user to input information and receive results. Examples include smartphones and tablets. Here, we will use a smartphone as an example. A user launches an app on the smartphone and uses the voice input function to input vague information. For example, a user might input a search request such as, "I want a book with a scary but moving story."

[0212] server

[0213] The server receives the ambiguous information sent from the user terminal and analyzes and processes it. Specifically, it uses the following hardware and software:

[0214] Server hardware: general data center servers and cloud servers

[0215] Software: Google Cloud Natural Language API, OpenAI GPT-4 model

[0216] Processing flow

[0217] 1. Data Reception

[0218] The server receives the voice data sent from the user's device, which is converted into text using the Google Cloud Speech-to-Text API.

[0219] 2. Data Analysis

[0220] The converted text data is then analyzed using the Google Cloud Natural Language API to extract meaningful keywords, such as "scary," "moving," and "story."

[0221] 3. Search Query Generation

[0222] Based on the extracted keywords, a search query is generated by generative AI (OpenAI GPT-4 model). For example, a prompt sentence such as "Generate a search query for a book that is scary yet emotional for bookstore inventory search" is input.

[0223] 4. Book database search

[0224] The generated query is then used to search a book database, which contains detailed information such as the book's title, author, genre, and summary, to produce a list of related books.

[0225] 5. Organizing search results

[0226] Search results are organized and presented in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0227] 6. Data Transmission

[0228] The organized search results are sent to the user's terminal and displayed on the screen.

[0229] 7. Bookstore location and route guidance

[0230] When a user selects a particular book, the location of that book in the bookstore is identified and displayed on the smartphone, and a store map is also displayed, providing directions to the selected book's location.

[0231] Specific examples

[0232] Example 1: Looking for a book with a scary but moving story

[0233] 1. User Device

[0234] A user types, "I want a book with scary but moving stories."

[0235] 2. Server

[0236] The server receives this vague information and converts it into text using the Google Cloud Speech-to-Text API.

[0237] Using the Google Cloud Natural Language API, keywords such as "scary," "moving," and "story" are extracted.

[0238] Based on the extracted keywords, the generative AI (OpenAI GPT-4 model) inputs the prompt sentence "Generate a search query for a book that is scary yet emotional for bookstore inventory search" and generates a search query.

[0239] A book database is searched based on the generated query, and related books are listed.

[0240] The search results are organized and sent to the user's terminal.

[0241] 3. User Device

[0242] The system presents the user with a list of books displayed as search results, and also displays the location and route to the selected book in the store, providing guidance to the specific location.

[0243] In this way, the present invention provides a system that can accurately identify a book and easily locate its location in a bookstore, even if the user has only vague information, making it possible for a wide range of users, including the elderly and young children, to easily search for books.

[0244] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0245] Step 1:

[0246] A user launches a smartphone application and uses the voice input function to enter a vague search request, specifically, "I want a book with a scary but moving story."

[0247] Input: User's voice data.

[0248] Output: Audio data collected through the smartphone microphone.

[0249] Specific behavior: The smartphone application receives voice input and prepares it to be sent to the system.

[0250] Step 2:

[0251] The device sends the voice data to the server, which converts it into text using the Google Cloud Speech-to-Text API.

[0252] Input: Audio data sent from a smartphone.

[0253] Output: Text data (e.g., "I want a book with a scary but moving story").

[0254] Specific operation: The server receives the audio data and converts it into text using the Google Cloud Speech-to-Text API.

[0255] Step 3:

[0256] The server analyzes the text data using the Google Cloud Natural Language API and extracts important keywords.

[0257] Input: Text data converted from audio data.

[0258] Output: A set of keywords (e.g. "scary", "emotional", "story").

[0259] Specific operation: The server analyzes the text data and performs tokenization, stop word removal, and keyword extraction.

[0260] Step 4:

[0261] The server inputs a prompt into the generative AI (OpenAI GPT-4 model) to generate a search query. Example prompt: "Generate a search query for a book that is scary yet emotional for bookstore inventory search."

[0262] Input: A set of keywords.

[0263] Output: The search query created by the generative AI.

[0264] Specific operation: The server inputs a prompt sentence into the generation AI to generate an appropriate search query.

[0265] Step 5:

[0266] The server searches a book database using the generated search query.

[0267] Input: A search query generated by the generative AI.

[0268] Output: A list of related books.

[0269] Specific operation: The server executes a search query against a book database and lists related books.

[0270] Step 6:

[0271] The server organizes the search results and summarizes important information (title, author, summary, etc.).

[0272] Input: A list of books.

[0273] Output: An organized list of book information.

[0274] Specific operation: The server collects book data and organizes it into a format that is easy for users to understand.

[0275] Step 7:

[0276] The server sends the organized search results to the user's terminal, which displays the information.

[0277] Input: An organized list of book information.

[0278] Output: Search results displayed on a smartphone screen.

[0279] Specific operation: The server sends the data and the smartphone displays the results on the screen.

[0280] Step 8:

[0281] When a user selects a particular book, the server identifies the book's location in the bookstore and sends that information to the user's device, which then displays a store map and route guidance on the smartphone.

[0282] Input: The book selected by the user.

[0283] Output: Book location and directions.

[0284] Specific operation: The server queries the book location information, generates route guidance, and sends it to the user's device. The smartphone displays a map of the store and provides route guidance.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] ---

[0287] The present invention provides a system that enables a user to effectively search for books using vague information or keywords even when the user does not know the specific title or author of the book, and further combines it with an emotion engine that recognizes the user's emotions and optimizes the search results.

[0288] System configuration

[0289] The system consists of four main components:

[0290] User terminal

[0291] server

[0292] Book Database

[0293] Emotion Engine

[0294] User terminal

[0295] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for a book that is an adventure story but has an inspiring story." In response to this input, the system collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[0296] server

[0297] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[0298] 1. Data Reception

[0299] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[0300] 2. Data Analysis

[0301] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[0302] 3. Search Query Generation

[0303] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into the generative AI.

[0304] 4. Book database search

[0305] The generative AI searches a book database based on the generated search query, taking into account the user's emotional state and identifying highly relevant books.

[0306] 5. Search Result Tailoring

[0307] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[0308] 6. Submit search results

[0309] The organized search results are sent to the user's terminal.

[0310] Emotion Engine

[0311] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[0312] Audio analysis

[0313] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[0314] facial expression analysis

[0315] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[0316] Specific examples

[0317] Example 1: Searching for moving adventure stories

[0318] 1. User Device

[0319] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[0320] 2. Server

[0321] The server receives this vague information, uses natural language processing technology to extract keywords such as "adventure," "emotional," and "story," and uses an emotion engine to analyze that the user is looking for something moving.

[0322] The generative AI searches a book database based on keywords and emotional state to produce a list of books such as "The Alchemist" and "Life of Pi."

[0323] Organize search results to prioritize inspiring stories at the top of the list.

[0324] 3. User Device

[0325] The search results list is displayed to the user, who then selects the desired book.

[0326] Example 2: Looking for an easy-to-understand children's book about science

[0327] 1. User Device

[0328] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[0329] 2. Server

[0330] The server receives this information, extracts keywords such as "science," "easy to understand," and "children's books," and uses an emotion engine to analyze whether the user is expressing curiosity.

[0331] Generative AI searches for the best books based on keywords and emotional state, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0332] It reflects your emotional state and prioritizes content that will interest you.

[0333] 3. User Device

[0334] A list of search results is displayed to the user, and the user can select books that interest them.

[0335] This invention combines an emotion engine and generative AI to provide a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This will greatly improve convenience for people, especially the elderly and young children, when searching for books.

[0336] The processing flow will be explained below.

[0337] ---

[0338] Step 1:

[0339] The user terminal displays an input interface for book searches, which includes a text input box, a voice input function, and a camera for face recognition.

[0340] Step 2:

[0341] The user inputs vague information or keywords about the book they are looking for. For example, they input vague criteria such as "a book that is an adventure story but also has an inspiring story" by voice or text. During this process, the user's voice and facial expression data are collected in real time.

[0342] Step 3:

[0343] The user terminal formats the input information from the user and sends it to the server along with emotion data. At this time, the data is encrypted and sent securely.

[0344] Step 4:

[0345] The server receives the ambiguous information and emotion data sent from the user terminal, and the received data is temporarily stored in a database.

[0346] Step 5:

[0347] The server analyzes the received fuzzy information using natural language processing techniques, which involves tokenizing the text and extracting key keywords.

[0348] Step 6:

[0349] The server uses an emotion engine to analyze the transmitted emotion data, identifying emotions from the voice data based on tone, speed, emphasis, etc., and from the facial expression data, determining the emotional state based on the movements and changes of facial muscles.

[0350] Step 7:

[0351] The server generates a search query based on the analyzed keywords and sentiment data, which is then converted into a format that the generative AI can understand.

[0352] Step 8:

[0353] The server uses a generation AI to search the book database, which takes into account the user's emotions and identifies highly relevant books from the database.

[0354] Step 9:

[0355] The server then performs a process of organizing the search results, listing data including title, author, publication year, and brief summary, and adjusting the display order to reflect the user's emotional state.

[0356] Step 10:

[0357] The server sends the organized search results to the user terminal, where the data is encrypted for efficient transmission.

[0358] Step 11:

[0359] The user terminal receives the search results sent from the server, analyzes the received data, and displays them to the user in a visually easy-to-understand list format.

[0360] Step 12:

[0361] The user selects the book they are interested in from the presented book list, and detailed information about the selected book and related further information is also displayed.

[0362] Examples:

[0363] Example 1: Searching for moving adventure stories

[0364] Step 1:

[0365] The user terminal displays an input interface.

[0366] Step 2:

[0367] The user speaks, "I'm looking for a book that is an adventure story but also has an inspiring plot," and voice and facial expression data are also collected.

[0368] Step 3:

[0369] The user terminal transmits the input information and emotion data to the server.

[0370] Step 4:

[0371] The server receives the ambiguous information and the emotion data.

[0372] Step 5:

[0373] The server uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0374] Step 6:

[0375] The server uses an emotion engine to analyze the strong desire for emotion.

[0376] Step 7:

[0377] The server generates a search query based on keywords and emotion data.

[0378] Step 8:

[0379] The server uses generative AI to search a book database and list "The Alchemist" and "Life of Pi."

[0380] Step 9:

[0381] The server prioritizes inspiring stories and displays them at the top of the list.

[0382] Step 10:

[0383] The server transmits the organized search results to the user terminal.

[0384] Step 11:

[0385] The user terminal receives and displays the search results.

[0386] Step 12:

[0387] The user selects the desired book from the displayed list.

[0388] ---

[0389] The above are the specific processing steps of the book search system that combines the emotion engine.

[0390] Example 2

[0391] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] In conventional book search systems, if a user does not know the specific book title or author name, it is difficult to find the appropriate book when entering vague keywords or information. Furthermore, since search results are provided without taking the user's emotional state into consideration, users may not be able to achieve the emotional satisfaction they are looking for. This makes it particularly difficult for elderly people and young children to search for books efficiently.

[0393] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0394] In this invention, the server includes means for analyzing ambiguous information and keywords entered by the user and the user's emotional data using natural language processing technology, means for searching a book database using a generative AI model, and means for adjusting and displaying search results according to the user's emotional state. This enables the user to effectively find the most suitable book according to their emotional state by simply entering ambiguous information and keywords, even if they do not know the specific information.

[0395] "Ambiguous information and keywords" refers to vague search criteria that a user enters when they do not know the specific title or author of a book.

[0396] "Emotional data" refers to information about the user's psychological state and emotions obtained by analyzing their voice and facial expressions.

[0397] "Natural language processing technology" refers to artificial intelligence technology for understanding, analyzing, and generating human language.

[0398] A "generative AI model" is a type of artificial intelligence that is trained in advance using large amounts of data and generates sentences and search results according to specific conditions.

[0399] A "book database" refers to a collection of data that aggregates information about books.

[0400] An "emotion engine" refers to a technology or system that analyzes data such as voice and facial expressions to recognize a user's emotions.

[0401] "Analysis results" refers to keywords extracted from input information using natural language processing technology and the emotional state recognized by the emotion engine.

[0402] "Search Results" refers to a list of relevant books searched through a book database using a generative AI model.

[0403] This invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. In addition, by combining it with an emotion engine, search results can be optimized based on the user's emotions.

[0404] System configuration

[0405] The system consists of the following main components:

[0406] User terminal

[0407] server

[0408] Book Database

[0409] Emotion Engine

[0410] Processing flow

[0411] User terminal

[0412] A user terminal is a device into which a user inputs information or keywords. Examples include a smartphone, tablet, or PC. When a user inputs, "I'm looking for a book that is an adventure story but also has an inspiring plot," the terminal uses voice input and a camera to collect the user's emotional data. This emotional data includes voice tone and facial expressions. Hardware such as a microphone and a camera are used. Software such as Google Speech-to-Text and OpenCV are used.

[0413] server

[0414] The server receives and temporarily stores ambiguous information and emotional data sent from the user's device. The server has the following functions:

[0415] Data analysis

[0416] The server uses TensorFlow to execute natural language processing techniques to extract important keywords from ambiguous information, such as "adventure," "moving," and "story."

[0417] At the same time, an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) is used to analyze the user's emotions from voice and facial expression data.

[0418] Search Query Generation

[0419] A prompt sentence is generated based on the extracted keywords and the analyzed sentiment data. This prompt sentence is configured as an input to a generative AI model (such as GPT-4). For example, "Please recommend the best book for an excited user looking for an inspiring adventure story."

[0420] Database search

[0421] The generative AI model searches a book database based on the generated prompt. This book database is configured using a database management system such as MySQL. The generative AI model lists highly relevant books based on the search results.

[0422] Tailoring and Submitting Search Results

[0423] The server will adjust search results based on the user's emotional state, for example, if the user is in a state where they want inspiring stories, it will prioritize books that tell such stories.

[0424] The adjusted search results are transmitted to the user terminal.

[0425] Emotion Engine

[0426] The emotion engine is a component that recognizes emotions from the user's voice and facial expression data. It has the following functions:

[0427] Audio analysis

[0428] Use speech analysis software (e.g., Nuance or Google Speech-to-Text) to analyze speech tone, speed, pauses, etc. to identify emotion.

[0429] facial expression analysis

[0430] Using facial expression analysis software (e.g., OpenCV or Microsoft Azure Face API), the system analyzes facial muscle movements and changes in facial expressions to recognize the user's emotions.

[0431] Specific examples

[0432] Example 1: Searching for moving adventure stories

[0433] User terminal

[0434] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[0435] server

[0436] 1. Receive vague information and use natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0437] 2. The emotion engine analyzes the user's emotions and determines that they are "seeking emotion."

[0438] 3. A generative AI model searches a book database based on keywords and emotional state to produce a list of books like "The Alchemist" and "Life of Pi."

[0439] 4. Organize search results to prioritize inspiring stories.

[0440] User terminal

[0441] The search results list is displayed to the user, who then selects the desired book.

[0442] Example 2: Looking for an easy-to-understand children's book about science

[0443] User terminal

[0444] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[0445] server

[0446] 1. Receive information and extract keywords such as "science," "easy to understand," and "children's book."

[0447] 2. The emotion engine analyzes the user's emotion as "showing curiosity."

[0448] 3. The generative AI model searches for suitable books and lists titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0449] 4. Reflect your emotional state and prioritize content that will interest you.

[0450] User terminal

[0451] A list of search results is displayed to the user, and the user can select books that interest them.

[0452] This invention provides a system that allows users who do not know specific information to easily find the most suitable book according to their emotional state by simply entering vague information or keywords, greatly improving the convenience of book searches, especially for the elderly and young children.

[0453] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0454] Step 1:

[0455] The user operates the device and inputs vague search criteria using text or voice. For example, they might input, "I'm looking for a book that is an adventure story but also has an inspiring plot." At the same time, the device uses a camera to capture the user's facial expressions and a microphone to record their voice. The inputs here are the user's text / voice input and emotional data. The device then collects the user's facial expression and voice data and sends it to the server.

[0456] Step 2:

[0457] The server receives and temporarily stores ambiguous information and emotion data sent from the user's device. The input data is the user's text / voice input and emotion data. Based on this, the server stores the data in a database and prepares it for subsequent analysis.

[0458] Step 3:

[0459] The server uses TensorFlow to execute natural language processing techniques and extract important keywords from ambiguous information. The input data is text / voice input provided by the user. The data is processed by tokenizing the text and extracting important keywords (e.g., "adventure," "moving," "story"). The output is the extracted keywords.

[0460] Step 4:

[0461] The server uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze emotions from the user's voice and facial expression data. The input data is the user's emotional data (voice tone and facial muscle movements). Data calculations identify the user's emotional state (e.g., "desiring to be moved"). The output is the analyzed emotional information.

[0462] Step 5:

[0463] The server generates a prompt sentence to be input to the generative AI model (GPT-4) based on the extracted keywords and analyzed emotional data. The input data is keywords and emotional state. Based on this, a prompt sentence such as "Recommend the best book for an excited user looking for an inspiring adventure story" is created as a search query. The output is the generated prompt sentence.

[0464] Step 6:

[0465] The generative AI model searches a book database based on the generated prompt. The input data is the prompt. A database search is performed to generate a list of highly relevant books. For example, search results may include "The Alchemist" and "Life of Pi." The output is a list of books.

[0466] Step 7:

[0467] The server adjusts search results based on the user's emotional state. The input data is a book list and the user's emotional state. The server processes the data to adjust the results and reorganizes the list to prioritize moving stories. The output is the adjusted book list.

[0468] Step 8:

[0469] The server sends the adjusted search results to the user terminal. The input data is the adjusted book list. Based on this, the server transfers the search results to the user terminal. The output is the data sent to the user terminal.

[0470] Step 9:

[0471] The user terminal receives the search results sent from the server and displays them on the screen. The input data is the adjusted book list. Based on this, the search results are visually presented to the user. The user can then select the book they want from the displayed list.

[0472] (Application example 2)

[0473] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0474] In modern book search systems, if a user does not know the specific title or author name, it is difficult to effectively search for a book using vague information or keywords. Furthermore, conventional systems do not optimize search results by taking into account the user's emotional state, making it difficult for users to find the book they want. Therefore, it is necessary to provide a book search system that reflects emotional data based on the user's vague information and keywords.

[0475] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0476] In this invention, the server includes means for receiving ambiguous information and keywords input by a user, means for automatically searching for related book data based on the ambiguous information and keywords, means for searching for the book data based on a generative AI model, means for inputting a prompt sentence to the generative AI model to obtain search results, means for optimizing the search results in consideration of the user's emotional data, and means for displaying the optimized search results to the user. This enables an optimal book search that reflects the user's emotional data based on the user's ambiguous information and keywords.

[0477] "Ambiguous information" refers to unclear keywords or fragmentary information that is entered when the user does not know the specific title or author's name.

[0478] A "keyword" refers to a specific word or short phrase that a user enters when searching, and is information that serves as a criterion for narrowing down search results.

[0479] A "generative AI model" is a system that uses artificial intelligence technology to generate responses based on specific search queries and prompts.

[0480] A "prompt" is a search query or instruction given to a generative AI model, which functions as a parameter to obtain appropriate search results.

[0481] "Emotion data" is data that indicates the emotional state of the user obtained from the user's voice and facial expressions, and is a quantitative evaluation of emotions such as joy and sadness.

[0482] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is a technique for unconsciously processing ambiguous information and keywords.

[0483] "Search results" are lists of related books presented by the system based on vague information, keywords, and emotional data entered by the user.

[0484] "Optimization" means taking emotional data into account and adjusting search results to best meet the user's needs and emotions.

[0485] "Display means" means an interface device for visually presenting search results and other related information to a user.

[0486] This system allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. Furthermore, by combining it with an emotion engine that recognizes the user's emotions and optimizes search results, it can recommend the most relevant books to the user.

[0487] System configuration

[0488] This system is broadly composed of the following components:

[0489] User terminal

[0490] server

[0491] Generative AI Models

[0492] Book Database

[0493] Emotion Engine

[0494] User terminal

[0495] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for adventure stories." In response to this input, the terminal collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[0496] server

[0497] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[0498] 1. Data Reception

[0499] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[0500] 2. Data Analysis

[0501] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[0502] 3. Search Query Generation

[0503] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into a generative AI model.

[0504] 4. Book database search

[0505] The generative AI model searches a book database based on the generated search query, taking into account the user's emotional state to identify highly relevant books.

[0506] 5. Search Result Tailoring

[0507] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[0508] 6. Submit search results

[0509] The organized search results are sent to the user's terminal.

[0510] Emotion Engine

[0511] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[0512] Audio analysis

[0513] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[0514] facial expression analysis

[0515] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[0516] Hardware and software used

[0517] User device: Smartphone (iPhone or Android device)

[0518] Analysis software:

[0519] Speech Recognition: Google Speech Recognition API (using the speech_recognition library)

[0520] Facial expression analysis: facial_recognition library

[0521] Sentiment Analysis: TextBlob (Sentiment analysis using natural language processing)

[0522] Server side:

[0523] REST API (e.g. bookrstore.com API)

[0524] Book Database

[0525] Examples and prompts

[0526] Examples:

[0527] A user launches a smartphone app and inputs "memorable." The user then selects "joy" as the emotion. Based on this information, books such as "The Alchemist" are recommended.

[0528] Prompt statement:

[0529] User-entered keyword: "memorable"

[0530] Emotion data: {"joy": 0.9, "sadness": 0.1}

[0531] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0532] Step 1:

[0533] The user uses a smartphone to input vague information or keywords. The user also simultaneously inputs emotional data using voice input or a camera. The input at this stage consists of text keywords and audio / image data.

[0534] Step 2:

[0535] The user's device sends the entered vague information or keywords, as well as audio and image data, to the server, where the audio data is converted into text using speech recognition software and the image data is analyzed using facial expression analysis software.

[0536] Step 3:

[0537] The server receives vague information, keywords, and emotion data sent from the user terminal. The received data is temporarily stored in a database. The input data is keyword text and emotion data.

[0538] Step 4:

[0539] The server uses natural language processing technology to analyze ambiguous information and extract important keywords. The server performs text analysis using TextBlob. It also uses an emotion engine to identify emotions from voice data and facial expression data. The analyzed keywords and emotion data are output.

[0540] Step 5:

[0541] The server generates a search query based on the analyzed keywords and sentiment data. This query is converted into a prompt sentence to be input to the generative AI model. The generated prompt sentence is output.

[0542] Step 6:

[0543] The server inputs a prompt into the generative AI model and searches the book database. The generative AI model retrieves relevant book data based on the prompt, and outputs the search results.

[0544] Step 7:

[0545] The server organizes the search results and optimizes them based on the emotional data recognized by the emotion engine. For example, if a user is looking for an inspiring story, relevant books will be prioritized in the list. The optimized search results are output.

[0546] Step 8:

[0547] The server sends the optimized search results to the user's device, which visually displays the results and presents them in a user-selectable format. The output at this stage is a list of related books and their detailed information.

[0548] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0549] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0550] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0551] [Second embodiment]

[0552] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0553] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0554] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0555] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0556] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0557] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0558] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0559] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0560] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0561] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0562] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0563] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0564] ---

[0565] The present invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. Below we will explain how this system works.

[0566] System configuration

[0567] The system consists of three main components:

[0568] User terminal

[0569] server

[0570] Book Database

[0571] User terminal

[0572] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[0573] server

[0574] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[0575] 1. Data Reception

[0576] The server receives the ambiguous information sent from the user terminal.

[0577] 2. Data Analysis

[0578] Analyzes received data and extracts important keywords. Natural language processing technology is used to tokenize text data and obtain meaningful keywords.

[0579] 3. Search Query Generation

[0580] Based on the extracted keywords, a search query is generated, which is then converted into a format that can be input to the generation AI.

[0581] 4. Book database search

[0582] Based on the generated query, the generative AI searches a book database and lists highly relevant books based on the information in the database.

[0583] 5. Organizing search results

[0584] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0585] 6. Data Transmission

[0586] The organized search results are sent to the user's terminal.

[0587] Book Database

[0588] The book database contains information about a huge number of books, which the server uses to find the most suitable book based on a search query. This database contains detailed information such as the book title, author, genre, and summary.

[0589] Specific examples

[0590] Example 1: Searching for moving adventure stories

[0591] 1. User Device

[0592] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[0593] 2. Server

[0594] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0595] The generative AI searches a book database based on the extracted keywords and lists books such as "The Alchemist" and "Life of Pi."

[0596] The resulting list is compiled and sent to the user's terminal.

[0597] 3. User Device

[0598] The search results are presented to the user as a list of books, from which the user can select books of interest.

[0599] Example 2: Looking for an easy-to-understand children's book about science

[0600] 1. User Device

[0601] A user types, "I'm looking for an easy-to-understand children's book about science."

[0602] 2. Server

[0603] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[0604] The generative AI searches for the best books based on these keywords, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0605] The organized results are sent to the user terminal.

[0606] 3. User Device

[0607] A list of search results is displayed to the user to help them select the most suitable book.

[0608] In this way, the present invention provides a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve the convenience of searching for books, especially for the elderly and young children.

[0609] The processing flow will be explained below.

[0610] ---

[0611] Step 1:

[0612] The user terminal displays an input interface for book searches to the user, which is intuitive and easy to operate, and is designed so that even the elderly and young children can use it.

[0613] Step 2:

[0614] Users input vague information or keywords about the book they are looking for, such as "a book that is an adventure story but also has an inspiring story."

[0615] Step 3:

[0616] The user terminal formats the ambiguous information entered by the user and sends it to the server, where the data has been converted into the appropriate format.

[0617] Step 4:

[0618] The server receives the ambiguous information sent from the user terminal and temporarily stores the received data.

[0619] Step 5:

[0620] The server then analyzes the received ambiguous information using natural language processing techniques, specifically tokenizing the text and extracting meaningful keywords.

[0621] Step 6:

[0622] The server generates a search query based on the analyzed keywords, which is then converted into a format that the AI ​​can understand.

[0623] Step 7:

[0624] The server uses a generation AI to search the book database, which references a large amount of book data and identifies books that are highly relevant.

[0625] Step 8:

[0626] The server organizes the generated search results and converts them into a user-friendly format, including, for example, book title, author, publication year, and a brief summary.

[0627] Step 9:

[0628] The server then sends the organized search results to the user's device, where the data is properly encrypted and securely transmitted.

[0629] Step 10:

[0630] The user terminal receives the search results sent from the server and displays them to the user in a visually easy-to-understand list format.

[0631] Step 11:

[0632] The user selects the book of interest from the displayed list of books, which may include more detailed information or related books.

[0633] ---

[0634] The above are the specific processing steps of the program.

[0635] Example 1

[0636] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional book search systems have the problem that it is difficult to search when you do not know the specific title or author name, or when you search based on only vague information. In addition, there is a lack of means to display search results in a format that is easy for users to understand, which makes them particularly difficult to use for the elderly and young children.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0639] In this invention, the server includes means for receiving ambiguous information or keywords entered by a user, means for analyzing the ambiguous information or keywords using natural language processing technology, and means for generating a search query based on the ambiguous information or keywords. This allows a user to easily find a desired book by simply entering ambiguous information or keywords, even if they do not know the specific information.

[0640] The server further includes a means for searching for related book data using the generative AI model based on the generated search query, and a means for organizing detailed information about books based on the search results and displaying it to the user. This makes it possible to provide a system that displays search results in an efficient and easy-to-understand format and is easy to use, especially for the elderly and young children.

[0641] "Ambiguous information or keywords entered by the user" refers to unclear information entered when the user does not clearly know specific information, or keywords used to identify the search target.

[0642] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and process human language, and includes technology for tokenizing text data and performing semantic analysis.

[0643] "Search query" means a sentence or phrase that describes a request to a search engine or database to find specific information.

[0644] A "generative AI model" is an artificial intelligence model that makes predictions and generates based on input data, and refers to an AI that generatively outputs text, for example.

[0645] A "prompt format" refers to a text sentence that has been formatted in a specific way to be input into a generative AI model.

[0646] "Searching book data" refers to searching through book information in a database to find relevant books based on a specific search query.

[0647] "Detailed book information" refers to information about a specific book, including the title, author name, publication year, summary, etc.

[0648] "Search Results" refers to the list of relevant books and information that the system finds based on the user's input.

[0649] "Displaying to the user" refers to providing the organized search results to the user through the interface of the user terminal.

[0650] The present invention is a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. How this system works will be explained in detail below.

[0651] System configuration

[0652] The system consists of three main components:

[0653] User terminal

[0654] server

[0655] Book Database

[0656] User terminal

[0657] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[0658] server

[0659] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[0660] 1. Data Reception

[0661] The server receives the ambiguous information sent from the user terminal, using a communication protocol (e.g., HTTPS) over the Internet.

[0662] 2. Data Analysis

[0663] The received data is analyzed and important keywords are extracted. Natural language processing techniques are used to tokenize the text data and obtain meaningful keywords. Specifically, Python's natural language processing libraries "NLTK" and "spaCy" are used.

[0664] 3. Search Query Generation

[0665] A search query is generated based on the extracted keywords. This query is converted into a format that can be input into a generative AI model. For example, "GPT-3" can be used as the generative AI model.

[0666] 4. Book database search

[0667] Based on the generated query, the generative AI model searches a book database and lists relevant books from the information in the database. The book database contains a large amount of book information, including detailed information such as book title, author, genre, and summary.

[0668] 5. Organizing search results

[0669] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0670] 6. Data Transmission

[0671] The organized search results are sent to the user's terminal, which receives the data and displays it to the user.

[0672] Specific examples

[0673] Example 1: Searching for moving adventure stories

[0674] 1. User Device

[0675] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[0676] 2. Server

[0677] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0678] Based on the extracted keywords, the generative AI model searches a book database and lists books such as "The Alchemist" and "Life of Pi."

[0679] The search results are organized and the list is sent to the user's terminal.

[0680] 3. User Device

[0681] The search results are presented to the user as a list of books from which the user can select books of interest.

[0682] Example 2: Looking for an easy-to-understand children's book about science

[0683] 1. User Device

[0684] A user types, "I'm looking for an easy-to-understand children's book about science."

[0685] 2. Server

[0686] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[0687] Based on the extracted keywords, the generative AI model searches for the most suitable books, listing titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0688] The organized results are sent to the user terminal.

[0689] 3. User Device

[0690] A list of search results is displayed to the user to help them select the most suitable book.

[0691] The above is a specific example of the "Mode for Carrying Out the Invention." This system allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve convenience for elderly people and young children when searching for books.

[0692] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0693] Step 1:

[0694] User input:

[0695] The user uses the device interface to input vague information or keywords, for example, "I'm looking for a book that is an adventure story but also has an inspiring plot." The input data are vague keywords or phrases related to the book the user is looking for.

[0696] Step 2:

[0697] Sending data:

[0698] The terminal sends the text data entered by the user to the server using the HTTPS protocol. This data includes the user's search request.

[0699] Step 3:

[0700] Receiving data:

[0701] The server receives the data sent from the terminal, which is an ambiguous keyword or phrase entered by the user.

[0702] Step 4:

[0703] Data analysis:

[0704] The server analyzes the received data. Specifically, it uses the Python natural language processing library "NLTK" or "spaCy" to tokenize the text data and extract meaningful keywords. For example, the keywords "adventure," "inspirational," and "story" are extracted from the text "A book that is both an adventure story and an inspiring one."

[0705] Step 5:

[0706] Search query generation:

[0707] The server generates a search query based on the extracted keywords. This query is converted into a format that can be input to the generative AI model. For example, by combining the keywords "adventure," "emotional," and "story," the prompt sentence "Please tell me a book that is both an adventure story and an emotional story" is generated.

[0708] Step 6:

[0709] Book database search:

[0710] The server searches a book database using a generative AI model (e.g., GPT-3) based on the generated query. The generative AI model lists relevant books based on the input prompt. The database contains detailed information such as book title, author, genre, and summary.

[0711] Step 7:

[0712] Search result organization:

[0713] The server organizes the search results and generates data to display them in a user-friendly format, including information such as the title, author, publication year, and a brief synopsis. For example, the server retrieves book information such as "The Alchemist," "Paulo Coelho," "1988," and "An adventure story that shows the journey of life."

[0714] Step 8:

[0715] Submit search results:

[0716] The server sends the organized search results to the user terminal, which receives the data and displays it to the user.

[0717] Step 9:

[0718] Displaying search results:

[0719] The user terminal displays the search results sent from the server to the user. The user can then select books that interest them based on the displayed search results. For example, a list of books such as "The Alchemist" and "Life of Pi" may be displayed.

[0720] (Application example 1)

[0721] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0722] When users search for a book, it is difficult to quickly and accurately find the desired book even if they only have vague information. It is also difficult to immediately identify where the book is actually located in the bookstore, which makes it inconvenient for a wide range of users, including the elderly and young children. To solve this problem, a system is needed that can accurately identify a book even with vague information and easily find where that book is located in the bookstore.

[0723] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0724] In this invention, the server includes means for receiving vague information or keywords entered by a user, means for automatically searching for related book data based on the vague information or keywords, means for displaying search results to the user, means for identifying the location of the book in the bookstore based on the search results, and means for displaying a route to the identified location. This allows the user to accurately identify a book even with vague information, and then easily find where it is located in the bookstore.

[0725] "Ambiguous information" refers to information that is general or vague and not specifically defined by the user.

[0726] "Keywords" are specific words or short phrases that are necessary to conduct a search for a book.

[0727] "Natural language processing technology" is a technology that enables computers to understand, interpret, and generate human language, and is used to analyze ambiguous user input.

[0728] "Book data" is data consisting of information such as the book title, author, genre, and summary.

[0729] "Search results" are a collection of related book data that are automatically searched based on the user's vague information or keywords.

[0730] A "user terminal" is a device used by a user to input information, including a smartphone or tablet.

[0731] "Server" refers to a central processing unit that receives data sent from user terminals and analyzes and processes it.

[0732] The "location of a book in a bookstore" is the specific location where a particular book is located in a bookstore.

[0733] "Displaying a route" means visually showing a customer the route they should take to reach a particular book within the bookstore.

[0734] The present invention provides a system for effectively searching for books in a bookstore. A specific embodiment for implementing this system will be described below.

[0735] System configuration

[0736] The system consists of three main components:

[0737] User terminal

[0738] server

[0739] Book Database

[0740] User terminal

[0741] A user terminal is a device that allows a user to input information and receive results. Examples include smartphones and tablets. Here, we will use a smartphone as an example. A user launches an app on the smartphone and uses the voice input function to input vague information. For example, a user might input a search request such as, "I want a book with a scary but moving story."

[0742] server

[0743] The server receives the ambiguous information sent from the user terminal and analyzes and processes it. Specifically, it uses the following hardware and software:

[0744] Server hardware: general data center servers and cloud servers

[0745] Software: Google Cloud Natural Language API, OpenAI GPT-4 model

[0746] Processing flow

[0747] 1. Data Reception

[0748] The server receives the voice data sent from the user's device, which is converted into text using the Google Cloud Speech-to-Text API.

[0749] 2. Data Analysis

[0750] The converted text data is then analyzed using the Google Cloud Natural Language API to extract meaningful keywords, such as "scary," "moving," and "story."

[0751] 3. Search Query Generation

[0752] Based on the extracted keywords, a search query is generated by generative AI (OpenAI GPT-4 model). For example, a prompt sentence such as "Generate a search query for a book that is scary yet emotional for bookstore inventory search" is input.

[0753] 4. Book database search

[0754] The generated query is then used to search a book database, which contains detailed information such as the book's title, author, genre, and summary, to produce a list of related books.

[0755] 5. Organizing search results

[0756] Search results are organized and presented in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[0757] 6. Data Transmission

[0758] The organized search results are sent to the user's terminal and displayed on the screen.

[0759] 7. Bookstore location and route guidance

[0760] When a user selects a particular book, the location of that book in the bookstore is identified and displayed on the smartphone, and a store map is also displayed, providing directions to the selected book's location.

[0761] Specific examples

[0762] Example 1: Looking for a book with a scary but moving story

[0763] 1. User Device

[0764] A user types, "I want a book with scary but moving stories."

[0765] 2. Server

[0766] The server receives this vague information and converts it into text using the Google Cloud Speech-to-Text API.

[0767] Using the Google Cloud Natural Language API, keywords such as "scary," "moving," and "story" are extracted.

[0768] Based on the extracted keywords, the generative AI (OpenAI GPT-4 model) inputs the prompt sentence "Generate a search query for a book that is scary yet emotional for bookstore inventory search" and generates a search query.

[0769] A book database is searched based on the generated query, and related books are listed.

[0770] The search results are organized and sent to the user's terminal.

[0771] 3. User Device

[0772] The system presents the user with a list of books displayed as search results, and also displays the location and route to the selected book in the store, providing guidance to the specific location.

[0773] In this way, the present invention provides a system that can accurately identify a book and easily locate its location in a bookstore, even if the user has only vague information, making it possible for a wide range of users, including the elderly and young children, to easily search for books.

[0774] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0775] Step 1:

[0776] A user launches a smartphone application and uses the voice input function to enter a vague search request, specifically, "I want a book with a scary but moving story."

[0777] Input: User's voice data.

[0778] Output: Audio data collected through the smartphone microphone.

[0779] Specific behavior: The smartphone application receives voice input and prepares it to be sent to the system.

[0780] Step 2:

[0781] The device sends the voice data to the server, which converts it into text using the Google Cloud Speech-to-Text API.

[0782] Input: Audio data sent from a smartphone.

[0783] Output: Text data (e.g., "I want a book with a scary but moving story").

[0784] Specific operation: The server receives the audio data and converts it into text using the Google Cloud Speech-to-Text API.

[0785] Step 3:

[0786] The server analyzes the text data using the Google Cloud Natural Language API and extracts important keywords.

[0787] Input: Text data converted from audio data.

[0788] Output: A set of keywords (e.g. "scary", "emotional", "story").

[0789] Specific operation: The server analyzes the text data and performs tokenization, stop word removal, and keyword extraction.

[0790] Step 4:

[0791] The server inputs a prompt into the generative AI (OpenAI GPT-4 model) to generate a search query. Example prompt: "Generate a search query for a book that is scary yet emotional for bookstore inventory search."

[0792] Input: A set of keywords.

[0793] Output: The search query created by the generative AI.

[0794] Specific operation: The server inputs a prompt sentence into the generation AI to generate an appropriate search query.

[0795] Step 5:

[0796] The server searches a book database using the generated search query.

[0797] Input: A search query generated by the generative AI.

[0798] Output: A list of related books.

[0799] Specific operation: The server executes a search query against a book database and lists related books.

[0800] Step 6:

[0801] The server organizes the search results and summarizes important information (title, author, summary, etc.).

[0802] Input: A list of books.

[0803] Output: An organized list of book information.

[0804] Specific operation: The server collects book data and organizes it into a format that is easy for users to understand.

[0805] Step 7:

[0806] The server sends the organized search results to the user's terminal, which displays the information.

[0807] Input: An organized list of book information.

[0808] Output: Search results displayed on a smartphone screen.

[0809] Specific operation: The server sends the data and the smartphone displays the results on the screen.

[0810] Step 8:

[0811] When a user selects a particular book, the server identifies the book's location in the bookstore and sends that information to the user's device, which then displays a store map and route guidance on the smartphone.

[0812] Input: The book selected by the user.

[0813] Output: Book location and directions.

[0814] Specific operation: The server queries the book location information, generates route guidance, and sends it to the user's device. The smartphone displays a map of the store and provides route guidance.

[0815] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0816] ---

[0817] The present invention provides a system that enables a user to effectively search for books using vague information or keywords even when the user does not know the specific title or author of the book, and further combines it with an emotion engine that recognizes the user's emotions and optimizes the search results.

[0818] System configuration

[0819] The system consists of four main components:

[0820] User terminal

[0821] server

[0822] Book Database

[0823] Emotion Engine

[0824] User terminal

[0825] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for a book that is an adventure story but has an inspiring story." In response to this input, the system collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[0826] server

[0827] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[0828] 1. Data Reception

[0829] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[0830] 2. Data Analysis

[0831] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[0832] 3. Search Query Generation

[0833] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into the generative AI.

[0834] 4. Book database search

[0835] The generative AI searches a book database based on the generated search query, taking into account the user's emotional state and identifying highly relevant books.

[0836] 5. Search Result Tailoring

[0837] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[0838] 6. Submit search results

[0839] The organized search results are sent to the user's terminal.

[0840] Emotion Engine

[0841] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[0842] Audio analysis

[0843] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[0844] facial expression analysis

[0845] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[0846] Specific examples

[0847] Example 1: Searching for moving adventure stories

[0848] 1. User Device

[0849] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[0850] 2. Server

[0851] The server receives this vague information, uses natural language processing technology to extract keywords such as "adventure," "emotional," and "story," and uses an emotion engine to analyze that the user is looking for something moving.

[0852] The generative AI searches a book database based on keywords and emotional state to produce a list of books such as "The Alchemist" and "Life of Pi."

[0853] Organize search results to prioritize inspiring stories at the top of the list.

[0854] 3. User Device

[0855] The search results list is displayed to the user, who then selects the desired book.

[0856] Example 2: Looking for an easy-to-understand children's book about science

[0857] 1. User Device

[0858] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[0859] 2. Server

[0860] The server receives this information, extracts keywords such as "science," "easy to understand," and "children's books," and uses an emotion engine to analyze whether the user is expressing curiosity.

[0861] Generative AI searches for the best books based on keywords and emotional state, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0862] It reflects your emotional state and prioritizes content that will interest you.

[0863] 3. User Device

[0864] A list of search results is displayed to the user, and the user can select books that interest them.

[0865] This invention combines an emotion engine and generative AI to provide a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This will greatly improve convenience for people, especially the elderly and young children, when searching for books.

[0866] The processing flow will be explained below.

[0867] ---

[0868] Step 1:

[0869] The user terminal displays an input interface for book searches, which includes a text input box, a voice input function, and a camera for face recognition.

[0870] Step 2:

[0871] The user inputs vague information or keywords about the book they are looking for. For example, they input vague criteria such as "a book that is an adventure story but also has an inspiring story" by voice or text. During this process, the user's voice and facial expression data are collected in real time.

[0872] Step 3:

[0873] The user terminal formats the input information from the user and sends it to the server along with emotion data. At this time, the data is encrypted and sent securely.

[0874] Step 4:

[0875] The server receives the ambiguous information and emotion data sent from the user terminal, and the received data is temporarily stored in a database.

[0876] Step 5:

[0877] The server analyzes the received fuzzy information using natural language processing techniques, which involves tokenizing the text and extracting key keywords.

[0878] Step 6:

[0879] The server uses an emotion engine to analyze the transmitted emotion data, identifying emotions from the voice data based on tone, speed, emphasis, etc., and from the facial expression data, determining the emotional state based on the movements and changes of facial muscles.

[0880] Step 7:

[0881] The server generates a search query based on the analyzed keywords and sentiment data, which is then converted into a format that the generative AI can understand.

[0882] Step 8:

[0883] The server uses a generation AI to search the book database, which takes into account the user's emotions and identifies highly relevant books from the database.

[0884] Step 9:

[0885] The server then performs a process of organizing the search results, listing data including title, author, publication year, and brief summary, and adjusting the display order to reflect the user's emotional state.

[0886] Step 10:

[0887] The server sends the organized search results to the user terminal, where the data is encrypted for efficient transmission.

[0888] Step 11:

[0889] The user terminal receives the search results sent from the server, analyzes the received data, and displays them to the user in a visually easy-to-understand list format.

[0890] Step 12:

[0891] The user selects the book they are interested in from the presented book list, and detailed information about the selected book and related further information is also displayed.

[0892] Examples:

[0893] Example 1: Searching for moving adventure stories

[0894] Step 1:

[0895] The user terminal displays an input interface.

[0896] Step 2:

[0897] The user speaks, "I'm looking for a book that is an adventure story but also has an inspiring plot," and voice and facial expression data are also collected.

[0898] Step 3:

[0899] The user terminal transmits the input information and emotion data to the server.

[0900] Step 4:

[0901] The server receives the ambiguous information and the emotion data.

[0902] Step 5:

[0903] The server uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0904] Step 6:

[0905] The server uses an emotion engine to analyze the strong desire for emotion.

[0906] Step 7:

[0907] The server generates a search query based on keywords and emotion data.

[0908] Step 8:

[0909] The server uses generative AI to search a book database and list "The Alchemist" and "Life of Pi."

[0910] Step 9:

[0911] The server prioritizes inspiring stories and displays them at the top of the list.

[0912] Step 10:

[0913] The server transmits the organized search results to the user terminal.

[0914] Step 11:

[0915] The user terminal receives and displays the search results.

[0916] Step 12:

[0917] The user selects the desired book from the displayed list.

[0918] ---

[0919] The above are the specific processing steps of the book search system that combines the emotion engine.

[0920] Example 2

[0921] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0922] In conventional book search systems, if a user does not know the specific book title or author name, it is difficult to find the appropriate book when entering vague keywords or information. Furthermore, since search results are provided without taking the user's emotional state into consideration, users may not be able to achieve the emotional satisfaction they are looking for. This makes it particularly difficult for elderly people and young children to search for books efficiently.

[0923] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0924] In this invention, the server includes means for analyzing ambiguous information and keywords entered by the user and the user's emotional data using natural language processing technology, means for searching a book database using a generative AI model, and means for adjusting and displaying search results according to the user's emotional state. This enables the user to effectively find the most suitable book according to their emotional state by simply entering ambiguous information and keywords, even if they do not know the specific information.

[0925] "Ambiguous information and keywords" refers to vague search criteria that a user enters when they do not know the specific title or author of a book.

[0926] "Emotional data" refers to information about the user's psychological state and emotions obtained by analyzing their voice and facial expressions.

[0927] "Natural language processing technology" refers to artificial intelligence technology for understanding, analyzing, and generating human language.

[0928] A "generative AI model" is a type of artificial intelligence that is trained in advance using large amounts of data and generates sentences and search results according to specific conditions.

[0929] A "book database" refers to a collection of data that aggregates information about books.

[0930] An "emotion engine" refers to a technology or system that analyzes data such as voice and facial expressions to recognize a user's emotions.

[0931] "Analysis results" refers to keywords extracted from input information using natural language processing technology and the emotional state recognized by the emotion engine.

[0932] "Search Results" refers to a list of relevant books searched through a book database using a generative AI model.

[0933] This invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. In addition, by combining it with an emotion engine, search results can be optimized based on the user's emotions.

[0934] System configuration

[0935] The system consists of the following main components:

[0936] User terminal

[0937] server

[0938] Book Database

[0939] Emotion Engine

[0940] Processing flow

[0941] User terminal

[0942] A user terminal is a device into which a user inputs information or keywords. Examples include a smartphone, tablet, or PC. When a user inputs, "I'm looking for a book that is an adventure story but also has an inspiring plot," the terminal uses voice input and a camera to collect the user's emotional data. This emotional data includes voice tone and facial expressions. Hardware such as a microphone and a camera are used. Software such as Google Speech-to-Text and OpenCV are used.

[0943] server

[0944] The server receives and temporarily stores ambiguous information and emotional data sent from the user's device. The server has the following functions:

[0945] Data analysis

[0946] The server uses TensorFlow to execute natural language processing techniques to extract important keywords from ambiguous information, such as "adventure," "moving," and "story."

[0947] At the same time, an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) is used to analyze the user's emotions from voice and facial expression data.

[0948] Search Query Generation

[0949] A prompt sentence is generated based on the extracted keywords and the analyzed sentiment data. This prompt sentence is configured as an input to a generative AI model (such as GPT-4). For example, "Please recommend the best book for an excited user looking for an inspiring adventure story."

[0950] Database search

[0951] The generative AI model searches a book database based on the generated prompt. This book database is configured using a database management system such as MySQL. The generative AI model lists highly relevant books based on the search results.

[0952] Tailoring and Submitting Search Results

[0953] The server will adjust search results based on the user's emotional state, for example, if the user is in a state where they want inspiring stories, it will prioritize books that tell such stories.

[0954] The adjusted search results are transmitted to the user terminal.

[0955] Emotion Engine

[0956] The emotion engine is a component that recognizes emotions from the user's voice and facial expression data. It has the following functions:

[0957] Audio analysis

[0958] Use speech analysis software (e.g., Nuance or Google Speech-to-Text) to analyze speech tone, speed, pauses, etc. to identify emotion.

[0959] facial expression analysis

[0960] Using facial expression analysis software (e.g., OpenCV or Microsoft Azure Face API), the system analyzes facial muscle movements and changes in facial expressions to recognize the user's emotions.

[0961] Specific examples

[0962] Example 1: Searching for moving adventure stories

[0963] User terminal

[0964] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[0965] server

[0966] 1. Receive vague information and use natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[0967] 2. The emotion engine analyzes the user's emotions and determines that they are "seeking emotion."

[0968] 3. A generative AI model searches a book database based on keywords and emotional state to produce a list of books like "The Alchemist" and "Life of Pi."

[0969] 4. Organize search results to prioritize inspiring stories.

[0970] User terminal

[0971] The search results list is displayed to the user, who then selects the desired book.

[0972] Example 2: Looking for an easy-to-understand children's book about science

[0973] User terminal

[0974] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[0975] server

[0976] 1. Receive information and extract keywords such as "science," "easy to understand," and "children's book."

[0977] 2. The emotion engine analyzes the user's emotion as "showing curiosity."

[0978] 3. The generative AI model searches for suitable books and lists titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[0979] 4. Reflect your emotional state and prioritize content that will interest you.

[0980] User terminal

[0981] A list of search results is displayed to the user, and the user can select books that interest them.

[0982] This invention provides a system that allows users who do not know specific information to easily find the most suitable book according to their emotional state by simply entering vague information or keywords, greatly improving the convenience of book searches, especially for the elderly and young children.

[0983] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0984] Step 1:

[0985] The user operates the device and inputs vague search criteria using text or voice. For example, they might input, "I'm looking for a book that is an adventure story but also has an inspiring plot." At the same time, the device uses a camera to capture the user's facial expressions and a microphone to record their voice. The inputs here are the user's text / voice input and emotional data. The device then collects the user's facial expression and voice data and sends it to the server.

[0986] Step 2:

[0987] The server receives and temporarily stores ambiguous information and emotion data sent from the user's device. The input data is the user's text / voice input and emotion data. Based on this, the server stores the data in a database and prepares it for subsequent analysis.

[0988] Step 3:

[0989] The server uses TensorFlow to execute natural language processing techniques and extract important keywords from ambiguous information. The input data is text / voice input provided by the user. The data is processed by tokenizing the text and extracting important keywords (e.g., "adventure," "moving," "story"). The output is the extracted keywords.

[0990] Step 4:

[0991] The server uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze emotions from the user's voice and facial expression data. The input data is the user's emotional data (voice tone and facial muscle movements). Data calculations identify the user's emotional state (e.g., "desiring to be moved"). The output is the analyzed emotional information.

[0992] Step 5:

[0993] The server generates a prompt sentence to be input to the generative AI model (GPT-4) based on the extracted keywords and analyzed emotional data. The input data is keywords and emotional state. Based on this, a prompt sentence such as "Recommend the best book for an excited user looking for an inspiring adventure story" is created as a search query. The output is the generated prompt sentence.

[0994] Step 6:

[0995] The generative AI model searches a book database based on the generated prompt. The input data is the prompt. A database search is performed to generate a list of highly relevant books. For example, search results may include "The Alchemist" and "Life of Pi." The output is a list of books.

[0996] Step 7:

[0997] The server adjusts search results based on the user's emotional state. The input data is a book list and the user's emotional state. The server processes the data to adjust the results and reorganizes the list to prioritize moving stories. The output is the adjusted book list.

[0998] Step 8:

[0999] The server sends the adjusted search results to the user terminal. The input data is the adjusted book list. Based on this, the server transfers the search results to the user terminal. The output is the data sent to the user terminal.

[1000] Step 9:

[1001] The user terminal receives the search results sent from the server and displays them on the screen. The input data is the adjusted book list. Based on this, the search results are visually presented to the user. The user can then select the book they want from the displayed list.

[1002] (Application example 2)

[1003] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1004] In modern book search systems, if a user does not know the specific title or author name, it is difficult to effectively search for a book using vague information or keywords. Furthermore, conventional systems do not optimize search results by taking into account the user's emotional state, making it difficult for users to find the book they want. Therefore, it is necessary to provide a book search system that reflects emotional data based on the user's vague information and keywords.

[1005] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1006] In this invention, the server includes means for receiving ambiguous information and keywords input by a user, means for automatically searching for related book data based on the ambiguous information and keywords, means for searching for the book data based on a generative AI model, means for inputting a prompt sentence to the generative AI model to obtain search results, means for optimizing the search results in consideration of the user's emotional data, and means for displaying the optimized search results to the user. This enables an optimal book search that reflects the user's emotional data based on the user's ambiguous information and keywords.

[1007] "Ambiguous information" refers to unclear keywords or fragmentary information that is entered when the user does not know the specific title or author's name.

[1008] A "keyword" refers to a specific word or short phrase that a user enters when searching, and is information that serves as a criterion for narrowing down search results.

[1009] A "generative AI model" is a system that uses artificial intelligence technology to generate responses based on specific search queries and prompts.

[1010] A "prompt" is a search query or instruction given to a generative AI model, which functions as a parameter to obtain appropriate search results.

[1011] "Emotion data" is data that indicates the emotional state of the user obtained from the user's voice and facial expressions, and is a quantitative evaluation of emotions such as joy and sadness.

[1012] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is a technique for unconsciously processing ambiguous information and keywords.

[1013] "Search results" are lists of related books presented by the system based on vague information, keywords, and emotional data entered by the user.

[1014] "Optimization" means taking emotional data into account and adjusting search results to best meet the user's needs and emotions.

[1015] "Display means" means an interface device for visually presenting search results and other related information to a user.

[1016] This system allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. Furthermore, by combining it with an emotion engine that recognizes the user's emotions and optimizes search results, it can recommend the most relevant books to the user.

[1017] System configuration

[1018] This system is broadly composed of the following components:

[1019] User terminal

[1020] server

[1021] Generative AI Models

[1022] Book Database

[1023] Emotion Engine

[1024] User terminal

[1025] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for adventure stories." In response to this input, the terminal collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[1026] server

[1027] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[1028] 1. Data Reception

[1029] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[1030] 2. Data Analysis

[1031] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[1032] 3. Search Query Generation

[1033] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into a generative AI model.

[1034] 4. Book database search

[1035] The generative AI model searches a book database based on the generated search query, taking into account the user's emotional state to identify highly relevant books.

[1036] 5. Search Result Tailoring

[1037] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[1038] 6. Submit search results

[1039] The organized search results are sent to the user's terminal.

[1040] Emotion Engine

[1041] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[1042] Audio analysis

[1043] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[1044] facial expression analysis

[1045] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[1046] Hardware and software used

[1047] User device: Smartphone (iPhone or Android device)

[1048] Analysis software:

[1049] Speech Recognition: Google Speech Recognition API (using the speech_recognition library)

[1050] Facial expression analysis: facial_recognition library

[1051] Sentiment Analysis: TextBlob (Sentiment analysis using natural language processing)

[1052] Server side:

[1053] REST API (e.g. bookrstore.com API)

[1054] Book Database

[1055] Examples and prompts

[1056] Examples:

[1057] A user launches a smartphone app and inputs "memorable." The user then selects "joy" as the emotion. Based on this information, books such as "The Alchemist" are recommended.

[1058] Prompt statement:

[1059] User-entered keyword: "memorable"

[1060] Emotion data: {"joy": 0.9, "sadness": 0.1}

[1061] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1062] Step 1:

[1063] The user uses a smartphone to input vague information or keywords. The user also simultaneously inputs emotional data using voice input or a camera. The input at this stage consists of text keywords and audio / image data.

[1064] Step 2:

[1065] The user's device sends the entered vague information or keywords, as well as audio and image data, to the server, where the audio data is converted into text using speech recognition software and the image data is analyzed using facial expression analysis software.

[1066] Step 3:

[1067] The server receives vague information, keywords, and emotion data sent from the user terminal. The received data is temporarily stored in a database. The input data is keyword text and emotion data.

[1068] Step 4:

[1069] The server uses natural language processing technology to analyze ambiguous information and extract important keywords. The server performs text analysis using TextBlob. It also uses an emotion engine to identify emotions from voice data and facial expression data. The analyzed keywords and emotion data are output.

[1070] Step 5:

[1071] The server generates a search query based on the analyzed keywords and sentiment data. This query is converted into a prompt sentence to be input to the generative AI model. The generated prompt sentence is output.

[1072] Step 6:

[1073] The server inputs a prompt into the generative AI model and searches the book database. The generative AI model retrieves relevant book data based on the prompt, and outputs the search results.

[1074] Step 7:

[1075] The server organizes the search results and optimizes them based on the emotional data recognized by the emotion engine. For example, if a user is looking for an inspiring story, relevant books will be prioritized in the list. The optimized search results are output.

[1076] Step 8:

[1077] The server sends the optimized search results to the user's device, which visually displays the results and presents them in a user-selectable format. The output at this stage is a list of related books and their detailed information.

[1078] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1079] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1080] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1081] [Third embodiment]

[1082] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1083] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1084] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1085] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1086] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1087] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1088] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1089] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1090] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1091] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1092] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1093] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1094] ---

[1095] The present invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. Below we will explain how this system works.

[1096] System configuration

[1097] The system consists of three main components:

[1098] User terminal

[1099] server

[1100] Book Database

[1101] User terminal

[1102] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[1103] server

[1104] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[1105] 1. Data Reception

[1106] The server receives the ambiguous information sent from the user terminal.

[1107] 2. Data Analysis

[1108] Analyzes received data and extracts important keywords. Natural language processing technology is used to tokenize text data and obtain meaningful keywords.

[1109] 3. Search Query Generation

[1110] Based on the extracted keywords, a search query is generated, which is then converted into a format that can be input to the generation AI.

[1111] 4. Book database search

[1112] Based on the generated query, the generative AI searches a book database and lists highly relevant books based on the information in the database.

[1113] 5. Organizing search results

[1114] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1115] 6. Data Transmission

[1116] The organized search results are sent to the user's terminal.

[1117] Book Database

[1118] The book database contains information about a huge number of books, which the server uses to find the most suitable book based on a search query. This database contains detailed information such as the book title, author, genre, and summary.

[1119] Specific examples

[1120] Example 1: Searching for moving adventure stories

[1121] 1. User Device

[1122] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[1123] 2. Server

[1124] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1125] The generative AI searches a book database based on the extracted keywords and lists books such as "The Alchemist" and "Life of Pi."

[1126] The resulting list is compiled and sent to the user's terminal.

[1127] 3. User Device

[1128] The search results are presented to the user as a list of books, from which the user can select books of interest.

[1129] Example 2: Looking for an easy-to-understand children's book about science

[1130] 1. User Device

[1131] A user types, "I'm looking for an easy-to-understand children's book about science."

[1132] 2. Server

[1133] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[1134] The generative AI searches for the best books based on these keywords, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1135] The organized results are sent to the user terminal.

[1136] 3. User Device

[1137] A list of search results is displayed to the user to help them select the most suitable book.

[1138] In this way, the present invention provides a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve the convenience of searching for books, especially for the elderly and young children.

[1139] The processing flow will be explained below.

[1140] ---

[1141] Step 1:

[1142] The user terminal displays an input interface for book searches to the user, which is intuitive and easy to operate, and is designed so that even the elderly and young children can use it.

[1143] Step 2:

[1144] Users input vague information or keywords about the book they are looking for, such as "a book that is an adventure story but also has an inspiring story."

[1145] Step 3:

[1146] The user terminal formats the ambiguous information entered by the user and sends it to the server, where the data has been converted into the appropriate format.

[1147] Step 4:

[1148] The server receives the ambiguous information sent from the user terminal and temporarily stores the received data.

[1149] Step 5:

[1150] The server then analyzes the received ambiguous information using natural language processing techniques, specifically tokenizing the text and extracting meaningful keywords.

[1151] Step 6:

[1152] The server generates a search query based on the analyzed keywords, which is then converted into a format that the AI ​​can understand.

[1153] Step 7:

[1154] The server uses a generation AI to search the book database, which references a large amount of book data and identifies books that are highly relevant.

[1155] Step 8:

[1156] The server organizes the generated search results and converts them into a user-friendly format, including, for example, book title, author, publication year, and a brief summary.

[1157] Step 9:

[1158] The server then sends the organized search results to the user's device, where the data is properly encrypted and securely transmitted.

[1159] Step 10:

[1160] The user terminal receives the search results sent from the server and displays them to the user in a visually easy-to-understand list format.

[1161] Step 11:

[1162] The user selects the book of interest from the displayed list of books, which may include more detailed information or related books.

[1163] ---

[1164] The above are the specific processing steps of the program.

[1165] Example 1

[1166] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1167] Conventional book search systems have the problem that it is difficult to search when you do not know the specific title or author name, or when you search based on only vague information. In addition, there is a lack of means to display search results in a format that is easy for users to understand, which makes them particularly difficult to use for the elderly and young children.

[1168] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1169] In this invention, the server includes means for receiving ambiguous information or keywords entered by a user, means for analyzing the ambiguous information or keywords using natural language processing technology, and means for generating a search query based on the ambiguous information or keywords. This allows a user to easily find a desired book by simply entering ambiguous information or keywords, even if they do not know the specific information.

[1170] The server further includes a means for searching for related book data using the generative AI model based on the generated search query, and a means for organizing detailed information about books based on the search results and displaying it to the user. This makes it possible to provide a system that displays search results in an efficient and easy-to-understand format and is easy to use, especially for the elderly and young children.

[1171] "Ambiguous information or keywords entered by the user" refers to unclear information entered when the user does not clearly know specific information, or keywords used to identify the search target.

[1172] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and process human language, and includes technology for tokenizing text data and performing semantic analysis.

[1173] "Search query" means a sentence or phrase that describes a request to a search engine or database to find specific information.

[1174] A "generative AI model" is an artificial intelligence model that makes predictions and generates based on input data, and refers to an AI that generatively outputs text, for example.

[1175] A "prompt format" refers to a text sentence that has been formatted in a specific way to be input into a generative AI model.

[1176] "Searching book data" refers to searching through book information in a database to find relevant books based on a specific search query.

[1177] "Detailed book information" refers to information about a specific book, including the title, author name, publication year, summary, etc.

[1178] "Search Results" refers to the list of relevant books and information that the system finds based on the user's input.

[1179] "Displaying to the user" refers to providing the organized search results to the user through the interface of the user terminal.

[1180] The present invention is a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. How this system works will be explained in detail below.

[1181] System configuration

[1182] The system consists of three main components:

[1183] User terminal

[1184] server

[1185] Book Database

[1186] User terminal

[1187] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[1188] server

[1189] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[1190] 1. Data Reception

[1191] The server receives the ambiguous information sent from the user terminal, using a communication protocol (e.g., HTTPS) over the Internet.

[1192] 2. Data Analysis

[1193] The received data is analyzed and important keywords are extracted. Natural language processing techniques are used to tokenize the text data and obtain meaningful keywords. Specifically, Python's natural language processing libraries "NLTK" and "spaCy" are used.

[1194] 3. Search Query Generation

[1195] A search query is generated based on the extracted keywords. This query is converted into a format that can be input into a generative AI model. For example, "GPT-3" can be used as the generative AI model.

[1196] 4. Book database search

[1197] Based on the generated query, the generative AI model searches a book database and lists relevant books from the information in the database. The book database contains a large amount of book information, including detailed information such as book title, author, genre, and summary.

[1198] 5. Organizing search results

[1199] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1200] 6. Data Transmission

[1201] The organized search results are sent to the user's terminal, which receives the data and displays it to the user.

[1202] Specific examples

[1203] Example 1: Searching for moving adventure stories

[1204] 1. User Device

[1205] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[1206] 2. Server

[1207] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1208] Based on the extracted keywords, the generative AI model searches a book database and lists books such as "The Alchemist" and "Life of Pi."

[1209] The search results are organized and the list is sent to the user's terminal.

[1210] 3. User Device

[1211] The search results are presented to the user as a list of books from which the user can select books of interest.

[1212] Example 2: Looking for an easy-to-understand children's book about science

[1213] 1. User Device

[1214] A user types, "I'm looking for an easy-to-understand children's book about science."

[1215] 2. Server

[1216] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[1217] Based on the extracted keywords, the generative AI model searches for the most suitable books, listing titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1218] The organized results are sent to the user terminal.

[1219] 3. User Device

[1220] A list of search results is displayed to the user to help them select the most suitable book.

[1221] The above is a specific example of the "Mode for Carrying Out the Invention." This system allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve convenience for elderly people and young children when searching for books.

[1222] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1223] Step 1:

[1224] User input:

[1225] The user uses the device interface to input vague information or keywords, for example, "I'm looking for a book that is an adventure story but also has an inspiring plot." The input data are vague keywords or phrases related to the book the user is looking for.

[1226] Step 2:

[1227] Sending data:

[1228] The terminal sends the text data entered by the user to the server using the HTTPS protocol. This data includes the user's search request.

[1229] Step 3:

[1230] Receiving data:

[1231] The server receives the data sent from the terminal, which is an ambiguous keyword or phrase entered by the user.

[1232] Step 4:

[1233] Data analysis:

[1234] The server analyzes the received data. Specifically, it uses the Python natural language processing library "NLTK" or "spaCy" to tokenize the text data and extract meaningful keywords. For example, the keywords "adventure," "inspirational," and "story" are extracted from the text "A book that is both an adventure story and an inspiring one."

[1235] Step 5:

[1236] Search query generation:

[1237] The server generates a search query based on the extracted keywords. This query is converted into a format that can be input to the generative AI model. For example, by combining the keywords "adventure," "emotional," and "story," the prompt sentence "Please tell me a book that is both an adventure story and an emotional story" is generated.

[1238] Step 6:

[1239] Book database search:

[1240] The server searches a book database using a generative AI model (e.g., GPT-3) based on the generated query. The generative AI model lists relevant books based on the input prompt. The database contains detailed information such as book title, author, genre, and summary.

[1241] Step 7:

[1242] Search result organization:

[1243] The server organizes the search results and generates data to display them in a user-friendly format, including information such as the title, author, publication year, and a brief synopsis. For example, the server retrieves book information such as "The Alchemist," "Paulo Coelho," "1988," and "An adventure story that shows the journey of life."

[1244] Step 8:

[1245] Submit search results:

[1246] The server sends the organized search results to the user terminal, which receives the data and displays it to the user.

[1247] Step 9:

[1248] Displaying search results:

[1249] The user terminal displays the search results sent from the server to the user. The user can then select books that interest them based on the displayed search results. For example, a list of books such as "The Alchemist" and "Life of Pi" may be displayed.

[1250] (Application example 1)

[1251] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] When users search for a book, it is difficult to quickly and accurately find the desired book even if they only have vague information. It is also difficult to immediately identify where the book is actually located in the bookstore, which makes it inconvenient for a wide range of users, including the elderly and young children. To solve this problem, a system is needed that can accurately identify a book even with vague information and easily find where that book is located in the bookstore.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1254] In this invention, the server includes means for receiving vague information or keywords entered by a user, means for automatically searching for related book data based on the vague information or keywords, means for displaying search results to the user, means for identifying the location of the book in the bookstore based on the search results, and means for displaying a route to the identified location. This allows the user to accurately identify a book even with vague information, and then easily find where it is located in the bookstore.

[1255] "Ambiguous information" refers to information that is general or vague and not specifically defined by the user.

[1256] "Keywords" are specific words or short phrases that are necessary to conduct a search for a book.

[1257] "Natural language processing technology" is a technology that enables computers to understand, interpret, and generate human language, and is used to analyze ambiguous user input.

[1258] "Book data" is data consisting of information such as the book title, author, genre, and summary.

[1259] "Search results" are a collection of related book data that are automatically searched based on the user's vague information or keywords.

[1260] A "user terminal" is a device used by a user to input information, including a smartphone or tablet.

[1261] "Server" refers to a central processing unit that receives data sent from user terminals and analyzes and processes it.

[1262] The "location of a book in a bookstore" is the specific location where a particular book is located in a bookstore.

[1263] "Displaying a route" means visually showing a customer the route they should take to reach a particular book within the bookstore.

[1264] The present invention provides a system for effectively searching for books in a bookstore. A specific embodiment for implementing this system will be described below.

[1265] System configuration

[1266] The system consists of three main components:

[1267] User terminal

[1268] server

[1269] Book Database

[1270] User terminal

[1271] A user terminal is a device that allows a user to input information and receive results. Examples include smartphones and tablets. Here, we will use a smartphone as an example. A user launches an app on the smartphone and uses the voice input function to input vague information. For example, a user might input a search request such as, "I want a book with a scary but moving story."

[1272] server

[1273] The server receives the ambiguous information sent from the user terminal and analyzes and processes it. Specifically, it uses the following hardware and software:

[1274] Server hardware: general data center servers and cloud servers

[1275] Software: Google Cloud Natural Language API, OpenAI GPT-4 model

[1276] Processing flow

[1277] 1. Data Reception

[1278] The server receives the voice data sent from the user's device, which is converted into text using the Google Cloud Speech-to-Text API.

[1279] 2. Data Analysis

[1280] The converted text data is then analyzed using the Google Cloud Natural Language API to extract meaningful keywords, such as "scary," "moving," and "story."

[1281] 3. Search Query Generation

[1282] Based on the extracted keywords, a search query is generated by generative AI (OpenAI GPT-4 model). For example, a prompt sentence such as "Generate a search query for a book that is scary yet emotional for bookstore inventory search" is input.

[1283] 4. Book database search

[1284] The generated query is then used to search a book database, which contains detailed information such as the book's title, author, genre, and summary, to produce a list of related books.

[1285] 5. Organizing search results

[1286] Search results are organized and presented in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1287] 6. Data Transmission

[1288] The organized search results are sent to the user's terminal and displayed on the screen.

[1289] 7. Bookstore location and route guidance

[1290] When a user selects a particular book, the location of that book in the bookstore is identified and displayed on the smartphone, and a store map is also displayed, providing directions to the selected book's location.

[1291] Specific examples

[1292] Example 1: Looking for a book with a scary but moving story

[1293] 1. User Device

[1294] A user types, "I want a book with scary but moving stories."

[1295] 2. Server

[1296] The server receives this vague information and converts it into text using the Google Cloud Speech-to-Text API.

[1297] Using the Google Cloud Natural Language API, keywords such as "scary," "moving," and "story" are extracted.

[1298] Based on the extracted keywords, the generative AI (OpenAI GPT-4 model) inputs the prompt sentence "Generate a search query for a book that is scary yet emotional for bookstore inventory search" and generates a search query.

[1299] A book database is searched based on the generated query, and related books are listed.

[1300] The search results are organized and sent to the user's terminal.

[1301] 3. User Device

[1302] The system presents the user with a list of books displayed as search results, and also displays the location and route to the selected book in the store, providing guidance to the specific location.

[1303] In this way, the present invention provides a system that can accurately identify a book and easily locate its location in a bookstore, even if the user has only vague information, making it possible for a wide range of users, including the elderly and young children, to easily search for books.

[1304] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1305] Step 1:

[1306] A user launches a smartphone application and uses the voice input function to enter a vague search request, specifically, "I want a book with a scary but moving story."

[1307] Input: User's voice data.

[1308] Output: Audio data collected through the smartphone microphone.

[1309] Specific behavior: The smartphone application receives voice input and prepares it to be sent to the system.

[1310] Step 2:

[1311] The device sends the voice data to the server, which converts it into text using the Google Cloud Speech-to-Text API.

[1312] Input: Audio data sent from a smartphone.

[1313] Output: Text data (e.g., "I want a book with a scary but moving story").

[1314] Specific operation: The server receives the audio data and converts it into text using the Google Cloud Speech-to-Text API.

[1315] Step 3:

[1316] The server analyzes the text data using the Google Cloud Natural Language API and extracts important keywords.

[1317] Input: Text data converted from audio data.

[1318] Output: A set of keywords (e.g. "scary", "emotional", "story").

[1319] Specific operation: The server analyzes the text data and performs tokenization, stop word removal, and keyword extraction.

[1320] Step 4:

[1321] The server inputs a prompt into the generative AI (OpenAI GPT-4 model) to generate a search query. Example prompt: "Generate a search query for a book that is scary yet emotional for bookstore inventory search."

[1322] Input: A set of keywords.

[1323] Output: The search query created by the generative AI.

[1324] Specific operation: The server inputs a prompt sentence into the generation AI to generate an appropriate search query.

[1325] Step 5:

[1326] The server searches a book database using the generated search query.

[1327] Input: A search query generated by the generative AI.

[1328] Output: A list of related books.

[1329] Specific operation: The server executes a search query against a book database and lists related books.

[1330] Step 6:

[1331] The server organizes the search results and summarizes important information (title, author, summary, etc.).

[1332] Input: A list of books.

[1333] Output: An organized list of book information.

[1334] Specific operation: The server collects book data and organizes it into a format that is easy for users to understand.

[1335] Step 7:

[1336] The server sends the organized search results to the user's terminal, which displays the information.

[1337] Input: An organized list of book information.

[1338] Output: Search results displayed on a smartphone screen.

[1339] Specific operation: The server sends the data and the smartphone displays the results on the screen.

[1340] Step 8:

[1341] When a user selects a particular book, the server identifies the book's location in the bookstore and sends that information to the user's device, which then displays a store map and route guidance on the smartphone.

[1342] Input: The book selected by the user.

[1343] Output: Book location and directions.

[1344] Specific operation: The server queries the book location information, generates route guidance, and sends it to the user's device. The smartphone displays a map of the store and provides route guidance.

[1345] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1346] ---

[1347] The present invention provides a system that enables a user to effectively search for books using vague information or keywords even when the user does not know the specific title or author of the book, and further combines it with an emotion engine that recognizes the user's emotions and optimizes the search results.

[1348] System configuration

[1349] The system consists of four main components:

[1350] User terminal

[1351] server

[1352] Book Database

[1353] Emotion Engine

[1354] User terminal

[1355] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for a book that is an adventure story but has an inspiring story." In response to this input, the system collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[1356] server

[1357] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[1358] 1. Data Reception

[1359] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[1360] 2. Data Analysis

[1361] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[1362] 3. Search Query Generation

[1363] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into the generative AI.

[1364] 4. Book database search

[1365] The generative AI searches a book database based on the generated search query, taking into account the user's emotional state and identifying highly relevant books.

[1366] 5. Search Result Tailoring

[1367] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[1368] 6. Submit search results

[1369] The organized search results are sent to the user's terminal.

[1370] Emotion Engine

[1371] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[1372] Audio analysis

[1373] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[1374] facial expression analysis

[1375] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[1376] Specific examples

[1377] Example 1: Searching for moving adventure stories

[1378] 1. User Device

[1379] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[1380] 2. Server

[1381] The server receives this vague information, uses natural language processing technology to extract keywords such as "adventure," "emotional," and "story," and uses an emotion engine to analyze that the user is looking for something moving.

[1382] The generative AI searches a book database based on keywords and emotional state to produce a list of books such as "The Alchemist" and "Life of Pi."

[1383] Organize search results to prioritize inspiring stories at the top of the list.

[1384] 3. User Device

[1385] The search results list is displayed to the user, who then selects the desired book.

[1386] Example 2: Looking for an easy-to-understand children's book about science

[1387] 1. User Device

[1388] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[1389] 2. Server

[1390] The server receives this information, extracts keywords such as "science," "easy to understand," and "children's books," and uses an emotion engine to analyze whether the user is expressing curiosity.

[1391] Generative AI searches for the best books based on keywords and emotional state, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1392] It reflects your emotional state and prioritizes content that will interest you.

[1393] 3. User Device

[1394] A list of search results is displayed to the user, and the user can select books that interest them.

[1395] This invention combines an emotion engine and generative AI to provide a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This will greatly improve convenience for people, especially the elderly and young children, when searching for books.

[1396] The processing flow will be explained below.

[1397] ---

[1398] Step 1:

[1399] The user terminal displays an input interface for book searches, which includes a text input box, a voice input function, and a camera for face recognition.

[1400] Step 2:

[1401] The user inputs vague information or keywords about the book they are looking for. For example, they input vague criteria such as "a book that is an adventure story but also has an inspiring story" by voice or text. During this process, the user's voice and facial expression data are collected in real time.

[1402] Step 3:

[1403] The user terminal formats the input information from the user and sends it to the server along with emotion data. At this time, the data is encrypted and sent securely.

[1404] Step 4:

[1405] The server receives the ambiguous information and emotion data sent from the user terminal, and the received data is temporarily stored in a database.

[1406] Step 5:

[1407] The server analyzes the received fuzzy information using natural language processing techniques, which involves tokenizing the text and extracting key keywords.

[1408] Step 6:

[1409] The server uses an emotion engine to analyze the transmitted emotion data, identifying emotions from the voice data based on tone, speed, emphasis, etc., and from the facial expression data, determining the emotional state based on the movements and changes of facial muscles.

[1410] Step 7:

[1411] The server generates a search query based on the analyzed keywords and sentiment data, which is then converted into a format that the generative AI can understand.

[1412] Step 8:

[1413] The server uses a generation AI to search the book database, which takes into account the user's emotions and identifies highly relevant books from the database.

[1414] Step 9:

[1415] The server then performs a process of organizing the search results, listing data including title, author, publication year, and brief summary, and adjusting the display order to reflect the user's emotional state.

[1416] Step 10:

[1417] The server sends the organized search results to the user terminal, where the data is encrypted for efficient transmission.

[1418] Step 11:

[1419] The user terminal receives the search results sent from the server, analyzes the received data, and displays them to the user in a visually easy-to-understand list format.

[1420] Step 12:

[1421] The user selects the book they are interested in from the presented book list, and detailed information about the selected book and related further information is also displayed.

[1422] Examples:

[1423] Example 1: Searching for moving adventure stories

[1424] Step 1:

[1425] The user terminal displays an input interface.

[1426] Step 2:

[1427] The user speaks, "I'm looking for a book that is an adventure story but also has an inspiring plot," and voice and facial expression data are also collected.

[1428] Step 3:

[1429] The user terminal transmits the input information and emotion data to the server.

[1430] Step 4:

[1431] The server receives the ambiguous information and the emotion data.

[1432] Step 5:

[1433] The server uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1434] Step 6:

[1435] The server uses an emotion engine to analyze the strong desire for emotion.

[1436] Step 7:

[1437] The server generates a search query based on keywords and emotion data.

[1438] Step 8:

[1439] The server uses generative AI to search a book database and list "The Alchemist" and "Life of Pi."

[1440] Step 9:

[1441] The server prioritizes inspiring stories and displays them at the top of the list.

[1442] Step 10:

[1443] The server transmits the organized search results to the user terminal.

[1444] Step 11:

[1445] The user terminal receives and displays the search results.

[1446] Step 12:

[1447] The user selects the desired book from the displayed list.

[1448] ---

[1449] The above are the specific processing steps of the book search system that combines the emotion engine.

[1450] Example 2

[1451] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1452] In conventional book search systems, if a user does not know the specific book title or author name, it is difficult to find the appropriate book when entering vague keywords or information. Furthermore, since search results are provided without taking the user's emotional state into consideration, users may not be able to achieve the emotional satisfaction they are looking for. This makes it particularly difficult for elderly people and young children to search for books efficiently.

[1453] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1454] In this invention, the server includes means for analyzing ambiguous information and keywords entered by the user and the user's emotional data using natural language processing technology, means for searching a book database using a generative AI model, and means for adjusting and displaying search results according to the user's emotional state. This enables the user to effectively find the most suitable book according to their emotional state by simply entering ambiguous information and keywords, even if they do not know the specific information.

[1455] "Ambiguous information and keywords" refers to vague search criteria that a user enters when they do not know the specific title or author of a book.

[1456] "Emotional data" refers to information about the user's psychological state and emotions obtained by analyzing their voice and facial expressions.

[1457] "Natural language processing technology" refers to artificial intelligence technology for understanding, analyzing, and generating human language.

[1458] A "generative AI model" is a type of artificial intelligence that is trained in advance using large amounts of data and generates sentences and search results according to specific conditions.

[1459] A "book database" refers to a collection of data that aggregates information about books.

[1460] An "emotion engine" refers to a technology or system that analyzes data such as voice and facial expressions to recognize a user's emotions.

[1461] "Analysis results" refers to keywords extracted from input information using natural language processing technology and the emotional state recognized by the emotion engine.

[1462] "Search Results" refers to a list of relevant books searched through a book database using a generative AI model.

[1463] This invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. In addition, by combining it with an emotion engine, search results can be optimized based on the user's emotions.

[1464] System configuration

[1465] The system consists of the following main components:

[1466] User terminal

[1467] server

[1468] Book Database

[1469] Emotion Engine

[1470] Processing flow

[1471] User terminal

[1472] A user terminal is a device into which a user inputs information or keywords. Examples include a smartphone, tablet, or PC. When a user inputs, "I'm looking for a book that is an adventure story but also has an inspiring plot," the terminal uses voice input and a camera to collect the user's emotional data. This emotional data includes voice tone and facial expressions. Hardware such as a microphone and a camera are used. Software such as Google Speech-to-Text and OpenCV are used.

[1473] server

[1474] The server receives and temporarily stores ambiguous information and emotional data sent from the user's device. The server has the following functions:

[1475] Data analysis

[1476] The server uses TensorFlow to execute natural language processing techniques to extract important keywords from ambiguous information, such as "adventure," "moving," and "story."

[1477] At the same time, an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) is used to analyze the user's emotions from voice and facial expression data.

[1478] Search Query Generation

[1479] A prompt sentence is generated based on the extracted keywords and the analyzed sentiment data. This prompt sentence is configured as an input to a generative AI model (such as GPT-4). For example, "Please recommend the best book for an excited user looking for an inspiring adventure story."

[1480] Database search

[1481] The generative AI model searches a book database based on the generated prompt. This book database is configured using a database management system such as MySQL. The generative AI model lists highly relevant books based on the search results.

[1482] Tailoring and Submitting Search Results

[1483] The server will adjust search results based on the user's emotional state, for example, if the user is in a state where they want inspiring stories, it will prioritize books that tell such stories.

[1484] The adjusted search results are transmitted to the user terminal.

[1485] Emotion Engine

[1486] The emotion engine is a component that recognizes emotions from the user's voice and facial expression data. It has the following functions:

[1487] Audio analysis

[1488] Use speech analysis software (e.g., Nuance or Google Speech-to-Text) to analyze speech tone, speed, pauses, etc. to identify emotion.

[1489] facial expression analysis

[1490] Using facial expression analysis software (e.g., OpenCV or Microsoft Azure Face API), the system analyzes facial muscle movements and changes in facial expressions to recognize the user's emotions.

[1491] Specific examples

[1492] Example 1: Searching for moving adventure stories

[1493] User terminal

[1494] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[1495] server

[1496] 1. Receive vague information and use natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1497] 2. The emotion engine analyzes the user's emotions and determines that they are "seeking emotion."

[1498] 3. A generative AI model searches a book database based on keywords and emotional state to produce a list of books like "The Alchemist" and "Life of Pi."

[1499] 4. Organize search results to prioritize inspiring stories.

[1500] User terminal

[1501] The search results list is displayed to the user, who then selects the desired book.

[1502] Example 2: Looking for an easy-to-understand children's book about science

[1503] User terminal

[1504] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[1505] server

[1506] 1. Receive information and extract keywords such as "science," "easy to understand," and "children's book."

[1507] 2. The emotion engine analyzes the user's emotion as "showing curiosity."

[1508] 3. The generative AI model searches for suitable books and lists titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1509] 4. Reflect your emotional state and prioritize content that will interest you.

[1510] User terminal

[1511] A list of search results is displayed to the user, and the user can select books that interest them.

[1512] This invention provides a system that allows users who do not know specific information to easily find the most suitable book according to their emotional state by simply entering vague information or keywords, greatly improving the convenience of book searches, especially for the elderly and young children.

[1513] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1514] Step 1:

[1515] The user operates the device and inputs vague search criteria using text or voice. For example, they might input, "I'm looking for a book that is an adventure story but also has an inspiring plot." At the same time, the device uses a camera to capture the user's facial expressions and a microphone to record their voice. The inputs here are the user's text / voice input and emotional data. The device then collects the user's facial expression and voice data and sends it to the server.

[1516] Step 2:

[1517] The server receives and temporarily stores ambiguous information and emotion data sent from the user's device. The input data is the user's text / voice input and emotion data. Based on this, the server stores the data in a database and prepares it for subsequent analysis.

[1518] Step 3:

[1519] The server uses TensorFlow to execute natural language processing techniques and extract important keywords from ambiguous information. The input data is text / voice input provided by the user. The data is processed by tokenizing the text and extracting important keywords (e.g., "adventure," "moving," "story"). The output is the extracted keywords.

[1520] Step 4:

[1521] The server uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze emotions from the user's voice and facial expression data. The input data is the user's emotional data (voice tone and facial muscle movements). Data calculations identify the user's emotional state (e.g., "desiring to be moved"). The output is the analyzed emotional information.

[1522] Step 5:

[1523] The server generates a prompt sentence to be input to the generative AI model (GPT-4) based on the extracted keywords and analyzed emotional data. The input data is keywords and emotional state. Based on this, a prompt sentence such as "Recommend the best book for an excited user looking for an inspiring adventure story" is created as a search query. The output is the generated prompt sentence.

[1524] Step 6:

[1525] The generative AI model searches a book database based on the generated prompt. The input data is the prompt. A database search is performed to generate a list of highly relevant books. For example, search results may include "The Alchemist" and "Life of Pi." The output is a list of books.

[1526] Step 7:

[1527] The server adjusts search results based on the user's emotional state. The input data is a book list and the user's emotional state. The server processes the data to adjust the results and reorganizes the list to prioritize moving stories. The output is the adjusted book list.

[1528] Step 8:

[1529] The server sends the adjusted search results to the user terminal. The input data is the adjusted book list. Based on this, the server transfers the search results to the user terminal. The output is the data sent to the user terminal.

[1530] Step 9:

[1531] The user terminal receives the search results sent from the server and displays them on the screen. The input data is the adjusted book list. Based on this, the search results are visually presented to the user. The user can then select the book they want from the displayed list.

[1532] (Application example 2)

[1533] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1534] In modern book search systems, if a user does not know the specific title or author name, it is difficult to effectively search for a book using vague information or keywords. Furthermore, conventional systems do not optimize search results by taking into account the user's emotional state, making it difficult for users to find the book they want. Therefore, it is necessary to provide a book search system that reflects emotional data based on the user's vague information and keywords.

[1535] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1536] In this invention, the server includes means for receiving ambiguous information and keywords input by a user, means for automatically searching for related book data based on the ambiguous information and keywords, means for searching for the book data based on a generative AI model, means for inputting a prompt sentence to the generative AI model to obtain search results, means for optimizing the search results in consideration of the user's emotional data, and means for displaying the optimized search results to the user. This enables an optimal book search that reflects the user's emotional data based on the user's ambiguous information and keywords.

[1537] "Ambiguous information" refers to unclear keywords or fragmentary information that is entered when the user does not know the specific title or author's name.

[1538] A "keyword" refers to a specific word or short phrase that a user enters when searching, and is information that serves as a criterion for narrowing down search results.

[1539] A "generative AI model" is a system that uses artificial intelligence technology to generate responses based on specific search queries and prompts.

[1540] A "prompt" is a search query or instruction given to a generative AI model, which functions as a parameter to obtain appropriate search results.

[1541] "Emotion data" is data that indicates the emotional state of the user obtained from the user's voice and facial expressions, and is a quantitative evaluation of emotions such as joy and sadness.

[1542] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is a technique for unconsciously processing ambiguous information and keywords.

[1543] "Search results" are lists of related books presented by the system based on vague information, keywords, and emotional data entered by the user.

[1544] "Optimization" means taking emotional data into account and adjusting search results to best meet the user's needs and emotions.

[1545] "Display means" means an interface device for visually presenting search results and other related information to a user.

[1546] This system allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. Furthermore, by combining it with an emotion engine that recognizes the user's emotions and optimizes search results, it can recommend the most relevant books to the user.

[1547] System configuration

[1548] This system is broadly composed of the following components:

[1549] User terminal

[1550] server

[1551] Generative AI Models

[1552] Book Database

[1553] Emotion Engine

[1554] User terminal

[1555] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for adventure stories." In response to this input, the terminal collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[1556] server

[1557] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[1558] 1. Data Reception

[1559] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[1560] 2. Data Analysis

[1561] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[1562] 3. Search Query Generation

[1563] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into a generative AI model.

[1564] 4. Book database search

[1565] The generative AI model searches a book database based on the generated search query, taking into account the user's emotional state to identify highly relevant books.

[1566] 5. Search Result Tailoring

[1567] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[1568] 6. Submit search results

[1569] The organized search results are sent to the user's terminal.

[1570] Emotion Engine

[1571] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[1572] Audio analysis

[1573] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[1574] facial expression analysis

[1575] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[1576] Hardware and software used

[1577] User device: Smartphone (iPhone or Android device)

[1578] Analysis software:

[1579] Speech Recognition: Google Speech Recognition API (using the speech_recognition library)

[1580] Facial expression analysis: facial_recognition library

[1581] Sentiment Analysis: TextBlob (Sentiment analysis using natural language processing)

[1582] Server side:

[1583] REST API (e.g. bookrstore.com API)

[1584] Book Database

[1585] Examples and prompts

[1586] Examples:

[1587] A user launches a smartphone app and inputs "memorable." The user then selects "joy" as the emotion. Based on this information, books such as "The Alchemist" are recommended.

[1588] Prompt statement:

[1589] User-entered keyword: "memorable"

[1590] Emotion data: {"joy": 0.9, "sadness": 0.1}

[1591] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1592] Step 1:

[1593] The user uses a smartphone to input vague information or keywords. The user also simultaneously inputs emotional data using voice input or a camera. The input at this stage consists of text keywords and audio / image data.

[1594] Step 2:

[1595] The user's device sends the entered vague information or keywords, as well as audio and image data, to the server, where the audio data is converted into text using speech recognition software and the image data is analyzed using facial expression analysis software.

[1596] Step 3:

[1597] The server receives vague information, keywords, and emotion data sent from the user terminal. The received data is temporarily stored in a database. The input data is keyword text and emotion data.

[1598] Step 4:

[1599] The server uses natural language processing technology to analyze ambiguous information and extract important keywords. The server performs text analysis using TextBlob. It also uses an emotion engine to identify emotions from voice data and facial expression data. The analyzed keywords and emotion data are output.

[1600] Step 5:

[1601] The server generates a search query based on the analyzed keywords and sentiment data. This query is converted into a prompt sentence to be input to the generative AI model. The generated prompt sentence is output.

[1602] Step 6:

[1603] The server inputs a prompt into the generative AI model and searches the book database. The generative AI model retrieves relevant book data based on the prompt, and outputs the search results.

[1604] Step 7:

[1605] The server organizes the search results and optimizes them based on the emotional data recognized by the emotion engine. For example, if a user is looking for an inspiring story, relevant books will be prioritized in the list. The optimized search results are output.

[1606] Step 8:

[1607] The server sends the optimized search results to the user's device, which visually displays the results and presents them in a user-selectable format. The output at this stage is a list of related books and their detailed information.

[1608] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1609] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1610] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1611] [Fourth embodiment]

[1612] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1613] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1615] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1616] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1618] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1619] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1620] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1623] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1624] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1625] ---

[1626] The present invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. Below we will explain how this system works.

[1627] System configuration

[1628] The system consists of three main components:

[1629] User terminal

[1630] server

[1631] Book Database

[1632] User terminal

[1633] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[1634] server

[1635] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[1636] 1. Data Reception

[1637] The server receives the ambiguous information sent from the user terminal.

[1638] 2. Data Analysis

[1639] Analyzes received data and extracts important keywords. Natural language processing technology is used to tokenize text data and obtain meaningful keywords.

[1640] 3. Search Query Generation

[1641] Based on the extracted keywords, a search query is generated, which is then converted into a format that can be input to the generation AI.

[1642] 4. Book database search

[1643] Based on the generated query, the generative AI searches a book database and lists highly relevant books based on the information in the database.

[1644] 5. Organizing search results

[1645] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1646] 6. Data Transmission

[1647] The organized search results are sent to the user's terminal.

[1648] Book Database

[1649] The book database contains information about a huge number of books, which the server uses to find the most suitable book based on a search query. This database contains detailed information such as the book title, author, genre, and summary.

[1650] Specific examples

[1651] Example 1: Searching for moving adventure stories

[1652] 1. User Device

[1653] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[1654] 2. Server

[1655] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1656] The generative AI searches a book database based on the extracted keywords and lists books such as "The Alchemist" and "Life of Pi."

[1657] The resulting list is compiled and sent to the user's terminal.

[1658] 3. User Device

[1659] The search results are presented to the user as a list of books, from which the user can select books of interest.

[1660] Example 2: Looking for an easy-to-understand children's book about science

[1661] 1. User Device

[1662] A user types, "I'm looking for an easy-to-understand children's book about science."

[1663] 2. Server

[1664] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[1665] The generative AI searches for the best books based on these keywords, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1666] The organized results are sent to the user terminal.

[1667] 3. User Device

[1668] A list of search results is displayed to the user to help them select the most suitable book.

[1669] In this way, the present invention provides a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve the convenience of searching for books, especially for the elderly and young children.

[1670] The processing flow will be explained below.

[1671] ---

[1672] Step 1:

[1673] The user terminal displays an input interface for book searches to the user, which is intuitive and easy to operate, and is designed so that even the elderly and young children can use it.

[1674] Step 2:

[1675] Users input vague information or keywords about the book they are looking for, such as "a book that is an adventure story but also has an inspiring story."

[1676] Step 3:

[1677] The user terminal formats the ambiguous information entered by the user and sends it to the server, where the data has been converted into the appropriate format.

[1678] Step 4:

[1679] The server receives the ambiguous information sent from the user terminal and temporarily stores the received data.

[1680] Step 5:

[1681] The server then analyzes the received ambiguous information using natural language processing techniques, specifically tokenizing the text and extracting meaningful keywords.

[1682] Step 6:

[1683] The server generates a search query based on the analyzed keywords, which is then converted into a format that the AI ​​can understand.

[1684] Step 7:

[1685] The server uses a generation AI to search the book database, which references a large amount of book data and identifies books that are highly relevant.

[1686] Step 8:

[1687] The server organizes the generated search results and converts them into a user-friendly format, including, for example, book title, author, publication year, and a brief summary.

[1688] Step 9:

[1689] The server then sends the organized search results to the user's device, where the data is properly encrypted and securely transmitted.

[1690] Step 10:

[1691] The user terminal receives the search results sent from the server and displays them to the user in a visually easy-to-understand list format.

[1692] Step 11:

[1693] The user selects the book of interest from the displayed list of books, which may include more detailed information or related books.

[1694] ---

[1695] The above are the specific processing steps of the program.

[1696] Example 1

[1697] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1698] Conventional book search systems have the problem that it is difficult to search when you do not know the specific title or author name, or when you search based on only vague information. In addition, there is a lack of means to display search results in a format that is easy for users to understand, which makes them particularly difficult to use for the elderly and young children.

[1699] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1700] In this invention, the server includes means for receiving ambiguous information or keywords entered by a user, means for analyzing the ambiguous information or keywords using natural language processing technology, and means for generating a search query based on the ambiguous information or keywords. This allows a user to easily find a desired book by simply entering ambiguous information or keywords, even if they do not know the specific information.

[1701] The server further includes a means for searching for related book data using the generative AI model based on the generated search query, and a means for organizing detailed information about books based on the search results and displaying it to the user. This makes it possible to provide a system that displays search results in an efficient and easy-to-understand format and is easy to use, especially for the elderly and young children.

[1702] "Ambiguous information or keywords entered by the user" refers to unclear information entered when the user does not clearly know specific information, or keywords used to identify the search target.

[1703] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and process human language, and includes technology for tokenizing text data and performing semantic analysis.

[1704] "Search query" means a sentence or phrase that describes a request to a search engine or database to find specific information.

[1705] A "generative AI model" is an artificial intelligence model that makes predictions and generates based on input data, and refers to an AI that generatively outputs text, for example.

[1706] A "prompt format" refers to a text sentence that has been formatted in a specific way to be input into a generative AI model.

[1707] "Searching book data" refers to searching through book information in a database to find relevant books based on a specific search query.

[1708] "Detailed book information" refers to information about a specific book, including the title, author name, publication year, summary, etc.

[1709] "Search Results" refers to the list of relevant books and information that the system finds based on the user's input.

[1710] "Displaying to the user" refers to providing the organized search results to the user through the interface of the user terminal.

[1711] The present invention is a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author of the book. How this system works will be explained in detail below.

[1712] System configuration

[1713] The system consists of three main components:

[1714] User terminal

[1715] server

[1716] Book Database

[1717] User terminal

[1718] The user terminal is a device where users input vague information or keywords through an input interface. This terminal is equipped with an interface that is easy to use for the elderly and young children, such as large buttons and a voice input function. Users can input vague search criteria such as "I'm looking for a book that is an adventure story but also has an inspiring story."

[1719] server

[1720] The server receives vague information and keywords sent from the user's device and analyzes them using natural language processing technology. Specifically, the following process is performed:

[1721] 1. Data Reception

[1722] The server receives the ambiguous information sent from the user terminal, using a communication protocol (e.g., HTTPS) over the Internet.

[1723] 2. Data Analysis

[1724] The received data is analyzed and important keywords are extracted. Natural language processing techniques are used to tokenize the text data and obtain meaningful keywords. Specifically, Python's natural language processing libraries "NLTK" and "spaCy" are used.

[1725] 3. Search Query Generation

[1726] A search query is generated based on the extracted keywords. This query is converted into a format that can be input into a generative AI model. For example, "GPT-3" can be used as the generative AI model.

[1727] 4. Book database search

[1728] Based on the generated query, the generative AI model searches a book database and lists relevant books from the information in the database. The book database contains a large amount of book information, including detailed information such as book title, author, genre, and summary.

[1729] 5. Organizing search results

[1730] Organize search results and generate data to display them in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1731] 6. Data Transmission

[1732] The organized search results are sent to the user's terminal, which receives the data and displays it to the user.

[1733] Specific examples

[1734] Example 1: Searching for moving adventure stories

[1735] 1. User Device

[1736] A user types, "I'm looking for a book that's both an adventure story and an inspiring one."

[1737] 2. Server

[1738] The server receives this vague information and uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1739] Based on the extracted keywords, the generative AI model searches a book database and lists books such as "The Alchemist" and "Life of Pi."

[1740] The search results are organized and the list is sent to the user's terminal.

[1741] 3. User Device

[1742] The search results are presented to the user as a list of books from which the user can select books of interest.

[1743] Example 2: Looking for an easy-to-understand children's book about science

[1744] 1. User Device

[1745] A user types, "I'm looking for an easy-to-understand children's book about science."

[1746] 2. Server

[1747] The server receives this information, analyzes it using natural language processing technology, and extracts keywords such as "science," "easy to understand," and "children's books."

[1748] Based on the extracted keywords, the generative AI model searches for the most suitable books, listing titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1749] The organized results are sent to the user terminal.

[1750] 3. User Device

[1751] A list of search results is displayed to the user to help them select the most suitable book.

[1752] The above is a specific example of the "Mode for Carrying Out the Invention." This system allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This can significantly improve convenience for elderly people and young children when searching for books.

[1753] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1754] Step 1:

[1755] User input:

[1756] The user uses the device interface to input vague information or keywords, for example, "I'm looking for a book that is an adventure story but also has an inspiring plot." The input data are vague keywords or phrases related to the book the user is looking for.

[1757] Step 2:

[1758] Sending data:

[1759] The terminal sends the text data entered by the user to the server using the HTTPS protocol. This data includes the user's search request.

[1760] Step 3:

[1761] Receiving data:

[1762] The server receives the data sent from the terminal, which is an ambiguous keyword or phrase entered by the user.

[1763] Step 4:

[1764] Data analysis:

[1765] The server analyzes the received data. Specifically, it uses the Python natural language processing library "NLTK" or "spaCy" to tokenize the text data and extract meaningful keywords. For example, the keywords "adventure," "inspirational," and "story" are extracted from the text "A book that is both an adventure story and an inspiring one."

[1766] Step 5:

[1767] Search query generation:

[1768] The server generates a search query based on the extracted keywords. This query is converted into a format that can be input to the generative AI model. For example, by combining the keywords "adventure," "emotional," and "story," the prompt sentence "Please tell me a book that is both an adventure story and an emotional story" is generated.

[1769] Step 6:

[1770] Book database search:

[1771] The server searches a book database using a generative AI model (e.g., GPT-3) based on the generated query. The generative AI model lists relevant books based on the input prompt. The database contains detailed information such as book title, author, genre, and summary.

[1772] Step 7:

[1773] Search result organization:

[1774] The server organizes the search results and generates data to display them in a user-friendly format, including information such as the title, author, publication year, and a brief synopsis. For example, the server retrieves book information such as "The Alchemist," "Paulo Coelho," "1988," and "An adventure story that shows the journey of life."

[1775] Step 8:

[1776] Submit search results:

[1777] The server sends the organized search results to the user terminal, which receives the data and displays it to the user.

[1778] Step 9:

[1779] Displaying search results:

[1780] The user terminal displays the search results sent from the server to the user. The user can then select books that interest them based on the displayed search results. For example, a list of books such as "The Alchemist" and "Life of Pi" may be displayed.

[1781] (Application example 1)

[1782] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1783] When users search for a book, it is difficult to quickly and accurately find the desired book even if they only have vague information. It is also difficult to immediately identify where the book is actually located in the bookstore, which makes it inconvenient for a wide range of users, including the elderly and young children. To solve this problem, a system is needed that can accurately identify a book even with vague information and easily find where that book is located in the bookstore.

[1784] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1785] In this invention, the server includes means for receiving vague information or keywords entered by a user, means for automatically searching for related book data based on the vague information or keywords, means for displaying search results to the user, means for identifying the location of the book in the bookstore based on the search results, and means for displaying a route to the identified location. This allows the user to accurately identify a book even with vague information, and then easily find where it is located in the bookstore.

[1786] "Ambiguous information" refers to information that is general or vague and not specifically defined by the user.

[1787] "Keywords" are specific words or short phrases that are necessary to conduct a search for a book.

[1788] "Natural language processing technology" is a technology that enables computers to understand, interpret, and generate human language, and is used to analyze ambiguous user input.

[1789] "Book data" is data consisting of information such as the book title, author, genre, and summary.

[1790] "Search results" are a collection of related book data that are automatically searched based on the user's vague information or keywords.

[1791] A "user terminal" is a device used by a user to input information, including a smartphone or tablet.

[1792] "Server" refers to a central processing unit that receives data sent from user terminals and analyzes and processes it.

[1793] The "location of a book in a bookstore" is the specific location where a particular book is located in a bookstore.

[1794] "Displaying a route" means visually showing a customer the route they should take to reach a particular book within the bookstore.

[1795] The present invention provides a system for effectively searching for books in a bookstore. A specific embodiment for implementing this system will be described below.

[1796] System configuration

[1797] The system consists of three main components:

[1798] User terminal

[1799] server

[1800] Book Database

[1801] User terminal

[1802] A user terminal is a device that allows a user to input information and receive results. Examples include smartphones and tablets. Here, we will use a smartphone as an example. A user launches an app on the smartphone and uses the voice input function to input vague information. For example, a user might input a search request such as, "I want a book with a scary but moving story."

[1803] server

[1804] The server receives the ambiguous information sent from the user terminal and analyzes and processes it. Specifically, it uses the following hardware and software:

[1805] Server hardware: general data center servers and cloud servers

[1806] Software: Google Cloud Natural Language API, OpenAI GPT-4 model

[1807] Processing flow

[1808] 1. Data Reception

[1809] The server receives the voice data sent from the user's device, which is converted into text using the Google Cloud Speech-to-Text API.

[1810] 2. Data Analysis

[1811] The converted text data is then analyzed using the Google Cloud Natural Language API to extract meaningful keywords, such as "scary," "moving," and "story."

[1812] 3. Search Query Generation

[1813] Based on the extracted keywords, a search query is generated by generative AI (OpenAI GPT-4 model). For example, a prompt sentence such as "Generate a search query for a book that is scary yet emotional for bookstore inventory search" is input.

[1814] 4. Book database search

[1815] The generated query is then used to search a book database, which contains detailed information such as the book's title, author, genre, and summary, to produce a list of related books.

[1816] 5. Organizing search results

[1817] Search results are organized and presented in a user-friendly format, including information such as title, author, publication year, and a brief summary.

[1818] 6. Data Transmission

[1819] The organized search results are sent to the user's terminal and displayed on the screen.

[1820] 7. Bookstore location and route guidance

[1821] When a user selects a particular book, the location of that book in the bookstore is identified and displayed on the smartphone, and a store map is also displayed, providing directions to the selected book's location.

[1822] Specific examples

[1823] Example 1: Looking for a book with a scary but moving story

[1824] 1. User Device

[1825] A user types, "I want a book with scary but moving stories."

[1826] 2. Server

[1827] The server receives this vague information and converts it into text using the Google Cloud Speech-to-Text API.

[1828] Using the Google Cloud Natural Language API, keywords such as "scary," "moving," and "story" are extracted.

[1829] Based on the extracted keywords, the generative AI (OpenAI GPT-4 model) inputs the prompt sentence "Generate a search query for a book that is scary yet emotional for bookstore inventory search" and generates a search query.

[1830] A book database is searched based on the generated query, and related books are listed.

[1831] The search results are organized and sent to the user's terminal.

[1832] 3. User Device

[1833] The system presents the user with a list of books displayed as search results, and also displays the location and route to the selected book in the store, providing guidance to the specific location.

[1834] In this way, the present invention provides a system that can accurately identify a book and easily locate its location in a bookstore, even if the user has only vague information, making it possible for a wide range of users, including the elderly and young children, to easily search for books.

[1835] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1836] Step 1:

[1837] A user launches a smartphone application and uses the voice input function to enter a vague search request, specifically, "I want a book with a scary but moving story."

[1838] Input: User's voice data.

[1839] Output: Audio data collected through the smartphone microphone.

[1840] Specific behavior: The smartphone application receives voice input and prepares it to be sent to the system.

[1841] Step 2:

[1842] The device sends the voice data to the server, which converts it into text using the Google Cloud Speech-to-Text API.

[1843] Input: Audio data sent from a smartphone.

[1844] Output: Text data (e.g., "I want a book with a scary but moving story").

[1845] Specific operation: The server receives the audio data and converts it into text using the Google Cloud Speech-to-Text API.

[1846] Step 3:

[1847] The server analyzes the text data using the Google Cloud Natural Language API and extracts important keywords.

[1848] Input: Text data converted from audio data.

[1849] Output: A set of keywords (e.g. "scary", "emotional", "story").

[1850] Specific operation: The server analyzes the text data and performs tokenization, stop word removal, and keyword extraction.

[1851] Step 4:

[1852] The server inputs a prompt into the generative AI (OpenAI GPT-4 model) to generate a search query. Example prompt: "Generate a search query for a book that is scary yet emotional for bookstore inventory search."

[1853] Input: A set of keywords.

[1854] Output: The search query created by the generative AI.

[1855] Specific operation: The server inputs a prompt sentence into the generation AI to generate an appropriate search query.

[1856] Step 5:

[1857] The server searches a book database using the generated search query.

[1858] Input: A search query generated by the generative AI.

[1859] Output: A list of related books.

[1860] Specific operation: The server executes a search query against a book database and lists related books.

[1861] Step 6:

[1862] The server organizes the search results and summarizes important information (title, author, summary, etc.).

[1863] Input: A list of books.

[1864] Output: An organized list of book information.

[1865] Specific operation: The server collects book data and organizes it into a format that is easy for users to understand.

[1866] Step 7:

[1867] The server sends the organized search results to the user's terminal, which displays the information.

[1868] Input: An organized list of book information.

[1869] Output: Search results displayed on a smartphone screen.

[1870] Specific operation: The server sends the data and the smartphone displays the results on the screen.

[1871] Step 8:

[1872] When a user selects a particular book, the server identifies the book's location in the bookstore and sends that information to the user's device, which then displays a store map and route guidance on the smartphone.

[1873] Input: The book selected by the user.

[1874] Output: Book location and directions.

[1875] Specific operation: The server queries the book location information, generates route guidance, and sends it to the user's device. The smartphone displays a map of the store and provides route guidance.

[1876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1877] ---

[1878] The present invention provides a system that enables a user to effectively search for books using vague information or keywords even when the user does not know the specific title or author of the book, and further combines it with an emotion engine that recognizes the user's emotions and optimizes the search results.

[1879] System configuration

[1880] The system consists of four main components:

[1881] User terminal

[1882] server

[1883] Book Database

[1884] Emotion Engine

[1885] User terminal

[1886] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for a book that is an adventure story but has an inspiring story." In response to this input, the system collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[1887] server

[1888] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[1889] 1. Data Reception

[1890] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[1891] 2. Data Analysis

[1892] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[1893] 3. Search Query Generation

[1894] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into the generative AI.

[1895] 4. Book database search

[1896] The generative AI searches a book database based on the generated search query, taking into account the user's emotional state and identifying highly relevant books.

[1897] 5. Search Result Tailoring

[1898] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[1899] 6. Submit search results

[1900] The organized search results are sent to the user's terminal.

[1901] Emotion Engine

[1902] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[1903] Audio analysis

[1904] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[1905] facial expression analysis

[1906] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[1907] Specific examples

[1908] Example 1: Searching for moving adventure stories

[1909] 1. User Device

[1910] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[1911] 2. Server

[1912] The server receives this vague information, uses natural language processing technology to extract keywords such as "adventure," "emotional," and "story," and uses an emotion engine to analyze that the user is looking for something moving.

[1913] The generative AI searches a book database based on keywords and emotional state to produce a list of books such as "The Alchemist" and "Life of Pi."

[1914] Organize search results to prioritize inspiring stories at the top of the list.

[1915] 3. User Device

[1916] The search results list is displayed to the user, who then selects the desired book.

[1917] Example 2: Looking for an easy-to-understand children's book about science

[1918] 1. User Device

[1919] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[1920] 2. Server

[1921] The server receives this information, extracts keywords such as "science," "easy to understand," and "children's books," and uses an emotion engine to analyze whether the user is expressing curiosity.

[1922] Generative AI searches for the best books based on keywords and emotional state, resulting in a list of titles like "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[1923] It reflects your emotional state and prioritizes content that will interest you.

[1924] 3. User Device

[1925] A list of search results is displayed to the user, and the user can select books that interest them.

[1926] This invention combines an emotion engine and generative AI to provide a system that allows users to easily find the book they want by simply entering vague information or keywords, even if they do not know the specific information. This will greatly improve convenience for people, especially the elderly and young children, when searching for books.

[1927] The processing flow will be explained below.

[1928] ---

[1929] Step 1:

[1930] The user terminal displays an input interface for book searches, which includes a text input box, a voice input function, and a camera for face recognition.

[1931] Step 2:

[1932] The user inputs vague information or keywords about the book they are looking for. For example, they input vague criteria such as "a book that is an adventure story but also has an inspiring story" by voice or text. During this process, the user's voice and facial expression data are collected in real time.

[1933] Step 3:

[1934] The user terminal formats the input information from the user and sends it to the server along with emotion data. At this time, the data is encrypted and sent securely.

[1935] Step 4:

[1936] The server receives the ambiguous information and emotion data sent from the user terminal, and the received data is temporarily stored in a database.

[1937] Step 5:

[1938] The server analyzes the received fuzzy information using natural language processing techniques, which involves tokenizing the text and extracting key keywords.

[1939] Step 6:

[1940] The server uses an emotion engine to analyze the transmitted emotion data, identifying emotions from the voice data based on tone, speed, emphasis, etc., and from the facial expression data, determining the emotional state based on the movements and changes of facial muscles.

[1941] Step 7:

[1942] The server generates a search query based on the analyzed keywords and sentiment data, which is then converted into a format that the generative AI can understand.

[1943] Step 8:

[1944] The server uses a generation AI to search the book database, which takes into account the user's emotions and identifies highly relevant books from the database.

[1945] Step 9:

[1946] The server then performs a process of organizing the search results, listing data including title, author, publication year, and brief summary, and adjusting the display order to reflect the user's emotional state.

[1947] Step 10:

[1948] The server sends the organized search results to the user terminal, where the data is encrypted for efficient transmission.

[1949] Step 11:

[1950] The user terminal receives the search results sent from the server, analyzes the received data, and displays them to the user in a visually easy-to-understand list format.

[1951] Step 12:

[1952] The user selects the book they are interested in from the presented book list, and detailed information about the selected book and related further information is also displayed.

[1953] Examples:

[1954] Example 1: Searching for moving adventure stories

[1955] Step 1:

[1956] The user terminal displays an input interface.

[1957] Step 2:

[1958] The user speaks, "I'm looking for a book that is an adventure story but also has an inspiring plot," and voice and facial expression data are also collected.

[1959] Step 3:

[1960] The user terminal transmits the input information and emotion data to the server.

[1961] Step 4:

[1962] The server receives the ambiguous information and the emotion data.

[1963] Step 5:

[1964] The server uses natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[1965] Step 6:

[1966] The server uses an emotion engine to analyze the strong desire for emotion.

[1967] Step 7:

[1968] The server generates a search query based on keywords and emotion data.

[1969] Step 8:

[1970] The server uses generative AI to search a book database and list "The Alchemist" and "Life of Pi."

[1971] Step 9:

[1972] The server prioritizes inspiring stories and displays them at the top of the list.

[1973] Step 10:

[1974] The server transmits the organized search results to the user terminal.

[1975] Step 11:

[1976] The user terminal receives and displays the search results.

[1977] Step 12:

[1978] The user selects the desired book from the displayed list.

[1979] ---

[1980] The above are the specific processing steps of the book search system that combines the emotion engine.

[1981] Example 2

[1982] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1983] In conventional book search systems, if a user does not know the specific book title or author name, it is difficult to find the appropriate book when entering vague keywords or information. Furthermore, since search results are provided without taking the user's emotional state into consideration, users may not be able to achieve the emotional satisfaction they are looking for. This makes it particularly difficult for elderly people and young children to search for books efficiently.

[1984] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1985] In this invention, the server includes means for analyzing ambiguous information and keywords entered by the user and the user's emotional data using natural language processing technology, means for searching a book database using a generative AI model, and means for adjusting and displaying search results according to the user's emotional state. This enables the user to effectively find the most suitable book according to their emotional state by simply entering ambiguous information and keywords, even if they do not know the specific information.

[1986] "Ambiguous information and keywords" refers to vague search criteria that a user enters when they do not know the specific title or author of a book.

[1987] "Emotional data" refers to information about the user's psychological state and emotions obtained by analyzing their voice and facial expressions.

[1988] "Natural language processing technology" refers to artificial intelligence technology for understanding, analyzing, and generating human language.

[1989] A "generative AI model" is a type of artificial intelligence that is trained in advance using large amounts of data and generates sentences and search results according to specific conditions.

[1990] A "book database" refers to a collection of data that aggregates information about books.

[1991] An "emotion engine" refers to a technology or system that analyzes data such as voice and facial expressions to recognize a user's emotions.

[1992] "Analysis results" refers to keywords extracted from input information using natural language processing technology and the emotional state recognized by the emotion engine.

[1993] "Search Results" refers to a list of relevant books searched through a book database using a generative AI model.

[1994] This invention provides a system that allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. In addition, by combining it with an emotion engine, search results can be optimized based on the user's emotions.

[1995] System configuration

[1996] The system consists of the following main components:

[1997] User terminal

[1998] server

[1999] Book Database

[2000] Emotion Engine

[2001] Processing flow

[2002] User terminal

[2003] A user terminal is a device into which a user inputs information or keywords. Examples include a smartphone, tablet, or PC. When a user inputs, "I'm looking for a book that is an adventure story but also has an inspiring plot," the terminal uses voice input and a camera to collect the user's emotional data. This emotional data includes voice tone and facial expressions. Hardware such as a microphone and a camera are used. Software such as Google Speech-to-Text and OpenCV are used.

[2004] server

[2005] The server receives and temporarily stores ambiguous information and emotional data sent from the user's device. The server has the following functions:

[2006] Data analysis

[2007] The server uses TensorFlow to execute natural language processing techniques to extract important keywords from ambiguous information, such as "adventure," "moving," and "story."

[2008] At the same time, an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) is used to analyze the user's emotions from voice and facial expression data.

[2009] Search Query Generation

[2010] A prompt sentence is generated based on the extracted keywords and the analyzed sentiment data. This prompt sentence is configured as an input to a generative AI model (such as GPT-4). For example, "Please recommend the best book for an excited user looking for an inspiring adventure story."

[2011] Database search

[2012] The generative AI model searches a book database based on the generated prompt. This book database is configured using a database management system such as MySQL. The generative AI model lists highly relevant books based on the search results.

[2013] Tailoring and Submitting Search Results

[2014] The server will adjust search results based on the user's emotional state, for example, if the user is in a state where they want inspiring stories, it will prioritize books that tell such stories.

[2015] The adjusted search results are transmitted to the user terminal.

[2016] Emotion Engine

[2017] The emotion engine is a component that recognizes emotions from the user's voice and facial expression data. It has the following functions:

[2018] Audio analysis

[2019] Use speech analysis software (e.g., Nuance or Google Speech-to-Text) to analyze speech tone, speed, pauses, etc. to identify emotion.

[2020] facial expression analysis

[2021] Using facial expression analysis software (e.g., OpenCV or Microsoft Azure Face API), the system analyzes facial muscle movements and changes in facial expressions to recognize the user's emotions.

[2022] Specific examples

[2023] Example 1: Searching for moving adventure stories

[2024] User terminal

[2025] The user types, "I'm looking for a book that is both an adventure story and an inspiring one," and at the same time, emotional data is collected via the camera and microphone.

[2026] server

[2027] 1. Receive vague information and use natural language processing technology to extract keywords such as "adventure," "moving," and "story."

[2028] 2. The emotion engine analyzes the user's emotions and determines that they are "seeking emotion."

[2029] 3. A generative AI model searches a book database based on keywords and emotional state to produce a list of books like "The Alchemist" and "Life of Pi."

[2030] 4. Organize search results to prioritize inspiring stories.

[2031] User terminal

[2032] The search results list is displayed to the user, who then selects the desired book.

[2033] Example 2: Looking for an easy-to-understand children's book about science

[2034] User terminal

[2035] The user types, "I'm looking for an easy-to-understand children's book about science," and speech and facial expression data is collected.

[2036] server

[2037] 1. Receive information and extract keywords such as "science," "easy to understand," and "children's book."

[2038] 2. The emotion engine analyzes the user's emotion as "showing curiosity."

[2039] 3. The generative AI model searches for suitable books and lists titles such as "A Brief History of Time for Young Readers" and "The Magic School Bus series."

[2040] 4. Reflect your emotional state and prioritize content that will interest you.

[2041] User terminal

[2042] A list of search results is displayed to the user, and the user can select books that interest them.

[2043] This invention provides a system that allows users who do not know specific information to easily find the most suitable book according to their emotional state by simply entering vague information or keywords, greatly improving the convenience of book searches, especially for the elderly and young children.

[2044] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2045] Step 1:

[2046] The user operates the device and inputs vague search criteria using text or voice. For example, they might input, "I'm looking for a book that is an adventure story but also has an inspiring plot." At the same time, the device uses a camera to capture the user's facial expressions and a microphone to record their voice. The inputs here are the user's text / voice input and emotional data. The device then collects the user's facial expression and voice data and sends it to the server.

[2047] Step 2:

[2048] The server receives and temporarily stores ambiguous information and emotion data sent from the user's device. The input data is the user's text / voice input and emotion data. Based on this, the server stores the data in a database and prepares it for subsequent analysis.

[2049] Step 3:

[2050] The server uses TensorFlow to execute natural language processing techniques and extract important keywords from ambiguous information. The input data is text / voice input provided by the user. The data is processed by tokenizing the text and extracting important keywords (e.g., "adventure," "moving," "story"). The output is the extracted keywords.

[2051] Step 4:

[2052] The server uses an emotion engine (e.g., Affectiva or Microsoft Azure Emotion API) to analyze emotions from the user's voice and facial expression data. The input data is the user's emotional data (voice tone and facial muscle movements). Data calculations identify the user's emotional state (e.g., "desiring to be moved"). The output is the analyzed emotional information.

[2053] Step 5:

[2054] The server generates a prompt sentence to be input to the generative AI model (GPT-4) based on the extracted keywords and analyzed emotional data. The input data is keywords and emotional state. Based on this, a prompt sentence such as "Recommend the best book for an excited user looking for an inspiring adventure story" is created as a search query. The output is the generated prompt sentence.

[2055] Step 6:

[2056] The generative AI model searches a book database based on the generated prompt. The input data is the prompt. A database search is performed to generate a list of highly relevant books. For example, search results may include "The Alchemist" and "Life of Pi." The output is a list of books.

[2057] Step 7:

[2058] The server adjusts search results based on the user's emotional state. The input data is a book list and the user's emotional state. The server processes the data to adjust the results and reorganizes the list to prioritize moving stories. The output is the adjusted book list.

[2059] Step 8:

[2060] The server sends the adjusted search results to the user terminal. The input data is the adjusted book list. Based on this, the server transfers the search results to the user terminal. The output is the data sent to the user terminal.

[2061] Step 9:

[2062] The user terminal receives the search results sent from the server and displays them on the screen. The input data is the adjusted book list. Based on this, the search results are visually presented to the user. The user can then select the book they want from the displayed list.

[2063] (Application example 2)

[2064] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2065] In modern book search systems, if a user does not know the specific title or author name, it is difficult to effectively search for a book using vague information or keywords. Furthermore, conventional systems do not optimize search results by taking into account the user's emotional state, making it difficult for users to find the book they want. Therefore, it is necessary to provide a book search system that reflects emotional data based on the user's vague information and keywords.

[2066] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2067] In this invention, the server includes means for receiving ambiguous information and keywords input by a user, means for automatically searching for related book data based on the ambiguous information and keywords, means for searching for the book data based on a generative AI model, means for inputting a prompt sentence to the generative AI model to obtain search results, means for optimizing the search results in consideration of the user's emotional data, and means for displaying the optimized search results to the user. This enables an optimal book search that reflects the user's emotional data based on the user's ambiguous information and keywords.

[2068] "Ambiguous information" refers to unclear keywords or fragmentary information that is entered when the user does not know the specific title or author's name.

[2069] A "keyword" refers to a specific word or short phrase that a user enters when searching, and is information that serves as a criterion for narrowing down search results.

[2070] A "generative AI model" is a system that uses artificial intelligence technology to generate responses based on specific search queries and prompts.

[2071] A "prompt" is a search query or instruction given to a generative AI model, which functions as a parameter to obtain appropriate search results.

[2072] "Emotion data" is data that indicates the emotional state of the user obtained from the user's voice and facial expressions, and is a quantitative evaluation of emotions such as joy and sadness.

[2073] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and is a technique for unconsciously processing ambiguous information and keywords.

[2074] "Search results" are lists of related books presented by the system based on vague information, keywords, and emotional data entered by the user.

[2075] "Optimization" means taking emotional data into account and adjusting search results to best meet the user's needs and emotions.

[2076] "Display means" means an interface device for visually presenting search results and other related information to a user.

[2077] This system allows users to effectively search for books using vague information or keywords, even if they do not know the specific title or author. Furthermore, by combining it with an emotion engine that recognizes the user's emotions and optimizes search results, it can recommend the most relevant books to the user.

[2078] System configuration

[2079] This system is broadly composed of the following components:

[2080] User terminal

[2081] server

[2082] Generative AI Models

[2083] Book Database

[2084] Emotion Engine

[2085] User terminal

[2086] The user terminal is a device into which the user inputs vague information or keywords through an input interface. This terminal is equipped with the function of recognizing the user's emotions through voice input and a camera. For example, a user can input vague search criteria such as "I'm looking for adventure stories." In response to this input, the terminal collects the user's emotions from their voice and facial expressions, and sends the data to the server.

[2087] server

[2088] The server receives the vague information, keywords, and emotion data sent from the user device and performs the following processes:

[2089] 1. Data Reception

[2090] The server receives the ambiguous information and emotion data sent from the user terminal and temporarily stores them.

[2091] 2. Data Analysis

[2092] It uses natural language processing technology to analyze ambiguous information and extract important keywords, while using an emotion engine to analyze emotional data and identify the user's current emotional state.

[2093] 3. Search Query Generation

[2094] Based on the analyzed keywords and sentiment data, a search query is generated, which is then converted into a format that can be fed into a generative AI model.

[2095] 4. Book database search

[2096] The generative AI model searches a book database based on the generated search query, taking into account the user's emotional state to identify highly relevant books.

[2097] 5. Search Result Tailoring

[2098] In the process of sorting the search results, the emotional engine recognizes the user's emotions and adjusts the display order and suggestions accordingly. For example, if the user is in an emotional state looking for a moving story, the system will prioritize books that fit that sentiment.

[2099] 6. Submit search results

[2100] The organized search results are sent to the user's terminal.

[2101] Emotion Engine

[2102] The emotion engine is a component that recognizes emotions from the user's voice and facial expressions. It includes the following functions:

[2103] Audio analysis

[2104] It analyzes the user's voice input to identify emotions, such as joy, sadness, and excitement, based on the tone, speed, and pauses of the voice.

[2105] facial expression analysis

[2106] The system analyzes the user's facial expressions through a camera and recognizes their emotions. It determines their emotional state based on the movements of their facial muscles and changes in their expressions.

[2107] Hardware and software used

[2108] User device: Smartphone (iPhone or Android device)

[2109] Analysis software:

[2110] Speech Recognition: Google Speech Recognition API (using the speech_recognition library)

[2111] Facial expression analysis: facial_recognition library

[2112] Sentiment Analysis: TextBlob (Sentiment analysis using natural language processing)

[2113] Server side:

[2114] REST API (e.g. bookrstore.com API)

[2115] Book Database

[2116] Examples and prompts

[2117] Examples:

[2118] A user launches a smartphone app and inputs "memorable." The user then selects "joy" as the emotion. Based on this information, books such as "The Alchemist" are recommended.

[2119] Prompt statement:

[2120] User-entered keyword: "memorable"

[2121] Emotion data: {"joy": 0.9, "sadness": 0.1}

[2122] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2123] Step 1:

[2124] The user uses a smartphone to input vague information or keywords. The user also simultaneously inputs emotional data using voice input or a camera. The input at this stage consists of text keywords and audio / image data.

[2125] Step 2:

[2126] The user's device sends the entered vague information or keywords, as well as audio and image data, to the server, where the audio data is converted into text using speech recognition software and the image data is analyzed using facial expression analysis software.

[2127] Step 3:

[2128] The server receives vague information, keywords, and emotion data sent from the user terminal. The received data is temporarily stored in a database. The input data is keyword text and emotion data.

[2129] Step 4:

[2130] The server uses natural language processing technology to analyze ambiguous information and extract important keywords. The server performs text analysis using TextBlob. It also uses an emotion engine to identify emotions from voice data and facial expression data. The analyzed keywords and emotion data are output.

[2131] Step 5:

[2132] The server generates a search query based on the analyzed keywords and sentiment data. This query is converted into a prompt sentence to be input to the generative AI model. The generated prompt sentence is output.

[2133] Step 6:

[2134] The server inputs a prompt into the generative AI model and searches the book database. The generative AI model retrieves relevant book data based on the prompt, and outputs the search results.

[2135] Step 7:

[2136] The server organizes the search results and optimizes them based on the emotional data recognized by the emotion engine. For example, if a user is looking for an inspiring story, relevant books will be prioritized in the list. The optimized search results are output.

[2137] Step 8:

[2138] The server sends the optimized search results to the user's device, which visually displays the results and presents them in a user-selectable format. The output at this stage is a list of related books and their detailed information.

[2139] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2140] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2141] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2142] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2143] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2144] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2145] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2146] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2147] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2148] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2149] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2150] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2151] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2152] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2153] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2154] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2155] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2156] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2157] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2158] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2159] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2160] The following is further disclosed regarding the above embodiment.

[2161] (Claim 1)

[2162] a means for receiving vague information or keywords input by a user;

[2163] A means for automatically searching for related book data based on the vague information or keywords;

[2164] means for displaying the search results to a user;

[2165] A system including:

[2166] (Claim 2)

[2167] 2. The system according to claim 1, further comprising means for analyzing the ambiguous information and keywords using natural language processing technology.

[2168] (Claim 3)

[2169] 2. The system according to claim 1, further comprising means for organizing detailed information about books based on the search results and displaying the information to the user.

[2170] "Example 1"

[2171] (Claim 1)

[2172] a means for receiving vague information or keywords input by a user;

[2173] A means for analyzing the ambiguous information and keywords using natural language processing technology;

[2174] means for generating a search query based on the fuzzy information or keywords;

[2175] A means for searching for related book data using a generation AI model based on the generated search query;

[2176] a means for organizing detailed information about books based on the search results and displaying the information to a user;

[2177] A system including:

[2178] (Claim 2)

[2179] 10. The system of claim 1, wherein the search query is converted into a prompt sentence format for a generative AI model.

[2180] (Claim 3)

[2181] The system of claim 1, wherein the generative AI model analyzes a search query sent from a user terminal and lists related books.

[2182] "Application Example 1"

[2183] Claiming a new invention

[2184] (Claim 1)

[2185] a means for receiving vague information or keywords input by a user;

[2186] A means for automatically searching for related book data based on the vague information or keywords;

[2187] means for displaying the search results to a user;

[2188] means for locating the book in the bookstore based on the search results;

[2189] a means for displaying a route to the identified location;

[2190] A system including:

[2191] (Claim 2)

[2192] 2. The system according to claim 1, further comprising means for analyzing the ambiguous information and keywords using natural language processing technology.

[2193] (Claim 3)

[2194] 2. The system according to claim 1, further comprising means for organizing detailed information about books based on the search results and displaying the information to the user.

[2195] "Example 2: Combining Emotion Engines"

[2196] (Claim 1)

[2197] a means for receiving vague information or keywords input by a user;

[2198] A means for analyzing the ambiguous information, keywords, and user emotion data using natural language processing technology;

[2199] A means for searching a book database using a generated AI model based on the analysis results;

[2200] means for adjusting and displaying the search results according to the emotional state of the user;

[2201] A system including:

[2202] (Claim 2)

[2203] 2. The system according to claim 1, further comprising means for recognizing emotions from the user's voice and facial expressions using an emotion engine.

[2204] (Claim 3)

[2205] 2. The system according to claim 1, further comprising means for organizing detailed information about books based on the search results and displaying the information in an order that is most suited to the emotional state of the user.

[2206] "Application example 2 when combining emotion engines"

[2207] (Claim 1)

[2208] a means for receiving vague information or keywords input by a user;

[2209] A means for automatically searching for related book data based on the vague information or keywords;

[2210] A means for searching the book data based on a generation AI model;

[2211] a means for inputting a prompt sentence into the generative AI model to obtain search results;

[2212] means for optimizing the search results by taking into account the emotion data of the user;

[2213] means for displaying the optimized search results to a user;

[2214] A system including:

[2215] (Claim 2)

[2216] 2. The system according to claim 1, further comprising means for analyzing the ambiguous information and keywords using natural language processing technology.

[2217] (Claim 3)

[2218] 2. The system according to claim 1, further comprising means for organizing detailed information about books based on the search results and displaying the information to the user. [Explanation of symbols]

[2219] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving vague information or keywords input by a user; A means for automatically searching for related book data based on the vague information or keywords; means for displaying the search results to a user; A system including:

2. The system according to claim 1, further comprising means for analyzing the ambiguous information and keywords using natural language processing technology.

3. 2. The system according to claim 1, further comprising means for organizing detailed information about books based on the search results and displaying the information to the user.

Images