System

Abstract

To provide a system.SOLUTION: A system includes: means for capturing expert knowledge, latest information, and personal information; means for converting a voice of a user into a text using a voice recognition technique; means for analyzing information collected using a natural language processing technique and utilizing the information as learning data of a generative AI model; means for generating an appropriate answer on the basis of an input of the user using the generative AI model; and means for converting a text response generated using a voice synthesis technique into a voice and reproducing it to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional generative AI has difficulty incorporating specialized knowledge, up-to-date information, and personal information, and there are no systems that can converse with humanoid AI beyond text. Furthermore, there is a demand for systems in which each AI operates independently and generates and provides information according to its role, as well as systems in which each AI shares information with other AIs to provide consistent information as a whole. [Means for solving the problem]

[0005] This invention provides a system that includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, and means for providing information together with security and platform information. Furthermore, it also provides means for each AI to operate independently and generate and provide information according to its respective role, and means for each AI to share information with other AIs and provide consistent information as a whole. This solves the problems of the past. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, information is collected from various databases and internet sources as a means of incorporating specialized knowledge, the latest information, and personal information, and this information is used as learning data for the AI. Furthermore, as a means of conversing with a humanoid AI, not just through text, speech recognition technology and natural language processing technology are used to enable dialogue with the user.

[0029] "Example 2"

[0030] As a means of assigning roles to each AI, for example, some AIs could provide the latest news, while others could provide recommended information based on the user's personal information, etc. In addition, as a means of providing information together with security and platform information, a system will be built to centrally manage the information generated by each AI and provide it to users in an appropriate format.

[0031] "Example 3"

[0032] Each AI operates independently, and as a means of generating and providing information according to its role, each AI has its own learning data and algorithms, and generates and provides information independently.In addition, each AI shares information with other AIs, and as a means of providing consistent information as a whole, a communication protocol is set up for sharing information between AIs, realizing consistent information provision as a whole.

[0033] The processing flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Gather specialized knowledge, current information, and personal information from various databases and online sources.

[0036] Step 2: Use the collected information as learning data for the AI ​​to update its knowledge.

[0037] Step 3: Use voice recognition and natural language processing technologies to enable dialogue with the user.

[0038] "Example 2"

[0039] Step 1: Assign each AI a specific role. For example, some AIs can provide news updates, while others can provide personalized recommendations.

[0040] Step 2: Build a system that centrally manages the information generated by each AI and provides it to users in an appropriate format.

[0041] "Example 3"

[0042] Step 1: Each AI has its own training data and algorithms, and each generates and provides information independently.

[0043] Step 2: Establish a communication protocol for sharing information between AIs to ensure consistent information provision across the board.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] Conventional information provision systems have had difficulty efficiently collecting specialized knowledge, the latest information, and personal information, and providing consistent information to users. Even in dialogue systems using voice recognition and natural language processing technologies, it was difficult to generate appropriate responses based on user input, limiting the user experience. Furthermore, each AI operates independently and cannot share information, making it difficult to provide consistent information overall.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes a means for collecting specialized knowledge, the latest information, and personal information, a means for conversing with a humanoid AI not only through text, a means for converting a user's voice into text using speech recognition technology, a means for analyzing collected information using natural language processing technology and using it as training data for a generative AI model, a means for generating appropriate responses based on user input using a generative AI model, a means for converting the generated text responses into speech using speech synthesis technology and playing them back to the user, a means for assigning roles to each AI, and a means for providing information together with security and platform information. This enables the efficient collection of specialized knowledge, the latest information, and personal information and the provision of consistent information to users. Furthermore, even in dialogue systems using speech recognition technology and natural language processing technology, appropriate responses can be generated based on user input, improving the user experience. Furthermore, each AI can share information, providing consistent information overall.

[0049] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0050] "Latest news" is the latest data and news about current situations and events.

[0051] "Personal Information" means data or information relating to an individual.

[0052] "Humanoid AI" is artificial intelligence that mimics human-like dialogue and behavior.

[0053] "Speech recognition technology" is a technology that converts speech into text.

[0054] "Natural language processing technology" is a technology for analyzing and understanding human language.

[0055] A "generative AI model" is an artificial intelligence model that generates new data or responses based on input data.

[0056] "Speech synthesis technology" is a technology that converts text into speech.

[0057] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0058] "Security" refers to the techniques and methods used to protect information and systems.

[0059] A "platform" is a system that serves as the foundation for providing software and services.

[0060] The present invention is a system for efficiently collecting specialized knowledge, up-to-date information, and personal information, and providing consistent information to users. A specific embodiment of this system will be described below.

[0061] Information gathering

[0062] The server collects specialized knowledge, up-to-date information, and personal information from various databases and internet sources. This information is collected using web crawlers and APIs. For example, it uses a news site's API to retrieve the latest news articles. The server periodically checks these sources to see if there is new information.

[0063] Data processing and learning

[0064] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The extracted data is saved as training data for the generative AI model.

[0065] Recognizing user voice

[0066] When a user speaks into the device, the device captures the audio using its built-in microphone. The device converts the captured audio into text using the Google® Cloud Speech-to-Text API. The converted text is then sent to the server.

[0067] Text analysis and response generation

[0068] The server analyzes the received text. For analysis, it uses OpenAI's GPT-3 (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response.

[0069] Audio Output

[0070] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is then played back to the user through the device's speaker. For example, if the generated response is "The latest news is...", that content is conveyed to the user as speech.

[0071] Specific examples

[0072] Example 1: Getting and interacting with news

[0073] When a user says, "Tell me the latest news," the device converts the speech into text and sends it to the server. The server retrieves the latest news from the news site's API and uses a generative AI model to generate a response such as, "The latest news is ____." The device then converts this response into audio and plays it back to the user.

[0074] Example 2: Use of personal information

[0075] When a user says, "What's on my agenda for today?", the device converts the speech into text and sends it to the server. The server retrieves the user's calendar information and uses a generative AI model to generate a response such as "Today's agenda is ____." The device then converts this response into audio and plays it back to the user.

[0076] Prompt Sentence Examples

[0077] "Tell me the latest news"

[0078] "What are my plans for today?"

[0079] "Tell me the weather forecast."

[0080] The system is designed to allow users to obtain information through natural interaction.

[0081] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0082] Step 1: Gather information

[0083] The server collects specialized knowledge, the latest information, and personal information from various databases and information sources on the Internet. Specifically, it acquires information using a web crawler or API. For example, it acquires the latest news articles using a news site's API. The input is the news site's API endpoint, and the output is the retrieved news article data.

[0084] Step 2: Data processing and training

[0085] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The input is the collected news article data, and the output is the extracted keywords and context information. This data is saved as training data for the generative AI model.

[0086] Step 3: Recognizing user speech

[0087] When a user speaks into the device, the device captures the voice using the built-in microphone. The device converts the captured voice into text using the Google® Cloud Speech-to-Text API. The input is the user's voice data, and the output is the converted text data. This text data is sent to the server.

[0088] Step 4: Text analysis and response generation

[0089] The server analyzes the received text. For the analysis, it uses GPT-3 (registered trademark) from OpenAI (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response. The input is the text data sent by the user, and the output is the generated response text.

[0090] Step 5: Audio Output

[0091] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is played back to the user through the device's speaker. For example, if the generated response is "The latest news is ____," that content is conveyed to the user as speech. The input is the response text received from the server, and the output is the converted speech data.

[0092] (Application example 1)

[0093] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Autonomous vehicles are required to collect traffic and weather information in real time and suggest optimal routes and rest stops through dialogue with the user. However, current systems collect and provide information in a fragmented manner, making it difficult to meet the user's personalized needs. In addition, dialogue systems using voice recognition technology and natural language processing technology are not fully integrated, limiting the user experience. This poses a challenge as the convenience and safety of autonomous vehicles are not fully realized.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0096] In this invention, the server includes means for acquiring specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs not only through text but also through other means, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with users using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and proposing optimal routes and rest stops. This enables autonomous vehicles to suggest personalized, optimal routes and rest stops through real-time information collection and dialogue with users.

[0097] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[0098] "Latest news" refers to the latest data and news about current situations and events.

[0099] "Personal information" is information about an individual, including personal preferences and history.

[0100] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[0101] "Means of conversation" refers to a method of interacting with the user using voice recognition technology or natural language processing technology.

[0102] "Roles" refers to the method of assigning specific functions and responsibilities to each AI.

[0103] "Security" refers to the technologies and measures used to protect information and systems and prevent unauthorized access and data leaks.

[0104] A "platform" refers to the system or environment that forms the foundation of software and hardware.

[0105] "Means of collecting traffic and weather information in real time" refers to a method of instantly obtaining data on current traffic conditions and weather.

[0106] "Speech recognition technology" refers to technology that converts speech into text.

[0107] "Natural language processing technology" refers to artificial intelligence technology for understanding and generating human language.

[0108] "Means for learning about the user's past driving history and preferences" refers to a method of collecting data on the user's past behavior and preferences and learning based on that data.

[0109] "Means for suggesting optimal routes and rest points" refers to methods that suggest the most suitable routes and rest points for users based on collected information.

[0110] The system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with the user using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and suggesting optimal routes and rest stops.

[0111] The server collects real-time traffic and weather information from various databases and internet sources using software (e.g., the requests library) that uses APIs to retrieve data. The collected information is then analyzed within the server and provided to the user.

[0112] The device converts the user's speech into text using speech recognition technology (e.g., the speech_recognition library). The converted text is then analyzed using natural language processing technology (e.g., the transformers library) to understand the user's intent, allowing for dialogue with the user.

[0113] Users interact with the system through voice. For example, by asking, "Where is the next rest stop?", the system will suggest the best rest stop based on real-time traffic information and the user's past driving history. The suggested information is conveyed to the user using voice synthesis technology (e.g., the pyttsx3 library).

[0114] As a concrete example, if a user asks, "Where is the next rest stop?" the system responds as follows:

[0115] User: Where is the next rest stop?

[0116] System: Based on current traffic conditions and past driving history, the system will suggest the next rest stop, "XX Service Area." This area has a restaurant and restrooms, making it an ideal place to take a break.

[0117] In this way, intelligent navigation systems in autonomous vehicles can provide users with a convenient and safe driving experience.

[0118] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0119] Step 1:

[0120] The server collects traffic and weather information in real time from various databases and information sources on the Internet. Specifically, it uses the requests library to obtain data using an API. An API endpoint is specified as input, and traffic and weather information in JSON format is obtained as output. This allows the latest traffic and weather information to be aggregated on the server.

[0121] Step 2:

[0122] The server analyzes the collected traffic and weather information. Specifically, it analyzes the acquired JSON data and extracts the necessary information (e.g., locations of traffic jams, weather changes). It receives JSON data as input and obtains the analyzed information as output. This prepares useful information to provide to users.

[0123] Step 3:

[0124] The device uses voice recognition technology to convert the user's voice into text. Specifically, it uses the speech_recognition library to convert the voice data input from the microphone into text data. It receives voice data as input and obtains text data as output. This allows the user's speech content to be obtained in text format.

[0125] Step 4:

[0126] The device uses natural language processing technology to analyze the user's text data and understand their intent. Specifically, it uses the transformers library to analyze the text data using a generative AI model. It receives text data as input and obtains data indicating the user's intent as output. This allows the device to understand the user's questions and requests.

[0127] Step 5:

[0128] The server generates an appropriate response based on the user's intent. Specifically, it suggests optimal routes and rest stops based on analyzed traffic and weather information, the user's past driving history, and preferences. It receives the user's intent data and analyzed information as input, and obtains response data as output. This allows it to generate appropriate suggestions for the user.

[0129] Step 6:

[0130] The device uses speech synthesis technology to generate response data and communicates it to the user. Specifically, it uses the pyttsx3 library to convert text-format response data into speech data. It receives response data as input and obtains speech data as output. This provides a speech response to the user.

[0131] Step 7:

[0132] The user receives the voice response provided by the system and decides on the next course of action. Specifically, the user continues driving according to the route and rest points suggested by the system. The system receives the voice response as input and decides on the next course of action as output. This allows the user to reach their destination safely and efficiently.

[0133] Example 2

[0134] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0135] Conventional information provision systems have difficulty efficiently incorporating specialized knowledge, the latest information, and personal information, making it difficult to provide consistent information to users. Furthermore, when each AI operates independently, centralized information management and security assurance become issues. Furthermore, there was a need for a system that could integrate multiple processes, such as news information collection and summarization, personal information input and storage, and recommended information generation.

[0136] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0137] In this invention, the server includes a means for taking in specialized knowledge, the latest information, and personal information, a means for conversing with the humanoid AI in addition to text, a means for assigning roles to each AI, a means for providing information together with security and platform information, a means for collecting and summarizing news information, a means for inputting and saving personal information, a means for generating recommended information, a means for centrally managing information, and a means for providing information. This makes it possible to provide consistent information to users, and achieves centralized information management and security while each AI operates independently.

[0138] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0139] "Latest news" is the latest data and news about current situations and events.

[0140] "Personal information" refers to information about an individual user, such as data about that user, their interests, and past behavioral history.

[0141] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[0142] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0143] "Security" refers to the technologies and methods used to protect information and systems from unauthorized access and data leaks.

[0144] A "platform" is a system or environment that serves as the foundation for providing software and services.

[0145] "Means for collecting and summarizing news information" refers to a method for obtaining news data and concisely summarizing its contents.

[0146] "Means for entering and storing personal information" refers to the method by which a user enters their own information and stores it in a database.

[0147] The "means for generating recommendation information" is a method for generating appropriate suggestions and advice based on the user's personal information.

[0148] A "means for centrally managing information" is a method for integrating and managing multiple pieces of information.

[0149] "Means of providing information" refers to the method of providing the necessary information to users in an appropriate format.

[0150] This invention is a system that assigns specific roles to multiple AIs, centrally manages the information generated by each AI, and provides it to users in an appropriate format. A specific embodiment of this system is described below.

[0151] Hardware and software used

[0152] Hardware: Servers, terminals (user devices)

[0153] Software: Generative AI models (e.g., GPT-4®), database management systems (e.g., MySQL®), security software (e.g., SSL / TLS)

[0154] News information gathering and summarization

[0155] The server uses a news API (e.g., NewsAPI) to collect news information. The server periodically sends requests to the news API to obtain the latest news data. The obtained news data is input into a generative AI model, and a prompt is sent to generate a summary. An example of a specific prompt is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The server stores the summarized news information in a database.

[0156] Entering and saving personal information

[0157] The user uses the device to input their personal information (e.g., interests, past browsing history), and the device sends the input personal information to the server, which stores the received personal information in a database.

[0158] Generate recommendations

[0159] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. An example of a specific prompt is, "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The server stores the generated recommendations in a database.

[0160] Centralized information management

[0161] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The server uses security software (e.g., SSL / TLS) to ensure the safety of data. Specifically, access to the database is encrypted with SSL / TLS to prevent unauthorized access.

[0162] Providing information

[0163] A user uses a device to request news information or recommended information. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The device then displays the received information to the user. Specifically, it displays summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[0164] In this way, a system will be created in which each AI has a specific role, and the information generated will be centrally managed and provided to users.

[0165] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0166] Step 1: Gather and summarize news information

[0167] The server sends a request to the news API to obtain the latest news data. The input is news data from the news API, and the output is the obtained news data. Specifically, the server sends a request to the news API to "obtain the latest news." The obtained news data is input into the generative AI model, and a prompt sentence is sent to generate a summary. An example of a prompt sentence is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The output is summarized news information. The server stores the summarized news information in a database.

[0168] Step 2: Enter and save your personal information

[0169] The user uses the device to input their own personal information (e.g., interests, past browsing history). The input is the personal information the user enters into the device, and the output is the personal information sent to the server. The device sends the entered personal information to the server. The server stores the received personal information in a database. Specifically, the user enters "Interests: Technology, Past Browsing History: AI-related news" into the device, and the device sends this to the server.

[0170] Step 3: Generate recommendations

[0171] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. The input is the personal information stored in the database, and the output is the prompt sent to the generative AI model. An example of a prompt is "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The output is the generated recommendations. The server stores the generated recommendations in a database.

[0172] Step 4: Centralize your information

[0173] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The input is news information and recommendation information, and the output is centrally managed data. The server uses security software (e.g., SSL / TLS) to ensure the security of the data. Specifically, it encrypts access to the database with SSL / TLS to prevent unauthorized access.

[0174] Step 5: Provide information

[0175] A user uses a device to request news information or recommended information. The input is the user's request, and the output is the request sent to the server. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The output is the information provided to the user. The device displays the received information to the user. A specific operation is to display summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[0176] (Application example 2)

[0177] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0178] Conventional information provision systems have difficulty in providing consistent information to users, and are particularly inadequate in providing personalized content. Furthermore, there is a lack of a way to centrally manage information generated by multiple AIs and provide it to users in an appropriate format, which results in a lack of an improved user experience.

[0179] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on the user's interests, and means for analyzing news data using natural language processing technology and classifying it by category. This enables consistent information provision to users, realizing the provision of personalized content.

[0180] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0181] "Latest news" is the latest data and news about current situations and events.

[0182] "Personal Information" means information about an individual user's personal data, interests, and preferences.

[0183] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0184] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0185] "Security information" means information relating to the safety of systems and data.

[0186] "Platform information" refers to information relating to the operation and management of the entire system.

[0187] "Centralized management" is a method of integrating and managing multiple pieces of information in one system or location.

[0188] "Recommended content" is information or media selected based on a user's interests and preferences.

[0189] "Natural language processing technology" is a technology that uses computers to understand and analyze human language.

[0190] "News Data" means information obtained from news organizations or news sources.

[0191] A "categorization method" is a method for separating information into specific categories.

[0192] A system for carrying out this invention is configured as follows: The server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on user interests, and means for analyzing news data using natural language processing technology and classifying it into categories.

[0193] Hardware and software used

[0194] Hardware: Smartphones, smart glasses

[0195] Software: Python, TENSORFLOW (registered trademark), Flask, SQLite, REST API

[0196] Data processing and calculation flow

[0197] 1. Data Collection:

[0198] The server retrieves the latest news data from a news API (e.g., NewsAPI) and stores user personal information (interests, browsing history, etc.) in a SQLite database.

[0199] 2. Data Processing:

[0200] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. Based on the user's personal information, a model is trained using TensorFlow (registered trademark) to generate recommended content.

[0201] 3. Information generation:

[0202] The server uses News AI to generate the latest news information, Recommendation AI to generate recommended content based on the user's interests, and Security AI to generate security information.

[0203] 4. Information Integration:

[0204] The server uses Flask to build a REST API that centrally manages the information generated by each AI, consolidates the information from each AI, and provides it to users in an appropriate format.

[0205] 5. User Interface:

[0206] The device (smartphone or smart glasses) displays the integrated information.

[0207] Examples and prompts

[0208] Specific examples

[0209] News AI: "Today's latest news is about new discoveries in the field of technology."

[0210] Recommendation AI: "We recommend the following content based on your recent topics of interest."

[0211] Security AI: "The latest security information is to beware of phishing scams."

[0212] Prompt Sentence Examples

[0213] Prompt for News AI: "Get the latest technology news."

[0214] Prompt for recommendation AI: "Recommend content based on the user's interests."

[0215] Prompt security AI: "Please provide us with the latest security information."

[0216] In this way, the server will enable each AI to have a specific role and provide personalized information to users, thereby realizing a richer content distribution service.

[0217] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0218] Step 1:

[0219] The server retrieves the latest news data from the news API. The input is the news API endpoint, and the output is the retrieved news data. Specifically, the server sends an HTTP request and receives news data in JSON format.

[0220] Step 2:

[0221] The server stores user personal information (interests, browsing history, etc.) in an SQLite database. The input is the user's personal information, and the output is the information stored in the database. Specifically, the server receives user information and inserts it into the database using an SQL query.

[0222] Step 3:

[0223] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. The input is news data, and the output is news data classified by category. Specifically, the server analyzes the news data using an NLP library (e.g., spaCy) and assigns category tags.

[0224] Step 4:

[0225] The server uses TensorFlow® to train a model for generating recommended content based on the user's personal information. The input is the user's personal information and news data, and the output is the trained recommendation model. Specifically, the server preprocesses the data and trains the model using TensorFlow®.

[0226] Step 5:

[0227] The server uses news AI to generate the latest news information. The input is news data categorized by category, and the output is the generated news information. Specifically, the server analyzes the news data and selects news appropriate for the user.

[0228] Step 6:

[0229] The server uses recommendation AI to generate recommended content based on the user's interests. The input is a trained recommendation model and the user's personal information, and the output is recommended content. Specifically, the server uses the model to recommend the most suitable content to the user.

[0230] Step 7:

[0231] The server generates security information using security AI. The input is security-related data, and the output is the generated security information. Specifically, the server analyzes the security data and provides warnings and advice to the user.

[0232] Step 8:

[0233] The server uses Flask to build a REST API that centrally manages the information generated by each AI. The input is the information generated by each AI, and the output is the integrated information. Specifically, the server uses the Flask framework to set up an API endpoint and integrate the information.

[0234] Step 9:

[0235] The device (smartphone or smart glasses) displays the integrated information. The input is the integrated information from the server, and the output is the information displayed on the user interface. Specifically, the device retrieves information from the API and displays it visually to the user.

[0236] Example 3

[0237] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0238] In conventional information provision systems, it was difficult for each AI to operate independently, and it was also difficult to provide the information generated by each AI consistently. Furthermore, the process from when a user sends a request to when they receive the information is complicated, making it difficult to provide information efficiently.

[0239] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0240] In this invention, the server includes means for a user to send a request from a terminal, means for the terminal to transfer the request to the server, means for the server to analyze the request and select an appropriate generative AI model, means for the server to input a prompt sentence into the generative AI model selected by the server and generate information, means for the server to share the generated information with other generative AI models, means for the server to return the final information to the terminal, and means for the terminal to display the information to the user. This makes it possible to provide consistent information as a whole even though each generative AI model operates independently.

[0241] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[0242] "Latest information" refers to the most current data, news, and events.

[0243] "Personal information" refers to data or information relating to an individual, including personal characteristics, preferences, and history.

[0244] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[0245] A "role" is a responsibility or function assigned to perform a specific task or function.

[0246] "Security" refers to the means and measures taken to protect information and systems from unauthorized access and attacks.

[0247] A "platform" is the environment or system on which software and hardware operate.

[0248] A "request" is a request or inquiry made by a user to a system.

[0249] A "server" is a computer that provides services to other computers on a network.

[0250] A "terminal" is a device or computer that is directly operated by a user.

[0251] A "generative AI model" is an artificial intelligence model that is trained to generate information for a specific task.

[0252] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[0253] "Generating information" means creating new data or answers based on input data and prompts.

[0254] "Sharing information" means exchanging generated data and knowledge with other AI models and systems.

[0255] "Consistent information provision" means that data obtained from multiple information sources and AI models is consistent and provided to users in a unified form.

[0256] This invention is a system in which each generative AI model operates independently and generates and provides information according to its role. Furthermore, each generative AI model can share information with other generative AI models, providing consistent information as a whole.

[0257] Hardware and software used

[0258] The server trains and deploys each generative AI model using machine learning frameworks such as TensorFlow (registered trademark) and PyTorch. The server also communicates with terminals using HTTP requests and WebSockets. Terminals are devices such as smartphones and PCs that provide an interface for users to input requests.

[0259] Data processing and calculation

[0260] The server receives a request from the user, analyzes it, and selects an appropriate generative AI model. The selected generative AI model inputs a prompt sentence to generate information. The generated information is shared with other generative AI models to provide consistent information overall. The final information is sent back from the server to the device and displayed to the user.

[0261] Specific examples

[0262] As a concrete example, consider a news article generation system. A news article generation AI, a sports article generation AI, and a weather forecast generation AI each operate independently to generate specific information. When a user requests "Tell me today's news," the device sends this request to a server. The server then calls the news article generation AI to generate the latest news articles. The generated news articles are shared with other generative AI models (for example, a sports article generation AI and a weather forecast generation AI), providing consistent information overall.

[0263] Prompt Sentence Examples

[0264] Here is an example prompt:

[0265] "Tell me the news today"

[0266] "Tell me the latest sports news"

[0267] "What's the weather forecast for today?"

[0268] In this way, a system is constructed that provides consistent information as a whole, even though each generative AI model operates independently. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0269] Step 1:

[0270] The user sends a request from the terminal.

[0271] Users open the application on their smartphone or computer, type "Tell me today's news" into the search bar, and press the send button.

[0272] Input: User request (e.g. "Tell me today's news")

[0273] Output: Request data sent to the terminal

[0274] Step 2:

[0275] The terminal forwards the request to the server.

[0276] The device receives the user's input and sends an HTTP POST request to the server, which includes the user's input.

[0277] Input: User request data

[0278] Output: HTTP request sent to the server

[0279] Step 3:

[0280] The server analyzes the request and selects the appropriate generative AI model.

[0281] The server analyzes the received request and detects the keyword "news." Based on this, it selects a news article generation AI.

[0282] Input: HTTP request data

[0283] Output: Selected generative AI model (news article generation AI)

[0284] Step 4:

[0285] The server inputs a prompt sentence into the selected generative AI model to generate information.

[0286] The server inputs the prompt "Tell me today's news" into the news article generation AI, and the AI ​​model generates the latest news articles. The generated news articles are returned to the server in text format.

[0287] Input: Prompt sentence (e.g. "Tell me today's news")

[0288] Output: Generated news article

[0289] Step 5:

[0290] The server shares the generated information with other generative AI models.

[0291] The server shares the generated news articles with the sports article generation AI and weather forecast generation AI, allowing other AIs to generate information based on the latest news information.

[0292] Input: Generated news article

[0293] Output: Information shared with other generative AI models

[0294] Step 6:

[0295] The server returns the final information to the terminal.

[0296] The server then compiles the news article, sports news, and weather forecast into a single response and sends it back to the device. This response is sent in JSON format.

[0297] Input: Shared information (news articles, sports news, weather forecasts)

[0298] Output: JSON response sent to the terminal

[0299] Step 7:

[0300] The terminal displays the information to the user.

[0301] The device parses the JSON response received from the server and displays news articles, sports news, and weather forecasts in their respective sections, allowing users to view this information on their device screen.

[0302] Input: JSON response from the server

[0303] Output: Information displayed to the user (news articles, sports news, weather forecasts)

[0304] (Application example 3)

[0305] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0306] In self-driving vehicles, multiple AIs are required to operate independently and generate and provide information according to their respective roles. However, in order for these AIs to provide consistent information, a means is needed to efficiently share and integrate the information generated by each AI. Furthermore, it is technically difficult to process and integrate a wide range of information in real time, such as obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation. A system that can solve these issues is needed.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information; means for conversing with humanoid AIs not only through text but also through other means; means for assigning roles to each AI; means for providing information together with security and platform information; means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in an autonomous vehicle; and means for sharing information generated by each AI through a communication protocol to provide consistent information. This enables multiple AIs to operate independently while efficiently sharing information and providing consistent information.

[0308] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0309] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[0310] "Personal information" refers to data and attribute information about individuals.

[0311] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0312] "Means of conversation" refers to technologies and interfaces that allow for natural dialogue with users.

[0313] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0314] "Security information" is information used to ensure the safety of systems and data.

[0315] "Platform information" is information about the platform on which the system operates.

[0316] The "means of provision" refers to the method by which information or services are provided to users.

[0317] An "autonomous vehicle" is a vehicle that drives autonomously without driver intervention.

[0318] "Obstacle detection" is a technology that detects obstacles that exist in the direction of travel of a vehicle.

[0319] "Route optimization" is a technique for calculating the optimal route to a destination.

[0320] "Traffic situation analysis" is a technique for evaluating and analyzing the current traffic situation.

[0321] "Passenger comfort assessment" is a technique for assessing passenger comfort.

[0322] A "communication protocol" is a set of rules and procedures for sending and receiving data.

[0323] "Providing consistent information" refers to integrating data obtained from multiple sources and presenting it in a consistent manner.

[0324] A system for carrying out this invention is configured as follows: The server includes means for taking in specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in autonomous vehicles, and means for sharing information generated by each AI via a communication protocol to provide consistent information.

[0325] Hardware and Software Configuration

[0326] The server uses the following hardware and software:

[0327] Hardware

[0328] Autonomous vehicle sensors (cameras, LIDAR, GPS, etc.)

[0329] In-vehicle computer

[0330] software

[0331] Python programming language

[0332] threading module (concurrency)

[0333] queue module (data queue)

[0334] Data processing and calculation

[0335] The server processes the data and performs calculations as follows:

[0336] 1. Sensor data acquisition: Acquire real-time data from the autonomous vehicle's sensors, such as video data from cameras and LIDAR, and location information from GPS.

[0337] 2. Data processing by AI: Each AI processes data using its own algorithm. For example, obstacle detection AI analyzes sensor data and determines whether there is an obstacle ahead. Route optimization AI calculates the optimal route based on map data. Traffic situation analysis AI analyzes traffic data and evaluates the current traffic situation. Passenger comfort assessment AI evaluates comfort based on passenger data.

[0338] 3. Data sharing via communication protocols: Information generated by each AI is shared via communication protocols, enabling consistent information provision. For example, obstacle detection AI can send information about detected obstacles to route optimization AI, which can then calculate the optimal route to avoid them.

[0339] Specific examples

[0340] For example, imagine an autonomous vehicle driving through a city: it detects obstacles ahead (e.g., pedestrians), recalculates the optimal route, analyzes traffic conditions, and evaluates passenger comfort. All this information is integrated in real time and reflected in the vehicle's operations.

[0341] Prompt Sentence Examples

[0342] "Design a system that detects pedestrians ahead, recalculates the optimal route, analyzes traffic conditions, and assesses passenger comfort while an autonomous vehicle is driving through a city."

[0343] In this way, it is possible to provide a specific embodiment of the invention.

[0344] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0345] Step 1:

[0346] The server receives real-time data from the autonomous vehicle's sensors. As input, it receives video data from cameras and LIDAR, and location information from GPS. It then takes this data and passes it on to the next processing step.

[0347] Step 2:

[0348] The server passes sensor data as input to the obstacle detection AI, which analyzes the video and LIDAR data to determine whether there is an obstacle ahead and generates the location information of the obstacle as output.

[0349] Step 3:

[0350] The server passes map data and obstacle location information to the route optimization AI as input. The route optimization AI calculates the optimal route based on the map data and generates a new route to avoid obstacles. The output is the optimal route information.

[0351] Step 4:

[0352] The server passes traffic data as input to the traffic situation analysis AI. The traffic situation analysis AI analyzes the current traffic data and evaluates traffic flow and congestion information. As output, it generates a traffic situation evaluation result.

[0353] Step 5:

[0354] The server passes passenger data as input to the passenger comfort evaluation AI, which evaluates comfort based on the passenger data and generates passenger comfort evaluation results as output.

[0355] Step 6:

[0356] The server shares the information generated by each AI through a communication protocol. As input, it receives information on the location of obstacles, optimal route information, traffic situation evaluation results, and passenger comfort evaluation results. This information is integrated to provide consistent information.

[0357] Step 7:

[0358] The server passes the integrated information to the autonomous vehicle's control system, which receives the integrated information as input and reflects it in the vehicle's operation. Specifically, it changes the route to avoid obstacles, adjusts the speed according to traffic conditions, and operates in a way that takes passenger comfort into consideration.

[0359] In this way, the server uses multiple AIs to optimize the operation of autonomous vehicles and provide consistent information.

[0360] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0361] "Example 1"

[0362] The invention, which combines the emotion engine, is an AI system that recognizes the user's emotions and engages in dialogue accordingly. Specifically, it infers emotions from the user's tone of voice, facial expressions, and choice of words, and generates a response appropriate to those emotions. For example, it provides words of comfort when the user is sad, and words of sympathy when the user is happy. This makes the dialogue between the AI ​​and the user more natural and human-like.

[0363] "Example 2"

[0364] The invention combining emotion engines is a system in which each AI operates independently to generate and provide information according to its respective role. Specifically, some AIs provide the latest news, others provide recommended information based on the user's personal information, and still others use the emotion engine to recognize the user's emotions and engage in dialogue accordingly. This allows users to obtain information tailored to their emotions.

[0365] "Example 3"

[0366] The form of the invention that combines emotion engines is a system in which each AI shares information with other AIs and provides consistent information as a whole. Specifically, some AIs share information generated or recognized user emotions with other AIs, and provide information and engage in dialogue based on that. For example, if one AI recognizes the user's emotion of joy, it shares that information with other AIs, and the other AIs also provide information and engage in dialogue that is tailored to the user's joy. This enables consistent dialogue as a whole.

[0367] The processing flow of each embodiment will be described below.

[0368] "Example 1"

[0369] Step 1: Initiate a dialogue with the user.

[0370] Step 2: Infer the user's emotions from their tone of voice, facial expressions, and choice of words.

[0371] Step 3: Generate a response appropriate to the estimated emotion.

[0372] Step 4: Provide the generated response to the user.

[0373] "Example 2"

[0374] Step 1: Each AI generates and provides information independently.

[0375] Step 2: Some AI will provide news updates.

[0376] Step 3: Another AI provides recommendations based on the user's personal information.

[0377] Step 4: Another AI uses an emotion engine to recognize the user's emotions and interact accordingly.

[0378] "Example 3"

[0379] Step 1: Each AI shares information with other AIs.

[0380] Step 2: Some AIs share the information they generate and the user's emotions they recognize with other AIs. Step 3: Other AIs provide information and engage in dialogue based on the shared information.

[0381] Example 1

[0382] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] Conventional AI systems have difficulty effectively incorporating specialized knowledge, up-to-date information, and personal information, resulting in a lack of natural, human-like interaction with users. Furthermore, they are unable to recognize user emotions and generate appropriate responses, limiting the user experience.

[0384] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for acquiring specialized knowledge, the latest information, and personal information, a means for converting a user's voice into text using speech recognition technology, a means for analyzing the text using natural language processing technology, a means for estimating the user's emotions from the user's tone of voice and facial expression, a means for generating an appropriate response based on the estimated emotions, a means for assigning roles to each AI, and a means for providing information together with security and platform information. This makes the dialogue with the user more natural and human-like, and makes it possible to provide an appropriate response according to the user's emotions.

[0385] "Expertise" refers to information that involves advanced understanding or skill in a particular field.

[0386] "Latest information" refers to the most recent data and knowledge available at the present time.

[0387] "Personal Information" means data or information relating to an individual.

[0388] "Speech recognition technology" refers to technology that converts speech into text.

[0389] "Natural language processing technology" refers to the technology of analyzing text data and understanding its meaning.

[0390] "Means for estimating emotions" refers to a method for inferring emotions from the user's tone of voice and facial expressions.

[0391] The "means for generating a response" refers to a method for creating an appropriate reply based on the analyzed information and estimated emotions.

[0392] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0393] "Security and platform information" refers to data related to the security and infrastructure of a system.

[0394] This invention is a system including means for incorporating specialized knowledge, up-to-date information, and personal information, means for converting a user's voice into text using speech recognition technology, means for analyzing the text using natural language processing technology, means for inferring the user's emotions from their tone of voice and facial expressions, and means for generating an appropriate response based on the inferred emotions.

[0395] The server collects specialized knowledge, the latest information, and personal information from various databases and online sources. This information is used as training data for AI. Specifically, the server obtains information using web scraping technology or APIs and stores it in a database. For example, Python's BeautifulSoup or Scrapy can be used to extract information from web pages and store it in a database.

[0396] The device uses speech recognition technology to convert the user's speech into text. Specifically, the device uses the Google® Cloud Speech-to-Text API or IBM Watson® natural language processing service to convert the user's speech into text, analyze the text, and generate an appropriate response based on the analysis results.

[0397] The device uses an emotion engine to infer emotions from the user's tone of voice and facial expressions. Specifically, it analyzes the user's facial expressions using Microsoft® Azure® Emotion API and OpenCV to infer emotions. Based on the inferred emotion, the AI ​​generates an appropriate response. For example, if the user is sad, it will provide comforting words such as, "Are you okay? Is there anything I can help you with?"

[0398] When a user asks a question or makes a request to the system, the device recognizes the speech and converts it into text. It then uses natural language processing technology to analyze the text and understand the user's intent. It then uses an emotion engine to estimate the user's emotions and generate an appropriate response.

[0399] For example, if a user asks, "What's the weather like today?", the device converts the voice to text and analyzes the text to provide weather information. If a user says, "I'm tired today," in a sad voice, the device analyzes the tone of the voice and recognizes the sad emotion. The device then provides comforting words such as, "You've had a hard time today. Please take a rest."

[0400] An example prompt is, "Design an AI system that offers words of comfort when the user is sad. Explain how you can recognize emotions from the user's tone of voice and facial expression and generate a response appropriate to those emotions."

[0401] In this way, the system can make interactions with the user more natural and human-like, and provide appropriate responses according to the user's emotions.

[0402] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0403] Step 1: Gather information

[0404] The server collects specialized knowledge, up-to-date information, and personal information from various databases and internet sources. It receives website URLs and API endpoints as input and retrieves information using web scraping techniques and APIs. Specifically, it uses Python's BeautifulSoup and Scrapy to extract information from web pages and store it in a database. The collected information is then stored in a database as output.

[0405] Step 2: Voice Recognition

[0406] The device receives the user's voice as input and converts it into text using speech recognition technology. Specifically, it converts the voice data into text data using the Google® Cloud Speech-to-Text API. The converted text is obtained as output.

[0407] Step 3: Natural Language Processing

[0408] The device receives text obtained by speech recognition as input and analyzes the text using natural language processing technology. Specifically, it uses the natural language processing service of IBM Watson (registered trademark) to analyze the meaning of the text and understand the user's intent. The analysis results are obtained as output.

[0409] Step 4: Emotion Recognition

[0410] The device receives the user's tone of voice and facial expression as input and uses an emotion engine to estimate the user's emotion. Specifically, it analyzes the user's facial expression using Microsoft® Azure® Emotion API and OpenCV, and estimates the emotion. The estimated emotion is obtained as output.

[0411] Step 5: Response Generation

[0412] The device receives the analysis results of natural language processing and the estimated results of emotion recognition as input and generates an appropriate response. Specifically, it uses a generative AI model to create a response based on the user's intention and emotion. The generated response is obtained as output.

[0413] Step 6: Providing a response

[0414] The terminal provides the generated response to the user by converting the text response into speech using speech synthesis technology and playing it for the user. The user receives the response as an output.

[0415] In this way, the system can make interactions with the user more natural and human-like, and provide appropriate responses according to the user's emotions.

[0416] (Application example 1)

[0417] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0418] Conventional security systems respond uniformly without considering the user's emotional state, making it impossible to provide appropriate security alerts according to the user's psychological state. Furthermore, even conversational AI systems using voice recognition and natural language processing technologies have difficulty recognizing the user's emotions and engaging in dialogue accordingly. This makes dialogue with users unnatural, making it impossible to properly manage security risks.

[0419] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions and providing appropriate security alerts, means for enabling dialogue with the user using voice recognition technology and natural language processing technology, and means for recognizing the user's facial emotions using a camera. This makes it possible to provide appropriate security alerts according to the user's emotional state and realize more natural, human-like dialogue.

[0420] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0421] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[0422] "Personal Information" means certain data or information about an individual.

[0423] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0424] "Means for conversation" refers to a method for interacting with users using voice recognition technology or natural language processing technology.

[0425] "Roles" are ways to assign specific functions and responsibilities to each AI.

[0426] "Security and platform information" refers to data related to the security and infrastructure of a system.

[0427] The "means of provision" is the method by which information or services are provided to users.

[0428] "Means for recognizing emotions and providing appropriate security alerts" refers to a method for detecting a user's emotional state and providing appropriate warnings or notifications.

[0429] "Speech recognition technology" is a technology that converts speech into text.

[0430] "Natural language processing technology" is a technology for understanding and generating human language.

[0431] "Means for recognizing the user's facial emotions using a camera" refers to a method for analyzing the user's facial expressions using a camera and estimating their emotions.

[0432] A system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing user emotions and providing appropriate security alerts, means for enabling dialogue with users using voice recognition technology and natural language processing technology, and means for recognizing user facial emotions using a camera.

[0433] System Program

[0434] This system operates using the following hardware and software:

[0435] Hardware: Your smartphone's microphone and camera

[0436] Software: Python, speech_recognition library, transformers library, OpenCV

[0437] Processing flow

[0438] The server first collects the user's voice using the smartphone's microphone and converts the voice to text using the speech_recognition library. It then inputs the converted text into an emotion recognition model in the transformers library to estimate the user's emotion. It then captures the user's face using the smartphone's camera and detects it using OpenCV. It then inputs the facial image into the emotion recognition model to estimate the facial emotion. Finally, if the user's emotion is "sadness" or "anger," it raises a security alert.

[0439] Specific examples

[0440] For example, if a user says, "Recently, things haven't been going well at work and I'm feeling stressed," the speech recognition system will recognize this text, and the emotion recognition model will determine it as "anger" or "sadness." As a result, the server will notify the user with a "Security Alert: The user is in an unstable state."

[0441] Prompt Sentence Examples

[0442] If a user says, "Work hasn't been going well lately and I'm feeling stressed," what emotion would the emotion recognition model infer?

[0443] In this way, an emotion-aware security assistant can monitor user emotions in real time and provide security alerts when necessary.

[0444] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0445] Step 1:

[0446] The server collects the user's voice using the smartphone's microphone. The input is the user's voice, and the output is voice data. This voice data is used for subsequent voice recognition processing.

[0447] Step 2:

[0448] The server converts the speech data to text using the speech_recognition library. The input is speech data and the output is text data. This text data is input to the emotion recognition model.

[0449] Step 3:

[0450] The server inputs text data into the emotion recognition model of the Transformers library to estimate the user's emotion. The input is text data, and the output is an emotion label (e.g., joy, sadness, anger, etc.). This emotion label is used to determine security alerts.

[0451] Step 4:

[0452] The server captures the user's face using the smartphone camera. The input is the camera image, and the output is facial image data. This facial image data is used for facial emotion recognition processing.

[0453] Step 5:

[0454] The server uses OpenCV to detect faces from the facial image data and inputs the facial images into an emotion recognition model. The input is the facial image data, and the output is the facial emotion label. This facial emotion label is also used to determine security alerts.

[0455] Step 6:

[0456] The server integrates emotion labels from the voice and the face to determine the user's overall emotional state. The input is the emotion labels from the voice and the face, and the output is the overall emotion label. Based on this overall emotion label, it decides whether to issue a security alert.

[0457] Step 7:

[0458] The server generates a security alert if the user's overall emotion label is "sad" or "anger." The input is the overall emotion label, and the output is a security alert notification. This notification provides the user with appropriate warnings and advice.

[0459] Example 2

[0460] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0461] Conventional information provision systems have difficulty in providing users with the latest information and personal information they need. Furthermore, they are unable to provide information that reflects the user's emotions, which limits the user experience. Furthermore, because each AI does not operate independently, there is a lack of consistency in the information provided.

[0462] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0463] In this invention, the server includes a means for importing specialized knowledge, the latest information, and personal information, a means for conversing with the humanoid AI not only through text, a means for assigning roles to each AI, a means for providing information together with security and platform information, a means for receiving user requests, a means for collecting news information, a means for generating recommended information based on personal information, a means for recognizing user emotions, and a means for integrating information from each AI and providing it to the user. This allows users to appropriately receive the latest information and personal information, and further enables information to be provided according to emotions. Furthermore, each AI operates independently, maintaining consistency of information.

[0464] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0465] "Latest news" is the latest data and news about current situations and events.

[0466] "Personal information" refers to information about an individual user, such as data, history, and interests.

[0467] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[0468] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0469] "Security" refers to the techniques and measures used to protect information and systems from unauthorized access and attacks.

[0470] A "platform" is the underlying environment on which software and hardware operate.

[0471] "Means for receiving requests" refers to the method for receiving requests or inquiries from users.

[0472] "Methods of gathering news information" are ways of obtaining the latest news from the internet and other sources.

[0473] The "means for generating recommended information" is a method for creating appropriate suggestions and advice based on the user's personal information.

[0474] "Means for recognizing emotions" refers to a method for analyzing and determining the user's emotional state.

[0475] "Means of integrating information" refers to a method of consolidating data obtained from multiple AIs and providing it as one.

[0476] This invention is a system in which a user accesses the system through a terminal and a server provides information using various AIs. A specific embodiment of this system will be described below.

[0477] Hardware and software used

[0478] The server uses a high-performance server (e.g., a high-performance server). As for software, a platform (e.g., a machine learning framework) is used to manage the operation of each AI. This allows the news AI, user AI, and emotion engine AI to operate efficiently.

[0479] Program processing

[0480] The server receives requests from users and issues instructions to each AI. The news AI collects the latest news information from online news sites and RSS feeds. The user AI generates recommended information based on the user's personal information (past browsing history, interests, etc.). The emotion engine AI analyzes the user's input, voice, facial expressions, etc. to determine the user's emotional state.

[0481] Specific examples

[0482] When a user accesses the system through a terminal, the server performs the following process.

[0483] 1. Providing news information:

[0484] News AI collects the latest news information and provides it to users.

[0485] Example: "The headline news today is that economic growth exceeded expectations."

[0486] 2. Providing Recommendations:

[0487] User AI recommends relevant information based on the user's past browsing history and personal information.

[0488] For example: "Here are some tech-related articles that have recently interested you."

[0489] 3. Emotionally-sensitive dialogue:

[0490] The emotion engine AI recognizes the user's emotions and interacts accordingly.

[0491] For example: "You seem a little tired today. I'd like some relaxing music."

[0492] Prompt Sentence Examples

[0493] Below is an example of a prompt sentence to input to the generative AI model.

[0494] "Please tell me the latest news."

[0495] "What movie would you recommend for me?"

[0496] "I'm feeling a little down today. Can you give me some cheer-up advice?"

[0497] This system allows users to obtain information tailored to their emotions and interests.

[0498] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0499] Step 1:

[0500] A user accesses the system through a terminal.

[0501] Specifically, the user opens the app on their smartphone and enters their username and password on the login screen.

[0502] Input: Username, Password

[0503] Output: Login success message, user ID

[0504] Step 2:

[0505] The server receives the user request.

[0506] As a specific operation, the user sends a request such as "Please tell me the latest news."

[0507] Input: User request (e.g., What's the latest news?)

[0508] Output: Analysis result of request content

[0509] Step 3:

[0510] The server has the news AI collect the latest news information.

[0511] Specifically, the server instructs the news AI to "collect the latest news." The news AI then collects the latest news articles from major news sites and sends them back to the server.

[0512] Input: news gathering instructions

[0513] Output: Latest news information

[0514] Step 4:

[0515] The server allows the user AI to generate recommendations based on the user's personal information.

[0516] Specifically, the server instructs the user AI to "generate recommendations appropriate for this user." The user AI analyzes the user's past browsing history and generates recommendations.

[0517] Input: User's personal information, instructions for generating recommended information

[0518] Output: Recommendations

[0519] Step 5:

[0520] The server makes the emotion engine AI recognize the user's emotions.

[0521] Specifically, the server instructs the emotion engine AI to "recognize the user's emotions." The emotion engine AI then determines the user's emotions from their tone of voice and input content.

[0522] Input: User voice tone, input content, emotion recognition instructions

[0523] Output: Emotion recognition result

[0524] Step 6:

[0525] The server consolidates the information from each AI and provides it to the user.

[0526] Specifically, the server integrates the latest news from the news AI, recommended information from the user AI, and emotion recognition results from the emotion engine AI, and provides them to the user.

[0527] Input: Latest news information, recommendation information, emotion recognition results

[0528] Output: Synthesized information (e.g., Today's top news story is that economic growth exceeded expectations. A movie recommendation for you is "Inception." You seem a little tired today. I recommend some relaxing music.)

[0529] (Application example 2)

[0530] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0531] Conventional information provision systems do not provide information according to the user's emotions, resulting in a limited user experience. Furthermore, when multiple AIs operate independently, it is difficult to centrally manage information, making it difficult to provide consistent information to users. Furthermore, in applications installed on smart devices and robots, the division of roles between AIs is unclear, making it difficult to provide information efficiently.

[0532] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0533] In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with the humanoid AI not only through text but also through other means, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions using an emotion engine and engaging in dialogue accordingly, means for centrally managing information generated by each AI and providing it to the user in an appropriate format, and means including an application installed on a smartphone, smart glasses, head-mounted display, or robot. This makes it possible to provide information according to the user's emotions, clarifies the division of roles between each AI, and realizes centralized information management and consistent information provision.

[0534] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[0535] "Latest news" refers to the latest data and news about current situations and events.

[0536] "Personal information" refers to information about an individual, including the user's preferences and behavioral history.

[0537] "Humanoid AI" is artificial intelligence that mimics the shape of a human and communicates with users through dialogue and actions.

[0538] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0539] "Security information" is information used to ensure the safety of systems and data.

[0540] "Platform information" is information about the platform on which the system operates.

[0541] An "emotion engine" is a technology that recognizes a user's emotions and generates a response accordingly.

[0542] "Centralized management" refers to the integrated management of multiple pieces of information and data in a single system.

[0543] A "smartphone" is a type of mobile phone, a device that can connect to the Internet and use applications.

[0544] "Smart glasses" are a wearable device in the form of glasses that can display information and allow interaction.

[0545] A "head-mounted display" is a display device worn on the head that provides visual information.

[0546] A "robot" is a mechanical device that operates autonomously or remotely to perform a specific task.

[0547] A system for implementing this invention comprises means for taking in specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions using an emotion engine and engaging in dialogue accordingly, means for centrally managing information generated by each AI and providing it to the user in an appropriate form, and means including an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[0548] The server first collects specialized knowledge, the latest information, and personal information and stores them in a database. This is done using a news API and algorithms that analyze user behavior history. Each AI is then assigned a specific role. For example, some AIs provide the latest news, while others provide recommendations based on the user's personal information. Additionally, an AI with an emotion engine recognizes the user's emotions and interacts accordingly.

[0549] The device (smartphone, smart glasses, head-mounted display, robot) receives the information provided by the server and provides it to the user in an appropriate format, either visually or audibly. For example, smart glasses display news headlines in the user's field of vision, and a head-mounted display enables interaction in a virtual space.

[0550] As an example, if the user is wearing smart glasses, the following prompts can be used to provide information:

[0551] Example prompt sentence:

[0552] "Good morning. Today's news headlines are: 1. Economic News: Stocks are rising. 2. Weather Forecast: It's sunny today. Also, based on your recent search history, we recommend the following movie: 'Latest Action Movies.' Additionally, you look a little tired. Would you like some relaxing music?"

[0553] In this way, users can receive information tailored to their emotions and situations in real time, improving the user experience and providing information efficiently.

[0554] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0555] Step 1:

[0556] The server collects specialized knowledge, the latest information, and personal information. As input, it receives news API and user behavior history data, and stores this in a database. As data processing, it analyzes the latest information obtained from the news API and extracts personal information based on the user's behavior history. As output, organized specialized knowledge, the latest information, and personal information are stored in a database.

[0557] Step 2:

[0558] The server assigns a specific role to each AI. It uses specialized knowledge, the latest information, and personal information stored in the database as input. It runs an algorithm to provide specific information to each AI as data calculation. The output is set to a role such as news provision AI, recommendation information provision AI, or emotion recognition AI.

[0559] Step 3:

[0560] The server uses an emotion engine to recognize the user's emotions. As input, it receives the user's voice data and facial expression data. As data processing, the emotion engine analyzes this data and identifies the user's emotional state. As output, the user's emotional state is recognized.

[0561] Step 4:

[0562] The server centrally manages the information generated by each AI and provides it to the user in an appropriate format. It receives the information generated by each AI as input. It integrates the information as a data calculation and converts it into a format that is optimally presented to the user. The integrated information is generated as output.

[0563] Step 5:

[0564] The device (smartphone, smart glasses, head-mounted display, robot) receives the information provided by the server and provides it to the user in an appropriate format, either visually or audibly. As input, it receives the integrated information sent from the server. In concrete terms, the smart glasses display news headlines in the user's field of vision, and the head-mounted display enables interaction in the virtual space. As output, information is provided to the user.

[0565] Step 6:

[0566] The user receives the information provided through the terminal and provides feedback as needed. As input, the information provided by the terminal is received. As a specific operation, the user can check the display on the smart glasses and request additional information by voice command. As output, the user's feedback is sent to the server.

[0567] Step 7:

[0568] The server receives user feedback to further personalize the system. As input, it receives user feedback data. As data processing, it analyzes the feedback and adjusts system settings based on the user's preferences and emotional state. As output, it enhances personalized information provision.

[0569] Example 3

[0570] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0571] Conventional systems had limited means of incorporating specialized knowledge, the latest information, and personal information, and interactions with users were limited to text-based. It was also difficult for each AI to operate independently, making it difficult to maintain consistency of information. Furthermore, the system was unable to recognize the user's emotions and provide information based on them, limiting the user experience.

[0572] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0573] In this invention, the server includes means for taking in specialized knowledge, the latest information, and personal information, means for conversing with the humanoid AI not only through text, means for assigning roles to each AI, means for providing information together with security and platform information, means for receiving user input data and distributing it to each AI, means for each AI to process data using its own algorithm, means for an emotion-recognition AI to recognize the user's emotions and share the information with other AIs, means for other AIs to generate information based on the shared emotional information, and means for integrating information from each AI and providing it to the user. This allows each AI to operate independently, but by sharing information, it is possible to provide consistent information as a whole, realizing dialogue and information provision based on the user's emotions.

[0574] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0575] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[0576] "Personal Information" means data or information about an individual that reflects that individual's characteristics or preferences.

[0577] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[0578] A "role" is the specific function or task each AI is responsible for.

[0579] "Security" refers to measures to protect systems and data from unauthorized access or attack.

[0580] A "platform" is the hardware and software environment that forms the basis for a system to operate.

[0581] "Input data" is information or instructions provided by a user to a system.

[0582] An "algorithm" is a procedure or computational method for solving a particular problem.

[0583] "Emotion recognition AI" is artificial intelligence that has the ability to analyze and recognize the user's emotions.

[0584] "Information sharing" means that multiple AIs exchange data and information with each other.

[0585] "Information generation" means that AI creates new data or responses based on input data or shared information.

[0586] "Information integration" means compiling data obtained from multiple AIs into a single, consistent piece of information.

[0587] This invention is a system in which multiple AIs operate independently, generating and providing information according to their respective roles. Each AI has its own learning data and algorithms, and processes user input data. In addition, a communication protocol is set up so that each AI can share information with other AIs and provide consistent information as a whole.

[0588] Hardware and software used

[0589] server

[0590] The server is responsible for managing the operation of each AI, receiving input data from the user, and distributing it to each AI. The server is a computer equipped with a high-performance processor and large memory, and uses HTTP or WebSocket as a communication protocol.

[0591] Terminal

[0592] A terminal is a device with an interface through which a user provides input data. Terminals include smartphones, tablets, and PCs. The terminal sends the user's input to a server and displays the server's response to the user.

[0593] AI model

[0594] Each AI is a generative AI model with a specific role. For example, emotion recognition AI analyzes the user's emotions, and dialogue generation AI generates appropriate responses. Each AI operates using its own algorithms and training data.

[0595] Data processing and calculation

[0596] Processing input data

[0597] When a user provides input data to the device, the device sends the data to the server, which distributes the input data to each AI, and each AI processes the data using its own algorithm.

[0598] emotion recognition

[0599] Emotion recognition AI analyzes emotions from user input data. For example, if a user inputs "I'm happy today," the emotion recognition AI will recognize the emotion of "joy."

[0600] Information sharing

[0601] Emotional information recognized by emotion-recognition AI is shared with other AIs using communication protocols such as HTTP and WebSocket.

[0602] information generation

[0603] Other AIs generate information based on shared emotional information. For example, a dialogue generation AI generates a response based on the emotion of "joy," such as, "That's great! Did something special happen today?"

[0604] Information Integration

[0605] The server integrates the information from each AI and sends it as a single response to the device, which then displays the integrated information to the user.

[0606] Specific examples

[0607] Consider the case where a user inputs "I'm happy today." In this case, the emotion-recognition AI recognizes the user's emotion as "joy" and shares this information with other AIs. Based on this information, the other AIs can provide information and dialogue tailored to the user's joy.

[0608] Example prompts to input to a generative AI model:

[0609] If a user types "I'm happy today," how will the emotion-aware AI respond and how will it share that information with other AIs?

[0610] In this way, while each AI operates independently, it is possible to provide consistent information as a whole by sharing information, and dialogue and information provision based on the user's emotions can be realized. The flow of the identification process in Example 3 will be described with reference to Figure 21.

[0611] Step 1:

[0612] The user types into the terminal.

[0613] The user inputs text or voice data through the device interface. For example, the user inputs "I'm happy today." The input data is sent to the device in text format.

[0614] Step 2:

[0615] The terminal sends the input data to the server.

[0616] The device converts the user's input data into JSON format and sends it to the server, which receives the data and proceeds to the next processing step.

[0617] Step 3:

[0618] The server distributes input data to each AI.

[0619] The server analyzes the input data it receives and distributes it to the appropriate AI, such as emotion recognition AI, dialogue generation AI, or information provision AI. Each AI prepares to process the data using its own algorithm.

[0620] Step 4:

[0621] Each AI processes data using its own algorithm.

[0622] Emotion recognition AI analyzes user input data and recognizes emotions. For example, it recognizes the emotion of "joy" from the input "I'm happy today." Dialogue generation AI generates appropriate responses based on user input.

[0623] Step 5:

[0624] Emotion-recognition AI recognizes the user's emotions and shares information with other AIs.

[0625] Emotion recognition AI shares the recognized emotional information with other AIs. Communication protocols such as HTTP and WebSocket are used for sharing. For example, emotional information of "joy" is sent to a dialogue generation AI.

[0626] Step 6:

[0627] Other AIs generate information based on shared emotional information.

[0628] The dialogue generation AI generates a response to the user based on the shared emotional information of "joy." For example, it generates a response like, "That's great! Did anything special happen today?"

[0629] Step 7:

[0630] The server consolidates the information from each AI and sends it to the terminal.

[0631] The server integrates the information from each AI and sends it to the terminal as a single response. The integration includes processing to maintain data consistency.

[0632] Step 8:

[0633] The terminal displays the integrated information to the user.

[0634] The terminal displays the integrated information received from the server to the user, allowing the user to receive consistent information. For example, a message such as "That's great! Is there anything special happening today?"

[0635] (Application example 3)

[0636] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0637] In conventional AI systems, it was difficult for each AI to operate independently, and it was also difficult to provide consistent information based on the user's emotions. Furthermore, there was a problem of a lack of smart device applications to support customer interaction and service provision in physical stores.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information; means for conversing with humanoid AIs not only through text but also through other means; means for assigning roles to each AI; means for providing information together with security and platform information; means for performing emotion recognition; means for generating and providing emotion-based information; means for setting a communication protocol for sharing information between AIs; and means for including an application installed on the smart device. This allows each AI to operate independently, enabling consistent information provision based on the user's emotions. Furthermore, smart device applications can be realized to support customer interactions and service provision in brick-and-mortar stores.

[0639] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0640] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[0641] "Personal information" refers to specific data and attribute information about an individual.

[0642] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0643] "Means for conversation" refers to techniques and methods for engaging in natural dialogue with the user.

[0644] "Roles" are ways to assign specific functions and responsibilities to each AI.

[0645] "Security" refers to technologies and measures for protecting information and systems.

[0646] A "platform" is the underlying environment for running applications and services.

[0647] "Emotion recognition" is a technology that detects and identifies a user's emotions.

[0648] "Means of generating and providing information" refers to the method of creating new information based on data and providing it to users.

[0649] A "communication protocol" is a set of rules and procedures for exchanging information between different systems.

[0650] A "smart device" is an electronic device that has Internet connectivity and can run a variety of applications.

[0651] An "application" is software that provides specific functions or services.

[0652] A system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing emotions, means for generating and providing information based on emotions, means for setting communication protocols for sharing information between AIs, and means including an application to be installed on a smart device.

[0653] System program generation

[0654] The server runs the following programs. First, it builds a database to collect specialized knowledge, the latest information, and personal information, and updates this information in real time. Next, it prepares a humanoid AI to interact with the user, and uses natural language processing technology to realize text-based conversations. Furthermore, each AI is assigned a specific role and is designed to operate independently.

[0655] Processing Description

[0656] The server uses image recognition technology to perform emotion recognition. Specifically, it receives a user's facial image as input and identifies the emotion. For this process, it uses, for example, Microsoft® Azure® Face API. Services such as Amazon Personalize are used to generate and provide appropriate information and products based on the recognized emotion. This information is shared among the AIs, allowing for consistent information provision overall.

[0657] Specific examples

[0658] For example, consider a system in which store clerks wear smart glasses to support conversations with customers in a physical store. If a customer is happy, that information is shared with other AIs, and the services and product suggestions provided by the clerk will be tailored to their happiness. In this way, it will be possible to provide consistent service based on the customer's emotions.

[0659] Prompt Sentence Examples

[0660] Below are some example prompts to input to a generative AI model:

[0661] Develop an application that takes a customer's image as input, recognizes their emotion, and suggests appropriate products. Use Microsoft (registered trademark) Azure (registered trademark) Face API for emotion recognition and Amazon Personalize for product suggestions. Use a RESTful API to share the recognized emotion and suggested product information with other AIs.

[0662] This prompt will help the generative AI model suggest specific code and designs.

[0663] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0664] Step 1:

[0665] The server retrieves specialized knowledge, the latest information, and personal information from the database. As input, it receives information from various databases, integrates it, and stores it. As data processing, it normalizes and filters the information, and generates an integrated database as output.

[0666] Step 2:

[0667] The server receives the user's facial image as input and performs emotion recognition. It receives the user's facial image as input and uses image recognition technology to identify emotions. Specifically, it uses Microsoft® Azure® Face API to analyze facial feature points as data calculations. It generates recognized emotion data as output.

[0668] Step 3:

[0669] The server then recommends appropriate products based on the recognized emotion data. It receives emotion data as input and uses services such as Amazon Personalize to make product suggestions. It processes the data by matching the emotion data with the product data and generates a list of suggested products as output.

[0670] Step 4:

[0671] The server shares the generated emotion data and product recommendation data with other AIs. It receives emotion data and product recommendation data as input and sends them to other AIs using a communication protocol. Specifically, it uses a RESTful API to serialize and send the data as data calculations. As output, it generates data that is shared with other AIs.

[0672] Step 5:

[0673] The terminal provides information to the user through an application installed on the smart device. As input, it receives emotion data and product recommendation data from the server and displays them through the user interface. Specific operations include displaying the information on the smart glasses display and notifying the user through the voice assistant. As output, it generates the information provided to the user.

[0674] Step 6:

[0675] The user selects a product based on the information provided through the smart device. As input, the system receives information from the smart device and makes a selection. As a specific operation, the selection is made using the touchpad on the smart glasses or voice commands. As output, the selected product data is generated.

[0676] Step 7:

[0677] The server receives the user's selection data and performs the purchase procedure. As input, it receives the user's selection data and processes the data for the purchase procedure. Specifically, it processes the payment using an electronic payment service. As output, it generates a notification of purchase completion.

[0678] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0679] The data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of the data generation model 58 include generative AIs such as ChatGPT (registered trademark) (Internet search <URL: https: / / OpenAI (registered trademark). com / blog / chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization, etc.

[0680] Other examples of generative AIs include Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0681] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0682] [Second embodiment]

[0683] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0684] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0685] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0686] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0687] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0688] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0689] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0690] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0691] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0692] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0693] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0694] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0695] "Example 1"

[0696] In one embodiment of the present invention, information is collected from various databases and internet sources as a means of incorporating specialized knowledge, the latest information, and personal information, and this information is used as learning data for the AI. Furthermore, as a means of conversing with a humanoid AI, not just through text, speech recognition technology and natural language processing technology are used to enable dialogue with the user.

[0697] "Example 2"

[0698] As a means of assigning roles to each AI, for example, some AIs could provide the latest news, while others could provide recommended information based on the user's personal information, etc. In addition, as a means of providing information together with security and platform information, a system will be built to centrally manage the information generated by each AI and provide it to users in an appropriate format.

[0699] "Example 3"

[0700] Each AI operates independently, and as a means of generating and providing information according to its role, each AI has its own learning data and algorithms, and generates and provides information independently.In addition, each AI shares information with other AIs, and as a means of providing consistent information as a whole, a communication protocol is set up for sharing information between AIs, realizing consistent information provision as a whole.

[0701] The processing flow of each embodiment will be described below.

[0702] "Example 1"

[0703] Step 1: Gather specialized knowledge, current information, and personal information from various databases and online sources.

[0704] Step 2: Use the collected information as learning data for the AI ​​to update its knowledge.

[0705] Step 3: Use voice recognition and natural language processing technologies to enable dialogue with the user.

[0706] "Example 2"

[0707] Step 1: Assign each AI a specific role. For example, some AIs can provide news updates, while others can provide personalized recommendations.

[0708] Step 2: Build a system that centrally manages the information generated by each AI and provides it to users in an appropriate format.

[0709] "Example 3"

[0710] Step 1: Each AI has its own training data and algorithms, and each generates and provides information independently.

[0711] Step 2: Establish a communication protocol for sharing information between AIs to ensure consistent information provision across the board.

[0712] Example 1

[0713] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0714] Conventional information provision systems have had difficulty efficiently collecting specialized knowledge, the latest information, and personal information, and providing consistent information to users. Even in dialogue systems using voice recognition and natural language processing technologies, it was difficult to generate appropriate responses based on user input, limiting the user experience. Furthermore, each AI operates independently and cannot share information, making it difficult to provide consistent information overall.

[0715] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0716] In this invention, the server includes a means for collecting specialized knowledge, the latest information, and personal information, a means for conversing with a humanoid AI not only through text, a means for converting a user's voice into text using speech recognition technology, a means for analyzing collected information using natural language processing technology and using it as training data for a generative AI model, a means for generating appropriate responses based on user input using a generative AI model, a means for converting the generated text responses into speech using speech synthesis technology and playing them back to the user, a means for assigning roles to each AI, and a means for providing information together with security and platform information. This enables the efficient collection of specialized knowledge, the latest information, and personal information and the provision of consistent information to users. Furthermore, even in dialogue systems using speech recognition technology and natural language processing technology, appropriate responses can be generated based on user input, improving the user experience. Furthermore, each AI can share information, providing consistent information overall.

[0717] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0718] "Latest news" is the latest data and news about current situations and events.

[0719] "Personal Information" means data or information relating to an individual.

[0720] "Humanoid AI" is artificial intelligence that mimics human-like dialogue and behavior.

[0721] "Speech recognition technology" is a technology that converts speech into text.

[0722] "Natural language processing technology" is a technology for analyzing and understanding human language.

[0723] A "generative AI model" is an artificial intelligence model that generates new data or responses based on input data.

[0724] "Speech synthesis technology" is a technology that converts text into speech.

[0725] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0726] "Security" refers to the techniques and methods used to protect information and systems.

[0727] A "platform" is a system that serves as the foundation for providing software and services.

[0728] The present invention is a system for efficiently collecting specialized knowledge, up-to-date information, and personal information, and providing consistent information to users. A specific embodiment of this system will be described below.

[0729] Information gathering

[0730] The server collects specialized knowledge, up-to-date information, and personal information from various databases and internet sources. This information is collected using web crawlers and APIs. For example, it uses a news site's API to retrieve the latest news articles. The server periodically checks these sources to see if there is new information.

[0731] Data processing and learning

[0732] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The extracted data is saved as training data for the generative AI model.

[0733] Recognizing user voice

[0734] When a user speaks into the device, the device captures the audio using its built-in microphone. The device converts the captured audio into text using the Google® Cloud Speech-to-Text API. The converted text is then sent to the server.

[0735] Text analysis and response generation

[0736] The server analyzes the received text. For analysis, it uses OpenAI's GPT-3 (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response.

[0737] Audio Output

[0738] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is then played back to the user through the device's speaker. For example, if the generated response is "The latest news is...", that content is conveyed to the user as speech.

[0739] Specific examples

[0740] Example 1: Getting and interacting with news

[0741] When a user says, "Tell me the latest news," the device converts the speech into text and sends it to the server. The server retrieves the latest news from the news site's API and uses a generative AI model to generate a response such as, "The latest news is ____." The device then converts this response into audio and plays it back to the user.

[0742] Example 2: Use of personal information

[0743] When a user says, "What's on my agenda for today?", the device converts the speech into text and sends it to the server. The server retrieves the user's calendar information and uses a generative AI model to generate a response such as "Today's agenda is ____." The device then converts this response into audio and plays it back to the user.

[0744] Prompt Sentence Examples

[0745] "Tell me the latest news"

[0746] "What are my plans for today?"

[0747] "Tell me the weather forecast."

[0748] The system is designed to allow users to obtain information through natural interaction.

[0749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0750] Step 1: Gather information

[0751] The server collects specialized knowledge, the latest information, and personal information from various databases and information sources on the Internet. Specifically, it acquires information using a web crawler or API. For example, it acquires the latest news articles using a news site's API. The input is the news site's API endpoint, and the output is the retrieved news article data.

[0752] Step 2: Data processing and training

[0753] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The input is the collected news article data, and the output is the extracted keywords and context information. This data is saved as training data for the generative AI model.

[0754] Step 3: Recognizing user speech

[0755] When a user speaks into the device, the device captures the voice using the built-in microphone. The device converts the captured voice into text using the Google® Cloud Speech-to-Text API. The input is the user's voice data, and the output is the converted text data. This text data is sent to the server.

[0756] Step 4: Text analysis and response generation

[0757] The server analyzes the received text. For the analysis, it uses GPT-3 (registered trademark) from OpenAI (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response. The input is the text data sent by the user, and the output is the generated response text.

[0758] Step 5: Audio Output

[0759] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is played back to the user through the device's speaker. For example, if the generated response is "The latest news is ____," that content is conveyed to the user as speech. The input is the response text received from the server, and the output is the converted speech data.

[0760] (Application example 1)

[0761] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] Autonomous vehicles are required to collect traffic and weather information in real time and suggest optimal routes and rest stops through dialogue with the user. However, current systems collect and provide information in a fragmented manner, making it difficult to meet the user's personalized needs. In addition, dialogue systems using voice recognition technology and natural language processing technology are not fully integrated, limiting the user experience. This poses a challenge as the convenience and safety of autonomous vehicles are not fully realized.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0764] In this invention, the server includes means for acquiring specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs not only through text but also through other means, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with users using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and proposing optimal routes and rest stops. This enables autonomous vehicles to suggest personalized, optimal routes and rest stops through real-time information collection and dialogue with users.

[0765] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[0766] "Latest news" refers to the latest data and news about current situations and events.

[0767] "Personal information" is information about an individual, including personal preferences and history.

[0768] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[0769] "Means of conversation" refers to a method of interacting with the user using voice recognition technology or natural language processing technology.

[0770] "Roles" refers to the method of assigning specific functions and responsibilities to each AI.

[0771] "Security" refers to the technologies and measures used to protect information and systems and prevent unauthorized access and data leaks.

[0772] A "platform" refers to the system or environment that forms the foundation of software and hardware.

[0773] "Means of collecting traffic and weather information in real time" refers to a method of instantly obtaining data on current traffic conditions and weather.

[0774] "Speech recognition technology" refers to technology that converts speech into text.

[0775] "Natural language processing technology" refers to artificial intelligence technology for understanding and generating human language.

[0776] "Means for learning about the user's past driving history and preferences" refers to a method of collecting data on the user's past behavior and preferences and learning based on that data.

[0777] "Means for suggesting optimal routes and rest points" refers to methods that suggest the most suitable routes and rest points for users based on collected information.

[0778] The system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with the user using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and suggesting optimal routes and rest stops.

[0779] The server collects real-time traffic and weather information from various databases and internet sources using software (e.g., the requests library) that uses APIs to retrieve data. The collected information is then analyzed within the server and provided to the user.

[0780] The device converts the user's speech into text using speech recognition technology (e.g., the speech_recognition library). The converted text is then analyzed using natural language processing technology (e.g., the transformers library) to understand the user's intent, allowing for dialogue with the user.

[0781] Users interact with the system through voice. For example, by asking, "Where is the next rest stop?", the system will suggest the best rest stop based on real-time traffic information and the user's past driving history. The suggested information is conveyed to the user using voice synthesis technology (e.g., the pyttsx3 library).

[0782] As a concrete example, if a user asks, "Where is the next rest stop?" the system responds as follows:

[0783] User: Where is the next rest stop?

[0784] System: Based on current traffic conditions and past driving history, the system will suggest the next rest stop, "XX Service Area." This area has a restaurant and restrooms, making it an ideal place to take a break.

[0785] In this way, intelligent navigation systems in autonomous vehicles can provide users with a convenient and safe driving experience.

[0786] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0787] Step 1:

[0788] The server collects traffic and weather information in real time from various databases and information sources on the Internet. Specifically, it uses the requests library to obtain data using an API. An API endpoint is specified as input, and traffic and weather information in JSON format is obtained as output. This allows the latest traffic and weather information to be aggregated on the server.

[0789] Step 2:

[0790] The server analyzes the collected traffic and weather information. Specifically, it analyzes the acquired JSON data and extracts the necessary information (e.g., locations of traffic jams, weather changes). It receives JSON data as input and obtains the analyzed information as output. This prepares useful information to provide to users.

[0791] Step 3:

[0792] The device uses voice recognition technology to convert the user's voice into text. Specifically, it uses the speech_recognition library to convert the voice data input from the microphone into text data. It receives voice data as input and obtains text data as output. This allows the user's speech content to be obtained in text format.

[0793] Step 4:

[0794] The device uses natural language processing technology to analyze the user's text data and understand their intent. Specifically, it uses the transformers library to analyze the text data using a generative AI model. It receives text data as input and obtains data indicating the user's intent as output. This allows the device to understand the user's questions and requests.

[0795] Step 5:

[0796] The server generates an appropriate response based on the user's intent. Specifically, it suggests optimal routes and rest stops based on analyzed traffic and weather information, the user's past driving history, and preferences. It receives the user's intent data and analyzed information as input, and obtains response data as output. This allows it to generate appropriate suggestions for the user.

[0797] Step 6:

[0798] The device uses speech synthesis technology to generate response data and communicates it to the user. Specifically, it uses the pyttsx3 library to convert text-format response data into speech data. It receives response data as input and obtains speech data as output. This provides a speech response to the user.

[0799] Step 7:

[0800] The user receives the voice response provided by the system and decides on the next course of action. Specifically, the user continues driving according to the route and rest points suggested by the system. The system receives the voice response as input and decides on the next course of action as output. This allows the user to reach their destination safely and efficiently.

[0801] Example 2

[0802] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0803] Conventional information provision systems have difficulty efficiently incorporating specialized knowledge, the latest information, and personal information, making it difficult to provide consistent information to users. Furthermore, when each AI operates independently, centralized information management and security assurance become issues. Furthermore, there was a need for a system that could integrate multiple processes, such as news information collection and summarization, personal information input and storage, and recommended information generation.

[0804] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0805] In this invention, the server includes a means for taking in specialized knowledge, the latest information, and personal information, a means for conversing with the humanoid AI in addition to text, a means for assigning roles to each AI, a means for providing information together with security and platform information, a means for collecting and summarizing news information, a means for inputting and saving personal information, a means for generating recommended information, a means for centrally managing information, and a means for providing information. This makes it possible to provide consistent information to users, and achieves centralized information management and security while each AI operates independently.

[0806] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0807] "Latest news" is the latest data and news about current situations and events.

[0808] "Personal information" refers to information about an individual user, such as data about that user, their interests, and past behavioral history.

[0809] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[0810] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0811] "Security" refers to the technologies and methods used to protect information and systems from unauthorized access and data leaks.

[0812] A "platform" is a system or environment that serves as the foundation for providing software and services.

[0813] "Means for collecting and summarizing news information" refers to a method for obtaining news data and concisely summarizing its contents.

[0814] "Means for entering and storing personal information" refers to the method by which a user enters their own information and stores it in a database.

[0815] The "means for generating recommendation information" is a method for generating appropriate suggestions and advice based on the user's personal information.

[0816] A "means for centrally managing information" is a method for integrating and managing multiple pieces of information.

[0817] "Means of providing information" refers to the method of providing the necessary information to users in an appropriate format.

[0818] This invention is a system that assigns specific roles to multiple AIs, centrally manages the information generated by each AI, and provides it to users in an appropriate format. A specific embodiment of this system is described below.

[0819] Hardware and software used

[0820] Hardware: Servers, terminals (user devices)

[0821] Software: Generative AI models (e.g., GPT-4®), database management systems (e.g., MySQL®), security software (e.g., SSL / TLS)

[0822] News information gathering and summarization

[0823] The server uses a news API (e.g., NewsAPI) to collect news information. The server periodically sends requests to the news API to obtain the latest news data. The obtained news data is input into a generative AI model, and a prompt is sent to generate a summary. An example of a specific prompt is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The server stores the summarized news information in a database.

[0824] Entering and saving personal information

[0825] The user uses the device to input their personal information (e.g., interests, past browsing history), and the device sends the input personal information to the server, which stores the received personal information in a database.

[0826] Generate recommendations

[0827] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. An example of a specific prompt is, "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The server stores the generated recommendations in a database.

[0828] Centralized information management

[0829] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The server uses security software (e.g., SSL / TLS) to ensure the safety of data. Specifically, access to the database is encrypted with SSL / TLS to prevent unauthorized access.

[0830] Providing information

[0831] A user uses a device to request news information or recommended information. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The device then displays the received information to the user. Specifically, it displays summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[0832] In this way, a system will be created in which each AI has a specific role, and the information generated will be centrally managed and provided to users.

[0833] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0834] Step 1: Gather and summarize news information

[0835] The server sends a request to the news API to obtain the latest news data. The input is news data from the news API, and the output is the obtained news data. Specifically, the server sends a request to the news API to "obtain the latest news." The obtained news data is input into the generative AI model, and a prompt sentence is sent to generate a summary. An example of a prompt sentence is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The output is summarized news information. The server stores the summarized news information in a database.

[0836] Step 2: Enter and save your personal information

[0837] The user uses the device to input their own personal information (e.g., interests, past browsing history). The input is the personal information the user enters into the device, and the output is the personal information sent to the server. The device sends the entered personal information to the server. The server stores the received personal information in a database. Specifically, the user enters "Interests: Technology, Past Browsing History: AI-related news" into the device, and the device sends this to the server.

[0838] Step 3: Generate recommendations

[0839] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. The input is the personal information stored in the database, and the output is the prompt sent to the generative AI model. An example of a prompt is "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The output is the generated recommendations. The server stores the generated recommendations in a database.

[0840] Step 4: Centralize your information

[0841] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The input is news information and recommendation information, and the output is centrally managed data. The server uses security software (e.g., SSL / TLS) to ensure the security of the data. Specifically, it encrypts access to the database with SSL / TLS to prevent unauthorized access.

[0842] Step 5: Provide information

[0843] A user uses a device to request news information or recommended information. The input is the user's request, and the output is the request sent to the server. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The output is the information provided to the user. The device displays the received information to the user. A specific operation is to display summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[0844] (Application example 2)

[0845] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0846] Conventional information provision systems have difficulty in providing consistent information to users, and are particularly inadequate in providing personalized content. Furthermore, there is a lack of a way to centrally manage information generated by multiple AIs and provide it to users in an appropriate format, which results in a lack of an improved user experience.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on the user's interests, and means for analyzing news data using natural language processing technology and classifying it by category. This enables consistent information provision to users, realizing the provision of personalized content.

[0848] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0849] "Latest news" is the latest data and news about current situations and events.

[0850] "Personal Information" means information about an individual user's personal data, interests, and preferences.

[0851] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0852] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0853] "Security information" means information relating to the safety of systems and data.

[0854] "Platform information" refers to information relating to the operation and management of the entire system.

[0855] "Centralized management" is a method of integrating and managing multiple pieces of information in one system or location.

[0856] "Recommended content" is information or media selected based on a user's interests and preferences.

[0857] "Natural language processing technology" is a technology that uses computers to understand and analyze human language.

[0858] "News Data" means information obtained from news organizations or news sources.

[0859] A "categorization method" is a method for separating information into specific categories.

[0860] A system for carrying out this invention is configured as follows: The server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on user interests, and means for analyzing news data using natural language processing technology and classifying it into categories.

[0861] Hardware and software used

[0862] Hardware: Smartphones, smart glasses

[0863] Software: Python, TensorFlow (registered trademark), Flask, SQLite, REST API

[0864] Data processing and calculation flow

[0865] 1. Data Collection:

[0866] The server retrieves the latest news data from a news API (e.g., NewsAPI) and stores user personal information (interests, browsing history, etc.) in a SQLite database.

[0867] 2. Data Processing:

[0868] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. Based on the user's personal information, a model is trained using TensorFlow (registered trademark) to generate recommended content.

[0869] 3. Information generation:

[0870] The server uses News AI to generate the latest news information, Recommendation AI to generate recommended content based on the user's interests, and Security AI to generate security information.

[0871] 4. Information Integration:

[0872] The server uses Flask to build a REST API that centrally manages the information generated by each AI, consolidates the information from each AI, and provides it to users in an appropriate format.

[0873] 5. User Interface:

[0874] The device (smartphone or smart glasses) displays the integrated information.

[0875] Examples and prompts

[0876] Specific examples

[0877] News AI: "Today's latest news is about new discoveries in the field of technology."

[0878] Recommendation AI: "We recommend the following content based on your recent topics of interest."

[0879] Security AI: "The latest security information is to beware of phishing scams."

[0880] Prompt Sentence Examples

[0881] Prompt for News AI: "Get the latest technology news."

[0882] Prompt for recommendation AI: "Recommend content based on the user's interests."

[0883] Prompt security AI: "Please provide us with the latest security information."

[0884] In this way, the server will enable each AI to have a specific role and provide personalized information to users, thereby realizing a richer content distribution service.

[0885] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0886] Step 1:

[0887] The server retrieves the latest news data from the news API. The input is the news API endpoint, and the output is the retrieved news data. Specifically, the server sends an HTTP request and receives news data in JSON format.

[0888] Step 2:

[0889] The server stores user personal information (interests, browsing history, etc.) in an SQLite database. The input is the user's personal information, and the output is the information stored in the database. Specifically, the server receives user information and inserts it into the database using an SQL query.

[0890] Step 3:

[0891] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. The input is news data, and the output is news data classified by category. Specifically, the server analyzes the news data using an NLP library (e.g., spaCy) and assigns category tags.

[0892] Step 4:

[0893] The server uses TensorFlow® to train a model for generating recommended content based on the user's personal information. The input is the user's personal information and news data, and the output is the trained recommendation model. Specifically, the server preprocesses the data and trains the model using TensorFlow®.

[0894] Step 5:

[0895] The server uses news AI to generate the latest news information. The input is news data categorized by category, and the output is the generated news information. Specifically, the server analyzes the news data and selects news appropriate for the user.

[0896] Step 6:

[0897] The server uses recommendation AI to generate recommended content based on the user's interests. The input is a trained recommendation model and the user's personal information, and the output is recommended content. Specifically, the server uses the model to recommend the most suitable content to the user.

[0898] Step 7:

[0899] The server generates security information using security AI. The input is security-related data, and the output is the generated security information. Specifically, the server analyzes the security data and provides warnings and advice to the user.

[0900] Step 8:

[0901] The server uses Flask to build a REST API that centrally manages the information generated by each AI. The input is the information generated by each AI, and the output is the integrated information. Specifically, the server uses the Flask framework to set up an API endpoint and integrate the information.

[0902] Step 9:

[0903] The device (smartphone or smart glasses) displays the integrated information. The input is the integrated information from the server, and the output is the information displayed on the user interface. Specifically, the device retrieves information from the API and displays it visually to the user.

[0904] Example 3

[0905] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0906] In conventional information provision systems, it was difficult for each AI to operate independently, and it was also difficult to provide the information generated by each AI consistently. Furthermore, the process from when a user sends a request to when they receive the information is complicated, making it difficult to provide information efficiently.

[0907] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0908] In this invention, the server includes means for a user to send a request from a terminal, means for the terminal to transfer the request to the server, means for the server to analyze the request and select an appropriate generative AI model, means for the server to input a prompt sentence into the generative AI model selected by the server and generate information, means for the server to share the generated information with other generative AI models, means for the server to return the final information to the terminal, and means for the terminal to display the information to the user. This makes it possible to provide consistent information as a whole even though each generative AI model operates independently.

[0909] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[0910] "Latest information" refers to the most current data, news, and events.

[0911] "Personal information" refers to data or information relating to an individual, including personal characteristics, preferences, and history.

[0912] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[0913] A "role" is a responsibility or function assigned to perform a specific task or function.

[0914] "Security" refers to the means and measures taken to protect information and systems from unauthorized access and attacks.

[0915] A "platform" is the environment or system on which software and hardware operate.

[0916] A "request" is a request or inquiry made by a user to a system.

[0917] A "server" is a computer that provides services to other computers on a network.

[0918] A "terminal" is a device or computer that is directly operated by a user.

[0919] A "generative AI model" is an artificial intelligence model that is trained to generate information for a specific task.

[0920] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[0921] "Generating information" means creating new data or answers based on input data and prompts.

[0922] "Sharing information" means exchanging generated data and knowledge with other AI models and systems.

[0923] "Consistent information provision" means that data obtained from multiple information sources and AI models is consistent and provided to users in a unified form.

[0924] This invention is a system in which each generative AI model operates independently and generates and provides information according to its role. Furthermore, each generative AI model can share information with other generative AI models, providing consistent information as a whole.

[0925] Hardware and software used

[0926] The server trains and deploys each generative AI model using machine learning frameworks such as TensorFlow (registered trademark) and PyTorch. The server also communicates with terminals using HTTP requests and WebSockets. Terminals are devices such as smartphones and PCs that provide an interface for users to input requests.

[0927] Data processing and calculation

[0928] The server receives a request from the user, analyzes it, and selects an appropriate generative AI model. The selected generative AI model inputs a prompt sentence to generate information. The generated information is shared with other generative AI models to provide consistent information overall. The final information is sent back from the server to the device and displayed to the user.

[0929] Specific examples

[0930] As a concrete example, consider a news article generation system. A news article generation AI, a sports article generation AI, and a weather forecast generation AI each operate independently to generate specific information. When a user requests "Tell me today's news," the device sends this request to a server. The server then calls the news article generation AI to generate the latest news articles. The generated news articles are shared with other generative AI models (for example, a sports article generation AI and a weather forecast generation AI), providing consistent information overall.

[0931] Prompt Sentence Examples

[0932] Here is an example prompt:

[0933] "Tell me the news today"

[0934] "Tell me the latest sports news"

[0935] "What's the weather forecast for today?"

[0936] In this way, a system is constructed that provides consistent information as a whole, even though each generative AI model operates independently. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0937] Step 1:

[0938] The user sends a request from the terminal.

[0939] Users open the application on their smartphone or computer, type "Tell me today's news" into the search bar, and press the send button.

[0940] Input: User request (e.g. "Tell me today's news")

[0941] Output: Request data sent to the terminal

[0942] Step 2:

[0943] The terminal forwards the request to the server.

[0944] The device receives the user's input and sends an HTTP POST request to the server, which includes the user's input.

[0945] Input: User request data

[0946] Output: HTTP request sent to the server

[0947] Step 3:

[0948] The server analyzes the request and selects the appropriate generative AI model.

[0949] The server analyzes the received request and detects the keyword "news." Based on this, it selects a news article generation AI.

[0950] Input: HTTP request data

[0951] Output: Selected generative AI model (news article generation AI)

[0952] Step 4:

[0953] The server inputs a prompt sentence into the selected generative AI model to generate information.

[0954] The server inputs the prompt "Tell me today's news" into the news article generation AI, and the AI ​​model generates the latest news articles. The generated news articles are returned to the server in text format.

[0955] Input: Prompt sentence (e.g. "Tell me today's news")

[0956] Output: Generated news article

[0957] Step 5:

[0958] The server shares the generated information with other generative AI models.

[0959] The server shares the generated news articles with the sports article generation AI and weather forecast generation AI, allowing other AIs to generate information based on the latest news information.

[0960] Input: Generated news article

[0961] Output: Information shared with other generative AI models

[0962] Step 6:

[0963] The server returns the final information to the terminal.

[0964] The server then compiles the news article, sports news, and weather forecast into a single response and sends it back to the device. This response is sent in JSON format.

[0965] Input: Shared information (news articles, sports news, weather forecasts)

[0966] Output: JSON response sent to the terminal

[0967] Step 7:

[0968] The terminal displays the information to the user.

[0969] The device parses the JSON response received from the server and displays news articles, sports news, and weather forecasts in their respective sections, allowing users to view this information on their device screen.

[0970] Input: JSON response from the server

[0971] Output: Information displayed to the user (news articles, sports news, weather forecasts)

[0972] (Application example 3)

[0973] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0974] In self-driving vehicles, multiple AIs are required to operate independently and generate and provide information according to their respective roles. However, in order for these AIs to provide consistent information, a means is needed to efficiently share and integrate the information generated by each AI. Furthermore, it is technically difficult to process and integrate a wide range of information in real time, such as obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation. A system that can solve these issues is needed.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information; means for conversing with humanoid AIs not only through text but also through other means; means for assigning roles to each AI; means for providing information together with security and platform information; means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in an autonomous vehicle; and means for sharing information generated by each AI through a communication protocol to provide consistent information. This enables multiple AIs to operate independently while efficiently sharing information and providing consistent information.

[0976] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[0977] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[0978] "Personal information" refers to data and attribute information about individuals.

[0979] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[0980] "Means of conversation" refers to technologies and interfaces that allow for natural dialogue with users.

[0981] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[0982] "Security information" is information used to ensure the safety of systems and data.

[0983] "Platform information" is information about the platform on which the system operates.

[0984] The "means of provision" refers to the method by which information or services are provided to users.

[0985] An "autonomous vehicle" is a vehicle that drives autonomously without driver intervention.

[0986] "Obstacle detection" is a technology that detects obstacles that exist in the direction of travel of a vehicle.

[0987] "Route optimization" is a technique for calculating the optimal route to a destination.

[0988] "Traffic situation analysis" is a technique for evaluating and analyzing the current traffic situation.

[0989] "Passenger comfort assessment" is a technique for assessing passenger comfort.

[0990] A "communication protocol" is a set of rules and procedures for sending and receiving data.

[0991] "Providing consistent information" refers to integrating data obtained from multiple sources and presenting it in a consistent manner.

[0992] A system for carrying out this invention is configured as follows: The server includes means for taking in specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in autonomous vehicles, and means for sharing information generated by each AI via a communication protocol to provide consistent information.

[0993] Hardware and Software Configuration

[0994] The server uses the following hardware and software:

[0995] Hardware

[0996] Autonomous vehicle sensors (cameras, LIDAR, GPS, etc.)

[0997] In-vehicle computer

[0998] software

[0999] Python programming language

[1000] threading module (concurrency)

[1001] queue module (data queue)

[1002] Data processing and calculation

[1003] The server processes the data and performs calculations as follows:

[1004] 1. Sensor data acquisition: Acquire real-time data from the autonomous vehicle's sensors, such as video data from cameras and LIDAR, and location information from GPS.

[1005] 2. Data processing by AI: Each AI processes data using its own algorithm. For example, obstacle detection AI analyzes sensor data and determines whether there is an obstacle ahead. Route optimization AI calculates the optimal route based on map data. Traffic situation analysis AI analyzes traffic data and evaluates the current traffic situation. Passenger comfort assessment AI evaluates comfort based on passenger data.

[1006] 3. Data sharing via communication protocols: Information generated by each AI is shared via communication protocols, enabling consistent information provision. For example, obstacle detection AI can send information about detected obstacles to route optimization AI, which can then calculate the optimal route to avoid them.

[1007] Specific examples

[1008] For example, imagine an autonomous vehicle driving through a city: it detects obstacles ahead (e.g., pedestrians), recalculates the optimal route, analyzes traffic conditions, and evaluates passenger comfort. All this information is integrated in real time and reflected in the vehicle's operations.

[1009] Prompt Sentence Examples

[1010] "Design a system that detects pedestrians ahead, recalculates the optimal route, analyzes traffic conditions, and assesses passenger comfort while an autonomous vehicle is driving through a city."

[1011] In this way, it is possible to provide a specific embodiment of the invention.

[1012] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1013] Step 1:

[1014] The server receives real-time data from the autonomous vehicle's sensors. As input, it receives video data from cameras and LIDAR, and location information from GPS. It then takes this data and passes it on to the next processing step.

[1015] Step 2:

[1016] The server passes sensor data as input to the obstacle detection AI, which analyzes the video and LIDAR data to determine whether there is an obstacle ahead and generates the location information of the obstacle as output.

[1017] Step 3:

[1018] The server passes map data and obstacle location information to the route optimization AI as input. The route optimization AI calculates the optimal route based on the map data and generates a new route to avoid obstacles. The output is the optimal route information.

[1019] Step 4:

[1020] The server passes traffic data as input to the traffic situation analysis AI. The traffic situation analysis AI analyzes the current traffic data and evaluates traffic flow and congestion information. As output, it generates a traffic situation evaluation result.

[1021] Step 5:

[1022] The server passes passenger data as input to the passenger comfort evaluation AI, which evaluates comfort based on the passenger data and generates passenger comfort evaluation results as output.

[1023] Step 6:

[1024] The server shares the information generated by each AI through a communication protocol. As input, it receives information on the location of obstacles, optimal route information, traffic situation evaluation results, and passenger comfort evaluation results. This information is integrated to provide consistent information.

[1025] Step 7:

[1026] The server passes the integrated information to the autonomous vehicle's control system, which receives the integrated information as input and reflects it in the vehicle's operation. Specifically, it changes the route to avoid obstacles, adjusts the speed according to traffic conditions, and operates in a way that takes passenger comfort into consideration.

[1027] In this way, the server uses multiple AIs to optimize the operation of autonomous vehicles and provide consistent information.

[1028] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1029] "Example 1"

[1030] The invention, which combines the emotion engine, is an AI system that recognizes the user's emotions and engages in dialogue accordingly. Specifically, it infers emotions from the user's tone of voice, facial expressions, and choice of words, and generates a response appropriate to those emotions. For example, it provides words of comfort when the user is sad, and words of sympathy when the user is happy. This makes the dialogue between the AI ​​and the user more natural and human-like.

[1031] "Example 2"

[1032] The invention combining emotion engines is a system in which each AI operates independently to generate and provide information according to its respective role. Specifically, some AIs provide the latest news, others provide recommended information based on the user's personal information, and still others use the emotion engine to recognize the user's emotions and engage in dialogue accordingly. This allows users to obtain information tailored to their emotions.

[1033] "Example 3"

[1034] The form of the invention that combines emotion engines is a system in which each AI shares information with other AIs and provides consistent information as a whole. Specifically, some AIs share information generated or recognized user emotions with other AIs, and provide information and engage in dialogue based on that. For example, if one AI recognizes the user's emotion of joy, it shares that information with other AIs, and the other AIs also provide information and engage in dialogue that is tailored to the user's joy. This enables consistent dialogue as a whole.

[1035] The processing flow of each embodiment will be described below.

[1036] "Example 1"

[1037] Step 1: Initiate a dialogue with the user.

[1038] Step 2: Infer the user's emotions from their tone of voice, facial expressions, and choice of words.

[1039] Step 3: Generate a response appropriate to the estimated emotion.

[1040] Step 4: Provide the generated response to the user.

[1041] "Example 2"

[1042] Step 1: Each AI generates and provides information independently.

[1043] Step 2: Some AI will provide news updates.

[1044] Step 3: Another AI provides recommendations based on the user's personal information.

[1045] Step 4: Another AI uses an emotion engine to recognize the user's emotions and interact accordingly.

[1046] "Example 3"

[1047] Step 1: Each AI shares information with other AIs.

[1048] Step 2: Some AIs share the information they generate and the user's emotions they recognize with other AIs. Step 3: Other AIs provide information and engage in dialogue based on the shared information.

[1049] Example 1

[1050] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1051] Conventional AI systems have difficulty effectively incorporating specialized knowledge, up-to-date information, and personal information, resulting in a lack of natural, human-like interaction with users. Furthermore, they are unable to recognize user emotions and generate appropriate responses, limiting the user experience.

[1052] The identification process by the identification processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means. In this invention, the server includes a means for acquiring specialized knowledge, the latest information, and personal information, a means for converting a user's voice into text using speech recognition technology, a means for analyzing the text using natural language processing technology, a means for estimating the user's emotions from the user's tone of voice and facial expression, a means for generating an appropriate response based on the estimated emotions, a means for assigning roles to each AI, and a means for providing information together with security and platform information. This makes the dialogue with the user more natural and human-like, and makes it possible to provide an appropriate response according to the user's emotions.

[1053] "Expertise" refers to information that involves advanced understanding or skill in a particular field.

[1054] "Latest information" refers to the most recent data and knowledge available at the present time.

[1055] "Personal Information" means data or information relating to an individual.

[1056] "Speech recognition technology" refers to technology that converts speech into text.

[1057] "Natural language processing technology" refers to the technology of analyzing text data and understanding its meaning.

[1058] "Means for estimating emotions" refers to a method for inferring emotions from the user's tone of voice and facial expressions.

[1059] The "means for generating a response" refers to a method for creating an appropriate reply based on the analyzed information and estimated emotions.

[1060] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1061] "Security and platform information" refers to data related to the security and infrastructure of a system.

[1062] This invention is a system including means for incorporating specialized knowledge, up-to-date information, and personal information, means for converting a user's voice into text using speech recognition technology, means for analyzing the text using natural language processing technology, means for inferring the user's emotions from their tone of voice and facial expressions, and means for generating an appropriate response based on the inferred emotions.

[1063] The server collects specialized knowledge, the latest information, and personal information from various databases and online sources. This information is used as training data for AI. Specifically, the server obtains information using web scraping technology or APIs and stores it in a database. For example, Python's BeautifulSoup or Scrapy can be used to extract information from web pages and store it in a database.

[1064] The device uses speech recognition technology to convert the user's speech into text. Specifically, the device uses the Google® Cloud Speech-to-Text API or IBM Watson® natural language processing service to convert the user's speech into text, analyze the text, and generate an appropriate response based on the analysis results.

[1065] The device uses an emotion engine to infer emotions from the user's tone of voice and facial expressions. Specifically, it analyzes the user's facial expressions using Microsoft® Azure® Emotion API and OpenCV to infer emotions. Based on the inferred emotion, the AI ​​generates an appropriate response. For example, if the user is sad, it will provide comforting words such as, "Are you okay? Is there anything I can help you with?"

[1066] When a user asks a question or makes a request to the system, the device recognizes the speech and converts it into text. It then uses natural language processing technology to analyze the text and understand the user's intent. It then uses an emotion engine to estimate the user's emotions and generate an appropriate response.

[1067] For example, if a user asks, "What's the weather like today?", the device converts the voice to text and analyzes the text to provide weather information. If a user says, "I'm tired today," in a sad voice, the device analyzes the tone of the voice and recognizes the sad emotion. The device then provides comforting words such as, "You've had a hard time today. Please take a rest."

[1068] An example prompt is, "Design an AI system that offers words of comfort when the user is sad. Explain how you can recognize emotions from the user's tone of voice and facial expression and generate a response appropriate to those emotions."

[1069] In this way, the system can make interactions with the user more natural and human-like, and provide appropriate responses according to the user's emotions.

[1070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1071] Step 1: Gather information

[1072] The server collects specialized knowledge, up-to-date information, and personal information from various databases and internet sources. It receives website URLs and API endpoints as input and retrieves information using web scraping techniques and APIs. Specifically, it uses Python's BeautifulSoup and Scrapy to extract information from web pages and store it in a database. The collected information is then stored in a database as output.

[1073] Step 2: Voice Recognition

[1074] The device receives the user's voice as input and converts it into text using speech recognition technology. Specifically, it converts the voice data into text data using the Google® Cloud Speech-to-Text API. The converted text is obtained as output.

[1075] Step 3: Natural Language Processing

[1076] The device receives text obtained by speech recognition as input and analyzes the text using natural language processing technology. Specifically, it uses the natural language processing service of IBM Watson (registered trademark) to analyze the meaning of the text and understand the user's intent. The analysis results are obtained as output.

[1077] Step 4: Emotion Recognition

[1078] The device receives the user's tone of voice and facial expression as input and uses an emotion engine to estimate the user's emotion. Specifically, it analyzes the user's facial expression using Microsoft® Azure® Emotion API and OpenCV, and estimates the emotion. The estimated emotion is obtained as output.

[1079] Step 5: Response Generation

[1080] The device receives the analysis results of natural language processing and the estimated results of emotion recognition as input and generates an appropriate response. Specifically, it uses a generative AI model to create a response based on the user's intention and emotion. The generated response is obtained as output.

[1081] Step 6: Providing a response

[1082] The terminal provides the generated response to the user by converting the text response into speech using speech synthesis technology and playing it for the user. The user receives the response as an output.

[1083] In this way, the system can make interactions with the user more natural and human-like, and provide appropriate responses according to the user's emotions.

[1084] (Application example 1)

[1085] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1086] Conventional security systems respond uniformly without considering the user's emotional state, making it impossible to provide appropriate security alerts according to the user's psychological state. Furthermore, even conversational AI systems using voice recognition and natural language processing technologies have difficulty recognizing the user's emotions and engaging in dialogue accordingly. This makes dialogue with users unnatural, making it impossible to properly manage security risks.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions and providing appropriate security alerts, means for enabling dialogue with the user using voice recognition technology and natural language processing technology, and means for recognizing the user's facial emotions using a camera. This makes it possible to provide appropriate security alerts according to the user's emotional state and realize more natural, human-like dialogue.

[1088] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1089] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[1090] "Personal Information" means certain data or information about an individual.

[1091] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[1092] "Means for conversation" refers to a method for interacting with users using voice recognition technology or natural language processing technology.

[1093] "Roles" are ways to assign specific functions and responsibilities to each AI.

[1094] "Security and platform information" refers to data related to the security and infrastructure of a system.

[1095] The "means of provision" is the method by which information or services are provided to users.

[1096] "Means for recognizing emotions and providing appropriate security alerts" refers to a method for detecting a user's emotional state and providing appropriate warnings or notifications.

[1097] "Speech recognition technology" is a technology that converts speech into text.

[1098] "Natural language processing technology" is a technology for understanding and generating human language.

[1099] "Means for recognizing the user's facial emotions using a camera" refers to a method for analyzing the user's facial expressions using a camera and estimating their emotions.

[1100] A system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing user emotions and providing appropriate security alerts, means for enabling dialogue with users using voice recognition technology and natural language processing technology, and means for recognizing user facial emotions using a camera.

[1101] System Program

[1102] This system operates using the following hardware and software:

[1103] Hardware: Your smartphone's microphone and camera

[1104] Software: Python, speech_recognition library, transformers library, OpenCV

[1105] Processing flow

[1106] The server first collects the user's voice using the smartphone's microphone and converts the voice to text using the speech_recognition library. It then inputs the converted text into an emotion recognition model in the transformers library to estimate the user's emotion. It then captures the user's face using the smartphone's camera and detects it using OpenCV. It then inputs the facial image into the emotion recognition model to estimate the facial emotion. Finally, if the user's emotion is "sadness" or "anger," it raises a security alert.

[1107] Specific examples

[1108] For example, if a user says, "Recently, things haven't been going well at work and I'm feeling stressed," the speech recognition system will recognize this text, and the emotion recognition model will determine it as "anger" or "sadness." As a result, the server will notify the user with a "Security Alert: The user is in an unstable state."

[1109] Prompt Sentence Examples

[1110] If a user says, "Work hasn't been going well lately and I'm feeling stressed," what emotion would the emotion recognition model infer?

[1111] In this way, an emotion-aware security assistant can monitor user emotions in real time and provide security alerts when necessary.

[1112] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1113] Step 1:

[1114] The server collects the user's voice using the smartphone's microphone. The input is the user's voice, and the output is voice data. This voice data is used for subsequent voice recognition processing.

[1115] Step 2:

[1116] The server converts the speech data to text using the speech_recognition library. The input is speech data and the output is text data. This text data is input to the emotion recognition model.

[1117] Step 3:

[1118] The server inputs text data into the emotion recognition model of the Transformers library to estimate the user's emotion. The input is text data, and the output is an emotion label (e.g., joy, sadness, anger, etc.). This emotion label is used to determine security alerts.

[1119] Step 4:

[1120] The server captures the user's face using the smartphone camera. The input is the camera image, and the output is facial image data. This facial image data is used for facial emotion recognition processing.

[1121] Step 5:

[1122] The server uses OpenCV to detect faces from the facial image data and inputs the facial images into an emotion recognition model. The input is the facial image data, and the output is the facial emotion label. This facial emotion label is also used to determine security alerts.

[1123] Step 6:

[1124] The server integrates emotion labels from the voice and the face to determine the user's overall emotional state. The input is the emotion labels from the voice and the face, and the output is the overall emotion label. Based on this overall emotion label, it decides whether to issue a security alert.

[1125] Step 7:

[1126] The server generates a security alert if the user's overall emotion label is "sad" or "anger." The input is the overall emotion label, and the output is a security alert notification. This notification provides the user with appropriate warnings and advice.

[1127] Example 2

[1128] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1129] Conventional information provision systems have difficulty in providing users with the latest information and personal information they need. Furthermore, they are unable to provide information that reflects the user's emotions, which limits the user experience. Furthermore, because each AI does not operate independently, there is a lack of consistency in the information provided.

[1130] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1131] In this invention, the server includes a means for importing specialized knowledge, the latest information, and personal information, a means for conversing with the humanoid AI not only through text, a means for assigning roles to each AI, a means for providing information together with security and platform information, a means for receiving user requests, a means for collecting news information, a means for generating recommended information based on personal information, a means for recognizing user emotions, and a means for integrating information from each AI and providing it to the user. This allows users to appropriately receive the latest information and personal information, and further enables information to be provided according to emotions. Furthermore, each AI operates independently, maintaining consistency of information.

[1132] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1133] "Latest news" is the latest data and news about current situations and events.

[1134] "Personal information" refers to information about an individual user, such as data, history, and interests.

[1135] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[1136] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1137] "Security" refers to the techniques and measures used to protect information and systems from unauthorized access and attacks.

[1138] A "platform" is the underlying environment on which software and hardware operate.

[1139] "Means for receiving requests" refers to the method for receiving requests or inquiries from users.

[1140] "Methods of gathering news information" are ways of obtaining the latest news from the internet and other sources.

[1141] The "means for generating recommended information" is a method for creating appropriate suggestions and advice based on the user's personal information.

[1142] "Means for recognizing emotions" refers to a method for analyzing and determining the user's emotional state.

[1143] "Means of integrating information" refers to a method of consolidating data obtained from multiple AIs and providing it as one.

[1144] This invention is a system in which a user accesses the system through a terminal and a server provides information using various AIs. A specific embodiment of this system will be described below.

[1145] Hardware and software used

[1146] The server uses a high-performance server (e.g., a high-performance server). As for software, a platform (e.g., a machine learning framework) is used to manage the operation of each AI. This allows the news AI, user AI, and emotion engine AI to operate efficiently.

[1147] Program processing

[1148] The server receives requests from users and issues instructions to each AI. The news AI collects the latest news information from online news sites and RSS feeds. The user AI generates recommended information based on the user's personal information (past browsing history, interests, etc.). The emotion engine AI analyzes the user's input, voice, facial expressions, etc. to determine the user's emotional state.

[1149] Specific examples

[1150] When a user accesses the system through a terminal, the server performs the following process.

[1151] 1. Providing news information:

[1152] News AI collects the latest news information and provides it to users.

[1153] Example: "The headline news today is that economic growth exceeded expectations."

[1154] 2. Providing Recommendations:

[1155] User AI recommends relevant information based on the user's past browsing history and personal information.

[1156] For example: "Here are some tech-related articles that have recently interested you."

[1157] 3. Emotionally-sensitive dialogue:

[1158] The emotion engine AI recognizes the user's emotions and interacts accordingly.

[1159] For example: "You seem a little tired today. I'd like some relaxing music."

[1160] Prompt Sentence Examples

[1161] Below is an example of a prompt sentence to input to the generative AI model.

[1162] "Please tell me the latest news."

[1163] "What movie would you recommend for me?"

[1164] "I'm feeling a little down today. Can you give me some cheer-up advice?"

[1165] This system allows users to obtain information tailored to their emotions and interests.

[1166] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1167] Step 1:

[1168] A user accesses the system through a terminal.

[1169] Specifically, the user opens the app on their smartphone and enters their username and password on the login screen.

[1170] Input: Username, Password

[1171] Output: Login success message, user ID

[1172] Step 2:

[1173] The server receives the user request.

[1174] As a specific operation, the user sends a request such as "Please tell me the latest news."

[1175] Input: User request (e.g., What's the latest news?)

[1176] Output: Analysis result of request content

[1177] Step 3:

[1178] The server has the news AI collect the latest news information.

[1179] Specifically, the server instructs the news AI to "collect the latest news." The news AI then collects the latest news articles from major news sites and sends them back to the server.

[1180] Input: news gathering instructions

[1181] Output: Latest news information

[1182] Step 4:

[1183] The server allows the user AI to generate recommendations based on the user's personal information.

[1184] Specifically, the server instructs the user AI to "generate recommendations appropriate for this user." The user AI analyzes the user's past browsing history and generates recommendations.

[1185] Input: User's personal information, instructions for generating recommended information

[1186] Output: Recommendations

[1187] Step 5:

[1188] The server makes the emotion engine AI recognize the user's emotions.

[1189] Specifically, the server instructs the emotion engine AI to "recognize the user's emotions." The emotion engine AI then determines the user's emotions from their tone of voice and input content.

[1190] Input: User voice tone, input content, emotion recognition instructions

[1191] Output: Emotion recognition result

[1192] Step 6:

[1193] The server consolidates the information from each AI and provides it to the user.

[1194] Specifically, the server integrates the latest news from the news AI, recommended information from the user AI, and emotion recognition results from the emotion engine AI, and provides them to the user.

[1195] Input: Latest news information, recommendation information, emotion recognition results

[1196] Output: Synthesized information (e.g., Today's top news story is that economic growth exceeded expectations. A movie recommendation for you is "Inception." You seem a little tired today. I recommend some relaxing music.)

[1197] (Application example 2)

[1198] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1199] Conventional information provision systems do not provide information according to the user's emotions, resulting in a limited user experience. Furthermore, when multiple AIs operate independently, it is difficult to centrally manage information, making it difficult to provide consistent information to users. Furthermore, in applications installed on smart devices and robots, the division of roles between AIs is unclear, making it difficult to provide information efficiently.

[1200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1201] In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with the humanoid AI not only through text but also through other means, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions using an emotion engine and engaging in dialogue accordingly, means for centrally managing information generated by each AI and providing it to the user in an appropriate format, and means including an application installed on a smartphone, smart glasses, head-mounted display, or robot. This makes it possible to provide information according to the user's emotions, clarifies the division of roles between each AI, and realizes centralized information management and consistent information provision.

[1202] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[1203] "Latest news" refers to the latest data and news about current situations and events.

[1204] "Personal information" refers to information about an individual, including the user's preferences and behavioral history.

[1205] "Humanoid AI" is artificial intelligence that mimics the shape of a human and communicates with users through dialogue and actions.

[1206] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1207] "Security information" is information used to ensure the safety of systems and data.

[1208] "Platform information" is information about the platform on which the system operates.

[1209] An "emotion engine" is a technology that recognizes a user's emotions and generates a response accordingly.

[1210] "Centralized management" refers to the integrated management of multiple pieces of information and data in a single system.

[1211] A "smartphone" is a type of mobile phone, a device that can connect to the Internet and use applications.

[1212] "Smart glasses" are a wearable device in the form of glasses that can display information and allow interaction.

[1213] A "head-mounted display" is a display device worn on the head that provides visual information.

[1214] A "robot" is a mechanical device that operates autonomously or remotely to perform a specific task.

[1215] A system for implementing this invention comprises means for taking in specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing the user's emotions using an emotion engine and engaging in dialogue accordingly, means for centrally managing information generated by each AI and providing it to the user in an appropriate form, and means including an application installed on a smartphone, smart glasses, head-mounted display, or robot.

[1216] The server first collects specialized knowledge, the latest information, and personal information and stores them in a database. This is done using a news API and algorithms that analyze user behavior history. Each AI is then assigned a specific role. For example, some AIs provide the latest news, while others provide recommendations based on the user's personal information. Additionally, an AI with an emotion engine recognizes the user's emotions and interacts accordingly.

[1217] The device (smartphone, smart glasses, head-mounted display, robot) receives the information provided by the server and provides it to the user in an appropriate format, either visually or audibly. For example, smart glasses display news headlines in the user's field of vision, and a head-mounted display enables interaction in a virtual space.

[1218] As an example, if the user is wearing smart glasses, the following prompts can be used to provide information:

[1219] Example prompt sentence:

[1220] "Good morning. Today's news headlines are: 1. Economic News: Stocks are rising. 2. Weather Forecast: It's sunny today. Also, based on your recent search history, we recommend the following movie: 'Latest Action Movies.' Additionally, you look a little tired. Would you like some relaxing music?"

[1221] In this way, users can receive information tailored to their emotions and situations in real time, improving the user experience and providing information efficiently.

[1222] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1223] Step 1:

[1224] The server collects specialized knowledge, the latest information, and personal information. As input, it receives news API and user behavior history data, and stores this in a database. As data processing, it analyzes the latest information obtained from the news API and extracts personal information based on the user's behavior history. As output, organized specialized knowledge, the latest information, and personal information are stored in a database.

[1225] Step 2:

[1226] The server assigns a specific role to each AI. It uses specialized knowledge, the latest information, and personal information stored in the database as input. It runs an algorithm to provide specific information to each AI as data calculation. The output is set to a role such as news provision AI, recommendation information provision AI, or emotion recognition AI.

[1227] Step 3:

[1228] The server uses an emotion engine to recognize the user's emotions. As input, it receives the user's voice data and facial expression data. As data processing, the emotion engine analyzes this data and identifies the user's emotional state. As output, the user's emotional state is recognized.

[1229] Step 4:

[1230] The server centrally manages the information generated by each AI and provides it to the user in an appropriate format. It receives the information generated by each AI as input. It integrates the information as a data calculation and converts it into a format that is optimally presented to the user. The integrated information is generated as output.

[1231] Step 5:

[1232] The device (smartphone, smart glasses, head-mounted display, robot) receives the information provided by the server and provides it to the user in an appropriate format, either visually or audibly. As input, it receives the integrated information sent from the server. In concrete terms, the smart glasses display news headlines in the user's field of vision, and the head-mounted display enables interaction in the virtual space. As output, information is provided to the user.

[1233] Step 6:

[1234] The user receives the information provided through the terminal and provides feedback as needed. As input, the information provided by the terminal is received. As a specific operation, the user can check the display on the smart glasses and request additional information by voice command. As output, the user's feedback is sent to the server.

[1235] Step 7:

[1236] The server receives user feedback to further personalize the system. As input, it receives user feedback data. As data processing, it analyzes the feedback and adjusts system settings based on the user's preferences and emotional state. As output, it enhances personalized information provision.

[1237] Example 3

[1238] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1239] Conventional systems had limited means of incorporating specialized knowledge, the latest information, and personal information, and interactions with users were limited to text-based. It was also difficult for each AI to operate independently, making it difficult to maintain consistency of information. Furthermore, the system was unable to recognize the user's emotions and provide information based on them, limiting the user experience.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1241] In this invention, the server includes means for taking in specialized knowledge, the latest information, and personal information, means for conversing with the humanoid AI not only through text, means for assigning roles to each AI, means for providing information together with security and platform information, means for receiving user input data and distributing it to each AI, means for each AI to process data using its own algorithm, means for an emotion-recognition AI to recognize the user's emotions and share the information with other AIs, means for other AIs to generate information based on the shared emotional information, and means for integrating information from each AI and providing it to the user. This allows each AI to operate independently, but by sharing information, it is possible to provide consistent information as a whole, realizing dialogue and information provision based on the user's emotions.

[1242] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1243] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[1244] "Personal Information" means data or information about an individual that reflects that individual's characteristics or preferences.

[1245] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[1246] A "role" is the specific function or task each AI is responsible for.

[1247] "Security" refers to measures to protect systems and data from unauthorized access or attack.

[1248] A "platform" is the hardware and software environment that forms the basis for a system to operate.

[1249] "Input data" is information or instructions provided by a user to a system.

[1250] An "algorithm" is a procedure or computational method for solving a particular problem.

[1251] "Emotion recognition AI" is artificial intelligence that has the ability to analyze and recognize the user's emotions.

[1252] "Information sharing" means that multiple AIs exchange data and information with each other.

[1253] "Information generation" means that AI creates new data or responses based on input data or shared information.

[1254] "Information integration" means compiling data obtained from multiple AIs into a single, consistent piece of information.

[1255] This invention is a system in which multiple AIs operate independently, generating and providing information according to their respective roles. Each AI has its own learning data and algorithms, and processes user input data. In addition, a communication protocol is set up so that each AI can share information with other AIs and provide consistent information as a whole.

[1256] Hardware and software used

[1257] server

[1258] The server is responsible for managing the operation of each AI, receiving input data from the user, and distributing it to each AI. The server is a computer equipped with a high-performance processor and large memory, and uses HTTP or WebSocket as a communication protocol.

[1259] Terminal

[1260] A terminal is a device with an interface through which a user provides input data. Terminals include smartphones, tablets, and PCs. The terminal sends the user's input to a server and displays the server's response to the user.

[1261] AI model

[1262] Each AI is a generative AI model with a specific role. For example, emotion recognition AI analyzes the user's emotions, and dialogue generation AI generates appropriate responses. Each AI operates using its own algorithms and training data.

[1263] Data processing and calculation

[1264] Processing input data

[1265] When a user provides input data to the device, the device sends the data to the server, which distributes the input data to each AI, and each AI processes the data using its own algorithm.

[1266] emotion recognition

[1267] Emotion recognition AI analyzes emotions from user input data. For example, if a user inputs "I'm happy today," the emotion recognition AI will recognize the emotion of "joy."

[1268] Information sharing

[1269] Emotional information recognized by emotion-recognition AI is shared with other AIs using communication protocols such as HTTP and WebSocket.

[1270] information generation

[1271] Other AIs generate information based on shared emotional information. For example, a dialogue generation AI generates a response based on the emotion of "joy," such as, "That's great! Did something special happen today?"

[1272] Information Integration

[1273] The server integrates the information from each AI and sends it as a single response to the device, which then displays the integrated information to the user.

[1274] Specific examples

[1275] Consider the case where a user inputs "I'm happy today." In this case, the emotion-recognition AI recognizes the user's emotion as "joy" and shares this information with other AIs. Based on this information, the other AIs can provide information and dialogue tailored to the user's joy.

[1276] Example prompts to input to a generative AI model:

[1277] If a user types "I'm happy today," how will the emotion-aware AI respond and how will it share that information with other AIs?

[1278] In this way, while each AI operates independently, it is possible to provide consistent information as a whole by sharing information, and dialogue and information provision based on the user's emotions can be realized. The flow of the identification process in Example 3 will be described with reference to Figure 21.

[1279] Step 1:

[1280] The user types into the terminal.

[1281] The user inputs text or voice data through the device interface. For example, the user inputs "I'm happy today." The input data is sent to the device in text format.

[1282] Step 2:

[1283] The terminal sends the input data to the server.

[1284] The device converts the user's input data into JSON format and sends it to the server, which receives the data and proceeds to the next processing step.

[1285] Step 3:

[1286] The server distributes input data to each AI.

[1287] The server analyzes the input data it receives and distributes it to the appropriate AI, such as emotion recognition AI, dialogue generation AI, or information provision AI. Each AI prepares to process the data using its own algorithm.

[1288] Step 4:

[1289] Each AI processes data using its own algorithm.

[1290] Emotion recognition AI analyzes user input data and recognizes emotions. For example, it recognizes the emotion of "joy" from the input "I'm happy today." Dialogue generation AI generates appropriate responses based on user input.

[1291] Step 5:

[1292] Emotion-recognition AI recognizes the user's emotions and shares information with other AIs.

[1293] Emotion recognition AI shares the recognized emotional information with other AIs. Communication protocols such as HTTP and WebSocket are used for sharing. For example, emotional information of "joy" is sent to a dialogue generation AI.

[1294] Step 6:

[1295] Other AIs generate information based on shared emotional information.

[1296] The dialogue generation AI generates a response to the user based on the shared emotional information of "joy." For example, it generates a response like, "That's great! Did anything special happen today?"

[1297] Step 7:

[1298] The server consolidates the information from each AI and sends it to the terminal.

[1299] The server integrates the information from each AI and sends it to the terminal as a single response. The integration includes processing to maintain data consistency.

[1300] Step 8:

[1301] The terminal displays the integrated information to the user.

[1302] The terminal displays the integrated information received from the server to the user, allowing the user to receive consistent information. For example, a message such as "That's great! Is there anything special happening today?"

[1303] (Application example 3)

[1304] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1305] In conventional AI systems, it was difficult for each AI to operate independently, and it was also difficult to provide consistent information based on the user's emotions. Furthermore, there was a problem of a lack of smart device applications to support customer interaction and service provision in physical stores.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information; means for conversing with humanoid AIs not only through text but also through other means; means for assigning roles to each AI; means for providing information together with security and platform information; means for performing emotion recognition; means for generating and providing emotion-based information; means for setting a communication protocol for sharing information between AIs; and means for including an application installed on the smart device. This allows each AI to operate independently, enabling consistent information provision based on the user's emotions. Furthermore, smart device applications can be realized to support customer interactions and service provision in brick-and-mortar stores.

[1307] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1308] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[1309] "Personal information" refers to specific data and attribute information about an individual.

[1310] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[1311] "Means for conversation" refers to techniques and methods for engaging in natural dialogue with the user.

[1312] "Roles" are ways to assign specific functions and responsibilities to each AI.

[1313] "Security" refers to technologies and measures for protecting information and systems.

[1314] A "platform" is the underlying environment for running applications and services.

[1315] "Emotion recognition" is a technology that detects and identifies a user's emotions.

[1316] "Means of generating and providing information" refers to the method of creating new information based on data and providing it to users.

[1317] A "communication protocol" is a set of rules and procedures for exchanging information between different systems.

[1318] A "smart device" is an electronic device that has Internet connectivity and can run a variety of applications.

[1319] An "application" is software that provides specific functions or services.

[1320] A system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for recognizing emotions, means for generating and providing information based on emotions, means for setting communication protocols for sharing information between AIs, and means including an application to be installed on a smart device.

[1321] System program generation

[1322] The server runs the following programs. First, it builds a database to collect specialized knowledge, the latest information, and personal information, and updates this information in real time. Next, it prepares a humanoid AI to interact with the user, and uses natural language processing technology to realize text-based conversations. Furthermore, each AI is assigned a specific role and is designed to operate independently.

[1323] Processing Description

[1324] The server uses image recognition technology to perform emotion recognition. Specifically, it receives a user's facial image as input and identifies the emotion. For this process, it uses, for example, Microsoft® Azure® Face API. Services such as Amazon Personalize are used to generate and provide appropriate information and products based on the recognized emotion. This information is shared among the AIs, allowing for consistent information provision overall.

[1325] Specific examples

[1326] For example, consider a system in which store clerks wear smart glasses to support conversations with customers in a physical store. If a customer is happy, that information is shared with other AIs, and the services and product suggestions provided by the clerk will be tailored to their happiness. In this way, it will be possible to provide consistent service based on the customer's emotions.

[1327] Prompt Sentence Examples

[1328] Below are some example prompts to input to a generative AI model:

[1329] Develop an application that takes a customer's image as input, recognizes their emotion, and suggests appropriate products. Use Microsoft (registered trademark) Azure (registered trademark) Face API for emotion recognition and Amazon Personalize for product suggestions. Use a RESTful API to share the recognized emotion and suggested product information with other AIs.

[1330] This prompt will help the generative AI model suggest specific code and designs.

[1331] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1332] Step 1:

[1333] The server retrieves specialized knowledge, the latest information, and personal information from the database. As input, it receives information from various databases, integrates it, and stores it. As data processing, it normalizes and filters the information, and generates an integrated database as output.

[1334] Step 2:

[1335] The server receives the user's facial image as input and performs emotion recognition. It receives the user's facial image as input and uses image recognition technology to identify emotions. Specifically, it uses Microsoft® Azure® Face API to analyze facial feature points as data calculations. It generates recognized emotion data as output.

[1336] Step 3:

[1337] The server then recommends appropriate products based on the recognized emotion data. It receives emotion data as input and uses services such as Amazon Personalize to make product suggestions. It processes the data by matching the emotion data with the product data and generates a list of suggested products as output.

[1338] Step 4:

[1339] The server shares the generated emotion data and product recommendation data with other AIs. It receives emotion data and product recommendation data as input and sends them to other AIs using a communication protocol. Specifically, it uses a RESTful API to serialize and send the data as data calculations. As output, it generates data that is shared with other AIs.

[1340] Step 5:

[1341] The terminal provides information to the user through an application installed on the smart device. As input, it receives emotion data and product recommendation data from the server and displays them through the user interface. Specific operations include displaying the information on the smart glasses display and notifying the user through the voice assistant. As output, it generates the information provided to the user.

[1342] Step 6:

[1343] The user selects a product based on the information provided through the smart device. As input, the system receives information from the smart device and makes a selection. As a specific operation, the selection is made using the touchpad on the smart glasses or voice commands. As output, the selected product data is generated.

[1344] Step 7:

[1345] The server receives the user's selection data and performs the purchase procedure. As input, it receives the user's selection data and processes the data for the purchase procedure. Specifically, it processes the payment using an electronic payment service. As output, it generates a notification of purchase completion.

[1346] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires voice indicating user input with respect to the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1347] The data generation model 58 is a so-called generative AI (Artificial Intelligence). Examples of the data generation model 58 include generative AIs such as ChatGPT (Internet search <URL: https: / / OpenAI (registered trademark).com / blog / chatgpt>). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is input thereto. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summary.

[1348] As another example of the generative AI, Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1349] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1350] [Third embodiment]

[1351] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1352] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1353] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1354] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1355] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1356] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1357] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1358] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1359] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1360] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1361] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1362] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1363] "Example 1"

[1364] In one embodiment of the present invention, information is collected from various databases and internet sources as a means of incorporating specialized knowledge, the latest information, and personal information, and this information is used as learning data for the AI. Furthermore, as a means of conversing with a humanoid AI, not just through text, speech recognition technology and natural language processing technology are used to enable dialogue with the user.

[1365] "Example 2"

[1366] As a means of assigning roles to each AI, for example, some AIs could provide the latest news, while others could provide recommended information based on the user's personal information, etc. In addition, as a means of providing information together with security and platform information, a system will be built to centrally manage the information generated by each AI and provide it to users in an appropriate format.

[1367] "Example 3"

[1368] Each AI operates independently, and as a means of generating and providing information according to its role, each AI has its own learning data and algorithms, and generates and provides information independently.In addition, each AI shares information with other AIs, and as a means of providing consistent information as a whole, a communication protocol is set up for sharing information between AIs, realizing consistent information provision as a whole.

[1369] The processing flow of each embodiment will be described below.

[1370] "Example 1"

[1371] Step 1: Gather specialized knowledge, current information, and personal information from various databases and online sources.

[1372] Step 2: Use the collected information as learning data for the AI ​​to update its knowledge.

[1373] Step 3: Use voice recognition and natural language processing technologies to enable dialogue with the user.

[1374] "Example 2"

[1375] Step 1: Assign specific roles to each AI. For example, some AIs may be tasked with updating news.

[1376] Another AI provides personalized recommendations based on the user's personal information.

[1377] Step 2: Build a system that centrally manages the information generated by each AI and provides it to users in an appropriate format.

[1378] "Example 3"

[1379] Step 1: Each AI has its own training data and algorithms, and each generates and provides information independently.

[1380] Step 2: Establish a communication protocol for sharing information between AIs to ensure consistent information provision across the board.

[1381] Example 1

[1382] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1383] Conventional information provision systems have had difficulty efficiently collecting specialized knowledge, the latest information, and personal information, and providing consistent information to users. Even in dialogue systems using voice recognition and natural language processing technologies, it was difficult to generate appropriate responses based on user input, limiting the user experience. Furthermore, each AI operates independently and cannot share information, making it difficult to provide consistent information overall.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1385] In this invention, the server includes a means for collecting specialized knowledge, the latest information, and personal information, a means for conversing with a humanoid AI not only through text, a means for converting a user's voice into text using speech recognition technology, a means for analyzing collected information using natural language processing technology and using it as training data for a generative AI model, a means for generating appropriate responses based on user input using a generative AI model, a means for converting the generated text responses into speech using speech synthesis technology and playing them back to the user, a means for assigning roles to each AI, and a means for providing information together with security and platform information. This enables the efficient collection of specialized knowledge, the latest information, and personal information and the provision of consistent information to users. Furthermore, even in dialogue systems using speech recognition technology and natural language processing technology, appropriate responses can be generated based on user input, improving the user experience. Furthermore, each AI can share information, providing consistent information overall.

[1386] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1387] "Latest news" is the latest data and news about current situations and events.

[1388] "Personal Information" means data or information relating to an individual.

[1389] "Humanoid AI" is artificial intelligence that mimics human-like dialogue and behavior.

[1390] "Speech recognition technology" is a technology that converts speech into text.

[1391] "Natural language processing technology" is a technology for analyzing and understanding human language.

[1392] A "generative AI model" is an artificial intelligence model that generates new data or responses based on input data.

[1393] "Speech synthesis technology" is a technology that converts text into speech.

[1394] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1395] "Security" refers to the techniques and methods used to protect information and systems.

[1396] A "platform" is a system that serves as the foundation for providing software and services.

[1397] The present invention is a system for efficiently collecting specialized knowledge, up-to-date information, and personal information, and providing consistent information to users. A specific embodiment of this system will be described below.

[1398] Information gathering

[1399] The server collects specialized knowledge, up-to-date information, and personal information from various databases and internet sources. This information is collected using web crawlers and APIs. For example, it uses a news site's API to retrieve the latest news articles. The server periodically checks these sources to see if there is new information.

[1400] Data processing and learning

[1401] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The extracted data is saved as training data for the generative AI model.

[1402] Recognizing user voice

[1403] When a user speaks into the device, the device captures the audio using its built-in microphone. The device converts the captured audio into text using the Google® Cloud Speech-to-Text API. The converted text is then sent to the server.

[1404] Text analysis and response generation

[1405] The server analyzes the received text. For analysis, it uses OpenAI's GPT-3 (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response.

[1406] Audio Output

[1407] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is then played back to the user through the device's speaker. For example, if the generated response is "The latest news is...", that content is conveyed to the user as speech.

[1408] Specific examples

[1409] Example 1: Getting and interacting with news

[1410] When a user says, "Tell me the latest news," the device converts the speech into text and sends it to the server. The server retrieves the latest news from the news site's API and uses a generative AI model to generate a response such as, "The latest news is ____." The device then converts this response into audio and plays it back to the user.

[1411] Example 2: Use of personal information

[1412] When a user says, "What's on my agenda for today?", the device converts the speech into text and sends it to the server. The server retrieves the user's calendar information and uses a generative AI model to generate a response such as "Today's agenda is ____." The device then converts this response into audio and plays it back to the user.

[1413] Prompt Sentence Examples

[1414] "Tell me the latest news"

[1415] "What are my plans for today?"

[1416] "Tell me the weather forecast."

[1417] The system is designed to allow users to obtain information through natural interaction.

[1418] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1419] Step 1: Gather information

[1420] The server collects specialized knowledge, the latest information, and personal information from various databases and information sources on the Internet. Specifically, it acquires information using a web crawler or API. For example, it acquires the latest news articles using a news site's API. The input is the news site's API endpoint, and the output is the retrieved news article data.

[1421] Step 2: Data processing and training

[1422] The server analyzes the collected information using natural language processing (NLP) techniques. Specifically, it uses the Python libraries NLTK and spaCy to tokenize the text data and extract important keywords and context. The input is the collected news article data, and the output is the extracted keywords and context information. This data is saved as training data for the generative AI model.

[1423] Step 3: Recognizing user speech

[1424] When a user speaks into the device, the device captures the voice using the built-in microphone. The device converts the captured voice into text using the Google® Cloud Speech-to-Text API. The input is the user's voice data, and the output is the converted text data. This text data is sent to the server.

[1425] Step 4: Text analysis and response generation

[1426] The server analyzes the received text. For the analysis, it uses GPT-3 (registered trademark) from OpenAI (registered trademark), a generative AI model. The server generates an appropriate response based on the user's input. For example, if a user types "Tell me the latest news," the server searches for the latest news articles, summarizes their contents, and generates a response. The input is the text data sent by the user, and the output is the generated response text.

[1427] Step 5: Audio Output

[1428] The device converts the text response received from the server into speech using a speech synthesis service such as Amazon Polly. The converted speech is played back to the user through the device's speaker. For example, if the generated response is "The latest news is ____," that content is conveyed to the user as speech. The input is the response text received from the server, and the output is the converted speech data.

[1429] (Application example 1)

[1430] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1431] Autonomous vehicles are required to collect traffic and weather information in real time and suggest optimal routes and rest stops through dialogue with the user. However, current systems collect and provide information in a fragmented manner, making it difficult to meet the user's personalized needs. In addition, dialogue systems using voice recognition technology and natural language processing technology are not fully integrated, limiting the user experience. This poses a challenge as the convenience and safety of autonomous vehicles are not fully realized.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1433] In this invention, the server includes means for acquiring specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs not only through text but also through other means, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with users using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and proposing optimal routes and rest stops. This enables autonomous vehicles to suggest personalized, optimal routes and rest stops through real-time information collection and dialogue with users.

[1434] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[1435] "Latest news" refers to the latest data and news about current situations and events.

[1436] "Personal information" is information about an individual, including personal preferences and history.

[1437] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[1438] "Means of conversation" refers to a method of interacting with the user using voice recognition technology or natural language processing technology.

[1439] "Roles" refers to the method of assigning specific functions and responsibilities to each AI.

[1440] "Security" refers to the technologies and measures used to protect information and systems and prevent unauthorized access and data leaks.

[1441] A "platform" refers to the system or environment that forms the foundation of software and hardware.

[1442] "Means of collecting traffic and weather information in real time" refers to a method of instantly obtaining data on current traffic conditions and weather.

[1443] "Speech recognition technology" refers to technology that converts speech into text.

[1444] "Natural language processing technology" refers to artificial intelligence technology for understanding and generating human language.

[1445] "Means for learning about the user's past driving history and preferences" refers to a method of collecting data on the user's past behavior and preferences and learning based on that data.

[1446] "Means for suggesting optimal routes and rest points" refers to methods that suggest the most suitable routes and rest points for users based on collected information.

[1447] The system for implementing this invention includes means for incorporating specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for collecting traffic and weather information in real time, means for interacting with the user using voice recognition technology and natural language processing technology to set destinations and change routes, and means for learning the user's past driving history and preferences and suggesting optimal routes and rest stops.

[1448] The server collects real-time traffic and weather information from various databases and internet sources using software (e.g., the requests library) that uses APIs to retrieve data. The collected information is then analyzed within the server and provided to the user.

[1449] The device converts the user's speech into text using speech recognition technology (e.g., the speech_recognition library). The converted text is then analyzed using natural language processing technology (e.g., the transformers library) to understand the user's intent, allowing for dialogue with the user.

[1450] Users interact with the system through voice. For example, by asking, "Where is the next rest stop?", the system will suggest the best rest stop based on real-time traffic information and the user's past driving history. The suggested information is conveyed to the user using voice synthesis technology (e.g., the pyttsx3 library).

[1451] As a concrete example, if a user asks, "Where is the next rest stop?" the system responds as follows:

[1452] User: Where is the next rest stop?

[1453] System: Based on current traffic conditions and past driving history, the system will suggest the next rest stop, "XX Service Area." This area has a restaurant and restrooms, making it an ideal place to take a break.

[1454] In this way, intelligent navigation systems in autonomous vehicles can provide users with a convenient and safe driving experience.

[1455] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1456] Step 1:

[1457] The server collects traffic and weather information in real time from various databases and information sources on the Internet. Specifically, it uses the requests library to obtain data using an API. An API endpoint is specified as input, and traffic and weather information in JSON format is obtained as output. This allows the latest traffic and weather information to be aggregated on the server.

[1458] Step 2:

[1459] The server analyzes the collected traffic and weather information. Specifically, it analyzes the acquired JSON data and extracts the necessary information (e.g., locations of traffic jams, weather changes). It receives JSON data as input and obtains the analyzed information as output. This prepares useful information to provide to users.

[1460] Step 3:

[1461] The device uses voice recognition technology to convert the user's voice into text. Specifically, it uses the speech_recognition library to convert the voice data input from the microphone into text data. It receives voice data as input and obtains text data as output. This allows the user's speech content to be obtained in text format.

[1462] Step 4:

[1463] The device uses natural language processing technology to analyze the user's text data and understand their intent. Specifically, it uses the transformers library to analyze the text data using a generative AI model. It receives text data as input and obtains data indicating the user's intent as output. This allows the device to understand the user's questions and requests.

[1464] Step 5:

[1465] The server generates an appropriate response based on the user's intent. Specifically, it suggests optimal routes and rest stops based on analyzed traffic and weather information, the user's past driving history, and preferences. It receives the user's intent data and analyzed information as input, and obtains response data as output. This allows it to generate appropriate suggestions for the user.

[1466] Step 6:

[1467] The device uses speech synthesis technology to generate response data and communicates it to the user. Specifically, it uses the pyttsx3 library to convert text-format response data into speech data. It receives response data as input and obtains speech data as output. This provides a speech response to the user.

[1468] Step 7:

[1469] The user receives the voice response provided by the system and decides on the next course of action. Specifically, the user continues driving according to the route and rest points suggested by the system. The system receives the voice response as input and decides on the next course of action as output. This allows the user to reach their destination safely and efficiently.

[1470] Example 2

[1471] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1472] Conventional information provision systems have difficulty efficiently incorporating specialized knowledge, the latest information, and personal information, making it difficult to provide consistent information to users. Furthermore, when each AI operates independently, centralized information management and security assurance become issues. Furthermore, there was a need for a system that could integrate multiple processes, such as news information collection and summarization, personal information input and storage, and recommended information generation.

[1473] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1474] In this invention, the server includes a means for taking in specialized knowledge, the latest information, and personal information, a means for conversing with the humanoid AI in addition to text, a means for assigning roles to each AI, a means for providing information together with security and platform information, a means for collecting and summarizing news information, a means for inputting and saving personal information, a means for generating recommended information, a means for centrally managing information, and a means for providing information. This makes it possible to provide consistent information to users, and achieves centralized information management and security while each AI operates independently.

[1475] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1476] "Latest news" is the latest data and news about current situations and events.

[1477] "Personal information" refers to information about an individual user, such as data about that user, their interests, and past behavioral history.

[1478] "Humanoid AI" is artificial intelligence that can converse and behave like a human.

[1479] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1480] "Security" refers to the technologies and methods used to protect information and systems from unauthorized access and data leaks.

[1481] A "platform" is a system or environment that serves as the foundation for providing software and services.

[1482] "Means for collecting and summarizing news information" refers to a method for obtaining news data and concisely summarizing its contents.

[1483] "Means for entering and storing personal information" refers to the method by which a user enters their own information and stores it in a database.

[1484] The "means for generating recommendation information" is a method for generating appropriate suggestions and advice based on the user's personal information.

[1485] A "means for centrally managing information" is a method for integrating and managing multiple pieces of information.

[1486] "Means of providing information" refers to the method of providing the necessary information to users in an appropriate format.

[1487] This invention is a system that assigns specific roles to multiple AIs, centrally manages the information generated by each AI, and provides it to users in an appropriate format. A specific embodiment of this system is described below.

[1488] Hardware and software used

[1489] Hardware: Servers, terminals (user devices)

[1490] Software: Generative AI models (e.g., GPT-4®), database management systems (e.g., MySQL®), security software (e.g., SSL / TLS)

[1491] News information gathering and summarization

[1492] The server uses a news API (e.g., NewsAPI) to collect news information. The server periodically sends requests to the news API to obtain the latest news data. The obtained news data is input into a generative AI model, and a prompt is sent to generate a summary. An example of a specific prompt is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The server stores the summarized news information in a database.

[1493] Entering and saving personal information

[1494] The user uses the device to input their personal information (e.g., interests, past browsing history), and the device sends the input personal information to the server, which stores the received personal information in a database.

[1495] Generate recommendations

[1496] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. An example of a specific prompt is, "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The server stores the generated recommendations in a database.

[1497] Centralized information management

[1498] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The server uses security software (e.g., SSL / TLS) to ensure the safety of data. Specifically, access to the database is encrypted with SSL / TLS to prevent unauthorized access.

[1499] Providing information

[1500] A user uses a device to request news information or recommended information. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The device then displays the received information to the user. Specifically, it displays summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[1501] In this way, a system will be created in which each AI has a specific role, and the information generated will be centrally managed and provided to users.

[1502] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1503] Step 1: Gather and summarize news information

[1504] The server sends a request to the news API to obtain the latest news data. The input is news data from the news API, and the output is the obtained news data. Specifically, the server sends a request to the news API to "obtain the latest news." The obtained news data is input into the generative AI model, and a prompt sentence is sent to generate a summary. An example of a prompt sentence is "Please summarize the latest technology news." The generative AI model summarizes the news data and extracts key points. The output is summarized news information. The server stores the summarized news information in a database.

[1505] Step 2: Enter and save your personal information

[1506] The user uses the device to input their own personal information (e.g., interests, past browsing history). The input is the personal information the user enters into the device, and the output is the personal information sent to the server. The device sends the entered personal information to the server. The server stores the received personal information in a database. Specifically, the user enters "Interests: Technology, Past Browsing History: AI-related news" into the device, and the device sends this to the server.

[1507] Step 3: Generate recommendations

[1508] The server sends a prompt to the generative AI model to generate recommendations based on the stored personal information. The input is the personal information stored in the database, and the output is the prompt sent to the generative AI model. An example of a prompt is "Please recommend some movies based on the user's interests." The generative AI model analyzes the personal information and generates recommendations appropriate for the user. The output is the generated recommendations. The server stores the generated recommendations in a database.

[1509] Step 4: Centralize your information

[1510] The server uses a database management system (e.g., MySQL (registered trademark)) to centrally manage news information and recommendation information. The input is news information and recommendation information, and the output is centrally managed data. The server uses security software (e.g., SSL / TLS) to ensure the security of the data. Specifically, it encrypts access to the database with SSL / TLS to prevent unauthorized access.

[1511] Step 5: Provide information

[1512] A user uses a device to request news information or recommended information. The input is the user's request, and the output is the request sent to the server. The device sends the request to the server. The server retrieves the requested information from a database and sends it to the device. The output is the information provided to the user. The device displays the received information to the user. A specific operation is to display summary information such as "Latest technology news: AI technology is advancing at an accelerating pace."

[1513] (Application example 2)

[1514] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1515] Conventional information provision systems have difficulty in providing consistent information to users, and are particularly inadequate in providing personalized content. Furthermore, there is a lack of a way to centrally manage information generated by multiple AIs and provide it to users in an appropriate format, which results in a lack of an improved user experience.

[1516] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on the user's interests, and means for analyzing news data using natural language processing technology and classifying it by category. This enables consistent information provision to users, realizing the provision of personalized content.

[1517] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1518] "Latest news" is the latest data and news about current situations and events.

[1519] "Personal Information" means information about an individual user's personal data, interests, and preferences.

[1520] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[1521] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1522] "Security information" means information relating to the safety of systems and data.

[1523] "Platform information" refers to information relating to the operation and management of the entire system.

[1524] "Centralized management" is a method of integrating and managing multiple pieces of information in one system or location.

[1525] "Recommended content" is information or media selected based on a user's interests and preferences.

[1526] "Natural language processing technology" is a technology that uses computers to understand and analyze human language.

[1527] "News Data" means information obtained from news organizations or news sources.

[1528] A "categorization method" is a method for separating information into specific categories.

[1529] A system for carrying out this invention is configured as follows: The server includes means for importing specialized knowledge, the latest information, and personal information, means for conversing with humanoid AI in addition to text, means for assigning roles to each AI, means for providing information together with security and platform information, means for centrally managing information generated by multiple AIs and providing it to users in an appropriate format, means for generating recommended content based on user interests, and means for analyzing news data using natural language processing technology and classifying it into categories.

[1530] Hardware and software used

[1531] Hardware: Smartphones, smart glasses

[1532] Software: Python, TensorFlow (registered trademark), Flask, SQLite, REST API

[1533] Data processing and calculation flow

[1534] 1. Data Collection:

[1535] The server retrieves the latest news data from a news API (e.g., NewsAPI) and stores user personal information (interests, browsing history, etc.) in a SQLite database.

[1536] 2. Data Processing:

[1537] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. Based on the user's personal information, a model is trained using TensorFlow (registered trademark) to generate recommended content.

[1538] 3. Information generation:

[1539] The server uses News AI to generate the latest news information, Recommendation AI to generate recommended content based on the user's interests, and Security AI to generate security information.

[1540] 4. Information Integration:

[1541] The server uses Flask to build a REST API that centrally manages the information generated by each AI, consolidates the information from each AI, and provides it to users in an appropriate format.

[1542] 5. User Interface:

[1543] The device (smartphone or smart glasses) displays the integrated information.

[1544] Examples and prompts

[1545] Specific examples

[1546] News AI: "Today's latest news is about new discoveries in the field of technology."

[1547] Recommendation AI: "We recommend the following content based on your recent topics of interest."

[1548] Security AI: "The latest security information is to beware of phishing scams."

[1549] Prompt Sentence Examples

[1550] Prompt for News AI: "Get the latest technology news."

[1551] Prompt for recommendation AI: "Recommend content based on the user's interests."

[1552] Prompt security AI: "Please provide us with the latest security information."

[1553] In this way, the server will enable each AI to have a specific role and provide personalized information to users, thereby realizing a richer content distribution service.

[1554] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1555] Step 1:

[1556] The server retrieves the latest news data from the news API. The input is the news API endpoint, and the output is the retrieved news data. Specifically, the server sends an HTTP request and receives news data in JSON format.

[1557] Step 2:

[1558] The server stores user personal information (interests, browsing history, etc.) in an SQLite database. The input is the user's personal information, and the output is the information stored in the database. Specifically, the server receives user information and inserts it into the database using an SQL query.

[1559] Step 3:

[1560] The server analyzes news data using natural language processing (NLP) technology and classifies it into categories. The input is news data, and the output is news data classified by category. Specifically, the server analyzes the news data using an NLP library (e.g., spaCy) and assigns category tags.

[1561] Step 4:

[1562] The server uses TensorFlow® to train a model for generating recommended content based on the user's personal information. The input is the user's personal information and news data, and the output is the trained recommendation model. Specifically, the server preprocesses the data and trains the model using TensorFlow®.

[1563] Step 5:

[1564] The server uses news AI to generate the latest news information. The input is news data categorized by category, and the output is the generated news information. Specifically, the server analyzes the news data and selects news appropriate for the user.

[1565] Step 6:

[1566] The server uses recommendation AI to generate recommended content based on the user's interests. The input is a trained recommendation model and the user's personal information, and the output is recommended content. Specifically, the server uses the model to recommend the most suitable content to the user.

[1567] Step 7:

[1568] The server generates security information using security AI. The input is security-related data, and the output is the generated security information. Specifically, the server analyzes the security data and provides warnings and advice to the user.

[1569] Step 8:

[1570] The server uses Flask to build a REST API that centrally manages the information generated by each AI. The input is the information generated by each AI, and the output is the integrated information. Specifically, the server uses the Flask framework to set up an API endpoint and integrate the information.

[1571] Step 9:

[1572] The device (smartphone or smart glasses) displays the integrated information. The input is the integrated information from the server, and the output is the information displayed on the user interface. Specifically, the device retrieves information from the API and displays it visually to the user.

[1573] Example 3

[1574] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1575] In conventional information provision systems, it was difficult for each AI to operate independently, and it was also difficult to provide the information generated by each AI consistently. Furthermore, the process from when a user sends a request to when they receive the information is complicated, making it difficult to provide information efficiently.

[1576] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1577] In this invention, the server includes means for a user to send a request from a terminal, means for the terminal to transfer the request to the server, means for the server to analyze the request and select an appropriate generative AI model, means for the server to input a prompt sentence into the generative AI model selected by the server and generate information, means for the server to share the generated information with other generative AI models, means for the server to return the final information to the terminal, and means for the terminal to display the information to the user. This makes it possible to provide consistent information as a whole even though each generative AI model operates independently.

[1578] "Specialist knowledge" refers to information or knowledge that represents a high level of understanding or skill in a particular field.

[1579] "Latest information" refers to the most current data, news, and events.

[1580] "Personal information" refers to data or information relating to an individual, including personal characteristics, preferences, and history.

[1581] "Humanoid AI" refers to artificial intelligence that mimics human-like dialogue and behavior.

[1582] A "role" is a responsibility or function assigned to perform a specific task or function.

[1583] "Security" refers to the means and measures taken to protect information and systems from unauthorized access and attacks.

[1584] A "platform" is the environment or system on which software and hardware operate.

[1585] A "request" is a request or inquiry made by a user to a system.

[1586] A "server" is a computer that provides services to other computers on a network.

[1587] A "terminal" is a device or computer that is directly operated by a user.

[1588] A "generative AI model" is an artificial intelligence model that is trained to generate information for a specific task.

[1589] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[1590] "Generating information" means creating new data or answers based on input data and prompts.

[1591] "Sharing information" means exchanging generated data and knowledge with other AI models and systems.

[1592] "Consistent information provision" means that data obtained from multiple information sources and AI models is consistent and provided to users in a unified form.

[1593] This invention is a system in which each generative AI model operates independently and generates and provides information according to its role. Furthermore, each generative AI model can share information with other generative AI models, providing consistent information as a whole.

[1594] Hardware and software used

[1595] The server trains and deploys each generative AI model using machine learning frameworks such as TensorFlow (registered trademark) and PyTorch. The server also communicates with terminals using HTTP requests and WebSockets. Terminals are devices such as smartphones and PCs that provide an interface for users to input requests.

[1596] Data processing and calculation

[1597] The server receives a request from the user, analyzes it, and selects an appropriate generative AI model. The selected generative AI model inputs a prompt sentence to generate information. The generated information is shared with other generative AI models to provide consistent information overall. The final information is sent back from the server to the device and displayed to the user.

[1598] Specific examples

[1599] As a concrete example, consider a news article generation system. A news article generation AI, a sports article generation AI, and a weather forecast generation AI each operate independently to generate specific information. When a user requests "Tell me today's news," the device sends this request to a server. The server then calls the news article generation AI to generate the latest news articles. The generated news articles are shared with other generative AI models (for example, a sports article generation AI and a weather forecast generation AI), providing consistent information overall.

[1600] Prompt Sentence Examples

[1601] Here is an example prompt:

[1602] "Tell me the news today"

[1603] "Tell me the latest sports news"

[1604] "What's the weather forecast for today?"

[1605] In this way, a system is constructed that provides consistent information as a whole, even though each generative AI model operates independently. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1606] Step 1:

[1607] The user sends a request from the terminal.

[1608] Users open the application on their smartphone or computer, type "Tell me today's news" into the search bar, and press the send button.

[1609] Input: User request (e.g. "Tell me today's news")

[1610] Output: Request data sent to the terminal

[1611] Step 2:

[1612] The terminal forwards the request to the server.

[1613] The device receives the user's input and sends an HTTP POST request to the server, which includes the user's input.

[1614] Input: User request data

[1615] Output: HTTP request sent to the server

[1616] Step 3:

[1617] The server analyzes the request and selects the appropriate generative AI model.

[1618] The server analyzes the received request and detects the keyword "news." Based on this, it selects a news article generation AI.

[1619] Input: HTTP request data

[1620] Output: Selected generative AI model (news article generation AI)

[1621] Step 4:

[1622] The server inputs a prompt sentence into the selected generative AI model to generate information.

[1623] The server inputs the prompt "Tell me today's news" into the news article generation AI, and the AI ​​model generates the latest news articles. The generated news articles are returned to the server in text format.

[1624] Input: Prompt sentence (e.g. "Tell me today's news")

[1625] Output: Generated news article

[1626] Step 5:

[1627] The server shares the generated information with other generative AI models.

[1628] The server shares the generated news articles with the sports article generation AI and weather forecast generation AI, allowing other AIs to generate information based on the latest news information.

[1629] Input: Generated news article

[1630] Output: Information shared with other generative AI models

[1631] Step 6:

[1632] The server returns the final information to the terminal.

[1633] The server then compiles the news article, sports news, and weather forecast into a single response and sends it back to the device. This response is sent in JSON format.

[1634] Input: Shared information (news articles, sports news, weather forecasts)

[1635] Output: JSON response sent to the terminal

[1636] Step 7:

[1637] The terminal displays the information to the user.

[1638] The device parses the JSON response received from the server and displays news articles, sports news, and weather forecasts in their respective sections, allowing users to view this information on their device screen.

[1639] Input: JSON response from the server

[1640] Output: Information displayed to the user (news articles, sports news, weather forecasts)

[1641] (Application example 3)

[1642] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1643] In self-driving vehicles, multiple AIs are required to operate independently and generate and provide information according to their respective roles. However, in order for these AIs to provide consistent information, a means is needed to efficiently share and integrate the information generated by each AI. Furthermore, it is technically difficult to process and integrate a wide range of information in real time, such as obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation. A system that can solve these issues is needed.

[1644] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for importing specialized knowledge, the latest information, and personal information; means for conversing with humanoid AIs not only through text but also through other means; means for assigning roles to each AI; means for providing information together with security and platform information; means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in an autonomous vehicle; and means for sharing information generated by each AI through a communication protocol to provide consistent information. This enables multiple AIs to operate independently while efficiently sharing information and providing consistent information.

[1645] "Expertise" is information that represents a high level of understanding or skill in a particular field.

[1646] "Latest information" refers to the latest data and knowledge about current conditions and technology.

[1647] "Personal information" refers to data and attribute information about individuals.

[1648] "Humanoid AI" is artificial intelligence that mimics the shape and movements of humans.

[1649] "Means of conversation" refers to technologies and interfaces that allow for natural dialogue with users.

[1650] "Roles" refers to the method of assigning specific functions or tasks to each AI.

[1651] "Security information" is information used to ensure the safety of systems and data.

[1652] "Platform information" is information about the platform on which the system operates.

[1653] The "means of provision" refers to the method by which information or services are provided to users.

[1654] An "autonomous vehicle" is a vehicle that drives autonomously without driver intervention.

[1655] "Obstacle detection" is a technology that detects obstacles that exist in the direction of travel of a vehicle.

[1656] "Route optimization" is a technique for calculating the optimal route to a destination.

[1657] "Traffic situation analysis" is a technique for evaluating and analyzing the current traffic situation.

[1658] "Passenger comfort assessment" is a technique for assessing passenger comfort.

[1659] A "communication protocol" is a set of rules and procedures for sending and receiving data.

[1660] "Providing consistent information" refers to integrating data obtained from multiple sources and presenting it in a consistent manner.

[1661] A system for carrying out this invention is configured as follows: The server includes means for taking in specialized knowledge, the latest information, and personal information, means for conversing with humanoid AIs rather than just text, means for assigning roles to each AI, means for providing information together with security and platform information, means for performing obstacle detection, route optimization, traffic situation analysis, and passenger comfort evaluation in autonomous vehicles, and means for sharing information generated by each AI via a communication protocol to provide consistent information.

[1662] Hardware and Software Configuration

[1663] The server uses the following hardware and software:

[1664] Hardware

[1665] Autonomous vehicle sensors (cameras, LIDAR, GPS, etc.)

[1666] In-vehicle computer

[1667] software

[1668] Python programming language

[1669] threading module (concurrency)

[1670] queue module (data queue)

[1671] Data processing and calculation

[1672] The server processes the data and performs calculations as follows:

[1673] 1. Sensor data acquisition: Acquire real-time data from the autonomous vehicle's sensors, such as video data from cameras and LIDAR, and location information from GPS.

[1674] 2. Data processing by AI: Each AI processes data using its own algorithm. For example, obstacle detection AI analyzes sensor data and determines whether there is an obstacle ahead. Route optimization AI calculates the optimal route based on map data. Traffic situation analysis AI analyzes traffic data and evaluates the current traffic situation. Passenger comfort assessment AI evaluates comfort based on passenger data.

[1675] 3. Data sharing via communication protocols: Information generated by each AI is shared via communication protocols, enabling consistent information provision. For example, obstacle detection AI can send information about detected obstacles to route optimization AI, which can then calculate the optimal route to avoid them.

[1676] Specific examples

[1677] For example, imagine an autonomous vehicle driving through a city: it detects obstacles ahead (e.g., pedestrians), recalculates the optimal route, analyzes traffic conditions, and evaluates passenger comfort. All this information is integrated in real time and reflected in the vehicle's operations.

[1678] Prompt Sentence Examples

[1679] "Design a system that detects pedestrians ahead, recalculates the optimal route, analyzes traffic conditions, and assesses passenger comfort while an autonomous vehicle is driving through a city."

[1680] In this way, it is possible to provide a specific embodiment of the invention.

[1681] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1682] Step 1:

[1683] The server receives real-time data from the autonomous vehicle's sensors. As input, it receives video data from cameras and LIDAR, and location information from GPS. It then takes this data and passes it on to the next processing step.

[1684] Step 2:

[1685] The server passes sensor data as input to the obstacle detection AI, which analyzes the video and LIDAR data to determine whether there is an obstacle ahead and generates the location information of the obstacle as output.

[1686] Step 3:

[1687] The server passes map data and obstacle location information to the route optimization AI as input. The route optimization AI calculates the optimal route based on the map data and generates a new route to avoid obstacles. The output is the optimal route information.

[1688] Step 4:

[1689] The server passes traffic data as input to the traffic situation analysis AI. The traffic situation analysis AI analyzes the current traffic data and evaluates traffic flow and congestion information. As output, it generates a traffic situation evaluation result.

[1690] Step 5:

[1691] The server passes passenger data as input to the passenger comfort evaluation AI, which evaluates comfort based on the passenger data and generates passenger comfort evaluation results as output.

[1692] Step 6:

[1693] The server shares the information generated by each AI through a communication protocol. As input, it receives information on the location of obstacles, optimal route information, traffic situation evaluation results, and passenger comfort evaluation results. This information is integrated to provide consistent information.

[1694] Step 7:

[1695] The server passes the integrated information to the autonomous vehicle's control system, which receives the integrated information as input and reflects it in the vehicle's operation. Specifically, it changes the route to avoid obstacles, adjusts the speed according to traffic conditions, and operates in a way that takes passenger comfort into consideration.

[1696] In this way, the server uses multiple AIs to optimize the operation of autonomous vehicles and provide consistent information.

[1697] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1698] "Example 1"

[1699] The invention, which combines the emotion engine, is an AI system that recognizes the user's emotions and engages in dialogue accordingly. Specifically, it infers emotions from the user's tone of voice, facial expressions, and choice of words, and generates a response appropriate to those emotions. For example, it provides words of comfort when the user is sad, and words of sympathy when the user is happy. This makes the dialogue between the AI ​​and the user more natural and human-like.

[1700] "Example 2"

[1701] The invention combining emotion engines is a system in which each AI operates independently to generate and provide information according to its respective role. Specifically, some AIs provide the latest news, others provide recommended information based on the user's personal information, and still others use the emotion engine to recognize the user's emotions and engage in dialogue accordingly. This allows users to obtain information tailored to their emotions.

[1702] "Example 3"

[1703] The form of the invention that combines emotion engines is a system in which each AI shares information with other AIs and provides consistent information as a whole. Specifically, some AIs share information generated or recognized user emotions with other AIs, and provide information and engage in dialogue based on that. For example, if one AI recognizes the user's emotion of joy, it shares that information with other AIs, and the other AIs also provide information and engage in dialogue that is tailored to the user's joy. This enables consistent dialogue as a whole.

[1704] The processing flow of each embodiment will be described below.

[1705] "Example 1"

[1706] Step 1: Initiate a dialogue with the user.

[1707] Step 2: Infer the user's emotions from their tone of voice, facial expressions, and choice of words.

[1708] Step 3: Generate a response appropriate to the estimated emotion.

[1709] Step 4: Provide the generated response to the user.

[1710] "Example 2"

[1711] Step 1: Each AI generates and provides information independently.

[1712] Step 2: Some AI will provide news updates.

[1713] Step 3: Another AI provides recommendations based on the user's personal information.

[1714] Step 4: Another AI uses an emotion engine to recognize the user's emotions and interact accordingly.

[1715] "Example 3"

[1716] Step 1: Each AI shares information with other AIs.

[1717] Step 2: Some AIs share the information they generate and the user's emotions they recognize with other AIs. Step 3: Other AIs provide information and engage in dialogue based on the shared information.

[1718] Example 1

[1719] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1720] Conventional AI systems have difficulty effectively incorporating specialized knowledge, up-to-date information, and personal information, resulting in a lack of natural, human-like interactio...

Claims

1. a means for capturing specialized knowledge, updates and personal information; A means for converting a user's speech into text using voice recognition technology; A means to analyze the collected information using natural language processing technology and use it as training data for generative AI models; means for generating an appropriate response based on user input using the generative AI model; means for converting the generated text response into speech using speech synthesis technology and playing it to the user; A system including:

2. Multiple generation AIs operate independently to generate and provide information according to their respective roles. The system of claim 1

3. Multiple generation AIs share information with other AIs to provide consistent information as a whole. A system according to claim 1 or claim 2.

Images