system

Abstract

Provide a system. [Solution] a means for obtaining information from publicly available internet sources; A means for analyzing the acquired information using natural language processing technology; a means for translating the analyzed information into multiple languages ​​using a multilingual translation system; means for storing the translated information in a database; means for providing information in a specified language based on a request from a user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Japan is prone to frequent natural disasters, including earthquakes, tsunamis, and typhoons, which often affect the operation of public transportation. While information about the status of public transportation is provided on official websites, it is often only available in Japanese, making it difficult for foreigners to understand. This poses a significant barrier to using public transportation and makes it difficult to make appropriate decisions in emergencies. Therefore, there is a need for a system that provides information about public transportation status in multiple languages. [Means for solving the problem]

[0005] The present invention relates to a system that acquires information from publicly available data sources on the Internet, analyzes the information using natural language processing technology, translates the analyzed information into multiple languages ​​using a multilingual translation system, and stores the translated information in a database. The system also includes a means for providing information in a specified language based on a user request. The system also includes a means for sending notifications to users based on preset conditions in an emergency, and a means for updating and providing information in real time based on a user request. This configuration enables the provision of information about train operation status in multiple languages ​​that is easy for foreign users to use, and enables rapid response in emergencies.

[0006] "Publicly available internet data sources" refers to websites and data feeds that public transport operators and related organizations publish on the internet to provide information on operation status, etc.

[0007] "Means of obtaining information" refers to the mechanism for collecting necessary information from publicly available data sources on the Internet using scraping technology or API communication.

[0008] "Natural language processing technology" is a technology for analyzing raw text data and understanding the meaning and structure of sentences, and in particular refers to a method for extracting important information and keywords.

[0009] A "multilingual translation system" is a technology or system for translating acquired information into multiple languages, and generally includes machine translation APIs.

[0010] "Database" refers to a system for systematically and efficiently storing and managing collected and translated information.

[0011] "Means for providing information based on user requests" refers to a mechanism for processing requests received through a user interface and returning information in the specified language.

[0012] "Means for sending notifications in an emergency" refers to a system or function for sending notifications to designated users in an emergency based on pre-set conditions.

[0013] "Means of updating and providing information in real time" refers to a system for obtaining, translating, and providing the latest information in real time, and refers to technology that allows users to always check the latest operating status. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific examples of the present invention are described below.

[0036] Server Processing

[0037] The server periodically retrieves the latest operating status from the official websites of public transportation organizations. This is done using the Python scraping libraries "BeautifulSoup" and "Selenium." For example, it scrapes the operation information for a specific line from the official JR line website and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0038] The acquired text information is analyzed using natural language processing (NLP) techniques, specifically using Python's "spaCy" and "NLTK" to extract important keywords (e.g., place names, operation status) from the information.

[0039] The analyzed information is then translated into multiple languages ​​using a multilingual translation system such as the Google® Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0040] The translated information is stored in a relational database (e.g., MySQL (registered trademark), PostgreSQL). The database stores operation status data, route names, acquisition dates and times, translated text, and other information.

[0041] Terminal handling

[0042] A device (such as a smartphone application or web browser) sends an API request to a server based on a user request. For example, if a user wants to check the "Tokaido Line operation status," the device sends a request such as " / api / route_status?line=tokaido" to the server.

[0043] The server reads the operation status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response may return data in JSON format such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0044] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0045] User Action

[0046] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0047] Users can check the latest train status in their preferred language. For example, if a user wants to check the "Tokaido Line train status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon."

[0048] In an emergency, users can receive push notifications based on their pre-set notification conditions. For example, if a service is suspended due to a typhoon, users who have set up notifications will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0049] The above is a specific embodiment of the present invention. This system allows foreign users to easily understand the operation status of public transportation in Japan in real time and in their own language, enabling them to respond quickly and appropriately in emergencies.

[0050] The processing flow will be explained below.

[0051] Step 1:

[0052] The server periodically accesses publicly available data sources on the Internet. Specifically, it accesses the official website of a public transportation company and extracts information about its operation status using Python's "BeautifulSoup" and "Selenium." For example, the server retrieves text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0053] Step 2:

[0054] The raw text data acquired by the server is analyzed using natural language processing (NLP). Specifically, Python's "spaCy" and "NLTK" are used to extract important keywords such as place names and service status from the text data. For example, information such as "Shinjuku Station," "Tokyo Station," and "service suspension" is analyzed and extracted.

[0055] Step 3:

[0056] The server translates the analyzed information using a multilingual translation system. Using a system such as the Google Translate API, the extracted information is translated into multiple languages. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0057] Step 4:

[0058] The server stores the translated information in a relational database, which stores information such as operation status data, route names, acquisition dates and times, and translated text. This allows operation information in each language to be stored in an organized manner.

[0059] Step 5:

[0060] The device receives a request from the user. For example, if the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0061] Step 6:

[0062] When the server receives a request from a device, it retrieves the corresponding data from the database. It uses an SQL query to retrieve the data and extracts the service information in the language specified in the request. For example, the server returns data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0063] Step 7:

[0064] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon" to help the user understand the situation.

[0065] Step 8:

[0066] The user sets up notifications in case of an emergency. For example, if the user wants to be notified of changes in service due to a typhoon, the user can turn on notifications in the app. The server stores this setting information and sends push notifications in case of an emergency according to the setting.

[0067] Step 9:

[0068] The server sends notifications in the event of an emergency. Based on pre-set conditions, for example, if service suspension information due to a typhoon is saved in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to registered users.

[0069] The above is a specific processing flow of the program according to the present invention.

[0070] Example 1

[0071] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0072] Understanding the status of public transportation is extremely important, especially during natural disasters, but foreigners face the challenge of quickly obtaining this information in their native language. Conventional systems lack multilingual support, real-time updates, and emergency notification functions, making it difficult for foreign users to take appropriate action.

[0073] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0074] In this invention, the server includes means for acquiring information from publicly available information sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a relational database, means for providing information in a specified language based on a user request, and means for displaying the information on the user's terminal, thereby enabling foreign users to understand the operation status of public transportation in real time in their own language.

[0075] "Publicly available internet sources" refers to information providers that are publicly accessible via the internet, such as official public transport websites or news sites.

[0076] "Natural language processing technology" is a computer technology for analyzing and classifying text data, extracting keywords, etc., and examples include morphological analysis and syntactic analysis.

[0077] A "multilingual translation system" is a system that automatically converts text written in one language into multiple other languages, and includes existing translation APIs and dedicated translation algorithms.

[0078] A "relational database" is a database system for storing and managing data based on a relational model, and allows information to be manipulated through queries using SQL.

[0079] "Request from the user" refers to a request for information acquisition or operation made by the user to the system via a terminal.

[0080] A "user device" is a device used by a user to receive and display information, such as a smartphone, tablet, or computer.

[0081] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific embodiments for carrying out the present invention are described below.

[0082] Server Processing

[0083] The server periodically retrieves the latest operation status from the official websites of public transportation organizations. This process uses the Python scraping libraries "BeautifulSoup" and "Selenium." For example, the server scrapes operation information for a specific line from the official website of a railway company and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0084] The acquired text information is analyzed using natural language processing (NLP) technology. Specifically, Python's "spaCy" or "NLTK" is used to extract important keywords from the information (e.g., place names, service status). The server then translates the analyzed information into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0085] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[0086] Terminal handling

[0087] A device (for example, a smartphone application or web browser) sends an API request to a server based on a user request. If a user wants to check the "Tokaido Line service status," the device sends a request such as " / api / route_status?line=tokaido" to the server. The server reads the service status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response could return data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0088] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0089] User Action

[0090] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0091] Users can check the latest service status in their preferred language. For example, if a user wants to check the "Tokaido Line service status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon." In an emergency, users can receive push notifications based on the notification conditions they have set up in advance. For example, a user who has set up notifications for service suspensions due to typhoons will receive a push notification on their device stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0092] In this way, the present invention enables foreign users to easily grasp the operation status of public transportation in Japan in real time and in their own native language, enabling them to respond quickly and appropriately in emergencies.

[0093] Specific examples

[0094] The server retrieves information from the official website of a railway company, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station," and uses Python's "spaCy" to extract the keywords "Shinjuku Station," "Tokyo Station," "earthquake," and "service suspended." It then translates this information into English using the Google Translate API and stores it in a relational database.

[0095] When a user checks the "Tokaido Line operation status" on a smartphone app, the server reads the information from the database and returns a response in English saying, "Service suspension between Shinagawa and Yokohama due to typhoon." The user's device then outputs this information on the display screen so that the user can check it.

[0096] An example of a specific prompt is, "Write Python code for a system that scrapes public transport status and translates it into multiple languages. Include code that parses the HTML using BeautifulSoup and Selenium, performs NLP analysis with spaCy, translates the results into multiple languages ​​using the Google Translate API, and stores them in a relational database."

[0097] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0098] Step 1:

[0099] The server periodically retrieves operation status data from publicly available information sources on the Internet. Specifically, it uses Python's "BeautifulSoup" and "Selenium" to scrape the official homepages of public transportation agencies. The URL of the public transportation agency is given as input, and HTML data is obtained as output. For example, text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station" is obtained.

[0100] Step 2:

[0101] The server analyzes the acquired HTML data using natural language processing technology. Here, "spaCy" and "NLTK" are used. The text information obtained in step 1 is given as input, and important keywords (e.g., place names, service status) are extracted as output. Specifically, keywords such as "Shinjuku Station," "Tokyo Station," "earthquake," and "suspension of service" are extracted from the text.

[0102] Step 3:

[0103] The server translates the analyzed keyword information using a multilingual translation system. This process is performed using the Google Translate API or similar. The keyword information obtained in step 2 is given as input, and text translated into multiple languages ​​is obtained as output. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English yields the result "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0104] Step 4:

[0105] The server stores the translated information in a relational database. Here, "MySQL" or "PostgreSQL" is used. The translated text obtained in step 3 is given as input, and is saved in the database as output. The database stores operation status data, route name, acquisition date and time, and translated text. Specifically, the translated text is inserted into the database using an SQL query.

[0106] Step 5:

[0107] The device sends an API request to the server based on the user's request. The user's request data is given as input, and the request to be sent to the server is formed as output. For example, if a user wants to check the "Tokaido Line operation status," the device sends the request " / api / route_status?line=tokaido" to the server.

[0108] Step 6:

[0109] The server receives the request from the terminal, reads the corresponding operation status data from the database, and generates a response. The request sent from the terminal in step 5 is given as input, and appropriate operation status information is generated in JSON format as output. For example, data such as "Service suspension between Shinagawa and Yokohama due to typhoon" is generated in JSON format.

[0110] Step 7:

[0111] The terminal displays the information received from the server to the user. The JSON-formatted response data obtained in step 6 is given as input, and the output is displayed in a format that is easy for the user to read. Here, a UI component is used to display text such as "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0112] Step 8:

[0113] Users set their preferred language through apps or websites. The input is the user's language selection, and the output is the preference stored on the server. For example, if a native English speaker selects English, this preference is stored on the server and applied to future requests.

[0114] Step 9:

[0115] In the event of an emergency, the user will receive a push notification based on the notification conditions they set. The input is emergency notification information generated by the server, and the output is a push notification sent to the user's device. Specifically, the device will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0116] (Application example 1)

[0117] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] The purpose of this invention is to solve the problem of the lack of means to provide real-time multilingual operation information to users of self-driving vehicles in emergency situations such as natural disasters and traffic accidents, and in particular to enable foreign users to respond quickly and accurately in emergency situations.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0120] In this invention, the server includes means for acquiring data from publicly available information sources on the Internet, means for analyzing the acquired data using natural language processing technology, means for translating the analyzed data into multiple languages ​​using a multilingual translation system, means for storing the translated data in a database, means for providing data in a specified language based on a user request, and means for displaying the acquired data on a dashboard or head-mounted display in the system of the autonomous vehicle. This enables users of the autonomous vehicle to acquire the latest operational information in multiple languages ​​even in an emergency and to take appropriate action promptly.

[0121] "Public internet resources" refers to websites and online services that are publicly accessible on the internet.

[0122] "Data" refers to elements that make up specific information or content, and in this case refers to things like public transportation operation information.

[0123] "Means of acquisition" refers to the technology or method for automatically collecting the required data from the source.

[0124] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and generate human language.

[0125] "Means of analysis" refers to the techniques and methods used to analyze acquired data and extract necessary information.

[0126] "Multilingual translation system" refers to any technology or service that translates from one language to another.

[0127] "Translation means" refers to techniques or methods for converting information written in one language into another language.

[0128] A "database" refers to a system that systematically stores large amounts of data and allows for efficient access, search, and management.

[0129] "Storage means" refers to the techniques and methods used to store the captured and translated data in a database.

[0130] A "request" refers to a request for specific information from a user.

[0131] "Preferred Language" refers to a particular language selected or set by a user.

[0132] "Means of providing" refers to the technology or method for appropriately displaying or transmitting the information required by the user.

[0133] An "autonomous vehicle" refers to a vehicle that can drive autonomously using artificial intelligence and sensor technology.

[0134] A "system" refers to an overall device or configuration that combines multiple technical elements and means to achieve a specific purpose.

[0135] "Dashboard" refers to a display device installed in the driver's seat of a vehicle that provides various information to the driver.

[0136] A "head-mounted display" refers to a display device worn on the head, and is a device for displaying information within the field of vision.

[0137] The present invention relates to a system for providing real-time operation information in multiple languages ​​to users of autonomous vehicles. Specific embodiments of the system are described below.

[0138] System Configuration

[0139] This system mainly consists of three elements: a server, a terminal, and a user.

[0140] 1. Server Processing

[0141] Data Acquisition

[0142] The server uses the Python scraping libraries "BeautifulSoup" and "Selenium" to retrieve operation information from the official websites of public transportation companies. For example, it retrieves text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specific official website.

[0143] Data analysis

[0144] The acquired text data is analyzed using natural language processing (NLP) techniques using Python's "spaCy" and "NLTK," which extracts important keywords such as place names and operation status.

[0145] Multilingual Translation

[0146] Information containing the analyzed keywords is translated into multiple languages ​​using the Google Translate API. For example, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0147] Data Storage

[0148] The translated information is stored in a relational database (e.g., MySQL or PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0149] 2. Terminal Processing

[0150] Data Acquisition and Display

[0151] The autonomous vehicle's terminal (dashboard or head-mounted display) sends API requests to the server based on the user's request. For example, if a user wants to check the "Tokaido Line operation status," the terminal will send a request such as " / api / route_status?line=tokaido" to the server. The server will read the corresponding operation status data from the database and return a response in the specified language. The terminal will then display this information in an easy-to-read format for the user.

[0152] 3. User Operation

[0153] Language settings and information confirmation

[0154] Users can set their preferred language on the app or vehicle dashboard. For example, a native English speaker can set English to check the latest service status. If they want to check service suspension information due to a typhoon, the device will display "Service suspension between Shinagawa and Yokohama due to typhoon."

[0155] Push notifications

[0156] In the event of an emergency, users will receive a push notification based on pre-set notification conditions. For example, if a service is suspended due to a typhoon, a notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama" will be sent to the user's device.

[0157] Examples and prompts

[0158] Specific examples

[0159] In the event of a natural disaster, the dashboard of the self-driving vehicle will display information such as, "Service between Tokyo Station and Shinjuku Station has been suspended due to an earthquake. Searching for an alternative route."

[0160] Prompt statement

[0161] An example prompt sent to a generative AI model is:

[0162] "Please provide the latest train service status between Tokyo Station and Shinjuku Station in English due to an earthquake."

[0163] This will enable users of autonomous vehicles to obtain the latest operational information in real time and take appropriate measures quickly.

[0164] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0165] Step 1:

[0166] Data acquisition (server)

[0167] The server periodically retrieves data from publicly available information sources on the Internet. Using Python's "BeautifulSoup" and "Selenium," it scrapes operation information from the official website of a public transportation company. For example, it retrieves text data from the official website stating, "Service suspended between Shinjuku Station and Tokyo Station due to earthquake." The input at this point is the URL of the official website and the HTML data to be retrieved, and the output is the text data of the operation information.

[0168] Step 2:

[0169] Data analysis (server)

[0170] The server analyzes the acquired text data using natural language processing (NLP) techniques such as "spaCy" and "NLTK." The purpose of the analysis is to extract important keywords such as place names and operation status. The input is the text data acquired by scraping, and the output is a set of extracted keywords.

[0171] Step 3:

[0172] Multilingual translation (server)

[0173] The server uses the Google Translate API to translate information containing the analyzed keywords into multiple languages. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English results in "Service suspension between Shinjuku Station and Tokyo Station due to earthquake." The input is the extracted keywords and the original text data, and the output is the text data translated into each language.

[0174] Step 4:

[0175] Data storage (server)

[0176] The server stores the translated information in a relational database (MySQL or PostgreSQL). The database stores operation status data, route names, acquisition dates and times, translated text, etc. The input is multilingually translated text data, and the output is data records stored in the database.

[0177] Step 5:

[0178] Receiving user requests (terminal)

[0179] The terminal (dashboard or head-mounted display of an autonomous vehicle) receives the user's request. When the user wants to check traffic information, the terminal sends an API request to the server. For example, it sends a request such as " / api / route_status?line=tokaido". The input is the user's request information, and the output is the API request.

[0180] Step 6:

[0181] Data provision (server)

[0182] The server retrieves the corresponding operation status data from the database based on the received API request, and returns the operation information in the specified language as a response. The input is the API request, and the output is the text data of the operation status translated into the specified language (for example, "Service suspension between Shinagawa and Yokohama due to typhoon").

[0183] Step 7:

[0184] Information display (terminal)

[0185] The terminal receives the traffic information from the server and displays it in a format that is easy for the user to see. For example, it displays "Shinagawa - Yokohama: Service suspended due to typhoon" on the dashboard or head-mounted display. The input is the text data of the traffic information received from the server, and the output is a visual display for the user.

[0186] Step 8:

[0187] Emergency notification (terminal)

[0188] In an emergency, push notifications are sent based on pre-set notification conditions. For example, if a user has set up notifications for earthquakes, a push notification stating "Earthquake alert: Service suspended between Shinagawa and Yokohama" will be sent to the device. The input is emergency operation information and user settings, and the output is a push notification.

[0189] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0190] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. Specific examples of the present invention are described below.

[0191] Server Processing

[0192] The server periodically retrieves the latest operational status from the official website of the public transport company. Using Python's "BeautifulSoup" and "Selenium," it extracts text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0193] The acquired text information is analyzed using natural language processing (NLP) techniques, such as using Python's "spaCy" and "NLTK" to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0194] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0195] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0196] Terminal handling

[0197] When the device receives a user request, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[0198] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format is returned.

[0199] The device displays the information received from the server in a format that is easy for the user to see. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0200] Emotion engine processing

[0201] An emotion engine is built into the device to recognize the user's emotional state. The emotion engine uses facial recognition and voice analysis technology to analyze the user's emotions in real time. For example, if the emotional state indicates "tension" or "stress," that information is sent to the server.

[0202] Emergency Notification

[0203] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0204] Customize your information

[0205] The emotional engine adjusts information provision based on the user's emotional state. For example, if a user is emotionally stressed, important information will be presented in a prominent format. The user interface adjusts based on the emotional state, allowing users to quickly obtain the information they need most.

[0206] The above is a specific embodiment of the present invention. This system allows users to accurately grasp the operation status of public transportation regardless of their emotional state and respond quickly to emergencies. By incorporating an emotion engine, more detailed information can be provided to users, improving user satisfaction.

[0207] The processing flow will be explained below.

[0208] Step 1:

[0209] The server periodically accesses the official website of the public transport company and scrapes information about the operation status. Using Python's "BeautifulSoup" and "Selenium," it obtains text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0210] Step 2:

[0211] The server analyzes the acquired text data using natural language processing (NLP) techniques. Python's "spaCy" and "NLTK" are used to extract important keywords from the extracted text, such as place names (e.g., Shinjuku Station, Tokyo Station) and service status (e.g., service suspension).

[0212] Step 3:

[0213] The server translates the analyzed information using a multilingual translation system. For example, using the Google Translate API, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0214] Step 4:

[0215] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL) as records containing the operation status data, route name, acquisition date and time, and the translated text.

[0216] Step 5:

[0217] The device receives a request from the user. If the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0218] Step 6:

[0219] The server receives a request from the device, retrieves the corresponding data from the database, and uses an SQL query to extract the service information in the language specified in the request, returning data in JSON format, such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0220] Step 7:

[0221] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0222] Step 8:

[0223] The device's built-in emotion engine recognizes the user's emotional state. Using facial recognition and voice analysis technology, it analyzes emotions from the user's facial expressions and voice, recognizing emotional states such as "tension" or "stress."

[0224] Step 9:

[0225] The device then sends the recognized emotional state to the server. For example, if the user is nervous, that information is sent to the server.

[0226] Step 10:

[0227] The server adjusts the information provided based on the user's emotional state. For example, if the user is emotionally tense, it may emphasize the display of traffic information or send notifications more frequently.

[0228] Step 11:

[0229] The server sends notifications in emergencies. For example, if a service is suspended due to a typhoon, the server sends a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to the user based on the notification conditions set by the user in advance.

[0230] Step 12:

[0231] The device displays the received notification to the user. Based on the analysis results of the emotion engine, the notification is displayed in an appropriate manner according to the level of urgency, allowing the user to respond immediately.

[0232] The above are the specific processing steps of the present invention, which includes an emotion engine. This system allows users to receive appropriate information according to their emotional state, enabling them to respond quickly and appropriately in emergencies.

[0233] Example 2

[0234] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0235] Conventional public transportation operation status systems can provide information in multiple languages, but they have the problem of being unable to provide information flexibly according to the user's emotional state. Furthermore, even in emergencies, appropriate notifications that take the user's emotional state into consideration are not provided, resulting in stress and inconvenience for users. Therefore, there is a need for a system that provides more detailed information customized according to the user's emotional state.

[0236] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0237] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state using facial recognition technology and voice analysis technology, and means for customizing the information to be provided based on the user's emotional state. This makes it possible to provide information according to the user's emotional state and to send notifications at an appropriate time even in emergencies.

[0238] "Internet data sources" refers to any information resource accessible via the Internet, including, for example, websites, databases, and APIs.

[0239] "Natural language processing technology" refers to a set of technologies used by computers to understand, analyze, and generate human language, and examples include morphological analysis, keyword extraction, and sentiment analysis.

[0240] A "multilingual translation system" refers to a system that automatically translates text from one language into multiple other languages, for example using a machine translation engine or a translation API.

[0241] "Database" refers to a system that enables the efficient storage, retrieval, and management of data, and includes relational databases and key-value databases.

[0242] "Request" means a request a User makes to a System for a particular action or piece of information, including, for example, an API call or a search query.

[0243] "Facial recognition technology" refers to technology that uses a camera or other device to detect, identify, and analyze a person's face, and is used to determine the user's emotional state.

[0244] "Voice analysis technology" refers to technology that analyzes voice data and understands its content, and includes speech recognition, emotion recognition, and natural language understanding.

[0245] "Emotional state" refers to the emotional state the user is currently feeling, and examples include tension, stress, joy, etc.

[0246] "Customization" refers to adjusting the information provided and its presentation format according to the user's needs and emotional state.

[0247] "Notification" refers to an alert or message sent to a user based on a particular event or situation.

[0248] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. A specific embodiment of the present invention will be described in detail below.

[0249] Server processing and technologies used

[0250] The server retrieves the latest train status information from publicly available data sources on the Internet. To this end, the server uses web scraping libraries such as Python's "BeautifulSoup" and "Selenium" to analyze the content of web pages and extract specific train status information, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station."

[0251] The acquired text information is analyzed using natural language processing (NLP) technology using Python's "spaCy" and "NLTK," which allows important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" to be extracted.

[0252] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. The translated information is then stored in a relational database (e.g., MySQL or PostgreSQL) and includes operation status data, route names, acquisition dates and times, translated text, and other information.

[0253] Terminal processing and data provision

[0254] When the device receives a user request, it sends an API request to the server. For example, if the user wants to check the "Tokaido Line service status," the device sends the request " / api / route_status?line=tokaido" to the server. The server retrieves the corresponding data from the database in response to the received request and returns a response to the device in the specified language. The device then displays the received information in an easy-to-read format for the user, for example, "Shinagawa - Yokohama: Service suspended due to typhoon" on the smartphone app.

[0255] Emotion engine processing

[0256] This system incorporates an emotion engine that uses facial recognition and voice analysis technologies to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions and voice in real time through a camera and microphone to determine their emotional state, such as "tension" or "stress." This information is sent to a server, which then customizes the information provided based on the user's emotional state.

[0257] Emergency Notification

[0258] If the user has set up emergency notifications, the server will take into account the emergency information stored in the database and data from the emotion engine to send notifications at the appropriate time, for example, "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0259] Customize your information

[0260] An emotion engine tailors information delivery based on the user's emotional state. For example, if the user is feeling stressed, the system will deliver important information in a prominent format, allowing the user to quickly get the information they need most.

[0261] Examples and prompts

[0262] For example, when providing information about service suspensions due to a typhoon, users who are in a state of tension can be notified by displaying "Shinagawa - Yokohama: Service suspended due to typhoon" in red and large font.In addition, in an emergency, a "Typhoon alert: Service suspended between Shinagawa and Yokohama" notification can be sent immediately.

[0263] The generative AI model generates a detailed description of the system by providing the following prompt:

[0264] Please explain the specific processing steps and detailed behavior of each step in a public transportation status information system that incorporates an emotion engine. For example, if the user is feeling stressed, please explain in detail how information is provided based on that user's emotional state.

[0265] This system not only allows users to accurately and timely grasp the status of public transport, but also allows them to receive customized information according to their emotional state, reducing stress and enabling them to respond quickly. This system is expected to significantly improve user satisfaction.

[0266] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0267] Step 1: Obtaining the operation status of public transportation

[0268] The server retrieves the latest train operation status from publicly available data sources on the Internet. A specific URL (e.g., the URL of a public transportation company's official website) is used as input. Specific processing involves analyzing the content of the webpage using Python's "BeautifulSoup" and "Selenium" to extract text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station." The output is text information about the train operation status.

[0269] Step 2: Natural Language Processing (NLP) Analysis

[0270] The server analyzes the text information acquired in step 1 using natural language processing technology. The text information on train operation status is used as input. Specifically, it uses Python's "spaCy" or "NLTK" to extract important keywords (e.g., "Shinjuku Station," "Tokyo Station," "suspension") and outputs a list of the extracted keywords.

[0271] Step 3: Multilingual Translation

[0272] The server translates the information analyzed in step 2 using a multilingual translation system. A list of important keywords is used as input. Specifically, the server calls the Google Translate API or similar to translate the keywords into multiple languages. The translated information is obtained as output. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0273] Step 4: Store in the database

[0274] The server stores the translated information from step 3 in a database. The translated information is used as input. Specifically, the server stores the information in a relational database (e.g., MySQL, PostgreSQL). The output is the translated information stored in the database. The stored information includes operation status data, route name, acquisition date and time, translated text, etc.

[0275] Step 5: Receiving the user request

[0276] The terminal receives a request from the user. As input, it uses the request information entered by the user (e.g., "Tokaido Line operation status"). In concrete terms, the terminal assembles the request content into an API request and sends it to the server. As output, it obtains the API request sent to the server. For example, a request in the format " / api / route_status?line=tokaido" is sent.

[0277] Step 6: Generate and return the server response

[0278] The server retrieves the corresponding data from the database in response to the request received in step 5. The API request is used as input. Specifically, the server executes a database query to retrieve the corresponding data. The retrieved data is obtained as output. The retrieved data is converted to JSON format and returned to the terminal as a response. For example, the data returned is "Service suspension between Shinagawa and Yokohama due to typhoon."

[0279] Step 7: Receiving and Displaying the Server Response

[0280] The terminal receives the response from the server and displays it in a format that is easy for the user to view. As input, it uses the JSON data received from the server. Specifically, it analyzes the received data and displays it in the appropriate UI component. As output, it displays information in a format that is easy for the user to view. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0281] Step 8: Emotional state recognition by the emotion engine

[0282] The emotion engine recognizes the user's emotional state in real time through a camera and microphone. It uses facial images and voice data as input. Specifically, it uses facial recognition and voice analysis technologies to analyze the user's facial expressions and tone of voice, and estimates emotions such as "tension" or "stress." The output is data on the recognized emotional state.

[0283] Step 9: Tailor information delivery based on emotional state

[0284] The server customizes the information it provides based on the emotional state data received from the emotion engine. It uses the emotional state data as input. Specific processing involves adjusting the display format and priority of the information to highlight the information the user most desires. The customized information is obtained as output. For example, if the user is feeling stressed, important information is displayed in red or in a larger font.

[0285] Step 10: Send emergency notifications

[0286] The server sends a notification to the user when the conditions are met in an emergency. The input is emergency information stored in the database and data from the emotion engine. Specifically, the server generates a notification message according to the level of urgency and sends it to the user's device. The output is a notification to the user. For example, a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" is sent.

[0287] This system not only allows users to get accurate and timely information about public transportation operations, but also allows them to receive customized information based on their emotional state, reducing stress and enabling them to respond quickly.

[0288] (Application example 2)

[0289] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0290] Autonomous vehicles are required to provide real-time information on public transportation and road conditions, as well as information that takes into account the user's emotional state, thereby reducing stress and anxiety and achieving a comfortable driving experience. Conventional systems have difficulty meeting these requirements simultaneously, and more advanced responses are required. Therefore, it is necessary to realize a system that can quickly and appropriately provide the information desired by users and take effective measures in emergencies.

[0291] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0292] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state, and means for adjusting the format of information provision based on the emotion analysis results. This enables real-time information provision that takes the user's emotional state into consideration, and allows for the prompt provision of appropriate information even in emergencies, thereby reducing user stress and anxiety.

[0293] "Data source" refers to websites, databases, etc. that provide publicly available information on the Internet.

[0294] "Means of obtaining information" refers to the systems and processes that automatically collect the required information from data sources.

[0295] "Natural language processing technology" refers to the technology of analyzing text data, understanding its meaning, and extracting important information.

[0296] A "multilingual translation system" refers to a system that translates text written in one language into multiple other languages.

[0297] "Database" refers to a system for organizing and storing information that has been acquired, analyzed, and translated.

[0298] A "request" refers to an action in which a user requests information from a system.

[0299] "Means of providing information" refers to a system or process that returns the necessary information in an appropriate format based on the user's request.

[0300] "Emotional state" refers to a user's psychological state, such as stress or anxiety.

[0301] "Means for analyzing emotional state" refers to technologies and systems that analyze a user's emotional state from facial expressions, voice, etc.

[0302] "Means for adjusting the format of information provision" refers to a system that changes the display method and content of the information provided depending on the user's emotional state.

[0303] To implement the present invention, a system must be constructed based on the following steps.

[0304] Server configuration and processing

[0305] The server retrieves the latest train operation information from publicly available data sources on the Internet using web scraping tools such as "BeautifulSoup" and "Selenium," which use the Python programming language. For example, it extracts text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specified website.

[0306] The extracted text information is then analyzed using natural language processing (NLP) techniques, using Python libraries such as spaCy and NLTK to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0307] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0308] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0309] Terminal configuration and handling

[0310] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent. The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. The connected device (e.g., a smartphone) displays the information received from the server in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0311] Emotion engine processing

[0312] The emotion engine uses facial recognition and voice analysis technologies to provide users with comfortable information in real time. Specifically, it uses the Python "dlib" library for face detection and "DeepFace" for emotion analysis.

[0313] If the emotional state indicates "tension" or "stress," that state is sent to the server and the way the information is displayed changes. For example, if the user is feeling stressed, important information will be displayed in a more prominent format.

[0314] Emergency Notification

[0315] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0316] Usage example

[0317] When a user inquires about the Tokaido Line's service status using a smartphone, the device displays "Shinagawa - Yokohama: Service suspended due to typhoon." If the emotion engine detects stress in the user's facial expression, this information is highlighted and, if necessary, an emergency notification is sent.

[0318] Example prompts for generative AI models

[0319] "Please tell me the Tokaido Line operation status in multiple languages."

[0320] "Customize traffic notifications based on emotions."

[0321] In this way, the system of the present invention allows users to obtain real-time operational information that is adapted to their emotional state, enabling them to respond quickly in emergencies.

[0322] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0323] Step 1:

[0324] The server retrieves information from publicly available internet data sources, uses Python's BeautifulSoup and Selenium to extract text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake," and converts the text into a format that can be used in the next processing step.

[0325] Input: URL of the data source

[0326] Output: Extracted service information text data

[0327] Step 2:

[0328] The server analyzes the acquired information using natural language processing technology, using Python's "spaCy" and "NLTK" to extract important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" from the text information.

[0329] Input: Text data of operation information

[0330] Output: Parsed keywords

[0331] Step 3:

[0332] The server translates the analyzed information into multiple languages ​​using a multilingual translation system (e.g., Google Translate API). For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0333] Input: Parsed keyword

[0334] Output: Translated text data

[0335] Step 4:

[0336] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operation status data, route name, acquisition date and time, and translated text.

[0337] Input: Translated text data

[0338] Output: Data stored in the database

[0339] Step 5:

[0340] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[0341] Input: User request

[0342] Output: API request to the server

[0343] Step 6:

[0344] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" is returned in JSON format.

[0345] Input: API request

[0346] Output: JSON formatted service information

[0347] Step 7:

[0348] The device receives the information from the server and displays it in a user-friendly format. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0349] Input: Response from the server

[0350] Output: Traffic information displayed on the user's device

[0351] Step 8:

[0352] The server analyzes the user's emotional state in real time using an emotion engine, which uses facial recognition technology (Python's "dlib" or "DeepFace") and voice analysis technology to analyze the user's emotional state and transmits the results to the server.

[0353] Input: User's facial image and voice data

[0354] Output: Sentiment analysis results

[0355] Step 9:

[0356] The server adjusts the format of information delivery based on the user's emotional state: for example, if the user is stressed, important information will be displayed prominently and, if necessary, sent as an urgent notification.

[0357] Input: Sentiment analysis results, traffic information data

[0358] Output: Form of tailored information, emergency notification

[0359] The above are the specific processing steps for carrying out the present invention.

[0360] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0361] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0362] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0363] [Second embodiment]

[0364] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0365] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0366] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0367] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0368] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0369] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0370] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0371] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0372] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0373] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0374] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0375] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0376] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific examples of the present invention are described below.

[0377] Server Processing

[0378] The server periodically retrieves the latest operating status from the official websites of public transportation organizations. This is done using the Python scraping libraries "BeautifulSoup" and "Selenium." For example, it scrapes the operation information for a specific line from the official JR line website and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0379] The acquired text information is analyzed using natural language processing (NLP) techniques, specifically using Python's "spaCy" and "NLTK" to extract important keywords (e.g., place names, operation status) from the information.

[0380] The analyzed information is then translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0381] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[0382] Terminal handling

[0383] A device (such as a smartphone application or web browser) sends an API request to a server based on a user request. For example, if a user wants to check the "Tokaido Line operation status," the device sends a request such as " / api / route_status?line=tokaido" to the server.

[0384] The server reads the operation status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response may return data in JSON format such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0385] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0386] User Action

[0387] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0388] Users can check the latest train status in their preferred language. For example, if a user wants to check the "Tokaido Line train status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon."

[0389] In an emergency, users can receive push notifications based on their pre-set notification conditions. For example, if a service is suspended due to a typhoon, users who have set up notifications will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0390] The above is a specific embodiment of the present invention. This system allows foreign users to easily understand the operation status of public transportation in Japan in real time and in their own language, enabling them to respond quickly and appropriately in emergencies.

[0391] The processing flow will be explained below.

[0392] Step 1:

[0393] The server periodically accesses publicly available data sources on the Internet. Specifically, it accesses the official website of a public transportation company and extracts information about its operation status using Python's "BeautifulSoup" and "Selenium." For example, the server retrieves text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0394] Step 2:

[0395] The raw text data acquired by the server is analyzed using natural language processing (NLP). Specifically, Python's "spaCy" and "NLTK" are used to extract important keywords such as place names and service status from the text data. For example, information such as "Shinjuku Station," "Tokyo Station," and "service suspension" is analyzed and extracted.

[0396] Step 3:

[0397] The server translates the analyzed information using a multilingual translation system. Using a system such as the Google Translate API, the extracted information is translated into multiple languages. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0398] Step 4:

[0399] The server stores the translated information in a relational database, which stores information such as operation status data, route names, acquisition dates and times, and translated text. This allows operation information in each language to be stored in an organized manner.

[0400] Step 5:

[0401] The device receives a request from the user. For example, if the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0402] Step 6:

[0403] When the server receives a request from a device, it retrieves the corresponding data from the database. It uses an SQL query to retrieve the data and extracts the service information in the language specified in the request. For example, the server returns data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0404] Step 7:

[0405] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon" to help the user understand the situation.

[0406] Step 8:

[0407] The user sets up notifications in case of an emergency. For example, if the user wants to be notified of changes in service due to a typhoon, the user can turn on notifications in the app. The server stores this setting information and sends push notifications in case of an emergency according to the setting.

[0408] Step 9:

[0409] The server sends notifications in the event of an emergency. Based on pre-set conditions, for example, if service suspension information due to a typhoon is saved in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to registered users.

[0410] The above is a specific processing flow of the program according to the present invention.

[0411] Example 1

[0412] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0413] Understanding the status of public transportation is extremely important, especially during natural disasters, but foreigners face the challenge of quickly obtaining this information in their native language. Conventional systems lack multilingual support, real-time updates, and emergency notification functions, making it difficult for foreign users to take appropriate action.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0415] In this invention, the server includes means for acquiring information from publicly available information sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a relational database, means for providing information in a specified language based on a user request, and means for displaying the information on the user's terminal, thereby enabling foreign users to understand the operation status of public transportation in real time in their own language.

[0416] "Publicly available internet sources" refers to information providers that are publicly accessible via the internet, such as official public transport websites or news sites.

[0417] "Natural language processing technology" is a computer technology for analyzing and classifying text data, extracting keywords, etc., and examples include morphological analysis and syntactic analysis.

[0418] A "multilingual translation system" is a system that automatically converts text written in one language into multiple other languages, and includes existing translation APIs and dedicated translation algorithms.

[0419] A "relational database" is a database system for storing and managing data based on a relational model, and allows information to be manipulated through queries using SQL.

[0420] "Request from the user" refers to a request for information acquisition or operation made by the user to the system via a terminal.

[0421] A "user device" is a device used by a user to receive and display information, such as a smartphone, tablet, or computer.

[0422] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific embodiments for carrying out the present invention are described below.

[0423] Server Processing

[0424] The server periodically retrieves the latest operation status from the official websites of public transportation organizations. This process uses the Python scraping libraries "BeautifulSoup" and "Selenium." For example, the server scrapes operation information for a specific line from the official website of a railway company and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0425] The acquired text information is analyzed using natural language processing (NLP) technology. Specifically, Python's "spaCy" or "NLTK" is used to extract important keywords from the information (e.g., place names, service status). The server then translates the analyzed information into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0426] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[0427] Terminal handling

[0428] A device (for example, a smartphone application or web browser) sends an API request to a server based on a user request. If a user wants to check the "Tokaido Line service status," the device sends a request such as " / api / route_status?line=tokaido" to the server. The server reads the service status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response could return data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0429] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0430] User Action

[0431] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0432] Users can check the latest service status in their preferred language. For example, if a user wants to check the "Tokaido Line service status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon." In an emergency, users can receive push notifications based on the notification conditions they have set up in advance. For example, a user who has set up notifications for service suspensions due to typhoons will receive a push notification on their device stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0433] In this way, the present invention enables foreign users to easily grasp the operation status of public transportation in Japan in real time and in their own native language, enabling them to respond quickly and appropriately in emergencies.

[0434] Specific examples

[0435] The server retrieves information from the official website of a railway company, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station," and uses Python's "spaCy" to extract the keywords "Shinjuku Station," "Tokyo Station," "earthquake," and "service suspended." It then translates this information into English using the Google Translate API and stores it in a relational database.

[0436] When a user checks the "Tokaido Line operation status" on a smartphone app, the server reads the information from the database and returns a response in English saying, "Service suspension between Shinagawa and Yokohama due to typhoon." The user's device then outputs this information on the display screen so that the user can check it.

[0437] An example of a specific prompt is, "Write Python code for a system that scrapes public transport status and translates it into multiple languages. Include code that parses the HTML using BeautifulSoup and Selenium, performs NLP analysis with spaCy, translates the results into multiple languages ​​using the Google Translate API, and stores them in a relational database."

[0438] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0439] Step 1:

[0440] The server periodically retrieves operation status data from publicly available information sources on the Internet. Specifically, it uses Python's "BeautifulSoup" and "Selenium" to scrape the official homepages of public transportation agencies. The URL of the public transportation agency is given as input, and HTML data is obtained as output. For example, text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station" is obtained.

[0441] Step 2:

[0442] The server analyzes the acquired HTML data using natural language processing technology. Here, "spaCy" and "NLTK" are used. The text information obtained in step 1 is given as input, and important keywords (e.g., place names, service status) are extracted as output. Specifically, keywords such as "Shinjuku Station," "Tokyo Station," "earthquake," and "suspension of service" are extracted from the text.

[0443] Step 3:

[0444] The server translates the analyzed keyword information using a multilingual translation system. This process is performed using the Google Translate API or similar. The keyword information obtained in step 2 is given as input, and text translated into multiple languages ​​is obtained as output. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English yields the result "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0445] Step 4:

[0446] The server stores the translated information in a relational database. Here, "MySQL" or "PostgreSQL" is used. The translated text obtained in step 3 is given as input, and is saved in the database as output. The database stores operation status data, route name, acquisition date and time, and translated text. Specifically, the translated text is inserted into the database using an SQL query.

[0447] Step 5:

[0448] The device sends an API request to the server based on the user's request. The user's request data is given as input, and the request to be sent to the server is formed as output. For example, if a user wants to check the "Tokaido Line operation status," the device sends the request " / api / route_status?line=tokaido" to the server.

[0449] Step 6:

[0450] The server receives the request from the terminal, reads the corresponding operation status data from the database, and generates a response. The request sent from the terminal in step 5 is given as input, and appropriate operation status information is generated in JSON format as output. For example, data such as "Service suspension between Shinagawa and Yokohama due to typhoon" is generated in JSON format.

[0451] Step 7:

[0452] The terminal displays the information received from the server to the user. The JSON-formatted response data obtained in step 6 is given as input, and the output is displayed in a format that is easy for the user to read. Here, a UI component is used to display text such as "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0453] Step 8:

[0454] Users set their preferred language through apps or websites. The input is the user's language selection, and the output is the preference stored on the server. For example, if a native English speaker selects English, this preference is stored on the server and applied to future requests.

[0455] Step 9:

[0456] In the event of an emergency, the user will receive a push notification based on the notification conditions they set. The input is emergency notification information generated by the server, and the output is a push notification sent to the user's device. Specifically, the device will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0457] (Application example 1)

[0458] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0459] The purpose of this invention is to solve the problem of the lack of means to provide real-time multilingual operation information to users of self-driving vehicles in emergency situations such as natural disasters and traffic accidents, and in particular to enable foreign users to respond quickly and accurately in emergency situations.

[0460] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0461] In this invention, the server includes means for acquiring data from publicly available information sources on the Internet, means for analyzing the acquired data using natural language processing technology, means for translating the analyzed data into multiple languages ​​using a multilingual translation system, means for storing the translated data in a database, means for providing data in a specified language based on a user request, and means for displaying the acquired data on a dashboard or head-mounted display in the system of the autonomous vehicle. This enables users of the autonomous vehicle to acquire the latest operational information in multiple languages ​​even in an emergency and to take appropriate action promptly.

[0462] "Public internet resources" refers to websites and online services that are publicly accessible on the internet.

[0463] "Data" refers to elements that make up specific information or content, and in this case refers to things like public transportation operation information.

[0464] "Means of acquisition" refers to the technology or method for automatically collecting the required data from the source.

[0465] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and generate human language.

[0466] "Means of analysis" refers to the techniques and methods used to analyze acquired data and extract necessary information.

[0467] "Multilingual translation system" refers to any technology or service that translates from one language to another.

[0468] "Translation means" refers to techniques or methods for converting information written in one language into another language.

[0469] A "database" refers to a system that systematically stores large amounts of data and allows for efficient access, search, and management.

[0470] "Storage means" refers to the techniques and methods used to store the captured and translated data in a database.

[0471] A "request" refers to a request for specific information from a user.

[0472] "Preferred Language" refers to a particular language selected or set by a user.

[0473] "Means of providing" refers to the technology or method for appropriately displaying or transmitting the information required by the user.

[0474] An "autonomous vehicle" refers to a vehicle that can drive autonomously using artificial intelligence and sensor technology.

[0475] A "system" refers to an overall device or configuration that combines multiple technical elements and means to achieve a specific purpose.

[0476] "Dashboard" refers to a display device installed in the driver's seat of a vehicle that provides various information to the driver.

[0477] A "head-mounted display" refers to a display device worn on the head, and is a device for displaying information within the field of vision.

[0478] The present invention relates to a system for providing real-time operation information in multiple languages ​​to users of autonomous vehicles. Specific embodiments of the system are described below.

[0479] System Configuration

[0480] This system mainly consists of three elements: a server, a terminal, and a user.

[0481] 1. Server Processing

[0482] Data Acquisition

[0483] The server uses the Python scraping libraries "BeautifulSoup" and "Selenium" to retrieve operation information from the official websites of public transportation companies. For example, it retrieves text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specific official website.

[0484] Data analysis

[0485] The acquired text data is analyzed using natural language processing (NLP) techniques using Python's "spaCy" and "NLTK," which extracts important keywords such as place names and operation status.

[0486] Multilingual Translation

[0487] Information containing the analyzed keywords is translated into multiple languages ​​using the Google Translate API. For example, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0488] Data Storage

[0489] The translated information is stored in a relational database (e.g., MySQL or PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0490] 2. Terminal Processing

[0491] Data Acquisition and Display

[0492] The autonomous vehicle's terminal (dashboard or head-mounted display) sends API requests to the server based on the user's request. For example, if a user wants to check the "Tokaido Line operation status," the terminal will send a request such as " / api / route_status?line=tokaido" to the server. The server will read the corresponding operation status data from the database and return a response in the specified language. The terminal will then display this information in an easy-to-read format for the user.

[0493] 3. User Operation

[0494] Language settings and information confirmation

[0495] Users can set their preferred language on the app or vehicle dashboard. For example, a native English speaker can set English to check the latest service status. If they want to check service suspension information due to a typhoon, the device will display "Service suspension between Shinagawa and Yokohama due to typhoon."

[0496] Push notifications

[0497] In the event of an emergency, users will receive a push notification based on pre-set notification conditions. For example, if a service is suspended due to a typhoon, a notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama" will be sent to the user's device.

[0498] Examples and prompts

[0499] Specific examples

[0500] In the event of a natural disaster, the dashboard of the self-driving vehicle will display information such as, "Service between Tokyo Station and Shinjuku Station has been suspended due to an earthquake. Searching for an alternative route."

[0501] Prompt statement

[0502] An example prompt sent to a generative AI model is:

[0503] "Please provide the latest train service status between Tokyo Station and Shinjuku Station in English due to an earthquake."

[0504] This will enable users of autonomous vehicles to obtain the latest operational information in real time and take appropriate measures quickly.

[0505] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0506] Step 1:

[0507] Data acquisition (server)

[0508] The server periodically retrieves data from publicly available information sources on the Internet. Using Python's "BeautifulSoup" and "Selenium," it scrapes operation information from the official website of a public transportation company. For example, it retrieves text data from the official website stating, "Service suspended between Shinjuku Station and Tokyo Station due to earthquake." The input at this point is the URL of the official website and the HTML data to be retrieved, and the output is the text data of the operation information.

[0509] Step 2:

[0510] Data analysis (server)

[0511] The server analyzes the acquired text data using natural language processing (NLP) techniques such as "spaCy" and "NLTK." The purpose of the analysis is to extract important keywords such as place names and operation status. The input is the text data acquired by scraping, and the output is a set of extracted keywords.

[0512] Step 3:

[0513] Multilingual translation (server)

[0514] The server uses the Google Translate API to translate information containing the analyzed keywords into multiple languages. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English results in "Service suspension between Shinjuku Station and Tokyo Station due to earthquake." The input is the extracted keywords and the original text data, and the output is the text data translated into each language.

[0515] Step 4:

[0516] Data storage (server)

[0517] The server stores the translated information in a relational database (MySQL or PostgreSQL). The database stores operation status data, route names, acquisition dates and times, translated text, etc. The input is multilingually translated text data, and the output is data records stored in the database.

[0518] Step 5:

[0519] Receiving user requests (terminal)

[0520] The terminal (dashboard or head-mounted display of an autonomous vehicle) receives the user's request. When the user wants to check traffic information, the terminal sends an API request to the server. For example, it sends a request such as " / api / route_status?line=tokaido". The input is the user's request information, and the output is the API request.

[0521] Step 6:

[0522] Data provision (server)

[0523] The server retrieves the corresponding operation status data from the database based on the received API request, and returns the operation information in the specified language as a response. The input is the API request, and the output is the text data of the operation status translated into the specified language (for example, "Service suspension between Shinagawa and Yokohama due to typhoon").

[0524] Step 7:

[0525] Information display (terminal)

[0526] The terminal receives the traffic information from the server and displays it in a format that is easy for the user to see. For example, it displays "Shinagawa - Yokohama: Service suspended due to typhoon" on the dashboard or head-mounted display. The input is the text data of the traffic information received from the server, and the output is a visual display for the user.

[0527] Step 8:

[0528] Emergency notification (terminal)

[0529] In an emergency, push notifications are sent based on pre-set notification conditions. For example, if a user has set up notifications for earthquakes, a push notification stating "Earthquake alert: Service suspended between Shinagawa and Yokohama" will be sent to the device. The input is emergency operation information and user settings, and the output is a push notification.

[0530] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0531] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. Specific examples of the present invention are described below.

[0532] Server Processing

[0533] The server periodically retrieves the latest operational status from the official website of the public transport company. Using Python's "BeautifulSoup" and "Selenium," it extracts text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0534] The acquired text information is analyzed using natural language processing (NLP) techniques, such as using Python's "spaCy" and "NLTK" to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0535] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0536] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0537] Terminal handling

[0538] When the device receives a user request, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[0539] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format is returned.

[0540] The device displays the information received from the server in a format that is easy for the user to see. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0541] Emotion engine processing

[0542] An emotion engine is built into the device to recognize the user's emotional state. The emotion engine uses facial recognition and voice analysis technology to analyze the user's emotions in real time. For example, if the emotional state indicates "tension" or "stress," that information is sent to the server.

[0543] Emergency Notification

[0544] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0545] Customize your information

[0546] The emotional engine adjusts information provision based on the user's emotional state. For example, if a user is emotionally stressed, important information will be presented in a prominent format. The user interface adjusts based on the emotional state, allowing users to quickly obtain the information they need most.

[0547] The above is a specific embodiment of the present invention. This system allows users to accurately grasp the operation status of public transportation regardless of their emotional state and respond quickly to emergencies. By incorporating an emotion engine, more detailed information can be provided to users, improving user satisfaction.

[0548] The processing flow will be explained below.

[0549] Step 1:

[0550] The server periodically accesses the official website of the public transport company and scrapes information about the operation status. Using Python's "BeautifulSoup" and "Selenium," it obtains text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0551] Step 2:

[0552] The server analyzes the acquired text data using natural language processing (NLP) techniques. Python's "spaCy" and "NLTK" are used to extract important keywords from the extracted text, such as place names (e.g., Shinjuku Station, Tokyo Station) and service status (e.g., service suspension).

[0553] Step 3:

[0554] The server translates the analyzed information using a multilingual translation system. For example, using the Google Translate API, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0555] Step 4:

[0556] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL) as records containing the operation status data, route name, acquisition date and time, and the translated text.

[0557] Step 5:

[0558] The device receives a request from the user. If the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0559] Step 6:

[0560] The server receives a request from the device, retrieves the corresponding data from the database, and uses an SQL query to extract the service information in the language specified in the request, returning data in JSON format, such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0561] Step 7:

[0562] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0563] Step 8:

[0564] The device's built-in emotion engine recognizes the user's emotional state. Using facial recognition and voice analysis technology, it analyzes emotions from the user's facial expressions and voice, recognizing emotional states such as "tension" or "stress."

[0565] Step 9:

[0566] The device then sends the recognized emotional state to the server. For example, if the user is nervous, that information is sent to the server.

[0567] Step 10:

[0568] The server adjusts the information provided based on the user's emotional state. For example, if the user is emotionally tense, it may emphasize the display of traffic information or send notifications more frequently.

[0569] Step 11:

[0570] The server sends notifications in emergencies. For example, if a service is suspended due to a typhoon, the server sends a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to the user based on the notification conditions set by the user in advance.

[0571] Step 12:

[0572] The device displays the received notification to the user. Based on the analysis results of the emotion engine, the notification is displayed in an appropriate manner according to the level of urgency, allowing the user to respond immediately.

[0573] The above are the specific processing steps of the present invention, which includes an emotion engine. This system allows users to receive appropriate information according to their emotional state, enabling them to respond quickly and appropriately in emergencies.

[0574] Example 2

[0575] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0576] Conventional public transportation operation status systems can provide information in multiple languages, but they have the problem of being unable to provide information flexibly according to the user's emotional state. Furthermore, even in emergencies, appropriate notifications that take the user's emotional state into consideration are not provided, resulting in stress and inconvenience for users. Therefore, there is a need for a system that provides more detailed information customized according to the user's emotional state.

[0577] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0578] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state using facial recognition technology and voice analysis technology, and means for customizing the information to be provided based on the user's emotional state. This makes it possible to provide information according to the user's emotional state and to send notifications at an appropriate time even in emergencies.

[0579] "Internet data sources" refers to any information resource accessible via the Internet, including, for example, websites, databases, and APIs.

[0580] "Natural language processing technology" refers to a set of technologies used by computers to understand, analyze, and generate human language, and examples include morphological analysis, keyword extraction, and sentiment analysis.

[0581] A "multilingual translation system" refers to a system that automatically translates text from one language into multiple other languages, for example using a machine translation engine or a translation API.

[0582] "Database" refers to a system that enables the efficient storage, retrieval, and management of data, and includes relational databases and key-value databases.

[0583] "Request" means a request a User makes to a System for a particular action or piece of information, including, for example, an API call or a search query.

[0584] "Facial recognition technology" refers to technology that uses a camera or other device to detect, identify, and analyze a person's face, and is used to determine the user's emotional state.

[0585] "Voice analysis technology" refers to technology that analyzes voice data and understands its content, and includes speech recognition, emotion recognition, and natural language understanding.

[0586] "Emotional state" refers to the emotional state the user is currently feeling, and examples include tension, stress, joy, etc.

[0587] "Customization" refers to adjusting the information provided and its presentation format according to the user's needs and emotional state.

[0588] "Notification" refers to an alert or message sent to a user based on a particular event or situation.

[0589] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. A specific embodiment of the present invention will be described in detail below.

[0590] Server processing and technologies used

[0591] The server retrieves the latest train status information from publicly available data sources on the Internet. To this end, the server uses web scraping libraries such as Python's "BeautifulSoup" and "Selenium" to analyze the content of web pages and extract specific train status information, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station."

[0592] The acquired text information is analyzed using natural language processing (NLP) technology using Python's "spaCy" and "NLTK," which allows important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" to be extracted.

[0593] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. The translated information is then stored in a relational database (e.g., MySQL or PostgreSQL) and includes operation status data, route names, acquisition dates and times, translated text, and other information.

[0594] Terminal processing and data provision

[0595] When the device receives a user request, it sends an API request to the server. For example, if the user wants to check the "Tokaido Line service status," the device sends the request " / api / route_status?line=tokaido" to the server. The server retrieves the corresponding data from the database in response to the received request and returns a response to the device in the specified language. The device then displays the received information in an easy-to-read format for the user, for example, "Shinagawa - Yokohama: Service suspended due to typhoon" on the smartphone app.

[0596] Emotion engine processing

[0597] This system incorporates an emotion engine that uses facial recognition and voice analysis technologies to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions and voice in real time through a camera and microphone to determine their emotional state, such as "tension" or "stress." This information is sent to a server, which then customizes the information provided based on the user's emotional state.

[0598] Emergency Notification

[0599] If the user has set up emergency notifications, the server will take into account the emergency information stored in the database and data from the emotion engine to send notifications at the appropriate time, for example, "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0600] Customize your information

[0601] An emotion engine tailors information delivery based on the user's emotional state. For example, if the user is feeling stressed, the system will deliver important information in a prominent format, allowing the user to quickly get the information they need most.

[0602] Examples and prompts

[0603] For example, when providing information about service suspensions due to a typhoon, users who are in a state of tension can be notified by displaying "Shinagawa - Yokohama: Service suspended due to typhoon" in red and large font.In addition, in an emergency, a "Typhoon alert: Service suspended between Shinagawa and Yokohama" notification can be sent immediately.

[0604] The generative AI model generates a detailed description of the system by providing the following prompt:

[0605] Please explain the specific processing steps and detailed behavior of each step in a public transportation status information system that incorporates an emotion engine. For example, if the user is feeling stressed, please explain in detail how information is provided based on that user's emotional state.

[0606] This system not only allows users to accurately and timely grasp the status of public transport, but also allows them to receive customized information according to their emotional state, reducing stress and enabling them to respond quickly. This system is expected to significantly improve user satisfaction.

[0607] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0608] Step 1: Obtaining the operation status of public transportation

[0609] The server retrieves the latest train operation status from publicly available data sources on the Internet. A specific URL (e.g., the URL of a public transportation company's official website) is used as input. Specific processing involves analyzing the content of the webpage using Python's "BeautifulSoup" and "Selenium" to extract text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station." The output is text information about the train operation status.

[0610] Step 2: Natural Language Processing (NLP) Analysis

[0611] The server analyzes the text information acquired in step 1 using natural language processing technology. The text information on train operation status is used as input. Specifically, it uses Python's "spaCy" or "NLTK" to extract important keywords (e.g., "Shinjuku Station," "Tokyo Station," "suspension") and outputs a list of the extracted keywords.

[0612] Step 3: Multilingual Translation

[0613] The server translates the information analyzed in step 2 using a multilingual translation system. A list of important keywords is used as input. Specifically, the server calls the Google Translate API or similar to translate the keywords into multiple languages. The translated information is obtained as output. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0614] Step 4: Store in the database

[0615] The server stores the translated information from step 3 in a database. The translated information is used as input. Specifically, the server stores the information in a relational database (e.g., MySQL, PostgreSQL). The output is the translated information stored in the database. The stored information includes operation status data, route name, acquisition date and time, translated text, etc.

[0616] Step 5: Receiving the user request

[0617] The terminal receives a request from the user. As input, it uses the request information entered by the user (e.g., "Tokaido Line operation status"). In concrete terms, the terminal assembles the request content into an API request and sends it to the server. As output, it obtains the API request sent to the server. For example, a request in the format " / api / route_status?line=tokaido" is sent.

[0618] Step 6: Generate and return the server response

[0619] The server retrieves the corresponding data from the database in response to the request received in step 5. The API request is used as input. Specifically, the server executes a database query to retrieve the corresponding data. The retrieved data is obtained as output. The retrieved data is converted to JSON format and returned to the terminal as a response. For example, the data returned is "Service suspension between Shinagawa and Yokohama due to typhoon."

[0620] Step 7: Receiving and Displaying the Server Response

[0621] The terminal receives the response from the server and displays it in a format that is easy for the user to view. As input, it uses the JSON data received from the server. Specifically, it analyzes the received data and displays it in the appropriate UI component. As output, it displays information in a format that is easy for the user to view. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0622] Step 8: Emotional state recognition by the emotion engine

[0623] The emotion engine recognizes the user's emotional state in real time through a camera and microphone. It uses facial images and voice data as input. Specifically, it uses facial recognition and voice analysis technologies to analyze the user's facial expressions and tone of voice, and estimates emotions such as "tension" or "stress." The output is data on the recognized emotional state.

[0624] Step 9: Tailor information delivery based on emotional state

[0625] The server customizes the information it provides based on the emotional state data received from the emotion engine. It uses the emotional state data as input. Specific processing involves adjusting the display format and priority of the information to highlight the information the user most desires. The customized information is obtained as output. For example, if the user is feeling stressed, important information is displayed in red or in a larger font.

[0626] Step 10: Send emergency notifications

[0627] The server sends a notification to the user when the conditions are met in an emergency. The input is emergency information stored in the database and data from the emotion engine. Specifically, the server generates a notification message according to the level of urgency and sends it to the user's device. The output is a notification to the user. For example, a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" is sent.

[0628] This system not only allows users to get accurate and timely information about public transportation operations, but also allows them to receive customized information based on their emotional state, reducing stress and enabling them to respond quickly.

[0629] (Application example 2)

[0630] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0631] Autonomous vehicles are required to provide real-time information on public transportation and road conditions, as well as information that takes into account the user's emotional state, thereby reducing stress and anxiety and achieving a comfortable driving experience. Conventional systems have difficulty meeting these requirements simultaneously, and more advanced responses are required. Therefore, it is necessary to realize a system that can quickly and appropriately provide the information desired by users and take effective measures in emergencies.

[0632] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0633] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state, and means for adjusting the format of information provision based on the emotion analysis results. This enables real-time information provision that takes the user's emotional state into consideration, and allows for the prompt provision of appropriate information even in emergencies, thereby reducing user stress and anxiety.

[0634] "Data source" refers to websites, databases, etc. that provide publicly available information on the Internet.

[0635] "Means of obtaining information" refers to the systems and processes that automatically collect the required information from data sources.

[0636] "Natural language processing technology" refers to the technology of analyzing text data, understanding its meaning, and extracting important information.

[0637] A "multilingual translation system" refers to a system that translates text written in one language into multiple other languages.

[0638] "Database" refers to a system for organizing and storing information that has been acquired, analyzed, and translated.

[0639] A "request" refers to an action in which a user requests information from a system.

[0640] "Means of providing information" refers to a system or process that returns the necessary information in an appropriate format based on the user's request.

[0641] "Emotional state" refers to a user's psychological state, such as stress or anxiety.

[0642] "Means for analyzing emotional state" refers to technologies and systems that analyze a user's emotional state from facial expressions, voice, etc.

[0643] "Means for adjusting the format of information provision" refers to a system that changes the display method and content of the information provided depending on the user's emotional state.

[0644] To implement the present invention, a system must be constructed based on the following steps.

[0645] Server configuration and processing

[0646] The server retrieves the latest train operation information from publicly available data sources on the Internet using web scraping tools such as "BeautifulSoup" and "Selenium," which use the Python programming language. For example, it extracts text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specified website.

[0647] The extracted text information is then analyzed using natural language processing (NLP) techniques, using Python libraries such as spaCy and NLTK to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0648] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0649] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0650] Terminal configuration and handling

[0651] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent. The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. The connected device (e.g., a smartphone) displays the information received from the server in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0652] Emotion engine processing

[0653] The emotion engine uses facial recognition and voice analysis technologies to provide users with comfortable information in real time. Specifically, it uses the Python "dlib" library for face detection and "DeepFace" for emotion analysis.

[0654] If the emotional state indicates "tension" or "stress," that state is sent to the server and the way the information is displayed changes. For example, if the user is feeling stressed, important information will be displayed in a more prominent format.

[0655] Emergency Notification

[0656] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0657] Usage example

[0658] When a user inquires about the Tokaido Line's service status using a smartphone, the device displays "Shinagawa - Yokohama: Service suspended due to typhoon." If the emotion engine detects stress in the user's facial expression, this information is highlighted and, if necessary, an emergency notification is sent.

[0659] Example prompts for generative AI models

[0660] "Please tell me the Tokaido Line operation status in multiple languages."

[0661] "Customize traffic notifications based on emotions."

[0662] In this way, the system of the present invention allows users to obtain real-time operational information that is adapted to their emotional state, enabling them to respond quickly in emergencies.

[0663] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0664] Step 1:

[0665] The server retrieves information from publicly available internet data sources, uses Python's BeautifulSoup and Selenium to extract text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake," and converts the text into a format that can be used in the next processing step.

[0666] Input: URL of the data source

[0667] Output: Extracted service information text data

[0668] Step 2:

[0669] The server analyzes the acquired information using natural language processing technology, using Python's "spaCy" and "NLTK" to extract important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" from the text information.

[0670] Input: Text data of operation information

[0671] Output: Parsed keywords

[0672] Step 3:

[0673] The server translates the analyzed information into multiple languages ​​using a multilingual translation system (e.g., Google Translate API). For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0674] Input: Parsed keyword

[0675] Output: Translated text data

[0676] Step 4:

[0677] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operation status data, route name, acquisition date and time, and translated text.

[0678] Input: Translated text data

[0679] Output: Data stored in the database

[0680] Step 5:

[0681] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[0682] Input: User request

[0683] Output: API request to the server

[0684] Step 6:

[0685] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" is returned in JSON format.

[0686] Input: API request

[0687] Output: JSON formatted service information

[0688] Step 7:

[0689] The device receives the information from the server and displays it in a user-friendly format. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0690] Input: Response from the server

[0691] Output: Traffic information displayed on the user's device

[0692] Step 8:

[0693] The server analyzes the user's emotional state in real time using an emotion engine, which uses facial recognition technology (Python's "dlib" or "DeepFace") and voice analysis technology to analyze the user's emotional state and transmits the results to the server.

[0694] Input: User's facial image and voice data

[0695] Output: Sentiment analysis results

[0696] Step 9:

[0697] The server adjusts the format of information delivery based on the user's emotional state: for example, if the user is stressed, important information will be displayed prominently and, if necessary, sent as an urgent notification.

[0698] Input: Sentiment analysis results, traffic information data

[0699] Output: Form of tailored information, emergency notification

[0700] The above are the specific processing steps for carrying out the present invention.

[0701] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0702] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0703] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0704] [Third embodiment]

[0705] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0706] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0707] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0708] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0709] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0710] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0711] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0712] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0713] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0714] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0715] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0716] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0717] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific examples of the present invention are described below.

[0718] Server Processing

[0719] The server periodically retrieves the latest operating status from the official websites of public transportation organizations. This is done using the Python scraping libraries "BeautifulSoup" and "Selenium." For example, it scrapes the operation information for a specific line from the official JR line website and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0720] The acquired text information is analyzed using natural language processing (NLP) techniques, specifically using Python's "spaCy" and "NLTK" to extract important keywords (e.g., place names, operation status) from the information.

[0721] The analyzed information is then translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0722] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[0723] Terminal handling

[0724] A device (such as a smartphone application or web browser) sends an API request to a server based on a user request. For example, if a user wants to check the "Tokaido Line operation status," the device sends a request such as " / api / route_status?line=tokaido" to the server.

[0725] The server reads the operation status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response may return data in JSON format such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0726] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0727] User Action

[0728] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0729] Users can check the latest train status in their preferred language. For example, if a user wants to check the "Tokaido Line train status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon."

[0730] In an emergency, users can receive push notifications based on their pre-set notification conditions. For example, if a service is suspended due to a typhoon, users who have set up notifications will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0731] The above is a specific embodiment of the present invention. This system allows foreign users to easily understand the operation status of public transportation in Japan in real time and in their own language, enabling them to respond quickly and appropriately in emergencies.

[0732] The processing flow will be explained below.

[0733] Step 1:

[0734] The server periodically accesses publicly available data sources on the Internet. Specifically, it accesses the official website of a public transportation company and extracts information about its operation status using Python's "BeautifulSoup" and "Selenium." For example, the server retrieves text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0735] Step 2:

[0736] The raw text data acquired by the server is analyzed using natural language processing (NLP). Specifically, Python's "spaCy" and "NLTK" are used to extract important keywords such as place names and service status from the text data. For example, information such as "Shinjuku Station," "Tokyo Station," and "service suspension" is analyzed and extracted.

[0737] Step 3:

[0738] The server translates the analyzed information using a multilingual translation system. Using a system such as the Google Translate API, the extracted information is translated into multiple languages. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0739] Step 4:

[0740] The server stores the translated information in a relational database, which stores information such as operation status data, route names, acquisition dates and times, and translated text. This allows operation information in each language to be stored in an organized manner.

[0741] Step 5:

[0742] The device receives a request from the user. For example, if the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0743] Step 6:

[0744] When the server receives a request from a device, it retrieves the corresponding data from the database. It uses an SQL query to retrieve the data and extracts the service information in the language specified in the request. For example, the server returns data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0745] Step 7:

[0746] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon" to help the user understand the situation.

[0747] Step 8:

[0748] The user sets up notifications in case of an emergency. For example, if the user wants to be notified of changes in service due to a typhoon, the user can turn on notifications in the app. The server stores this setting information and sends push notifications in case of an emergency according to the setting.

[0749] Step 9:

[0750] The server sends notifications in the event of an emergency. Based on pre-set conditions, for example, if service suspension information due to a typhoon is saved in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to registered users.

[0751] The above is a specific processing flow of the program according to the present invention.

[0752] Example 1

[0753] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0754] Understanding the status of public transportation is extremely important, especially during natural disasters, but foreigners face the challenge of quickly obtaining this information in their native language. Conventional systems lack multilingual support, real-time updates, and emergency notification functions, making it difficult for foreign users to take appropriate action.

[0755] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0756] In this invention, the server includes means for acquiring information from publicly available information sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a relational database, means for providing information in a specified language based on a user request, and means for displaying the information on the user's terminal, thereby enabling foreign users to understand the operation status of public transportation in real time in their own language.

[0757] "Publicly available internet sources" refers to information providers that are publicly accessible via the internet, such as official public transport websites or news sites.

[0758] "Natural language processing technology" is a computer technology for analyzing and classifying text data, extracting keywords, etc., and examples include morphological analysis and syntactic analysis.

[0759] A "multilingual translation system" is a system that automatically converts text written in one language into multiple other languages, and includes existing translation APIs and dedicated translation algorithms.

[0760] A "relational database" is a database system for storing and managing data based on a relational model, and allows information to be manipulated through queries using SQL.

[0761] "Request from the user" refers to a request for information acquisition or operation made by the user to the system via a terminal.

[0762] A "user device" is a device used by a user to receive and display information, such as a smartphone, tablet, or computer.

[0763] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific embodiments for carrying out the present invention are described below.

[0764] Server Processing

[0765] The server periodically retrieves the latest operation status from the official websites of public transportation organizations. This process uses the Python scraping libraries "BeautifulSoup" and "Selenium." For example, the server scrapes operation information for a specific line from the official website of a railway company and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0766] The acquired text information is analyzed using natural language processing (NLP) technology. Specifically, Python's "spaCy" or "NLTK" is used to extract important keywords from the information (e.g., place names, service status). The server then translates the analyzed information into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0767] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[0768] Terminal handling

[0769] A device (for example, a smartphone application or web browser) sends an API request to a server based on a user request. If a user wants to check the "Tokaido Line service status," the device sends a request such as " / api / route_status?line=tokaido" to the server. The server reads the service status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response could return data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[0770] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[0771] User Action

[0772] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[0773] Users can check the latest service status in their preferred language. For example, if a user wants to check the "Tokaido Line service status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon." In an emergency, users can receive push notifications based on the notification conditions they have set up in advance. For example, a user who has set up notifications for service suspensions due to typhoons will receive a push notification on their device stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0774] In this way, the present invention enables foreign users to easily grasp the operation status of public transportation in Japan in real time and in their own native language, enabling them to respond quickly and appropriately in emergencies.

[0775] Specific examples

[0776] The server retrieves information from the official website of a railway company, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station," and uses Python's "spaCy" to extract the keywords "Shinjuku Station," "Tokyo Station," "earthquake," and "service suspended." It then translates this information into English using the Google Translate API and stores it in a relational database.

[0777] When a user checks the "Tokaido Line operation status" on a smartphone app, the server reads the information from the database and returns a response in English saying, "Service suspension between Shinagawa and Yokohama due to typhoon." The user's device then outputs this information on the display screen so that the user can check it.

[0778] An example of a specific prompt is, "Write Python code for a system that scrapes public transport status and translates it into multiple languages. Include code that parses the HTML using BeautifulSoup and Selenium, performs NLP analysis with spaCy, translates the results into multiple languages ​​using the Google Translate API, and stores them in a relational database."

[0779] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0780] Step 1:

[0781] The server periodically retrieves operation status data from publicly available information sources on the Internet. Specifically, it uses Python's "BeautifulSoup" and "Selenium" to scrape the official homepages of public transportation agencies. The URL of the public transportation agency is given as input, and HTML data is obtained as output. For example, text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station" is obtained.

[0782] Step 2:

[0783] The server analyzes the acquired HTML data using natural language processing technology. Here, "spaCy" and "NLTK" are used. The text information obtained in step 1 is given as input, and important keywords (e.g., place names, service status) are extracted as output. Specifically, keywords such as "Shinjuku Station," "Tokyo Station," "earthquake," and "suspension of service" are extracted from the text.

[0784] Step 3:

[0785] The server translates the analyzed keyword information using a multilingual translation system. This process is performed using the Google Translate API or similar. The keyword information obtained in step 2 is given as input, and text translated into multiple languages ​​is obtained as output. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English yields the result "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0786] Step 4:

[0787] The server stores the translated information in a relational database. Here, "MySQL" or "PostgreSQL" is used. The translated text obtained in step 3 is given as input, and is saved in the database as output. The database stores operation status data, route name, acquisition date and time, and translated text. Specifically, the translated text is inserted into the database using an SQL query.

[0788] Step 5:

[0789] The device sends an API request to the server based on the user's request. The user's request data is given as input, and the request to be sent to the server is formed as output. For example, if a user wants to check the "Tokaido Line operation status," the device sends the request " / api / route_status?line=tokaido" to the server.

[0790] Step 6:

[0791] The server receives the request from the terminal, reads the corresponding operation status data from the database, and generates a response. The request sent from the terminal in step 5 is given as input, and appropriate operation status information is generated in JSON format as output. For example, data such as "Service suspension between Shinagawa and Yokohama due to typhoon" is generated in JSON format.

[0792] Step 7:

[0793] The terminal displays the information received from the server to the user. The JSON-formatted response data obtained in step 6 is given as input, and the output is displayed in a format that is easy for the user to read. Here, a UI component is used to display text such as "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0794] Step 8:

[0795] Users set their preferred language through apps or websites. The input is the user's language selection, and the output is the preference stored on the server. For example, if a native English speaker selects English, this preference is stored on the server and applied to future requests.

[0796] Step 9:

[0797] In the event of an emergency, the user will receive a push notification based on the notification conditions they set. The input is emergency notification information generated by the server, and the output is a push notification sent to the user's device. Specifically, the device will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0798] (Application example 1)

[0799] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0800] The purpose of this invention is to solve the problem of the lack of means to provide real-time multilingual operation information to users of self-driving vehicles in emergency situations such as natural disasters and traffic accidents, and in particular to enable foreign users to respond quickly and accurately in emergency situations.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0802] In this invention, the server includes means for acquiring data from publicly available information sources on the Internet, means for analyzing the acquired data using natural language processing technology, means for translating the analyzed data into multiple languages ​​using a multilingual translation system, means for storing the translated data in a database, means for providing data in a specified language based on a user request, and means for displaying the acquired data on a dashboard or head-mounted display in the system of the autonomous vehicle. This enables users of the autonomous vehicle to acquire the latest operational information in multiple languages ​​even in an emergency and to take appropriate action promptly.

[0803] "Public internet resources" refers to websites and online services that are publicly accessible on the internet.

[0804] "Data" refers to elements that make up specific information or content, and in this case refers to things like public transportation operation information.

[0805] "Means of acquisition" refers to the technology or method for automatically collecting the required data from the source.

[0806] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and generate human language.

[0807] "Means of analysis" refers to the techniques and methods used to analyze acquired data and extract necessary information.

[0808] "Multilingual translation system" refers to any technology or service that translates from one language to another.

[0809] "Translation means" refers to techniques or methods for converting information written in one language into another language.

[0810] A "database" refers to a system that systematically stores large amounts of data and allows for efficient access, search, and management.

[0811] "Storage means" refers to the techniques and methods used to store the captured and translated data in a database.

[0812] A "request" refers to a request for specific information from a user.

[0813] "Preferred Language" refers to a particular language selected or set by a user.

[0814] "Means of providing" refers to the technology or method for appropriately displaying or transmitting the information required by the user.

[0815] An "autonomous vehicle" refers to a vehicle that can drive autonomously using artificial intelligence and sensor technology.

[0816] A "system" refers to an overall device or configuration that combines multiple technical elements and means to achieve a specific purpose.

[0817] "Dashboard" refers to a display device installed in the driver's seat of a vehicle that provides various information to the driver.

[0818] A "head-mounted display" refers to a display device worn on the head, and is a device for displaying information within the field of vision.

[0819] The present invention relates to a system for providing real-time operation information in multiple languages ​​to users of autonomous vehicles. Specific embodiments of the system are described below.

[0820] System Configuration

[0821] This system mainly consists of three elements: a server, a terminal, and a user.

[0822] 1. Server Processing

[0823] Data Acquisition

[0824] The server uses the Python scraping libraries "BeautifulSoup" and "Selenium" to retrieve operation information from the official websites of public transportation companies. For example, it retrieves text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specific official website.

[0825] Data analysis

[0826] The acquired text data is analyzed using natural language processing (NLP) techniques using Python's "spaCy" and "NLTK," which extracts important keywords such as place names and operation status.

[0827] Multilingual Translation

[0828] Information containing the analyzed keywords is translated into multiple languages ​​using the Google Translate API. For example, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0829] Data Storage

[0830] The translated information is stored in a relational database (e.g., MySQL or PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0831] 2. Terminal Processing

[0832] Data Acquisition and Display

[0833] The autonomous vehicle's terminal (dashboard or head-mounted display) sends API requests to the server based on the user's request. For example, if a user wants to check the "Tokaido Line operation status," the terminal will send a request such as " / api / route_status?line=tokaido" to the server. The server will read the corresponding operation status data from the database and return a response in the specified language. The terminal will then display this information in an easy-to-read format for the user.

[0834] 3. User Operation

[0835] Language settings and information confirmation

[0836] Users can set their preferred language on the app or vehicle dashboard. For example, a native English speaker can set English to check the latest service status. If they want to check service suspension information due to a typhoon, the device will display "Service suspension between Shinagawa and Yokohama due to typhoon."

[0837] Push notifications

[0838] In the event of an emergency, users will receive a push notification based on pre-set notification conditions. For example, if a service is suspended due to a typhoon, a notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama" will be sent to the user's device.

[0839] Examples and prompts

[0840] Specific examples

[0841] In the event of a natural disaster, the dashboard of the self-driving vehicle will display information such as, "Service between Tokyo Station and Shinjuku Station has been suspended due to an earthquake. Searching for an alternative route."

[0842] Prompt statement

[0843] An example prompt sent to a generative AI model is:

[0844] "Please provide the latest train service status between Tokyo Station and Shinjuku Station in English due to an earthquake."

[0845] This will enable users of autonomous vehicles to obtain the latest operational information in real time and take appropriate measures quickly.

[0846] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0847] Step 1:

[0848] Data acquisition (server)

[0849] The server periodically retrieves data from publicly available information sources on the Internet. Using Python's "BeautifulSoup" and "Selenium," it scrapes operation information from the official website of a public transportation company. For example, it retrieves text data from the official website stating, "Service suspended between Shinjuku Station and Tokyo Station due to earthquake." The input at this point is the URL of the official website and the HTML data to be retrieved, and the output is the text data of the operation information.

[0850] Step 2:

[0851] Data analysis (server)

[0852] The server analyzes the acquired text data using natural language processing (NLP) techniques such as "spaCy" and "NLTK." The purpose of the analysis is to extract important keywords such as place names and operation status. The input is the text data acquired by scraping, and the output is a set of extracted keywords.

[0853] Step 3:

[0854] Multilingual translation (server)

[0855] The server uses the Google Translate API to translate information containing the analyzed keywords into multiple languages. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English results in "Service suspension between Shinjuku Station and Tokyo Station due to earthquake." The input is the extracted keywords and the original text data, and the output is the text data translated into each language.

[0856] Step 4:

[0857] Data storage (server)

[0858] The server stores the translated information in a relational database (MySQL or PostgreSQL). The database stores operation status data, route names, acquisition dates and times, translated text, etc. The input is multilingually translated text data, and the output is data records stored in the database.

[0859] Step 5:

[0860] Receiving user requests (terminal)

[0861] The terminal (dashboard or head-mounted display of an autonomous vehicle) receives the user's request. When the user wants to check traffic information, the terminal sends an API request to the server. For example, it sends a request such as " / api / route_status?line=tokaido". The input is the user's request information, and the output is the API request.

[0862] Step 6:

[0863] Data provision (server)

[0864] The server retrieves the corresponding operation status data from the database based on the received API request, and returns the operation information in the specified language as a response. The input is the API request, and the output is the text data of the operation status translated into the specified language (for example, "Service suspension between Shinagawa and Yokohama due to typhoon").

[0865] Step 7:

[0866] Information display (terminal)

[0867] The terminal receives the traffic information from the server and displays it in a format that is easy for the user to see. For example, it displays "Shinagawa - Yokohama: Service suspended due to typhoon" on the dashboard or head-mounted display. The input is the text data of the traffic information received from the server, and the output is a visual display for the user.

[0868] Step 8:

[0869] Emergency notification (terminal)

[0870] In an emergency, push notifications are sent based on pre-set notification conditions. For example, if a user has set up notifications for earthquakes, a push notification stating "Earthquake alert: Service suspended between Shinagawa and Yokohama" will be sent to the device. The input is emergency operation information and user settings, and the output is a push notification.

[0871] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0872] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. Specific examples of the present invention are described below.

[0873] Server Processing

[0874] The server periodically retrieves the latest operational status from the official website of the public transport company. Using Python's "BeautifulSoup" and "Selenium," it extracts text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0875] The acquired text information is analyzed using natural language processing (NLP) techniques, such as using Python's "spaCy" and "NLTK" to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0876] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0877] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0878] Terminal handling

[0879] When the device receives a user request, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[0880] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format is returned.

[0881] The device displays the information received from the server in a format that is easy for the user to see. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0882] Emotion engine processing

[0883] An emotion engine is built into the device to recognize the user's emotional state. The emotion engine uses facial recognition and voice analysis technology to analyze the user's emotions in real time. For example, if the emotional state indicates "tension" or "stress," that information is sent to the server.

[0884] Emergency Notification

[0885] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0886] Customize your information

[0887] The emotional engine adjusts information provision based on the user's emotional state. For example, if a user is emotionally stressed, important information will be presented in a prominent format. The user interface adjusts based on the emotional state, allowing users to quickly obtain the information they need most.

[0888] The above is a specific embodiment of the present invention. This system allows users to accurately grasp the operation status of public transportation regardless of their emotional state and respond quickly to emergencies. By incorporating an emotion engine, more detailed information can be provided to users, improving user satisfaction.

[0889] The processing flow will be explained below.

[0890] Step 1:

[0891] The server periodically accesses the official website of the public transport company and scrapes information about the operation status. Using Python's "BeautifulSoup" and "Selenium," it obtains text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[0892] Step 2:

[0893] The server analyzes the acquired text data using natural language processing (NLP) techniques. Python's "spaCy" and "NLTK" are used to extract important keywords from the extracted text, such as place names (e.g., Shinjuku Station, Tokyo Station) and service status (e.g., service suspension).

[0894] Step 3:

[0895] The server translates the analyzed information using a multilingual translation system. For example, using the Google Translate API, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0896] Step 4:

[0897] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL) as records containing the operation status data, route name, acquisition date and time, and the translated text.

[0898] Step 5:

[0899] The device receives a request from the user. If the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[0900] Step 6:

[0901] The server receives a request from the device, retrieves the corresponding data from the database, and uses an SQL query to extract the service information in the language specified in the request, returning data in JSON format, such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[0902] Step 7:

[0903] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[0904] Step 8:

[0905] The device's built-in emotion engine recognizes the user's emotional state. Using facial recognition and voice analysis technology, it analyzes emotions from the user's facial expressions and voice, recognizing emotional states such as "tension" or "stress."

[0906] Step 9:

[0907] The device then sends the recognized emotional state to the server. For example, if the user is nervous, that information is sent to the server.

[0908] Step 10:

[0909] The server adjusts the information provided based on the user's emotional state. For example, if the user is emotionally tense, it may emphasize the display of traffic information or send notifications more frequently.

[0910] Step 11:

[0911] The server sends notifications in emergencies. For example, if a service is suspended due to a typhoon, the server sends a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to the user based on the notification conditions set by the user in advance.

[0912] Step 12:

[0913] The device displays the received notification to the user. Based on the analysis results of the emotion engine, the notification is displayed in an appropriate manner according to the level of urgency, allowing the user to respond immediately.

[0914] The above are the specific processing steps of the present invention, which includes an emotion engine. This system allows users to receive appropriate information according to their emotional state, enabling them to respond quickly and appropriately in emergencies.

[0915] Example 2

[0916] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0917] Conventional public transportation operation status systems can provide information in multiple languages, but they have the problem of being unable to provide information flexibly according to the user's emotional state. Furthermore, even in emergencies, appropriate notifications that take the user's emotional state into consideration are not provided, resulting in stress and inconvenience for users. Therefore, there is a need for a system that provides more detailed information customized according to the user's emotional state.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0919] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state using facial recognition technology and voice analysis technology, and means for customizing the information to be provided based on the user's emotional state. This makes it possible to provide information according to the user's emotional state and to send notifications at an appropriate time even in emergencies.

[0920] "Internet data sources" refers to any information resource accessible via the Internet, including, for example, websites, databases, and APIs.

[0921] "Natural language processing technology" refers to a set of technologies used by computers to understand, analyze, and generate human language, and examples include morphological analysis, keyword extraction, and sentiment analysis.

[0922] A "multilingual translation system" refers to a system that automatically translates text from one language into multiple other languages, for example using a machine translation engine or a translation API.

[0923] "Database" refers to a system that enables the efficient storage, retrieval, and management of data, and includes relational databases and key-value databases.

[0924] "Request" means a request a User makes to a System for a particular action or piece of information, including, for example, an API call or a search query.

[0925] "Facial recognition technology" refers to technology that uses a camera or other device to detect, identify, and analyze a person's face, and is used to determine the user's emotional state.

[0926] "Voice analysis technology" refers to technology that analyzes voice data and understands its content, and includes speech recognition, emotion recognition, and natural language understanding.

[0927] "Emotional state" refers to the emotional state the user is currently feeling, and examples include tension, stress, joy, etc.

[0928] "Customization" refers to adjusting the information provided and its presentation format according to the user's needs and emotional state.

[0929] "Notification" refers to an alert or message sent to a user based on a particular event or situation.

[0930] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. A specific embodiment of the present invention will be described in detail below.

[0931] Server processing and technologies used

[0932] The server retrieves the latest train status information from publicly available data sources on the Internet. To this end, the server uses web scraping libraries such as Python's "BeautifulSoup" and "Selenium" to analyze the content of web pages and extract specific train status information, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station."

[0933] The acquired text information is analyzed using natural language processing (NLP) technology using Python's "spaCy" and "NLTK," which allows important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" to be extracted.

[0934] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. The translated information is then stored in a relational database (e.g., MySQL or PostgreSQL) and includes operation status data, route names, acquisition dates and times, translated text, and other information.

[0935] Terminal processing and data provision

[0936] When the device receives a user request, it sends an API request to the server. For example, if the user wants to check the "Tokaido Line service status," the device sends the request " / api / route_status?line=tokaido" to the server. The server retrieves the corresponding data from the database in response to the received request and returns a response to the device in the specified language. The device then displays the received information in an easy-to-read format for the user, for example, "Shinagawa - Yokohama: Service suspended due to typhoon" on the smartphone app.

[0937] Emotion engine processing

[0938] This system incorporates an emotion engine that uses facial recognition and voice analysis technologies to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions and voice in real time through a camera and microphone to determine their emotional state, such as "tension" or "stress." This information is sent to a server, which then customizes the information provided based on the user's emotional state.

[0939] Emergency Notification

[0940] If the user has set up emergency notifications, the server will take into account the emergency information stored in the database and data from the emotion engine to send notifications at the appropriate time, for example, "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[0941] Customize your information

[0942] An emotion engine tailors information delivery based on the user's emotional state. For example, if the user is feeling stressed, the system will deliver important information in a prominent format, allowing the user to quickly get the information they need most.

[0943] Examples and prompts

[0944] For example, when providing information about service suspensions due to a typhoon, users who are in a state of tension can be notified by displaying "Shinagawa - Yokohama: Service suspended due to typhoon" in red and large font.In addition, in an emergency, a "Typhoon alert: Service suspended between Shinagawa and Yokohama" notification can be sent immediately.

[0945] The generative AI model generates a detailed description of the system by providing the following prompt:

[0946] Please explain the specific processing steps and detailed behavior of each step in a public transportation status information system that incorporates an emotion engine. For example, if the user is feeling stressed, please explain in detail how information is provided based on that user's emotional state.

[0947] This system not only allows users to accurately and timely grasp the status of public transport, but also allows them to receive customized information according to their emotional state, reducing stress and enabling them to respond quickly. This system is expected to significantly improve user satisfaction.

[0948] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0949] Step 1: Obtaining the operation status of public transportation

[0950] The server retrieves the latest train operation status from publicly available data sources on the Internet. A specific URL (e.g., the URL of a public transportation company's official website) is used as input. Specific processing involves analyzing the content of the webpage using Python's "BeautifulSoup" and "Selenium" to extract text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station." The output is text information about the train operation status.

[0951] Step 2: Natural Language Processing (NLP) Analysis

[0952] The server analyzes the text information acquired in step 1 using natural language processing technology. The text information on train operation status is used as input. Specifically, it uses Python's "spaCy" or "NLTK" to extract important keywords (e.g., "Shinjuku Station," "Tokyo Station," "suspension") and outputs a list of the extracted keywords.

[0953] Step 3: Multilingual Translation

[0954] The server translates the information analyzed in step 2 using a multilingual translation system. A list of important keywords is used as input. Specifically, the server calls the Google Translate API or similar to translate the keywords into multiple languages. The translated information is obtained as output. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0955] Step 4: Store in the database

[0956] The server stores the translated information from step 3 in a database. The translated information is used as input. Specifically, the server stores the information in a relational database (e.g., MySQL, PostgreSQL). The output is the translated information stored in the database. The stored information includes operation status data, route name, acquisition date and time, translated text, etc.

[0957] Step 5: Receiving the user request

[0958] The terminal receives a request from the user. As input, it uses the request information entered by the user (e.g., "Tokaido Line operation status"). In concrete terms, the terminal assembles the request content into an API request and sends it to the server. As output, it obtains the API request sent to the server. For example, a request in the format " / api / route_status?line=tokaido" is sent.

[0959] Step 6: Generate and return the server response

[0960] The server retrieves the corresponding data from the database in response to the request received in step 5. The API request is used as input. Specifically, the server executes a database query to retrieve the corresponding data. The retrieved data is obtained as output. The retrieved data is converted to JSON format and returned to the terminal as a response. For example, the data returned is "Service suspension between Shinagawa and Yokohama due to typhoon."

[0961] Step 7: Receiving and Displaying the Server Response

[0962] The terminal receives the response from the server and displays it in a format that is easy for the user to view. As input, it uses the JSON data received from the server. Specifically, it analyzes the received data and displays it in the appropriate UI component. As output, it displays information in a format that is easy for the user to view. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0963] Step 8: Emotional state recognition by the emotion engine

[0964] The emotion engine recognizes the user's emotional state in real time through a camera and microphone. It uses facial images and voice data as input. Specifically, it uses facial recognition and voice analysis technologies to analyze the user's facial expressions and tone of voice, and estimates emotions such as "tension" or "stress." The output is data on the recognized emotional state.

[0965] Step 9: Tailor information delivery based on emotional state

[0966] The server customizes the information it provides based on the emotional state data received from the emotion engine. It uses the emotional state data as input. Specific processing involves adjusting the display format and priority of the information to highlight the information the user most desires. The customized information is obtained as output. For example, if the user is feeling stressed, important information is displayed in red or in a larger font.

[0967] Step 10: Send emergency notifications

[0968] The server sends a notification to the user when the conditions are met in an emergency. The input is emergency information stored in the database and data from the emotion engine. Specifically, the server generates a notification message according to the level of urgency and sends it to the user's device. The output is a notification to the user. For example, a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" is sent.

[0969] This system not only allows users to get accurate and timely information about public transportation operations, but also allows them to receive customized information based on their emotional state, reducing stress and enabling them to respond quickly.

[0970] (Application example 2)

[0971] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0972] Autonomous vehicles are required to provide real-time information on public transportation and road conditions, as well as information that takes into account the user's emotional state, thereby reducing stress and anxiety and achieving a comfortable driving experience. Conventional systems have difficulty meeting these requirements simultaneously, and more advanced responses are required. Therefore, it is necessary to realize a system that can quickly and appropriately provide the information desired by users and take effective measures in emergencies.

[0973] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0974] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state, and means for adjusting the format of information provision based on the emotion analysis results. This enables real-time information provision that takes the user's emotional state into consideration, and allows for the prompt provision of appropriate information even in emergencies, thereby reducing user stress and anxiety.

[0975] "Data source" refers to websites, databases, etc. that provide publicly available information on the Internet.

[0976] "Means of obtaining information" refers to the systems and processes that automatically collect the required information from data sources.

[0977] "Natural language processing technology" refers to the technology of analyzing text data, understanding its meaning, and extracting important information.

[0978] A "multilingual translation system" refers to a system that translates text written in one language into multiple other languages.

[0979] "Database" refers to a system for organizing and storing information that has been acquired, analyzed, and translated.

[0980] A "request" refers to an action in which a user requests information from a system.

[0981] "Means of providing information" refers to a system or process that returns the necessary information in an appropriate format based on the user's request.

[0982] "Emotional state" refers to a user's psychological state, such as stress or anxiety.

[0983] "Means for analyzing emotional state" refers to technologies and systems that analyze a user's emotional state from facial expressions, voice, etc.

[0984] "Means for adjusting the format of information provision" refers to a system that changes the display method and content of the information provided depending on the user's emotional state.

[0985] To implement the present invention, a system must be constructed based on the following steps.

[0986] Server configuration and processing

[0987] The server retrieves the latest train operation information from publicly available data sources on the Internet using web scraping tools such as "BeautifulSoup" and "Selenium," which use the Python programming language. For example, it extracts text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specified website.

[0988] The extracted text information is then analyzed using natural language processing (NLP) techniques, using Python libraries such as spaCy and NLTK to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[0989] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[0990] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[0991] Terminal configuration and handling

[0992] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent. The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. The connected device (e.g., a smartphone) displays the information received from the server in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[0993] Emotion engine processing

[0994] The emotion engine uses facial recognition and voice analysis technologies to provide users with comfortable information in real time. Specifically, it uses the Python "dlib" library for face detection and "DeepFace" for emotion analysis.

[0995] If the emotional state indicates "tension" or "stress," that state is sent to the server and the way the information is displayed changes. For example, if the user is feeling stressed, important information will be displayed in a more prominent format.

[0996] Emergency Notification

[0997] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[0998] Usage example

[0999] When a user inquires about the Tokaido Line's service status using a smartphone, the device displays "Shinagawa - Yokohama: Service suspended due to typhoon." If the emotion engine detects stress in the user's facial expression, this information is highlighted and, if necessary, an emergency notification is sent.

[1000] Example prompts for generative AI models

[1001] "Please tell me the Tokaido Line operation status in multiple languages."

[1002] "Customize traffic notifications based on emotions."

[1003] In this way, the system of the present invention allows users to obtain real-time operational information that is adapted to their emotional state, enabling them to respond quickly in emergencies.

[1004] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1005] Step 1:

[1006] The server retrieves information from publicly available internet data sources, uses Python's BeautifulSoup and Selenium to extract text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake," and converts the text into a format that can be used in the next processing step.

[1007] Input: URL of the data source

[1008] Output: Extracted service information text data

[1009] Step 2:

[1010] The server analyzes the acquired information using natural language processing technology, using Python's "spaCy" and "NLTK" to extract important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" from the text information.

[1011] Input: Text data of operation information

[1012] Output: Parsed keywords

[1013] Step 3:

[1014] The server translates the analyzed information into multiple languages ​​using a multilingual translation system (e.g., Google Translate API). For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1015] Input: Parsed keyword

[1016] Output: Translated text data

[1017] Step 4:

[1018] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operation status data, route name, acquisition date and time, and translated text.

[1019] Input: Translated text data

[1020] Output: Data stored in the database

[1021] Step 5:

[1022] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[1023] Input: User request

[1024] Output: API request to the server

[1025] Step 6:

[1026] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" is returned in JSON format.

[1027] Input: API request

[1028] Output: JSON formatted service information

[1029] Step 7:

[1030] The device receives the information from the server and displays it in a user-friendly format. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[1031] Input: Response from the server

[1032] Output: Traffic information displayed on the user's device

[1033] Step 8:

[1034] The server analyzes the user's emotional state in real time using an emotion engine, which uses facial recognition technology (Python's "dlib" or "DeepFace") and voice analysis technology to analyze the user's emotional state and transmits the results to the server.

[1035] Input: User's facial image and voice data

[1036] Output: Sentiment analysis results

[1037] Step 9:

[1038] The server adjusts the format of information delivery based on the user's emotional state: for example, if the user is stressed, important information will be displayed prominently and, if necessary, sent as an urgent notification.

[1039] Input: Sentiment analysis results, traffic information data

[1040] Output: Form of tailored information, emergency notification

[1041] The above are the specific processing steps for carrying out the present invention.

[1042] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1043] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1044] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1045] [Fourth embodiment]

[1046] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1047] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1048] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1049] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1050] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1051] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1052] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1053] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1054] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1055] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1056] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1057] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1058] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1059] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific examples of the present invention are described below.

[1060] Server Processing

[1061] The server periodically retrieves the latest operating status from the official websites of public transportation organizations. This is done using the Python scraping libraries "BeautifulSoup" and "Selenium." For example, it scrapes the operation information for a specific line from the official JR line website and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[1062] The acquired text information is analyzed using natural language processing (NLP) techniques, specifically using Python's "spaCy" and "NLTK" to extract important keywords (e.g., place names, operation status) from the information.

[1063] The analyzed information is then translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1064] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[1065] Terminal handling

[1066] A device (such as a smartphone application or web browser) sends an API request to a server based on a user request. For example, if a user wants to check the "Tokaido Line operation status," the device sends a request such as " / api / route_status?line=tokaido" to the server.

[1067] The server reads the operation status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response may return data in JSON format such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[1068] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[1069] User Action

[1070] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[1071] Users can check the latest train status in their preferred language. For example, if a user wants to check the "Tokaido Line train status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon."

[1072] In an emergency, users can receive push notifications based on their pre-set notification conditions. For example, if a service is suspended due to a typhoon, users who have set up notifications will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[1073] The above is a specific embodiment of the present invention. This system allows foreign users to easily understand the operation status of public transportation in Japan in real time and in their own language, enabling them to respond quickly and appropriately in emergencies.

[1074] The processing flow will be explained below.

[1075] Step 1:

[1076] The server periodically accesses publicly available data sources on the Internet. Specifically, it accesses the official website of a public transportation company and extracts information about its operation status using Python's "BeautifulSoup" and "Selenium." For example, the server retrieves text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[1077] Step 2:

[1078] The raw text data acquired by the server is analyzed using natural language processing (NLP). Specifically, Python's "spaCy" and "NLTK" are used to extract important keywords such as place names and service status from the text data. For example, information such as "Shinjuku Station," "Tokyo Station," and "service suspension" is analyzed and extracted.

[1079] Step 3:

[1080] The server translates the analyzed information using a multilingual translation system. Using a system such as the Google Translate API, the extracted information is translated into multiple languages. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1081] Step 4:

[1082] The server stores the translated information in a relational database, which stores information such as operation status data, route names, acquisition dates and times, and translated text. This allows operation information in each language to be stored in an organized manner.

[1083] Step 5:

[1084] The device receives a request from the user. For example, if the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[1085] Step 6:

[1086] When the server receives a request from a device, it retrieves the corresponding data from the database. It uses an SQL query to retrieve the data and extracts the service information in the language specified in the request. For example, the server returns data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[1087] Step 7:

[1088] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon" to help the user understand the situation.

[1089] Step 8:

[1090] The user sets up notifications in case of an emergency. For example, if the user wants to be notified of changes in service due to a typhoon, the user can turn on notifications in the app. The server stores this setting information and sends push notifications in case of an emergency according to the setting.

[1091] Step 9:

[1092] The server sends notifications in the event of an emergency. Based on pre-set conditions, for example, if service suspension information due to a typhoon is saved in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to registered users.

[1093] The above is a specific processing flow of the program according to the present invention.

[1094] Example 1

[1095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1096] Understanding the status of public transportation is extremely important, especially during natural disasters, but foreigners face the challenge of quickly obtaining this information in their native language. Conventional systems lack multilingual support, real-time updates, and emergency notification functions, making it difficult for foreign users to take appropriate action.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1098] In this invention, the server includes means for acquiring information from publicly available information sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a relational database, means for providing information in a specified language based on a user request, and means for displaying the information on the user's terminal, thereby enabling foreign users to understand the operation status of public transportation in real time in their own language.

[1099] "Publicly available internet sources" refers to information providers that are publicly accessible via the internet, such as official public transport websites or news sites.

[1100] "Natural language processing technology" is a computer technology for analyzing and classifying text data, extracting keywords, etc., and examples include morphological analysis and syntactic analysis.

[1101] A "multilingual translation system" is a system that automatically converts text written in one language into multiple other languages, and includes existing translation APIs and dedicated translation algorithms.

[1102] A "relational database" is a database system for storing and managing data based on a relational model, and allows information to be manipulated through queries using SQL.

[1103] "Request from the user" refers to a request for information acquisition or operation made by the user to the system via a terminal.

[1104] A "user device" is a device used by a user to receive and display information, such as a smartphone, tablet, or computer.

[1105] The present invention relates to a system for providing information on the operation status of public transportation in multiple languages, and in particular to a technology for supporting foreign users in understanding operation information broadcast during natural disasters. Specific embodiments for carrying out the present invention are described below.

[1106] Server Processing

[1107] The server periodically retrieves the latest operation status from the official websites of public transportation organizations. This process uses the Python scraping libraries "BeautifulSoup" and "Selenium." For example, the server scrapes operation information for a specific line from the official website of a railway company and obtains text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[1108] The acquired text information is analyzed using natural language processing (NLP) technology. Specifically, Python's "spaCy" or "NLTK" is used to extract important keywords from the information (e.g., place names, service status). The server then translates the analyzed information into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1109] This translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route names, acquisition dates and times, and translated text.

[1110] Terminal handling

[1111] A device (for example, a smartphone application or web browser) sends an API request to a server based on a user request. If a user wants to check the "Tokaido Line service status," the device sends a request such as " / api / route_status?line=tokaido" to the server. The server reads the service status data corresponding to the received request from the database and returns a response in the specified language. For example, the API response could return data such as "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format.

[1112] The device displays the information received from the server in a UI component that is easy for the user to see. For example, a smartphone app might display the text "Shinagawa - Yokohama: Service suspended due to typhoon."

[1113] User Action

[1114] Users can set their preferred language for apps and websites. For example, a native English speaker can set English in the app's settings screen. This preference information is stored on the server and applied whenever the user requests it.

[1115] Users can check the latest service status in their preferred language. For example, if a user wants to check the "Tokaido Line service status," the device will issue an API request and display the returned data in the form of "Service suspension between Shinagawa and Yokohama due to typhoon." In an emergency, users can receive push notifications based on the notification conditions they have set up in advance. For example, a user who has set up notifications for service suspensions due to typhoons will receive a push notification on their device stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[1116] In this way, the present invention enables foreign users to easily grasp the operation status of public transportation in Japan in real time and in their own native language, enabling them to respond quickly and appropriately in emergencies.

[1117] Specific examples

[1118] The server retrieves information from the official website of a railway company, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station," and uses Python's "spaCy" to extract the keywords "Shinjuku Station," "Tokyo Station," "earthquake," and "service suspended." It then translates this information into English using the Google Translate API and stores it in a relational database.

[1119] When a user checks the "Tokaido Line operation status" on a smartphone app, the server reads the information from the database and returns a response in English saying, "Service suspension between Shinagawa and Yokohama due to typhoon." The user's device then outputs this information on the display screen so that the user can check it.

[1120] An example of a specific prompt is, "Write Python code for a system that scrapes public transport status and translates it into multiple languages. Include code that parses the HTML using BeautifulSoup and Selenium, performs NLP analysis with spaCy, translates the results into multiple languages ​​using the Google Translate API, and stores them in a relational database."

[1121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1122] Step 1:

[1123] The server periodically retrieves operation status data from publicly available information sources on the Internet. Specifically, it uses Python's "BeautifulSoup" and "Selenium" to scrape the official homepages of public transportation agencies. The URL of the public transportation agency is given as input, and HTML data is obtained as output. For example, text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station" is obtained.

[1124] Step 2:

[1125] The server analyzes the acquired HTML data using natural language processing technology. Here, "spaCy" and "NLTK" are used. The text information obtained in step 1 is given as input, and important keywords (e.g., place names, service status) are extracted as output. Specifically, keywords such as "Shinjuku Station," "Tokyo Station," "earthquake," and "suspension of service" are extracted from the text.

[1126] Step 3:

[1127] The server translates the analyzed keyword information using a multilingual translation system. This process is performed using the Google Translate API or similar. The keyword information obtained in step 2 is given as input, and text translated into multiple languages ​​is obtained as output. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English yields the result "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1128] Step 4:

[1129] The server stores the translated information in a relational database. Here, "MySQL" or "PostgreSQL" is used. The translated text obtained in step 3 is given as input, and is saved in the database as output. The database stores operation status data, route name, acquisition date and time, and translated text. Specifically, the translated text is inserted into the database using an SQL query.

[1130] Step 5:

[1131] The device sends an API request to the server based on the user's request. The user's request data is given as input, and the request to be sent to the server is formed as output. For example, if a user wants to check the "Tokaido Line operation status," the device sends the request " / api / route_status?line=tokaido" to the server.

[1132] Step 6:

[1133] The server receives the request from the terminal, reads the corresponding operation status data from the database, and generates a response. The request sent from the terminal in step 5 is given as input, and appropriate operation status information is generated in JSON format as output. For example, data such as "Service suspension between Shinagawa and Yokohama due to typhoon" is generated in JSON format.

[1134] Step 7:

[1135] The terminal displays the information received from the server to the user. The JSON-formatted response data obtained in step 6 is given as input, and the output is displayed in a format that is easy for the user to read. Here, a UI component is used to display text such as "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[1136] Step 8:

[1137] Users set their preferred language through apps or websites. The input is the user's language selection, and the output is the preference stored on the server. For example, if a native English speaker selects English, this preference is stored on the server and applied to future requests.

[1138] Step 9:

[1139] In the event of an emergency, the user will receive a push notification based on the notification conditions they set. The input is emergency notification information generated by the server, and the output is a push notification sent to the user's device. Specifically, the device will receive a push notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[1140] (Application example 1)

[1141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1142] The purpose of this invention is to solve the problem of the lack of means to provide real-time multilingual operation information to users of self-driving vehicles in emergency situations such as natural disasters and traffic accidents, and in particular to enable foreign users to respond quickly and accurately in emergency situations.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1144] In this invention, the server includes means for acquiring data from publicly available information sources on the Internet, means for analyzing the acquired data using natural language processing technology, means for translating the analyzed data into multiple languages ​​using a multilingual translation system, means for storing the translated data in a database, means for providing data in a specified language based on a user request, and means for displaying the acquired data on a dashboard or head-mounted display in the system of the autonomous vehicle. This enables users of the autonomous vehicle to acquire the latest operational information in multiple languages ​​even in an emergency and to take appropriate action promptly.

[1145] "Public internet resources" refers to websites and online services that are publicly accessible on the internet.

[1146] "Data" refers to elements that make up specific information or content, and in this case refers to things like public transportation operation information.

[1147] "Means of acquisition" refers to the technology or method for automatically collecting the required data from the source.

[1148] "Natural language processing technology" refers to technology that enables computers to understand, analyze, and generate human language.

[1149] "Means of analysis" refers to the techniques and methods used to analyze acquired data and extract necessary information.

[1150] "Multilingual translation system" refers to any technology or service that translates from one language to another.

[1151] "Translation means" refers to techniques or methods for converting information written in one language into another language.

[1152] A "database" refers to a system that systematically stores large amounts of data and allows for efficient access, search, and management.

[1153] "Storage means" refers to the techniques and methods used to store the captured and translated data in a database.

[1154] A "request" refers to a request for specific information from a user.

[1155] "Preferred Language" refers to a particular language selected or set by a user.

[1156] "Means of providing" refers to the technology or method for appropriately displaying or transmitting the information required by the user.

[1157] An "autonomous vehicle" refers to a vehicle that can drive autonomously using artificial intelligence and sensor technology.

[1158] A "system" refers to an overall device or configuration that combines multiple technical elements and means to achieve a specific purpose.

[1159] "Dashboard" refers to a display device installed in the driver's seat of a vehicle that provides various information to the driver.

[1160] A "head-mounted display" refers to a display device worn on the head, and is a device for displaying information within the field of vision.

[1161] The present invention relates to a system for providing real-time operation information in multiple languages ​​to users of autonomous vehicles. Specific embodiments of the system are described below.

[1162] System Configuration

[1163] This system mainly consists of three elements: a server, a terminal, and a user.

[1164] 1. Server Processing

[1165] Data Acquisition

[1166] The server uses the Python scraping libraries "BeautifulSoup" and "Selenium" to retrieve operation information from the official websites of public transportation companies. For example, it retrieves text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specific official website.

[1167] Data analysis

[1168] The acquired text data is analyzed using natural language processing (NLP) techniques using Python's "spaCy" and "NLTK," which extracts important keywords such as place names and operation status.

[1169] Multilingual Translation

[1170] Information containing the analyzed keywords is translated into multiple languages ​​using the Google Translate API. For example, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into English as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1171] Data Storage

[1172] The translated information is stored in a relational database (e.g., MySQL or PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[1173] 2. Terminal Processing

[1174] Data Acquisition and Display

[1175] The autonomous vehicle's terminal (dashboard or head-mounted display) sends API requests to the server based on the user's request. For example, if a user wants to check the "Tokaido Line operation status," the terminal will send a request such as " / api / route_status?line=tokaido" to the server. The server will read the corresponding operation status data from the database and return a response in the specified language. The terminal will then display this information in an easy-to-read format for the user.

[1176] 3. User Operation

[1177] Language settings and information confirmation

[1178] Users can set their preferred language on the app or vehicle dashboard. For example, a native English speaker can set English to check the latest service status. If they want to check service suspension information due to a typhoon, the device will display "Service suspension between Shinagawa and Yokohama due to typhoon."

[1179] Push notifications

[1180] In the event of an emergency, users will receive a push notification based on pre-set notification conditions. For example, if a service is suspended due to a typhoon, a notification stating "Typhoon alert: Service suspended between Shinagawa and Yokohama" will be sent to the user's device.

[1181] Examples and prompts

[1182] Specific examples

[1183] In the event of a natural disaster, the dashboard of the self-driving vehicle will display information such as, "Service between Tokyo Station and Shinjuku Station has been suspended due to an earthquake. Searching for an alternative route."

[1184] Prompt statement

[1185] An example prompt sent to a generative AI model is:

[1186] "Please provide the latest train service status between Tokyo Station and Shinjuku Station in English due to an earthquake."

[1187] This will enable users of autonomous vehicles to obtain the latest operational information in real time and take appropriate measures quickly.

[1188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1189] Step 1:

[1190] Data acquisition (server)

[1191] The server periodically retrieves data from publicly available information sources on the Internet. Using Python's "BeautifulSoup" and "Selenium," it scrapes operation information from the official website of a public transportation company. For example, it retrieves text data from the official website stating, "Service suspended between Shinjuku Station and Tokyo Station due to earthquake." The input at this point is the URL of the official website and the HTML data to be retrieved, and the output is the text data of the operation information.

[1192] Step 2:

[1193] Data analysis (server)

[1194] The server analyzes the acquired text data using natural language processing (NLP) techniques such as "spaCy" and "NLTK." The purpose of the analysis is to extract important keywords such as place names and operation status. The input is the text data acquired by scraping, and the output is a set of extracted keywords.

[1195] Step 3:

[1196] Multilingual translation (server)

[1197] The server uses the Google Translate API to translate information containing the analyzed keywords into multiple languages. For example, translating the information "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" into English results in "Service suspension between Shinjuku Station and Tokyo Station due to earthquake." The input is the extracted keywords and the original text data, and the output is the text data translated into each language.

[1198] Step 4:

[1199] Data storage (server)

[1200] The server stores the translated information in a relational database (MySQL or PostgreSQL). The database stores operation status data, route names, acquisition dates and times, translated text, etc. The input is multilingually translated text data, and the output is data records stored in the database.

[1201] Step 5:

[1202] Receiving user requests (terminal)

[1203] The terminal (dashboard or head-mounted display of an autonomous vehicle) receives the user's request. When the user wants to check traffic information, the terminal sends an API request to the server. For example, it sends a request such as " / api / route_status?line=tokaido". The input is the user's request information, and the output is the API request.

[1204] Step 6:

[1205] Data provision (server)

[1206] The server retrieves the corresponding operation status data from the database based on the received API request, and returns the operation information in the specified language as a response. The input is the API request, and the output is the text data of the operation status translated into the specified language (for example, "Service suspension between Shinagawa and Yokohama due to typhoon").

[1207] Step 7:

[1208] Information display (terminal)

[1209] The terminal receives the traffic information from the server and displays it in a format that is easy for the user to see. For example, it displays "Shinagawa - Yokohama: Service suspended due to typhoon" on the dashboard or head-mounted display. The input is the text data of the traffic information received from the server, and the output is a visual display for the user.

[1210] Step 8:

[1211] Emergency notification (terminal)

[1212] In an emergency, push notifications are sent based on pre-set notification conditions. For example, if a user has set up notifications for earthquakes, a push notification stating "Earthquake alert: Service suspended between Shinagawa and Yokohama" will be sent to the device. The input is emergency operation information and user settings, and the output is a push notification.

[1213] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1214] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. Specific examples of the present invention are described below.

[1215] Server Processing

[1216] The server periodically retrieves the latest operational status from the official website of the public transport company. Using Python's "BeautifulSoup" and "Selenium," it extracts text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[1217] The acquired text information is analyzed using natural language processing (NLP) techniques, such as using Python's "spaCy" and "NLTK" to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[1218] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" can be translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1219] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[1220] Terminal handling

[1221] When the device receives a user request, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[1222] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" in JSON format is returned.

[1223] The device displays the information received from the server in a format that is easy for the user to see. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[1224] Emotion engine processing

[1225] An emotion engine is built into the device to recognize the user's emotional state. The emotion engine uses facial recognition and voice analysis technology to analyze the user's emotions in real time. For example, if the emotional state indicates "tension" or "stress," that information is sent to the server.

[1226] Emergency Notification

[1227] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[1228] Customize your information

[1229] The emotional engine adjusts information provision based on the user's emotional state. For example, if a user is emotionally stressed, important information will be presented in a prominent format. The user interface adjusts based on the emotional state, allowing users to quickly obtain the information they need most.

[1230] The above is a specific embodiment of the present invention. This system allows users to accurately grasp the operation status of public transportation regardless of their emotional state and respond quickly to emergencies. By incorporating an emotion engine, more detailed information can be provided to users, improving user satisfaction.

[1231] The processing flow will be explained below.

[1232] Step 1:

[1233] The server periodically accesses the official website of the public transport company and scrapes information about the operation status. Using Python's "BeautifulSoup" and "Selenium," it obtains text data such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake."

[1234] Step 2:

[1235] The server analyzes the acquired text data using natural language processing (NLP) techniques. Python's "spaCy" and "NLTK" are used to extract important keywords from the extracted text, such as place names (e.g., Shinjuku Station, Tokyo Station) and service status (e.g., service suspension).

[1236] Step 3:

[1237] The server translates the analyzed information using a multilingual translation system. For example, using the Google Translate API, information such as "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1238] Step 4:

[1239] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL) as records containing the operation status data, route name, acquisition date and time, and the translated text.

[1240] Step 5:

[1241] The device receives a request from the user. If the user wants to check the "Tokaido Line operation status," the device sends an API request such as " / api / route_status?line=tokaido" to the server.

[1242] Step 6:

[1243] The server receives a request from the device, retrieves the corresponding data from the database, and uses an SQL query to extract the service information in the language specified in the request, returning data in JSON format, such as "Service suspension between Shinagawa and Yokohama due to typhoon."

[1244] Step 7:

[1245] The device receives the response from the server and displays the acquired data in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon" on the screen.

[1246] Step 8:

[1247] The device's built-in emotion engine recognizes the user's emotional state. Using facial recognition and voice analysis technology, it analyzes emotions from the user's facial expressions and voice, recognizing emotional states such as "tension" or "stress."

[1248] Step 9:

[1249] The device then sends the recognized emotional state to the server. For example, if the user is nervous, that information is sent to the server.

[1250] Step 10:

[1251] The server adjusts the information provided based on the user's emotional state. For example, if the user is emotionally tense, it may emphasize the display of traffic information or send notifications more frequently.

[1252] Step 11:

[1253] The server sends notifications in emergencies. For example, if a service is suspended due to a typhoon, the server sends a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" to the user based on the notification conditions set by the user in advance.

[1254] Step 12:

[1255] The device displays the received notification to the user. Based on the analysis results of the emotion engine, the notification is displayed in an appropriate manner according to the level of urgency, allowing the user to respond immediately.

[1256] The above are the specific processing steps of the present invention, which includes an emotion engine. This system allows users to receive appropriate information according to their emotional state, enabling them to respond quickly and appropriately in emergencies.

[1257] Example 2

[1258] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1259] Conventional public transportation operation status systems can provide information in multiple languages, but they have the problem of being unable to provide information flexibly according to the user's emotional state. Furthermore, even in emergencies, appropriate notifications that take the user's emotional state into consideration are not provided, resulting in stress and inconvenience for users. Therefore, there is a need for a system that provides more detailed information customized according to the user's emotional state.

[1260] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1261] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state using facial recognition technology and voice analysis technology, and means for customizing the information to be provided based on the user's emotional state. This makes it possible to provide information according to the user's emotional state and to send notifications at an appropriate time even in emergencies.

[1262] "Internet data sources" refers to any information resource accessible via the Internet, including, for example, websites, databases, and APIs.

[1263] "Natural language processing technology" refers to a set of technologies used by computers to understand, analyze, and generate human language, and examples include morphological analysis, keyword extraction, and sentiment analysis.

[1264] A "multilingual translation system" refers to a system that automatically translates text from one language into multiple other languages, for example using a machine translation engine or a translation API.

[1265] "Database" refers to a system that enables the efficient storage, retrieval, and management of data, and includes relational databases and key-value databases.

[1266] "Request" means a request a User makes to a System for a particular action or piece of information, including, for example, an API call or a search query.

[1267] "Facial recognition technology" refers to technology that uses a camera or other device to detect, identify, and analyze a person's face, and is used to determine the user's emotional state.

[1268] "Voice analysis technology" refers to technology that analyzes voice data and understands its content, and includes speech recognition, emotion recognition, and natural language understanding.

[1269] "Emotional state" refers to the emotional state the user is currently feeling, and examples include tension, stress, joy, etc.

[1270] "Customization" refers to adjusting the information provided and its presentation format according to the user's needs and emotional state.

[1271] "Notification" refers to an alert or message sent to a user based on a particular event or situation.

[1272] The present invention relates to a technology for providing information according to a user's emotional state by adding an emotion engine to a system that provides the status of public transportation in multiple languages. A specific embodiment of the present invention will be described in detail below.

[1273] Server processing and technologies used

[1274] The server retrieves the latest train status information from publicly available data sources on the Internet. To this end, the server uses web scraping libraries such as Python's "BeautifulSoup" and "Selenium" to analyze the content of web pages and extract specific train status information, such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station."

[1275] The acquired text information is analyzed using natural language processing (NLP) technology using Python's "spaCy" and "NLTK," which allows important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" to be extracted.

[1276] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. The translated information is then stored in a relational database (e.g., MySQL or PostgreSQL) and includes operation status data, route names, acquisition dates and times, translated text, and other information.

[1277] Terminal processing and data provision

[1278] When the device receives a user request, it sends an API request to the server. For example, if the user wants to check the "Tokaido Line service status," the device sends the request " / api / route_status?line=tokaido" to the server. The server retrieves the corresponding data from the database in response to the received request and returns a response to the device in the specified language. The device then displays the received information in an easy-to-read format for the user, for example, "Shinagawa - Yokohama: Service suspended due to typhoon" on the smartphone app.

[1279] Emotion engine processing

[1280] This system incorporates an emotion engine that uses facial recognition and voice analysis technologies to recognize the user's emotional state. The emotion engine analyzes the user's facial expressions and voice in real time through a camera and microphone to determine their emotional state, such as "tension" or "stress." This information is sent to a server, which then customizes the information provided based on the user's emotional state.

[1281] Emergency Notification

[1282] If the user has set up emergency notifications, the server will take into account the emergency information stored in the database and data from the emotion engine to send notifications at the appropriate time, for example, "Typhoon alert: Service suspended between Shinagawa and Yokohama."

[1283] Customize your information

[1284] An emotion engine tailors information delivery based on the user's emotional state. For example, if the user is feeling stressed, the system will deliver important information in a prominent format, allowing the user to quickly get the information they need most.

[1285] Examples and prompts

[1286] For example, when providing information about service suspensions due to a typhoon, users who are in a state of tension can be notified by displaying "Shinagawa - Yokohama: Service suspended due to typhoon" in red and large font.In addition, in an emergency, a "Typhoon alert: Service suspended between Shinagawa and Yokohama" notification can be sent immediately.

[1287] The generative AI model generates a detailed description of the system by providing the following prompt:

[1288] Please explain the specific processing steps and detailed behavior of each step in a public transportation status information system that incorporates an emotion engine. For example, if the user is feeling stressed, please explain in detail how information is provided based on that user's emotional state.

[1289] This system not only allows users to accurately and timely grasp the status of public transport, but also allows them to receive customized information according to their emotional state, reducing stress and enabling them to respond quickly. This system is expected to significantly improve user satisfaction.

[1290] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1291] Step 1: Obtaining the operation status of public transportation

[1292] The server retrieves the latest train operation status from publicly available data sources on the Internet. A specific URL (e.g., the URL of a public transportation company's official website) is used as input. Specific processing involves analyzing the content of the webpage using Python's "BeautifulSoup" and "Selenium" to extract text information such as "Service suspended due to earthquake between Shinjuku Station and Tokyo Station." The output is text information about the train operation status.

[1293] Step 2: Natural Language Processing (NLP) Analysis

[1294] The server analyzes the text information acquired in step 1 using natural language processing technology. The text information on train operation status is used as input. Specifically, it uses Python's "spaCy" or "NLTK" to extract important keywords (e.g., "Shinjuku Station," "Tokyo Station," "suspension") and outputs a list of the extracted keywords.

[1295] Step 3: Multilingual Translation

[1296] The server translates the information analyzed in step 2 using a multilingual translation system. A list of important keywords is used as input. Specifically, the server calls the Google Translate API or similar to translate the keywords into multiple languages. The translated information is obtained as output. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1297] Step 4: Store in the database

[1298] The server stores the translated information from step 3 in a database. The translated information is used as input. Specifically, the server stores the information in a relational database (e.g., MySQL, PostgreSQL). The output is the translated information stored in the database. The stored information includes operation status data, route name, acquisition date and time, translated text, etc.

[1299] Step 5: Receiving the user request

[1300] The terminal receives a request from the user. As input, it uses the request information entered by the user (e.g., "Tokaido Line operation status"). In concrete terms, the terminal assembles the request content into an API request and sends it to the server. As output, it obtains the API request sent to the server. For example, a request in the format " / api / route_status?line=tokaido" is sent.

[1301] Step 6: Generate and return the server response

[1302] The server retrieves the corresponding data from the database in response to the request received in step 5. The API request is used as input. Specifically, the server executes a database query to retrieve the corresponding data. The retrieved data is obtained as output. The retrieved data is converted to JSON format and returned to the terminal as a response. For example, the data returned is "Service suspension between Shinagawa and Yokohama due to typhoon."

[1303] Step 7: Receiving and Displaying the Server Response

[1304] The terminal receives the response from the server and displays it in a format that is easy for the user to view. As input, it uses the JSON data received from the server. Specifically, it analyzes the received data and displays it in the appropriate UI component. As output, it displays information in a format that is easy for the user to view. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[1305] Step 8: Emotional state recognition by the emotion engine

[1306] The emotion engine recognizes the user's emotional state in real time through a camera and microphone. It uses facial images and voice data as input. Specifically, it uses facial recognition and voice analysis technologies to analyze the user's facial expressions and tone of voice, and estimates emotions such as "tension" or "stress." The output is data on the recognized emotional state.

[1307] Step 9: Tailor information delivery based on emotional state

[1308] The server customizes the information it provides based on the emotional state data received from the emotion engine. It uses the emotional state data as input. Specific processing involves adjusting the display format and priority of the information to highlight the information the user most desires. The customized information is obtained as output. For example, if the user is feeling stressed, important information is displayed in red or in a larger font.

[1309] Step 10: Send emergency notifications

[1310] The server sends a notification to the user when the conditions are met in an emergency. The input is emergency information stored in the database and data from the emotion engine. Specifically, the server generates a notification message according to the level of urgency and sends it to the user's device. The output is a notification to the user. For example, a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" is sent.

[1311] This system not only allows users to get accurate and timely information about public transportation operations, but also allows them to receive customized information based on their emotional state, reducing stress and enabling them to respond quickly.

[1312] (Application example 2)

[1313] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1314] Autonomous vehicles are required to provide real-time information on public transportation and road conditions, as well as information that takes into account the user's emotional state, thereby reducing stress and anxiety and achieving a comfortable driving experience. Conventional systems have difficulty meeting these requirements simultaneously, and more advanced responses are required. Therefore, it is necessary to realize a system that can quickly and appropriately provide the information desired by users and take effective measures in emergencies.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1316] In this invention, the server includes means for acquiring information from publicly available data sources on the Internet, means for analyzing the acquired information using natural language processing technology, means for translating the analyzed information into multiple languages ​​using a multilingual translation system, means for storing the translated information in a database, means for providing information in a specified language based on a user request, means for analyzing the user's emotional state, and means for adjusting the format of information provision based on the emotion analysis results. This enables real-time information provision that takes the user's emotional state into consideration, and allows for the prompt provision of appropriate information even in emergencies, thereby reducing user stress and anxiety.

[1317] "Data source" refers to websites, databases, etc. that provide publicly available information on the Internet.

[1318] "Means of obtaining information" refers to the systems and processes that automatically collect the required information from data sources.

[1319] "Natural language processing technology" refers to the technology of analyzing text data, understanding its meaning, and extracting important information.

[1320] A "multilingual translation system" refers to a system that translates text written in one language into multiple other languages.

[1321] "Database" refers to a system for organizing and storing information that has been acquired, analyzed, and translated.

[1322] A "request" refers to an action in which a user requests information from a system.

[1323] "Means of providing information" refers to a system or process that returns the necessary information in an appropriate format based on the user's request.

[1324] "Emotional state" refers to a user's psychological state, such as stress or anxiety.

[1325] "Means for analyzing emotional state" refers to technologies and systems that analyze a user's emotional state from facial expressions, voice, etc.

[1326] "Means for adjusting the format of information provision" refers to a system that changes the display method and content of the information provided depending on the user's emotional state.

[1327] To implement the present invention, a system must be constructed based on the following steps.

[1328] Server configuration and processing

[1329] The server retrieves the latest train operation information from publicly available data sources on the Internet using web scraping tools such as "BeautifulSoup" and "Selenium," which use the Python programming language. For example, it extracts text information such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake" from a specified website.

[1330] The extracted text information is then analyzed using natural language processing (NLP) techniques, using Python libraries such as spaCy and NLTK to extract important keywords, such as "Shinjuku Station," "Tokyo Station," and "service suspension."

[1331] The analyzed information is translated into multiple languages ​​using a multilingual translation system such as the Google Translate API. For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1332] The translated information is stored in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operational status data, route name, acquisition date and time, and translated text.

[1333] Terminal configuration and handling

[1334] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent. The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. The connected device (e.g., a smartphone) displays the information received from the server in an easy-to-read format for the user. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[1335] Emotion engine processing

[1336] The emotion engine uses facial recognition and voice analysis technologies to provide users with comfortable information in real time. Specifically, it uses the Python "dlib" library for face detection and "DeepFace" for emotion analysis.

[1337] If the emotional state indicates "tension" or "stress," that state is sent to the server and the way the information is displayed changes. For example, if the user is feeling stressed, important information will be displayed in a more prominent format.

[1338] Emergency Notification

[1339] If the user has set up emergency notifications, the server will take into account the data from the emotion engine and send notifications at the appropriate time. For example, if service suspension information due to a typhoon is stored in the database, the server will send a notification such as "Typhoon alert: Service suspended between Shinagawa and Yokohama" depending on the urgency.

[1340] Usage example

[1341] When a user inquires about the Tokaido Line's service status using a smartphone, the device displays "Shinagawa - Yokohama: Service suspended due to typhoon." If the emotion engine detects stress in the user's facial expression, this information is highlighted and, if necessary, an emergency notification is sent.

[1342] Example prompts for generative AI models

[1343] "Please tell me the Tokaido Line operation status in multiple languages."

[1344] "Customize traffic notifications based on emotions."

[1345] In this way, the system of the present invention allows users to obtain real-time operational information that is adapted to their emotional state, enabling them to respond quickly in emergencies.

[1346] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1347] Step 1:

[1348] The server retrieves information from publicly available internet data sources, uses Python's BeautifulSoup and Selenium to extract text information, such as "Service suspended between Shinjuku Station and Tokyo Station due to earthquake," and converts the text into a format that can be used in the next processing step.

[1349] Input: URL of the data source

[1350] Output: Extracted service information text data

[1351] Step 2:

[1352] The server analyzes the acquired information using natural language processing technology, using Python's "spaCy" and "NLTK" to extract important keywords such as "Shinjuku Station," "Tokyo Station," and "service suspension" from the text information.

[1353] Input: Text data of operation information

[1354] Output: Parsed keywords

[1355] Step 3:

[1356] The server translates the analyzed information into multiple languages ​​using a multilingual translation system (e.g., Google Translate API). For example, "Service suspension between Shinjuku Station and Tokyo Station due to earthquake" is translated into "Service suspension between Shinjuku Station and Tokyo Station due to earthquake."

[1357] Input: Parsed keyword

[1358] Output: Translated text data

[1359] Step 4:

[1360] The server stores the translated information in a relational database (e.g., MySQL, PostgreSQL), which stores information such as operation status data, route name, acquisition date and time, and translated text.

[1361] Input: Translated text data

[1362] Output: Data stored in the database

[1363] Step 5:

[1364] When a device receives a request from a user, it sends an API request to the server. For example, if a user wants to check the "Tokaido Line operation status," a request such as " / api / route_status?line=tokaido" is sent.

[1365] Input: User request

[1366] Output: API request to the server

[1367] Step 6:

[1368] The server retrieves the corresponding data from the database in response to the received request and returns a response in the specified language. For example, the data "Service suspension between Shinagawa and Yokohama due to typhoon" is returned in JSON format.

[1369] Input: API request

[1370] Output: JSON formatted service information

[1371] Step 7:

[1372] The device receives the information from the server and displays it in a user-friendly format. For example, a smartphone app might display "Shinagawa - Yokohama: Service suspended due to typhoon."

[1373] Input: Response from the server

[1374] Output: Traffic information displayed on the user's device

[1375] Step 8:

[1376] The server analyzes the user's emotional state in real time using an emotion engine, which uses facial recognition technology (Python's "dlib" or "DeepFace") and voice analysis technology to analyze the user's emotional state and transmits the results to the server.

[1377] Input: User's facial image and voice data

[1378] Output: Sentiment analysis results

[1379] Step 9:

[1380] The server adjusts the format of information delivery based on the user's emotional state: for example, if the user is stressed, important information will be displayed prominently and, if necessary, sent as an urgent notification.

[1381] Input: Sentiment analysis results, traffic information data

[1382] Output: Form of tailored information, emergency notification

[1383] The above are the specific processing steps for carrying out the present invention.

[1384] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1385] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1386] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1387] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1388] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1389] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1390] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1391] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1392] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1393] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1394] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1395] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1396] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1397] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1398] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1399] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1400] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1401] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1402] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1403] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1404] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1405] The following is further disclosed regarding the above embodiment.

[1406] (Claim 1)

[1407] a means for obtaining information from publicly available internet sources;

[1408] A means for analyzing the acquired information using natural language processing technology;

[1409] a means for translating the analyzed information into multiple languages ​​using a multilingual translation system;

[1410] means for storing the translated information in a database;

[1411] means for providing information in a specified language based on a request from a user;

[1412] A system including:

[1413] (Claim 2)

[1414] 10. The system of claim 1, further comprising: means for sending a notification to the user in an emergency based on a preset condition.

[1415] (Claim 3)

[1416] 10. The system of claim 1, further comprising means for updating and providing information in real time based on user requests.

[1417] "Example 1"

[1418] (Claim 1)

[1419] a means of obtaining information from publicly available internet sources;

[1420] A means for analyzing the acquired information using natural language processing technology;

[1421] a means for translating the analyzed information into multiple languages ​​using a multilingual translation system;

[1422] means for storing the translated information in a relational database;

[1423] means for providing information in a specified language based on a request from a user;

[1424] means for displaying information on a user's terminal;

[1425] A system including:

[1426] (Claim 2)

[1427] 10. The system of claim 1, further comprising: means for sending a notification to the user in an emergency based on a preset condition.

[1428] (Claim 3)

[1429] 10. The system of claim 1, further comprising means for updating and providing information in real time based on user requests.

[1430] "Application Example 1"

[1431] (Claim 1)

[1432] a means of obtaining data from publicly available internet sources;

[1433] A means for analyzing the acquired data using natural language processing technology;

[1434] means for translating the analyzed data into multiple languages ​​using a multilingual translation system;

[1435] means for storing the translated data in a database;

[1436] means for providing data in a specified language upon request from a user;

[1437] In an autonomous vehicle system, a means for displaying the acquired data on a dashboard or a head-mounted display;

[1438] A system including:

[1439] (Claim 2)

[1440] 10. The system of claim 1, further comprising: means for sending a notification to the user in an emergency based on a preset condition.

[1441] (Claim 3)

[1442] 10. The system of claim 1, further comprising means for updating and providing the data in real time based on a user request.

[1443] "Example 2: Combining Emotion Engines"

[1444] (Claim 1)

[1445] a means for obtaining information from publicly available internet sources;

[1446] A means for analyzing the acquired information using natural language processing technology;

[1447] a means for translating the analyzed information into multiple languages ​​using a multilingual translation system;

[1448] means for storing the translated information in a database;

[1449] means for providing information in a specified language based on a request from a user;

[1450] A means for analyzing the user's emotional state using facial recognition technology and voice analysis technology;

[1451] means for customizing information provided based on the user's emotional state;

[1452] A system including:

[1453] (Claim 2)

[1454] 10. The system of claim 1, further comprising: means for sending a notification to the user in an emergency based on a preset condition.

[1455] (Claim 3)

[1456] 10. The system of claim 1, further comprising means for updating and providing information in real time based on user requests.

[1457] "Application example 2 when combining emotion engines"

[1458] (Claim 1)

[1459] a means for obtaining information from publicly available internet sources;

[1460] A means for analyzing the acquired information using natural language processing technology;

[1461] a means for translating the analyzed information into multiple languages ​​using a multilingual translation system;

[1462] means for storing the translated information in a database;

[1463] means for providing information in a specified language based on a request from a user;

[1464] means for analyzing the emotional state of a user;

[1465] A means for adjusting the format of information provision based on the result of the sentiment analysis;

[1466] A system including:

[1467] (Claim 2)

[1468] 10. The system of claim 1, further comprising: means for sending a notification to the user in an emergency based on a preset condition.

[1469] (Claim 3)

[1470] 10. The system of claim 1, further comprising means for updating and providing information in real time based on user requests. [Explanation of symbols]

[1471] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining information from publicly available internet sources; A means for analyzing the acquired information using natural language processing technology; a means for translating the analyzed information into multiple languages ​​using a multilingual translation system; means for storing the translated information in a database; means for providing information in a specified language based on a request from a user; A system including:

2. The system of claim 1 , further comprising means for sending a notification to a user in an emergency based on a preset condition.

3. 10. The system of claim 1, further comprising means for updating and providing information in real time based on user requests.

Images