System

Abstract

A system is provided.SOLUTION: The system includes a means for acquiring charging spot information, a means for analyzing a user's request by voice recognition, a means for calculating an optimum route on the basis of information acquired from a server, and a means for displaying the route information on a navigation screen.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Current automobile navigation systems can only provide fixed route guidance, making it difficult to meet the diverse needs of users. Furthermore, with the spread of electric vehicles, there is a demand for information on charging spots and appropriate route guidance, but currently there is a lack of systems that provide this information in real time. The purpose of this invention is to solve these problems and improve user convenience. [Means for solving the problem]

[0005] The present invention provides a system including a means for acquiring charging spot information, a means for analyzing a user's request using voice recognition, a means for calculating an optimal route based on information acquired from a server, and a means for displaying the route information on a navigation screen. The means for calculating the optimal route also includes a means for dynamically changing the route based on congestion information acquired from the server, and the means for acquiring charging spot information includes a means for displaying the current availability of charging spots in real time. Furthermore, the system includes a means for a user to input information using a smartphone from outside the vehicle and transmit that information to the vehicle's navigation system, and a means for caching the information acquired from the server in the terminal and processing the content requested by voice recognition based on that information, thereby enabling real-time provision of charging spot information and optimal route guidance for electric vehicles.

[0006] "Means for acquiring charging spot information" is a function for collecting location information and availability of spots where electric vehicles can be charged, and providing this information to users.

[0007] "Means for analyzing user requests through voice recognition" refers to a function that converts voice commands issued by users into text data and analyzes it to perform appropriate operations or provide information.

[0008] "Means for calculating the optimal route based on information obtained from the server" refers to a function that calculates the optimal route to the destination in real time, taking into account the latest road conditions and traffic congestion information provided by the server.

[0009] The "means for displaying the route information on the navigation screen" is a function for displaying the calculated optimum route information on the vehicle's navigation display so that the user can visually confirm it.

[0010] "Means for dynamically changing routes based on congestion information" refers to a function that calculates the optimal detour route each time based on congestion and traffic condition data obtained in real time, and automatically changes the navigation route.

[0011] "Means for displaying the current availability of charging spots in real time" refers to a function that displays the availability of charging spots collected in real time on a navigation screen or the like in order to provide the information to users promptly.

[0012] "Means for inputting information using a smartphone from outside the vehicle and transmitting that information to the vehicle's navigation system" refers to a function that transmits information input by the user via a smartphone to the vehicle's navigation system via wireless communication, linking it with the system inside the vehicle.

[0013] "Means for caching information obtained from the server in the terminal and processing the content requested by voice recognition based on that information" is a function for temporarily storing information received from the server in the terminal and quickly processing the content requested by voice recognition. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] Overall system configuration

[0036] This system consists of a server, a terminal, and a user. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone.

[0037] What the program does

[0038] Server-side processing

[0039] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0040] Examples:

[0041] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0042] Terminal side processing

[0043] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on the commands, and displays it on the navigation screen. It also constantly monitors the current location and destination information, and calculates the optimal route based on the information from the server. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0044] Examples:

[0045] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If traffic jams occur during driving, the device suggests a new detour route based on the latest traffic information from the server.

[0046] User operation

[0047] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0048] Examples:

[0049] While taking a break at a cafe, the user can set their next destination on their smartphone and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and starts guiding the user once they get in the car.

[0050] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and by linking with a smartphone, they can also be operated comfortably from outside the vehicle.

[0051] The processing flow will be explained below.

[0052] Server-side processing

[0053] Step 1:

[0054] The server sends a request to the charging spot API, which returns data such as the latest location of the charging spot, availability, and type of charger.

[0055] Step 2:

[0056] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0057] Step 3:

[0058] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0059] Step 4:

[0060] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0061] Terminal side processing

[0062] Step 1:

[0063] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0064] Step 2:

[0065] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0066] Step 3:

[0067] The user issues a voice command, for example, "Find nearby charging spots."

[0068] Step 4:

[0069] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0070] Step 5:

[0071] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0072] Step 6:

[0073] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0074] Step 7:

[0075] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0076] User operation

[0077] Step 1:

[0078] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[0079] Step 2:

[0080] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[0081] Step 3:

[0082] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[0083] Step 4:

[0084] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[0085] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] Conventional navigation systems face challenges in terms of their ability to acquire and update charging station and traffic information in real time and dynamically present optimal routes. Furthermore, they lack the functionality to allow users to set a destination outside the vehicle and instantly synchronize it with the vehicle's navigation system, significantly reducing user convenience. Furthermore, the limited use of voice recognition systems presents a problem in that they are unable to efficiently analyze diverse voice commands.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes a means for periodically collecting charging station information, traffic information, and congestion information and updating the internal database, a means for calculating the optimal route, and a means for analyzing user instructions using voice recognition and converting the obtained instructions into text data. This makes it possible to dynamically calculate and present the optimal route based on charging station information and traffic information updated in real time. Furthermore, the destination set by the user using a smart device outside the vehicle can be synchronized with the vehicle's navigation system, greatly improving convenience. In addition, advanced voice recognition technology allows for efficient analysis and execution of a variety of voice instructions.

[0091] "Charging station information" refers to data such as the location, availability, and type of charger of the equipment that supplies power to electric vehicles.

[0092] "Speech recognition" refers to the technology that analyzes the voice spoken by the user and converts it into corresponding text data.

[0093] "User instructions" refer to requests or commands made by a user to a system that direct a particular operation of the system.

[0094] "Means for calculating the optimal route" refers to a device or software that uses an algorithm to calculate the optimal route to the destination based on information obtained by the server and terminal.

[0095] "Navigation screen" means a display device installed inside a vehicle that visually presents route information and other related data.

[0096] "Server" refers to a computer system that collects, processes, and manages data via a network.

[0097] "Periodic collection means" refers to a method or device that accesses external APIs or other data sources at regular intervals to obtain the required data.

[0098] "Internal database" refers to a data management system that stores collected data and enables quick retrieval of required information.

[0099] "Smart device" refers to a mobile device (such as a smartphone or tablet) with internet connectivity that allows access to and operation of the system.

[0100] "Means of synchronization" refers to the technology or method that keeps data consistent across multiple devices.

[0101] "Means for displaying in real time" refers to a device or software that has the function of instantly displaying acquired information to a user.

[0102] This invention relates to a system that collects charging station information, traffic information, and congestion information in real time and dynamically presents optimal routes. This system consists of three main parts: a server, a terminal, and a user.

[0103] Server Features

[0104] The server is responsible for periodically collecting charging station information, traffic information, and congestion information. Charging station information includes location information, availability, charger type, etc., and is obtained from an external charging spot API via an HTTP GET request. The obtained data is received in JSON format, parsed, and saved / updated in the internal database.

[0105] Similarly, traffic and congestion information is periodically obtained from an external traffic information API. The server analyzes this information and keeps it up to date.

[0106] Examples:

[0107] The server accesses the charging spot API every hour to obtain the latest charging station information. After obtaining the information, the server analyzes the data and updates the nationwide charging station database. Traffic information is also updated in the same way.

[0108] Device Features

[0109] The terminal is the main device that receives real-time charging station and traffic information from the server. The terminal is equipped with a voice recognition system that analyzes the user's voice commands and converts them into text data. The terminal then sends the appropriate request to the server, retrieves the necessary information, and displays it on the navigation screen.

[0110] Based on the acquired information, the device constantly monitors the current location and destination information and calculates the optimal route. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0111] Examples:

[0112] When a user says "Find nearby charging spots," the device's voice recognition engine (for example, Google Speech-to-Text API) converts this into text. The device then retrieves information about the nearest charging spots from the server, displays it on the screen, and provides directions. If traffic jams occur during a drive, the device will suggest a new route based on the new traffic conditions.

[0113] User operations

[0114] Users can issue instructions to the system using voice commands from inside the car. For example, the system responds to voice commands such as "Find nearby charging spots" or "Set my next destination." Users can also access the system from outside the car using a smart device to search for charging stations or set destinations. The information set from the smart device is synchronized with the vehicle's navigation system, making it available the moment the user gets into the car.

[0115] Examples:

[0116] While taking a break at a cafe, a user can use their smart device to set their next destination and check nearby charging stations in advance. This information is automatically synchronized with the vehicle's navigation system, so that directions can begin immediately when the user gets in the car.

[0117] Example prompt sentence:

[0118] "My system involves a server collecting charging spot information and traffic congestion information, and an in-car terminal using that information to navigate the vehicle. The user can issue instructions to the system via voice commands or a smartphone. Please explain the process of this system in detail, dividing it into specific steps. Please also include the roles of the server, terminal, and user, as well as the specific flow of processing. Please also include a specific example of how it works."

[0119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0120] Step 1:

[0121] The server sends an HTTP GET request to the charging station API to obtain the latest charging station information.

[0122] Input: Charging Station API Endpoint URL

[0123] Output: Charging station information data in JSON format

[0124] Specifically, the server calls the charging station API at specific time intervals (e.g., every hour) to obtain data such as location, availability, and type of charger.

[0125] Step 2:

[0126] The server analyzes the acquired JSON-formatted data, extracts information such as the location of the charging station, availability, and type of charger, and stores it in an internal database.

[0127] Input: Charging station information data in JSON format

[0128] Output: Charging station information stored in a database

[0129] Specifically, the server performs parsing (analysis) processing, extracts necessary information, and stores it in a "charging station information" table.

[0130] Step 3:

[0131] The server sends an HTTP GET request to the traffic information API to obtain the latest traffic conditions and congestion information.

[0132] Input: Traffic Information API endpoint URL

[0133] Output: Traffic and congestion information data in JSON format

[0134] Specifically, the server accesses the traffic information API at regular intervals to obtain the latest traffic conditions and congestion information.

[0135] Step 4:

[0136] The server analyzes the acquired traffic information and stores it in an internal database.

[0137] Input: Traffic and congestion information data in JSON format

[0138] Output: Traffic situation and congestion information stored in a database

[0139] Specifically, the server parses the acquired JSON data, extracts the necessary traffic information, and stores it in a database.

[0140] Step 5:

[0141] The terminal sends an HTTP GET request to the server to obtain charging station information and traffic information in real time.

[0142] Input: Server endpoint URL

[0143] Output: Real-time charging station information and traffic information

[0144] Specifically, the terminal obtains the necessary information from the server in real time in response to a user request.

[0145] Step 6:

[0146] The terminal uses a voice recognition system to analyze the user's voice commands and convert them into text data.

[0147] Input: User's voice command

[0148] Output: Text data

[0149] Specifically, a speech recognition engine (for example, Google Speech-to-Text API) is used to convert the user's speech into text data.

[0150] Step 7:

[0151] The terminal sends an appropriate request to the server based on the acquired text data and acquires the required information.

[0152] Input: Text data (the result of converting the user's voice command)

[0153] Output: Response data from the server (charging station information, traffic information, etc.)

[0154] Specifically, the terminal sends a request to the server based on the analyzed text data and obtains related information from the server.

[0155] Step 8:

[0156] Based on the acquired information, the device displays charging station information and the optimal route on the navigation screen.

[0157] Input: Charging station information and traffic information obtained from the server

[0158] Output: Charging station information and optimal route displayed on the navigation screen

[0159] Specifically, the device uses Google Maps API and other tools to display charging station information and traffic information on a map, and presents the optimal route on the navigation screen.

[0160] Step 9:

[0161] The device calculates the optimal route based on the current location and destination information, and if traffic congestion occurs on the route, it calculates a new detour route.

[0162] Input: current location, destination, traffic information

[0163] Output: Optimal route and detour route information

[0164] Specifically, the device dynamically calculates the optimal route based on traffic information updated in real time, and presents new detour routes if necessary.

[0165] Step 10:

[0166] Users can access the system from outside the vehicle using a smart device to search for charging stations and set destinations, and then synchronize that information with the in-car navigation system.

[0167] Input: Destination setting information from smart device, charging station search results

[0168] Output: Destination and charging station information synchronized with the in-car navigation system

[0169] Specifically, information set by the user using a smartphone app is synchronized in real time with the vehicle's navigation system, making the information instantly available in the vehicle.

[0170] (Application example 1)

[0171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] Conventional navigation systems for autonomous vehicles lack the ability to collect information on charging spots and traffic congestion, making it difficult to calculate optimal routes in real time. Furthermore, they lack connectivity with smartphones and other devices, making it difficult for users to operate them easily. Furthermore, there was also the issue of incomplete operational support for autonomous vehicles in response to user voice commands.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0174] In this invention, the server includes means for acquiring charging spot information, means for analyzing user requests through voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, and means for synchronizing and manipulating data in real time via a smartphone or terminal. This makes it possible to acquire charging spot information and traffic congestion information in real time, dynamically calculate an optimal route based on the user's voice commands, and efficiently support the operation of autonomous vehicles.

[0175] "Charging spot information" is information about locations where electric vehicles can be charged, and includes location information, availability, type of charger, and the like.

[0176] "Speech recognition" is a technology that analyzes voice input and converts it into text data, in order to recognize a user's voice commands.

[0177] A "server" is a computer system that manages and processes data on a network, and is responsible for acquiring, storing, and providing information on charging spots and traffic congestion.

[0178] "Means for calculating the optimal route" refers to methods or technologies that derive the optimal route based on the user's current location and destination, and refer to charging spot information and traffic congestion information.

[0179] The "navigation screen" is a screen that shows route information, charging spot information, and the like, displayed on a display inside the vehicle.

[0180] "Means for synchronizing and manipulating data in real time through smartphones and terminals" refers to technology that allows charging spot information and route information to be updated and manipulated in real time using smartphones and other devices.

[0181] "Traffic congestion information" is data relating to traffic conditions on roads, including vehicle speeds, congestion, accidents, and the like.

[0182] An "autonomous vehicle" is a vehicle that can drive autonomously without driver intervention.

[0183] "Means for assisting the operation of autonomous vehicles in response to voice commands" refers to technology that enables autonomous vehicles to perform appropriate operations and route changes based on the user's voice input.

[0184] "Means for displaying the current availability of charging spots in real time" refers to methods or technologies that constantly update the availability of charging spots and display it to users immediately.

[0185] "Location information of the nearest charging spot" is data indicating the location of the charging spot that is the shortest distance from the user's current location.

[0186] To implement the present invention, the roles played by the server, the terminal, and the user will be described.

[0187] Server Roles

[0188] The server collects and processes charging spot and traffic congestion information, and provides this information to the terminal. A general server computer is used as the hardware, and Flask (framework) and Python (programming language) are used as the software. The server periodically accesses the charging spot API and traffic information API, and stores the collected data in a database.

[0189] Specific server operations

[0190] The server periodically makes API requests to collect data such as the availability of charging spots, their locations, and the type of charger. This data is stored in a database on the server and updated in real time. Traffic and congestion information is also collected and added to the database in real time.

[0191] Device Role

[0192] The in-vehicle device uses information obtained from the server to perform navigation and voice recognition operations. Google's Speech-to-Text API is used for voice recognition, and the entire system is built using Python and Flask. It also has the ability to synchronize data in real time with smartphones and other devices.

[0193] Specific operation of the device

[0194] The device receives the user's voice command and converts it into text using Google's Speech-to-Text API. For example, if the user says, "Find nearby charging spots," the speech is converted into text and sent to the server as an appropriate request. The server searches for charging spots based on the latest information and returns the results to the device. The device then displays the search results on the navigation screen and provides the user with directions to the nearest charging spot.

[0195] User Roles

[0196] Users can operate the system via voice commands or a smartphone app. Information and settings acquired through user operations are synchronized in real time with the device inside the vehicle, allowing the system to be easily operated even from outside the vehicle.

[0197] Specific user operation examples

[0198] While taking a break at a cafe, the user can use their smartphone to set their next destination and check the nearest charging spot. This information is automatically synchronized with the vehicle's navigation system, and guidance begins as soon as the user gets in the car. When the user uses the smartphone app to enter the prompt "Find nearby charging spots," the system will provide information on the optimal charging spots.

[0199] Example prompt sentence:

[0200] Find a nearby charging spot

[0201] These functions allow users to receive information on charging spots and optimal route guidance in real time, and the system can be conveniently operated by linking it with a smartphone or other device.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] The server periodically sends requests to the charging spot API and traffic information API. The server parses the data obtained from the API and stores the available charging spots, their locations, charger types, and traffic congestion information in a database. The input is the API response, and the output is the parsed charging spot information and traffic congestion information updates.

[0205] Step 2:

[0206] The user speaks a voice command. The device in the vehicle converts the speech to text using Google's Speech-to-Text API. The voice command is the input, and the generated text is the output. For example, the voice command might be "Find nearby charging spots."

[0207] Step 3:

[0208] The device parses the voice command converted to text and requests the necessary information from the server. The device sends a request to the appropriate API endpoint based on the text generated by Google's Speech-to-Text API. The input is the voice command converted to text, and the output is a request to the server.

[0209] Step 4:

[0210] When the server receives a request from the device, it retrieves the appropriate charging spot information and traffic congestion information from the database and returns it to the device in JSON format. The input is the request from the device, and the output is the data obtained and returned by the server.

[0211] Step 5:

[0212] The device displays the information received from the server and shows the location information and route to the nearest charging spot on the navigation screen. The input is information from the server, and the output is what is displayed on the navigation screen. For example, the device may display that the nearest charging spot is 500 meters from the vehicle and provide route information.

[0213] Step 6:

[0214] The user refers to the navigation screen and drives the vehicle to the displayed charging spot or uses the automatic driving function. The input is the information on the navigation screen, and the output is the user's movement or the automatic driving of the vehicle.

[0215] Step 7:

[0216] When the vehicle arrives at a charging spot, charging begins. The server then periodically updates the data and provides new information in real time. The input is the current status of the charging spot, and the output is the start of use of the charging spot.

[0217] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0218] Overall system configuration

[0219] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[0220] What the program does

[0221] Server-side processing

[0222] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0223] Examples:

[0224] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0225] Terminal side processing

[0226] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on those commands, and displays it on the navigation screen. It also uses an emotion engine to analyze the user's emotions from their voice and facial expressions, and adjusts the system's response based on those results.

[0227] Examples:

[0228] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0229] The device constantly monitors the user's current location and destination, and calculates the optimal route based on the latest traffic congestion information. If a traffic jam occurs, the device calculates a new detour route and presents it to the user. At the same time, based on information from the emotion engine, if the user is feeling stressed, the device adjusts the guidance to make route selection as simple as possible.

[0230] User operation

[0231] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0232] Examples:

[0233] While taking a break at a cafe, the user sets their next destination on their smartphone and checks available charging spots in advance. Based on this, the car's navigation system automatically sets a route and starts guiding the user once they get in the car. If the emotion engine determines that the user's stress level is high during the journey, it will suggest a smoother route or relaxing music.

[0234] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and the emotion engine also provides a comfortable driving experience.

[0235] The processing flow will be explained below.

[0236] Processing flow of a system that combines emotion engines

[0237] Server-side processing

[0238] Step 1:

[0239] The server periodically sends requests to the charging spot API, which returns data such as the latest location, availability, and type of charger for the charging spot.

[0240] Step 2:

[0241] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0242] Step 3:

[0243] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0244] Step 4:

[0245] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0246] Terminal side processing

[0247] Step 1:

[0248] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0249] Step 2:

[0250] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0251] Step 3:

[0252] The user issues a voice command, for example, "Find nearby charging spots."

[0253] Step 4:

[0254] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0255] Step 5:

[0256] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0257] Step 6:

[0258] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0259] Step 7:

[0260] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0261] Step 8:

[0262] The device analyzes the user's voice and facial expressions using an emotion engine, which determines the user's stress level and emotional state.

[0263] Step 9:

[0264] After the emotion engine recognizes the user's emotions, the device can adjust navigation and system responses based on that information. For example, if the user is feeling stressed, it can provide a calming voice prompt to encourage the user to relax.

[0265] Examples:

[0266] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0267] User operation

[0268] Step 1:

[0269] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[0270] Step 2:

[0271] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[0272] Step 3:

[0273] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[0274] Step 4:

[0275] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[0276] Step 5:

[0277] If the emotion engine determines that the user's stress level is high along the way, the device will adjust the guidance to make route selection as simple as possible.

[0278] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably. In addition, the introduction of an emotion engine provides optimal support according to the user's emotional state.

[0279] Example 2

[0280] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0281] Conventional navigation systems can provide real-time information on charging spots and traffic congestion, but they are unable to respond appropriately taking into account the user's emotional state. This can lead to stress for drivers and can lead to a less comfortable driving environment.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0283] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server and the user's emotions, means for displaying the route information on a navigation screen, and means for recognizing emotions from the user's voice and facial expressions and adjusting the system's response, thereby making it possible to provide optimal route guidance and appropriate responses according to the user's emotional state.

[0284] "Charging spot information" is data such as location information, availability, and type of charger regarding places where electric vehicles can be charged.

[0285] "Speech recognition" is a technology that converts a user's voice commands into text and analyzes their intent.

[0286] The "optimal route" is the most efficient and effective route to your destination, calculated based on charging spot information and traffic congestion information.

[0287] A "server" is a computer system that collects information on charging spots, traffic congestion, and other real-time data and provides it to terminals.

[0288] The "navigation screen" is a screen on which the terminal visually presents route guidance and charging spot information to the user.

[0289] The "emotion engine" is a technology that analyzes the user's voice and facial expressions to recognize their emotional state.

[0290] "Traffic congestion information" is data relating to road traffic conditions and the degree of congestion.

[0291] "Means for displaying in real time" refers to techniques or methods that instantly display the current situation to the user.

[0292] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[0293] Server-side processing

[0294] The server sends requests to the charging spot API and traffic information API at regular intervals to obtain the latest charging spot information and traffic congestion information. The obtained data includes information such as location, availability, and charger type, and is parsed and updated in an internal database. Specifically, it obtains data from major charging spot APIs (for example, the EV charging network API) and traffic congestion information from Google Maps' traffic data API.

[0295] Terminal side processing

[0296] The device obtains charging spot and traffic congestion information in real time from the server. The device is equipped with a voice recognition system that analyzes the user's voice commands, retrieves appropriate information, and displays it on the navigation screen. The emotion engine analyzes the user's voice and facial expressions, recognizes their emotions, and adjusts the system's response accordingly. For example, if a user says, "Find nearby charging spots," the device's voice recognition engine analyzes this and retrieves and displays the most appropriate charging spot information from the server. If the emotion engine analyzes the user's tone of voice and facial expressions and determines that they are stressed, the device will respond in a calm voice, saying, "Relax and drive safely."

[0297] User operation

[0298] Users can issue instructions to the system using voice commands. They can also access the system from outside the car using a smartphone app to search for charging spots and set destinations. Information set on the smartphone is synchronized with the car's navigation system and is immediately available when they get in. For example, while taking a break at a cafe, a user can use their smartphone to set their next destination and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and begins guiding the user as soon as they get in the car. If the emotion engine determines that the user's stress level is high, it will suggest smooth routes and relaxing music.

[0299] Examples of prompt statements

[0300] Example prompt for explaining server-side processing: "Please explain the server-side process that collects the latest charging spot information and traffic congestion information."

[0301] Example prompt for device-side processing: "Describe the device-side process where a user uses voice commands to find a charging spot. Also, explain how you use an emotion engine to tailor the response."

[0302] Example prompt for user interaction: "Describe how a user would set their next destination and search for charging spots on their smartphone. Include an example of how the emotion engine would respond if their stress level was high."

[0303] This system allows users to receive real-time information on electric vehicle charging spots and optimal route guidance, and also provides a comfortable driving experience through an emotion engine.

[0304] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0305] Step 1: Data Collection (Server)

[0306] The server sends periodic requests to the charging spot API and traffic information API. The input for this step is the API endpoint URL and necessary authentication information. The server uses these inputs to generate and send a data request. As output, it receives charging spot information and traffic information in JSON format. Specifically, the server sends a request every hour to the main charging spot API (e.g., EV charging network API) to obtain the latest data. It also sends a request to Google Maps traffic data API to obtain traffic congestion information.

[0307] Step 2: Data Parsing (Server)

[0308] The server parses the JSON format data obtained in step 1. The input for this step is the obtained JSON data. The server parses this data and extracts the necessary information (e.g., location information, availability, and type of charger). As an output, the extracted data is imported into an internal database. Specifically, the server parses the responses obtained from the charging spot API and traffic information API, and classifies and extracts the respective information.

[0309] Step 3: Database Update (Server)

[0310] The server immediately updates the information extracted in step 2 to its internal database. The input to this step is the extracted charging spot information and traffic information. The server updates the corresponding fields in the database based on this input. The output is an updated database that reflects the new information. Specifically, the server adds and updates the extracted charging spot information to the nationwide charging spot database. Similarly, traffic congestion information is also updated in the database.

[0311] Step 4: Data Acquisition (Device)

[0312] The terminal obtains charging spot information and traffic congestion information in real time from the server. The input to this step is request information to the server. Based on this, the terminal requests data from the server and receives the latest charging spot information and traffic information as output. Specifically, the terminal sends requests to the server at regular intervals to obtain the latest information.

[0313] Step 5: Voice command analysis (device)

[0314] The device analyzes the user's voice command with a voice recognition system. The input for this step is the user's voice command. The device converts this voice into text and analyzes it for appropriate processing. The output is the analyzed text data. Specifically, the device's voice recognition engine converts the voice "Find nearby charging spots" into text and retrieves information based on the request.

[0315] Step 6: Sentiment Analysis (Device)

[0316] The device uses an emotion engine to analyze emotions from the user's voice and facial expressions. The input for this step is the user's voice and facial expression data. The device analyzes these and determines the user's emotional state. The output is a system response based on the emotional state. Specifically, if the device's emotion engine analyzes the user's tone of voice and facial expression and recognizes that the user is irritated, the device will respond in a calm voice, saying, "Relax and drive safely."

[0317] Step 7: Navigation Display (Device)

[0318] The device updates the navigation screen based on the data obtained from the server. The input for this step is the latest charging spot information and traffic information. The device calculates the optimal route based on this input and displays it on the navigation screen. The output is the latest route guidance. Specifically, the device displays information on the charging spot closest to the user and navigates the route. It also recalculates the optimal route based on traffic congestion information and suggests detour routes.

[0319] Step 8: Voice command input (user)

[0320] The user issues a voice command from inside the car to give instructions to the system. The input for this step is the user's voice command. Based on this, the user sends a request to the system. The output is the analyzed voice command, which is processed by the system. In concrete terms, if the user says, "Find a nearby charging spot," the device will recognize the voice and provide appropriate information.

[0321] Step 9: Smartphone operation (user)

[0322] The user accesses the system from outside the vehicle using a smartphone app. The input for this step is the operation information of the smartphone app. The user uses this information to search for charging spots and set their destination. The output is the setting information synchronized with the in-vehicle navigation system. In concrete terms, the user sets their next destination using their smartphone while taking a break at a cafe and checks available charging spots in advance.

[0323] Step 10: System Sync (User)

[0324] The information set by the user is synchronized with the navigation system in the vehicle. The input for this step is the setting information from a smartphone. The user uses this input to make the system immediately available when getting in the car. The setting information is then reflected in the navigation system as an output. In concrete terms, the moment the user gets into the car, the navigation system begins guiding the user along the pre-set route. If the emotion engine determines that the stress level is high, it will suggest smooth routes and relaxing music.

[0325] (Application example 2)

[0326] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Current self-driving vehicles and navigation systems can provide information on charging spots and traffic congestion, but they cannot respond to the user's emotions or make suggestions to reduce stress. As a result, they cannot reduce the stress and anxiety caused by long drives, making it difficult to improve the overall driving experience.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0329] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, an emotion engine for analyzing the user's emotions, and means for adjusting the system's response based on the emotions analyzed by the emotion engine. This enables the user to not only obtain information about charging spots and the optimal route, but also receive suggestions for reducing emotional stress.

[0330] "Charging spot information" is data about locations where electric vehicles can be charged, including location information, availability, and type of charger.

[0331] "Speech recognition" is a technology that analyzes a user's voice and converts it into text data.

[0332] An "optimal route" is the most efficient and time-saving route for a user to reach their destination.

[0333] A "navigation screen" is a display device that visually provides the user with information about the current location and route to the destination.

[0334] The "emotion engine" is a device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0335] A "server" is an information processing device that collects data via a network, processes and stores it, and provides it to clients.

[0336] "Adjusting the system's response based on emotions" refers to controlling the system to respond appropriately based on the user's emotional state.

[0337] The embodiment of the present invention is composed of a server, a terminal, a user, and an emotion engine. This system enables users of autonomous vehicles to enjoy a comfortable and stress-free driving experience.

[0338] Server-side processing

[0339] The server periodically sends requests to the API to collect charging spot information and obtains the latest data. This information includes location, availability, and type of charger. This information is stored in an internal database and provided in response to requests from users' devices. Traffic and congestion information is also collected from another API and stored in the database.

[0340] Terminal side processing

[0341] The device receives real-time information on charging spots and traffic congestion from the server and provides it to the user. It uses a voice recognition system to analyze the user's voice commands, obtain the necessary information, and display it on the navigation screen. It also uses an emotion engine to analyze the user's emotions and adjust the system's response.

[0342] User operations

[0343] Users can issue voice commands to the system from inside the car, and can also access the system from outside the car using a smartphone application to search for charging spots and set destinations. The information set on the smartphone is automatically synchronized with the vehicle's navigation system.

[0344] Hardware and software used

[0345] The hardware used includes a smartphone, microphone, camera, internet connection, etc. The software used includes Python, the Requests library, the Geopy library, the SpeechRecognition library, and the EmotionEngine. This hardware and software enables voice recognition, emotion analysis, and the display of navigation data.

[0346] Specific examples

[0347] In a specific scenario, when a user issues a voice command such as "Find a nearby charging spot," the device's voice recognition engine converts this into text, retrieves appropriate charging spot information from the server, and displays it on the screen. At the same time, if the emotion engine determines that the user is feeling stressed based on their tone of voice or facial expression, the device will notify them by saying, "Relax and drive safely."

[0348] Example prompts for generative AI models

[0349] An example of a prompt for a generative AI model is:

[0350] "Explanation of the goal

[0351] I want to develop an application that allows users of autonomous vehicles to receive charging spot information and navigation.

[0352] We would also like to include a function that recognizes the user's emotions and provides advice to reduce stress.

[0353] Input data

[0354] 1. Charging spot API data (location, availability, charger type)

[0355] 2. Traffic congestion information API data

[0356] 3. User voice commands

[0357] 4. User emotion analysis data (voice tone, facial expressions)

[0358] Output Data

[0359] 1. Information on the nearest charging spot

[0360] 2. Real-time navigation directions

[0361] 3. Response messages based on user sentiment

[0362] Suggested Application Features

[0363] 1. Charging spot search and reservation function

[0364] 2. Real-time navigation function

[0365] 3. Voice command reception function

[0366] 4. Response adjustment function based on emotion analysis

[0367] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0368] Step 1:

[0369] The server sends a request to the charging spot API to get the latest charging spot information. The input is the charging spot API URL, and the output is JSON data including location information, availability, and charger type. This data is parsed and stored in an internal database.

[0370] Step 2:

[0371] The server sends a request to the traffic information API to get the latest traffic congestion information. The input is the traffic information API URL, and the output is JSON data about traffic congestion. This data is also parsed and stored in the internal database.

[0372] Step 3:

[0373] The user issues a voice command. The device inputs the voice through the microphone and converts it into text using speech recognition technology (SpeechRecognition library). For example, if the input is "Find nearby charging spots," the output is this text data.

[0374] Step 4:

[0375] The device that receives the text-translated voice command requests charging spot information from the server based on the command. The input is the text data "Search for nearby charging spots," and the output is information about the nearest charging spot (location information, availability, etc.) sent from the server.

[0376] Step 5:

[0377] The device combines the charging spot information received from the server with the current location information to calculate the optimal route. The input is charging spot information and current location information, and the output is route data for the optimal route. This calculation uses the Geopy library.

[0378] Step 6:

[0379] After the optimal route is calculated, the device displays this route information on the navigation screen. The input is the route data of the optimal route, and the output is a visual navigation display. The user can check the route to the charging spot by looking at the screen.

[0380] Step 7:

[0381] At the same time, the device uses an emotion engine to analyze the user's emotions. The input is the user's voice tone and facial expression data, and the output is data indicating the user's emotional state (e.g., stressed, angry, relaxed). This analysis is performed using the EmotionEngine.

[0382] Step 8:

[0383] Based on the analyzed emotional data, the device adjusts the response message. The input is emotional data, and the output is an appropriate response message (e.g., "Relax and drive safely."). This gives the user a sense of security.

[0384] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0385] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0386] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0387] [Second embodiment]

[0388] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0389] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0390] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0391] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0392] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0393] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0394] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0395] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0396] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0397] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0398] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0399] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0400] Overall system configuration

[0401] This system consists of a server, a terminal, and a user. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone.

[0402] What the program does

[0403] Server-side processing

[0404] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0405] Examples:

[0406] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0407] Terminal side processing

[0408] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on the commands, and displays it on the navigation screen. It also constantly monitors the current location and destination information, and calculates the optimal route based on the information from the server. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0409] Examples:

[0410] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If traffic jams occur during driving, the device suggests a new detour route based on the latest traffic information from the server.

[0411] User operation

[0412] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0413] Examples:

[0414] While taking a break at a cafe, the user can set their next destination on their smartphone and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and starts guiding the user once they get in the car.

[0415] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and by linking with a smartphone, they can also be operated comfortably from outside the vehicle.

[0416] The processing flow will be explained below.

[0417] Server-side processing

[0418] Step 1:

[0419] The server sends a request to the charging spot API, which returns data such as the latest location of the charging spot, availability, and type of charger.

[0420] Step 2:

[0421] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0422] Step 3:

[0423] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0424] Step 4:

[0425] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0426] Terminal side processing

[0427] Step 1:

[0428] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0429] Step 2:

[0430] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0431] Step 3:

[0432] The user issues a voice command, for example, "Find nearby charging spots."

[0433] Step 4:

[0434] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0435] Step 5:

[0436] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0437] Step 6:

[0438] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0439] Step 7:

[0440] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0441] User operation

[0442] Step 1:

[0443] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[0444] Step 2:

[0445] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[0446] Step 3:

[0447] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[0448] Step 4:

[0449] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[0450] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably.

[0451] Example 1

[0452] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0453] Conventional navigation systems face challenges in terms of their ability to acquire and update charging station and traffic information in real time and dynamically present optimal routes. Furthermore, they lack the functionality to allow users to set a destination outside the vehicle and instantly synchronize it with the vehicle's navigation system, significantly reducing user convenience. Furthermore, the limited use of voice recognition systems presents a problem in that they are unable to efficiently analyze diverse voice commands.

[0454] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0455] In this invention, the server includes a means for periodically collecting charging station information, traffic information, and congestion information and updating the internal database, a means for calculating the optimal route, and a means for analyzing user instructions using voice recognition and converting the obtained instructions into text data. This makes it possible to dynamically calculate and present the optimal route based on charging station information and traffic information updated in real time. Furthermore, the destination set by the user using a smart device outside the vehicle can be synchronized with the vehicle's navigation system, greatly improving convenience. In addition, advanced voice recognition technology allows for efficient analysis and execution of a variety of voice instructions.

[0456] "Charging station information" refers to data such as the location, availability, and type of charger of the equipment that supplies power to electric vehicles.

[0457] "Speech recognition" refers to the technology that analyzes the voice spoken by the user and converts it into corresponding text data.

[0458] "User instructions" refer to requests or commands made by a user to a system that direct a particular operation of the system.

[0459] "Means for calculating the optimal route" refers to a device or software that uses an algorithm to calculate the optimal route to the destination based on information obtained by the server and terminal.

[0460] "Navigation screen" means a display device installed inside a vehicle that visually presents route information and other related data.

[0461] "Server" refers to a computer system that collects, processes, and manages data via a network.

[0462] "Periodic collection means" refers to a method or device that accesses external APIs or other data sources at regular intervals to obtain the required data.

[0463] "Internal database" refers to a data management system that stores collected data and enables quick retrieval of required information.

[0464] "Smart device" refers to a mobile device (such as a smartphone or tablet) with internet connectivity that allows access to and operation of the system.

[0465] "Means of synchronization" refers to the technology or method that keeps data consistent across multiple devices.

[0466] "Means for displaying in real time" refers to a device or software that has the function of instantly displaying acquired information to a user.

[0467] This invention relates to a system that collects charging station information, traffic information, and congestion information in real time and dynamically presents optimal routes. This system consists of three main parts: a server, a terminal, and a user.

[0468] Server Features

[0469] The server is responsible for periodically collecting charging station information, traffic information, and congestion information. Charging station information includes location information, availability, charger type, etc., and is obtained from an external charging spot API via an HTTP GET request. The obtained data is received in JSON format, parsed, and saved / updated in the internal database.

[0470] Similarly, traffic and congestion information is periodically obtained from an external traffic information API. The server analyzes this information and keeps it up to date.

[0471] Examples:

[0472] The server accesses the charging spot API every hour to obtain the latest charging station information. After obtaining the information, the server analyzes the data and updates the nationwide charging station database. Traffic information is also updated in the same way.

[0473] Device Features

[0474] The terminal is the main device that receives real-time charging station and traffic information from the server. The terminal is equipped with a voice recognition system that analyzes the user's voice commands and converts them into text data. The terminal then sends the appropriate request to the server, retrieves the necessary information, and displays it on the navigation screen.

[0475] Based on the acquired information, the device constantly monitors the current location and destination information and calculates the optimal route. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0476] Examples:

[0477] When a user says "Find nearby charging spots," the device's voice recognition engine (for example, Google Speech-to-Text API) converts this into text. The device then retrieves information about the nearest charging spots from the server, displays it on the screen, and provides directions. If traffic jams occur during a drive, the device will suggest a new route based on the new traffic conditions.

[0478] User operations

[0479] Users can issue instructions to the system using voice commands from inside the car. For example, the system responds to voice commands such as "Find nearby charging spots" or "Set my next destination." Users can also access the system from outside the car using a smart device to search for charging stations or set destinations. The information set from the smart device is synchronized with the vehicle's navigation system, making it available the moment the user gets into the car.

[0480] Examples:

[0481] While taking a break at a cafe, a user can use their smart device to set their next destination and check nearby charging stations in advance. This information is automatically synchronized with the vehicle's navigation system, so that directions can begin immediately when the user gets in the car.

[0482] Example prompt sentence:

[0483] "My system involves a server collecting charging spot information and traffic congestion information, and an in-car terminal using that information to navigate the vehicle. The user can issue instructions to the system via voice commands or a smartphone. Please explain the process of this system in detail, dividing it into specific steps. Please also include the roles of the server, terminal, and user, as well as the specific flow of processing. Please also include a specific example of how it works."

[0484] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0485] Step 1:

[0486] The server sends an HTTP GET request to the charging station API to obtain the latest charging station information.

[0487] Input: Charging Station API Endpoint URL

[0488] Output: Charging station information data in JSON format

[0489] Specifically, the server calls the charging station API at specific time intervals (e.g., every hour) to obtain data such as location, availability, and type of charger.

[0490] Step 2:

[0491] The server analyzes the acquired JSON-formatted data, extracts information such as the location of the charging station, availability, and type of charger, and stores it in an internal database.

[0492] Input: Charging station information data in JSON format

[0493] Output: Charging station information stored in a database

[0494] Specifically, the server performs parsing (analysis) processing, extracts necessary information, and stores it in a "charging station information" table.

[0495] Step 3:

[0496] The server sends an HTTP GET request to the traffic information API to obtain the latest traffic conditions and congestion information.

[0497] Input: Traffic Information API endpoint URL

[0498] Output: Traffic and congestion information data in JSON format

[0499] Specifically, the server accesses the traffic information API at regular intervals to obtain the latest traffic conditions and congestion information.

[0500] Step 4:

[0501] The server analyzes the acquired traffic information and stores it in an internal database.

[0502] Input: Traffic and congestion information data in JSON format

[0503] Output: Traffic situation and congestion information stored in a database

[0504] Specifically, the server parses the acquired JSON data, extracts the necessary traffic information, and stores it in a database.

[0505] Step 5:

[0506] The terminal sends an HTTP GET request to the server to obtain charging station information and traffic information in real time.

[0507] Input: Server endpoint URL

[0508] Output: Real-time charging station information and traffic information

[0509] Specifically, the terminal obtains the necessary information from the server in real time in response to a user request.

[0510] Step 6:

[0511] The terminal uses a voice recognition system to analyze the user's voice commands and convert them into text data.

[0512] Input: User's voice command

[0513] Output: Text data

[0514] Specifically, a speech recognition engine (for example, Google Speech-to-Text API) is used to convert the user's speech into text data.

[0515] Step 7:

[0516] The terminal sends an appropriate request to the server based on the acquired text data and acquires the required information.

[0517] Input: Text data (the result of converting the user's voice command)

[0518] Output: Response data from the server (charging station information, traffic information, etc.)

[0519] Specifically, the terminal sends a request to the server based on the analyzed text data and obtains related information from the server.

[0520] Step 8:

[0521] Based on the acquired information, the device displays charging station information and the optimal route on the navigation screen.

[0522] Input: Charging station information and traffic information obtained from the server

[0523] Output: Charging station information and optimal route displayed on the navigation screen

[0524] Specifically, the device uses Google Maps API and other tools to display charging station information and traffic information on a map, and presents the optimal route on the navigation screen.

[0525] Step 9:

[0526] The device calculates the optimal route based on the current location and destination information, and if traffic congestion occurs on the route, it calculates a new detour route.

[0527] Input: current location, destination, traffic information

[0528] Output: Optimal route and detour route information

[0529] Specifically, the device dynamically calculates the optimal route based on traffic information updated in real time, and presents new detour routes if necessary.

[0530] Step 10:

[0531] Users can access the system from outside the vehicle using a smart device to search for charging stations and set destinations, and then synchronize that information with the in-car navigation system.

[0532] Input: Destination setting information from smart device, charging station search results

[0533] Output: Destination and charging station information synchronized with the in-car navigation system

[0534] Specifically, information set by the user using a smartphone app is synchronized in real time with the vehicle's navigation system, making the information instantly available in the vehicle.

[0535] (Application example 1)

[0536] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0537] Conventional navigation systems for autonomous vehicles lack the ability to collect information on charging spots and traffic congestion, making it difficult to calculate optimal routes in real time. Furthermore, they lack connectivity with smartphones and other devices, making it difficult for users to operate them easily. Furthermore, there was also the issue of incomplete operational support for autonomous vehicles in response to user voice commands.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0539] In this invention, the server includes means for acquiring charging spot information, means for analyzing user requests through voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, and means for synchronizing and manipulating data in real time via a smartphone or terminal. This makes it possible to acquire charging spot information and traffic congestion information in real time, dynamically calculate an optimal route based on the user's voice commands, and efficiently support the operation of autonomous vehicles.

[0540] "Charging spot information" is information about locations where electric vehicles can be charged, and includes location information, availability, type of charger, and the like.

[0541] "Speech recognition" is a technology that analyzes voice input and converts it into text data, in order to recognize a user's voice commands.

[0542] A "server" is a computer system that manages and processes data on a network, and is responsible for acquiring, storing, and providing information on charging spots and traffic congestion.

[0543] "Means for calculating the optimal route" refers to methods or technologies that derive the optimal route based on the user's current location and destination, and refer to charging spot information and traffic congestion information.

[0544] The "navigation screen" is a screen that shows route information, charging spot information, and the like, displayed on a display inside the vehicle.

[0545] "Means for synchronizing and manipulating data in real time through smartphones and terminals" refers to technology that allows charging spot information and route information to be updated and manipulated in real time using smartphones and other devices.

[0546] "Traffic congestion information" is data relating to traffic conditions on roads, including vehicle speeds, congestion, accidents, and the like.

[0547] An "autonomous vehicle" is a vehicle that can drive autonomously without driver intervention.

[0548] "Means for assisting the operation of autonomous vehicles in response to voice commands" refers to technology that enables autonomous vehicles to perform appropriate operations and route changes based on the user's voice input.

[0549] "Means for displaying the current availability of charging spots in real time" refers to methods or technologies that constantly update the availability of charging spots and display it to users immediately.

[0550] "Location information of the nearest charging spot" is data indicating the location of the charging spot that is the shortest distance from the user's current location.

[0551] To implement the present invention, the roles played by the server, the terminal, and the user will be described.

[0552] Server Roles

[0553] The server collects and processes charging spot and traffic congestion information, and provides this information to the terminal. A general server computer is used as the hardware, and Flask (framework) and Python (programming language) are used as the software. The server periodically accesses the charging spot API and traffic information API, and stores the collected data in a database.

[0554] Specific server operations

[0555] The server periodically makes API requests to collect data such as the availability of charging spots, their locations, and the type of charger. This data is stored in a database on the server and updated in real time. Traffic and congestion information is also collected and added to the database in real time.

[0556] Device Role

[0557] The in-vehicle device uses information obtained from the server to perform navigation and voice recognition operations. Google's Speech-to-Text API is used for voice recognition, and the entire system is built using Python and Flask. It also has the ability to synchronize data in real time with smartphones and other devices.

[0558] Specific operation of the device

[0559] The device receives the user's voice command and converts it into text using Google's Speech-to-Text API. For example, if the user says, "Find nearby charging spots," the speech is converted into text and sent to the server as an appropriate request. The server searches for charging spots based on the latest information and returns the results to the device. The device then displays the search results on the navigation screen and provides the user with directions to the nearest charging spot.

[0560] User Roles

[0561] Users can operate the system via voice commands or a smartphone app. Information and settings acquired through user operations are synchronized in real time with the device inside the vehicle, allowing the system to be easily operated even from outside the vehicle.

[0562] Specific user operation examples

[0563] While taking a break at a cafe, the user can use their smartphone to set their next destination and check the nearest charging spot. This information is automatically synchronized with the vehicle's navigation system, and guidance begins as soon as the user gets in the car. When the user uses the smartphone app to enter the prompt "Find nearby charging spots," the system will provide information on the optimal charging spots.

[0564] Example prompt sentence:

[0565] Find a nearby charging spot

[0566] These functions allow users to receive information on charging spots and optimal route guidance in real time, and the system can be conveniently operated by linking it with a smartphone or other device.

[0567] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0568] Step 1:

[0569] The server periodically sends requests to the charging spot API and traffic information API. The server parses the data obtained from the API and stores the available charging spots, their locations, charger types, and traffic congestion information in a database. The input is the API response, and the output is the parsed charging spot information and traffic congestion information updates.

[0570] Step 2:

[0571] The user speaks a voice command. The device in the vehicle converts the speech to text using Google's Speech-to-Text API. The voice command is the input, and the generated text is the output. For example, the voice command might be "Find nearby charging spots."

[0572] Step 3:

[0573] The device parses the voice command converted to text and requests the necessary information from the server. The device sends a request to the appropriate API endpoint based on the text generated by Google's Speech-to-Text API. The input is the voice command converted to text, and the output is a request to the server.

[0574] Step 4:

[0575] When the server receives a request from the device, it retrieves the appropriate charging spot information and traffic congestion information from the database and returns it to the device in JSON format. The input is the request from the device, and the output is the data obtained and returned by the server.

[0576] Step 5:

[0577] The device displays the information received from the server and shows the location information and route to the nearest charging spot on the navigation screen. The input is information from the server, and the output is what is displayed on the navigation screen. For example, the device may display that the nearest charging spot is 500 meters from the vehicle and provide route information.

[0578] Step 6:

[0579] The user refers to the navigation screen and drives the vehicle to the displayed charging spot or uses the automatic driving function. The input is the information on the navigation screen, and the output is the user's movement or the automatic driving of the vehicle.

[0580] Step 7:

[0581] When the vehicle arrives at a charging spot, charging begins. The server then periodically updates the data and provides new information in real time. The input is the current status of the charging spot, and the output is the start of use of the charging spot.

[0582] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0583] Overall system configuration

[0584] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[0585] What the program does

[0586] Server-side processing

[0587] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0588] Examples:

[0589] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0590] Terminal side processing

[0591] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on those commands, and displays it on the navigation screen. It also uses an emotion engine to analyze the user's emotions from their voice and facial expressions, and adjusts the system's response based on those results.

[0592] Examples:

[0593] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0594] The device constantly monitors the user's current location and destination, and calculates the optimal route based on the latest traffic congestion information. If a traffic jam occurs, the device calculates a new detour route and presents it to the user. At the same time, based on information from the emotion engine, if the user is feeling stressed, the device adjusts the guidance to make route selection as simple as possible.

[0595] User operation

[0596] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0597] Examples:

[0598] While taking a break at a cafe, the user sets their next destination on their smartphone and checks available charging spots in advance. Based on this, the car's navigation system automatically sets a route and starts guiding the user once they get in the car. If the emotion engine determines that the user's stress level is high during the journey, it will suggest a smoother route or relaxing music.

[0599] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and the emotion engine also provides a comfortable driving experience.

[0600] The processing flow will be explained below.

[0601] Processing flow of a system that combines emotion engines

[0602] Server-side processing

[0603] Step 1:

[0604] The server periodically sends requests to the charging spot API, which returns data such as the latest location, availability, and type of charger for the charging spot.

[0605] Step 2:

[0606] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0607] Step 3:

[0608] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0609] Step 4:

[0610] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0611] Terminal side processing

[0612] Step 1:

[0613] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0614] Step 2:

[0615] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0616] Step 3:

[0617] The user issues a voice command, for example, "Find nearby charging spots."

[0618] Step 4:

[0619] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0620] Step 5:

[0621] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0622] Step 6:

[0623] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0624] Step 7:

[0625] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0626] Step 8:

[0627] The device analyzes the user's voice and facial expressions using an emotion engine, which determines the user's stress level and emotional state.

[0628] Step 9:

[0629] After the emotion engine recognizes the user's emotions, the device can adjust navigation and system responses based on that information. For example, if the user is feeling stressed, it can provide a calming voice prompt to encourage the user to relax.

[0630] Examples:

[0631] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0632] User operation

[0633] Step 1:

[0634] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[0635] Step 2:

[0636] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[0637] Step 3:

[0638] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[0639] Step 4:

[0640] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[0641] Step 5:

[0642] If the emotion engine determines that the user's stress level is high along the way, the device will adjust the guidance to make route selection as simple as possible.

[0643] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably. In addition, the introduction of an emotion engine provides optimal support according to the user's emotional state.

[0644] Example 2

[0645] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0646] Conventional navigation systems can provide real-time information on charging spots and traffic congestion, but they are unable to respond appropriately taking into account the user's emotional state. This can lead to stress for drivers and can lead to a less comfortable driving environment.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0648] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server and the user's emotions, means for displaying the route information on a navigation screen, and means for recognizing emotions from the user's voice and facial expressions and adjusting the system's response, thereby making it possible to provide optimal route guidance and appropriate responses according to the user's emotional state.

[0649] "Charging spot information" is data such as location information, availability, and type of charger regarding places where electric vehicles can be charged.

[0650] "Speech recognition" is a technology that converts a user's voice commands into text and analyzes their intent.

[0651] The "optimal route" is the most efficient and effective route to your destination, calculated based on charging spot information and traffic congestion information.

[0652] A "server" is a computer system that collects information on charging spots, traffic congestion, and other real-time data and provides it to terminals.

[0653] The "navigation screen" is a screen on which the terminal visually presents route guidance and charging spot information to the user.

[0654] The "emotion engine" is a technology that analyzes the user's voice and facial expressions to recognize their emotional state.

[0655] "Traffic congestion information" is data relating to road traffic conditions and the degree of congestion.

[0656] "Means for displaying in real time" refers to techniques or methods that instantly display the current situation to the user.

[0657] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[0658] Server-side processing

[0659] The server sends requests to the charging spot API and traffic information API at regular intervals to obtain the latest charging spot information and traffic congestion information. The obtained data includes information such as location, availability, and charger type, and is parsed and updated in an internal database. Specifically, it obtains data from major charging spot APIs (for example, the EV charging network API) and traffic congestion information from Google Maps' traffic data API.

[0660] Terminal side processing

[0661] The device obtains charging spot and traffic congestion information in real time from the server. The device is equipped with a voice recognition system that analyzes the user's voice commands, retrieves appropriate information, and displays it on the navigation screen. The emotion engine analyzes the user's voice and facial expressions, recognizes their emotions, and adjusts the system's response accordingly. For example, if a user says, "Find nearby charging spots," the device's voice recognition engine analyzes this and retrieves and displays the most appropriate charging spot information from the server. If the emotion engine analyzes the user's tone of voice and facial expressions and determines that they are stressed, the device will respond in a calm voice, saying, "Relax and drive safely."

[0662] User operation

[0663] Users can issue instructions to the system using voice commands. They can also access the system from outside the car using a smartphone app to search for charging spots and set destinations. Information set on the smartphone is synchronized with the car's navigation system and is immediately available when they get in. For example, while taking a break at a cafe, a user can use their smartphone to set their next destination and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and begins guiding the user as soon as they get in the car. If the emotion engine determines that the user's stress level is high, it will suggest smooth routes and relaxing music.

[0664] Examples of prompt statements

[0665] Example prompt for explaining server-side processing: "Please explain the server-side process that collects the latest charging spot information and traffic congestion information."

[0666] Example prompt for device-side processing: "Describe the device-side process where a user uses voice commands to find a charging spot. Also, explain how you use an emotion engine to tailor the response."

[0667] Example prompt for user interaction: "Describe how a user would set their next destination and search for charging spots on their smartphone. Include an example of how the emotion engine would respond if their stress level was high."

[0668] This system allows users to receive real-time information on electric vehicle charging spots and optimal route guidance, and also provides a comfortable driving experience through an emotion engine.

[0669] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0670] Step 1: Data Collection (Server)

[0671] The server sends periodic requests to the charging spot API and traffic information API. The input for this step is the API endpoint URL and necessary authentication information. The server uses these inputs to generate and send a data request. As output, it receives charging spot information and traffic information in JSON format. Specifically, the server sends a request every hour to the main charging spot API (e.g., EV charging network API) to obtain the latest data. It also sends a request to Google Maps traffic data API to obtain traffic congestion information.

[0672] Step 2: Data Parsing (Server)

[0673] The server parses the JSON format data obtained in step 1. The input for this step is the obtained JSON data. The server parses this data and extracts the necessary information (e.g., location information, availability, and type of charger). As an output, the extracted data is imported into an internal database. Specifically, the server parses the responses obtained from the charging spot API and traffic information API, and classifies and extracts the respective information.

[0674] Step 3: Database Update (Server)

[0675] The server immediately updates the information extracted in step 2 to its internal database. The input to this step is the extracted charging spot information and traffic information. The server updates the corresponding fields in the database based on this input. The output is an updated database that reflects the new information. Specifically, the server adds and updates the extracted charging spot information to the nationwide charging spot database. Similarly, traffic congestion information is also updated in the database.

[0676] Step 4: Data Acquisition (Device)

[0677] The terminal obtains charging spot information and traffic congestion information in real time from the server. The input to this step is request information to the server. Based on this, the terminal requests data from the server and receives the latest charging spot information and traffic information as output. Specifically, the terminal sends requests to the server at regular intervals to obtain the latest information.

[0678] Step 5: Voice command analysis (device)

[0679] The device analyzes the user's voice command with a voice recognition system. The input for this step is the user's voice command. The device converts this voice into text and analyzes it for appropriate processing. The output is the analyzed text data. Specifically, the device's voice recognition engine converts the voice "Find nearby charging spots" into text and retrieves information based on the request.

[0680] Step 6: Sentiment Analysis (Device)

[0681] The device uses an emotion engine to analyze emotions from the user's voice and facial expressions. The input for this step is the user's voice and facial expression data. The device analyzes these and determines the user's emotional state. The output is a system response based on the emotional state. Specifically, if the device's emotion engine analyzes the user's tone of voice and facial expression and recognizes that the user is irritated, the device will respond in a calm voice, saying, "Relax and drive safely."

[0682] Step 7: Navigation Display (Device)

[0683] The device updates the navigation screen based on the data obtained from the server. The input for this step is the latest charging spot information and traffic information. The device calculates the optimal route based on this input and displays it on the navigation screen. The output is the latest route guidance. Specifically, the device displays information on the charging spot closest to the user and navigates the route. It also recalculates the optimal route based on traffic congestion information and suggests detour routes.

[0684] Step 8: Voice command input (user)

[0685] The user issues a voice command from inside the car to give instructions to the system. The input for this step is the user's voice command. Based on this, the user sends a request to the system. The output is the analyzed voice command, which is processed by the system. In concrete terms, if the user says, "Find a nearby charging spot," the device will recognize the voice and provide appropriate information.

[0686] Step 9: Smartphone operation (user)

[0687] The user accesses the system from outside the vehicle using a smartphone app. The input for this step is the operation information of the smartphone app. The user uses this information to search for charging spots and set their destination. The output is the setting information synchronized with the in-vehicle navigation system. In concrete terms, the user sets their next destination using their smartphone while taking a break at a cafe and checks available charging spots in advance.

[0688] Step 10: System Sync (User)

[0689] The information set by the user is synchronized with the navigation system in the vehicle. The input for this step is the setting information from a smartphone. The user uses this input to make the system immediately available when getting in the car. The setting information is then reflected in the navigation system as an output. In concrete terms, the moment the user gets into the car, the navigation system begins guiding the user along the pre-set route. If the emotion engine determines that the stress level is high, it will suggest smooth routes and relaxing music.

[0690] (Application example 2)

[0691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Current self-driving vehicles and navigation systems can provide information on charging spots and traffic congestion, but they cannot respond to the user's emotions or make suggestions to reduce stress. As a result, they cannot reduce the stress and anxiety caused by long drives, making it difficult to improve the overall driving experience.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0694] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, an emotion engine for analyzing the user's emotions, and means for adjusting the system's response based on the emotions analyzed by the emotion engine. This enables the user to not only obtain information about charging spots and the optimal route, but also receive suggestions for reducing emotional stress.

[0695] "Charging spot information" is data about locations where electric vehicles can be charged, including location information, availability, and type of charger.

[0696] "Speech recognition" is a technology that analyzes a user's voice and converts it into text data.

[0697] An "optimal route" is the most efficient and time-saving route for a user to reach their destination.

[0698] A "navigation screen" is a display device that visually provides the user with information about the current location and route to the destination.

[0699] The "emotion engine" is a device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[0700] A "server" is an information processing device that collects data via a network, processes and stores it, and provides it to clients.

[0701] "Adjusting the system's response based on emotions" refers to controlling the system to respond appropriately based on the user's emotional state.

[0702] The embodiment of the present invention is composed of a server, a terminal, a user, and an emotion engine. This system enables users of autonomous vehicles to enjoy a comfortable and stress-free driving experience.

[0703] Server-side processing

[0704] The server periodically sends requests to the API to collect charging spot information and obtains the latest data. This information includes location, availability, and type of charger. This information is stored in an internal database and provided in response to requests from users' devices. Traffic and congestion information is also collected from another API and stored in the database.

[0705] Terminal side processing

[0706] The device receives real-time information on charging spots and traffic congestion from the server and provides it to the user. It uses a voice recognition system to analyze the user's voice commands, obtain the necessary information, and display it on the navigation screen. It also uses an emotion engine to analyze the user's emotions and adjust the system's response.

[0707] User operations

[0708] Users can issue voice commands to the system from inside the car, and can also access the system from outside the car using a smartphone application to search for charging spots and set destinations. The information set on the smartphone is automatically synchronized with the vehicle's navigation system.

[0709] Hardware and software used

[0710] The hardware used includes a smartphone, microphone, camera, internet connection, etc. The software used includes Python, the Requests library, the Geopy library, the SpeechRecognition library, and the EmotionEngine. This hardware and software enables voice recognition, emotion analysis, and the display of navigation data.

[0711] Specific examples

[0712] In a specific scenario, when a user issues a voice command such as "Find a nearby charging spot," the device's voice recognition engine converts this into text, retrieves appropriate charging spot information from the server, and displays it on the screen. At the same time, if the emotion engine determines that the user is feeling stressed based on their tone of voice or facial expression, the device will notify them by saying, "Relax and drive safely."

[0713] Example prompts for generative AI models

[0714] An example of a prompt for a generative AI model is:

[0715] "Explanation of the goal

[0716] I want to develop an application that allows users of autonomous vehicles to receive charging spot information and navigation.

[0717] We would also like to include a function that recognizes the user's emotions and provides advice to reduce stress.

[0718] Input data

[0719] 1. Charging spot API data (location, availability, charger type)

[0720] 2. Traffic congestion information API data

[0721] 3. User voice commands

[0722] 4. User emotion analysis data (voice tone, facial expressions)

[0723] Output Data

[0724] 1. Information on the nearest charging spot

[0725] 2. Real-time navigation directions

[0726] 3. Response messages based on user sentiment

[0727] Suggested Application Features

[0728] 1. Charging spot search and reservation function

[0729] 2. Real-time navigation function

[0730] 3. Voice command reception function

[0731] 4. Response adjustment function based on emotion analysis

[0732] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0733] Step 1:

[0734] The server sends a request to the charging spot API to get the latest charging spot information. The input is the charging spot API URL, and the output is JSON data including location information, availability, and charger type. This data is parsed and stored in an internal database.

[0735] Step 2:

[0736] The server sends a request to the traffic information API to get the latest traffic congestion information. The input is the traffic information API URL, and the output is JSON data about traffic congestion. This data is also parsed and stored in the internal database.

[0737] Step 3:

[0738] The user issues a voice command. The device inputs the voice through the microphone and converts it into text using speech recognition technology (SpeechRecognition library). For example, if the input is "Find nearby charging spots," the output is this text data.

[0739] Step 4:

[0740] The device that receives the text-translated voice command requests charging spot information from the server based on the command. The input is the text data "Search for nearby charging spots," and the output is information about the nearest charging spot (location information, availability, etc.) sent from the server.

[0741] Step 5:

[0742] The device combines the charging spot information received from the server with the current location information to calculate the optimal route. The input is charging spot information and current location information, and the output is route data for the optimal route. This calculation uses the Geopy library.

[0743] Step 6:

[0744] After the optimal route is calculated, the device displays this route information on the navigation screen. The input is the route data of the optimal route, and the output is a visual navigation display. The user can check the route to the charging spot by looking at the screen.

[0745] Step 7:

[0746] At the same time, the device uses an emotion engine to analyze the user's emotions. The input is the user's voice tone and facial expression data, and the output is data indicating the user's emotional state (e.g., stressed, angry, relaxed). This analysis is performed using the EmotionEngine.

[0747] Step 8:

[0748] Based on the analyzed emotional data, the device adjusts the response message. The input is emotional data, and the output is an appropriate response message (e.g., "Relax and drive safely."). This gives the user a sense of security.

[0749] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0751] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0752] [Third embodiment]

[0753] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0754] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0755] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0756] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0757] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0758] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0759] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0760] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0761] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0762] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0763] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0764] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0765] Overall system configuration

[0766] This system consists of a server, a terminal, and a user. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone.

[0767] What the program does

[0768] Server-side processing

[0769] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0770] Examples:

[0771] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0772] Terminal side processing

[0773] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on the commands, and displays it on the navigation screen. It also constantly monitors the current location and destination information, and calculates the optimal route based on the information from the server. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0774] Examples:

[0775] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If traffic jams occur during driving, the device suggests a new detour route based on the latest traffic information from the server.

[0776] User operation

[0777] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0778] Examples:

[0779] While taking a break at a cafe, the user can set their next destination on their smartphone and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and starts guiding the user once they get in the car.

[0780] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and by linking with a smartphone, they can also be operated comfortably from outside the vehicle.

[0781] The processing flow will be explained below.

[0782] Server-side processing

[0783] Step 1:

[0784] The server sends a request to the charging spot API, which returns data such as the latest location of the charging spot, availability, and type of charger.

[0785] Step 2:

[0786] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0787] Step 3:

[0788] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0789] Step 4:

[0790] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0791] Terminal side processing

[0792] Step 1:

[0793] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0794] Step 2:

[0795] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0796] Step 3:

[0797] The user issues a voice command, for example, "Find nearby charging spots."

[0798] Step 4:

[0799] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0800] Step 5:

[0801] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0802] Step 6:

[0803] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0804] Step 7:

[0805] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0806] User operation

[0807] Step 1:

[0808] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[0809] Step 2:

[0810] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[0811] Step 3:

[0812] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[0813] Step 4:

[0814] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[0815] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably.

[0816] Example 1

[0817] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0818] Conventional navigation systems face challenges in terms of their ability to acquire and update charging station and traffic information in real time and dynamically present optimal routes. Furthermore, they lack the functionality to allow users to set a destination outside the vehicle and instantly synchronize it with the vehicle's navigation system, significantly reducing user convenience. Furthermore, the limited use of voice recognition systems presents a problem in that they are unable to efficiently analyze diverse voice commands.

[0819] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0820] In this invention, the server includes a means for periodically collecting charging station information, traffic information, and congestion information and updating the internal database, a means for calculating the optimal route, and a means for analyzing user instructions using voice recognition and converting the obtained instructions into text data. This makes it possible to dynamically calculate and present the optimal route based on charging station information and traffic information updated in real time. Furthermore, the destination set by the user using a smart device outside the vehicle can be synchronized with the vehicle's navigation system, greatly improving convenience. In addition, advanced voice recognition technology allows for efficient analysis and execution of a variety of voice instructions.

[0821] "Charging station information" refers to data such as the location, availability, and type of charger of the equipment that supplies power to electric vehicles.

[0822] "Speech recognition" refers to the technology that analyzes the voice spoken by the user and converts it into corresponding text data.

[0823] "User instructions" refer to requests or commands made by a user to a system that direct a particular operation of the system.

[0824] "Means for calculating the optimal route" refers to a device or software that uses an algorithm to calculate the optimal route to the destination based on information obtained by the server and terminal.

[0825] "Navigation screen" means a display device installed inside a vehicle that visually presents route information and other related data.

[0826] "Server" refers to a computer system that collects, processes, and manages data via a network.

[0827] "Periodic collection means" refers to a method or device that accesses external APIs or other data sources at regular intervals to obtain the required data.

[0828] "Internal database" refers to a data management system that stores collected data and enables quick retrieval of required information.

[0829] "Smart device" refers to a mobile device (such as a smartphone or tablet) with internet connectivity that allows access to and operation of the system.

[0830] "Means of synchronization" refers to the technology or method that keeps data consistent across multiple devices.

[0831] "Means for displaying in real time" refers to a device or software that has the function of instantly displaying acquired information to a user.

[0832] This invention relates to a system that collects charging station information, traffic information, and congestion information in real time and dynamically presents optimal routes. This system consists of three main parts: a server, a terminal, and a user.

[0833] Server Features

[0834] The server is responsible for periodically collecting charging station information, traffic information, and congestion information. Charging station information includes location information, availability, charger type, etc., and is obtained from an external charging spot API via an HTTP GET request. The obtained data is received in JSON format, parsed, and saved / updated in the internal database.

[0835] Similarly, traffic and congestion information is periodically obtained from an external traffic information API. The server analyzes this information and keeps it up to date.

[0836] Examples:

[0837] The server accesses the charging spot API every hour to obtain the latest charging station information. After obtaining the information, the server analyzes the data and updates the nationwide charging station database. Traffic information is also updated in the same way.

[0838] Device Features

[0839] The terminal is the main device that receives real-time charging station and traffic information from the server. The terminal is equipped with a voice recognition system that analyzes the user's voice commands and converts them into text data. The terminal then sends the appropriate request to the server, retrieves the necessary information, and displays it on the navigation screen.

[0840] Based on the acquired information, the device constantly monitors the current location and destination information and calculates the optimal route. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[0841] Examples:

[0842] When a user says "Find nearby charging spots," the device's voice recognition engine (for example, Google Speech-to-Text API) converts this into text. The device then retrieves information about the nearest charging spots from the server, displays it on the screen, and provides directions. If traffic jams occur during a drive, the device will suggest a new route based on the new traffic conditions.

[0843] User operations

[0844] Users can issue instructions to the system using voice commands from inside the car. For example, the system responds to voice commands such as "Find nearby charging spots" or "Set my next destination." Users can also access the system from outside the car using a smart device to search for charging stations or set destinations. The information set from the smart device is synchronized with the vehicle's navigation system, making it available the moment the user gets into the car.

[0845] Examples:

[0846] While taking a break at a cafe, a user can use their smart device to set their next destination and check nearby charging stations in advance. This information is automatically synchronized with the vehicle's navigation system, so that directions can begin immediately when the user gets in the car.

[0847] Example prompt sentence:

[0848] "My system involves a server collecting charging spot information and traffic congestion information, and an in-car terminal using that information to navigate the vehicle. The user can issue instructions to the system via voice commands or a smartphone. Please explain the process of this system in detail, dividing it into specific steps. Please also include the roles of the server, terminal, and user, as well as the specific flow of processing. Please also include a specific example of how it works."

[0849] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0850] Step 1:

[0851] The server sends an HTTP GET request to the charging station API to obtain the latest charging station information.

[0852] Input: Charging Station API Endpoint URL

[0853] Output: Charging station information data in JSON format

[0854] Specifically, the server calls the charging station API at specific time intervals (e.g., every hour) to obtain data such as location, availability, and type of charger.

[0855] Step 2:

[0856] The server analyzes the acquired JSON-formatted data, extracts information such as the location of the charging station, availability, and type of charger, and stores it in an internal database.

[0857] Input: Charging station information data in JSON format

[0858] Output: Charging station information stored in a database

[0859] Specifically, the server performs parsing (analysis) processing, extracts necessary information, and stores it in a "charging station information" table.

[0860] Step 3:

[0861] The server sends an HTTP GET request to the traffic information API to obtain the latest traffic conditions and congestion information.

[0862] Input: Traffic Information API endpoint URL

[0863] Output: Traffic and congestion information data in JSON format

[0864] Specifically, the server accesses the traffic information API at regular intervals to obtain the latest traffic conditions and congestion information.

[0865] Step 4:

[0866] The server analyzes the acquired traffic information and stores it in an internal database.

[0867] Input: Traffic and congestion information data in JSON format

[0868] Output: Traffic situation and congestion information stored in a database

[0869] Specifically, the server parses the acquired JSON data, extracts the necessary traffic information, and stores it in a database.

[0870] Step 5:

[0871] The terminal sends an HTTP GET request to the server to obtain charging station information and traffic information in real time.

[0872] Input: Server endpoint URL

[0873] Output: Real-time charging station information and traffic information

[0874] Specifically, the terminal obtains the necessary information from the server in real time in response to a user request.

[0875] Step 6:

[0876] The terminal uses a voice recognition system to analyze the user's voice commands and convert them into text data.

[0877] Input: User's voice command

[0878] Output: Text data

[0879] Specifically, a speech recognition engine (for example, Google Speech-to-Text API) is used to convert the user's speech into text data.

[0880] Step 7:

[0881] The terminal sends an appropriate request to the server based on the acquired text data and acquires the required information.

[0882] Input: Text data (the result of converting the user's voice command)

[0883] Output: Response data from the server (charging station information, traffic information, etc.)

[0884] Specifically, the terminal sends a request to the server based on the analyzed text data and obtains related information from the server.

[0885] Step 8:

[0886] Based on the acquired information, the device displays charging station information and the optimal route on the navigation screen.

[0887] Input: Charging station information and traffic information obtained from the server

[0888] Output: Charging station information and optimal route displayed on the navigation screen

[0889] Specifically, the device uses Google Maps API and other tools to display charging station information and traffic information on a map, and presents the optimal route on the navigation screen.

[0890] Step 9:

[0891] The device calculates the optimal route based on the current location and destination information, and if traffic congestion occurs on the route, it calculates a new detour route.

[0892] Input: current location, destination, traffic information

[0893] Output: Optimal route and detour route information

[0894] Specifically, the device dynamically calculates the optimal route based on traffic information updated in real time, and presents new detour routes if necessary.

[0895] Step 10:

[0896] Users can access the system from outside the vehicle using a smart device to search for charging stations and set destinations, and then synchronize that information with the in-car navigation system.

[0897] Input: Destination setting information from smart device, charging station search results

[0898] Output: Destination and charging station information synchronized with the in-car navigation system

[0899] Specifically, information set by the user using a smartphone app is synchronized in real time with the vehicle's navigation system, making the information instantly available in the vehicle.

[0900] (Application example 1)

[0901] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0902] Conventional navigation systems for autonomous vehicles lack the ability to collect information on charging spots and traffic congestion, making it difficult to calculate optimal routes in real time. Furthermore, they lack connectivity with smartphones and other devices, making it difficult for users to operate them easily. Furthermore, there was also the issue of incomplete operational support for autonomous vehicles in response to user voice commands.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0904] In this invention, the server includes means for acquiring charging spot information, means for analyzing user requests through voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, and means for synchronizing and manipulating data in real time via a smartphone or terminal. This makes it possible to acquire charging spot information and traffic congestion information in real time, dynamically calculate an optimal route based on the user's voice commands, and efficiently support the operation of autonomous vehicles.

[0905] "Charging spot information" is information about locations where electric vehicles can be charged, and includes location information, availability, type of charger, and the like.

[0906] "Speech recognition" is a technology that analyzes voice input and converts it into text data, in order to recognize a user's voice commands.

[0907] A "server" is a computer system that manages and processes data on a network, and is responsible for acquiring, storing, and providing information on charging spots and traffic congestion.

[0908] "Means for calculating the optimal route" refers to methods or technologies that derive the optimal route based on the user's current location and destination, and refer to charging spot information and traffic congestion information.

[0909] The "navigation screen" is a screen that shows route information, charging spot information, and the like, displayed on a display inside the vehicle.

[0910] "Means for synchronizing and manipulating data in real time through smartphones and terminals" refers to technology that allows charging spot information and route information to be updated and manipulated in real time using smartphones and other devices.

[0911] "Traffic congestion information" is data relating to traffic conditions on roads, including vehicle speeds, congestion, accidents, and the like.

[0912] An "autonomous vehicle" is a vehicle that can drive autonomously without driver intervention.

[0913] "Means for assisting the operation of autonomous vehicles in response to voice commands" refers to technology that enables autonomous vehicles to perform appropriate operations and route changes based on the user's voice input.

[0914] "Means for displaying the current availability of charging spots in real time" refers to methods or technologies that constantly update the availability of charging spots and display it to users immediately.

[0915] "Location information of the nearest charging spot" is data indicating the location of the charging spot that is the shortest distance from the user's current location.

[0916] To implement the present invention, the roles played by the server, the terminal, and the user will be described.

[0917] Server Roles

[0918] The server collects and processes charging spot and traffic congestion information, and provides this information to the terminal. A general server computer is used as the hardware, and Flask (framework) and Python (programming language) are used as the software. The server periodically accesses the charging spot API and traffic information API, and stores the collected data in a database.

[0919] Specific server operations

[0920] The server periodically makes API requests to collect data such as the availability of charging spots, their locations, and the type of charger. This data is stored in a database on the server and updated in real time. Traffic and congestion information is also collected and added to the database in real time.

[0921] Device Role

[0922] The in-vehicle device uses information obtained from the server to perform navigation and voice recognition operations. Google's Speech-to-Text API is used for voice recognition, and the entire system is built using Python and Flask. It also has the ability to synchronize data in real time with smartphones and other devices.

[0923] Specific operation of the device

[0924] The device receives the user's voice command and converts it into text using Google's Speech-to-Text API. For example, if the user says, "Find nearby charging spots," the speech is converted into text and sent to the server as an appropriate request. The server searches for charging spots based on the latest information and returns the results to the device. The device then displays the search results on the navigation screen and provides the user with directions to the nearest charging spot.

[0925] User Roles

[0926] Users can operate the system via voice commands or a smartphone app. Information and settings acquired through user operations are synchronized in real time with the device inside the vehicle, allowing the system to be easily operated even from outside the vehicle.

[0927] Specific user operation examples

[0928] While taking a break at a cafe, the user can use their smartphone to set their next destination and check the nearest charging spot. This information is automatically synchronized with the vehicle's navigation system, and guidance begins as soon as the user gets in the car. When the user uses the smartphone app to enter the prompt "Find nearby charging spots," the system will provide information on the optimal charging spots.

[0929] Example prompt sentence:

[0930] Find a nearby charging spot

[0931] These functions allow users to receive information on charging spots and optimal route guidance in real time, and the system can be conveniently operated by linking it with a smartphone or other device.

[0932] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0933] Step 1:

[0934] The server periodically sends requests to the charging spot API and traffic information API. The server parses the data obtained from the API and stores the available charging spots, their locations, charger types, and traffic congestion information in a database. The input is the API response, and the output is the parsed charging spot information and traffic congestion information updates.

[0935] Step 2:

[0936] The user speaks a voice command. The device in the vehicle converts the speech to text using Google's Speech-to-Text API. The voice command is the input, and the generated text is the output. For example, the voice command might be "Find nearby charging spots."

[0937] Step 3:

[0938] The device parses the voice command converted to text and requests the necessary information from the server. The device sends a request to the appropriate API endpoint based on the text generated by Google's Speech-to-Text API. The input is the voice command converted to text, and the output is a request to the server.

[0939] Step 4:

[0940] When the server receives a request from the device, it retrieves the appropriate charging spot information and traffic congestion information from the database and returns it to the device in JSON format. The input is the request from the device, and the output is the data obtained and returned by the server.

[0941] Step 5:

[0942] The device displays the information received from the server and shows the location information and route to the nearest charging spot on the navigation screen. The input is information from the server, and the output is what is displayed on the navigation screen. For example, the device may display that the nearest charging spot is 500 meters from the vehicle and provide route information.

[0943] Step 6:

[0944] The user refers to the navigation screen and drives the vehicle to the displayed charging spot or uses the automatic driving function. The input is the information on the navigation screen, and the output is the user's movement or the automatic driving of the vehicle.

[0945] Step 7:

[0946] When the vehicle arrives at a charging spot, charging begins. The server then periodically updates the data and provides new information in real time. The input is the current status of the charging spot, and the output is the start of use of the charging spot.

[0947] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0948] Overall system configuration

[0949] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[0950] What the program does

[0951] Server-side processing

[0952] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[0953] Examples:

[0954] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[0955] Terminal side processing

[0956] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on those commands, and displays it on the navigation screen. It also uses an emotion engine to analyze the user's emotions from their voice and facial expressions, and adjusts the system's response based on those results.

[0957] Examples:

[0958] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0959] The device constantly monitors the user's current location and destination, and calculates the optimal route based on the latest traffic congestion information. If a traffic jam occurs, the device calculates a new detour route and presents it to the user. At the same time, based on information from the emotion engine, if the user is feeling stressed, the device adjusts the guidance to make route selection as simple as possible.

[0960] User operation

[0961] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[0962] Examples:

[0963] While taking a break at a cafe, the user sets their next destination on their smartphone and checks available charging spots in advance. Based on this, the car's navigation system automatically sets a route and starts guiding the user once they get in the car. If the emotion engine determines that the user's stress level is high during the journey, it will suggest a smoother route or relaxing music.

[0964] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and the emotion engine also provides a comfortable driving experience.

[0965] The processing flow will be explained below.

[0966] Processing flow of a system that combines emotion engines

[0967] Server-side processing

[0968] Step 1:

[0969] The server periodically sends requests to the charging spot API, which returns data such as the latest location, availability, and type of charger for the charging spot.

[0970] Step 2:

[0971] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[0972] Step 3:

[0973] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[0974] Step 4:

[0975] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[0976] Terminal side processing

[0977] Step 1:

[0978] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[0979] Step 2:

[0980] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[0981] Step 3:

[0982] The user issues a voice command, for example, "Find nearby charging spots."

[0983] Step 4:

[0984] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[0985] Step 5:

[0986] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[0987] Step 6:

[0988] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[0989] Step 7:

[0990] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[0991] Step 8:

[0992] The device analyzes the user's voice and facial expressions using an emotion engine, which determines the user's stress level and emotional state.

[0993] Step 9:

[0994] After the emotion engine recognizes the user's emotions, the device can adjust navigation and system responses based on that information. For example, if the user is feeling stressed, it can provide a calming voice prompt to encourage the user to relax.

[0995] Examples:

[0996] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[0997] User operation

[0998] Step 1:

[0999] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[1000] Step 2:

[1001] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[1002] Step 3:

[1003] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[1004] Step 4:

[1005] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[1006] Step 5:

[1007] If the emotion engine determines that the user's stress level is high along the way, the device will adjust the guidance to make route selection as simple as possible.

[1008] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably. In addition, the introduction of an emotion engine provides optimal support according to the user's emotional state.

[1009] Example 2

[1010] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1011] Conventional navigation systems can provide real-time information on charging spots and traffic congestion, but they are unable to respond appropriately taking into account the user's emotional state. This can lead to stress for drivers and can lead to a less comfortable driving environment.

[1012] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1013] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server and the user's emotions, means for displaying the route information on a navigation screen, and means for recognizing emotions from the user's voice and facial expressions and adjusting the system's response, thereby making it possible to provide optimal route guidance and appropriate responses according to the user's emotional state.

[1014] "Charging spot information" is data such as location information, availability, and type of charger regarding places where electric vehicles can be charged.

[1015] "Speech recognition" is a technology that converts a user's voice commands into text and analyzes their intent.

[1016] The "optimal route" is the most efficient and effective route to your destination, calculated based on charging spot information and traffic congestion information.

[1017] A "server" is a computer system that collects information on charging spots, traffic congestion, and other real-time data and provides it to terminals.

[1018] The "navigation screen" is a screen on which the terminal visually presents route guidance and charging spot information to the user.

[1019] The "emotion engine" is a technology that analyzes the user's voice and facial expressions to recognize their emotional state.

[1020] "Traffic congestion information" is data relating to road traffic conditions and the degree of congestion.

[1021] "Means for displaying in real time" refers to techniques or methods that instantly display the current situation to the user.

[1022] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[1023] Server-side processing

[1024] The server sends requests to the charging spot API and traffic information API at regular intervals to obtain the latest charging spot information and traffic congestion information. The obtained data includes information such as location, availability, and charger type, and is parsed and updated in an internal database. Specifically, it obtains data from major charging spot APIs (for example, the EV charging network API) and traffic congestion information from Google Maps' traffic data API.

[1025] Terminal side processing

[1026] The device obtains charging spot and traffic congestion information in real time from the server. The device is equipped with a voice recognition system that analyzes the user's voice commands, retrieves appropriate information, and displays it on the navigation screen. The emotion engine analyzes the user's voice and facial expressions, recognizes their emotions, and adjusts the system's response accordingly. For example, if a user says, "Find nearby charging spots," the device's voice recognition engine analyzes this and retrieves and displays the most appropriate charging spot information from the server. If the emotion engine analyzes the user's tone of voice and facial expressions and determines that they are stressed, the device will respond in a calm voice, saying, "Relax and drive safely."

[1027] User operation

[1028] Users can issue instructions to the system using voice commands. They can also access the system from outside the car using a smartphone app to search for charging spots and set destinations. Information set on the smartphone is synchronized with the car's navigation system and is immediately available when they get in. For example, while taking a break at a cafe, a user can use their smartphone to set their next destination and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and begins guiding the user as soon as they get in the car. If the emotion engine determines that the user's stress level is high, it will suggest smooth routes and relaxing music.

[1029] Examples of prompt statements

[1030] Example prompt for explaining server-side processing: "Please explain the server-side process that collects the latest charging spot information and traffic congestion information."

[1031] Example prompt for device-side processing: "Describe the device-side process where a user uses voice commands to find a charging spot. Also, explain how you use an emotion engine to tailor the response."

[1032] Example prompt for user interaction: "Describe how a user would set their next destination and search for charging spots on their smartphone. Include an example of how the emotion engine would respond if their stress level was high."

[1033] This system allows users to receive real-time information on electric vehicle charging spots and optimal route guidance, and also provides a comfortable driving experience through an emotion engine.

[1034] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1035] Step 1: Data Collection (Server)

[1036] The server sends periodic requests to the charging spot API and traffic information API. The input for this step is the API endpoint URL and necessary authentication information. The server uses these inputs to generate and send a data request. As output, it receives charging spot information and traffic information in JSON format. Specifically, the server sends a request every hour to the main charging spot API (e.g., EV charging network API) to obtain the latest data. It also sends a request to Google Maps traffic data API to obtain traffic congestion information.

[1037] Step 2: Data Parsing (Server)

[1038] The server parses the JSON format data obtained in step 1. The input for this step is the obtained JSON data. The server parses this data and extracts the necessary information (e.g., location information, availability, and type of charger). As an output, the extracted data is imported into an internal database. Specifically, the server parses the responses obtained from the charging spot API and traffic information API, and classifies and extracts the respective information.

[1039] Step 3: Database Update (Server)

[1040] The server immediately updates the information extracted in step 2 to its internal database. The input to this step is the extracted charging spot information and traffic information. The server updates the corresponding fields in the database based on this input. The output is an updated database that reflects the new information. Specifically, the server adds and updates the extracted charging spot information to the nationwide charging spot database. Similarly, traffic congestion information is also updated in the database.

[1041] Step 4: Data Acquisition (Device)

[1042] The terminal obtains charging spot information and traffic congestion information in real time from the server. The input to this step is request information to the server. Based on this, the terminal requests data from the server and receives the latest charging spot information and traffic information as output. Specifically, the terminal sends requests to the server at regular intervals to obtain the latest information.

[1043] Step 5: Voice command analysis (device)

[1044] The device analyzes the user's voice command with a voice recognition system. The input for this step is the user's voice command. The device converts this voice into text and analyzes it for appropriate processing. The output is the analyzed text data. Specifically, the device's voice recognition engine converts the voice "Find nearby charging spots" into text and retrieves information based on the request.

[1045] Step 6: Sentiment Analysis (Device)

[1046] The device uses an emotion engine to analyze emotions from the user's voice and facial expressions. The input for this step is the user's voice and facial expression data. The device analyzes these and determines the user's emotional state. The output is a system response based on the emotional state. Specifically, if the device's emotion engine analyzes the user's tone of voice and facial expression and recognizes that the user is irritated, the device will respond in a calm voice, saying, "Relax and drive safely."

[1047] Step 7: Navigation Display (Device)

[1048] The device updates the navigation screen based on the data obtained from the server. The input for this step is the latest charging spot information and traffic information. The device calculates the optimal route based on this input and displays it on the navigation screen. The output is the latest route guidance. Specifically, the device displays information on the charging spot closest to the user and navigates the route. It also recalculates the optimal route based on traffic congestion information and suggests detour routes.

[1049] Step 8: Voice command input (user)

[1050] The user issues a voice command from inside the car to give instructions to the system. The input for this step is the user's voice command. Based on this, the user sends a request to the system. The output is the analyzed voice command, which is processed by the system. In concrete terms, if the user says, "Find a nearby charging spot," the device will recognize the voice and provide appropriate information.

[1051] Step 9: Smartphone operation (user)

[1052] The user accesses the system from outside the vehicle using a smartphone app. The input for this step is the operation information of the smartphone app. The user uses this information to search for charging spots and set their destination. The output is the setting information synchronized with the in-vehicle navigation system. In concrete terms, the user sets their next destination using their smartphone while taking a break at a cafe and checks available charging spots in advance.

[1053] Step 10: System Sync (User)

[1054] The information set by the user is synchronized with the navigation system in the vehicle. The input for this step is the setting information from a smartphone. The user uses this input to make the system immediately available when getting in the car. The setting information is then reflected in the navigation system as an output. In concrete terms, the moment the user gets into the car, the navigation system begins guiding the user along the pre-set route. If the emotion engine determines that the stress level is high, it will suggest smooth routes and relaxing music.

[1055] (Application example 2)

[1056] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1057] Current self-driving vehicles and navigation systems can provide information on charging spots and traffic congestion, but they cannot respond to the user's emotions or make suggestions to reduce stress. As a result, they cannot reduce the stress and anxiety caused by long drives, making it difficult to improve the overall driving experience.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1059] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, an emotion engine for analyzing the user's emotions, and means for adjusting the system's response based on the emotions analyzed by the emotion engine. This enables the user to not only obtain information about charging spots and the optimal route, but also receive suggestions for reducing emotional stress.

[1060] "Charging spot information" is data about locations where electric vehicles can be charged, including location information, availability, and type of charger.

[1061] "Speech recognition" is a technology that analyzes a user's voice and converts it into text data.

[1062] An "optimal route" is the most efficient and time-saving route for a user to reach their destination.

[1063] A "navigation screen" is a display device that visually provides the user with information about the current location and route to the destination.

[1064] The "emotion engine" is a device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1065] A "server" is an information processing device that collects data via a network, processes and stores it, and provides it to clients.

[1066] "Adjusting the system's response based on emotions" refers to controlling the system to respond appropriately based on the user's emotional state.

[1067] The embodiment of the present invention is composed of a server, a terminal, a user, and an emotion engine. This system enables users of autonomous vehicles to enjoy a comfortable and stress-free driving experience.

[1068] Server-side processing

[1069] The server periodically sends requests to the API to collect charging spot information and obtains the latest data. This information includes location, availability, and type of charger. This information is stored in an internal database and provided in response to requests from users' devices. Traffic and congestion information is also collected from another API and stored in the database.

[1070] Terminal side processing

[1071] The device receives real-time information on charging spots and traffic congestion from the server and provides it to the user. It uses a voice recognition system to analyze the user's voice commands, obtain the necessary information, and display it on the navigation screen. It also uses an emotion engine to analyze the user's emotions and adjust the system's response.

[1072] User operations

[1073] Users can issue voice commands to the system from inside the car, and can also access the system from outside the car using a smartphone application to search for charging spots and set destinations. The information set on the smartphone is automatically synchronized with the vehicle's navigation system.

[1074] Hardware and software used

[1075] The hardware used includes a smartphone, microphone, camera, internet connection, etc. The software used includes Python, the Requests library, the Geopy library, the SpeechRecognition library, and the EmotionEngine. This hardware and software enables voice recognition, emotion analysis, and the display of navigation data.

[1076] Specific examples

[1077] In a specific scenario, when a user issues a voice command such as "Find a nearby charging spot," the device's voice recognition engine converts this into text, retrieves appropriate charging spot information from the server, and displays it on the screen. At the same time, if the emotion engine determines that the user is feeling stressed based on their tone of voice or facial expression, the device will notify them by saying, "Relax and drive safely."

[1078] Example prompts for generative AI models

[1079] An example of a prompt for a generative AI model is:

[1080] "Explanation of the goal

[1081] I want to develop an application that allows users of autonomous vehicles to receive charging spot information and navigation.

[1082] We would also like to include a function that recognizes the user's emotions and provides advice to reduce stress.

[1083] Input data

[1084] 1. Charging spot API data (location, availability, charger type)

[1085] 2. Traffic congestion information API data

[1086] 3. User voice commands

[1087] 4. User emotion analysis data (voice tone, facial expressions)

[1088] Output Data

[1089] 1. Information on the nearest charging spot

[1090] 2. Real-time navigation directions

[1091] 3. Response messages based on user sentiment

[1092] Suggested Application Features

[1093] 1. Charging spot search and reservation function

[1094] 2. Real-time navigation function

[1095] 3. Voice command reception function

[1096] 4. Response adjustment function based on emotion analysis

[1097] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1098] Step 1:

[1099] The server sends a request to the charging spot API to get the latest charging spot information. The input is the charging spot API URL, and the output is JSON data including location information, availability, and charger type. This data is parsed and stored in an internal database.

[1100] Step 2:

[1101] The server sends a request to the traffic information API to get the latest traffic congestion information. The input is the traffic information API URL, and the output is JSON data about traffic congestion. This data is also parsed and stored in the internal database.

[1102] Step 3:

[1103] The user issues a voice command. The device inputs the voice through the microphone and converts it into text using speech recognition technology (SpeechRecognition library). For example, if the input is "Find nearby charging spots," the output is this text data.

[1104] Step 4:

[1105] The device that receives the text-translated voice command requests charging spot information from the server based on the command. The input is the text data "Search for nearby charging spots," and the output is information about the nearest charging spot (location information, availability, etc.) sent from the server.

[1106] Step 5:

[1107] The device combines the charging spot information received from the server with the current location information to calculate the optimal route. The input is charging spot information and current location information, and the output is route data for the optimal route. This calculation uses the Geopy library.

[1108] Step 6:

[1109] After the optimal route is calculated, the device displays this route information on the navigation screen. The input is the route data of the optimal route, and the output is a visual navigation display. The user can check the route to the charging spot by looking at the screen.

[1110] Step 7:

[1111] At the same time, the device uses an emotion engine to analyze the user's emotions. The input is the user's voice tone and facial expression data, and the output is data indicating the user's emotional state (e.g., stressed, angry, relaxed). This analysis is performed using the EmotionEngine.

[1112] Step 8:

[1113] Based on the analyzed emotional data, the device adjusts the response message. The input is emotional data, and the output is an appropriate response message (e.g., "Relax and drive safely."). This gives the user a sense of security.

[1114] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1115] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1116] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1117] [Fourth embodiment]

[1118] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1119] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1120] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1121] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1122] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1123] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1124] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1125] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1126] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1127] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1128] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1129] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1130] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1131] Overall system configuration

[1132] This system consists of a server, a terminal, and a user. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone.

[1133] What the program does

[1134] Server-side processing

[1135] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[1136] Examples:

[1137] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[1138] Terminal side processing

[1139] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on the commands, and displays it on the navigation screen. It also constantly monitors the current location and destination information, and calculates the optimal route based on the information from the server. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[1140] Examples:

[1141] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If traffic jams occur during driving, the device suggests a new detour route based on the latest traffic information from the server.

[1142] User operation

[1143] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[1144] Examples:

[1145] While taking a break at a cafe, the user can set their next destination on their smartphone and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and starts guiding the user once they get in the car.

[1146] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and by linking with a smartphone, they can also be operated comfortably from outside the vehicle.

[1147] The processing flow will be explained below.

[1148] Server-side processing

[1149] Step 1:

[1150] The server sends a request to the charging spot API, which returns data such as the latest location of the charging spot, availability, and type of charger.

[1151] Step 2:

[1152] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[1153] Step 3:

[1154] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[1155] Step 4:

[1156] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[1157] Terminal side processing

[1158] Step 1:

[1159] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[1160] Step 2:

[1161] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[1162] Step 3:

[1163] The user issues a voice command, for example, "Find nearby charging spots."

[1164] Step 4:

[1165] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[1166] Step 5:

[1167] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[1168] Step 6:

[1169] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[1170] Step 7:

[1171] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[1172] User operation

[1173] Step 1:

[1174] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[1175] Step 2:

[1176] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[1177] Step 3:

[1178] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[1179] Step 4:

[1180] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[1181] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably.

[1182] Example 1

[1183] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1184] Conventional navigation systems face challenges in terms of their ability to acquire and update charging station and traffic information in real time and dynamically present optimal routes. Furthermore, they lack the functionality to allow users to set a destination outside the vehicle and instantly synchronize it with the vehicle's navigation system, significantly reducing user convenience. Furthermore, the limited use of voice recognition systems presents a problem in that they are unable to efficiently analyze diverse voice commands.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1186] In this invention, the server includes a means for periodically collecting charging station information, traffic information, and congestion information and updating the internal database, a means for calculating the optimal route, and a means for analyzing user instructions using voice recognition and converting the obtained instructions into text data. This makes it possible to dynamically calculate and present the optimal route based on charging station information and traffic information updated in real time. Furthermore, the destination set by the user using a smart device outside the vehicle can be synchronized with the vehicle's navigation system, greatly improving convenience. In addition, advanced voice recognition technology allows for efficient analysis and execution of a variety of voice instructions.

[1187] "Charging station information" refers to data such as the location, availability, and type of charger of the equipment that supplies power to electric vehicles.

[1188] "Speech recognition" refers to the technology that analyzes the voice spoken by the user and converts it into corresponding text data.

[1189] "User instructions" refer to requests or commands made by a user to a system that direct a particular operation of the system.

[1190] "Means for calculating the optimal route" refers to a device or software that uses an algorithm to calculate the optimal route to the destination based on information obtained by the server and terminal.

[1191] "Navigation screen" means a display device installed inside a vehicle that visually presents route information and other related data.

[1192] "Server" refers to a computer system that collects, processes, and manages data via a network.

[1193] "Periodic collection means" refers to a method or device that accesses external APIs or other data sources at regular intervals to obtain the required data.

[1194] "Internal database" refers to a data management system that stores collected data and enables quick retrieval of required information.

[1195] "Smart device" refers to a mobile device (such as a smartphone or tablet) with internet connectivity that allows access to and operation of the system.

[1196] "Means of synchronization" refers to the technology or method that keeps data consistent across multiple devices.

[1197] "Means for displaying in real time" refers to a device or software that has the function of instantly displaying acquired information to a user.

[1198] This invention relates to a system that collects charging station information, traffic information, and congestion information in real time and dynamically presents optimal routes. This system consists of three main parts: a server, a terminal, and a user.

[1199] Server Features

[1200] The server is responsible for periodically collecting charging station information, traffic information, and congestion information. Charging station information includes location information, availability, charger type, etc., and is obtained from an external charging spot API via an HTTP GET request. The obtained data is received in JSON format, parsed, and saved / updated in the internal database.

[1201] Similarly, traffic and congestion information is periodically obtained from an external traffic information API. The server analyzes this information and keeps it up to date.

[1202] Examples:

[1203] The server accesses the charging spot API every hour to obtain the latest charging station information. After obtaining the information, the server analyzes the data and updates the nationwide charging station database. Traffic information is also updated in the same way.

[1204] Device Features

[1205] The terminal is the main device that receives real-time charging station and traffic information from the server. The terminal is equipped with a voice recognition system that analyzes the user's voice commands and converts them into text data. The terminal then sends the appropriate request to the server, retrieves the necessary information, and displays it on the navigation screen.

[1206] Based on the acquired information, the device constantly monitors the current location and destination information and calculates the optimal route. If traffic congestion occurs, the device calculates a new detour route and presents it to the user.

[1207] Examples:

[1208] When a user says "Find nearby charging spots," the device's voice recognition engine (for example, Google Speech-to-Text API) converts this into text. The device then retrieves information about the nearest charging spots from the server, displays it on the screen, and provides directions. If traffic jams occur during a drive, the device will suggest a new route based on the new traffic conditions.

[1209] User operations

[1210] Users can issue instructions to the system using voice commands from inside the car. For example, the system responds to voice commands such as "Find nearby charging spots" or "Set my next destination." Users can also access the system from outside the car using a smart device to search for charging stations or set destinations. The information set from the smart device is synchronized with the vehicle's navigation system, making it available the moment the user gets into the car.

[1211] Examples:

[1212] While taking a break at a cafe, a user can use their smart device to set their next destination and check nearby charging stations in advance. This information is automatically synchronized with the vehicle's navigation system, so that directions can begin immediately when the user gets in the car.

[1213] Example prompt sentence:

[1214] "My system involves a server collecting charging spot information and traffic congestion information, and an in-car terminal using that information to navigate the vehicle. The user can issue instructions to the system via voice commands or a smartphone. Please explain the process of this system in detail, dividing it into specific steps. Please also include the roles of the server, terminal, and user, as well as the specific flow of processing. Please also include a specific example of how it works."

[1215] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1216] Step 1:

[1217] The server sends an HTTP GET request to the charging station API to obtain the latest charging station information.

[1218] Input: Charging Station API Endpoint URL

[1219] Output: Charging station information data in JSON format

[1220] Specifically, the server calls the charging station API at specific time intervals (e.g., every hour) to obtain data such as location, availability, and type of charger.

[1221] Step 2:

[1222] The server analyzes the acquired JSON-formatted data, extracts information such as the location of the charging station, availability, and type of charger, and stores it in an internal database.

[1223] Input: Charging station information data in JSON format

[1224] Output: Charging station information stored in a database

[1225] Specifically, the server performs parsing (analysis) processing, extracts necessary information, and stores it in a "charging station information" table.

[1226] Step 3:

[1227] The server sends an HTTP GET request to the traffic information API to obtain the latest traffic conditions and congestion information.

[1228] Input: Traffic Information API endpoint URL

[1229] Output: Traffic and congestion information data in JSON format

[1230] Specifically, the server accesses the traffic information API at regular intervals to obtain the latest traffic conditions and congestion information.

[1231] Step 4:

[1232] The server analyzes the acquired traffic information and stores it in an internal database.

[1233] Input: Traffic and congestion information data in JSON format

[1234] Output: Traffic situation and congestion information stored in a database

[1235] Specifically, the server parses the acquired JSON data, extracts the necessary traffic information, and stores it in a database.

[1236] Step 5:

[1237] The terminal sends an HTTP GET request to the server to obtain charging station information and traffic information in real time.

[1238] Input: Server endpoint URL

[1239] Output: Real-time charging station information and traffic information

[1240] Specifically, the terminal obtains the necessary information from the server in real time in response to a user request.

[1241] Step 6:

[1242] The terminal uses a voice recognition system to analyze the user's voice commands and convert them into text data.

[1243] Input: User's voice command

[1244] Output: Text data

[1245] Specifically, a speech recognition engine (for example, Google Speech-to-Text API) is used to convert the user's speech into text data.

[1246] Step 7:

[1247] The terminal sends an appropriate request to the server based on the acquired text data and acquires the required information.

[1248] Input: Text data (the result of converting the user's voice command)

[1249] Output: Response data from the server (charging station information, traffic information, etc.)

[1250] Specifically, the terminal sends a request to the server based on the analyzed text data and obtains related information from the server.

[1251] Step 8:

[1252] Based on the acquired information, the device displays charging station information and the optimal route on the navigation screen.

[1253] Input: Charging station information and traffic information obtained from the server

[1254] Output: Charging station information and optimal route displayed on the navigation screen

[1255] Specifically, the device uses Google Maps API and other tools to display charging station information and traffic information on a map, and presents the optimal route on the navigation screen.

[1256] Step 9:

[1257] The device calculates the optimal route based on the current location and destination information, and if traffic congestion occurs on the route, it calculates a new detour route.

[1258] Input: current location, destination, traffic information

[1259] Output: Optimal route and detour route information

[1260] Specifically, the device dynamically calculates the optimal route based on traffic information updated in real time, and presents new detour routes if necessary.

[1261] Step 10:

[1262] Users can access the system from outside the vehicle using a smart device to search for charging stations and set destinations, and then synchronize that information with the in-car navigation system.

[1263] Input: Destination setting information from smart device, charging station search results

[1264] Output: Destination and charging station information synchronized with the in-car navigation system

[1265] Specifically, information set by the user using a smartphone app is synchronized in real time with the vehicle's navigation system, making the information instantly available in the vehicle.

[1266] (Application example 1)

[1267] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1268] Conventional navigation systems for autonomous vehicles lack the ability to collect information on charging spots and traffic congestion, making it difficult to calculate optimal routes in real time. Furthermore, they lack connectivity with smartphones and other devices, making it difficult for users to operate them easily. Furthermore, there was also the issue of incomplete operational support for autonomous vehicles in response to user voice commands.

[1269] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1270] In this invention, the server includes means for acquiring charging spot information, means for analyzing user requests through voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, and means for synchronizing and manipulating data in real time via a smartphone or terminal. This makes it possible to acquire charging spot information and traffic congestion information in real time, dynamically calculate an optimal route based on the user's voice commands, and efficiently support the operation of autonomous vehicles.

[1271] "Charging spot information" is information about locations where electric vehicles can be charged, and includes location information, availability, type of charger, and the like.

[1272] "Speech recognition" is a technology that analyzes voice input and converts it into text data, in order to recognize a user's voice commands.

[1273] A "server" is a computer system that manages and processes data on a network, and is responsible for acquiring, storing, and providing information on charging spots and traffic congestion.

[1274] "Means for calculating the optimal route" refers to methods or technologies that derive the optimal route based on the user's current location and destination, and refer to charging spot information and traffic congestion information.

[1275] The "navigation screen" is a screen that shows route information, charging spot information, and the like, displayed on a display inside the vehicle.

[1276] "Means for synchronizing and manipulating data in real time through smartphones and terminals" refers to technology that allows charging spot information and route information to be updated and manipulated in real time using smartphones and other devices.

[1277] "Traffic congestion information" is data relating to traffic conditions on roads, including vehicle speeds, congestion, accidents, and the like.

[1278] An "autonomous vehicle" is a vehicle that can drive autonomously without driver intervention.

[1279] "Means for assisting the operation of autonomous vehicles in response to voice commands" refers to technology that enables autonomous vehicles to perform appropriate operations and route changes based on the user's voice input.

[1280] "Means for displaying the current availability of charging spots in real time" refers to methods or technologies that constantly update the availability of charging spots and display it to users immediately.

[1281] "Location information of the nearest charging spot" is data indicating the location of the charging spot that is the shortest distance from the user's current location.

[1282] To implement the present invention, the roles played by the server, the terminal, and the user will be described.

[1283] Server Roles

[1284] The server collects and processes charging spot and traffic congestion information, and provides this information to the terminal. A general server computer is used as the hardware, and Flask (framework) and Python (programming language) are used as the software. The server periodically accesses the charging spot API and traffic information API, and stores the collected data in a database.

[1285] Specific server operations

[1286] The server periodically makes API requests to collect data such as the availability of charging spots, their locations, and the type of charger. This data is stored in a database on the server and updated in real time. Traffic and congestion information is also collected and added to the database in real time.

[1287] Device Role

[1288] The in-vehicle device uses information obtained from the server to perform navigation and voice recognition operations. Google's Speech-to-Text API is used for voice recognition, and the entire system is built using Python and Flask. It also has the ability to synchronize data in real time with smartphones and other devices.

[1289] Specific operation of the device

[1290] The device receives the user's voice command and converts it into text using Google's Speech-to-Text API. For example, if the user says, "Find nearby charging spots," the speech is converted into text and sent to the server as an appropriate request. The server searches for charging spots based on the latest information and returns the results to the device. The device then displays the search results on the navigation screen and provides the user with directions to the nearest charging spot.

[1291] User Roles

[1292] Users can operate the system via voice commands or a smartphone app. Information and settings acquired through user operations are synchronized in real time with the device inside the vehicle, allowing the system to be easily operated even from outside the vehicle.

[1293] Specific user operation examples

[1294] While taking a break at a cafe, the user can use their smartphone to set their next destination and check the nearest charging spot. This information is automatically synchronized with the vehicle's navigation system, and guidance begins as soon as the user gets in the car. When the user uses the smartphone app to enter the prompt "Find nearby charging spots," the system will provide information on the optimal charging spots.

[1295] Example prompt sentence:

[1296] Find a nearby charging spot

[1297] These functions allow users to receive information on charging spots and optimal route guidance in real time, and the system can be conveniently operated by linking it with a smartphone or other device.

[1298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1299] Step 1:

[1300] The server periodically sends requests to the charging spot API and traffic information API. The server parses the data obtained from the API and stores the available charging spots, their locations, charger types, and traffic congestion information in a database. The input is the API response, and the output is the parsed charging spot information and traffic congestion information updates.

[1301] Step 2:

[1302] The user speaks a voice command. The device in the vehicle converts the speech to text using Google's Speech-to-Text API. The voice command is the input, and the generated text is the output. For example, the voice command might be "Find nearby charging spots."

[1303] Step 3:

[1304] The device parses the voice command converted to text and requests the necessary information from the server. The device sends a request to the appropriate API endpoint based on the text generated by Google's Speech-to-Text API. The input is the voice command converted to text, and the output is a request to the server.

[1305] Step 4:

[1306] When the server receives a request from the device, it retrieves the appropriate charging spot information and traffic congestion information from the database and returns it to the device in JSON format. The input is the request from the device, and the output is the data obtained and returned by the server.

[1307] Step 5:

[1308] The device displays the information received from the server and shows the location information and route to the nearest charging spot on the navigation screen. The input is information from the server, and the output is what is displayed on the navigation screen. For example, the device may display that the nearest charging spot is 500 meters from the vehicle and provide route information.

[1309] Step 6:

[1310] The user refers to the navigation screen and drives the vehicle to the displayed charging spot or uses the automatic driving function. The input is the information on the navigation screen, and the output is the user's movement or the automatic driving of the vehicle.

[1311] Step 7:

[1312] When the vehicle arrives at a charging spot, charging begins. The server then periodically updates the data and provides new information in real time. The input is the current status of the charging spot, and the output is the start of use of the charging spot.

[1313] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1314] Overall system configuration

[1315] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[1316] What the program does

[1317] Server-side processing

[1318] The server periodically sends requests to the charging spot API to collect the latest charging spot information. This information includes data such as location, availability, and type of charger. The server parses this data and immediately updates it in its internal database. In addition, the server periodically obtains data from the traffic information API to collect traffic conditions and congestion information, and stores this data in the database as well.

[1319] Examples:

[1320] The server calls the main charging spot API every hour to get the latest charging spot information, parses that data, and updates it to the nationwide charging spot database. Traffic congestion information is also updated periodically.

[1321] Terminal side processing

[1322] The device receives real-time information on charging spots and traffic congestion from the server. It uses a voice recognition system to analyze the user's voice commands, obtains the necessary information based on those commands, and displays it on the navigation screen. It also uses an emotion engine to analyze the user's emotions from their voice and facial expressions, and adjusts the system's response based on those results.

[1323] Examples:

[1324] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[1325] The device constantly monitors the user's current location and destination, and calculates the optimal route based on the latest traffic congestion information. If a traffic jam occurs, the device calculates a new detour route and presents it to the user. At the same time, based on information from the emotion engine, if the user is feeling stressed, the device adjusts the guidance to make route selection as simple as possible.

[1326] User operation

[1327] Users can issue voice commands to the system from inside the car, or access the system from outside the car using a smartphone app to search for charging spots and set destinations. The information set by the user on their smartphone is synchronized with the vehicle's navigation system and is immediately available when they get in the car.

[1328] Examples:

[1329] While taking a break at a cafe, the user sets their next destination on their smartphone and checks available charging spots in advance. Based on this, the car's navigation system automatically sets a route and starts guiding the user once they get in the car. If the emotion engine determines that the user's stress level is high during the journey, it will suggest a smoother route or relaxing music.

[1330] These operations allow users to receive real-time information on electric vehicle charging spots and optimal route guidance, and the emotion engine also provides a comfortable driving experience.

[1331] The processing flow will be explained below.

[1332] Processing flow of a system that combines emotion engines

[1333] Server-side processing

[1334] Step 1:

[1335] The server periodically sends requests to the charging spot API, which returns data such as the latest location, availability, and type of charger for the charging spot.

[1336] Step 2:

[1337] The server parses the received data and extracts information about each charging spot, including its location, the number of available chargers, and its current status.

[1338] Step 3:

[1339] The server updates the extracted information into an internal database that is accessible in real time and is available for use by terminals and other systems.

[1340] Step 4:

[1341] The server sends a request to the traffic information API to get the latest traffic congestion and accident information, which is also updated in the database.

[1342] Terminal side processing

[1343] Step 1:

[1344] The device sends data requests to the server at regular intervals, obtaining the latest information on charging spots and traffic congestion.

[1345] Step 2:

[1346] The terminal stores the information received from the server in an internal cache, allowing it to respond quickly to user requests later.

[1347] Step 3:

[1348] The user issues a voice command, for example, "Find nearby charging spots."

[1349] Step 4:

[1350] The device's speech recognition engine converts this voice command into text, which is then analyzed internally to determine the appropriate action to take.

[1351] Step 5:

[1352] The device identifies the nearest charging spot based on the charging spot information retrieved from the cache or server, and the identified information is displayed on the navigation screen.

[1353] Step 6:

[1354] The device monitors current location and destination information in real time and calculates the optimal route based on the latest traffic congestion information.

[1355] Step 7:

[1356] If a traffic jam or accident occurs, the device will calculate a new detour route and display it on the navigation screen. The user can then decide whether to select the proposed detour route or stay on the current route.

[1357] Step 8:

[1358] The device analyzes the user's voice and facial expressions using an emotion engine, which determines the user's stress level and emotional state.

[1359] Step 9:

[1360] After the emotion engine recognizes the user's emotions, the device can adjust navigation and system responses based on that information. For example, if the user is feeling stressed, it can provide a calming voice prompt to encourage the user to relax.

[1361] Examples:

[1362] When a user says, "Find a nearby charging spot," the device's voice recognition engine converts this into text and retrieves appropriate charging spots from the server. Information about the nearest charging spot is displayed on the screen and directions are provided. If the emotion engine detects irritation from the user's tone of voice or facial expression, the device will calmly advise, "Relax and drive safely."

[1363] User operation

[1364] Step 1:

[1365] Users can search for charging spots and receive route guidance by issuing voice commands from inside the vehicle.

[1366] Step 2:

[1367] Users can launch the smartphone app and check charging spot information and set destinations from outside the vehicle.

[1368] Step 3:

[1369] When a user sets a destination or charging spot information on their smartphone, the smartphone sends that information to the vehicle's navigation system.

[1370] Step 4:

[1371] After getting into the car, the user will see the navigation system automatically begin providing route guidance using the information sent from the smartphone.

[1372] Step 5:

[1373] If the emotion engine determines that the user's stress level is high along the way, the device will adjust the guidance to make route selection as simple as possible.

[1374] These processing steps enable users to quickly and accurately obtain information from inside and outside the vehicle, enabling them to travel comfortably. In addition, the introduction of an emotion engine provides optimal support according to the user's emotional state.

[1375] Example 2

[1376] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1377] Conventional navigation systems can provide real-time information on charging spots and traffic congestion, but they are unable to respond appropriately taking into account the user's emotional state. This can lead to stress for drivers and can lead to a less comfortable driving environment.

[1378] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1379] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server and the user's emotions, means for displaying the route information on a navigation screen, and means for recognizing emotions from the user's voice and facial expressions and adjusting the system's response, thereby making it possible to provide optimal route guidance and appropriate responses according to the user's emotional state.

[1380] "Charging spot information" is data such as location information, availability, and type of charger regarding places where electric vehicles can be charged.

[1381] "Speech recognition" is a technology that converts a user's voice commands into text and analyzes their intent.

[1382] The "optimal route" is the most efficient and effective route to your destination, calculated based on charging spot information and traffic congestion information.

[1383] A "server" is a computer system that collects information on charging spots, traffic congestion, and other real-time data and provides it to terminals.

[1384] The "navigation screen" is a screen on which the terminal visually presents route guidance and charging spot information to the user.

[1385] The "emotion engine" is a technology that analyzes the user's voice and facial expressions to recognize their emotional state.

[1386] "Traffic congestion information" is data relating to road traffic conditions and the degree of congestion.

[1387] "Means for displaying in real time" refers to techniques or methods that instantly display the current situation to the user.

[1388] This system consists of a server, a terminal, a user, and an emotion engine. The server collects information on charging spots, traffic congestion, and other real-time data and provides it to the terminal. The terminal operates inside the vehicle and navigates based on the information from the server. The user issues instructions to the system through voice commands or input from a smartphone. Furthermore, the emotion engine recognizes emotions from the user's voice and facial expressions and adjusts the system's response.

[1389] Server-side processing

[1390] The server sends requests to the charging spot API and traffic information API at regular intervals to obtain the latest charging spot information and traffic congestion information. The obtained data includes information such as location, availability, and charger type, and is parsed and updated in an internal database. Specifically, it obtains data from major charging spot APIs (for example, the EV charging network API) and traffic congestion information from Google Maps' traffic data API.

[1391] Terminal side processing

[1392] The device obtains charging spot and traffic congestion information in real time from the server. The device is equipped with a voice recognition system that analyzes the user's voice commands, retrieves appropriate information, and displays it on the navigation screen. The emotion engine analyzes the user's voice and facial expressions, recognizes their emotions, and adjusts the system's response accordingly. For example, if a user says, "Find nearby charging spots," the device's voice recognition engine analyzes this and retrieves and displays the most appropriate charging spot information from the server. If the emotion engine analyzes the user's tone of voice and facial expressions and determines that they are stressed, the device will respond in a calm voice, saying, "Relax and drive safely."

[1393] User operation

[1394] Users can issue instructions to the system using voice commands. They can also access the system from outside the car using a smartphone app to search for charging spots and set destinations. Information set on the smartphone is synchronized with the car's navigation system and is immediately available when they get in. For example, while taking a break at a cafe, a user can use their smartphone to set their next destination and check available charging spots in advance. Based on this, the car's navigation system automatically plans a route and begins guiding the user as soon as they get in the car. If the emotion engine determines that the user's stress level is high, it will suggest smooth routes and relaxing music.

[1395] Examples of prompt statements

[1396] Example prompt for explaining server-side processing: "Please explain the server-side process that collects the latest charging spot information and traffic congestion information."

[1397] Example prompt for device-side processing: "Describe the device-side process where a user uses voice commands to find a charging spot. Also, explain how you use an emotion engine to tailor the response."

[1398] Example prompt for user interaction: "Describe how a user would set their next destination and search for charging spots on their smartphone. Include an example of how the emotion engine would respond if their stress level was high."

[1399] This system allows users to receive real-time information on electric vehicle charging spots and optimal route guidance, and also provides a comfortable driving experience through an emotion engine.

[1400] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1401] Step 1: Data Collection (Server)

[1402] The server sends periodic requests to the charging spot API and traffic information API. The input for this step is the API endpoint URL and necessary authentication information. The server uses these inputs to generate and send a data request. As output, it receives charging spot information and traffic information in JSON format. Specifically, the server sends a request every hour to the main charging spot API (e.g., EV charging network API) to obtain the latest data. It also sends a request to Google Maps traffic data API to obtain traffic congestion information.

[1403] Step 2: Data Parsing (Server)

[1404] The server parses the JSON format data obtained in step 1. The input for this step is the obtained JSON data. The server parses this data and extracts the necessary information (e.g., location information, availability, and type of charger). As an output, the extracted data is imported into an internal database. Specifically, the server parses the responses obtained from the charging spot API and traffic information API, and classifies and extracts the respective information.

[1405] Step 3: Database Update (Server)

[1406] The server immediately updates the information extracted in step 2 to its internal database. The input to this step is the extracted charging spot information and traffic information. The server updates the corresponding fields in the database based on this input. The output is an updated database that reflects the new information. Specifically, the server adds and updates the extracted charging spot information to the nationwide charging spot database. Similarly, traffic congestion information is also updated in the database.

[1407] Step 4: Data Acquisition (Device)

[1408] The terminal obtains charging spot information and traffic congestion information in real time from the server. The input to this step is request information to the server. Based on this, the terminal requests data from the server and receives the latest charging spot information and traffic information as output. Specifically, the terminal sends requests to the server at regular intervals to obtain the latest information.

[1409] Step 5: Voice command analysis (device)

[1410] The device analyzes the user's voice command with a voice recognition system. The input for this step is the user's voice command. The device converts this voice into text and analyzes it for appropriate processing. The output is the analyzed text data. Specifically, the device's voice recognition engine converts the voice "Find nearby charging spots" into text and retrieves information based on the request.

[1411] Step 6: Sentiment Analysis (Device)

[1412] The device uses an emotion engine to analyze emotions from the user's voice and facial expressions. The input for this step is the user's voice and facial expression data. The device analyzes these and determines the user's emotional state. The output is a system response based on the emotional state. Specifically, if the device's emotion engine analyzes the user's tone of voice and facial expression and recognizes that the user is irritated, the device will respond in a calm voice, saying, "Relax and drive safely."

[1413] Step 7: Navigation Display (Device)

[1414] The device updates the navigation screen based on the data obtained from the server. The input for this step is the latest charging spot information and traffic information. The device calculates the optimal route based on this input and displays it on the navigation screen. The output is the latest route guidance. Specifically, the device displays information on the charging spot closest to the user and navigates the route. It also recalculates the optimal route based on traffic congestion information and suggests detour routes.

[1415] Step 8: Voice command input (user)

[1416] The user issues a voice command from inside the car to give instructions to the system. The input for this step is the user's voice command. Based on this, the user sends a request to the system. The output is the analyzed voice command, which is processed by the system. In concrete terms, if the user says, "Find a nearby charging spot," the device will recognize the voice and provide appropriate information.

[1417] Step 9: Smartphone operation (user)

[1418] The user accesses the system from outside the vehicle using a smartphone app. The input for this step is the operation information of the smartphone app. The user uses this information to search for charging spots and set their destination. The output is the setting information synchronized with the in-vehicle navigation system. In concrete terms, the user sets their next destination using their smartphone while taking a break at a cafe and checks available charging spots in advance.

[1419] Step 10: System Sync (User)

[1420] The information set by the user is synchronized with the navigation system in the vehicle. The input for this step is the setting information from a smartphone. The user uses this input to make the system immediately available when getting in the car. The setting information is then reflected in the navigation system as an output. In concrete terms, the moment the user gets into the car, the navigation system begins guiding the user along the pre-set route. If the emotion engine determines that the stress level is high, it will suggest smooth routes and relaxing music.

[1421] (Application example 2)

[1422] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1423] Current self-driving vehicles and navigation systems can provide information on charging spots and traffic congestion, but they cannot respond to the user's emotions or make suggestions to reduce stress. As a result, they cannot reduce the stress and anxiety caused by long drives, making it difficult to improve the overall driving experience.

[1424] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1425] In this invention, the server includes means for acquiring charging spot information, means for analyzing a user's request by voice recognition, means for calculating an optimal route based on the information acquired from the server, means for displaying the route information on a navigation screen, an emotion engine for analyzing the user's emotions, and means for adjusting the system's response based on the emotions analyzed by the emotion engine. This enables the user to not only obtain information about charging spots and the optimal route, but also receive suggestions for reducing emotional stress.

[1426] "Charging spot information" is data about locations where electric vehicles can be charged, including location information, availability, and type of charger.

[1427] "Speech recognition" is a technology that analyzes a user's voice and converts it into text data.

[1428] An "optimal route" is the most efficient and time-saving route for a user to reach their destination.

[1429] A "navigation screen" is a display device that visually provides the user with information about the current location and route to the destination.

[1430] The "emotion engine" is a device that analyzes the user's tone of voice and facial expressions to determine their emotional state.

[1431] A "server" is an information processing device that collects data via a network, processes and stores it, and provides it to clients.

[1432] "Adjusting the system's response based on emotions" refers to controlling the system to respond appropriately based on the user's emotional state.

[1433] The embodiment of the present invention is composed of a server, a terminal, a user, and an emotion engine. This system enables users of autonomous vehicles to enjoy a comfortable and stress-free driving experience.

[1434] Server-side processing

[1435] The server periodically sends requests to the API to collect charging spot information and obtains the latest data. This information includes location, availability, and type of charger. This information is stored in an internal database and provided in response to requests from users' devices. Traffic and congestion information is also collected from another API and stored in the database.

[1436] Terminal side processing

[1437] The device receives real-time information on charging spots and traffic congestion from the server and provides it to the user. It uses a voice recognition system to analyze the user's voice commands, obtain the necessary information, and display it on the navigation screen. It also uses an emotion engine to analyze the user's emotions and adjust the system's response.

[1438] User operations

[1439] Users can issue voice commands to the system from inside the car, and can also access the system from outside the car using a smartphone application to search for charging spots and set destinations. The information set on the smartphone is automatically synchronized with the vehicle's navigation system.

[1440] Hardware and software used

[1441] The hardware used includes a smartphone, microphone, camera, internet connection, etc. The software used includes Python, the Requests library, the Geopy library, the SpeechRecognition library, and the EmotionEngine. This hardware and software enables voice recognition, emotion analysis, and the display of navigation data.

[1442] Specific examples

[1443] In a specific scenario, when a user issues a voice command such as "Find a nearby charging spot," the device's voice recognition engine converts this into text, retrieves appropriate charging spot information from the server, and displays it on the screen. At the same time, if the emotion engine determines that the user is feeling stressed based on their tone of voice or facial expression, the device will notify them by saying, "Relax and drive safely."

[1444] Example prompts for generative AI models

[1445] An example of a prompt for a generative AI model is:

[1446] "Explanation of the goal

[1447] I want to develop an application that allows users of autonomous vehicles to receive charging spot information and navigation.

[1448] We would also like to include a function that recognizes the user's emotions and provides advice to reduce stress.

[1449] Input data

[1450] 1. Charging spot API data (location, availability, charger type)

[1451] 2. Traffic congestion information API data

[1452] 3. User voice commands

[1453] 4. User emotion analysis data (voice tone, facial expressions)

[1454] Output Data

[1455] 1. Information on the nearest charging spot

[1456] 2. Real-time navigation directions

[1457] 3. Response messages based on user sentiment

[1458] Suggested Application Features

[1459] 1. Charging spot search and reservation function

[1460] 2. Real-time navigation function

[1461] 3. Voice command reception function

[1462] 4. Response adjustment function based on emotion analysis

[1463] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1464] Step 1:

[1465] The server sends a request to the charging spot API to get the latest charging spot information. The input is the charging spot API URL, and the output is JSON data including location information, availability, and charger type. This data is parsed and stored in an internal database.

[1466] Step 2:

[1467] The server sends a request to the traffic information API to get the latest traffic congestion information. The input is the traffic information API URL, and the output is JSON data about traffic congestion. This data is also parsed and stored in the internal database.

[1468] Step 3:

[1469] The user issues a voice command. The device inputs the voice through the microphone and converts it into text using speech recognition technology (SpeechRecognition library). For example, if the input is "Find nearby charging spots," the output is this text data.

[1470] Step 4:

[1471] The device that receives the text-translated voice command requests charging spot information from the server based on the command. The input is the text data "Search for nearby charging spots," and the output is information about the nearest charging spot (location information, availability, etc.) sent from the server.

[1472] Step 5:

[1473] The device combines the charging spot information received from the server with the current location information to calculate the optimal route. The input is charging spot information and current location information, and the output is route data for the optimal route. This calculation uses the Geopy library.

[1474] Step 6:

[1475] After the optimal route is calculated, the device displays this route information on the navigation screen. The input is the route data of the optimal route, and the output is a visual navigation display. The user can check the route to the charging spot by looking at the screen.

[1476] Step 7:

[1477] At the same time, the device uses an emotion engine to analyze the user's emotions. The input is the user's voice tone and facial expression data, and the output is data indicating the user's emotional state (e.g., stressed, angry, relaxed). This analysis is performed using the EmotionEngine.

[1478] Step 8:

[1479] Based on the analyzed emotional data, the device adjusts the response message. The input is emotional data, and the output is an appropriate response message (e.g., "Relax and drive safely."). This gives the user a sense of security.

[1480] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1481] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1482] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1483] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1484] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1485] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1486] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1487] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1488] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1489] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1490] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1491] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1492] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1493] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1494] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1495] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1496] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1497] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1498] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1499] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1500] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1501] The following is further disclosed regarding the above embodiment.

[1502] (Claim 1)

[1503] A means for acquiring charging spot information;

[1504] means for analyzing a user's request by speech recognition;

[1505] A means for calculating an optimal route based on information obtained from the server;

[1506] means for displaying the route information on a navigation screen;

[1507] A system including:

[1508] (Claim 2)

[1509] 2. The system according to claim 1, wherein the means for calculating the optimum route further comprises means for dynamically changing the route based on congestion information obtained from the server.

[1510] (Claim 3)

[1511] 2. The system according to claim 1, wherein the means for acquiring charging spot information comprises means for displaying the current availability of charging spots in real time.

[1512] (Claim 4)

[1513] 10. The system according to claim 1, further comprising means for a user to input information using a smartphone from outside the vehicle and transmit the information to the vehicle's navigation system.

[1514] (Claim 5)

[1515] 2. The system according to claim 1, further comprising means for caching information acquired from the server in the terminal and processing content requested by voice recognition based on that information.

[1516] "Example 1"

[1517] (Claim 1)

[1518] A means for acquiring charging station information;

[1519] means for analyzing user instructions by voice recognition;

[1520] A means for calculating an optimal route based on information obtained from the server;

[1521] means for displaying the route information on a navigation screen;

[1522] Regularly collect charging station information, traffic information, and congestion information,

[1523] a means for updating the internal database;

[1524] A system including:

[1525] (Claim 2)

[1526] 2. The system according to claim 1, wherein the means for calculating the optimum route further comprises means for dynamically changing the route based on congestion information obtained from the server.

[1527] (Claim 3)

[1528] 2. The system according to claim 1, wherein the means for acquiring charging station information comprises means for displaying current availability of charging stations in real time.

[1529] (Claim 4)

[1530] The system according to claim 1, characterized in that it comprises means for a user to set a next destination using a smart device from outside the vehicle and synchronize it with the vehicle's navigation system.

[1531] (Claim 5)

[1532] 2. The system according to claim 1, further comprising means for converting instructions obtained by voice recognition into text data.

[1533] (Claim 6)

[1534] 2. The system according to claim 1, further comprising means for presenting new guidance in real time based on the acquired charging station information and traffic information.

[1535] "Application Example 1"

[1536] (Claim 1)

[1537] A means for acquiring charging spot information;

[1538] means for analyzing a user's request by speech recognition;

[1539] A means for calculating an optimal route based on information obtained from the server;

[1540] means for displaying the route information on a navigation screen;

[1541] A means to synchronize and manipulate data in real time via smartphones and devices,

[1542] A system including:

[1543] (Claim 2)

[1544] The system of claim 1, wherein the means for calculating the optimal route further comprises means for dynamically changing the route based on traffic congestion information obtained from the server, and means for assisting the operation of the autonomous vehicle in response to a user's voice command.

[1545] (Claim 3)

[1546] 2. The system according to claim 1, wherein the means for acquiring the charging spot information includes means for displaying the current availability of charging spots in real time and acquiring location information of the charging spot closest to the user.

[1547] "Example 2: Combining Emotion Engines"

[1548] (Claim 1)

[1549] A means for acquiring charging spot information;

[1550] means for analyzing a user's request by speech recognition;

[1551] A means for calculating an optimal route based on information obtained from the server and user emotions;

[1552] means for displaying the route information on a navigation screen;

[1553] A means for recognizing emotions from the user's voice and facial expressions and adjusting the system's response;

[1554] A system including:

[1555] (Claim 2)

[1556] 2. The system according to claim 1, further comprising means for dynamically changing the route based on congestion information obtained from the server.

[1557] (Claim 3)

[1558] The system according to claim 1, further comprising means for displaying the current availability of charging spots in real time.

[1559] "Application example 2 when combining emotion engines"

[1560] (Claim 1)

[1561] A means for acquiring charging spot information;

[1562] means for analyzing a user's request by speech recognition;

[1563] A means for calculating an optimal route based on information obtained from the server;

[1564] means for displaying the route information on a navigation screen;

[1565] An emotion engine that analyzes the user's emotions,

[1566] means for adjusting the response of the system based on the emotions analyzed by the emotion engine;

[1567] A system including:

[1568] (Claim 2)

[1569] 2. The system according to claim 1, wherein the means for calculating the optimum route further comprises means for dynamically changing the route based on congestion information obtained from the server.

[1570] (Claim 3)

[1571] 2. The system according to claim 1, wherein the means for acquiring charging spot information comprises means for displaying the current availability of charging spots in real time.

[1572] (Claim 4)

[1573] 2. The system according to claim 1, further comprising means for recognizing the user's emotions from voice tone and facial expressions and making suggestions for reducing stress.

[1574] (Claim 5)

[1575] 2. The system according to claim 1, characterized in that it is realized as an application installed on a smartphone, smart glasses, a head-mounted display, or a robot. [Explanation of symbols]

[1576] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring charging spot information; means for analyzing a user's request by speech recognition; A means for calculating an optimal route based on information obtained from the server; means for displaying the route information on a navigation screen; A system including:

2. 2. The system according to claim 1, wherein the means for calculating the optimum route further comprises means for dynamically changing the route based on congestion information obtained from the server.

3. 2. The system according to claim 1, wherein the means for acquiring charging spot information includes means for displaying the current availability of charging spots in real time.

4. 2. The system according to claim 1, further comprising means for allowing a user to input information using a smartphone from outside the vehicle and for transmitting the information to the vehicle's navigation system.

5. 2. The system according to claim 1, further comprising means for caching information acquired from the server in the terminal and processing content requested by voice recognition based on said information.

Images