System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring position information of a user; means for calculating an optimum route based on the position information; means for generating map data of a horizontal viewpoint based on the route; means for displaying the map data; means for recording geographic information in a photographing mode; means for receiving the recorded geographic information and checking quality thereof; means for storing the checked geographic information; and means for calculating and providing a reward for providing the geographic information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional map applications display maps from a top-down perspective, making them difficult to understand for users who are not adept at reading maps, causing confusion and stress when navigating. Furthermore, despite the growing demand for visually easy-to-understand map display methods as map applications evolve, there has been a lack of safety-conscious design. Furthermore, collecting geographic information using dedicated vehicles is expensive and updates are limited, resulting in a lack of map data that can be updated in real time. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system including: means for acquiring a user's current location information; means for calculating an optimal route based on the location information; means for generating horizontal perspective map data based on the route; means for displaying the map data; means for a user to record geographic information in a photographic mode; means for receiving the recorded geographic information and confirming its quality; means for saving the confirmed geographic information; and means for calculating and awarding a reward for providing the geographic information. Providing horizontal perspective map data makes it easier for users to intuitively understand maps and improves navigation safety. Furthermore, by having users collect geographic information in a photographic mode and then saving it after confirming its quality, map data can be updated efficiently. As a result, costly map update work using dedicated vehicles is no longer necessary, and the latest map data can be provided in real time.

[0006] "User" refers to an individual or organization that uses the system to obtain location information, display maps, navigate, record geographic information, etc.

[0007] "Location information" refers to geographic coordinate data such as latitude and longitude that indicates a specific point on the Earth.

[0008] A "route" refers to a possible route between a user's current location and a destination.

[0009] "Horizontal perspective" refers to a visual presentation in which the user's viewpoint is parallel to the horizon.

[0010] "Map Data" means visually displayed information, including geographic features and landmarks.

[0011] "Display means" refers to a device, such as a terminal screen or display, that visually presents map data and guidance information to the user.

[0012] "Photography mode" refers to an application function that allows users to record geographic information using the camera or GPS function.

[0013] "Geographic information" refers to image data and location information relating to a specific location.

[0014] "Quality" refers to criteria such as resolution, clarity, and accuracy of the geographic information received by the system.

[0015] "Storage means" refers to the system's function of accumulating recorded geographic information and storing it in a database, etc.

[0016] "Rewards" refers to points or other forms of incentives awarded by the system for User Contributions.

[0017] "Map application" refers to software for obtaining location information, calculating routes, displaying map data, navigation, etc. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] MODE FOR CARRYING OUT THE INVENTION

[0040] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[0041] 1. Obtaining user location information

[0042] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[0043] 2. Horizontal Perspective Map Generation

[0044] Based on the received location information, the server references map data in its database and calculates the optimal route from the current location to the destination. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[0045] 3. Displaying and operating horizontal perspective maps

[0046] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[0047] 4. Starting the shooting mode and saving data

[0048] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[0049] 5. Providing Rewards

[0050] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[0051] Specific examples

[0052] Below are some specific examples of how this system can be used.

[0053] 1. User A starts the app and confirms that their current location is Chiyoda-ku, Tokyo. The server calculates the shortest route to Shibuya-ku, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[0054] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0055] These features enable the present invention to provide an intuitive and safe navigation experience for users, while maintaining real-time, up-to-date map data.

[0056] The processing flow will be explained below.

[0057] Specific flow of program processing

[0058] Obtaining user location information

[0059] Step 1:

[0060] The user launches the app.

[0061] Specific behavior:

[0062] A user taps an app icon on their smartphone, launching the application.

[0063] Step 2:

[0064] The device activates the GPS function and obtains the current location information.

[0065] Specific behavior:

[0066] The device activates the GPS module and measures latitude and longitude data.

[0067] Step 3:

[0068] The terminal transmits the acquired location information to the server.

[0069] Specific behavior:

[0070] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[0071] Generating horizontal perspective maps

[0072] Step 1:

[0073] The server refers to a database based on the received location information and calculates the optimal route.

[0074] Specific behavior:

[0075] The server retrieves road and destination information from the database and applies a shortest path calculation algorithm (e.g., Dijkstra's algorithm).

[0076] Step 2:

[0077] The server generates horizontal perspective map data based on the calculated route.

[0078] Specific behavior:

[0079] The server calls the Google Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[0080] Step 3:

[0081] The server transmits the generated map data to the terminal.

[0082] Specific behavior:

[0083] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[0084] Displaying and operating horizontal perspective maps

[0085] Step 1:

[0086] The terminal stores the received map data in a cache.

[0087] Specific behavior:

[0088] Map data is temporarily stored in the device's storage in preparation for display processing.

[0089] Step 2:

[0090] The terminal displays the map data on the display.

[0091] Specific behavior:

[0092] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[0093] Step 3:

[0094] Users manipulate the map using their fingers or voice commands.

[0095] Specific behavior:

[0096] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[0097] Step 4:

[0098] The terminal acquires new map data from the server in response to a user operation.

[0099] Specific behavior:

[0100] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[0101] Starting the shooting mode and saving data

[0102] Step 1:

[0103] The user initiates the capture mode.

[0104] Specific behavior:

[0105] The user taps the "Shooting Mode" button within the app.

[0106] Step 2:

[0107] The device will activate the camera and GPS and begin recording location information and video data.

[0108] Specific behavior:

[0109] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[0110] Step 3:

[0111] The terminal transmits the recorded data to the server in real time.

[0112] Specific behavior:

[0113] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[0114] Step 4:

[0115] The server analyzes the received data and checks the quality criteria.

[0116] Specific behavior:

[0117] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[0118] Step 5:

[0119] The server stores data that meets the criteria in the system.

[0120] Specific behavior:

[0121] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[0122] Providing rewards

[0123] Step 1:

[0124] The server calculates points based on the amount and quality of data provided by the user.

[0125] Specific behavior:

[0126] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[0127] Step 2:

[0128] The server adds the calculated points to the user's account.

[0129] Specific behavior:

[0130] Update the user's account information and add the calculated points.

[0131] Step 3:

[0132] The server notifies the user that the reward has been granted.

[0133] Specific behavior:

[0134] The user will be notified by push notification or email that the reward has been granted.

[0135] In this way, the present invention provides an intuitive and safe navigation experience for users and allows for real-time updates of map data.

[0136] Example 1

[0137] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0138] Conventional navigation systems often lack the latest map data due to insufficient real-time location information acquisition and geographic information recording. Furthermore, there was no appropriate reward system for providing geographic information, and incentives based on users' contributions were lacking. This resulted in low user satisfaction and frequency of use, ultimately impairing the usefulness of the system as a whole.

[0139] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0140] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in a shooting mode, means for receiving the recorded geographic information in real time and analyzing the quality, means for storing the geographic information confirmed based on the analysis, and means for evaluating and awarding a reward for providing the geographic information. This allows users to use the latest map data in real time and receive an appropriate reward for providing geographic information, thereby improving the usability of the system.

[0141] "User location information" refers to the user's current latitude and longitude data obtained using a mobile device or GPS function.

[0142] An "optimal route" is the most efficient travel route between a user's starting point and their destination, calculated taking into account factors such as time, distance, and traffic conditions.

[0143] "Horizontal perspective map data" is map display data viewed from a horizontal direction, generated based on real-world scenery or street view images.

[0144] The "means for displaying map data" is an interface that uses the display of the terminal to visually present the acquired and generated map information to the user.

[0145] "Shooting mode" is an operation mode that uses the device's camera and GPS functions to record video data and location information.

[0146] "Geographic information" is data that includes location information within a specific area and visual information (images and video) of that location.

[0147] "Receiving in real time" means receiving data sent from a terminal immediately without delay.

[0148] "Means for analyzing quality" refers to the process of evaluating the accuracy, resolution, blur, etc. of the received geographic information data and confirming its quality.

[0149] "Means for storing" refers to storing the confirmed geographic information data in a storage device such as a database or cloud storage.

[0150] The "means for evaluating and awarding rewards" is a process for calculating the value of the geographic information provided based on the quantity and quality of the information and adding reward points to the user.

[0151] The following describes in detail the mode for carrying out the present invention. The present invention is a navigation system that acquires a user's location information, calculates an optimal route based on the acquired information, and generates and displays horizontal perspective map data. This system also includes a function to record geographic information in a photographing mode, send it to a server for quality check, and provide rewards to the user.

[0152] Obtaining user location information

[0153] When a user launches the application, the device uses its built-in GPS to obtain its current location. This location information is obtained in the form of latitude and longitude and sent to the server. Specifically, the location information is structured in JSON format and sent to the server using an HTTP request.

[0154] Generating horizontal perspective maps

[0155] The server analyzes the received location information and references map data in a database. It uses Python's GeoPandas library to calculate the optimal route from the current location to the destination. Based on this route information, it generates horizontal perspective map data using the Google Street View API. The generated map data is sent to the device as a JPEG image file.

[0156] Displaying and operating horizontal perspective maps

[0157] The device temporarily stores (caches) the received map data and displays it on the display. The map data is rendered using OpenGL, and the user can zoom in and out and move the map by pinching in, pinching out, and swiping. In response to user operations, the device requests new map data from the server and dynamically updates the map display.

[0158] Starting the shooting mode and saving data

[0159] When a user starts the shooting mode, the device activates the camera and GPS, simultaneously recording the captured video data and location information. This video data is encoded in H.264 format and sent to the server in real time via WebSocket communication along with GPS information. The server uses the OpenCV library to analyze the quality of the video data and check whether it meets certain criteria, such as blur and resolution. Data that meets the criteria is stored in cloud storage such as AWS S3.

[0160] Reward calculation and awarding

[0161] The server evaluates the quality and quantity of the geographic information provided and calculates reward points. This evaluation takes into account factors such as image quality and information coverage. The calculated reward points are immediately reflected in the user's account and managed in a MySQL database. The device uses push notifications to notify the user that a reward has been awarded.

[0162] Specific examples

[0163] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[0164] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in the database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0165] Prompt Sentence Examples

[0166] When a user launches the app, this system uses GPS to obtain their current location and sends it to the server. The server calculates the optimal route based on the received location information, generates horizontal perspective map data, and sends it to the device. The device displays this and navigates according to the user's operations. The user can then use the camera mode to record video data and location information and send it to the server. The server analyzes this, stores it in a database, and provides rewards to the user.

[0167] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0168] Step 1:

[0169] The user launches the app.

[0170] Specific behavior: When a user taps the app on their smartphone to launch it, the application invokes the GPS function in the background.

[0171] Step 2:

[0172] The device uses the GPS function to obtain the current location information.

[0173] Input: GPS satellite signal

[0174] Output: Latitude and longitude location (e.g. 35.6895, 139.6917)

[0175] How it works: The device's GPS module receives signals from satellites, analyzes them, and calculates the current latitude and longitude. This location information is structured in JSON format.

[0176] Step 3:

[0177] The location information acquired by the device is sent to the server.

[0178] Input: Latitude and longitude location information (JSON format)

[0179] Output: HTTP request sent to the server

[0180] What it does: The device packages the location information in JSON format and sends it to the server using the HTTPS protocol, along with the user's authentication token.

[0181] Step 4:

[0182] The server receives the user's location information.

[0183] Input: Location information (JSON format), authentication token

[0184] Output: Location information analysis results

[0185] Specific operation: The server deserializes the received JSON data and compares it with the map data stored in the database.

[0186] Step 5:

[0187] The server references the map data in the database.

[0188] Input:Location

[0189] Output: Map data reference results

[0190] How it works: The server uses Python's GeoPandas library to get the current location and related map data.

[0191] Step 6:

[0192] The server calculates the optimal route from the current location to the destination.

[0193] Input: Location information of starting point and destination

[0194] Output: Optimal route information (JSON format)

[0195] Specific operation: Calculates the optimal route taking into account distance, time, and traffic conditions using the GeoPandas library. The calculation results are saved in JSON format.

[0196] Step 7:

[0197] The server converts and generates map data based on Street View images into a horizontal perspective.

[0198] Input: Optimal route information

[0199] Output: Horizontal perspective map data (JPEG format)

[0200] Specific operation: Uses the Google Street View API to obtain image data of the area, converts it to a horizontal perspective, and generates an image in JPEG format.

[0201] Step 8:

[0202] The server transmits the generated map data to the terminal.

[0203] Input: Horizontal perspective map data (JPEG format)

[0204] Output: Sending data to the terminal

[0205] Specific operation: The server sends the generated map data to the terminal using the HTTPS protocol.

[0206] Step 9:

[0207] The terminal caches the received map data and displays the map data on the display.

[0208] Input: Horizontal perspective map data (JPEG format)

[0209] Output: Display

[0210] Specific operation: The device stores the received map data in a local cache and uses OpenGL to draw the map on the display.

[0211] Step 10:

[0212] The user manipulates the map to navigate.

[0213] Input: Touch, voice commands

[0214] Output: Updated navigation route

[0215] How it works: Users can zoom in and out and pan the map by pinching, pinching out, and swiping. They can also specify a destination using voice commands. The device will request new data from the server as needed and update the display.

[0216] Step 11:

[0217] The user initiates the capture mode.

[0218] Specific operation: When the user presses the "Photo mode" button in the app, the device activates the camera and GPS.

[0219] Step 12:

[0220] The device activates the camera and GPS to record video data and location information.

[0221] Input: Local image, latitude and longitude location information

[0222] Output: Encoded video data (H.264 format), location information

[0223] Specific operation: Local video captured by the camera is encoded in H.264 format and simultaneously recorded along with GPS information (latitude and longitude).

[0224] Step 13:

[0225] The device transmits the recorded data to the server in real time.

[0226] Input: Encoded video data, location information

[0227] Output: Send data to the server

[0228] Specific operation: The device uses WebSocket communication to send encoded video data and GPS location information to the server in real time.

[0229] Step 14:

[0230] The server analyzes the data and checks its quality.

[0231] Input: Video data, location information

[0232] Output: Quality analysis results

[0233] How it works: The server uses the OpenCV library to analyze the video for blur and resolution, checks the quality, and if it meets the criteria, proceeds to the next processing step.

[0234] Step 15:

[0235] The server stores the data that meets the criteria in a database.

[0236] Input: Quality checked data

[0237] Output: Save to database

[0238] Specific operation: Quality-confirmed data is stored in cloud storage such as AWS S3.

[0239] Step 16:

[0240] The server evaluates the quality and quantity of the geographic information provided and calculates the reward.

[0241] Input: Quality checked data

[0242] Output: Reward points

[0243] Specific operation: The server evaluates the quality and coverage of the video data and calculates reward points.

[0244] Step 17:

[0245] The server credits the reward points to the user's account.

[0246] Input: Reward Points

[0247] Output: Points added to user account

[0248] Specific operation: The server calculates reward points and adds them to the user's account, which is managed in a MySQL database.

[0249] Step 18:

[0250] The terminal notifies the user that the reward has been granted.

[0251] Input: Reward point award notification

[0252] Output: Push notification

[0253] Specific operation: The device uses a push notification to notify the user that the reward has been added.

[0254] (Application example 1)

[0255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0256] Navigation systems for autonomous vehicles are required to reflect the latest road conditions in real time and ensure the safety of driving routes. Conventional navigation systems mainly provide route guidance based on static map information, making it difficult to immediately reflect information on road changes and obstacles. In addition, there is no adequate reward system for the vehicle's ability to recognize and record its surrounding environment, and there is a lack of means to encourage user participation. A new system is needed to solve these issues.

[0257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0258] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route, means for generating horizontal viewpoint map data, means for controlling the onboard system of the autonomous vehicle and providing navigation information, means for photographing road conditions and transmitting the obtained data to the server in real time, and means for updating the map database based on the photographed data and location information. This allows the autonomous vehicle to always receive route guidance that reflects the latest road conditions, enabling safer operation. In addition, users are rewarded for providing photographed data, which has the effect of increasing their motivation to participate.

[0259] "Means for obtaining user location information" refers to a function that obtains the current location from the user's device or in-vehicle system using technology such as GPS.

[0260] "Means for calculating the optimal route" refers to a function in which a server or computer system uses an algorithm to calculate the shortest or safest route to a destination based on location information.

[0261] The "means for generating horizontal perspective map data" is a function that creates map data that displays the route from the current location to the destination from a horizontal perspective using street view images, etc., based on the received location information.

[0262] "Means for displaying map data" refers to a function that displays the generated map data on a display or monitor, providing it visually to the user.

[0263] "Means for users to record geographic information in photography mode" refers to a function that allows users to use a device or in-vehicle camera to take videos or photographs of their surroundings and simultaneously record their location information.

[0264] "Means for receiving recorded geographic information and checking the quality" refers to a function that sends the recorded location information along with the captured images and videos to a server and evaluates and checks the quality of the data.

[0265] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a database and preserving it for later use.

[0266] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points according to the quantity and quality of geographic information provided by the user and awards the results to the user's account.

[0267] "Means for controlling the on-board systems of autonomous vehicles and providing navigation information" refers to a function that operates the autonomous driving system in the vehicle and provides guidance and route information to the destination in real time.

[0268] "Means for photographing road conditions and transmitting the obtained data to a server in real time" refers to a function that uses the vehicle's camera to photograph road and surrounding conditions and instantly uploads the data to a server.

[0269] The "means for updating the map database based on the photographing data and location information" is a function that analyzes the received photographing data and location information and updates the map database with the latest information.

[0270] The present invention will be described in detail below with reference to an embodiment thereof. This system is applied to a navigation system for an autonomous vehicle, and operates in the following manner.

[0271] First, the server obtains the user's current location information. Specifically, it obtains location information in real time from the vehicle's GPS unit and sends it to the server. The server then calculates the optimal route to the destination based on the received location information. The software used for this is a map database and a route calculation algorithm.

[0272] Next, the server generates map data for a horizontal perspective based on the calculated optimal route. This map data is created by converting real-world landscape images, such as street views, into a horizontal perspective, making it easy for users to understand intuitively. This map data is then sent to the onboard display of the autonomous vehicle.

[0273] The in-vehicle display displays the received map data and provides real-time navigation information, allowing the autonomous driving system to safely navigate the vehicle based on the most up-to-date route information.

[0274] Furthermore, the vehicle's onboard camera and GPS work together to record road conditions in real time. This allows the vehicle to send video data and location information captured while driving to a server. The server analyzes the received data and checks its quality. Data that meets the criteria is stored to update the map database within the system. The updated map database is then immediately reflected in other autonomous vehicles.

[0275] Users will receive reward points calculated based on the quantity and quality of the data they provide, which will be added to their account, giving them an incentive to actively provide road condition data.

[0276] Examples of hardware and software used

[0277] Hardware:

[0278] GPS unit: Get your current location.

[0279] Camera: Captures road conditions.

[0280] Display: Displays a horizontal perspective map.

[0281] software:

[0282] GPS module: Obtains current location.

[0283] Camera module: Implements photography functionality.

[0284] Display module: displays map data and shows routes.

[0285] Server connection module: Implements communication with the server.

[0286] Reward System Module: Manages reward points.

[0287] Specific prompt examples

[0288] Prompt statement:

[0289] Please send the longitude and latitude data obtained from the GPS unit to the server in real time, calculate the optimal route on the server side, generate and return horizontal perspective map data, and also send road condition data captured by the in-vehicle camera to the server.

[0290] This will enable autonomous vehicles to receive route guidance based on the latest road conditions at all times, improving safety. In addition, a reward system for users will encourage active data provision.

[0291] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0292] Step 1:

[0293] The device obtains current location information from the GPS unit and sends that data to the server in real time. The input is location information (longitude and latitude) from the GPS, and the output is the transmission of location information to the server. This location information data is sent in the format "The longitude of the current location is xxx, and the latitude is yyy."

[0294] Step 2:

[0295] The server calculates the optimal route based on the received location information. The input is the location information received from the device, and the output is map data for the optimal route. The server uses a map database and a route calculation algorithm to derive the optimal route. For example, it calculates the shortest distance between the departure point and destination, and creates horizontal perspective map data by referencing Street View data.

[0296] Step 3:

[0297] The server generates horizontal perspective map data and sends it to the terminal. The input is the route calculation results and map data from the server, and the output is the transmission of map data to the terminal. The map data is sent as image data and displayed on the on-board display of the autonomous vehicle.

[0298] Step 4:

[0299] The terminal displays the received map data on the display and instructs the route to the automated driving system. The input is the map data received from the server, and the output is the display on the display and route instructions to the automated driving system. The map displayed on the display has a horizontal perspective, allowing the user to intuitively understand it.

[0300] Step 5:

[0301] The device uses a camera to capture road conditions and sends the data along with GPS information to a server in real time. The input is the video data captured by the camera and location information from GPS, and the output is the data sent to the server. For example, the data is sent in the format "Longitude of the shooting location: xxx, Latitude: yyy, Video data: zzz."

[0302] Step 6:

[0303] The server analyzes the received video data and location information and checks its quality. The input is the video data and location information sent from the device, and the output is the quality check result. The server checks the clarity, accuracy, and consistency of the data with the location information, and only stores data that meets the criteria in the database.

[0304] Step 7:

[0305] The server updates the map database based on the quality-checked data. The input is quality-checked video data and location information, and the output is an updated map database. For example, new road or building information is added, and this information is immediately reflected in other autonomous vehicles.

[0306] Step 8:

[0307] The server calculates reward points for the user based on the quantity and quality of the geographic information data provided and grants them to the user's account. The input is the provided geographic information data and its quality evaluation, and the output is the calculation result and grant of reward points. The reward points are notified to the user's application interface.

[0308] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0309] MODE FOR CARRYING OUT THE INVENTION

[0310] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[0311] 1. Obtaining user location information

[0312] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[0313] 2. Operation of the Emotion Engine

[0314] The device uses a camera to capture the user's facial expressions in real time and analyzes them with an emotion engine. Based on the analysis results, the device identifies the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[0315] 3. Generating a horizontal viewpoint map

[0316] The server calculates the optimal route by referencing map data in the database based on the received location information and the emotion recognition results from the emotion engine. This calculation also takes into account the user's emotional state. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[0317] 4. Displaying and operating horizontal perspective maps

[0318] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[0319] 5. Starting the shooting mode and saving data

[0320] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[0321] 6. Providing Rewards

[0322] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[0323] Specific examples

[0324] Below are some specific examples of how this system can be used.

[0325] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination. Along the way, the device analyzes User A's facial expressions, and if it detects an anxious state, it adjusts the navigation tone, such as by softening it.

[0326] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0327] These features allow the present invention to provide an intuitive and safe navigation experience for users, flexibly respond to the user's emotional state, and realize real-time updates of map data.

[0328] The processing flow will be explained below.

[0329] Specific flow of program processing

[0330] Obtaining user location information

[0331] Step 1:

[0332] The user launches the app.

[0333] Specific behavior:

[0334] A user taps an app icon on their smartphone, launching the application.

[0335] Step 2:

[0336] The device activates the GPS function and obtains the current location information.

[0337] Specific behavior:

[0338] The device activates the GPS module and measures latitude and longitude data.

[0339] Step 3:

[0340] The terminal transmits the acquired location information to the server.

[0341] Specific behavior:

[0342] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[0343] Emotion Engine Operation

[0344] Step 1:

[0345] The device activates the camera and captures the user's facial expression.

[0346] Specific behavior:

[0347] Activate the camera module and fix the camera in a position where you want to recognize your face.

[0348] Step 2:

[0349] The device analyzes the captured facial expression data using an emotion engine.

[0350] Specific behavior:

[0351] Facial expression data is processed by image analysis algorithms to identify emotional states (e.g., happiness, surprise, anger, anxiety).

[0352] Step 3:

[0353] The terminal transmits the recognized emotional state to the server.

[0354] Specific behavior:

[0355] The emotional state is converted into JSON format as text data and sent to the server using an HTTP POST request.

[0356] Generating horizontal perspective maps

[0357] Step 1:

[0358] The server refers to a database based on the received location information and emotion data and calculates the optimal route.

[0359] Specific behavior:

[0360] The server retrieves road and destination information from the database and applies a shortest route calculation algorithm (e.g., Dijkstra's algorithm), taking into account the flexibility of the route depending on the emotion data.

[0361] Step 2:

[0362] The server generates horizontal perspective map data based on the calculated route.

[0363] Specific behavior:

[0364] The server calls the Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[0365] Step 3:

[0366] The server transmits the generated map data to the terminal.

[0367] Specific behavior:

[0368] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[0369] Displaying and operating horizontal perspective maps

[0370] Step 1:

[0371] The terminal stores the received map data in a cache.

[0372] Specific behavior:

[0373] Map data is temporarily stored in the device's storage in preparation for display processing.

[0374] Step 2:

[0375] The terminal displays the map data on the display.

[0376] Specific behavior:

[0377] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[0378] Step 3:

[0379] Users manipulate the map using their fingers or voice commands.

[0380] Specific behavior:

[0381] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[0382] Step 4:

[0383] The terminal acquires new map data from the server in response to a user operation.

[0384] Specific behavior:

[0385] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[0386] Starting the shooting mode and saving data

[0387] Step 1:

[0388] The user initiates the capture mode.

[0389] Specific behavior:

[0390] The user taps the "Shooting Mode" button within the app.

[0391] Step 2:

[0392] The device will activate the camera and GPS and begin recording location information and video data.

[0393] Specific behavior:

[0394] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[0395] Step 3:

[0396] The terminal transmits the recorded data to the server in real time.

[0397] Specific behavior:

[0398] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[0399] Step 4:

[0400] The server analyzes the received data and checks the quality criteria.

[0401] Specific behavior:

[0402] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[0403] Step 5:

[0404] The server stores data that meets the criteria in the system.

[0405] Specific behavior:

[0406] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[0407] Providing rewards

[0408] Step 1:

[0409] The server calculates points based on the amount and quality of data provided by the user.

[0410] Specific behavior:

[0411] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[0412] Step 2:

[0413] The server adds the calculated points to the user's account.

[0414] Specific behavior:

[0415] Update the user's account information and add the calculated points.

[0416] Step 3:

[0417] The server notifies the user that the reward has been granted.

[0418] Specific behavior:

[0419] The user will be notified by push notification or email that the reward has been granted.

[0420] In this way, the present invention not only provides an intuitive and safe navigation experience for users, real-time updates of map data, but also provides flexible services that respond to the user's emotional state.

[0421] Example 2

[0422] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0423] Current navigation systems only provide routes based on the user's location information, and lack the flexibility to adapt to the user's emotional state or real-time changes in the environment. While geographic information is updated periodically, it lacks real-time functionality and is difficult to reflect the latest geographic information. While there are systems that update map data based on geographic information provided by users, the specific rewards for such contributions are unclear. Furthermore, there is little feedback based on the user's usage rhythm, and improvements in user satisfaction are needed.

[0424] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0425] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, and means for generating horizontal viewpoint map data based on the route, thereby making it possible to provide an optimal route according to the user's location information.

[0426] Furthermore, the terminal includes a means for analyzing the user's facial expression and identifying the user's emotional state, a means for the user to record geographic information in a photographing mode, a means for receiving the recorded geographic information and checking its quality, a means for storing the checked geographic information, and a means for calculating and awarding a reward for providing the geographic information. This allows navigation based on the user's emotional state to be provided, and the latest geographic information provided by the user is updated in real time in the system. Furthermore, by clarifying the reward for users' contributions, it is expected that the user's motivation will increase and the accuracy of the entire system will improve.

[0427] The "means for acquiring user location information" is a function by which the electronic device acquires latitude and longitude data indicating the user's current location.

[0428] The "means for calculating a route" is a function for calculating the optimal travel route from the departure point to the destination based on the acquired location information.

[0429] The "means for generating horizontal viewpoint map data" is a function for converting map data based on calculated route information so that the data can be displayed at a horizontal viewpoint that is easy for the user to view.

[0430] The "means for displaying map data" is a function for displaying the generated horizontal viewpoint map data on the display screen of the electronic device.

[0431] "Means for analyzing the user's facial expressions and identifying the emotional state" refers to a function in which an electronic device captures and analyzes the user's facial expressions to identify the current emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[0432] The "means for recording geographic information in a photographing mode" is a function that records geographic information using the camera and the location information acquisition function when a user selects a specified photographing mode.

[0433] The "means for receiving recorded geographic information and checking the quality" is a function in which the server receives the geographic information acquired in the shooting mode and checks the quality (resolution, accuracy, etc.) of the information.

[0434] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a recording device such as a database.

[0435] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points based on the quantity and quality of geographic information provided by the user and awards them to the user's account.

[0436] A "generative AI model" is an algorithm or network model used to generate or analyze data based on artificial intelligence techniques.

[0437] A "prompt" is an instruction or question entered into a generative AI model to make it perform a specific task.

[0438] The present invention is a system that provides optimal navigation information based on a user's location information and emotional state, and updates geographic information in real time based on that information. The system includes a terminal carried by the user, a server that calculates location information and provides routes, and an emotion engine that analyzes the user's emotional state.

[0439] System Configuration

[0440] The system consists of the following main components:

[0441] Device: A mobile device used by a user, such as a smartphone or tablet. The device is equipped with a GPS function, a camera, a display, a touch panel, and a voice input function.

[0442] Server: A cloud computing environment or data center for large-scale data processing and analysis, including databases.

[0443] Emotion Engine: A software module that analyzes the user's facial expressions and determines their emotional state in real time. It incorporates specific emotion recognition algorithms.

[0444] Processing Details

[0445] 1. Obtaining user location information

[0446] When a user launches the app, the device uses its GPS function to obtain current location information in the form of latitude and longitude, which is then sent by the device to the server.

[0447] Example: When a user taps an app to launch it, the device automatically activates the GPS module and obtains the current location.

[0448] 2. Operation of the Emotion Engine

[0449] The device's camera captures the user's facial expressions and sends the video data to the emotion engine for analysis. The emotion engine uses a generative AI model to identify the user's emotional state in real time and sends the information to a server.

[0450] Example: Video data is analyzed through an emotion recognition algorithm (e.g., a convolutional neural network) to determine emotional states such as "joy" or "anxiety."

[0451] 3. Generating a horizontal viewpoint map

[0452] The server calculates the optimal route based on the received location information and emotion recognition results, referring to map data in the database. This calculation also takes into account the user's emotional state. Based on the calculated route information, the server generates horizontal perspective map data and sends it to the device.

[0453] Example: The server calculates a route using Dijkstra's algorithm, and if the user's emotional state is "anxious," it prioritizes a safer route.

[0454] 4. Displaying and operating horizontal perspective maps

[0455] The device displays the received map data on the display and allows the user to operate the map using touch operations or voice commands. If new map data is required for the operation, the device retrieves the additional data from the server.

[0456] For example, the user drags the map around or gives a voice command such as "next intersection."

[0457] 5. Starting the shooting mode and saving data

[0458] When a user activates the camera mode, the device's camera and GPS are activated, recording video data and location information, which are then sent to the server in real time. The server analyzes the received data, checks its quality, and stores data that meets the criteria in a database.

[0459] Example: A user takes a video of a tourist spot and sends it to the server along with its location information.

[0460] 6. Providing Rewards

[0461] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode and grants the reward points to the user's account. The terminal notifies the user that the reward has been granted.

[0462] For example: Points are calculated and added to your account based on the resolution and accuracy of the data you capture.

[0463] Examples and prompts

[0464] 1. Example 1

[0465] User A confirms that his current location is Chiyoda-ku, Tokyo, and launches the app. The server calculates the shortest route to Shibuya-ku, Tokyo, and sends it to the device. The device then guides User A using the displayed map and changes the navigation tone according to his emotional state.

[0466] 2. Example 2

[0467] User B uses the camera mode to visit tourist spots and record video data and location information. The server receives the data, checks the quality, and stores it in the database. User B is awarded reward points according to his / her contribution.

[0468] Prompt Sentence Examples

[0469] 1. "How can this system determine my location and generate the best route to navigate me?"

[0470] 2. "Please explain in detail how the system analyzes the user's facial expressions and reflects them in the navigation."

[0471] As a result, the present invention provides users with an intuitive and safe navigation experience, and is capable of flexibly responding to the user's emotional state. Furthermore, by real-time updating of map data and clarifying the reward system for user contributions, we expect to improve system accuracy and user satisfaction.

[0472] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0473] Step 1:

[0474] The user launches the app.

[0475] Input: User action (launching the app)

[0476] Output: Application startup status

[0477] Specific action: The user taps the app icon on the smartphone's home screen.

[0478] Step 2:

[0479] The device uses the GPS function to obtain the current location information and sends it to the server.

[0480] Input: User location request

[0481] Data processing: The device's GPS module obtains the current latitude and longitude.

[0482] Output: Latitude and longitude data is sent to the server

[0483] Specific operation: The GPS module is activated, receives signals from satellites, obtains location information, and sends the obtained data to the server via an HTTP request.

[0484] Step 3:

[0485] The device uses a camera to capture the user's facial expressions in real time and sends them to the emotion engine for analysis.

[0486] Input: Video data of the user's facial expressions

[0487] Data Computation: Emotion engine uses generative AI models to recognize emotional states (e.g., joy, surprise, anger, anxiety)

[0488] Output: Emotion recognition results are sent to the server

[0489] How it works: The front camera captures a picture of the user's face and sends the video data to the emotion engine. The emotion recognition algorithm analyzes the data to determine the user's emotional state and sends the result to the server via an HTTP request.

[0490] Step 4:

[0491] Based on the location information and emotion recognition results received by the server, the optimal route is calculated by referencing map data in the database, and horizontal perspective map data is generated.

[0492] Input: Latitude and longitude location, emotion recognition results

[0493] Data processing: The server calculates the route using Dijkstra's algorithm and converts the Street View image data into a horizontal perspective.

[0494] Output: Horizontal perspective map data is generated and sent to the device.

[0495] How it works: The server references an existing map database and uses Dijkstra's algorithm to calculate the optimal route based on location and emotion data. The result is then used to convert the Street View image into a horizontal perspective map, which is then packaged and sent to the device.

[0496] Step 5:

[0497] The device displays the received map data on the display, and the user can operate the map using touch operations or voice commands. If necessary, new map data is obtained from the server and the display is updated.

[0498] Input: Horizontal perspective map data

[0499] Output: Map displayed in the user interface, map data update requests according to user actions

[0500] What it does: It uses the GPU to draw map data and display it on the screen. The user can then manipulate the map using touch or voice commands, requesting new map data from the server as needed.

[0501] Step 6:

[0502] When a user starts the shooting mode, the device activates the camera and GPS, records the video data and location information, and sends it to the server, where it analyzes the data, checks its quality, and stores it in a database.

[0503] Input: User's action to activate the shooting mode, video data, location information

[0504] Output: High-quality geographic information stored in the cloud, reward points added to user accounts

[0505] Specific operation: The device's camera and GPS are activated, and the user taps the record button to capture video data. At the same time, location information is recorded, the data is compressed, and sent to the server. The server checks the quality of the data, and data that meets the standards is stored in the database.

[0506] Step 7:

[0507] The server calculates and awards a reward to the user according to the amount and quality of the geographic information provided in the shooting mode, and the terminal notifies the user that the reward has been awarded.

[0508] Input: Amount of geographic information provided, quality check results

[0509] Output: Notification that reward points have been added to the user's account.

[0510] Specific operation: The server calculates the volume and quality score of geographic information, and calculates the user's reward points based on the calculated score. The calculated points are recorded in the database, and the device notifies the user via push notification.

[0511] The above are the specific processing steps of the system, which enable the system to provide users with an intuitive and emotionally sensitive navigation experience and update the latest geographic information in real time.

[0512] (Application example 2)

[0513] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0514] Conventional navigation systems do not provide route guidance that takes into account the user's emotional state, which can lead to anxiety and stress. Furthermore, even systems that offer reward systems for providing geographic information do not provide real-time updates of map data or adjust navigation based on emotions, making it difficult to improve the user experience. There is a need to resolve these issues and provide more personalized and flexible route guidance.

[0515] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in shooting mode, means for receiving the recorded geographic information and checking the quality, means for saving the checked geographic information, means for calculating and awarding a reward for providing the geographic information, means for analyzing the user's emotional state using an emotion engine, means for automatically adjusting the tone of the navigation based on the emotional state, and means for recalculating an optimal route taking the emotional state into consideration. This enables flexible and personalized navigation according to the user's emotional state.

[0516] "User" means a person who uses the system.

[0517] "Location information" is latitude and longitude data that indicates the user's current location.

[0518] A "route" is the optimal travel route from a user's starting point to their destination.

[0519] "Horizontal viewpoint map data" refers to map display data viewed from the same height as the user's eye level.

[0520] "Geographic information" is data related to a specific geographic location, such as location information or video data.

[0521] "Rewards" means points or other forms of compensation calculated and awarded for geographic information provided by a user.

[0522] The "emotion engine" is a technology that uses cameras and sensors to analyze a user's facial expressions and identify their emotional state.

[0523] "Navigation tone" refers to the tone and expressions used in the voice and display when providing route guidance.

[0524] A "server" is a computer system that receives data from users and processes and stores various information.

[0525] DETAILED DESCRIPTION OF THE INVENTION The present invention provides a navigation system based on the emotional state of a user.

[0526] 1. Obtaining user location information

[0527] The device uses a GPS module to obtain the user's current location, which is obtained in the form of latitude and longitude and sent to the server.

[0528] 2. Operation of the Emotion Engine

[0529] The device uses a camera to capture the user's facial expressions in real time, which are then analyzed by an emotion engine, using software such as EmotionRecognizer to identify the user's emotional state, and the analysis results are sent to a server.

[0530] 3. Generating a horizontal viewpoint map

[0531] The server calculates the optimal route based on the received location information and emotion recognition results, referencing map data in its database. This calculation also takes into account the user's emotional state. Based on the optimal route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[0532] 4. Displaying and operating horizontal perspective maps

[0533] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[0534] 5. Starting the shooting mode and saving data

[0535] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database.

[0536] 6. Providing Rewards

[0537] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[0538] Specific examples

[0539] 1. A user boards an autonomous vehicle and requests navigation to their destination. Location information is acquired, and the server calculates the optimal route. The server analyzes the user's facial expressions along the way, and if it detects anxiety, it changes the tone of the navigation to be gentler.

[0540] 2. The user uses the photo mode to tour tourist spots in an autonomous vehicle. The camera and GPS are used to record video data and location information of the tourist spots. The server receives the captured data, checks its quality, and stores it in a database. Map data is updated based on the saved data. When other users visit the area at a later date, they can use the latest map data. The server awards reward points to the user based on the contribution of the captured data.

[0541] Prompt Sentence Examples

[0542] "Please show me a route to Tokyo Tower based on my current location."

[0543] "I'm feeling stressed, so please change to a more scenic route."

[0544] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0545] Step 1:

[0546] The device uses the GPS module to obtain the user's current location information (latitude and longitude). The obtained location information is sent to the server. The input is the location information data from the GPS module, and the output is the location information data sent to the server. This location information is used for subsequent route calculation.

[0547] Step 2:

[0548] The device uses a camera to capture the user's facial expressions in real time and inputs the video data into an emotion engine (e.g., EmotionRecognizer). The emotion engine analyzes the facial expressions to identify the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.). This emotional state is sent to a server. The input is the video data from the camera, and the output is the emotion recognition result.

[0549] Step 3:

[0550] The server calculates the optimal route by referencing map data in the database based on the location information acquired in step 1 and the emotion recognition results acquired in step 2. This calculation also takes into account the user's emotional state. The calculated optimal route is stored on the server. The input is location information and emotion data, and the output is optimal route data.

[0551] Step 4:

[0552] The server generates map data displayed in a horizontal perspective based on the optimal route. This map data is converted to a horizontal perspective using Street View images. The generated map data is sent to the device. The input is the optimal route data, and the output is horizontal perspective map data.

[0553] Step 5:

[0554] The terminal displays the received map data on the display and provides navigation to the user. The user operates the map using touch operations or voice commands. New map data required in response to the operation is obtained from the server and the display is updated. The input is horizontal perspective map data, and the output is navigation information displayed on the display.

[0555] Step 6:

[0556] When a user starts the shooting mode, the device activates the camera and GPS to record the video data and location information. The recorded data is sent to the server in real time. The input is the data from the camera and GPS, and the output is the video and location data sent to the server.

[0557] Step 7:

[0558] The server analyzes the received video and location data and checks its quality. Data that meets the standards is stored in a database within the system. The input is video and location data, and the output is high-quality geographic information stored in the database.

[0559] Step 8:

[0560] The server calculates and awards reward points to the user according to the quantity and quality of the geographic information provided in the shooting mode. The terminal is notified that the reward has been awarded. The input is the geographic information provided in the shooting mode, and the output is the calculated reward points and a notification of the reward.

[0561] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0562] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0563] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0564] [Second embodiment]

[0565] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0566] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0567] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0568] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0569] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0570] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0571] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0572] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0573] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0574] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0575] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0576] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0577] MODE FOR CARRYING OUT THE INVENTION

[0578] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[0579] 1. Obtaining user location information

[0580] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[0581] 2. Horizontal Perspective Map Generation

[0582] Based on the received location information, the server references map data in its database and calculates the optimal route from the current location to the destination. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[0583] 3. Displaying and operating horizontal perspective maps

[0584] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[0585] 4. Starting the shooting mode and saving data

[0586] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[0587] 5. Providing Rewards

[0588] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[0589] Specific examples

[0590] Below are some specific examples of how this system can be used.

[0591] 1. User A starts the app and confirms that their current location is Chiyoda-ku, Tokyo. The server calculates the shortest route to Shibuya-ku, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[0592] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0593] These features enable the present invention to provide an intuitive and safe navigation experience for users, while maintaining real-time, up-to-date map data.

[0594] The processing flow will be explained below.

[0595] Specific flow of program processing

[0596] Obtaining user location information

[0597] Step 1:

[0598] The user launches the app.

[0599] Specific behavior:

[0600] A user taps an app icon on their smartphone, launching the application.

[0601] Step 2:

[0602] The device activates the GPS function and obtains the current location information.

[0603] Specific behavior:

[0604] The device activates the GPS module and measures latitude and longitude data.

[0605] Step 3:

[0606] The terminal transmits the acquired location information to the server.

[0607] Specific behavior:

[0608] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[0609] Generating horizontal perspective maps

[0610] Step 1:

[0611] The server refers to a database based on the received location information and calculates the optimal route.

[0612] Specific behavior:

[0613] The server retrieves road and destination information from the database and applies a shortest path calculation algorithm (e.g., Dijkstra's algorithm).

[0614] Step 2:

[0615] The server generates horizontal perspective map data based on the calculated route.

[0616] Specific behavior:

[0617] The server calls the Google Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[0618] Step 3:

[0619] The server transmits the generated map data to the terminal.

[0620] Specific behavior:

[0621] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[0622] Displaying and operating horizontal perspective maps

[0623] Step 1:

[0624] The terminal stores the received map data in a cache.

[0625] Specific behavior:

[0626] Map data is temporarily stored in the device's storage in preparation for display processing.

[0627] Step 2:

[0628] The terminal displays the map data on the display.

[0629] Specific behavior:

[0630] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[0631] Step 3:

[0632] Users manipulate the map using their fingers or voice commands.

[0633] Specific behavior:

[0634] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[0635] Step 4:

[0636] The terminal acquires new map data from the server in response to a user operation.

[0637] Specific behavior:

[0638] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[0639] Starting the shooting mode and saving data

[0640] Step 1:

[0641] The user initiates the capture mode.

[0642] Specific behavior:

[0643] The user taps the "Shooting Mode" button within the app.

[0644] Step 2:

[0645] The device will activate the camera and GPS and begin recording location information and video data.

[0646] Specific behavior:

[0647] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[0648] Step 3:

[0649] The terminal transmits the recorded data to the server in real time.

[0650] Specific behavior:

[0651] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[0652] Step 4:

[0653] The server analyzes the received data and checks the quality criteria.

[0654] Specific behavior:

[0655] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[0656] Step 5:

[0657] The server stores data that meets the criteria in the system.

[0658] Specific behavior:

[0659] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[0660] Providing rewards

[0661] Step 1:

[0662] The server calculates points based on the amount and quality of data provided by the user.

[0663] Specific behavior:

[0664] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[0665] Step 2:

[0666] The server adds the calculated points to the user's account.

[0667] Specific behavior:

[0668] Update the user's account information and add the calculated points.

[0669] Step 3:

[0670] The server notifies the user that the reward has been granted.

[0671] Specific behavior:

[0672] The user will be notified by push notification or email that the reward has been granted.

[0673] In this way, the present invention provides an intuitive and safe navigation experience for users and allows for real-time updates of map data.

[0674] Example 1

[0675] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0676] Conventional navigation systems often lack the latest map data due to insufficient real-time location information acquisition and geographic information recording. Furthermore, there was no appropriate reward system for providing geographic information, and incentives based on users' contributions were lacking. This resulted in low user satisfaction and frequency of use, ultimately impairing the usefulness of the system as a whole.

[0677] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0678] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in a shooting mode, means for receiving the recorded geographic information in real time and analyzing the quality, means for storing the geographic information confirmed based on the analysis, and means for evaluating and awarding a reward for providing the geographic information. This allows users to use the latest map data in real time and receive an appropriate reward for providing geographic information, thereby improving the usability of the system.

[0679] "User location information" refers to the user's current latitude and longitude data obtained using a mobile device or GPS function.

[0680] An "optimal route" is the most efficient travel route between a user's starting point and their destination, calculated taking into account factors such as time, distance, and traffic conditions.

[0681] "Horizontal perspective map data" is map display data viewed from a horizontal direction, generated based on real-world scenery or street view images.

[0682] The "means for displaying map data" is an interface that uses the display of the terminal to visually present the acquired and generated map information to the user.

[0683] "Shooting mode" is an operation mode that uses the device's camera and GPS functions to record video data and location information.

[0684] "Geographic information" is data that includes location information within a specific area and visual information (images and video) of that location.

[0685] "Receiving in real time" means receiving data sent from a terminal immediately without delay.

[0686] "Means for analyzing quality" refers to the process of evaluating the accuracy, resolution, blur, etc. of the received geographic information data and confirming its quality.

[0687] "Means for storing" refers to storing the confirmed geographic information data in a storage device such as a database or cloud storage.

[0688] The "means for evaluating and awarding rewards" is a process for calculating the value of the geographic information provided based on the quantity and quality of the information and adding reward points to the user.

[0689] The following describes in detail the mode for carrying out the present invention. The present invention is a navigation system that acquires a user's location information, calculates an optimal route based on the acquired information, and generates and displays horizontal perspective map data. This system also includes a function to record geographic information in a photographing mode, send it to a server for quality check, and provide rewards to the user.

[0690] Obtaining user location information

[0691] When a user launches the application, the device uses its built-in GPS to obtain its current location. This location information is obtained in the form of latitude and longitude and sent to the server. Specifically, the location information is structured in JSON format and sent to the server using an HTTP request.

[0692] Generating horizontal perspective maps

[0693] The server analyzes the received location information and references map data in a database. It uses Python's GeoPandas library to calculate the optimal route from the current location to the destination. Based on this route information, it generates horizontal perspective map data using the Google Street View API. The generated map data is sent to the device as a JPEG image file.

[0694] Displaying and operating horizontal perspective maps

[0695] The device temporarily stores (caches) the received map data and displays it on the display. The map data is rendered using OpenGL, and the user can zoom in and out and move the map by pinching in, pinching out, and swiping. In response to user operations, the device requests new map data from the server and dynamically updates the map display.

[0696] Starting the shooting mode and saving data

[0697] When a user starts the shooting mode, the device activates the camera and GPS, simultaneously recording the captured video data and location information. This video data is encoded in H.264 format and sent to the server in real time via WebSocket communication along with GPS information. The server uses the OpenCV library to analyze the quality of the video data and check whether it meets certain criteria, such as blur and resolution. Data that meets the criteria is stored in cloud storage such as AWS S3.

[0698] Reward calculation and awarding

[0699] The server evaluates the quality and quantity of the geographic information provided and calculates reward points. This evaluation takes into account factors such as image quality and information coverage. The calculated reward points are immediately reflected in the user's account and managed in a MySQL database. The device uses push notifications to notify the user that a reward has been awarded.

[0700] Specific examples

[0701] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[0702] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in the database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0703] Prompt Sentence Examples

[0704] When a user launches the app, this system uses GPS to obtain their current location and sends it to the server. The server calculates the optimal route based on the received location information, generates horizontal perspective map data, and sends it to the device. The device displays this and navigates according to the user's operations. The user can then use the camera mode to record video data and location information and send it to the server. The server analyzes this, stores it in a database, and provides rewards to the user.

[0705] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0706] Step 1:

[0707] The user launches the app.

[0708] Specific behavior: When a user taps the app on their smartphone to launch it, the application invokes the GPS function in the background.

[0709] Step 2:

[0710] The device uses the GPS function to obtain the current location information.

[0711] Input: GPS satellite signal

[0712] Output: Latitude and longitude location (e.g. 35.6895, 139.6917)

[0713] How it works: The device's GPS module receives signals from satellites, analyzes them, and calculates the current latitude and longitude. This location information is structured in JSON format.

[0714] Step 3:

[0715] The location information acquired by the device is sent to the server.

[0716] Input: Latitude and longitude location information (JSON format)

[0717] Output: HTTP request sent to the server

[0718] What it does: The device packages the location information in JSON format and sends it to the server using the HTTPS protocol, along with the user's authentication token.

[0719] Step 4:

[0720] The server receives the user's location information.

[0721] Input: Location information (JSON format), authentication token

[0722] Output: Location information analysis results

[0723] Specific operation: The server deserializes the received JSON data and compares it with the map data stored in the database.

[0724] Step 5:

[0725] The server references the map data in the database.

[0726] Input:Location

[0727] Output: Map data reference results

[0728] How it works: The server uses Python's GeoPandas library to get the current location and related map data.

[0729] Step 6:

[0730] The server calculates the optimal route from the current location to the destination.

[0731] Input: Location information of starting point and destination

[0732] Output: Optimal route information (JSON format)

[0733] Specific operation: Calculates the optimal route taking into account distance, time, and traffic conditions using the GeoPandas library. The calculation results are saved in JSON format.

[0734] Step 7:

[0735] The server converts and generates map data based on Street View images into a horizontal perspective.

[0736] Input: Optimal route information

[0737] Output: Horizontal perspective map data (JPEG format)

[0738] Specific operation: Uses the Google Street View API to obtain image data of the area, converts it to a horizontal perspective, and generates an image in JPEG format.

[0739] Step 8:

[0740] The server transmits the generated map data to the terminal.

[0741] Input: Horizontal perspective map data (JPEG format)

[0742] Output: Sending data to the terminal

[0743] Specific operation: The server sends the generated map data to the terminal using the HTTPS protocol.

[0744] Step 9:

[0745] The terminal caches the received map data and displays the map data on the display.

[0746] Input: Horizontal perspective map data (JPEG format)

[0747] Output: Display

[0748] Specific operation: The device stores the received map data in a local cache and uses OpenGL to draw the map on the display.

[0749] Step 10:

[0750] The user manipulates the map to navigate.

[0751] Input: Touch, voice commands

[0752] Output: Updated navigation route

[0753] How it works: Users can zoom in and out and pan the map by pinching, pinching out, and swiping. They can also specify a destination using voice commands. The device will request new data from the server as needed and update the display.

[0754] Step 11:

[0755] The user initiates the capture mode.

[0756] Specific operation: When the user presses the "Photo mode" button in the app, the device activates the camera and GPS.

[0757] Step 12:

[0758] The device activates the camera and GPS to record video data and location information.

[0759] Input: Local image, latitude and longitude location information

[0760] Output: Encoded video data (H.264 format), location information

[0761] Specific operation: Local video captured by the camera is encoded in H.264 format and simultaneously recorded along with GPS information (latitude and longitude).

[0762] Step 13:

[0763] The device transmits the recorded data to the server in real time.

[0764] Input: Encoded video data, location information

[0765] Output: Send data to the server

[0766] Specific operation: The device uses WebSocket communication to send encoded video data and GPS location information to the server in real time.

[0767] Step 14:

[0768] The server analyzes the data and checks its quality.

[0769] Input: Video data, location information

[0770] Output: Quality analysis results

[0771] How it works: The server uses the OpenCV library to analyze the video for blur and resolution, checks the quality, and if it meets the criteria, proceeds to the next processing step.

[0772] Step 15:

[0773] The server stores the data that meets the criteria in a database.

[0774] Input: Quality checked data

[0775] Output: Save to database

[0776] Specific operation: Quality-confirmed data is stored in cloud storage such as AWS S3.

[0777] Step 16:

[0778] The server evaluates the quality and quantity of the geographic information provided and calculates the reward.

[0779] Input: Quality checked data

[0780] Output: Reward points

[0781] Specific operation: The server evaluates the quality and coverage of the video data and calculates reward points.

[0782] Step 17:

[0783] The server credits the reward points to the user's account.

[0784] Input: Reward Points

[0785] Output: Points added to user account

[0786] Specific operation: The server calculates reward points and adds them to the user's account, which is managed in a MySQL database.

[0787] Step 18:

[0788] The terminal notifies the user that the reward has been granted.

[0789] Input: Reward point award notification

[0790] Output: Push notification

[0791] Specific operation: The device uses a push notification to notify the user that the reward has been added.

[0792] (Application example 1)

[0793] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Navigation systems for autonomous vehicles are required to reflect the latest road conditions in real time and ensure the safety of driving routes. Conventional navigation systems mainly provide route guidance based on static map information, making it difficult to immediately reflect information on road changes and obstacles. In addition, there is no adequate reward system for the vehicle's ability to recognize and record its surrounding environment, and there is a lack of means to encourage user participation. A new system is needed to solve these issues.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0796] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route, means for generating horizontal viewpoint map data, means for controlling the onboard system of the autonomous vehicle and providing navigation information, means for photographing road conditions and transmitting the obtained data to the server in real time, and means for updating the map database based on the photographed data and location information. This allows the autonomous vehicle to always receive route guidance that reflects the latest road conditions, enabling safer operation. In addition, users are rewarded for providing photographed data, which has the effect of increasing their motivation to participate.

[0797] "Means for obtaining user location information" refers to a function that obtains the current location from the user's device or in-vehicle system using technology such as GPS.

[0798] "Means for calculating the optimal route" refers to a function in which a server or computer system uses an algorithm to calculate the shortest or safest route to a destination based on location information.

[0799] The "means for generating horizontal perspective map data" is a function that creates map data that displays the route from the current location to the destination from a horizontal perspective using street view images, etc., based on the received location information.

[0800] "Means for displaying map data" refers to a function that displays the generated map data on a display or monitor, providing it visually to the user.

[0801] "Means for users to record geographic information in photography mode" refers to a function that allows users to use a device or in-vehicle camera to take videos or photographs of their surroundings and simultaneously record their location information.

[0802] "Means for receiving recorded geographic information and checking the quality" refers to a function that sends the recorded location information along with the captured images and videos to a server and evaluates and checks the quality of the data.

[0803] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a database and preserving it for later use.

[0804] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points according to the quantity and quality of geographic information provided by the user and awards the results to the user's account.

[0805] "Means for controlling the on-board systems of autonomous vehicles and providing navigation information" refers to a function that operates the autonomous driving system in the vehicle and provides guidance and route information to the destination in real time.

[0806] "Means for photographing road conditions and transmitting the obtained data to a server in real time" refers to a function that uses the vehicle's camera to photograph road and surrounding conditions and instantly uploads the data to a server.

[0807] The "means for updating the map database based on the photographing data and location information" is a function that analyzes the received photographing data and location information and updates the map database with the latest information.

[0808] The present invention will be described in detail below with reference to an embodiment thereof. This system is applied to a navigation system for an autonomous vehicle, and operates in the following manner.

[0809] First, the server obtains the user's current location information. Specifically, it obtains location information in real time from the vehicle's GPS unit and sends it to the server. The server then calculates the optimal route to the destination based on the received location information. The software used for this is a map database and a route calculation algorithm.

[0810] Next, the server generates map data for a horizontal perspective based on the calculated optimal route. This map data is created by converting real-world landscape images, such as street views, into a horizontal perspective, making it easy for users to understand intuitively. This map data is then sent to the onboard display of the autonomous vehicle.

[0811] The in-vehicle display displays the received map data and provides real-time navigation information, allowing the autonomous driving system to safely navigate the vehicle based on the most up-to-date route information.

[0812] Furthermore, the vehicle's onboard camera and GPS work together to record road conditions in real time. This allows the vehicle to send video data and location information captured while driving to a server. The server analyzes the received data and checks its quality. Data that meets the criteria is stored to update the map database within the system. The updated map database is then immediately reflected in other autonomous vehicles.

[0813] Users will receive reward points calculated based on the quantity and quality of the data they provide, which will be added to their account, giving them an incentive to actively provide road condition data.

[0814] Examples of hardware and software used

[0815] Hardware:

[0816] GPS unit: Get your current location.

[0817] Camera: Captures road conditions.

[0818] Display: Displays a horizontal perspective map.

[0819] software:

[0820] GPS module: Obtains current location.

[0821] Camera module: Implements photography functionality.

[0822] Display module: displays map data and shows routes.

[0823] Server connection module: Implements communication with the server.

[0824] Reward System Module: Manages reward points.

[0825] Specific prompt examples

[0826] Prompt statement:

[0827] Please send the longitude and latitude data obtained from the GPS unit to the server in real time, calculate the optimal route on the server side, generate and return horizontal perspective map data, and also send road condition data captured by the in-vehicle camera to the server.

[0828] This will enable autonomous vehicles to receive route guidance based on the latest road conditions at all times, improving safety. In addition, a reward system for users will encourage active data provision.

[0829] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0830] Step 1:

[0831] The device obtains current location information from the GPS unit and sends that data to the server in real time. The input is location information (longitude and latitude) from the GPS, and the output is the transmission of location information to the server. This location information data is sent in the format "The longitude of the current location is xxx, and the latitude is yyy."

[0832] Step 2:

[0833] The server calculates the optimal route based on the received location information. The input is the location information received from the device, and the output is map data for the optimal route. The server uses a map database and a route calculation algorithm to derive the optimal route. For example, it calculates the shortest distance between the departure point and destination, and creates horizontal perspective map data by referencing Street View data.

[0834] Step 3:

[0835] The server generates horizontal perspective map data and sends it to the terminal. The input is the route calculation results and map data from the server, and the output is the transmission of map data to the terminal. The map data is sent as image data and displayed on the on-board display of the autonomous vehicle.

[0836] Step 4:

[0837] The terminal displays the received map data on the display and instructs the route to the automated driving system. The input is the map data received from the server, and the output is the display on the display and route instructions to the automated driving system. The map displayed on the display has a horizontal perspective, allowing the user to intuitively understand it.

[0838] Step 5:

[0839] The device uses a camera to capture road conditions and sends the data along with GPS information to a server in real time. The input is the video data captured by the camera and location information from GPS, and the output is the data sent to the server. For example, the data is sent in the format "Longitude of the shooting location: xxx, Latitude: yyy, Video data: zzz."

[0840] Step 6:

[0841] The server analyzes the received video data and location information and checks its quality. The input is the video data and location information sent from the device, and the output is the quality check result. The server checks the clarity, accuracy, and consistency of the data with the location information, and only stores data that meets the criteria in the database.

[0842] Step 7:

[0843] The server updates the map database based on the quality-checked data. The input is quality-checked video data and location information, and the output is an updated map database. For example, new road or building information is added, and this information is immediately reflected in other autonomous vehicles.

[0844] Step 8:

[0845] The server calculates reward points for the user based on the quantity and quality of the geographic information data provided and grants them to the user's account. The input is the provided geographic information data and its quality evaluation, and the output is the calculation result and grant of reward points. The reward points are notified to the user's application interface.

[0846] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0847] MODE FOR CARRYING OUT THE INVENTION

[0848] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[0849] 1. Obtaining user location information

[0850] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[0851] 2. Operation of the Emotion Engine

[0852] The device uses a camera to capture the user's facial expressions in real time and analyzes them with an emotion engine. Based on the analysis results, the device identifies the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[0853] 3. Generating a horizontal viewpoint map

[0854] The server calculates the optimal route by referencing map data in the database based on the received location information and the emotion recognition results from the emotion engine. This calculation also takes into account the user's emotional state. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[0855] 4. Displaying and operating horizontal perspective maps

[0856] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[0857] 5. Starting the shooting mode and saving data

[0858] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[0859] 6. Providing Rewards

[0860] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[0861] Specific examples

[0862] Below are some specific examples of how this system can be used.

[0863] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination. Along the way, the device analyzes User A's facial expressions, and if it detects an anxious state, it adjusts the navigation tone, such as by softening it.

[0864] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[0865] These features allow the present invention to provide an intuitive and safe navigation experience for users, flexibly respond to the user's emotional state, and realize real-time updates of map data.

[0866] The processing flow will be explained below.

[0867] Specific flow of program processing

[0868] Obtaining user location information

[0869] Step 1:

[0870] The user launches the app.

[0871] Specific behavior:

[0872] A user taps an app icon on their smartphone, launching the application.

[0873] Step 2:

[0874] The device activates the GPS function and obtains the current location information.

[0875] Specific behavior:

[0876] The device activates the GPS module and measures latitude and longitude data.

[0877] Step 3:

[0878] The terminal transmits the acquired location information to the server.

[0879] Specific behavior:

[0880] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[0881] Emotion Engine Operation

[0882] Step 1:

[0883] The device activates the camera and captures the user's facial expression.

[0884] Specific behavior:

[0885] Activate the camera module and fix the camera in a position where you want to recognize your face.

[0886] Step 2:

[0887] The device analyzes the captured facial expression data using an emotion engine.

[0888] Specific behavior:

[0889] Facial expression data is processed by image analysis algorithms to identify emotional states (e.g., happiness, surprise, anger, anxiety).

[0890] Step 3:

[0891] The terminal transmits the recognized emotional state to the server.

[0892] Specific behavior:

[0893] The emotional state is converted into JSON format as text data and sent to the server using an HTTP POST request.

[0894] Generating horizontal perspective maps

[0895] Step 1:

[0896] The server refers to a database based on the received location information and emotion data and calculates the optimal route.

[0897] Specific behavior:

[0898] The server retrieves road and destination information from the database and applies a shortest route calculation algorithm (e.g., Dijkstra's algorithm), taking into account the flexibility of the route depending on the emotion data.

[0899] Step 2:

[0900] The server generates horizontal perspective map data based on the calculated route.

[0901] Specific behavior:

[0902] The server calls the Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[0903] Step 3:

[0904] The server transmits the generated map data to the terminal.

[0905] Specific behavior:

[0906] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[0907] Displaying and operating horizontal perspective maps

[0908] Step 1:

[0909] The terminal stores the received map data in a cache.

[0910] Specific behavior:

[0911] Map data is temporarily stored in the device's storage in preparation for display processing.

[0912] Step 2:

[0913] The terminal displays the map data on the display.

[0914] Specific behavior:

[0915] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[0916] Step 3:

[0917] Users manipulate the map using their fingers or voice commands.

[0918] Specific behavior:

[0919] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[0920] Step 4:

[0921] The terminal acquires new map data from the server in response to a user operation.

[0922] Specific behavior:

[0923] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[0924] Starting the shooting mode and saving data

[0925] Step 1:

[0926] The user initiates the capture mode.

[0927] Specific behavior:

[0928] The user taps the "Shooting Mode" button within the app.

[0929] Step 2:

[0930] The device will activate the camera and GPS and begin recording location information and video data.

[0931] Specific behavior:

[0932] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[0933] Step 3:

[0934] The terminal transmits the recorded data to the server in real time.

[0935] Specific behavior:

[0936] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[0937] Step 4:

[0938] The server analyzes the received data and checks the quality criteria.

[0939] Specific behavior:

[0940] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[0941] Step 5:

[0942] The server stores data that meets the criteria in the system.

[0943] Specific behavior:

[0944] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[0945] Providing rewards

[0946] Step 1:

[0947] The server calculates points based on the amount and quality of data provided by the user.

[0948] Specific behavior:

[0949] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[0950] Step 2:

[0951] The server adds the calculated points to the user's account.

[0952] Specific behavior:

[0953] Update the user's account information and add the calculated points.

[0954] Step 3:

[0955] The server notifies the user that the reward has been granted.

[0956] Specific behavior:

[0957] The user will be notified by push notification or email that the reward has been granted.

[0958] In this way, the present invention not only provides an intuitive and safe navigation experience for users, real-time updates of map data, but also provides flexible services that respond to the user's emotional state.

[0959] Example 2

[0960] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0961] Current navigation systems only provide routes based on the user's location information, and lack the flexibility to adapt to the user's emotional state or real-time changes in the environment. While geographic information is updated periodically, it lacks real-time functionality and is difficult to reflect the latest geographic information. While there are systems that update map data based on geographic information provided by users, the specific rewards for such contributions are unclear. Furthermore, there is little feedback based on the user's usage rhythm, and improvements in user satisfaction are needed.

[0962] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0963] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, and means for generating horizontal viewpoint map data based on the route, thereby making it possible to provide an optimal route according to the user's location information.

[0964] Furthermore, the terminal includes a means for analyzing the user's facial expression and identifying the user's emotional state, a means for the user to record geographic information in a photographing mode, a means for receiving the recorded geographic information and checking its quality, a means for storing the checked geographic information, and a means for calculating and awarding a reward for providing the geographic information. This allows navigation based on the user's emotional state to be provided, and the latest geographic information provided by the user is updated in real time in the system. Furthermore, by clarifying the reward for users' contributions, it is expected that the user's motivation will increase and the accuracy of the entire system will improve.

[0965] The "means for acquiring user location information" is a function by which the electronic device acquires latitude and longitude data indicating the user's current location.

[0966] The "means for calculating a route" is a function for calculating the optimal travel route from the departure point to the destination based on the acquired location information.

[0967] The "means for generating horizontal viewpoint map data" is a function for converting map data based on calculated route information so that the data can be displayed at a horizontal viewpoint that is easy for the user to view.

[0968] The "means for displaying map data" is a function for displaying the generated horizontal viewpoint map data on the display screen of the electronic device.

[0969] "Means for analyzing the user's facial expressions and identifying the emotional state" refers to a function in which an electronic device captures and analyzes the user's facial expressions to identify the current emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[0970] The "means for recording geographic information in a photographing mode" is a function that records geographic information using the camera and the location information acquisition function when a user selects a specified photographing mode.

[0971] The "means for receiving recorded geographic information and checking the quality" is a function in which the server receives the geographic information acquired in the shooting mode and checks the quality (resolution, accuracy, etc.) of the information.

[0972] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a recording device such as a database.

[0973] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points based on the quantity and quality of geographic information provided by the user and awards them to the user's account.

[0974] A "generative AI model" is an algorithm or network model used to generate or analyze data based on artificial intelligence techniques.

[0975] A "prompt" is an instruction or question entered into a generative AI model to make it perform a specific task.

[0976] The present invention is a system that provides optimal navigation information based on a user's location information and emotional state, and updates geographic information in real time based on that information. The system includes a terminal carried by the user, a server that calculates location information and provides routes, and an emotion engine that analyzes the user's emotional state.

[0977] System Configuration

[0978] The system consists of the following main components:

[0979] Device: A mobile device used by a user, such as a smartphone or tablet. The device is equipped with a GPS function, a camera, a display, a touch panel, and a voice input function.

[0980] Server: A cloud computing environment or data center for large-scale data processing and analysis, including databases.

[0981] Emotion Engine: A software module that analyzes the user's facial expressions and determines their emotional state in real time. It incorporates specific emotion recognition algorithms.

[0982] Processing Details

[0983] 1. Obtaining user location information

[0984] When a user launches the app, the device uses its GPS function to obtain current location information in the form of latitude and longitude, which is then sent by the device to the server.

[0985] Example: When a user taps an app to launch it, the device automatically activates the GPS module and obtains the current location.

[0986] 2. Operation of the Emotion Engine

[0987] The device's camera captures the user's facial expressions and sends the video data to the emotion engine for analysis. The emotion engine uses a generative AI model to identify the user's emotional state in real time and sends the information to a server.

[0988] Example: Video data is analyzed through an emotion recognition algorithm (e.g., a convolutional neural network) to determine emotional states such as "joy" or "anxiety."

[0989] 3. Generating a horizontal viewpoint map

[0990] The server calculates the optimal route based on the received location information and emotion recognition results, referring to map data in the database. This calculation also takes into account the user's emotional state. Based on the calculated route information, the server generates horizontal perspective map data and sends it to the device.

[0991] Example: The server calculates a route using Dijkstra's algorithm, and if the user's emotional state is "anxious," it prioritizes a safer route.

[0992] 4. Displaying and operating horizontal perspective maps

[0993] The device displays the received map data on the display and allows the user to operate the map using touch operations or voice commands. If new map data is required for the operation, the device retrieves the additional data from the server.

[0994] For example, the user drags the map around or gives a voice command such as "next intersection."

[0995] 5. Starting the shooting mode and saving data

[0996] When a user activates the camera mode, the device's camera and GPS are activated, recording video data and location information, which are then sent to the server in real time. The server analyzes the received data, checks its quality, and stores data that meets the criteria in a database.

[0997] Example: A user takes a video of a tourist spot and sends it to the server along with its location information.

[0998] 6. Providing Rewards

[0999] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode and grants the reward points to the user's account. The terminal notifies the user that the reward has been granted.

[1000] For example: Points are calculated and added to your account based on the resolution and accuracy of the data you capture.

[1001] Examples and prompts

[1002] 1. Example 1

[1003] User A confirms that his current location is Chiyoda-ku, Tokyo, and launches the app. The server calculates the shortest route to Shibuya-ku, Tokyo, and sends it to the device. The device then guides User A using the displayed map and changes the navigation tone according to his emotional state.

[1004] 2. Example 2

[1005] User B uses the camera mode to visit tourist spots and record video data and location information. The server receives the data, checks the quality, and stores it in the database. User B is awarded reward points according to his / her contribution.

[1006] Prompt Sentence Examples

[1007] 1. "How can this system determine my location and generate the best route to navigate me?"

[1008] 2. "Please explain in detail how the system analyzes the user's facial expressions and reflects them in the navigation."

[1009] As a result, the present invention provides users with an intuitive and safe navigation experience, and is capable of flexibly responding to the user's emotional state. Furthermore, by real-time updating of map data and clarifying the reward system for user contributions, we expect to improve system accuracy and user satisfaction.

[1010] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1011] Step 1:

[1012] The user launches the app.

[1013] Input: User action (launching the app)

[1014] Output: Application startup status

[1015] Specific action: The user taps the app icon on the smartphone's home screen.

[1016] Step 2:

[1017] The device uses the GPS function to obtain the current location information and sends it to the server.

[1018] Input: User location request

[1019] Data processing: The device's GPS module obtains the current latitude and longitude.

[1020] Output: Latitude and longitude data is sent to the server

[1021] Specific operation: The GPS module is activated, receives signals from satellites, obtains location information, and sends the obtained data to the server via an HTTP request.

[1022] Step 3:

[1023] The device uses a camera to capture the user's facial expressions in real time and sends them to the emotion engine for analysis.

[1024] Input: Video data of the user's facial expressions

[1025] Data Computation: Emotion engine uses generative AI models to recognize emotional states (e.g., joy, surprise, anger, anxiety)

[1026] Output: Emotion recognition results are sent to the server

[1027] How it works: The front camera captures a picture of the user's face and sends the video data to the emotion engine. The emotion recognition algorithm analyzes the data to determine the user's emotional state and sends the result to the server via an HTTP request.

[1028] Step 4:

[1029] Based on the location information and emotion recognition results received by the server, the optimal route is calculated by referencing map data in the database, and horizontal perspective map data is generated.

[1030] Input: Latitude and longitude location, emotion recognition results

[1031] Data processing: The server calculates the route using Dijkstra's algorithm and converts the Street View image data into a horizontal perspective.

[1032] Output: Horizontal perspective map data is generated and sent to the device.

[1033] How it works: The server references an existing map database and uses Dijkstra's algorithm to calculate the optimal route based on location and emotion data. The result is then used to convert the Street View image into a horizontal perspective map, which is then packaged and sent to the device.

[1034] Step 5:

[1035] The device displays the received map data on the display, and the user can operate the map using touch operations or voice commands. If necessary, new map data is obtained from the server and the display is updated.

[1036] Input: Horizontal perspective map data

[1037] Output: Map displayed in the user interface, map data update requests according to user actions

[1038] What it does: It uses the GPU to draw map data and display it on the screen. The user can then manipulate the map using touch or voice commands, requesting new map data from the server as needed.

[1039] Step 6:

[1040] When a user starts the shooting mode, the device activates the camera and GPS, records the video data and location information, and sends it to the server, where it analyzes the data, checks its quality, and stores it in a database.

[1041] Input: User's action to activate the shooting mode, video data, location information

[1042] Output: High-quality geographic information stored in the cloud, reward points added to user accounts

[1043] Specific operation: The device's camera and GPS are activated, and the user taps the record button to capture video data. At the same time, location information is recorded, the data is compressed, and sent to the server. The server checks the quality of the data, and data that meets the standards is stored in the database.

[1044] Step 7:

[1045] The server calculates and awards a reward to the user according to the amount and quality of the geographic information provided in the shooting mode, and the terminal notifies the user that the reward has been awarded.

[1046] Input: Amount of geographic information provided, quality check results

[1047] Output: Notification that reward points have been added to the user's account.

[1048] Specific operation: The server calculates the volume and quality score of geographic information, and calculates the user's reward points based on the calculated score. The calculated points are recorded in the database, and the device notifies the user via push notification.

[1049] The above are the specific processing steps of the system, which enable the system to provide users with an intuitive and emotionally sensitive navigation experience and update the latest geographic information in real time.

[1050] (Application example 2)

[1051] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1052] Conventional navigation systems do not provide route guidance that takes into account the user's emotional state, which can lead to anxiety and stress. Furthermore, even systems that offer reward systems for providing geographic information do not provide real-time updates of map data or adjust navigation based on emotions, making it difficult to improve the user experience. There is a need to resolve these issues and provide more personalized and flexible route guidance.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in shooting mode, means for receiving the recorded geographic information and checking the quality, means for saving the checked geographic information, means for calculating and awarding a reward for providing the geographic information, means for analyzing the user's emotional state using an emotion engine, means for automatically adjusting the tone of the navigation based on the emotional state, and means for recalculating an optimal route taking the emotional state into consideration. This enables flexible and personalized navigation according to the user's emotional state.

[1054] "User" means a person who uses the system.

[1055] "Location information" is latitude and longitude data that indicates the user's current location.

[1056] A "route" is the optimal travel route from a user's starting point to their destination.

[1057] "Horizontal viewpoint map data" refers to map display data viewed from the same height as the user's eye level.

[1058] "Geographic information" is data related to a specific geographic location, such as location information or video data.

[1059] "Rewards" means points or other forms of compensation calculated and awarded for geographic information provided by a user.

[1060] The "emotion engine" is a technology that uses cameras and sensors to analyze a user's facial expressions and identify their emotional state.

[1061] "Navigation tone" refers to the tone and expressions used in the voice and display when providing route guidance.

[1062] A "server" is a computer system that receives data from users and processes and stores various information.

[1063] DETAILED DESCRIPTION OF THE INVENTION The present invention provides a navigation system based on the emotional state of a user.

[1064] 1. Obtaining user location information

[1065] The device uses a GPS module to obtain the user's current location, which is obtained in the form of latitude and longitude and sent to the server.

[1066] 2. Operation of the Emotion Engine

[1067] The device uses a camera to capture the user's facial expressions in real time, which are then analyzed by an emotion engine, using software such as EmotionRecognizer to identify the user's emotional state, and the analysis results are sent to a server.

[1068] 3. Generating a horizontal viewpoint map

[1069] The server calculates the optimal route based on the received location information and emotion recognition results, referencing map data in its database. This calculation also takes into account the user's emotional state. Based on the optimal route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1070] 4. Displaying and operating horizontal perspective maps

[1071] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1072] 5. Starting the shooting mode and saving data

[1073] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database.

[1074] 6. Providing Rewards

[1075] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1076] Specific examples

[1077] 1. A user boards an autonomous vehicle and requests navigation to their destination. Location information is acquired, and the server calculates the optimal route. The server analyzes the user's facial expressions along the way, and if it detects anxiety, it changes the tone of the navigation to be gentler.

[1078] 2. The user uses the photo mode to tour tourist spots in an autonomous vehicle. The camera and GPS are used to record video data and location information of the tourist spots. The server receives the captured data, checks its quality, and stores it in a database. Map data is updated based on the saved data. When other users visit the area at a later date, they can use the latest map data. The server awards reward points to the user based on the contribution of the captured data.

[1079] Prompt Sentence Examples

[1080] "Please show me a route to Tokyo Tower based on my current location."

[1081] "I'm feeling stressed, so please change to a more scenic route."

[1082] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1083] Step 1:

[1084] The device uses the GPS module to obtain the user's current location information (latitude and longitude). The obtained location information is sent to the server. The input is the location information data from the GPS module, and the output is the location information data sent to the server. This location information is used for subsequent route calculation.

[1085] Step 2:

[1086] The device uses a camera to capture the user's facial expressions in real time and inputs the video data into an emotion engine (e.g., EmotionRecognizer). The emotion engine analyzes the facial expressions to identify the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.). This emotional state is sent to a server. The input is the video data from the camera, and the output is the emotion recognition result.

[1087] Step 3:

[1088] The server calculates the optimal route by referencing map data in the database based on the location information acquired in step 1 and the emotion recognition results acquired in step 2. This calculation also takes into account the user's emotional state. The calculated optimal route is stored on the server. The input is location information and emotion data, and the output is optimal route data.

[1089] Step 4:

[1090] The server generates map data displayed in a horizontal perspective based on the optimal route. This map data is converted to a horizontal perspective using Street View images. The generated map data is sent to the device. The input is the optimal route data, and the output is horizontal perspective map data.

[1091] Step 5:

[1092] The terminal displays the received map data on the display and provides navigation to the user. The user operates the map using touch operations or voice commands. New map data required in response to the operation is obtained from the server and the display is updated. The input is horizontal perspective map data, and the output is navigation information displayed on the display.

[1093] Step 6:

[1094] When a user starts the shooting mode, the device activates the camera and GPS to record the video data and location information. The recorded data is sent to the server in real time. The input is the data from the camera and GPS, and the output is the video and location data sent to the server.

[1095] Step 7:

[1096] The server analyzes the received video and location data and checks its quality. Data that meets the standards is stored in a database within the system. The input is video and location data, and the output is high-quality geographic information stored in the database.

[1097] Step 8:

[1098] The server calculates and awards reward points to the user according to the quantity and quality of the geographic information provided in the shooting mode. The terminal is notified that the reward has been awarded. The input is the geographic information provided in the shooting mode, and the output is the calculated reward points and a notification of the reward.

[1099] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1100] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1101] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1102] [Third embodiment]

[1103] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1104] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1105] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1106] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1107] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1108] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1109] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1110] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1111] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1112] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1113] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1114] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1115] MODE FOR CARRYING OUT THE INVENTION

[1116] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[1117] 1. Obtaining user location information

[1118] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[1119] 2. Horizontal Perspective Map Generation

[1120] Based on the received location information, the server references map data in its database and calculates the optimal route from the current location to the destination. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1121] 3. Displaying and operating horizontal perspective maps

[1122] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1123] 4. Starting the shooting mode and saving data

[1124] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[1125] 5. Providing Rewards

[1126] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1127] Specific examples

[1128] Below are some specific examples of how this system can be used.

[1129] 1. User A starts the app and confirms that their current location is Chiyoda-ku, Tokyo. The server calculates the shortest route to Shibuya-ku, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[1130] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1131] These features enable the present invention to provide an intuitive and safe navigation experience for users, while maintaining real-time, up-to-date map data.

[1132] The processing flow will be explained below.

[1133] Specific flow of program processing

[1134] Obtaining user location information

[1135] Step 1:

[1136] The user launches the app.

[1137] Specific behavior:

[1138] A user taps an app icon on their smartphone, launching the application.

[1139] Step 2:

[1140] The device activates the GPS function and obtains the current location information.

[1141] Specific behavior:

[1142] The device activates the GPS module and measures latitude and longitude data.

[1143] Step 3:

[1144] The terminal transmits the acquired location information to the server.

[1145] Specific behavior:

[1146] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[1147] Generating horizontal perspective maps

[1148] Step 1:

[1149] The server refers to a database based on the received location information and calculates the optimal route.

[1150] Specific behavior:

[1151] The server retrieves road and destination information from the database and applies a shortest path calculation algorithm (e.g., Dijkstra's algorithm).

[1152] Step 2:

[1153] The server generates horizontal perspective map data based on the calculated route.

[1154] Specific behavior:

[1155] The server calls the Google Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[1156] Step 3:

[1157] The server transmits the generated map data to the terminal.

[1158] Specific behavior:

[1159] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[1160] Displaying and operating horizontal perspective maps

[1161] Step 1:

[1162] The terminal stores the received map data in a cache.

[1163] Specific behavior:

[1164] Map data is temporarily stored in the device's storage in preparation for display processing.

[1165] Step 2:

[1166] The terminal displays the map data on the display.

[1167] Specific behavior:

[1168] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[1169] Step 3:

[1170] Users manipulate the map using their fingers or voice commands.

[1171] Specific behavior:

[1172] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[1173] Step 4:

[1174] The terminal acquires new map data from the server in response to a user operation.

[1175] Specific behavior:

[1176] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[1177] Starting the shooting mode and saving data

[1178] Step 1:

[1179] The user initiates the capture mode.

[1180] Specific behavior:

[1181] The user taps the "Shooting Mode" button within the app.

[1182] Step 2:

[1183] The device will activate the camera and GPS and begin recording location information and video data.

[1184] Specific behavior:

[1185] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[1186] Step 3:

[1187] The terminal transmits the recorded data to the server in real time.

[1188] Specific behavior:

[1189] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[1190] Step 4:

[1191] The server analyzes the received data and checks the quality criteria.

[1192] Specific behavior:

[1193] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[1194] Step 5:

[1195] The server stores data that meets the criteria in the system.

[1196] Specific behavior:

[1197] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[1198] Providing rewards

[1199] Step 1:

[1200] The server calculates points based on the amount and quality of data provided by the user.

[1201] Specific behavior:

[1202] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[1203] Step 2:

[1204] The server adds the calculated points to the user's account.

[1205] Specific behavior:

[1206] Update the user's account information and add the calculated points.

[1207] Step 3:

[1208] The server notifies the user that the reward has been granted.

[1209] Specific behavior:

[1210] The user will be notified by push notification or email that the reward has been granted.

[1211] In this way, the present invention provides an intuitive and safe navigation experience for users and allows for real-time updates of map data.

[1212] Example 1

[1213] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1214] Conventional navigation systems often lack the latest map data due to insufficient real-time location information acquisition and geographic information recording. Furthermore, there was no appropriate reward system for providing geographic information, and incentives based on users' contributions were lacking. This resulted in low user satisfaction and frequency of use, ultimately impairing the usefulness of the system as a whole.

[1215] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1216] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in a shooting mode, means for receiving the recorded geographic information in real time and analyzing the quality, means for storing the geographic information confirmed based on the analysis, and means for evaluating and awarding a reward for providing the geographic information. This allows users to use the latest map data in real time and receive an appropriate reward for providing geographic information, thereby improving the usability of the system.

[1217] "User location information" refers to the user's current latitude and longitude data obtained using a mobile device or GPS function.

[1218] An "optimal route" is the most efficient travel route between a user's starting point and their destination, calculated taking into account factors such as time, distance, and traffic conditions.

[1219] "Horizontal perspective map data" is map display data viewed from a horizontal direction, generated based on real-world scenery or street view images.

[1220] The "means for displaying map data" is an interface that uses the display of the terminal to visually present the acquired and generated map information to the user.

[1221] "Shooting mode" is an operation mode that uses the device's camera and GPS functions to record video data and location information.

[1222] "Geographic information" is data that includes location information within a specific area and visual information (images and video) of that location.

[1223] "Receiving in real time" means receiving data sent from a terminal immediately without delay.

[1224] "Means for analyzing quality" refers to the process of evaluating the accuracy, resolution, blur, etc. of the received geographic information data and confirming its quality.

[1225] "Means for storing" refers to storing the confirmed geographic information data in a storage device such as a database or cloud storage.

[1226] The "means for evaluating and awarding rewards" is a process for calculating the value of the geographic information provided based on the quantity and quality of the information and adding reward points to the user.

[1227] The following describes in detail the mode for carrying out the present invention. The present invention is a navigation system that acquires a user's location information, calculates an optimal route based on the acquired information, and generates and displays horizontal perspective map data. This system also includes a function to record geographic information in a photographing mode, send it to a server for quality check, and provide rewards to the user.

[1228] Obtaining user location information

[1229] When a user launches the application, the device uses its built-in GPS to obtain its current location. This location information is obtained in the form of latitude and longitude and sent to the server. Specifically, the location information is structured in JSON format and sent to the server using an HTTP request.

[1230] Generating horizontal perspective maps

[1231] The server analyzes the received location information and references map data in a database. It uses Python's GeoPandas library to calculate the optimal route from the current location to the destination. Based on this route information, it generates horizontal perspective map data using the Google Street View API. The generated map data is sent to the device as a JPEG image file.

[1232] Displaying and operating horizontal perspective maps

[1233] The device temporarily stores (caches) the received map data and displays it on the display. The map data is rendered using OpenGL, and the user can zoom in and out and move the map by pinching in, pinching out, and swiping. In response to user operations, the device requests new map data from the server and dynamically updates the map display.

[1234] Starting the shooting mode and saving data

[1235] When a user starts the shooting mode, the device activates the camera and GPS, simultaneously recording the captured video data and location information. This video data is encoded in H.264 format and sent to the server in real time via WebSocket communication along with GPS information. The server uses the OpenCV library to analyze the quality of the video data and check whether it meets certain criteria, such as blur and resolution. Data that meets the criteria is stored in cloud storage such as AWS S3.

[1236] Reward calculation and awarding

[1237] The server evaluates the quality and quantity of the geographic information provided and calculates reward points. This evaluation takes into account factors such as image quality and information coverage. The calculated reward points are immediately reflected in the user's account and managed in a MySQL database. The device uses push notifications to notify the user that a reward has been awarded.

[1238] Specific examples

[1239] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[1240] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in the database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1241] Prompt Sentence Examples

[1242] When a user launches the app, this system uses GPS to obtain their current location and sends it to the server. The server calculates the optimal route based on the received location information, generates horizontal perspective map data, and sends it to the device. The device displays this and navigates according to the user's operations. The user can then use the camera mode to record video data and location information and send it to the server. The server analyzes this, stores it in a database, and provides rewards to the user.

[1243] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1244] Step 1:

[1245] The user launches the app.

[1246] Specific behavior: When a user taps the app on their smartphone to launch it, the application invokes the GPS function in the background.

[1247] Step 2:

[1248] The device uses the GPS function to obtain the current location information.

[1249] Input: GPS satellite signal

[1250] Output: Latitude and longitude location (e.g. 35.6895, 139.6917)

[1251] How it works: The device's GPS module receives signals from satellites, analyzes them, and calculates the current latitude and longitude. This location information is structured in JSON format.

[1252] Step 3:

[1253] The location information acquired by the device is sent to the server.

[1254] Input: Latitude and longitude location information (JSON format)

[1255] Output: HTTP request sent to the server

[1256] What it does: The device packages the location information in JSON format and sends it to the server using the HTTPS protocol, along with the user's authentication token.

[1257] Step 4:

[1258] The server receives the user's location information.

[1259] Input: Location information (JSON format), authentication token

[1260] Output: Location information analysis results

[1261] Specific operation: The server deserializes the received JSON data and compares it with the map data stored in the database.

[1262] Step 5:

[1263] The server references the map data in the database.

[1264] Input:Location

[1265] Output: Map data reference results

[1266] How it works: The server uses Python's GeoPandas library to get the current location and related map data.

[1267] Step 6:

[1268] The server calculates the optimal route from the current location to the destination.

[1269] Input: Location information of starting point and destination

[1270] Output: Optimal route information (JSON format)

[1271] Specific operation: Calculates the optimal route taking into account distance, time, and traffic conditions using the GeoPandas library. The calculation results are saved in JSON format.

[1272] Step 7:

[1273] The server converts and generates map data based on Street View images into a horizontal perspective.

[1274] Input: Optimal route information

[1275] Output: Horizontal perspective map data (JPEG format)

[1276] Specific operation: Uses the Google Street View API to obtain image data of the area, converts it to a horizontal perspective, and generates an image in JPEG format.

[1277] Step 8:

[1278] The server transmits the generated map data to the terminal.

[1279] Input: Horizontal perspective map data (JPEG format)

[1280] Output: Sending data to the terminal

[1281] Specific operation: The server sends the generated map data to the terminal using the HTTPS protocol.

[1282] Step 9:

[1283] The terminal caches the received map data and displays the map data on the display.

[1284] Input: Horizontal perspective map data (JPEG format)

[1285] Output: Display

[1286] Specific operation: The device stores the received map data in a local cache and uses OpenGL to draw the map on the display.

[1287] Step 10:

[1288] The user manipulates the map to navigate.

[1289] Input: Touch, voice commands

[1290] Output: Updated navigation route

[1291] How it works: Users can zoom in and out and pan the map by pinching, pinching out, and swiping. They can also specify a destination using voice commands. The device will request new data from the server as needed and update the display.

[1292] Step 11:

[1293] The user initiates the capture mode.

[1294] Specific operation: When the user presses the "Photo mode" button in the app, the device activates the camera and GPS.

[1295] Step 12:

[1296] The device activates the camera and GPS to record video data and location information.

[1297] Input: Local image, latitude and longitude location information

[1298] Output: Encoded video data (H.264 format), location information

[1299] Specific operation: Local video captured by the camera is encoded in H.264 format and simultaneously recorded along with GPS information (latitude and longitude).

[1300] Step 13:

[1301] The device transmits the recorded data to the server in real time.

[1302] Input: Encoded video data, location information

[1303] Output: Send data to the server

[1304] Specific operation: The device uses WebSocket communication to send encoded video data and GPS location information to the server in real time.

[1305] Step 14:

[1306] The server analyzes the data and checks its quality.

[1307] Input: Video data, location information

[1308] Output: Quality analysis results

[1309] How it works: The server uses the OpenCV library to analyze the video for blur and resolution, checks the quality, and if it meets the criteria, proceeds to the next processing step.

[1310] Step 15:

[1311] The server stores the data that meets the criteria in a database.

[1312] Input: Quality checked data

[1313] Output: Save to database

[1314] Specific operation: Quality-confirmed data is stored in cloud storage such as AWS S3.

[1315] Step 16:

[1316] The server evaluates the quality and quantity of the geographic information provided and calculates the reward.

[1317] Input: Quality checked data

[1318] Output: Reward points

[1319] Specific operation: The server evaluates the quality and coverage of the video data and calculates reward points.

[1320] Step 17:

[1321] The server credits the reward points to the user's account.

[1322] Input: Reward Points

[1323] Output: Points added to user account

[1324] Specific operation: The server calculates reward points and adds them to the user's account, which is managed in a MySQL database.

[1325] Step 18:

[1326] The terminal notifies the user that the reward has been granted.

[1327] Input: Reward point award notification

[1328] Output: Push notification

[1329] Specific operation: The device uses a push notification to notify the user that the reward has been added.

[1330] (Application example 1)

[1331] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1332] Navigation systems for autonomous vehicles are required to reflect the latest road conditions in real time and ensure the safety of driving routes. Conventional navigation systems mainly provide route guidance based on static map information, making it difficult to immediately reflect information on road changes and obstacles. In addition, there is no adequate reward system for the vehicle's ability to recognize and record its surrounding environment, and there is a lack of means to encourage user participation. A new system is needed to solve these issues.

[1333] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1334] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route, means for generating horizontal viewpoint map data, means for controlling the onboard system of the autonomous vehicle and providing navigation information, means for photographing road conditions and transmitting the obtained data to the server in real time, and means for updating the map database based on the photographed data and location information. This allows the autonomous vehicle to always receive route guidance that reflects the latest road conditions, enabling safer operation. In addition, users are rewarded for providing photographed data, which has the effect of increasing their motivation to participate.

[1335] "Means for obtaining user location information" refers to a function that obtains the current location from the user's device or in-vehicle system using technology such as GPS.

[1336] "Means for calculating the optimal route" refers to a function in which a server or computer system uses an algorithm to calculate the shortest or safest route to a destination based on location information.

[1337] The "means for generating horizontal perspective map data" is a function that creates map data that displays the route from the current location to the destination from a horizontal perspective using street view images, etc., based on the received location information.

[1338] "Means for displaying map data" refers to a function that displays the generated map data on a display or monitor, providing it visually to the user.

[1339] "Means for users to record geographic information in photography mode" refers to a function that allows users to use a device or in-vehicle camera to take videos or photographs of their surroundings and simultaneously record their location information.

[1340] "Means for receiving recorded geographic information and checking the quality" refers to a function that sends the recorded location information along with the captured images and videos to a server and evaluates and checks the quality of the data.

[1341] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a database and preserving it for later use.

[1342] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points according to the quantity and quality of geographic information provided by the user and awards the results to the user's account.

[1343] "Means for controlling the on-board systems of autonomous vehicles and providing navigation information" refers to a function that operates the autonomous driving system in the vehicle and provides guidance and route information to the destination in real time.

[1344] "Means for photographing road conditions and transmitting the obtained data to a server in real time" refers to a function that uses the vehicle's camera to photograph road and surrounding conditions and instantly uploads the data to a server.

[1345] The "means for updating the map database based on the photographing data and location information" is a function that analyzes the received photographing data and location information and updates the map database with the latest information.

[1346] The present invention will be described in detail below with reference to an embodiment thereof. This system is applied to a navigation system for an autonomous vehicle, and operates in the following manner.

[1347] First, the server obtains the user's current location information. Specifically, it obtains location information in real time from the vehicle's GPS unit and sends it to the server. The server then calculates the optimal route to the destination based on the received location information. The software used for this is a map database and a route calculation algorithm.

[1348] Next, the server generates map data for a horizontal perspective based on the calculated optimal route. This map data is created by converting real-world landscape images, such as street views, into a horizontal perspective, making it easy for users to understand intuitively. This map data is then sent to the onboard display of the autonomous vehicle.

[1349] The in-vehicle display displays the received map data and provides real-time navigation information, allowing the autonomous driving system to safely navigate the vehicle based on the most up-to-date route information.

[1350] Furthermore, the vehicle's onboard camera and GPS work together to record road conditions in real time. This allows the vehicle to send video data and location information captured while driving to a server. The server analyzes the received data and checks its quality. Data that meets the criteria is stored to update the map database within the system. The updated map database is then immediately reflected in other autonomous vehicles.

[1351] Users will receive reward points calculated based on the quantity and quality of the data they provide, which will be added to their account, giving them an incentive to actively provide road condition data.

[1352] Examples of hardware and software used

[1353] Hardware:

[1354] GPS unit: Get your current location.

[1355] Camera: Captures road conditions.

[1356] Display: Displays a horizontal perspective map.

[1357] software:

[1358] GPS module: Obtains current location.

[1359] Camera module: Implements photography functionality.

[1360] Display module: displays map data and shows routes.

[1361] Server connection module: Implements communication with the server.

[1362] Reward System Module: Manages reward points.

[1363] Specific prompt examples

[1364] Prompt statement:

[1365] Please send the longitude and latitude data obtained from the GPS unit to the server in real time, calculate the optimal route on the server side, generate and return horizontal perspective map data, and also send road condition data captured by the in-vehicle camera to the server.

[1366] This will enable autonomous vehicles to receive route guidance based on the latest road conditions at all times, improving safety. In addition, a reward system for users will encourage active data provision.

[1367] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1368] Step 1:

[1369] The device obtains current location information from the GPS unit and sends that data to the server in real time. The input is location information (longitude and latitude) from the GPS, and the output is the transmission of location information to the server. This location information data is sent in the format "The longitude of the current location is xxx, and the latitude is yyy."

[1370] Step 2:

[1371] The server calculates the optimal route based on the received location information. The input is the location information received from the device, and the output is map data for the optimal route. The server uses a map database and a route calculation algorithm to derive the optimal route. For example, it calculates the shortest distance between the departure point and destination, and creates horizontal perspective map data by referencing Street View data.

[1372] Step 3:

[1373] The server generates horizontal perspective map data and sends it to the terminal. The input is the route calculation results and map data from the server, and the output is the transmission of map data to the terminal. The map data is sent as image data and displayed on the on-board display of the autonomous vehicle.

[1374] Step 4:

[1375] The terminal displays the received map data on the display and instructs the route to the automated driving system. The input is the map data received from the server, and the output is the display on the display and route instructions to the automated driving system. The map displayed on the display has a horizontal perspective, allowing the user to intuitively understand it.

[1376] Step 5:

[1377] The device uses a camera to capture road conditions and sends the data along with GPS information to a server in real time. The input is the video data captured by the camera and location information from GPS, and the output is the data sent to the server. For example, the data is sent in the format "Longitude of the shooting location: xxx, Latitude: yyy, Video data: zzz."

[1378] Step 6:

[1379] The server analyzes the received video data and location information and checks its quality. The input is the video data and location information sent from the device, and the output is the quality check result. The server checks the clarity, accuracy, and consistency of the data with the location information, and only stores data that meets the criteria in the database.

[1380] Step 7:

[1381] The server updates the map database based on the quality-checked data. The input is quality-checked video data and location information, and the output is an updated map database. For example, new road or building information is added, and this information is immediately reflected in other autonomous vehicles.

[1382] Step 8:

[1383] The server calculates reward points for the user based on the quantity and quality of the geographic information data provided and grants them to the user's account. The input is the provided geographic information data and its quality evaluation, and the output is the calculation result and grant of reward points. The reward points are notified to the user's application interface.

[1384] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1385] MODE FOR CARRYING OUT THE INVENTION

[1386] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[1387] 1. Obtaining user location information

[1388] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[1389] 2. Operation of the Emotion Engine

[1390] The device uses a camera to capture the user's facial expressions in real time and analyzes them with an emotion engine. Based on the analysis results, the device identifies the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[1391] 3. Generating a horizontal viewpoint map

[1392] The server calculates the optimal route by referencing map data in the database based on the received location information and the emotion recognition results from the emotion engine. This calculation also takes into account the user's emotional state. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1393] 4. Displaying and operating horizontal perspective maps

[1394] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1395] 5. Starting the shooting mode and saving data

[1396] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[1397] 6. Providing Rewards

[1398] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1399] Specific examples

[1400] Below are some specific examples of how this system can be used.

[1401] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination. Along the way, the device analyzes User A's facial expressions, and if it detects an anxious state, it adjusts the navigation tone, such as by softening it.

[1402] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1403] These features allow the present invention to provide an intuitive and safe navigation experience for users, flexibly respond to the user's emotional state, and realize real-time updates of map data.

[1404] The processing flow will be explained below.

[1405] Specific flow of program processing

[1406] Obtaining user location information

[1407] Step 1:

[1408] The user launches the app.

[1409] Specific behavior:

[1410] A user taps an app icon on their smartphone, launching the application.

[1411] Step 2:

[1412] The device activates the GPS function and obtains the current location information.

[1413] Specific behavior:

[1414] The device activates the GPS module and measures latitude and longitude data.

[1415] Step 3:

[1416] The terminal transmits the acquired location information to the server.

[1417] Specific behavior:

[1418] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[1419] Emotion Engine Operation

[1420] Step 1:

[1421] The device activates the camera and captures the user's facial expression.

[1422] Specific behavior:

[1423] Activate the camera module and fix the camera in a position where you want to recognize your face.

[1424] Step 2:

[1425] The device analyzes the captured facial expression data using an emotion engine.

[1426] Specific behavior:

[1427] Facial expression data is processed by image analysis algorithms to identify emotional states (e.g., happiness, surprise, anger, anxiety).

[1428] Step 3:

[1429] The terminal transmits the recognized emotional state to the server.

[1430] Specific behavior:

[1431] The emotional state is converted into JSON format as text data and sent to the server using an HTTP POST request.

[1432] Generating horizontal perspective maps

[1433] Step 1:

[1434] The server refers to a database based on the received location information and emotion data and calculates the optimal route.

[1435] Specific behavior:

[1436] The server retrieves road and destination information from the database and applies a shortest route calculation algorithm (e.g., Dijkstra's algorithm), taking into account the flexibility of the route depending on the emotion data.

[1437] Step 2:

[1438] The server generates horizontal perspective map data based on the calculated route.

[1439] Specific behavior:

[1440] The server calls the Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[1441] Step 3:

[1442] The server transmits the generated map data to the terminal.

[1443] Specific behavior:

[1444] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[1445] Displaying and operating horizontal perspective maps

[1446] Step 1:

[1447] The terminal stores the received map data in a cache.

[1448] Specific behavior:

[1449] Map data is temporarily stored in the device's storage in preparation for display processing.

[1450] Step 2:

[1451] The terminal displays the map data on the display.

[1452] Specific behavior:

[1453] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[1454] Step 3:

[1455] Users manipulate the map using their fingers or voice commands.

[1456] Specific behavior:

[1457] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[1458] Step 4:

[1459] The terminal acquires new map data from the server in response to a user operation.

[1460] Specific behavior:

[1461] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[1462] Starting the shooting mode and saving data

[1463] Step 1:

[1464] The user initiates the capture mode.

[1465] Specific behavior:

[1466] The user taps the "Shooting Mode" button within the app.

[1467] Step 2:

[1468] The device will activate the camera and GPS and begin recording location information and video data.

[1469] Specific behavior:

[1470] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[1471] Step 3:

[1472] The terminal transmits the recorded data to the server in real time.

[1473] Specific behavior:

[1474] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[1475] Step 4:

[1476] The server analyzes the received data and checks the quality criteria.

[1477] Specific behavior:

[1478] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[1479] Step 5:

[1480] The server stores data that meets the criteria in the system.

[1481] Specific behavior:

[1482] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[1483] Providing rewards

[1484] Step 1:

[1485] The server calculates points based on the amount and quality of data provided by the user.

[1486] Specific behavior:

[1487] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[1488] Step 2:

[1489] The server adds the calculated points to the user's account.

[1490] Specific behavior:

[1491] Update the user's account information and add the calculated points.

[1492] Step 3:

[1493] The server notifies the user that the reward has been granted.

[1494] Specific behavior:

[1495] The user will be notified by push notification or email that the reward has been granted.

[1496] In this way, the present invention not only provides an intuitive and safe navigation experience for users, real-time updates of map data, but also provides flexible services that respond to the user's emotional state.

[1497] Example 2

[1498] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1499] Current navigation systems only provide routes based on the user's location information, and lack the flexibility to adapt to the user's emotional state or real-time changes in the environment. While geographic information is updated periodically, it lacks real-time functionality and is difficult to reflect the latest geographic information. While there are systems that update map data based on geographic information provided by users, the specific rewards for such contributions are unclear. Furthermore, there is little feedback based on the user's usage rhythm, and improvements in user satisfaction are needed.

[1500] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1501] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, and means for generating horizontal viewpoint map data based on the route, thereby making it possible to provide an optimal route according to the user's location information.

[1502] Furthermore, the terminal includes a means for analyzing the user's facial expression and identifying the user's emotional state, a means for the user to record geographic information in a photographing mode, a means for receiving the recorded geographic information and checking its quality, a means for storing the checked geographic information, and a means for calculating and awarding a reward for providing the geographic information. This allows navigation based on the user's emotional state to be provided, and the latest geographic information provided by the user is updated in real time in the system. Furthermore, by clarifying the reward for users' contributions, it is expected that the user's motivation will increase and the accuracy of the entire system will improve.

[1503] The "means for acquiring user location information" is a function by which the electronic device acquires latitude and longitude data indicating the user's current location.

[1504] The "means for calculating a route" is a function for calculating the optimal travel route from the departure point to the destination based on the acquired location information.

[1505] The "means for generating horizontal viewpoint map data" is a function for converting map data based on calculated route information so that the data can be displayed at a horizontal viewpoint that is easy for the user to view.

[1506] The "means for displaying map data" is a function for displaying the generated horizontal viewpoint map data on the display screen of the electronic device.

[1507] "Means for analyzing the user's facial expressions and identifying the emotional state" refers to a function in which an electronic device captures and analyzes the user's facial expressions to identify the current emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[1508] The "means for recording geographic information in a photographing mode" is a function that records geographic information using the camera and the location information acquisition function when a user selects a specified photographing mode.

[1509] The "means for receiving recorded geographic information and checking the quality" is a function in which the server receives the geographic information acquired in the shooting mode and checks the quality (resolution, accuracy, etc.) of the information.

[1510] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a recording device such as a database.

[1511] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points based on the quantity and quality of geographic information provided by the user and awards them to the user's account.

[1512] A "generative AI model" is an algorithm or network model used to generate or analyze data based on artificial intelligence techniques.

[1513] A "prompt" is an instruction or question entered into a generative AI model to make it perform a specific task.

[1514] The present invention is a system that provides optimal navigation information based on a user's location information and emotional state, and updates geographic information in real time based on that information. The system includes a terminal carried by the user, a server that calculates location information and provides routes, and an emotion engine that analyzes the user's emotional state.

[1515] System Configuration

[1516] The system consists of the following main components:

[1517] Device: A mobile device used by a user, such as a smartphone or tablet. The device is equipped with a GPS function, a camera, a display, a touch panel, and a voice input function.

[1518] Server: A cloud computing environment or data center for large-scale data processing and analysis, including databases.

[1519] Emotion Engine: A software module that analyzes the user's facial expressions and determines their emotional state in real time. It incorporates specific emotion recognition algorithms.

[1520] Processing Details

[1521] 1. Obtaining user location information

[1522] When a user launches the app, the device uses its GPS function to obtain current location information in the form of latitude and longitude, which is then sent by the device to the server.

[1523] Example: When a user taps an app to launch it, the device automatically activates the GPS module and obtains the current location.

[1524] 2. Operation of the Emotion Engine

[1525] The device's camera captures the user's facial expressions and sends the video data to the emotion engine for analysis. The emotion engine uses a generative AI model to identify the user's emotional state in real time and sends the information to a server.

[1526] Example: Video data is analyzed through an emotion recognition algorithm (e.g., a convolutional neural network) to determine emotional states such as "joy" or "anxiety."

[1527] 3. Generating a horizontal viewpoint map

[1528] The server calculates the optimal route based on the received location information and emotion recognition results, referring to map data in the database. This calculation also takes into account the user's emotional state. Based on the calculated route information, the server generates horizontal perspective map data and sends it to the device.

[1529] Example: The server calculates a route using Dijkstra's algorithm, and if the user's emotional state is "anxious," it prioritizes a safer route.

[1530] 4. Displaying and operating horizontal perspective maps

[1531] The device displays the received map data on the display and allows the user to operate the map using touch operations or voice commands. If new map data is required for the operation, the device retrieves the additional data from the server.

[1532] For example, the user drags the map around or gives a voice command such as "next intersection."

[1533] 5. Starting the shooting mode and saving data

[1534] When a user activates the camera mode, the device's camera and GPS are activated, recording video data and location information, which are then sent to the server in real time. The server analyzes the received data, checks its quality, and stores data that meets the criteria in a database.

[1535] Example: A user takes a video of a tourist spot and sends it to the server along with its location information.

[1536] 6. Providing Rewards

[1537] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode and grants the reward points to the user's account. The terminal notifies the user that the reward has been granted.

[1538] For example: Points are calculated and added to your account based on the resolution and accuracy of the data you capture.

[1539] Examples and prompts

[1540] 1. Example 1

[1541] User A confirms that his current location is Chiyoda-ku, Tokyo, and launches the app. The server calculates the shortest route to Shibuya-ku, Tokyo, and sends it to the device. The device then guides User A using the displayed map and changes the navigation tone according to his emotional state.

[1542] 2. Example 2

[1543] User B uses the camera mode to visit tourist spots and record video data and location information. The server receives the data, checks the quality, and stores it in the database. User B is awarded reward points according to his / her contribution.

[1544] Prompt Sentence Examples

[1545] 1. "How can this system determine my location and generate the best route to navigate me?"

[1546] 2. "Please explain in detail how the system analyzes the user's facial expressions and reflects them in the navigation."

[1547] As a result, the present invention provides users with an intuitive and safe navigation experience, and is capable of flexibly responding to the user's emotional state. Furthermore, by real-time updating of map data and clarifying the reward system for user contributions, we expect to improve system accuracy and user satisfaction.

[1548] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1549] Step 1:

[1550] The user launches the app.

[1551] Input: User action (launching the app)

[1552] Output: Application startup status

[1553] Specific action: The user taps the app icon on the smartphone's home screen.

[1554] Step 2:

[1555] The device uses the GPS function to obtain the current location information and sends it to the server.

[1556] Input: User location request

[1557] Data processing: The device's GPS module obtains the current latitude and longitude.

[1558] Output: Latitude and longitude data is sent to the server

[1559] Specific operation: The GPS module is activated, receives signals from satellites, obtains location information, and sends the obtained data to the server via an HTTP request.

[1560] Step 3:

[1561] The device uses a camera to capture the user's facial expressions in real time and sends them to the emotion engine for analysis.

[1562] Input: Video data of the user's facial expressions

[1563] Data Computation: Emotion engine uses generative AI models to recognize emotional states (e.g., joy, surprise, anger, anxiety)

[1564] Output: Emotion recognition results are sent to the server

[1565] How it works: The front camera captures a picture of the user's face and sends the video data to the emotion engine. The emotion recognition algorithm analyzes the data to determine the user's emotional state and sends the result to the server via an HTTP request.

[1566] Step 4:

[1567] Based on the location information and emotion recognition results received by the server, the optimal route is calculated by referencing map data in the database, and horizontal perspective map data is generated.

[1568] Input: Latitude and longitude location, emotion recognition results

[1569] Data processing: The server calculates the route using Dijkstra's algorithm and converts the Street View image data into a horizontal perspective.

[1570] Output: Horizontal perspective map data is generated and sent to the device.

[1571] How it works: The server references an existing map database and uses Dijkstra's algorithm to calculate the optimal route based on location and emotion data. The result is then used to convert the Street View image into a horizontal perspective map, which is then packaged and sent to the device.

[1572] Step 5:

[1573] The device displays the received map data on the display, and the user can operate the map using touch operations or voice commands. If necessary, new map data is obtained from the server and the display is updated.

[1574] Input: Horizontal perspective map data

[1575] Output: Map displayed in the user interface, map data update requests according to user actions

[1576] What it does: It uses the GPU to draw map data and display it on the screen. The user can then manipulate the map using touch or voice commands, requesting new map data from the server as needed.

[1577] Step 6:

[1578] When a user starts the shooting mode, the device activates the camera and GPS, records the video data and location information, and sends it to the server, where it analyzes the data, checks its quality, and stores it in a database.

[1579] Input: User's action to activate the shooting mode, video data, location information

[1580] Output: High-quality geographic information stored in the cloud, reward points added to user accounts

[1581] Specific operation: The device's camera and GPS are activated, and the user taps the record button to capture video data. At the same time, location information is recorded, the data is compressed, and sent to the server. The server checks the quality of the data, and data that meets the standards is stored in the database.

[1582] Step 7:

[1583] The server calculates and awards a reward to the user according to the amount and quality of the geographic information provided in the shooting mode, and the terminal notifies the user that the reward has been awarded.

[1584] Input: Amount of geographic information provided, quality check results

[1585] Output: Notification that reward points have been added to the user's account.

[1586] Specific operation: The server calculates the volume and quality score of geographic information, and calculates the user's reward points based on the calculated score. The calculated points are recorded in the database, and the device notifies the user via push notification.

[1587] The above are the specific processing steps of the system, which enable the system to provide users with an intuitive and emotionally sensitive navigation experience and update the latest geographic information in real time.

[1588] (Application example 2)

[1589] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1590] Conventional navigation systems do not provide route guidance that takes into account the user's emotional state, which can lead to anxiety and stress. Furthermore, even systems that offer reward systems for providing geographic information do not provide real-time updates of map data or adjust navigation based on emotions, making it difficult to improve the user experience. There is a need to resolve these issues and provide more personalized and flexible route guidance.

[1591] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in shooting mode, means for receiving the recorded geographic information and checking the quality, means for saving the checked geographic information, means for calculating and awarding a reward for providing the geographic information, means for analyzing the user's emotional state using an emotion engine, means for automatically adjusting the tone of the navigation based on the emotional state, and means for recalculating an optimal route taking the emotional state into consideration. This enables flexible and personalized navigation according to the user's emotional state.

[1592] "User" means a person who uses the system.

[1593] "Location information" is latitude and longitude data that indicates the user's current location.

[1594] A "route" is the optimal travel route from a user's starting point to their destination.

[1595] "Horizontal viewpoint map data" refers to map display data viewed from the same height as the user's eye level.

[1596] "Geographic information" is data related to a specific geographic location, such as location information or video data.

[1597] "Rewards" means points or other forms of compensation calculated and awarded for geographic information provided by a user.

[1598] The "emotion engine" is a technology that uses cameras and sensors to analyze a user's facial expressions and identify their emotional state.

[1599] "Navigation tone" refers to the tone and expressions used in the voice and display when providing route guidance.

[1600] A "server" is a computer system that receives data from users and processes and stores various information.

[1601] DETAILED DESCRIPTION OF THE INVENTION The present invention provides a navigation system based on the emotional state of a user.

[1602] 1. Obtaining user location information

[1603] The device uses a GPS module to obtain the user's current location, which is obtained in the form of latitude and longitude and sent to the server.

[1604] 2. Operation of the Emotion Engine

[1605] The device uses a camera to capture the user's facial expressions in real time, which are then analyzed by an emotion engine, using software such as EmotionRecognizer to identify the user's emotional state, and the analysis results are sent to a server.

[1606] 3. Generating a horizontal viewpoint map

[1607] The server calculates the optimal route based on the received location information and emotion recognition results, referencing map data in its database. This calculation also takes into account the user's emotional state. Based on the optimal route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1608] 4. Displaying and operating horizontal perspective maps

[1609] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1610] 5. Starting the shooting mode and saving data

[1611] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database.

[1612] 6. Providing Rewards

[1613] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1614] Specific examples

[1615] 1. A user boards an autonomous vehicle and requests navigation to their destination. Location information is acquired, and the server calculates the optimal route. The server analyzes the user's facial expressions along the way, and if it detects anxiety, it changes the tone of the navigation to be gentler.

[1616] 2. The user uses the photo mode to tour tourist spots in an autonomous vehicle. The camera and GPS are used to record video data and location information of the tourist spots. The server receives the captured data, checks its quality, and stores it in a database. Map data is updated based on the saved data. When other users visit the area at a later date, they can use the latest map data. The server awards reward points to the user based on the contribution of the captured data.

[1617] Prompt Sentence Examples

[1618] "Please show me a route to Tokyo Tower based on my current location."

[1619] "I'm feeling stressed, so please change to a more scenic route."

[1620] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1621] Step 1:

[1622] The device uses the GPS module to obtain the user's current location information (latitude and longitude). The obtained location information is sent to the server. The input is the location information data from the GPS module, and the output is the location information data sent to the server. This location information is used for subsequent route calculation.

[1623] Step 2:

[1624] The device uses a camera to capture the user's facial expressions in real time and inputs the video data into an emotion engine (e.g., EmotionRecognizer). The emotion engine analyzes the facial expressions to identify the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.). This emotional state is sent to a server. The input is the video data from the camera, and the output is the emotion recognition result.

[1625] Step 3:

[1626] The server calculates the optimal route by referencing map data in the database based on the location information acquired in step 1 and the emotion recognition results acquired in step 2. This calculation also takes into account the user's emotional state. The calculated optimal route is stored on the server. The input is location information and emotion data, and the output is optimal route data.

[1627] Step 4:

[1628] The server generates map data displayed in a horizontal perspective based on the optimal route. This map data is converted to a horizontal perspective using Street View images. The generated map data is sent to the device. The input is the optimal route data, and the output is horizontal perspective map data.

[1629] Step 5:

[1630] The terminal displays the received map data on the display and provides navigation to the user. The user operates the map using touch operations or voice commands. New map data required in response to the operation is obtained from the server and the display is updated. The input is horizontal perspective map data, and the output is navigation information displayed on the display.

[1631] Step 6:

[1632] When a user starts the shooting mode, the device activates the camera and GPS to record the video data and location information. The recorded data is sent to the server in real time. The input is the data from the camera and GPS, and the output is the video and location data sent to the server.

[1633] Step 7:

[1634] The server analyzes the received video and location data and checks its quality. Data that meets the standards is stored in a database within the system. The input is video and location data, and the output is high-quality geographic information stored in the database.

[1635] Step 8:

[1636] The server calculates and awards reward points to the user according to the quantity and quality of the geographic information provided in the shooting mode. The terminal is notified that the reward has been awarded. The input is the geographic information provided in the shooting mode, and the output is the calculated reward points and a notification of the reward.

[1637] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1638] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1639] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1640] [Fourth embodiment]

[1641] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1642] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1643] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1644] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1645] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1646] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1647] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1648] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1649] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1650] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1651] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1652] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1653] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1654] MODE FOR CARRYING OUT THE INVENTION

[1655] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[1656] 1. Obtaining user location information

[1657] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[1658] 2. Horizontal Perspective Map Generation

[1659] Based on the received location information, the server references map data in its database and calculates the optimal route from the current location to the destination. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1660] 3. Displaying and operating horizontal perspective maps

[1661] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1662] 4. Starting the shooting mode and saving data

[1663] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[1664] 5. Providing Rewards

[1665] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1666] Specific examples

[1667] Below are some specific examples of how this system can be used.

[1668] 1. User A starts the app and confirms that their current location is Chiyoda-ku, Tokyo. The server calculates the shortest route to Shibuya-ku, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[1669] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1670] These features enable the present invention to provide an intuitive and safe navigation experience for users, while maintaining real-time, up-to-date map data.

[1671] The processing flow will be explained below.

[1672] Specific flow of program processing

[1673] Obtaining user location information

[1674] Step 1:

[1675] The user launches the app.

[1676] Specific behavior:

[1677] A user taps an app icon on their smartphone, launching the application.

[1678] Step 2:

[1679] The device activates the GPS function and obtains the current location information.

[1680] Specific behavior:

[1681] The device activates the GPS module and measures latitude and longitude data.

[1682] Step 3:

[1683] The terminal transmits the acquired location information to the server.

[1684] Specific behavior:

[1685] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[1686] Generating horizontal perspective maps

[1687] Step 1:

[1688] The server refers to a database based on the received location information and calculates the optimal route.

[1689] Specific behavior:

[1690] The server retrieves road and destination information from the database and applies a shortest path calculation algorithm (e.g., Dijkstra's algorithm).

[1691] Step 2:

[1692] The server generates horizontal perspective map data based on the calculated route.

[1693] Specific behavior:

[1694] The server calls the Google Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[1695] Step 3:

[1696] The server transmits the generated map data to the terminal.

[1697] Specific behavior:

[1698] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[1699] Displaying and operating horizontal perspective maps

[1700] Step 1:

[1701] The terminal stores the received map data in a cache.

[1702] Specific behavior:

[1703] Map data is temporarily stored in the device's storage in preparation for display processing.

[1704] Step 2:

[1705] The terminal displays the map data on the display.

[1706] Specific behavior:

[1707] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[1708] Step 3:

[1709] Users manipulate the map using their fingers or voice commands.

[1710] Specific behavior:

[1711] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[1712] Step 4:

[1713] The terminal acquires new map data from the server in response to a user operation.

[1714] Specific behavior:

[1715] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[1716] Starting the shooting mode and saving data

[1717] Step 1:

[1718] The user initiates the capture mode.

[1719] Specific behavior:

[1720] The user taps the "Shooting Mode" button within the app.

[1721] Step 2:

[1722] The device will activate the camera and GPS and begin recording location information and video data.

[1723] Specific behavior:

[1724] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[1725] Step 3:

[1726] The terminal transmits the recorded data to the server in real time.

[1727] Specific behavior:

[1728] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[1729] Step 4:

[1730] The server analyzes the received data and checks the quality criteria.

[1731] Specific behavior:

[1732] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[1733] Step 5:

[1734] The server stores data that meets the criteria in the system.

[1735] Specific behavior:

[1736] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[1737] Providing rewards

[1738] Step 1:

[1739] The server calculates points based on the amount and quality of data provided by the user.

[1740] Specific behavior:

[1741] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[1742] Step 2:

[1743] The server adds the calculated points to the user's account.

[1744] Specific behavior:

[1745] Update the user's account information and add the calculated points.

[1746] Step 3:

[1747] The server notifies the user that the reward has been granted.

[1748] Specific behavior:

[1749] The user will be notified by push notification or email that the reward has been granted.

[1750] In this way, the present invention provides an intuitive and safe navigation experience for users and allows for real-time updates of map data.

[1751] Example 1

[1752] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1753] Conventional navigation systems often lack the latest map data due to insufficient real-time location information acquisition and geographic information recording. Furthermore, there was no appropriate reward system for providing geographic information, and incentives based on users' contributions were lacking. This resulted in low user satisfaction and frequency of use, ultimately impairing the usefulness of the system as a whole.

[1754] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1755] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in a shooting mode, means for receiving the recorded geographic information in real time and analyzing the quality, means for storing the geographic information confirmed based on the analysis, and means for evaluating and awarding a reward for providing the geographic information. This allows users to use the latest map data in real time and receive an appropriate reward for providing geographic information, thereby improving the usability of the system.

[1756] "User location information" refers to the user's current latitude and longitude data obtained using a mobile device or GPS function.

[1757] An "optimal route" is the most efficient travel route between a user's starting point and their destination, calculated taking into account factors such as time, distance, and traffic conditions.

[1758] "Horizontal perspective map data" is map display data viewed from a horizontal direction, generated based on real-world scenery or street view images.

[1759] The "means for displaying map data" is an interface that uses the display of the terminal to visually present the acquired and generated map information to the user.

[1760] "Shooting mode" is an operation mode that uses the device's camera and GPS functions to record video data and location information.

[1761] "Geographic information" is data that includes location information within a specific area and visual information (images and video) of that location.

[1762] "Receiving in real time" means receiving data sent from a terminal immediately without delay.

[1763] "Means for analyzing quality" refers to the process of evaluating the accuracy, resolution, blur, etc. of the received geographic information data and confirming its quality.

[1764] "Means for storing" refers to storing the confirmed geographic information data in a storage device such as a database or cloud storage.

[1765] The "means for evaluating and awarding rewards" is a process for calculating the value of the geographic information provided based on the quantity and quality of the information and adding reward points to the user.

[1766] The following describes in detail the mode for carrying out the present invention. The present invention is a navigation system that acquires a user's location information, calculates an optimal route based on the acquired information, and generates and displays horizontal perspective map data. This system also includes a function to record geographic information in a photographing mode, send it to a server for quality check, and provide rewards to the user.

[1767] Obtaining user location information

[1768] When a user launches the application, the device uses its built-in GPS to obtain its current location. This location information is obtained in the form of latitude and longitude and sent to the server. Specifically, the location information is structured in JSON format and sent to the server using an HTTP request.

[1769] Generating horizontal perspective maps

[1770] The server analyzes the received location information and references map data in a database. It uses Python's GeoPandas library to calculate the optimal route from the current location to the destination. Based on this route information, it generates horizontal perspective map data using the Google Street View API. The generated map data is sent to the device as a JPEG image file.

[1771] Displaying and operating horizontal perspective maps

[1772] The device temporarily stores (caches) the received map data and displays it on the display. The map data is rendered using OpenGL, and the user can zoom in and out and move the map by pinching in, pinching out, and swiping. In response to user operations, the device requests new map data from the server and dynamically updates the map display.

[1773] Starting the shooting mode and saving data

[1774] When a user starts the shooting mode, the device activates the camera and GPS, simultaneously recording the captured video data and location information. This video data is encoded in H.264 format and sent to the server in real time via WebSocket communication along with GPS information. The server uses the OpenCV library to analyze the quality of the video data and check whether it meets certain criteria, such as blur and resolution. Data that meets the criteria is stored in cloud storage such as AWS S3.

[1775] Reward calculation and awarding

[1776] The server evaluates the quality and quantity of the geographic information provided and calculates reward points. This evaluation takes into account factors such as image quality and information coverage. The calculated reward points are immediately reflected in the user's account and managed in a MySQL database. The device uses push notifications to notify the user that a reward has been awarded.

[1777] Specific examples

[1778] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination.

[1779] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in the database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1780] Prompt Sentence Examples

[1781] When a user launches the app, this system uses GPS to obtain their current location and sends it to the server. The server calculates the optimal route based on the received location information, generates horizontal perspective map data, and sends it to the device. The device displays this and navigates according to the user's operations. The user can then use the camera mode to record video data and location information and send it to the server. The server analyzes this, stores it in a database, and provides rewards to the user.

[1782] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1783] Step 1:

[1784] The user launches the app.

[1785] Specific behavior: When a user taps the app on their smartphone to launch it, the application invokes the GPS function in the background.

[1786] Step 2:

[1787] The device uses the GPS function to obtain the current location information.

[1788] Input: GPS satellite signal

[1789] Output: Latitude and longitude location (e.g. 35.6895, 139.6917)

[1790] How it works: The device's GPS module receives signals from satellites, analyzes them, and calculates the current latitude and longitude. This location information is structured in JSON format.

[1791] Step 3:

[1792] The location information acquired by the device is sent to the server.

[1793] Input: Latitude and longitude location information (JSON format)

[1794] Output: HTTP request sent to the server

[1795] What it does: The device packages the location information in JSON format and sends it to the server using the HTTPS protocol, along with the user's authentication token.

[1796] Step 4:

[1797] The server receives the user's location information.

[1798] Input: Location information (JSON format), authentication token

[1799] Output: Location information analysis results

[1800] Specific operation: The server deserializes the received JSON data and compares it with the map data stored in the database.

[1801] Step 5:

[1802] The server references the map data in the database.

[1803] Input:Location

[1804] Output: Map data reference results

[1805] How it works: The server uses Python's GeoPandas library to get the current location and related map data.

[1806] Step 6:

[1807] The server calculates the optimal route from the current location to the destination.

[1808] Input: Location information of starting point and destination

[1809] Output: Optimal route information (JSON format)

[1810] Specific operation: Calculates the optimal route taking into account distance, time, and traffic conditions using the GeoPandas library. The calculation results are saved in JSON format.

[1811] Step 7:

[1812] The server converts and generates map data based on Street View images into a horizontal perspective.

[1813] Input: Optimal route information

[1814] Output: Horizontal perspective map data (JPEG format)

[1815] Specific operation: Uses the Google Street View API to obtain image data of the area, converts it to a horizontal perspective, and generates an image in JPEG format.

[1816] Step 8:

[1817] The server transmits the generated map data to the terminal.

[1818] Input: Horizontal perspective map data (JPEG format)

[1819] Output: Sending data to the terminal

[1820] Specific operation: The server sends the generated map data to the terminal using the HTTPS protocol.

[1821] Step 9:

[1822] The terminal caches the received map data and displays the map data on the display.

[1823] Input: Horizontal perspective map data (JPEG format)

[1824] Output: Display

[1825] Specific operation: The device stores the received map data in a local cache and uses OpenGL to draw the map on the display.

[1826] Step 10:

[1827] The user manipulates the map to navigate.

[1828] Input: Touch, voice commands

[1829] Output: Updated navigation route

[1830] How it works: Users can zoom in and out and pan the map by pinching, pinching out, and swiping. They can also specify a destination using voice commands. The device will request new data from the server as needed and update the display.

[1831] Step 11:

[1832] The user initiates the capture mode.

[1833] Specific operation: When the user presses the "Photo mode" button in the app, the device activates the camera and GPS.

[1834] Step 12:

[1835] The device activates the camera and GPS to record video data and location information.

[1836] Input: Local image, latitude and longitude location information

[1837] Output: Encoded video data (H.264 format), location information

[1838] Specific operation: Local video captured by the camera is encoded in H.264 format and simultaneously recorded along with GPS information (latitude and longitude).

[1839] Step 13:

[1840] The device transmits the recorded data to the server in real time.

[1841] Input: Encoded video data, location information

[1842] Output: Send data to the server

[1843] Specific operation: The device uses WebSocket communication to send encoded video data and GPS location information to the server in real time.

[1844] Step 14:

[1845] The server analyzes the data and checks its quality.

[1846] Input: Video data, location information

[1847] Output: Quality analysis results

[1848] How it works: The server uses the OpenCV library to analyze the video for blur and resolution, checks the quality, and if it meets the criteria, proceeds to the next processing step.

[1849] Step 15:

[1850] The server stores the data that meets the criteria in a database.

[1851] Input: Quality checked data

[1852] Output: Save to database

[1853] Specific operation: Quality-confirmed data is stored in cloud storage such as AWS S3.

[1854] Step 16:

[1855] The server evaluates the quality and quantity of the geographic information provided and calculates the reward.

[1856] Input: Quality checked data

[1857] Output: Reward points

[1858] Specific operation: The server evaluates the quality and coverage of the video data and calculates reward points.

[1859] Step 17:

[1860] The server credits the reward points to the user's account.

[1861] Input: Reward Points

[1862] Output: Points added to user account

[1863] Specific operation: The server calculates reward points and adds them to the user's account, which is managed in a MySQL database.

[1864] Step 18:

[1865] The terminal notifies the user that the reward has been granted.

[1866] Input: Reward point award notification

[1867] Output: Push notification

[1868] Specific operation: The device uses a push notification to notify the user that the reward has been added.

[1869] (Application example 1)

[1870] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1871] Navigation systems for autonomous vehicles are required to reflect the latest road conditions in real time and ensure the safety of driving routes. Conventional navigation systems mainly provide route guidance based on static map information, making it difficult to immediately reflect information on road changes and obstacles. In addition, there is no adequate reward system for the vehicle's ability to recognize and record its surrounding environment, and there is a lack of means to encourage user participation. A new system is needed to solve these issues.

[1872] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1873] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route, means for generating horizontal viewpoint map data, means for controlling the onboard system of the autonomous vehicle and providing navigation information, means for photographing road conditions and transmitting the obtained data to the server in real time, and means for updating the map database based on the photographed data and location information. This allows the autonomous vehicle to always receive route guidance that reflects the latest road conditions, enabling safer operation. In addition, users are rewarded for providing photographed data, which has the effect of increasing their motivation to participate.

[1874] "Means for obtaining user location information" refers to a function that obtains the current location from the user's device or in-vehicle system using technology such as GPS.

[1875] "Means for calculating the optimal route" refers to a function in which a server or computer system uses an algorithm to calculate the shortest or safest route to a destination based on location information.

[1876] The "means for generating horizontal perspective map data" is a function that creates map data that displays the route from the current location to the destination from a horizontal perspective using street view images, etc., based on the received location information.

[1877] "Means for displaying map data" refers to a function that displays the generated map data on a display or monitor, providing it visually to the user.

[1878] "Means for users to record geographic information in photography mode" refers to a function that allows users to use a device or in-vehicle camera to take videos or photographs of their surroundings and simultaneously record their location information.

[1879] "Means for receiving recorded geographic information and checking the quality" refers to a function that sends the recorded location information along with the captured images and videos to a server and evaluates and checks the quality of the data.

[1880] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a database and preserving it for later use.

[1881] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points according to the quantity and quality of geographic information provided by the user and awards the results to the user's account.

[1882] "Means for controlling the on-board systems of autonomous vehicles and providing navigation information" refers to a function that operates the autonomous driving system in the vehicle and provides guidance and route information to the destination in real time.

[1883] "Means for photographing road conditions and transmitting the obtained data to a server in real time" refers to a function that uses the vehicle's camera to photograph road and surrounding conditions and instantly uploads the data to a server.

[1884] The "means for updating the map database based on the photographing data and location information" is a function that analyzes the received photographing data and location information and updates the map database with the latest information.

[1885] The present invention will be described in detail below with reference to an embodiment thereof. This system is applied to a navigation system for an autonomous vehicle, and operates in the following manner.

[1886] First, the server obtains the user's current location information. Specifically, it obtains location information in real time from the vehicle's GPS unit and sends it to the server. The server then calculates the optimal route to the destination based on the received location information. The software used for this is a map database and a route calculation algorithm.

[1887] Next, the server generates map data for a horizontal perspective based on the calculated optimal route. This map data is created by converting real-world landscape images, such as street views, into a horizontal perspective, making it easy for users to understand intuitively. This map data is then sent to the onboard display of the autonomous vehicle.

[1888] The in-vehicle display displays the received map data and provides real-time navigation information, allowing the autonomous driving system to safely navigate the vehicle based on the most up-to-date route information.

[1889] Furthermore, the vehicle's onboard camera and GPS work together to record road conditions in real time. This allows the vehicle to send video data and location information captured while driving to a server. The server analyzes the received data and checks its quality. Data that meets the criteria is stored to update the map database within the system. The updated map database is then immediately reflected in other autonomous vehicles.

[1890] Users will receive reward points calculated based on the quantity and quality of the data they provide, which will be added to their account, giving them an incentive to actively provide road condition data.

[1891] Examples of hardware and software used

[1892] Hardware:

[1893] GPS unit: Get your current location.

[1894] Camera: Captures road conditions.

[1895] Display: Displays a horizontal perspective map.

[1896] software:

[1897] GPS module: Obtains current location.

[1898] Camera module: Implements photography functionality.

[1899] Display module: displays map data and shows routes.

[1900] Server connection module: Implements communication with the server.

[1901] Reward System Module: Manages reward points.

[1902] Specific prompt examples

[1903] Prompt statement:

[1904] Please send the longitude and latitude data obtained from the GPS unit to the server in real time, calculate the optimal route on the server side, generate and return horizontal perspective map data, and also send road condition data captured by the in-vehicle camera to the server.

[1905] This will enable autonomous vehicles to receive route guidance based on the latest road conditions at all times, improving safety. In addition, a reward system for users will encourage active data provision.

[1906] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1907] Step 1:

[1908] The device obtains current location information from the GPS unit and sends that data to the server in real time. The input is location information (longitude and latitude) from the GPS, and the output is the transmission of location information to the server. This location information data is sent in the format "The longitude of the current location is xxx, and the latitude is yyy."

[1909] Step 2:

[1910] The server calculates the optimal route based on the received location information. The input is the location information received from the device, and the output is map data for the optimal route. The server uses a map database and a route calculation algorithm to derive the optimal route. For example, it calculates the shortest distance between the departure point and destination, and creates horizontal perspective map data by referencing Street View data.

[1911] Step 3:

[1912] The server generates horizontal perspective map data and sends it to the terminal. The input is the route calculation results and map data from the server, and the output is the transmission of map data to the terminal. The map data is sent as image data and displayed on the on-board display of the autonomous vehicle.

[1913] Step 4:

[1914] The terminal displays the received map data on the display and instructs the route to the automated driving system. The input is the map data received from the server, and the output is the display on the display and route instructions to the automated driving system. The map displayed on the display has a horizontal perspective, allowing the user to intuitively understand it.

[1915] Step 5:

[1916] The device uses a camera to capture road conditions and sends the data along with GPS information to a server in real time. The input is the video data captured by the camera and location information from GPS, and the output is the data sent to the server. For example, the data is sent in the format "Longitude of the shooting location: xxx, Latitude: yyy, Video data: zzz."

[1917] Step 6:

[1918] The server analyzes the received video data and location information and checks its quality. The input is the video data and location information sent from the device, and the output is the quality check result. The server checks the clarity, accuracy, and consistency of the data with the location information, and only stores data that meets the criteria in the database.

[1919] Step 7:

[1920] The server updates the map database based on the quality-checked data. The input is quality-checked video data and location information, and the output is an updated map database. For example, new road or building information is added, and this information is immediately reflected in other autonomous vehicles.

[1921] Step 8:

[1922] The server calculates reward points for the user based on the quantity and quality of the geographic information data provided and grants them to the user's account. The input is the provided geographic information data and its quality evaluation, and the output is the calculation result and grant of reward points. The reward points are notified to the user's application interface.

[1923] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1924] MODE FOR CARRYING OUT THE INVENTION

[1925] The following describes in detail the mode for carrying out the present invention. The program processing of the present system will be explained in natural language, with specific examples also included.

[1926] 1. Obtaining user location information

[1927] When a user launches the app, the device uses its GPS to obtain its current location, which is then sent to the server in the form of latitude and longitude.

[1928] 2. Operation of the Emotion Engine

[1929] The device uses a camera to capture the user's facial expressions in real time and analyzes them with an emotion engine. Based on the analysis results, the device identifies the user's emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[1930] 3. Generating a horizontal viewpoint map

[1931] The server calculates the optimal route by referencing map data in the database based on the received location information and the emotion recognition results from the emotion engine. This calculation also takes into account the user's emotional state. Based on the calculated route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[1932] 4. Displaying and operating horizontal perspective maps

[1933] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the server in response to user operations and updates the display.

[1934] 5. Starting the shooting mode and saving data

[1935] When a user starts the shooting mode, the device activates the camera and GPS and records the captured image data and location information. The recorded data is sent to the server in real time. The server analyzes the received data and checks its quality. Data that meets the criteria is stored in the system's database. The stored data is used to update map data.

[1936] 6. Providing Rewards

[1937] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode, and the reward points are credited to the user's account, and the terminal notifies the user that the reward has been credited.

[1938] Specific examples

[1939] Below are some specific examples of how this system can be used.

[1940] 1. User A launches the app and confirms that their current location is Chiyoda Ward, Tokyo. The server calculates the shortest route to Shibuya Ward, Tokyo, generates horizontal perspective map data, and sends it to the device. The device then uses the displayed map to guide User A and navigate to their destination. Along the way, the device analyzes User A's facial expressions, and if it detects an anxious state, it adjusts the navigation tone, such as by softening it.

[1941] 2. User B uses the photo mode to walk around the tourist spots. The camera and GPS are used to record the video data and location information of the tourist spots. The server receives the captured data, checks the quality, and stores it in a database. The map data is updated based on the saved data. When other users use this area at a later date, they can use the latest map data. The server awards reward points to User B according to the contribution of the captured data.

[1942] These features allow the present invention to provide an intuitive and safe navigation experience for users, flexibly respond to the user's emotional state, and realize real-time updates of map data.

[1943] The processing flow will be explained below.

[1944] Specific flow of program processing

[1945] Obtaining user location information

[1946] Step 1:

[1947] The user launches the app.

[1948] Specific behavior:

[1949] A user taps an app icon on their smartphone, launching the application.

[1950] Step 2:

[1951] The device activates the GPS function and obtains the current location information.

[1952] Specific behavior:

[1953] The device activates the GPS module and measures latitude and longitude data.

[1954] Step 3:

[1955] The terminal transmits the acquired location information to the server.

[1956] Specific behavior:

[1957] The acquired location information is converted into JSON format and sent to the server using an HTTP POST request.

[1958] Emotion Engine Operation

[1959] Step 1:

[1960] The device activates the camera and captures the user's facial expression.

[1961] Specific behavior:

[1962] Activate the camera module and fix the camera in a position where you want to recognize your face.

[1963] Step 2:

[1964] The device analyzes the captured facial expression data using an emotion engine.

[1965] Specific behavior:

[1966] Facial expression data is processed by image analysis algorithms to identify emotional states (e.g., happiness, surprise, anger, anxiety).

[1967] Step 3:

[1968] The terminal transmits the recognized emotional state to the server.

[1969] Specific behavior:

[1970] The emotional state is converted into JSON format as text data and sent to the server using an HTTP POST request.

[1971] Generating horizontal perspective maps

[1972] Step 1:

[1973] The server refers to a database based on the received location information and emotion data and calculates the optimal route.

[1974] Specific behavior:

[1975] The server retrieves road and destination information from the database and applies a shortest route calculation algorithm (e.g., Dijkstra's algorithm), taking into account the flexibility of the route depending on the emotion data.

[1976] Step 2:

[1977] The server generates horizontal perspective map data based on the calculated route.

[1978] Specific behavior:

[1979] The server calls the Street View API and performs panoramic image processing to convert the acquired image data into a horizontal perspective.

[1980] Step 3:

[1981] The server transmits the generated map data to the terminal.

[1982] Specific behavior:

[1983] The map data converted to a horizontal perspective is packaged in a digital format and sent to the terminal as an HTTP response.

[1984] Displaying and operating horizontal perspective maps

[1985] Step 1:

[1986] The terminal stores the received map data in a cache.

[1987] Specific behavior:

[1988] Map data is temporarily stored in the device's storage in preparation for display processing.

[1989] Step 2:

[1990] The terminal displays the map data on the display.

[1991] Specific behavior:

[1992] It displays map data as a panoramic view on the device display and provides a user interface (UI).

[1993] Step 3:

[1994] Users manipulate the map using their fingers or voice commands.

[1995] Specific behavior:

[1996] Users can change viewpoints by swiping or tapping, and give navigation instructions by voice.

[1997] Step 4:

[1998] The terminal acquires new map data from the server in response to a user operation.

[1999] Specific behavior:

[2000] If new viewpoint data is required based on user operations, an HTTP GET request is sent to the server to obtain additional map data.

[2001] Starting the shooting mode and saving data

[2002] Step 1:

[2003] The user initiates the capture mode.

[2004] Specific behavior:

[2005] The user taps the "Shooting Mode" button within the app.

[2006] Step 2:

[2007] The device will activate the camera and GPS and begin recording location information and video data.

[2008] Specific behavior:

[2009] The device launches the camera app and simultaneously uses GPS signals to obtain location data and link the video and location information.

[2010] Step 3:

[2011] The terminal transmits the recorded data to the server in real time.

[2012] Specific behavior:

[2013] It uses a communication protocol (e.g., WebSocket) to stream video data and location information to a server in real time.

[2014] Step 4:

[2015] The server analyzes the received data and checks the quality criteria.

[2016] Specific behavior:

[2017] The server uses a dedicated algorithm to analyze the video data's resolution, clarity, blur, etc., and evaluates whether it meets the standards.

[2018] Step 5:

[2019] The server stores data that meets the criteria in the system.

[2020] Specific behavior:

[2021] Data that meets the criteria is permanently stored in a database and used to update map data in the future.

[2022] Providing rewards

[2023] Step 1:

[2024] The server calculates points based on the amount and quality of data provided by the user.

[2025] Specific behavior:

[2026] It uses a reward system based on the quantity and quality of data provided and applies an algorithm to calculate points.

[2027] Step 2:

[2028] The server adds the calculated points to the user's account.

[2029] Specific behavior:

[2030] Update the user's account information and add the calculated points.

[2031] Step 3:

[2032] The server notifies the user that the reward has been granted.

[2033] Specific behavior:

[2034] The user will be notified by push notification or email that the reward has been granted.

[2035] In this way, the present invention not only provides an intuitive and safe navigation experience for users, real-time updates of map data, but also provides flexible services that respond to the user's emotional state.

[2036] Example 2

[2037] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2038] Current navigation systems only provide routes based on the user's location information, and lack the flexibility to adapt to the user's emotional state or real-time changes in the environment. While geographic information is updated periodically, it lacks real-time functionality and is difficult to reflect the latest geographic information. While there are systems that update map data based on geographic information provided by users, the specific rewards for such contributions are unclear. Furthermore, there is little feedback based on the user's usage rhythm, and improvements in user satisfaction are needed.

[2039] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2040] In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, and means for generating horizontal viewpoint map data based on the route, thereby making it possible to provide an optimal route according to the user's location information.

[2041] Furthermore, the terminal includes a means for analyzing the user's facial expression and identifying the user's emotional state, a means for the user to record geographic information in a photographing mode, a means for receiving the recorded geographic information and checking its quality, a means for storing the checked geographic information, and a means for calculating and awarding a reward for providing the geographic information. This allows navigation based on the user's emotional state to be provided, and the latest geographic information provided by the user is updated in real time in the system. Furthermore, by clarifying the reward for users' contributions, it is expected that the user's motivation will increase and the accuracy of the entire system will improve.

[2042] The "means for acquiring user location information" is a function by which the electronic device acquires latitude and longitude data indicating the user's current location.

[2043] The "means for calculating a route" is a function for calculating the optimal travel route from the departure point to the destination based on the acquired location information.

[2044] The "means for generating horizontal viewpoint map data" is a function for converting map data based on calculated route information so that the data can be displayed at a horizontal viewpoint that is easy for the user to view.

[2045] The "means for displaying map data" is a function for displaying the generated horizontal viewpoint map data on the display screen of the electronic device.

[2046] "Means for analyzing the user's facial expressions and identifying the emotional state" refers to a function in which an electronic device captures and analyzes the user's facial expressions to identify the current emotional state (e.g., joy, surprise, anger, anxiety, etc.).

[2047] The "means for recording geographic information in a photographing mode" is a function that records geographic information using the camera and the location information acquisition function when a user selects a specified photographing mode.

[2048] The "means for receiving recorded geographic information and checking the quality" is a function in which the server receives the geographic information acquired in the shooting mode and checks the quality (resolution, accuracy, etc.) of the information.

[2049] The "means for storing the confirmed geographic information" is a function for storing the geographic information whose quality has been confirmed in a recording device such as a database.

[2050] The "means for calculating and awarding rewards for providing geographic information" is a function that calculates reward points based on the quantity and quality of geographic information provided by the user and awards them to the user's account.

[2051] A "generative AI model" is an algorithm or network model used to generate or analyze data based on artificial intelligence techniques.

[2052] A "prompt" is an instruction or question entered into a generative AI model to make it perform a specific task.

[2053] The present invention is a system that provides optimal navigation information based on a user's location information and emotional state, and updates geographic information in real time based on that information. The system includes a terminal carried by the user, a server that calculates location information and provides routes, and an emotion engine that analyzes the user's emotional state.

[2054] System Configuration

[2055] The system consists of the following main components:

[2056] Device: A mobile device used by a user, such as a smartphone or tablet. The device is equipped with a GPS function, a camera, a display, a touch panel, and a voice input function.

[2057] Server: A cloud computing environment or data center for large-scale data processing and analysis, including databases.

[2058] Emotion Engine: A software module that analyzes the user's facial expressions and determines their emotional state in real time. It incorporates specific emotion recognition algorithms.

[2059] Processing Details

[2060] 1. Obtaining user location information

[2061] When a user launches the app, the device uses its GPS function to obtain current location information in the form of latitude and longitude, which is then sent by the device to the server.

[2062] Example: When a user taps an app to launch it, the device automatically activates the GPS module and obtains the current location.

[2063] 2. Operation of the Emotion Engine

[2064] The device's camera captures the user's facial expressions and sends the video data to the emotion engine for analysis. The emotion engine uses a generative AI model to identify the user's emotional state in real time and sends the information to a server.

[2065] Example: Video data is analyzed through an emotion recognition algorithm (e.g., a convolutional neural network) to determine emotional states such as "joy" or "anxiety."

[2066] 3. Generating a horizontal viewpoint map

[2067] The server calculates the optimal route based on the received location information and emotion recognition results, referring to map data in the database. This calculation also takes into account the user's emotional state. Based on the calculated route information, the server generates horizontal perspective map data and sends it to the device.

[2068] Example: The server calculates a route using Dijkstra's algorithm, and if the user's emotional state is "anxious," it prioritizes a safer route.

[2069] 4. Displaying and operating horizontal perspective maps

[2070] The device displays the received map data on the display and allows the user to operate the map using touch operations or voice commands. If new map data is required for the operation, the device retrieves the additional data from the server.

[2071] For example, the user drags the map around or gives a voice command such as "next intersection."

[2072] 5. Starting the shooting mode and saving data

[2073] When a user activates the camera mode, the device's camera and GPS are activated, recording video data and location information, which are then sent to the server in real time. The server analyzes the received data, checks its quality, and stores data that meets the criteria in a database.

[2074] Example: A user takes a video of a tourist spot and sends it to the server along with its location information.

[2075] 6. Providing Rewards

[2076] The server calculates a reward for the user according to the quantity and quality of the geographic information provided in the shooting mode and grants the reward points to the user's account. The terminal notifies the user that the reward has been granted.

[2077] For example: Points are calculated and added to your account based on the resolution and accuracy of the data you capture.

[2078] Examples and prompts

[2079] 1. Example 1

[2080] User A confirms that his current location is Chiyoda-ku, Tokyo, and launches the app. The server calculates the shortest route to Shibuya-ku, Tokyo, and sends it to the device. The device then guides User A using the displayed map and changes the navigation tone according to his emotional state.

[2081] 2. Example 2

[2082] User B uses the camera mode to visit tourist spots and record video data and location information. The server receives the data, checks the quality, and stores it in the database. User B is awarded reward points according to his / her contribution.

[2083] Prompt Sentence Examples

[2084] 1. "How can this system determine my location and generate the best route to navigate me?"

[2085] 2. "Please explain in detail how the system analyzes the user's facial expressions and reflects them in the navigation."

[2086] As a result, the present invention provides users with an intuitive and safe navigation experience, and is capable of flexibly responding to the user's emotional state. Furthermore, by real-time updating of map data and clarifying the reward system for user contributions, we expect to improve system accuracy and user satisfaction.

[2087] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2088] Step 1:

[2089] The user launches the app.

[2090] Input: User action (launching the app)

[2091] Output: Application startup status

[2092] Specific action: The user taps the app icon on the smartphone's home screen.

[2093] Step 2:

[2094] The device uses the GPS function to obtain the current location information and sends it to the server.

[2095] Input: User location request

[2096] Data processing: The device's GPS module obtains the current latitude and longitude.

[2097] Output: Latitude and longitude data is sent to the server

[2098] Specific operation: The GPS module is activated, receives signals from satellites, obtains location information, and sends the obtained data to the server via an HTTP request.

[2099] Step 3:

[2100] The device uses a camera to capture the user's facial expressions in real time and sends them to the emotion engine for analysis.

[2101] Input: Video data of the user's facial expressions

[2102] Data Computation: Emotion engine uses generative AI models to recognize emotional states (e.g., joy, surprise, anger, anxiety)

[2103] Output: Emotion recognition results are sent to the server

[2104] How it works: The front camera captures a picture of the user's face and sends the video data to the emotion engine. The emotion recognition algorithm analyzes the data to determine the user's emotional state and sends the result to the server via an HTTP request.

[2105] Step 4:

[2106] Based on the location information and emotion recognition results received by the server, the optimal route is calculated by referencing map data in the database, and horizontal perspective map data is generated.

[2107] Input: Latitude and longitude location, emotion recognition results

[2108] Data processing: The server calculates the route using Dijkstra's algorithm and converts the Street View image data into a horizontal perspective.

[2109] Output: Horizontal perspective map data is generated and sent to the device.

[2110] How it works: The server references an existing map database and uses Dijkstra's algorithm to calculate the optimal route based on location and emotion data. The result is then used to convert the Street View image into a horizontal perspective map, which is then packaged and sent to the device.

[2111] Step 5:

[2112] The device displays the received map data on the display, and the user can operate the map using touch operations or voice commands. If necessary, new map data is obtained from the server and the display is updated.

[2113] Input: Horizontal perspective map data

[2114] Output: Map displayed in the user interface, map data update requests according to user actions

[2115] What it does: It uses the GPU to draw map data and display it on the screen. The user can then manipulate the map using touch or voice commands, requesting new map data from the server as needed.

[2116] Step 6:

[2117] When a user starts the shooting mode, the device activates the camera and GPS, records the video data and location information, and sends it to the server, where it analyzes the data, checks its quality, and stores it in a database.

[2118] Input: User's action to activate the shooting mode, video data, location information

[2119] Output: High-quality geographic information stored in the cloud, reward points added to user accounts

[2120] Specific operation: The device's camera and GPS are activated, and the user taps the record button to capture video data. At the same time, location information is recorded, the data is compressed, and sent to the server. The server checks the quality of the data, and data that meets the standards is stored in the database.

[2121] Step 7:

[2122] The server calculates and awards a reward to the user according to the amount and quality of the geographic information provided in the shooting mode, and the terminal notifies the user that the reward has been awarded.

[2123] Input: Amount of geographic information provided, quality check results

[2124] Output: Notification that reward points have been added to the user's account.

[2125] Specific operation: The server calculates the volume and quality score of geographic information, and calculates the user's reward points based on the calculated score. The calculated points are recorded in the database, and the device notifies the user via push notification.

[2126] The above are the specific processing steps of the system, which enable the system to provide users with an intuitive and emotionally sensitive navigation experience and update the latest geographic information in real time.

[2127] (Application example 2)

[2128] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2129] Conventional navigation systems do not provide route guidance that takes into account the user's emotional state, which can lead to anxiety and stress. Furthermore, even systems that offer reward systems for providing geographic information do not provide real-time updates of map data or adjust navigation based on emotions, making it difficult to improve the user experience. There is a need to resolve these issues and provide more personalized and flexible route guidance.

[2130] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for calculating an optimal route based on the location information, means for generating horizontal viewpoint map data based on the route, means for displaying the map data, means for the user to record geographic information in shooting mode, means for receiving the recorded geographic information and checking the quality, means for saving the checked geographic information, means for calculating and awarding a reward for providing the geographic information, means for analyzing the user's emotional state using an emotion engine, means for automatically adjusting the tone of the navigation based on the emotional state, and means for recalculating an optimal route taking the emotional state into consideration. This enables flexible and personalized navigation according to the user's emotional state.

[2131] "User" means a person who uses the system.

[2132] "Location information" is latitude and longitude data that indicates the user's current location.

[2133] A "route" is the optimal travel route from a user's starting point to their destination.

[2134] "Horizontal viewpoint map data" refers to map display data viewed from the same height as the user's eye level.

[2135] "Geographic information" is data related to a specific geographic location, such as location information or video data.

[2136] "Rewards" means points or other forms of compensation calculated and awarded for geographic information provided by a user.

[2137] The "emotion engine" is a technology that uses cameras and sensors to analyze a user's facial expressions and identify their emotional state.

[2138] "Navigation tone" refers to the tone and expressions used in the voice and display when providing route guidance.

[2139] A "server" is a computer system that receives data from users and processes and stores various information.

[2140] DETAILED DESCRIPTION OF THE INVENTION The present invention provides a navigation system based on the emotional state of a user.

[2141] 1. Obtaining user location information

[2142] The device uses a GPS module to obtain the user's current location, which is obtained in the form of latitude and longitude and sent to the server.

[2143] 2. Operation of the Emotion Engine

[2144] The device uses a camera to capture the user's facial expressions in real time, which are then analyzed by an emotion engine, using software such as EmotionRecognizer to identify the user's emotional state, and the analysis results are sent to a server.

[2145] 3. Generating a horizontal viewpoint map

[2146] The server calculates the optimal route based on the received location information and emotion recognition results, referencing map data in its database. This calculation also takes into account the user's emotional state. Based on the optimal route, the server generates map data displayed in a horizontal perspective. This map data is converted to a horizontal perspective using Street View images. The generated map data is then sent to the device.

[2147] 4. Displaying and operating horizontal perspective maps

[2148] The device caches the received map data and displays it on the display. The displayed map is presented in a horizontal perspective to allow users to operate it intuitively. Users can operate the map using touch operations or voice commands to navigate to their destination. The device retrieves new map data as needed from the s...

Claims

1. A means for acquiring user location information; means for calculating an optimal route based on the location information; means for generating horizontal viewpoint map data based on the route; means for displaying the map data; a means for a user to record geographic information in a photographic mode; means for receiving and verifying the quality of the recorded geographic information; means for storing the identified geographic information; A means for calculating and awarding a reward for providing the geographic information. A system including:

2. The system of claim 1 , wherein the horizontal perspective map data is generated based on a real-world scene.

3. The system of claim 1 , wherein the data recorded in the capture mode includes video data and location information.

4. The system of claim 1 , wherein the reward is credited to the user's account as points.

Images