System

Abstract

To provide information based on the evaluation of driving technique, the grasp of road conditions, and the interest of individual users.SOLUTION: An analysis unit configured to transmit moving image information and position information to a server and extract information such as a road surface condition, a road width, and a traffic sign from the moving image information; A system comprising: a database management means for updating; a selection means for selecting shop information that a user is likely to be interested in; a sentence generation means for generating information selected by the selection means and a result of analysis in a short sentence format; a transmission means for transmitting information generated by the sentence generation means to a user terminal; an evaluation means for evaluating driving skill based on information obtained by the analysis means and calculating a driving risk; and a display means for displaying a result calculated by the evaluation means to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional driving assistance systems have had problems in that they do not adequately evaluate driving skills in detail, grasp road conditions in real time, or provide information based on the interests of individual users. As a result, driving safety and comfort are not sufficiently improved, and it is difficult to meet the needs of drivers. The present invention aims to solve these problems and simultaneously improve driving safety and enjoyment. [Means for solving the problem]

[0005] The present invention provides a system equipped with a camera means for acquiring video information, a location information acquisition means for acquiring location information, and a transmission means for transmitting this information to a server. The server has an analysis means for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs. The extracted information is registered and updated in a database and used by a selection means for selecting store information that is likely to be of interest to the user. The information and analysis results from the selection means are generated in short sentence format by a sentence generation means and transmitted to a user terminal. The system also includes an evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means, and this evaluation result is also transmitted to the user terminal. In this way, the present invention provides drivers with safe and useful information in real time, proposing a means for improving driving skills and achieving safe driving.

[0006] "Camera means" refers to a device for capturing images of road conditions in real time, and includes a video camera or a smartphone camera function.

[0007] "Location information acquisition means" refers to a device or system for acquiring the user's current location in digital form, and includes a GPS sensor and the like.

[0008] The "transmitting means" refers to a device and system for transmitting data obtained from the camera means and the location information obtaining means to the server.

[0009] The "analysis means" refers to devices and algorithms for analyzing the acquired video information and extracting information such as road conditions and traffic signs.

[0010] The "database management means" refers to a device and system for storing data generated by the analysis means and updating it as necessary.

[0011] The "selection means" refers to a device and algorithm for selecting information and stores that are likely to be of interest to the user based on location information, past search history, etc.

[0012] The "sentence generation means" refers to a device or system that generates text to be displayed to the user in short sentence format based on the analysis results and selected information.

[0013] The "evaluation means" refers to a device and algorithm for quantifying driving skills and calculating driving risks based on acquired video information and location information.

[0014] The "display means" refers to a device or system for displaying the generated text and evaluation results on a user terminal. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The system of the present invention is mainly composed of a terminal, a server, and a user. The programs and processes of this system will be specifically described below.

[0037] 1. Data Acquisition and Transmission

[0038] Device:

[0039] The terminal first acquires video information of the road using the camera means. This video information is captured in real time. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[0040] Examples:

[0041] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[0042] 2. Video analysis and database update

[0043] server:

[0044] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[0045] Examples:

[0046] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[0047] 3. Selection of Interest Information

[0048] server:

[0049] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[0050] Examples:

[0051] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[0052] 4. Short sentence generation and sending

[0053] server:

[0054] The selected information and the analysis results are generated in the form of short sentences using a sentence generation means, and after the short sentences are generated, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0055] Examples:

[0056] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[0057] 5. Display of Information

[0058] Device:

[0059] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[0060] Examples:

[0061] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and users can follow that information to drive.

[0062] 6. Driving skill evaluation and risk calculation

[0063] server:

[0064] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[0065] Examples:

[0066] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[0067] 7. Display of evaluation results

[0068] Device:

[0069] The device displays the results of the driving skill evaluation and risk assessment to the user, allowing the user to objectively understand their own driving skills.

[0070] Examples:

[0071] The device screen will display a rating of "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[0072] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Device:

[0076] The device activates the camera means and captures road images in real time through the windshield of the vehicle. The camera means acquires high-resolution images and temporarily stores them as video files. At the same time, the device uses the location information acquisition means to acquire current location information from the GPS sensor.

[0077] Step 2:

[0078] Device:

[0079] The device combines the captured video file and the acquired location information into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmits this data packet to the server using a transmission means.

[0080] Step 3:

[0081] server:

[0082] The server receives the data packets sent from the device, stores them in storage, and separates and extracts the video information and location information.

[0083] Step 4:

[0084] server:

[0085] The server processes the video information using analytical means, specifically, image recognition algorithms to analyze and extract information such as road surface conditions (e.g., wear, potholes), road width, and traffic signs (e.g., speed limits, caution signs).

[0086] Step 5:

[0087] server:

[0088] The information analyzed and extracted by the analysis means is registered and updated in an existing database by the database management means. The database stores road conditions and related information corresponding to specific location information.

[0089] Step 6:

[0090] server:

[0091] The server uses a selection method to select information about stores and facilities that the user is likely to be interested in based on the analyzed location information and past search history. The selected information is also reflected in the navigation system.

[0092] Step 7:

[0093] server:

[0094] The selected information and analysis results are converted into user-friendly short-form text using a text generation means, which is optimized for intuitive understanding while the user is driving.

[0095] Step 8:

[0096] server:

[0097] The text generated in the short sentence format is again packed into a data packet and transmitted from the server to the user terminal using the transmission means.

[0098] Step 9:

[0099] Device:

[0100] The terminal extracts text information from the received data packets and provides it to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[0101] Step 10:

[0102] server:

[0103] The server then uses the analysis means to evaluate the user's driving skills based on the acquired video and location information. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[0104] Step 11:

[0105] server:

[0106] The driving skill value and driving risk assessment calculated by the assessment means are transmitted from the server to the terminal as information to be provided to the user.

[0107] Step 12:

[0108] Device:

[0109] The device provides the user with the results of the driving skill evaluation and driving risk using a display, allowing the user to visually understand their own driving skills and improve their driving style as necessary.

[0110] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In modern transportation systems, drivers have difficulty obtaining real-time road and traffic information. Furthermore, there is a lack of efficient means to objectively evaluate their own driving skills and collect and analyze data that can be used to promote safe driving. This poses a challenge in improving driver safety and providing efficient driving assistance.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: a photographing means for acquiring video information; a positioning means for acquiring location information; a communication means for transmitting information obtained from the photographing means and the positioning means to an information processing device; an information analysis means for analyzing the video information and extracting information such as traffic conditions, road width, and signs; a data management means for registering and updating the information obtained by the information analysis means in a data structure; an information selection means for selecting location information that is likely to be of interest to a user based on the location information; an information generation means for generating the information selected by the information selection means and the analysis results in a short sentence format; a communication means for transmitting the information generated by the information generation means to a user terminal; an evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the information analysis means; a communication means for transmitting the results calculated by the evaluation means to the user terminal; and a display means for displaying information to a user using the communication means. This makes it possible to provide useful information to drivers in real time and support the improvement of driving skills and safe driving.

[0116] text

[0117] "Photographing means" refers to a device or equipment for acquiring video information.

[0118] "Positioning means" refers to a device or equipment for acquiring location information.

[0119] A "communication means" is a device or mechanism for transmitting acquired information to another device or system.

[0120] "Information analysis means" refers to equipment or mechanisms for analyzing acquired video information and extracting useful information.

[0121] A "data management means" is a device or mechanism for registering and updating acquired and analyzed information in a data structure.

[0122] An "information selection tool" is a device or mechanism for selecting information that may be of interest to a user based on location information or other data.

[0123] "Information generation means" refers to a device or mechanism for generating selected information or analytical results in short form.

[0124] "Assessment tools" are devices or mechanisms for assessing driving skills and calculating driving risk.

[0125] "Display means" refers to a device or equipment that provides acquired information, analysis results, etc. to users.

[0126] MODE FOR CARRYING OUT THE INVENTION

[0127] The system of the present invention comprises an imaging means, a positioning means, a communication means, an information analysis means, a data management means, an information selection means, an information generation means, an evaluation means, and a display means. A specific implementation method of the system using each means will be described below.

[0128] Photography and positioning methods

[0129] Device:

[0130] The device first acquires video information of the road using a camera as a means of capturing images. This camera can be a smartphone or an in-vehicle camera. At the same time, it acquires location information using a GPS sensor as a means of positioning. This makes it possible to collect video information and its location information in real time.

[0131] Examples:

[0132] While driving, the user points the smartphone camera at the windshield, which captures the road ahead. At the same time, the GPS sensor acquires the current location information and sends this data to a server in real time.

[0133] communication means

[0134] Device:

[0135] The acquired video information and location information are sent to a server as data packets via a communication method such as Wi-Fi or mobile data communication.

[0136] Information analysis means

[0137] server:

[0138] The server analyzes the received video and location information using an image analysis library such as OpenCV. The analysis extracts information such as road surface conditions, traffic signs, and road width.

[0139] Examples:

[0140] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only traffic light at the next intersection."

[0141] Data Management Measures

[0142] server:

[0143] The extracted information is stored in a database using a data management tool. The database system can be MySQL or MongoDB. The database is updated as needed.

[0144] Information selection means

[0145] server:

[0146] The server uses information selection methods based on the user's location information and past search history to select location information that the user is likely to be interested in. A machine learning model is used for the selection.

[0147] Examples:

[0148] If the user has frequently searched for cafes in the past, the server will select information such as "There is a popular cafe 500 meters ahead."

[0149] Information generation means

[0150] server:

[0151] The selected information and analysis results are generated in short sentence format using an information generation means (e.g., GPT-3).

[0152] Example prompt sentence:

[0153] "If you turn right at the next intersection, there's a bakery I recommend."

[0154] Communication and display means

[0155] Server and Device:

[0156] The generated short sentence information is transmitted to the user terminal using the communication means and is displayed on the display means of the terminal.

[0157] Examples:

[0158] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[0159] Evaluation methods

[0160] server:

[0161] The server uses a proprietary algorithm to evaluate the driving technique and calculate the driving risk based on the acquired video and location information.

[0162] Examples:

[0163] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[0164] Display means

[0165] Device:

[0166] The evaluation results are presented to the user using the display means of the terminal, allowing the user to objectively grasp their own driving skills.

[0167] Examples:

[0168] The evaluation results, "Driving score: 85 (good), risk: low," are displayed on the device screen, allowing users to visually check their own driving skills.

[0169] In this way, the system of the present invention can provide useful information to the driver in real time, and support the improvement of driving skills and safe driving.

[0170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0171] System program processing flow

[0172] Step 1: Data Acquisition

[0173] Device:

[0174] The device uses a camera to capture video information of the road in real time, and also uses a GPS sensor to acquire location information, allowing it to collect video data and location information.

[0175] Input: None (activated by user action)

[0176] Processing: Record video with the camera and obtain location information with the GPS sensor

[0177] Output: Video data, location data

[0178] Specific behavior:

[0179] The user points the smartphone camera at the windshield and launches the app, which then captures video of the road and uses the GPS sensor to obtain the user's current location.

[0180] Step 2: Send data

[0181] Device:

[0182] The acquired video data and location data are sent to the server as data packets using wireless communication technology (Wi-Fi or mobile data communication).

[0183] Input: Video data, location data

[0184] Processing: Generate a data packet and send it to the server

[0185] Output: Data packet (including video data and location data)

[0186] Specific behavior:

[0187] The device combines the video data and GPS location information into a single data packet and sends it to the server.

[0188] Step 3: Data reception and analysis

[0189] server:

[0190] The server receives the data packets sent from the terminal, separates the video data and the location information data, and then analyzes the video data using information analysis means to extract information on road conditions and traffic signs as analysis results.

[0191] Input: Data packet

[0192] Processing: Breaking down the data packets and analyzing the video and location data

[0193] Output: Analysis results (road surface conditions, traffic sign information, etc.)

[0194] Specific behavior:

[0195] The video data received by the server is analyzed using an image analysis library such as OpenCV, and information such as the "road wear condition" and "location of traffic signs" is extracted.

[0196] Step 4: Update the database

[0197] server:

[0198] The server stores the parsed information in a database and updates the data as needed. Data management is performed using a database management system (e.g., MySQL or MongoDB).

[0199] Input: Analysis results

[0200] Processing: Register and update the analysis results in the database.

[0201] Output: Updated database

[0202] Specific behavior:

[0203] The server stores the analysis results in a database and updates the data when new information is added or existing information is changed.

[0204] Step 5: Selecting Interest Information

[0205] server:

[0206] The server analyzes location information and past search history and uses machine learning models to select location information that is likely to be of interest to the user.

[0207] Input: Location data, past search history

[0208] Processing: Analyzing information using machine learning models

[0209] Output: Selected interest information

[0210] Specific behavior:

[0211] The server selects information about nearby popular cafes based on the user's past cafe search history.

[0212] Step 6: Short sentence generation

[0213] server:

[0214] The selected information of interest and analysis results are generated in short sentence format using an AI model (e.g., GPT-3).

[0215] Input: Interest information, analysis results

[0216] Processing: Generate short sentences using AI models

[0217] Output: Short message

[0218] Specific behavior:

[0219] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery."

[0220] Step 7: Send a short message

[0221] Server and Device:

[0222] The generated short sentence is sent to the user terminal using a communication means and displayed on the terminal.

[0223] Input: A short message

[0224] Processing: Send the message as a data packet

[0225] Output: Messages displayed on the terminal

[0226] Specific behavior:

[0227] Short sentences are displayed on the device screen, allowing users to obtain useful information in real time.

[0228] Step 8: Driving Skills Assessment

[0229] server:

[0230] The server uses a proprietary algorithm to evaluate driving skills based on video data and location data, and calculates driving risk.

[0231] Input: Video data, location data

[0232] Processing: Evaluation and risk calculation using proprietary algorithms

[0233] Output: Evaluation results, risk calculation results

[0234] Specific behavior:

[0235] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[0236] Step 9: View the evaluation results

[0237] Device:

[0238] The device displays the evaluation results and risk calculation results on the screen, allowing the user to objectively understand their own driving skills.

[0239] Input: Evaluation results, risk calculation results

[0240] Action: Display on screen

[0241] Output: Rating information displayed on the terminal

[0242] Specific behavior:

[0243] The device screen will display "Driving score: 85 (good), Risk: low," allowing the user to check their driving skills.

[0244] (Application example 1)

[0245] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0246] Conventional driver assistance and navigation systems do not adequately assess changes in road conditions and driving skills in real time, and therefore do not provide sufficient information useful to drivers. In particular, autonomous vehicles require accurate and up-to-date road information and rapid calculation of driving risks, but current technology lacks a comprehensive system to solve this problem.

[0247] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0248] In this invention, the server includes an analysis means, a data storage means, and a selection means, which enable real-time analysis of road conditions. Also, by including a means for evaluating driving behavior and providing real-time updates to the vehicle's navigation system, it is possible to provide the driver with quick and useful information.

[0249] "Photographing device means" refers to a device used to capture video information.

[0250] A "positioning means" is a device or system used to obtain location information.

[0251] The "communication means" is a means for transmitting information obtained from the photographing device means and the positioning means to the data processing device.

[0252] "Analysis means" refers to technology for analyzing video information and extracting information such as road surface conditions, road width, and traffic signs.

[0253] The "data storage means" is a system for registering and updating information obtained by the analysis means in a database.

[0254] The "selection means" is a technique for selecting facility information that is likely to interest the user based on location information.

[0255] The "text generation means" is a technique for generating the information and analysis results selected by the selection means in a short sentence format.

[0256] A "display means" is a device or system for displaying information to a user using a communication means.

[0257] The "evaluation means" is a technique for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means.

[0258] An "automobile navigation system" is a system that provides real-time updates and presents appropriate driving routes and information to drivers.

[0259] MODE FOR CARRYING OUT THE INVENTION

[0260] The present invention is a system that includes real-time analysis of road conditions and evaluation of driving skills, and detailed embodiments of the system are described below.

[0261] System Configuration

[0262] Terminal

[0263] The device used is a smartphone or tablet and includes the following elements:

[0264] Camera means: Video information of the road is acquired using a smartphone or tablet camera.

[0265] Positioning method: Location information is obtained using the built-in GPS module.

[0266] Communication means: A communication means for transmitting the acquired video information and location information to the data processing device.

[0267] server

[0268] On the server, the following elements are included:

[0269] Analysis method: Technology to analyze received video information and extract information such as road conditions and traffic signs. This uses a trained artificial intelligence model (for example, using the OpenCV library).

[0270] Data storage means: A system for registering and updating information obtained by the analysis means in a database.

[0271] Selection method: Technology for selecting facility information (e.g., cafes, bakeries, etc.) that may be of interest to the user based on location information.

[0272] Text generation means: A technology that generates the information selected by the selection means and the analysis results in short sentences. The short sentences are generated using a generative AI model.

[0273] Evaluation method: A technology that evaluates driving skills and calculates driving risks based on information obtained by analytical methods.

[0274] User

[0275] Users interact with the system using smartphones and tablets.

[0276] Display Means: A device or system for displaying information to a user using a communication means.

[0277] Automotive navigation system: Refers to a system that provides real-time updates and presents appropriate driving routes and information to the driver.

[0278] Program processing explanation

[0279] When the system is operating, video information captured by the terminal's camera means and GPS location information are sent to the server via the communication means. Based on this information, the server's analysis means analyzes road conditions and traffic signs, and the data storage means stores the results in a database. At the same time, the server uses the selection means to select facility information likely to be of interest to the user, and generates the information in short sentence format using the text generation means.

[0280] The analysis results and selected information are sent back to the terminal via the communication means and displayed to the user via the display means. This makes it possible to provide useful information to the driver in real time. In addition, the driver's driving skills can be evaluated using the evaluation means, and driving risks can be calculated.

[0281] Specific examples

[0282] For example, if a user frequently searches for cafes, the information displayed might look like this:

[0283] "Turn right at the next intersection and you'll find a popular cafe 50 meters away."

[0284] "The road is in good condition at this point, so please be careful."

[0285] Prompt Sentence Examples

[0286] "Users often search for cafes. Please generate a short sentence like, 'Turn right at the next intersection and there's a popular cafe 50 meters away.'"

[0287] In this way, the system of the present invention analyzes road conditions in real time through data communication between the terminal and the server, and provides useful information to drivers. In addition, by using a generative AI model, it is possible to provide highly accurate information for autonomous vehicles.

[0288] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0289] Step 1:

[0290] The terminal acquires video information of roads using a camera means. At the same time, it also acquires GPS location information using a positioning means. The input data obtained from this is video captured in real time and location information of the location where the video was taken. This video information and location information are sent to a server via a communication means.

[0291] Step 2:

[0292] The server receives the video information and location information sent from the device. It passes this received data to the analysis means. The analysis means analyzes the video information using a trained artificial intelligence model (for example, using the OpenCV library) and extracts information such as road conditions and traffic signs. The data is processed based on the analysis results and organized for storage in a database in a clear format.

[0293] Step 3:

[0294] The server's data storage means registers the organized analysis results in a database and updates the information as necessary. This process ensures that the latest road conditions and traffic sign data is always available for use in other processes.

[0295] Step 4:

[0296] The server uses a selection means to analyze the user's location information and interest information (e.g., past search history) and select facility information that is likely to be of interest to the user. The selected information becomes useful information tailored to the user's current location and driving route. This process uses the user's location information as input data and outputs a list of facilities of interest.

[0297] Step 5:

[0298] The server uses a text generation means to generate the information selected by the selection means and the analysis results in short sentence format. Utilizing a generative AI model, it creates a short sentence such as, "Turn right at the next intersection and you'll find a popular cafe 50 meters away." The input data for this generation process is the selected facility information and the analysis results, and the output data is short sentence information to be provided to the user.

[0299] Step 6:

[0300] The server then uses the communication means to send the generated short message back to the terminal. The terminal then displays the received message to the user via a display. For example, a message such as "Turn right at the next intersection and you'll find a popular cafe 50 meters away" may be displayed on the smartphone screen.

[0301] Step 7:

[0302] At the same time, the evaluation means in the server evaluates the user's driving skills and calculates the driving risk based on the information obtained by the analysis means. For the evaluation, the number of sudden brakings, the frequency of speeding, etc. are quantified, and the driving risk is calculated based on this. In this way, the user's driving behavior is evaluated as a number, and further output as a driving risk.

[0303] Step 8:

[0304] The server transmits the driving skill evaluation results and driving risk calculated by the evaluation means to the user terminal via communication means. The terminal provides these evaluation results and risk information to the user via display means. For example, the smartphone screen may display "Driving score: 85 (good), Risk: low."

[0305] Through these steps, the system analyzes road conditions in real time, provides useful information to drivers, and evaluates driving skills and calculates risks.

[0306] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0307] The system of the present invention is mainly composed of a terminal, a server, and a user, and also has an emotion engine built in. The program of this system and its processing will be explained in detail below.

[0308] 1. Data Acquisition and Transmission

[0309] Device:

[0310] The terminal activates the camera means and captures road images in real time through the windshield of the car. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[0311] Examples:

[0312] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[0313] 2. Video analysis and database update

[0314] server:

[0315] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[0316] Examples:

[0317] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[0318] 3. Selection of Interest Information

[0319] server:

[0320] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[0321] Examples:

[0322] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[0323] 4. Short sentence generation and sending

[0324] server:

[0325] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0326] Examples:

[0327] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[0328] 5. Display of Information

[0329] Device:

[0330] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[0331] Examples:

[0332] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[0333] 6. Driving skill evaluation and risk calculation

[0334] server:

[0335] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[0336] Examples:

[0337] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[0338] 7. Display of evaluation results

[0339] Device:

[0340] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[0341] Examples:

[0342] The device screen displays the rating "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[0343] 8. User Emotion Recognition

[0344] Device:

[0345] The device captures a user's face in real time using a built-in or external camera. The emotion engine analyzes facial features and classifies the user's emotion. The classified emotion information is then sent to the server as a data packet.

[0346] Examples:

[0347] While the user is driving, a camera captures their face, and AI analyzes their emotions, such as "happiness," "anger," and "sadness," in real time. The results are then sent to a server.

[0348] 9. Emotion-based information regulation

[0349] server:

[0350] The server adjusts the content and display method of the information it provides based on the emotional information received from the emotion engine. For example, if the user is feeling stressed, it will prioritize providing information about places where they can relax or change their mood.

[0351] Examples:

[0352] If the server determines through emotion analysis that the user is tired, it provides information such as, "There is a parking area up ahead where you can rest."

[0353] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[0354] The processing flow will be explained below.

[0355] Step 1:

[0356] Device:

[0357] The camera means is activated and captures road images in real time through the windshield of the vehicle. The images are temporarily saved as video files. At the same time, GPS data is acquired using the location information acquisition means.

[0358] Step 2:

[0359] Device:

[0360] The captured video file and the acquired location information are combined into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmitted to a server using a transmission means.

[0361] Step 3:

[0362] server:

[0363] Receives data packets sent from the device. The received data packets are stored in storage, and video information and location information are separated and extracted.

[0364] Step 4:

[0365] server:

[0366] Video information is processed using analytical means, specifically, image recognition algorithms are used to analyze and extract information such as road surface conditions (wear, potholes), road width, and traffic signs (speed limits, caution signs).

[0367] Step 5:

[0368] server:

[0369] The analysis results obtained by the analysis means are registered and updated in an existing database by the database management means, and road conditions and related information corresponding to specific location information are stored.

[0370] Step 6:

[0371] server:

[0372] Based on the analyzed location information and past search history, the server uses a selection method to select information about stores and facilities that the user is likely to be interested in. The selected information is also reflected in the navigation system.

[0373] Step 7:

[0374] server:

[0375] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0376] Step 8:

[0377] Device:

[0378] The text information is extracted from the received data packet and provided to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[0379] Step 9:

[0380] server:

[0381] The system evaluates the user's driving skills based on video and location information acquired using analytical tools. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[0382] Step 10:

[0383] server:

[0384] The driving skill value and driving risk assessment calculated by the assessment means are again compiled into a data packet as information to be provided to the user, and are transmitted to the user terminal using the transmission means.

[0385] Step 11:

[0386] Device:

[0387] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[0388] Step 12:

[0389] Device:

[0390] The device then captures a real-time image of the user's face using a built-in or external camera. The emotion engine analyzes the facial features and classifies the user's emotion. This classified emotion information is then sent to the server as a data packet.

[0391] Step 13:

[0392] server:

[0393] The server analyzes the emotion information received from the emotion engine and determines the emotional state the user is in. For example, if the user is feeling stressed or tired, that emotion information is sent to the server.

[0394] Step 14:

[0395] server:

[0396] The emotion engine recognizes the user's emotions and adjusts the content and display method of the information provided accordingly. For example, if it determines that the user is feeling stressed, it will prioritize providing information about places to relax and rest areas.

[0397] Step 15:

[0398] Device:

[0399] The terminal again receives the personalized information sent from the server and provides it to the user using the display means, allowing the user to obtain optimal information in real time according to their current emotional state.

[0400] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[0401] Example 2

[0402] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0403] Conventional driving assistance systems have difficulty obtaining real-time road conditions and location information and providing specific navigation based on that information. They also lack sufficient capabilities for evaluating driving skills and calculating driving risks, and do not provide information that takes into account the user's emotional state. To solve these issues, a more advanced and integrated system is needed.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0405] In this invention, the server includes: a camera for capturing video information; a location information sensor for capturing location information; a communication device for transmitting information obtained from the camera and location information sensor to a central processing unit; an analysis device for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs; a database management device for registering and updating the information obtained by the analysis device in a database; a selection device for selecting location information likely to be of interest to a user based on the location information; a sentence generation device for generating the information selected by the selection device and the analysis results in short sentences; a communication device for transmitting the information generated by the sentence generation device to a user terminal; an evaluation device for evaluating driving skills and calculating driving risks based on the information obtained by the analysis device; a communication device for transmitting the results calculated by the evaluation device to a user terminal; a display device for displaying information to a user using the communication device; a processing device including a camera for recognizing user emotions and an emotion analysis engine; and an information adjustment device for appropriately adjusting information based on the emotions recognized by the emotion analysis engine. This enables real-time information provision, driving skill evaluation, and personalized information provision according to user emotions.

[0406] The "capture means" is a device for capturing images of the road from inside the vehicle in real time, and is usually a device including a camera.

[0407] The "location information sensor means" is a device for acquiring the current location of the vehicle, and is usually a device including a GPS sensor.

[0408] The "communication means" is an interface for transmitting acquired data to a destination, and is a device that includes a wired or wireless communication protocol.

[0409] The "analysis device" is a device that analyzes the received video data and extracts information such as road surface conditions, road width, and traffic signs.

[0410] The "database management device" is software or hardware for registering and updating information obtained by the analysis device in a database.

[0411] The "selection device" is a device for selecting location information that is likely to interest a user based on the user's location information and past history.

[0412] A "sentence generation device" is software or hardware for generating selected information and analysis results in short sentence format.

[0413] The "evaluation device" is a device for evaluating driving skills based on video data and location information and calculating driving risks.

[0414] A "display device" is a device for displaying information on a user terminal, typically including a display or screen.

[0415] An "emotion analysis engine" is software or hardware that analyzes the features of a user's face and classifies the user's emotions.

[0416] An "information adjustment device" is a device that adjusts the content and display method of information provided based on emotions recognized by an emotion analysis engine.

[0417] The system of the present invention is implemented by a combination of a terminal, a server, and a user. A specific embodiment of the system will be described in detail below.

[0418] Device behavior

[0419] The terminal is installed inside the vehicle. It performs the following operations using a camera and GPS sensor. The camera captures real-time images of the road through the vehicle's windshield, and the GPS sensor acquires the vehicle's current location. This allows real-time video and location information to be obtained.

[0420] Examples:

[0421] When a user launches the application in a vehicle, the device's camera starts capturing images of the road ahead, and the GPS sensor continuously acquires the vehicle's current location. This video and location information is then formed into a data packet and sent to the server.

[0422] Server Operation

[0423] The server acts as a central processing unit and receives data sent from the terminals, performs multiple analyses and data processing. Specifically, it performs the following operations using an analysis device, a database management device, a selection device, a sentence generation device, an evaluation device, and a sentiment analysis engine.

[0424] Video Analysis

[0425] The server's analysis device analyzes the received video data and extracts information such as road surface conditions, road width, traffic signs, etc. This information is stored in a database using a database management device and updated as necessary.

[0426] Examples:

[0427] By analyzing the video data, information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" is extracted and registered in a database.

[0428] Selection of information of interest

[0429] The selection device of the server selects information (for example, stores and facilities) that is likely to interest the user based on the user's location information and past history.

[0430] Examples:

[0431] If the user has a history of searching for cafes in the past, the server will select and provide information such as "There is a popular cafe 500 meters ahead."

[0432] Short sentence generation

[0433] The server's sentence generation device generates the selected information and analysis results in an easy-to-understand short sentence format and transmits them again to the terminal as a data packet.

[0434] Examples:

[0435] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's terminal.

[0436] Assessment and Risk Calculation

[0437] The server evaluates driving skills and calculates driving risk. The evaluation device calculates a driving score based on the number of sudden braking and speeding, and performs risk assessment.

[0438] Examples:

[0439] The evaluation result "Driving score 85 (good), low risk" is generated and sent to the user's terminal.

[0440] Emotion recognition and information regulation

[0441] The device's camera and emotion analysis engine are used to analyze the user's emotions in real time. The emotion analysis engine classifies the emotions and sends them to the server. The server then adjusts the content and display method of the information based on this emotion information.

[0442] Examples:

[0443] If the emotion analysis engine determines that the user is feeling tired, the server will provide information such as, "There is a parking area up ahead where you can take a rest."

[0444] Adjusted short sentence generation

[0445] The adjusted information is then converted back into short sentences and sent to the device, enabling personalized information to be provided according to the user's emotional state.

[0446] Prompt Sentence Examples

[0447] Below is an example of a prompt that the server might input to the generative AI model:

[0448] "If you turn right at the next intersection, there's a cafe I recommend."

[0449] "Driving score: 85 (good), risk: low"

[0450] "There's a parking area up ahead where you can rest."

[0451] In this way, the entire system works together to provide real-time information and driving evaluation, as well as personalized information based on emotions.

[0452] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0453] Processing Steps

[0454] Step 1: Data Acquisition and Transmission

[0455] Device:

[0456] The device activates the camera and GPS sensor to capture real-time road images through the vehicle's windshield. At the same time, the GPS sensor acquires the device's current location. Based on this, the device assembles the image data and location information into a data packet and sends it to the server.

[0457] Specific behavior:

[0458] 1. The device will activate the built-in camera.

[0459] 2. The camera captures the road scenery ahead.

[0460] 3. The device activates its built-in GPS sensor and acquires location information.

[0461] 4. The video data and location information are packaged in a packet format and sent to the server.

[0462] Input: Camera footage, GPS location information

[0463] Output: Data packet (video data + location information)

[0464] Step 2: Video analysis

[0465] server:

[0466] The server processes the received data packets (video data and location information) using an analysis device to extract necessary information such as road surface conditions, traffic signs, road width, etc. The analysis results are then registered in a database and updated as necessary.

[0467] Specific behavior:

[0468] 1. The server receives the data packet.

[0469] 2. The analysis device analyzes the video data and extracts data such as road wear conditions and traffic signs.

[0470] 3. The extracted data is registered and updated in the database by the database management device.

[0471] Input: Data packet (video data + location information)

[0472] Output: Extracted information (road conditions, traffic signs, etc.)

[0473] Step 3: Selecting information of interest

[0474] server:

[0475] The server uses the user's current location and past search history to select information that is likely to be of interest to the user.

[0476] Specific behavior:

[0477] 1. The server references the location information and the user's search history.

[0478] 2. The selection device applies an algorithm to select stores and facility information that may be of interest.

[0479] Input: User's current location, past search history

[0480] Output: Information that may be of interest to the user

[0481] Step 4: Create and send a short message

[0482] server:

[0483] The server generates text in short sentence format based on the selected information and the analysis results, and transmits it to the user terminal as a data packet.

[0484] Specific behavior:

[0485] 1. The server retrieves the selected information.

[0486] 2. A sentence generator generates text in short sentence format.

[0487] 3. The generated short sentence is sent to the user terminal as a data packet.

[0488] Input: Selection information, analysis results

[0489] Output: Short text

[0490] Step 5: Viewing information

[0491] Device:

[0492] The terminal displays the received data to the user.

[0493] Specific behavior:

[0494] 1. The terminal receives a data packet sent from the server.

[0495] 2. Decode the received information and display it on the screen.

[0496] Input: Data packet (short text format)

[0497] Output: The displayed information (display)

[0498] Step 6: Driving skill assessment and risk calculation

[0499] server:

[0500] The server uses video data and location information to evaluate driving skills and calculate driving risks.

[0501] Specific behavior:

[0502] 1. The server obtains the video data and location information.

[0503] 2. The evaluation device evaluates driving skills based on the number of sudden braking and speeding.

[0504] 3. The calculated driving risk is quantified and an evaluation result is generated.

[0505] Input: Video data, location information

[0506] Output: Driving skill evaluation results, driving risk

[0507] Step 7: View the evaluation results

[0508] Device:

[0509] The terminal displays the driving skill evaluation results sent from the server.

[0510] Specific behavior:

[0511] 1. The terminal receives the evaluation result.

[0512] 2. The received evaluation results are displayed on the screen.

[0513] Input: Evaluation result, driving risk

[0514] Output: Displayed evaluation results (display)

[0515] Step 8: Recognizing User Emotions

[0516] Device:

[0517] The device uses a built-in or external camera to capture a picture of the user's face and classifies the emotion using an emotion analysis engine.

[0518] Specific behavior:

[0519] 1. The device activates the camera and takes a picture of the user's face.

[0520] 2. The emotion analysis engine analyzes facial feature points and classifies emotions.

[0521] 3. The classified emotion information is sent to the server as a data packet.

[0522] Input: A photographed face image

[0523] Output: Emotion information (data packet)

[0524] Step 9: Emotionally regulated information

[0525] server:

[0526] The server adjusts the information provided based on the emotion information and provides the most suitable information to the user.

[0527] Specific behavior:

[0528] 1. The server receives emotion information.

[0529] 2. The sentiment analysis engine adjusts the information based on the analysis results.

[0530] 3. Convert the adjusted information into short-text format and send it to the terminal.

[0531] Input: Emotion information

[0532] Output: Adjusted information (sent to terminal in short form)

[0533] (Application example 2)

[0534] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0535] Modern self-driving vehicles require more accurate information, such as analysis of road conditions and traffic signs. It is also important to analyze passengers' emotional states in real time to provide a comfortable driving service. In this environment, a comprehensive system is needed to provide optimal information to users, evaluate driving skills, and calculate driving risks.

[0536] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a camera means for acquiring video information, a location information acquisition means for acquiring location information, and an emotion analysis means including an emotion engine for analyzing the user's emotional state. This makes it possible to analyze information on road conditions and traffic signs in real time, evaluate driving skills and calculate driving risks, and provide information according to the passenger's emotional state.

[0537] "Moving image information" is video data captured by a camera means.

[0538] The "camera means" is a device for photographing an object and acquiring video information.

[0539] "Location information" refers to coordinate data of the current location obtained using GPS or other technologies.

[0540] "Location information acquisition means" refers to a device or technology for acquiring location information.

[0541] "Transmission means" refers to a technique or device for transmitting acquired data to an external device such as a server.

[0542] "Analysis means" refers to techniques or devices for analyzing acquired data and extracting useful information.

[0543] "Road surface conditions" is information that indicates the condition of the road surface.

[0544] "Road width" is information indicating the width of the road.

[0545] "Traffic signs" refers to various signs installed on roads.

[0546] "Database management means" refers to the technology and devices that register, manage, and update acquired information in a database.

[0547] A "selection means" is a technique or device for selecting information based on specific conditions.

[0548] "Text generation means" refers to a technique or device for generating selected information or analysis results in text format.

[0549] "Evaluation means" refers to technology or equipment that evaluates driving skills based on data and calculates driving risks.

[0550] An "emotion engine" is a technology or device for analyzing and classifying a user's emotional state.

[0551] "Emotion analysis means" refers to a technique or device that uses an emotion engine to analyze the user's emotional state.

[0552] "Display means" refers to technology or devices for visually presenting acquired information and analysis results to the user.

[0553] A "user terminal" is a terminal device that allows a user to obtain information.

[0554] In this invention, by using a system having the following configuration, it is possible to provide information on road conditions and traffic signs in real time, as well as information based on an evaluation of driving skills and the emotional state of passengers.

[0555] 1. Data acquisition and transmission

[0556] Device:

[0557] The terminal is equipped with a camera means and a location information acquisition means. The camera means of the terminal acquires video information in real time, and the location information acquisition means acquires accurate location information using GPS. These data are transmitted to the server using a transmission means.

[0558] For example, a camera mounted on an autonomous vehicle captures the road ahead, and a GPS device acquires the vehicle's current location, which is then transmitted to a server via wireless communication.

[0559] 2.Video analysis and database updates

[0560] server:

[0561] The server analyzes the received video information and location information using an analysis means. The analysis means uses a trained generative AI model to extract information such as road surface conditions, road width, and traffic signs. The extracted information is registered in a database by a database management means and updated as necessary.

[0562] As a specific example, the server's analysis results include information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" that is registered in the database.

[0563] 3. Selection and provision of information of interest

[0564] server:

[0565] Based on the location information and past data, the server selects store and facility information that the user is likely to be interested in. This information is converted into short text using a sentence generation means and sent to the terminal.

[0566] As a specific example, if the user has frequently searched for cafes in the past, the server will generate information such as "There is a popular cafe 500 meters ahead" and send it to the terminal.

[0567] 4. Evaluation of driving skills and calculation of driving risks

[0568] server:

[0569] The server has an evaluation means for evaluating driving skills based on the acquired video information and location information, and calculating driving risk. The evaluation results are sent to the user's terminal.

[0570] As a specific example, the server measures the number of sudden brakings and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[0571] 5. Sentiment analysis and information provision adjustment

[0572] Device:

[0573] The device captures the user's face in real time using a built-in or external camera, analyzes the user's emotional state using an emotion engine, and transmits the analyzed emotional information to a server as a data packet.

[0574] server:

[0575] The server adjusts the information it provides based on the emotional information received from the emotion analysis means. For example, if the user is feeling stressed, it will prioritize providing information about places where the user can relax or change their mood.

[0576] As a specific example, if the server determines that the user is tired, it provides information such as, "There is a parking area up ahead where you can take a rest."

[0577] Prompt Sentence Examples

[0578] An example of a prompt to input to a generative AI model is as follows:

[0579] Prompt: Design an application that uses in-car cameras and GPS information to analyze road conditions and traffic signs in real time, suggest optimal routes, and use an emotion engine to assess passengers' emotional state and provide a comfortable ride.

[0580] In this way, the invention provides useful information to drivers and passengers in real time, helping to improve driving skills and ensure comfortable driving.

[0581] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0582] Step 1:

[0583] Terminal: The terminal's camera means is activated and video information is acquired in real time. GPS data is acquired using the location information acquisition means. These data (video information and location information) are sent to the server using the transmission means. The input is the video image from the camera and the location information from the GPS device, and the output is a data packet sent to the server. Through this process, real-time road conditions and location information are acquired.

[0584] Step 2:

[0585] Server: The server receives the video information and location information sent from the device. After receiving the video information, it uses analysis means to analyze it and extract important information such as road surface conditions, road width, and traffic signs. The input is the video information and location information sent from the device, and the output is the extracted road conditions and traffic signs. Specific operations include video analysis using a machine learning model.

[0586] Step 3:

[0587] Server: The analysis results are registered in a database by the database management means and updated as necessary. The input is the information obtained from the analysis means, and the output is an updated database. This ensures that the latest road information and traffic sign data is always maintained.

[0588] Step 4:

[0589] Server: The server selects information about stores and facilities that the user may be interested in based on their location information and past data. It uses a selection method to identify locations that the user may have chosen as their destination. The input is location information and past search history, and the output is store information that the user may be interested in. For example, if the user frequently searches for cafes, the server selects information about nearby cafes.

[0590] Step 5:

[0591] Server: The selected information and analysis results are converted into short sentences using a sentence generation tool. The input is the selected store information and analysis results, and the output is short text information. Specifically, a short sentence such as "There is a popular cafe 500 meters ahead" is generated.

[0592] Step 6:

[0593] Server: Sends short-form information to the user terminal using a transmission means. The input is short-form text information, and the output is the information sent to the user terminal. This allows the user to obtain the information they need in real time.

[0594] Step 7:

[0595] Terminal: The terminal provides the received information to the user using a display means. The input is short text information sent from the server, and the output is information displayed on the terminal's display. Specifically, it displays "Turn right at the next intersection and you'll find a recommended bakery."

[0596] Step 8:

[0597] Server: Using analytical means, the server evaluates the user's driving skills based on the acquired video information and location information, and calculates the driving risk. The evaluation means quantifies the driving skills and calculates the driving risk. The input is video information and location information, and the output is a driving skill score and driving risk assessment. Specific examples such as "driving score 85 (good), low risk" are generated.

[0598] Step 9:

[0599] Server: Sends the evaluation results to the user's device. The input is the driving skill score and driving risk assessment, and the output is the evaluation result sent to the user's device. This allows the user to objectively understand their own driving skills.

[0600] Step 10:

[0601] Device: The device captures the user's face in real time using a built-in or external camera, analyzes facial features using an emotion engine, and classifies the user's emotion. The input is the user's facial image, and the output is emotion classification data as the analysis result.

[0602] Step 11:

[0603] Server: Adjusts the content and display method of the information provided based on the emotional information received from the emotion engine. The input is emotional classification data, and the output is the adjusted information provided. For example, if the user is feeling stressed, "information about places to relax and change your mood" will be provided preferentially.

[0604] This allows the system to provide useful information to the driver in real time, supporting improved driving skills and safe driving. In addition, by incorporating an emotion engine, it becomes possible to provide information optimized for each individual user, further enhancing driving comfort.

[0605] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0606] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0607] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0608] [Second embodiment]

[0609] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0610] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0611] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0612] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0613] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0614] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0615] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0616] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0617] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0620] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0621] The system of the present invention is mainly composed of a terminal, a server, and a user. The programs and processes of this system will be specifically described below.

[0622] 1. Data Acquisition and Transmission

[0623] Device:

[0624] The terminal first acquires video information of the road using the camera means. This video information is captured in real time. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[0625] Examples:

[0626] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[0627] 2. Video analysis and database update

[0628] server:

[0629] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[0630] Examples:

[0631] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[0632] 3. Selection of Interest Information

[0633] server:

[0634] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[0635] Examples:

[0636] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[0637] 4. Short sentence generation and sending

[0638] server:

[0639] The selected information and the analysis results are generated in the form of short sentences using a sentence generation means, and after the short sentences are generated, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0640] Examples:

[0641] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[0642] 5. Display of Information

[0643] Device:

[0644] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[0645] Examples:

[0646] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and users can follow that information to drive.

[0647] 6. Driving skill evaluation and risk calculation

[0648] server:

[0649] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[0650] Examples:

[0651] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[0652] 7. Display of evaluation results

[0653] Device:

[0654] The device displays the results of the driving skill evaluation and risk assessment to the user, allowing the user to objectively understand their own driving skills.

[0655] Examples:

[0656] The device screen will display a rating of "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[0657] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[0658] The processing flow will be explained below.

[0659] Step 1:

[0660] Device:

[0661] The device activates the camera means and captures road images in real time through the windshield of the vehicle. The camera means acquires high-resolution images and temporarily stores them as video files. At the same time, the device uses the location information acquisition means to acquire current location information from the GPS sensor.

[0662] Step 2:

[0663] Device:

[0664] The device combines the captured video file and the acquired location information into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmits this data packet to the server using a transmission means.

[0665] Step 3:

[0666] server:

[0667] The server receives the data packets sent from the device, stores them in storage, and separates and extracts the video information and location information.

[0668] Step 4:

[0669] server:

[0670] The server processes the video information using analytical means, specifically, image recognition algorithms to analyze and extract information such as road surface conditions (e.g., wear, potholes), road width, and traffic signs (e.g., speed limits, caution signs).

[0671] Step 5:

[0672] server:

[0673] The information analyzed and extracted by the analysis means is registered and updated in an existing database by the database management means. The database stores road conditions and related information corresponding to specific location information.

[0674] Step 6:

[0675] server:

[0676] The server uses a selection method to select information about stores and facilities that the user is likely to be interested in based on the analyzed location information and past search history. The selected information is also reflected in the navigation system.

[0677] Step 7:

[0678] server:

[0679] The selected information and analysis results are converted into user-friendly short-form text using a text generation means, which is optimized for intuitive understanding while the user is driving.

[0680] Step 8:

[0681] server:

[0682] The text generated in the short sentence format is again packed into a data packet and transmitted from the server to the user terminal using the transmission means.

[0683] Step 9:

[0684] Device:

[0685] The terminal extracts text information from the received data packets and provides it to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[0686] Step 10:

[0687] server:

[0688] The server then uses the analysis means to evaluate the user's driving skills based on the acquired video and location information. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[0689] Step 11:

[0690] server:

[0691] The driving skill value and driving risk assessment calculated by the assessment means are transmitted from the server to the terminal as information to be provided to the user.

[0692] Step 12:

[0693] Device:

[0694] The device provides the user with the results of the driving skill evaluation and driving risk using a display, allowing the user to visually understand their own driving skills and improve their driving style as necessary.

[0695] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[0696] Example 1

[0697] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0698] In modern transportation systems, drivers have difficulty obtaining real-time road and traffic information. Furthermore, there is a lack of efficient means to objectively evaluate their own driving skills and collect and analyze data that can be used to promote safe driving. This poses a challenge in improving driver safety and providing efficient driving assistance.

[0699] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0700] In this invention, the server includes: a photographing means for acquiring video information; a positioning means for acquiring location information; a communication means for transmitting information obtained from the photographing means and the positioning means to an information processing device; an information analysis means for analyzing the video information and extracting information such as traffic conditions, road width, and signs; a data management means for registering and updating the information obtained by the information analysis means in a data structure; an information selection means for selecting location information that is likely to be of interest to a user based on the location information; an information generation means for generating the information selected by the information selection means and the analysis results in a short sentence format; a communication means for transmitting the information generated by the information generation means to a user terminal; an evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the information analysis means; a communication means for transmitting the results calculated by the evaluation means to the user terminal; and a display means for displaying information to a user using the communication means. This makes it possible to provide useful information to drivers in real time and support the improvement of driving skills and safe driving.

[0701] text

[0702] "Photographing means" refers to a device or equipment for acquiring video information.

[0703] "Positioning means" refers to a device or equipment for acquiring location information.

[0704] A "communication means" is a device or mechanism for transmitting acquired information to another device or system.

[0705] "Information analysis means" refers to equipment or mechanisms for analyzing acquired video information and extracting useful information.

[0706] A "data management means" is a device or mechanism for registering and updating acquired and analyzed information in a data structure.

[0707] An "information selection tool" is a device or mechanism for selecting information that may be of interest to a user based on location information or other data.

[0708] "Information generation means" refers to a device or mechanism for generating selected information or analytical results in short form.

[0709] "Assessment tools" are devices or mechanisms for assessing driving skills and calculating driving risk.

[0710] "Display means" refers to a device or equipment that provides acquired information, analysis results, etc. to users.

[0711] MODE FOR CARRYING OUT THE INVENTION

[0712] The system of the present invention comprises an imaging means, a positioning means, a communication means, an information analysis means, a data management means, an information selection means, an information generation means, an evaluation means, and a display means. A specific implementation method of the system using each means will be described below.

[0713] Photography and positioning methods

[0714] Device:

[0715] The device first acquires video information of the road using a camera as a means of capturing images. This camera can be a smartphone or an in-vehicle camera. At the same time, it acquires location information using a GPS sensor as a means of positioning. This makes it possible to collect video information and its location information in real time.

[0716] Examples:

[0717] While driving, the user points the smartphone camera at the windshield, which captures the road ahead. At the same time, the GPS sensor acquires the current location information and sends this data to a server in real time.

[0718] communication means

[0719] Device:

[0720] The acquired video information and location information are sent to a server as data packets via a communication method such as Wi-Fi or mobile data communication.

[0721] Information analysis means

[0722] server:

[0723] The server analyzes the received video and location information using an image analysis library such as OpenCV. The analysis extracts information such as road surface conditions, traffic signs, and road width.

[0724] Examples:

[0725] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only traffic light at the next intersection."

[0726] Data Management Measures

[0727] server:

[0728] The extracted information is stored in a database using a data management tool. The database system can be MySQL or MongoDB. The database is updated as needed.

[0729] Information selection means

[0730] server:

[0731] The server uses information selection methods based on the user's location information and past search history to select location information that the user is likely to be interested in. A machine learning model is used for the selection.

[0732] Examples:

[0733] If the user has frequently searched for cafes in the past, the server will select information such as "There is a popular cafe 500 meters ahead."

[0734] Information generation means

[0735] server:

[0736] The selected information and analysis results are generated in short sentence format using an information generation means (e.g., GPT-3).

[0737] Example prompt sentence:

[0738] "If you turn right at the next intersection, there's a bakery I recommend."

[0739] Communication and display means

[0740] Server and Device:

[0741] The generated short sentence information is transmitted to the user terminal using the communication means and is displayed on the display means of the terminal.

[0742] Examples:

[0743] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[0744] Evaluation methods

[0745] server:

[0746] The server uses a proprietary algorithm to evaluate the driving technique and calculate the driving risk based on the acquired video and location information.

[0747] Examples:

[0748] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[0749] Display means

[0750] Device:

[0751] The evaluation results are presented to the user using the display means of the terminal, allowing the user to objectively grasp their own driving skills.

[0752] Examples:

[0753] The evaluation results, "Driving score: 85 (good), risk: low," are displayed on the device screen, allowing users to visually check their own driving skills.

[0754] In this way, the system of the present invention can provide useful information to the driver in real time, and support the improvement of driving skills and safe driving.

[0755] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0756] System program processing flow

[0757] Step 1: Data Acquisition

[0758] Device:

[0759] The device uses a camera to capture video information of the road in real time, and also uses a GPS sensor to acquire location information, allowing it to collect video data and location information.

[0760] Input: None (activated by user action)

[0761] Processing: Record video with the camera and obtain location information with the GPS sensor

[0762] Output: Video data, location data

[0763] Specific behavior:

[0764] The user points the smartphone camera at the windshield and launches the app, which then captures video of the road and uses the GPS sensor to obtain the user's current location.

[0765] Step 2: Send data

[0766] Device:

[0767] The acquired video data and location data are sent to the server as data packets using wireless communication technology (Wi-Fi or mobile data communication).

[0768] Input: Video data, location data

[0769] Processing: Generate a data packet and send it to the server

[0770] Output: Data packet (including video data and location data)

[0771] Specific behavior:

[0772] The device combines the video data and GPS location information into a single data packet and sends it to the server.

[0773] Step 3: Data reception and analysis

[0774] server:

[0775] The server receives the data packets sent from the terminal, separates the video data and the location information data, and then analyzes the video data using information analysis means to extract information on road conditions and traffic signs as analysis results.

[0776] Input: Data packet

[0777] Processing: Breaking down the data packets and analyzing the video and location data

[0778] Output: Analysis results (road surface conditions, traffic sign information, etc.)

[0779] Specific behavior:

[0780] The video data received by the server is analyzed using an image analysis library such as OpenCV, and information such as the "road wear condition" and "location of traffic signs" is extracted.

[0781] Step 4: Update the database

[0782] server:

[0783] The server stores the parsed information in a database and updates the data as needed. Data management is performed using a database management system (e.g., MySQL or MongoDB).

[0784] Input: Analysis results

[0785] Processing: Register and update the analysis results in the database.

[0786] Output: Updated database

[0787] Specific behavior:

[0788] The server stores the analysis results in a database and updates the data when new information is added or existing information is changed.

[0789] Step 5: Selecting Interest Information

[0790] server:

[0791] The server analyzes location information and past search history and uses machine learning models to select location information that is likely to be of interest to the user.

[0792] Input: Location data, past search history

[0793] Processing: Analyzing information using machine learning models

[0794] Output: Selected interest information

[0795] Specific behavior:

[0796] The server selects information about nearby popular cafes based on the user's past cafe search history.

[0797] Step 6: Short sentence generation

[0798] server:

[0799] The selected information of interest and analysis results are generated in short sentence format using an AI model (e.g., GPT-3).

[0800] Input: Interest information, analysis results

[0801] Processing: Generate short sentences using AI models

[0802] Output: Short message

[0803] Specific behavior:

[0804] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery."

[0805] Step 7: Send a short message

[0806] Server and Device:

[0807] The generated short sentence is sent to the user terminal using a communication means and displayed on the terminal.

[0808] Input: A short message

[0809] Processing: Send the message as a data packet

[0810] Output: Messages displayed on the terminal

[0811] Specific behavior:

[0812] Short sentences are displayed on the device screen, allowing users to obtain useful information in real time.

[0813] Step 8: Driving Skills Assessment

[0814] server:

[0815] The server uses a proprietary algorithm to evaluate driving skills based on video data and location data, and calculates driving risk.

[0816] Input: Video data, location data

[0817] Processing: Evaluation and risk calculation using proprietary algorithms

[0818] Output: Evaluation results, risk calculation results

[0819] Specific behavior:

[0820] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[0821] Step 9: View the evaluation results

[0822] Device:

[0823] The device displays the evaluation results and risk calculation results on the screen, allowing the user to objectively understand their own driving skills.

[0824] Input: Evaluation results, risk calculation results

[0825] Action: Display on screen

[0826] Output: Rating information displayed on the terminal

[0827] Specific behavior:

[0828] The device screen will display "Driving score: 85 (good), Risk: low," allowing the user to check their driving skills.

[0829] (Application example 1)

[0830] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0831] Conventional driver assistance and navigation systems do not adequately assess changes in road conditions and driving skills in real time, and therefore do not provide sufficient information useful to drivers. In particular, autonomous vehicles require accurate and up-to-date road information and rapid calculation of driving risks, but current technology lacks a comprehensive system to solve this problem.

[0832] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0833] In this invention, the server includes an analysis means, a data storage means, and a selection means, which enable real-time analysis of road conditions. Also, by including a means for evaluating driving behavior and providing real-time updates to the vehicle's navigation system, it is possible to provide the driver with quick and useful information.

[0834] "Photographing device means" refers to a device used to capture video information.

[0835] A "positioning means" is a device or system used to obtain location information.

[0836] The "communication means" is a means for transmitting information obtained from the photographing device means and the positioning means to the data processing device.

[0837] "Analysis means" refers to technology for analyzing video information and extracting information such as road surface conditions, road width, and traffic signs.

[0838] The "data storage means" is a system for registering and updating information obtained by the analysis means in a database.

[0839] The "selection means" is a technique for selecting facility information that is likely to interest the user based on location information.

[0840] The "text generation means" is a technique for generating the information and analysis results selected by the selection means in a short sentence format.

[0841] A "display means" is a device or system for displaying information to a user using a communication means.

[0842] The "evaluation means" is a technique for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means.

[0843] An "automobile navigation system" is a system that provides real-time updates and presents appropriate driving routes and information to drivers.

[0844] MODE FOR CARRYING OUT THE INVENTION

[0845] The present invention is a system that includes real-time analysis of road conditions and evaluation of driving skills, and detailed embodiments of the system are described below.

[0846] System Configuration

[0847] Terminal

[0848] The device used is a smartphone or tablet and includes the following elements:

[0849] Camera means: Video information of the road is acquired using a smartphone or tablet camera.

[0850] Positioning method: Location information is obtained using the built-in GPS module.

[0851] Communication means: A communication means for transmitting the acquired video information and location information to the data processing device.

[0852] server

[0853] On the server, the following elements are included:

[0854] Analysis method: Technology to analyze received video information and extract information such as road conditions and traffic signs. This uses a trained artificial intelligence model (for example, using the OpenCV library).

[0855] Data storage means: A system for registering and updating information obtained by the analysis means in a database.

[0856] Selection method: Technology for selecting facility information (e.g., cafes, bakeries, etc.) that may be of interest to the user based on location information.

[0857] Text generation means: A technology that generates the information selected by the selection means and the analysis results in short sentences. The short sentences are generated using a generative AI model.

[0858] Evaluation method: A technology that evaluates driving skills and calculates driving risks based on information obtained by analytical methods.

[0859] User

[0860] Users interact with the system using smartphones and tablets.

[0861] Display Means: A device or system for displaying information to a user using a communication means.

[0862] Automotive navigation system: Refers to a system that provides real-time updates and presents appropriate driving routes and information to the driver.

[0863] Program processing explanation

[0864] When the system is operating, video information captured by the terminal's camera means and GPS location information are sent to the server via the communication means. Based on this information, the server's analysis means analyzes road conditions and traffic signs, and the data storage means stores the results in a database. At the same time, the server uses the selection means to select facility information likely to be of interest to the user, and generates the information in short sentence format using the text generation means.

[0865] The analysis results and selected information are sent back to the terminal via the communication means and displayed to the user via the display means. This makes it possible to provide useful information to the driver in real time. In addition, the driver's driving skills can be evaluated using the evaluation means, and driving risks can be calculated.

[0866] Specific examples

[0867] For example, if a user frequently searches for cafes, the information displayed might look like this:

[0868] "Turn right at the next intersection and you'll find a popular cafe 50 meters away."

[0869] "The road is in good condition at this point, so please be careful."

[0870] Prompt Sentence Examples

[0871] "Users often search for cafes. Please generate a short sentence like, 'Turn right at the next intersection and there's a popular cafe 50 meters away.'"

[0872] In this way, the system of the present invention analyzes road conditions in real time through data communication between the terminal and the server, and provides useful information to drivers. In addition, by using a generative AI model, it is possible to provide highly accurate information for autonomous vehicles.

[0873] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0874] Step 1:

[0875] The terminal acquires video information of roads using a camera means. At the same time, it also acquires GPS location information using a positioning means. The input data obtained from this is video captured in real time and location information of the location where the video was taken. This video information and location information are sent to a server via a communication means.

[0876] Step 2:

[0877] The server receives the video information and location information sent from the device. It passes this received data to the analysis means. The analysis means analyzes the video information using a trained artificial intelligence model (for example, using the OpenCV library) and extracts information such as road conditions and traffic signs. The data is processed based on the analysis results and organized for storage in a database in a clear format.

[0878] Step 3:

[0879] The server's data storage means registers the organized analysis results in a database and updates the information as necessary. This process ensures that the latest road conditions and traffic sign data is always available for use in other processes.

[0880] Step 4:

[0881] The server uses a selection means to analyze the user's location information and interest information (e.g., past search history) and select facility information that is likely to be of interest to the user. The selected information becomes useful information tailored to the user's current location and driving route. This process uses the user's location information as input data and outputs a list of facilities of interest.

[0882] Step 5:

[0883] The server uses a text generation means to generate the information selected by the selection means and the analysis results in short sentence format. Utilizing a generative AI model, it creates a short sentence such as, "Turn right at the next intersection and you'll find a popular cafe 50 meters away." The input data for this generation process is the selected facility information and the analysis results, and the output data is short sentence information to be provided to the user.

[0884] Step 6:

[0885] The server then uses the communication means to send the generated short message back to the terminal. The terminal then displays the received message to the user via a display. For example, a message such as "Turn right at the next intersection and you'll find a popular cafe 50 meters away" may be displayed on the smartphone screen.

[0886] Step 7:

[0887] At the same time, the evaluation means in the server evaluates the user's driving skills and calculates the driving risk based on the information obtained by the analysis means. For the evaluation, the number of sudden brakings, the frequency of speeding, etc. are quantified, and the driving risk is calculated based on this. In this way, the user's driving behavior is evaluated as a number, and further output as a driving risk.

[0888] Step 8:

[0889] The server transmits the driving skill evaluation results and driving risk calculated by the evaluation means to the user terminal via communication means. The terminal provides these evaluation results and risk information to the user via display means. For example, the smartphone screen may display "Driving score: 85 (good), Risk: low."

[0890] Through these steps, the system analyzes road conditions in real time, provides useful information to drivers, and evaluates driving skills and calculates risks.

[0891] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0892] The system of the present invention is mainly composed of a terminal, a server, and a user, and also has an emotion engine built in. The program of this system and its processing will be explained in detail below.

[0893] 1. Data Acquisition and Transmission

[0894] Device:

[0895] The terminal activates the camera means and captures road images in real time through the windshield of the car. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[0896] Examples:

[0897] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[0898] 2. Video analysis and database update

[0899] server:

[0900] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[0901] Examples:

[0902] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[0903] 3. Selection of Interest Information

[0904] server:

[0905] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[0906] Examples:

[0907] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[0908] 4. Short sentence generation and sending

[0909] server:

[0910] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0911] Examples:

[0912] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[0913] 5. Display of Information

[0914] Device:

[0915] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[0916] Examples:

[0917] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[0918] 6. Driving skill evaluation and risk calculation

[0919] server:

[0920] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[0921] Examples:

[0922] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[0923] 7. Display of evaluation results

[0924] Device:

[0925] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[0926] Examples:

[0927] The device screen displays the rating "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[0928] 8. User Emotion Recognition

[0929] Device:

[0930] The device captures a user's face in real time using a built-in or external camera. The emotion engine analyzes facial features and classifies the user's emotion. The classified emotion information is then sent to the server as a data packet.

[0931] Examples:

[0932] While the user is driving, a camera captures their face, and AI analyzes their emotions, such as "happiness," "anger," and "sadness," in real time. The results are then sent to a server.

[0933] 9. Emotion-based information regulation

[0934] server:

[0935] The server adjusts the content and display method of the information it provides based on the emotional information received from the emotion engine. For example, if the user is feeling stressed, it will prioritize providing information about places where they can relax or change their mood.

[0936] Examples:

[0937] If the server determines through emotion analysis that the user is tired, it provides information such as, "There is a parking area up ahead where you can rest."

[0938] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[0939] The processing flow will be explained below.

[0940] Step 1:

[0941] Device:

[0942] The camera means is activated and captures road images in real time through the windshield of the vehicle. The images are temporarily saved as video files. At the same time, GPS data is acquired using the location information acquisition means.

[0943] Step 2:

[0944] Device:

[0945] The captured video file and the acquired location information are combined into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmitted to a server using a transmission means.

[0946] Step 3:

[0947] server:

[0948] Receives data packets sent from the device. The received data packets are stored in storage, and video information and location information are separated and extracted.

[0949] Step 4:

[0950] server:

[0951] Video information is processed using analytical means, specifically, image recognition algorithms are used to analyze and extract information such as road surface conditions (wear, potholes), road width, and traffic signs (speed limits, caution signs).

[0952] Step 5:

[0953] server:

[0954] The analysis results obtained by the analysis means are registered and updated in an existing database by the database management means, and road conditions and related information corresponding to specific location information are stored.

[0955] Step 6:

[0956] server:

[0957] Based on the analyzed location information and past search history, the server uses a selection method to select information about stores and facilities that the user is likely to be interested in. The selected information is also reflected in the navigation system.

[0958] Step 7:

[0959] server:

[0960] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[0961] Step 8:

[0962] Device:

[0963] The text information is extracted from the received data packet and provided to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[0964] Step 9:

[0965] server:

[0966] The system evaluates the user's driving skills based on video and location information acquired using analytical tools. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[0967] Step 10:

[0968] server:

[0969] The driving skill value and driving risk assessment calculated by the assessment means are again compiled into a data packet as information to be provided to the user, and are transmitted to the user terminal using the transmission means.

[0970] Step 11:

[0971] Device:

[0972] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[0973] Step 12:

[0974] Device:

[0975] The device then captures a real-time image of the user's face using a built-in or external camera. The emotion engine analyzes the facial features and classifies the user's emotion. This classified emotion information is then sent to the server as a data packet.

[0976] Step 13:

[0977] server:

[0978] The server analyzes the emotion information received from the emotion engine and determines the emotional state the user is in. For example, if the user is feeling stressed or tired, that emotion information is sent to the server.

[0979] Step 14:

[0980] server:

[0981] The emotion engine recognizes the user's emotions and adjusts the content and display method of the information provided accordingly. For example, if it determines that the user is feeling stressed, it will prioritize providing information about places to relax and rest areas.

[0982] Step 15:

[0983] Device:

[0984] The terminal again receives the personalized information sent from the server and provides it to the user using the display means, allowing the user to obtain optimal information in real time according to their current emotional state.

[0985] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[0986] Example 2

[0987] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0988] Conventional driving assistance systems have difficulty obtaining real-time road conditions and location information and providing specific navigation based on that information. They also lack sufficient capabilities for evaluating driving skills and calculating driving risks, and do not provide information that takes into account the user's emotional state. To solve these issues, a more advanced and integrated system is needed.

[0989] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0990] In this invention, the server includes: a camera for capturing video information; a location information sensor for capturing location information; a communication device for transmitting information obtained from the camera and location information sensor to a central processing unit; an analysis device for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs; a database management device for registering and updating the information obtained by the analysis device in a database; a selection device for selecting location information likely to be of interest to a user based on the location information; a sentence generation device for generating the information selected by the selection device and the analysis results in short sentences; a communication device for transmitting the information generated by the sentence generation device to a user terminal; an evaluation device for evaluating driving skills and calculating driving risks based on the information obtained by the analysis device; a communication device for transmitting the results calculated by the evaluation device to a user terminal; a display device for displaying information to a user using the communication device; a processing device including a camera for recognizing user emotions and an emotion analysis engine; and an information adjustment device for appropriately adjusting information based on the emotions recognized by the emotion analysis engine. This enables real-time information provision, driving skill evaluation, and personalized information provision according to user emotions.

[0991] The "capture means" is a device for capturing images of the road from inside the vehicle in real time, and is usually a device including a camera.

[0992] The "location information sensor means" is a device for acquiring the current location of the vehicle, and is usually a device including a GPS sensor.

[0993] The "communication means" is an interface for transmitting acquired data to a destination, and is a device that includes a wired or wireless communication protocol.

[0994] The "analysis device" is a device that analyzes the received video data and extracts information such as road surface conditions, road width, and traffic signs.

[0995] The "database management device" is software or hardware for registering and updating information obtained by the analysis device in a database.

[0996] The "selection device" is a device for selecting location information that is likely to interest a user based on the user's location information and past history.

[0997] A "sentence generation device" is software or hardware for generating selected information and analysis results in short sentence format.

[0998] The "evaluation device" is a device for evaluating driving skills based on video data and location information and calculating driving risks.

[0999] A "display device" is a device for displaying information on a user terminal, typically including a display or screen.

[1000] An "emotion analysis engine" is software or hardware that analyzes the features of a user's face and classifies the user's emotions.

[1001] An "information adjustment device" is a device that adjusts the content and display method of information provided based on emotions recognized by an emotion analysis engine.

[1002] The system of the present invention is implemented by a combination of a terminal, a server, and a user. A specific embodiment of the system will be described in detail below.

[1003] Device behavior

[1004] The terminal is installed inside the vehicle. It performs the following operations using a camera and GPS sensor. The camera captures real-time images of the road through the vehicle's windshield, and the GPS sensor acquires the vehicle's current location. This allows real-time video and location information to be obtained.

[1005] Examples:

[1006] When a user launches the application in a vehicle, the device's camera starts capturing images of the road ahead, and the GPS sensor continuously acquires the vehicle's current location. This video and location information is then formed into a data packet and sent to the server.

[1007] Server Operation

[1008] The server acts as a central processing unit and receives data sent from the terminals, performs multiple analyses and data processing. Specifically, it performs the following operations using an analysis device, a database management device, a selection device, a sentence generation device, an evaluation device, and a sentiment analysis engine.

[1009] Video Analysis

[1010] The server's analysis device analyzes the received video data and extracts information such as road surface conditions, road width, traffic signs, etc. This information is stored in a database using a database management device and updated as necessary.

[1011] Examples:

[1012] By analyzing the video data, information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" is extracted and registered in a database.

[1013] Selection of information of interest

[1014] The selection device of the server selects information (for example, stores and facilities) that is likely to interest the user based on the user's location information and past history.

[1015] Examples:

[1016] If the user has a history of searching for cafes in the past, the server will select and provide information such as "There is a popular cafe 500 meters ahead."

[1017] Short sentence generation

[1018] The server's sentence generation device generates the selected information and analysis results in an easy-to-understand short sentence format and transmits them again to the terminal as a data packet.

[1019] Examples:

[1020] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's terminal.

[1021] Assessment and Risk Calculation

[1022] The server evaluates driving skills and calculates driving risk. The evaluation device calculates a driving score based on the number of sudden braking and speeding, and performs risk assessment.

[1023] Examples:

[1024] The evaluation result "Driving score 85 (good), low risk" is generated and sent to the user's terminal.

[1025] Emotion recognition and information regulation

[1026] The device's camera and emotion analysis engine are used to analyze the user's emotions in real time. The emotion analysis engine classifies the emotions and sends them to the server. The server then adjusts the content and display method of the information based on this emotion information.

[1027] Examples:

[1028] If the emotion analysis engine determines that the user is feeling tired, the server will provide information such as, "There is a parking area up ahead where you can take a rest."

[1029] Adjusted short sentence generation

[1030] The adjusted information is then converted back into short sentences and sent to the device, enabling personalized information to be provided according to the user's emotional state.

[1031] Prompt Sentence Examples

[1032] Below is an example of a prompt that the server might input to the generative AI model:

[1033] "If you turn right at the next intersection, there's a cafe I recommend."

[1034] "Driving score: 85 (good), risk: low"

[1035] "There's a parking area up ahead where you can rest."

[1036] In this way, the entire system works together to provide real-time information and driving evaluation, as well as personalized information based on emotions.

[1037] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1038] Processing Steps

[1039] Step 1: Data Acquisition and Transmission

[1040] Device:

[1041] The device activates the camera and GPS sensor to capture real-time road images through the vehicle's windshield. At the same time, the GPS sensor acquires the device's current location. Based on this, the device assembles the image data and location information into a data packet and sends it to the server.

[1042] Specific behavior:

[1043] 1. The device will activate the built-in camera.

[1044] 2. The camera captures the road scenery ahead.

[1045] 3. The device activates its built-in GPS sensor and acquires location information.

[1046] 4. The video data and location information are packaged in a packet format and sent to the server.

[1047] Input: Camera footage, GPS location information

[1048] Output: Data packet (video data + location information)

[1049] Step 2: Video analysis

[1050] server:

[1051] The server processes the received data packets (video data and location information) using an analysis device to extract necessary information such as road surface conditions, traffic signs, road width, etc. The analysis results are then registered in a database and updated as necessary.

[1052] Specific behavior:

[1053] 1. The server receives the data packet.

[1054] 2. The analysis device analyzes the video data and extracts data such as road wear conditions and traffic signs.

[1055] 3. The extracted data is registered and updated in the database by the database management device.

[1056] Input: Data packet (video data + location information)

[1057] Output: Extracted information (road conditions, traffic signs, etc.)

[1058] Step 3: Selecting information of interest

[1059] server:

[1060] The server uses the user's current location and past search history to select information that is likely to be of interest to the user.

[1061] Specific behavior:

[1062] 1. The server references the location information and the user's search history.

[1063] 2. The selection device applies an algorithm to select stores and facility information that may be of interest.

[1064] Input: User's current location, past search history

[1065] Output: Information that may be of interest to the user

[1066] Step 4: Create and send a short message

[1067] server:

[1068] The server generates text in short sentence format based on the selected information and the analysis results, and transmits it to the user terminal as a data packet.

[1069] Specific behavior:

[1070] 1. The server retrieves the selected information.

[1071] 2. A sentence generator generates text in short sentence format.

[1072] 3. The generated short sentence is sent to the user terminal as a data packet.

[1073] Input: Selection information, analysis results

[1074] Output: Short text

[1075] Step 5: Viewing information

[1076] Device:

[1077] The terminal displays the received data to the user.

[1078] Specific behavior:

[1079] 1. The terminal receives a data packet sent from the server.

[1080] 2. Decode the received information and display it on the screen.

[1081] Input: Data packet (short text format)

[1082] Output: The displayed information (display)

[1083] Step 6: Driving skill assessment and risk calculation

[1084] server:

[1085] The server uses video data and location information to evaluate driving skills and calculate driving risks.

[1086] Specific behavior:

[1087] 1. The server obtains the video data and location information.

[1088] 2. The evaluation device evaluates driving skills based on the number of sudden braking and speeding.

[1089] 3. The calculated driving risk is quantified and an evaluation result is generated.

[1090] Input: Video data, location information

[1091] Output: Driving skill evaluation results, driving risk

[1092] Step 7: View the evaluation results

[1093] Device:

[1094] The terminal displays the driving skill evaluation results sent from the server.

[1095] Specific behavior:

[1096] 1. The terminal receives the evaluation result.

[1097] 2. The received evaluation results are displayed on the screen.

[1098] Input: Evaluation result, driving risk

[1099] Output: Displayed evaluation results (display)

[1100] Step 8: Recognizing User Emotions

[1101] Device:

[1102] The device uses a built-in or external camera to capture a picture of the user's face and classifies the emotion using an emotion analysis engine.

[1103] Specific behavior:

[1104] 1. The device activates the camera and takes a picture of the user's face.

[1105] 2. The emotion analysis engine analyzes facial feature points and classifies emotions.

[1106] 3. The classified emotion information is sent to the server as a data packet.

[1107] Input: A photographed face image

[1108] Output: Emotion information (data packet)

[1109] Step 9: Emotionally regulated information

[1110] server:

[1111] The server adjusts the information provided based on the emotion information and provides the most suitable information to the user.

[1112] Specific behavior:

[1113] 1. The server receives emotion information.

[1114] 2. The sentiment analysis engine adjusts the information based on the analysis results.

[1115] 3. Convert the adjusted information into short-text format and send it to the terminal.

[1116] Input: Emotion information

[1117] Output: Adjusted information (sent to terminal in short form)

[1118] (Application example 2)

[1119] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1120] Modern self-driving vehicles require more accurate information, such as analysis of road conditions and traffic signs. It is also important to analyze passengers' emotional states in real time to provide a comfortable driving service. In this environment, a comprehensive system is needed to provide optimal information to users, evaluate driving skills, and calculate driving risks.

[1121] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a camera means for acquiring video information, a location information acquisition means for acquiring location information, and an emotion analysis means including an emotion engine for analyzing the user's emotional state. This makes it possible to analyze information on road conditions and traffic signs in real time, evaluate driving skills and calculate driving risks, and provide information according to the passenger's emotional state.

[1122] "Moving image information" is video data captured by a camera means.

[1123] The "camera means" is a device for photographing an object and acquiring video information.

[1124] "Location information" refers to coordinate data of the current location obtained using GPS or other technologies.

[1125] "Location information acquisition means" refers to a device or technology for acquiring location information.

[1126] "Transmission means" refers to a technique or device for transmitting acquired data to an external device such as a server.

[1127] "Analysis means" refers to techniques or devices for analyzing acquired data and extracting useful information.

[1128] "Road surface conditions" is information that indicates the condition of the road surface.

[1129] "Road width" is information indicating the width of the road.

[1130] "Traffic signs" refers to various signs installed on roads.

[1131] "Database management means" refers to the technology and devices that register, manage, and update acquired information in a database.

[1132] A "selection means" is a technique or device for selecting information based on specific conditions.

[1133] "Text generation means" refers to a technique or device for generating selected information or analysis results in text format.

[1134] "Evaluation means" refers to technology or equipment that evaluates driving skills based on data and calculates driving risks.

[1135] An "emotion engine" is a technology or device for analyzing and classifying a user's emotional state.

[1136] "Emotion analysis means" refers to a technique or device that uses an emotion engine to analyze the user's emotional state.

[1137] "Display means" refers to technology or devices for visually presenting acquired information and analysis results to the user.

[1138] A "user terminal" is a terminal device that allows a user to obtain information.

[1139] In this invention, by using a system having the following configuration, it is possible to provide information on road conditions and traffic signs in real time, as well as information based on an evaluation of driving skills and the emotional state of passengers.

[1140] 1. Data acquisition and transmission

[1141] Device:

[1142] The terminal is equipped with a camera means and a location information acquisition means. The camera means of the terminal acquires video information in real time, and the location information acquisition means acquires accurate location information using GPS. These data are transmitted to the server using a transmission means.

[1143] For example, a camera mounted on an autonomous vehicle captures the road ahead, and a GPS device acquires the vehicle's current location, which is then transmitted to a server via wireless communication.

[1144] 2.Video analysis and database updates

[1145] server:

[1146] The server analyzes the received video information and location information using an analysis means. The analysis means uses a trained generative AI model to extract information such as road surface conditions, road width, and traffic signs. The extracted information is registered in a database by a database management means and updated as necessary.

[1147] As a specific example, the server's analysis results include information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" that is registered in the database.

[1148] 3. Selection and provision of information of interest

[1149] server:

[1150] Based on the location information and past data, the server selects store and facility information that the user is likely to be interested in. This information is converted into short text using a sentence generation means and sent to the terminal.

[1151] As a specific example, if the user has frequently searched for cafes in the past, the server will generate information such as "There is a popular cafe 500 meters ahead" and send it to the terminal.

[1152] 4. Evaluation of driving skills and calculation of driving risks

[1153] server:

[1154] The server has an evaluation means for evaluating driving skills based on the acquired video information and location information, and calculating driving risk. The evaluation results are sent to the user's terminal.

[1155] As a specific example, the server measures the number of sudden brakings and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[1156] 5. Sentiment analysis and information provision adjustment

[1157] Device:

[1158] The device captures the user's face in real time using a built-in or external camera, analyzes the user's emotional state using an emotion engine, and transmits the analyzed emotional information to a server as a data packet.

[1159] server:

[1160] The server adjusts the information it provides based on the emotional information received from the emotion analysis means. For example, if the user is feeling stressed, it will prioritize providing information about places where the user can relax or change their mood.

[1161] As a specific example, if the server determines that the user is tired, it provides information such as, "There is a parking area up ahead where you can take a rest."

[1162] Prompt Sentence Examples

[1163] An example of a prompt to input to a generative AI model is as follows:

[1164] Prompt: Design an application that uses in-car cameras and GPS information to analyze road conditions and traffic signs in real time, suggest optimal routes, and use an emotion engine to assess passengers' emotional state and provide a comfortable ride.

[1165] In this way, the invention provides useful information to drivers and passengers in real time, helping to improve driving skills and ensure comfortable driving.

[1166] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1167] Step 1:

[1168] Terminal: The terminal's camera means is activated and video information is acquired in real time. GPS data is acquired using the location information acquisition means. These data (video information and location information) are sent to the server using the transmission means. The input is the video image from the camera and the location information from the GPS device, and the output is a data packet sent to the server. Through this process, real-time road conditions and location information are acquired.

[1169] Step 2:

[1170] Server: The server receives the video information and location information sent from the device. After receiving the video information, it uses analysis means to analyze it and extract important information such as road surface conditions, road width, and traffic signs. The input is the video information and location information sent from the device, and the output is the extracted road conditions and traffic signs. Specific operations include video analysis using a machine learning model.

[1171] Step 3:

[1172] Server: The analysis results are registered in a database by the database management means and updated as necessary. The input is the information obtained from the analysis means, and the output is an updated database. This ensures that the latest road information and traffic sign data is always maintained.

[1173] Step 4:

[1174] Server: The server selects information about stores and facilities that the user may be interested in based on their location information and past data. It uses a selection method to identify locations that the user may have chosen as their destination. The input is location information and past search history, and the output is store information that the user may be interested in. For example, if the user frequently searches for cafes, the server selects information about nearby cafes.

[1175] Step 5:

[1176] Server: The selected information and analysis results are converted into short sentences using a sentence generation tool. The input is the selected store information and analysis results, and the output is short text information. Specifically, a short sentence such as "There is a popular cafe 500 meters ahead" is generated.

[1177] Step 6:

[1178] Server: Sends short-form information to the user terminal using a transmission means. The input is short-form text information, and the output is the information sent to the user terminal. This allows the user to obtain the information they need in real time.

[1179] Step 7:

[1180] Terminal: The terminal provides the received information to the user using a display means. The input is short text information sent from the server, and the output is information displayed on the terminal's display. Specifically, it displays "Turn right at the next intersection and you'll find a recommended bakery."

[1181] Step 8:

[1182] Server: Using analytical means, the server evaluates the user's driving skills based on the acquired video information and location information, and calculates the driving risk. The evaluation means quantifies the driving skills and calculates the driving risk. The input is video information and location information, and the output is a driving skill score and driving risk assessment. Specific examples such as "driving score 85 (good), low risk" are generated.

[1183] Step 9:

[1184] Server: Sends the evaluation results to the user's device. The input is the driving skill score and driving risk assessment, and the output is the evaluation result sent to the user's device. This allows the user to objectively understand their own driving skills.

[1185] Step 10:

[1186] Device: The device captures the user's face in real time using a built-in or external camera, analyzes facial features using an emotion engine, and classifies the user's emotion. The input is the user's facial image, and the output is emotion classification data as the analysis result.

[1187] Step 11:

[1188] Server: Adjusts the content and display method of the information provided based on the emotional information received from the emotion engine. The input is emotional classification data, and the output is the adjusted information provided. For example, if the user is feeling stressed, "information about places to relax and change your mood" will be provided preferentially.

[1189] This allows the system to provide useful information to the driver in real time, supporting improved driving skills and safe driving. In addition, by incorporating an emotion engine, it becomes possible to provide information optimized for each individual user, further enhancing driving comfort.

[1190] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1191] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1192] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1193] [Third embodiment]

[1194] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1195] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1196] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1197] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1198] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1199] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1200] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1201] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1202] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1203] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1204] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1205] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1206] The system of the present invention is mainly composed of a terminal, a server, and a user. The programs and processes of this system will be specifically described below.

[1207] 1. Data Acquisition and Transmission

[1208] Device:

[1209] The terminal first acquires video information of the road using the camera means. This video information is captured in real time. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[1210] Examples:

[1211] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[1212] 2. Video analysis and database update

[1213] server:

[1214] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[1215] Examples:

[1216] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[1217] 3. Selection of Interest Information

[1218] server:

[1219] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[1220] Examples:

[1221] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[1222] 4. Short sentence generation and sending

[1223] server:

[1224] The selected information and the analysis results are generated in the form of short sentences using a sentence generation means, and after the short sentences are generated, the information is again transmitted to the user terminal as a data packet by a transmission means.

[1225] Examples:

[1226] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[1227] 5. Display of Information

[1228] Device:

[1229] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[1230] Examples:

[1231] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and users can follow that information to drive.

[1232] 6. Driving skill evaluation and risk calculation

[1233] server:

[1234] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[1235] Examples:

[1236] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[1237] 7. Display of evaluation results

[1238] Device:

[1239] The device displays the results of the driving skill evaluation and risk assessment to the user, allowing the user to objectively understand their own driving skills.

[1240] Examples:

[1241] The device screen will display a rating of "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[1242] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[1243] The processing flow will be explained below.

[1244] Step 1:

[1245] Device:

[1246] The device activates the camera means and captures road images in real time through the windshield of the vehicle. The camera means acquires high-resolution images and temporarily stores them as video files. At the same time, the device uses the location information acquisition means to acquire current location information from the GPS sensor.

[1247] Step 2:

[1248] Device:

[1249] The device combines the captured video file and the acquired location information into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmits this data packet to the server using a transmission means.

[1250] Step 3:

[1251] server:

[1252] The server receives the data packets sent from the device, stores them in storage, and separates and extracts the video information and location information.

[1253] Step 4:

[1254] server:

[1255] The server processes the video information using analytical means, specifically, image recognition algorithms to analyze and extract information such as road surface conditions (e.g., wear, potholes), road width, and traffic signs (e.g., speed limits, caution signs).

[1256] Step 5:

[1257] server:

[1258] The information analyzed and extracted by the analysis means is registered and updated in an existing database by the database management means. The database stores road conditions and related information corresponding to specific location information.

[1259] Step 6:

[1260] server:

[1261] The server uses a selection method to select information about stores and facilities that the user is likely to be interested in based on the analyzed location information and past search history. The selected information is also reflected in the navigation system.

[1262] Step 7:

[1263] server:

[1264] The selected information and analysis results are converted into user-friendly short-form text using a text generation means, which is optimized for intuitive understanding while the user is driving.

[1265] Step 8:

[1266] server:

[1267] The text generated in the short sentence format is again packed into a data packet and transmitted from the server to the user terminal using the transmission means.

[1268] Step 9:

[1269] Device:

[1270] The terminal extracts text information from the received data packets and provides it to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[1271] Step 10:

[1272] server:

[1273] The server then uses the analysis means to evaluate the user's driving skills based on the acquired video and location information. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[1274] Step 11:

[1275] server:

[1276] The driving skill value and driving risk assessment calculated by the assessment means are transmitted from the server to the terminal as information to be provided to the user.

[1277] Step 12:

[1278] Device:

[1279] The device provides the user with the results of the driving skill evaluation and driving risk using a display, allowing the user to visually understand their own driving skills and improve their driving style as necessary.

[1280] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[1281] Example 1

[1282] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1283] In modern transportation systems, drivers have difficulty obtaining real-time road and traffic information. Furthermore, there is a lack of efficient means to objectively evaluate their own driving skills and collect and analyze data that can be used to promote safe driving. This poses a challenge in improving driver safety and providing efficient driving assistance.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1285] In this invention, the server includes: a photographing means for acquiring video information; a positioning means for acquiring location information; a communication means for transmitting information obtained from the photographing means and the positioning means to an information processing device; an information analysis means for analyzing the video information and extracting information such as traffic conditions, road width, and signs; a data management means for registering and updating the information obtained by the information analysis means in a data structure; an information selection means for selecting location information that is likely to be of interest to a user based on the location information; an information generation means for generating the information selected by the information selection means and the analysis results in a short sentence format; a communication means for transmitting the information generated by the information generation means to a user terminal; an evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the information analysis means; a communication means for transmitting the results calculated by the evaluation means to the user terminal; and a display means for displaying information to a user using the communication means. This makes it possible to provide useful information to drivers in real time and support the improvement of driving skills and safe driving.

[1286] text

[1287] "Photographing means" refers to a device or equipment for acquiring video information.

[1288] "Positioning means" refers to a device or equipment for acquiring location information.

[1289] A "communication means" is a device or mechanism for transmitting acquired information to another device or system.

[1290] "Information analysis means" refers to equipment or mechanisms for analyzing acquired video information and extracting useful information.

[1291] A "data management means" is a device or mechanism for registering and updating acquired and analyzed information in a data structure.

[1292] An "information selection tool" is a device or mechanism for selecting information that may be of interest to a user based on location information or other data.

[1293] "Information generation means" refers to a device or mechanism for generating selected information or analytical results in short form.

[1294] "Assessment tools" are devices or mechanisms for assessing driving skills and calculating driving risk.

[1295] "Display means" refers to a device or equipment that provides acquired information, analysis results, etc. to users.

[1296] MODE FOR CARRYING OUT THE INVENTION

[1297] The system of the present invention comprises an imaging means, a positioning means, a communication means, an information analysis means, a data management means, an information selection means, an information generation means, an evaluation means, and a display means. A specific implementation method of the system using each means will be described below.

[1298] Photography and positioning methods

[1299] Device:

[1300] The device first acquires video information of the road using a camera as a means of capturing images. This camera can be a smartphone or an in-vehicle camera. At the same time, it acquires location information using a GPS sensor as a means of positioning. This makes it possible to collect video information and its location information in real time.

[1301] Examples:

[1302] While driving, the user points the smartphone camera at the windshield, which captures the road ahead. At the same time, the GPS sensor acquires the current location information and sends this data to a server in real time.

[1303] communication means

[1304] Device:

[1305] The acquired video information and location information are sent to a server as data packets via a communication method such as Wi-Fi or mobile data communication.

[1306] Information analysis means

[1307] server:

[1308] The server analyzes the received video and location information using an image analysis library such as OpenCV. The analysis extracts information such as road surface conditions, traffic signs, and road width.

[1309] Examples:

[1310] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only traffic light at the next intersection."

[1311] Data Management Measures

[1312] server:

[1313] The extracted information is stored in a database using a data management tool. The database system can be MySQL or MongoDB. The database is updated as needed.

[1314] Information selection means

[1315] server:

[1316] The server uses information selection methods based on the user's location information and past search history to select location information that the user is likely to be interested in. A machine learning model is used for the selection.

[1317] Examples:

[1318] If the user has frequently searched for cafes in the past, the server will select information such as "There is a popular cafe 500 meters ahead."

[1319] Information generation means

[1320] server:

[1321] The selected information and analysis results are generated in short sentence format using an information generation means (e.g., GPT-3).

[1322] Example prompt sentence:

[1323] "If you turn right at the next intersection, there's a bakery I recommend."

[1324] Communication and display means

[1325] Server and Device:

[1326] The generated short sentence information is transmitted to the user terminal using the communication means and is displayed on the display means of the terminal.

[1327] Examples:

[1328] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[1329] Evaluation methods

[1330] server:

[1331] The server uses a proprietary algorithm to evaluate the driving technique and calculate the driving risk based on the acquired video and location information.

[1332] Examples:

[1333] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[1334] Display means

[1335] Device:

[1336] The evaluation results are presented to the user using the display means of the terminal, allowing the user to objectively grasp their own driving skills.

[1337] Examples:

[1338] The evaluation results, "Driving score: 85 (good), risk: low," are displayed on the device screen, allowing users to visually check their own driving skills.

[1339] In this way, the system of the present invention can provide useful information to the driver in real time, and support the improvement of driving skills and safe driving.

[1340] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1341] System program processing flow

[1342] Step 1: Data Acquisition

[1343] Device:

[1344] The device uses a camera to capture video information of the road in real time, and also uses a GPS sensor to acquire location information, allowing it to collect video data and location information.

[1345] Input: None (activated by user action)

[1346] Processing: Record video with the camera and obtain location information with the GPS sensor

[1347] Output: Video data, location data

[1348] Specific behavior:

[1349] The user points the smartphone camera at the windshield and launches the app, which then captures video of the road and uses the GPS sensor to obtain the user's current location.

[1350] Step 2: Send data

[1351] Device:

[1352] The acquired video data and location data are sent to the server as data packets using wireless communication technology (Wi-Fi or mobile data communication).

[1353] Input: Video data, location data

[1354] Processing: Generate a data packet and send it to the server

[1355] Output: Data packet (including video data and location data)

[1356] Specific behavior:

[1357] The device combines the video data and GPS location information into a single data packet and sends it to the server.

[1358] Step 3: Data reception and analysis

[1359] server:

[1360] The server receives the data packets sent from the terminal, separates the video data and the location information data, and then analyzes the video data using information analysis means to extract information on road conditions and traffic signs as analysis results.

[1361] Input: Data packet

[1362] Processing: Breaking down the data packets and analyzing the video and location data

[1363] Output: Analysis results (road surface conditions, traffic sign information, etc.)

[1364] Specific behavior:

[1365] The video data received by the server is analyzed using an image analysis library such as OpenCV, and information such as the "road wear condition" and "location of traffic signs" is extracted.

[1366] Step 4: Update the database

[1367] server:

[1368] The server stores the parsed information in a database and updates the data as needed. Data management is performed using a database management system (e.g., MySQL or MongoDB).

[1369] Input: Analysis results

[1370] Processing: Register and update the analysis results in the database.

[1371] Output: Updated database

[1372] Specific behavior:

[1373] The server stores the analysis results in a database and updates the data when new information is added or existing information is changed.

[1374] Step 5: Selecting Interest Information

[1375] server:

[1376] The server analyzes location information and past search history and uses machine learning models to select location information that is likely to be of interest to the user.

[1377] Input: Location data, past search history

[1378] Processing: Analyzing information using machine learning models

[1379] Output: Selected interest information

[1380] Specific behavior:

[1381] The server selects information about nearby popular cafes based on the user's past cafe search history.

[1382] Step 6: Short sentence generation

[1383] server:

[1384] The selected information of interest and analysis results are generated in short sentence format using an AI model (e.g., GPT-3).

[1385] Input: Interest information, analysis results

[1386] Processing: Generate short sentences using AI models

[1387] Output: Short message

[1388] Specific behavior:

[1389] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery."

[1390] Step 7: Send a short message

[1391] Server and Device:

[1392] The generated short sentence is sent to the user terminal using a communication means and displayed on the terminal.

[1393] Input: A short message

[1394] Processing: Send the message as a data packet

[1395] Output: Messages displayed on the terminal

[1396] Specific behavior:

[1397] Short sentences are displayed on the device screen, allowing users to obtain useful information in real time.

[1398] Step 8: Driving Skills Assessment

[1399] server:

[1400] The server uses a proprietary algorithm to evaluate driving skills based on video data and location data, and calculates driving risk.

[1401] Input: Video data, location data

[1402] Processing: Evaluation and risk calculation using proprietary algorithms

[1403] Output: Evaluation results, risk calculation results

[1404] Specific behavior:

[1405] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[1406] Step 9: View the evaluation results

[1407] Device:

[1408] The device displays the evaluation results and risk calculation results on the screen, allowing the user to objectively understand their own driving skills.

[1409] Input: Evaluation results, risk calculation results

[1410] Action: Display on screen

[1411] Output: Rating information displayed on the terminal

[1412] Specific behavior:

[1413] The device screen will display "Driving score: 85 (good), Risk: low," allowing the user to check their driving skills.

[1414] (Application example 1)

[1415] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1416] Conventional driver assistance and navigation systems do not adequately assess changes in road conditions and driving skills in real time, and therefore do not provide sufficient information useful to drivers. In particular, autonomous vehicles require accurate and up-to-date road information and rapid calculation of driving risks, but current technology lacks a comprehensive system to solve this problem.

[1417] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1418] In this invention, the server includes an analysis means, a data storage means, and a selection means, which enable real-time analysis of road conditions. Also, by including a means for evaluating driving behavior and providing real-time updates to the vehicle's navigation system, it is possible to provide the driver with quick and useful information.

[1419] "Photographing device means" refers to a device used to capture video information.

[1420] A "positioning means" is a device or system used to obtain location information.

[1421] The "communication means" is a means for transmitting information obtained from the photographing device means and the positioning means to the data processing device.

[1422] "Analysis means" refers to technology for analyzing video information and extracting information such as road surface conditions, road width, and traffic signs.

[1423] The "data storage means" is a system for registering and updating information obtained by the analysis means in a database.

[1424] The "selection means" is a technique for selecting facility information that is likely to interest the user based on location information.

[1425] The "text generation means" is a technique for generating the information and analysis results selected by the selection means in a short sentence format.

[1426] A "display means" is a device or system for displaying information to a user using a communication means.

[1427] The "evaluation means" is a technique for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means.

[1428] An "automobile navigation system" is a system that provides real-time updates and presents appropriate driving routes and information to drivers.

[1429] MODE FOR CARRYING OUT THE INVENTION

[1430] The present invention is a system that includes real-time analysis of road conditions and evaluation of driving skills, and detailed embodiments of the system are described below.

[1431] System Configuration

[1432] Terminal

[1433] The device used is a smartphone or tablet and includes the following elements:

[1434] Camera means: Video information of the road is acquired using a smartphone or tablet camera.

[1435] Positioning method: Location information is obtained using the built-in GPS module.

[1436] Communication means: A communication means for transmitting the acquired video information and location information to the data processing device.

[1437] server

[1438] On the server, the following elements are included:

[1439] Analysis method: Technology to analyze received video information and extract information such as road conditions and traffic signs. This uses a trained artificial intelligence model (for example, using the OpenCV library).

[1440] Data storage means: A system for registering and updating information obtained by the analysis means in a database.

[1441] Selection method: Technology for selecting facility information (e.g., cafes, bakeries, etc.) that may be of interest to the user based on location information.

[1442] Text generation means: A technology that generates the information selected by the selection means and the analysis results in short sentences. The short sentences are generated using a generative AI model.

[1443] Evaluation method: A technology that evaluates driving skills and calculates driving risks based on information obtained by analytical methods.

[1444] User

[1445] Users interact with the system using smartphones and tablets.

[1446] Display Means: A device or system for displaying information to a user using a communication means.

[1447] Automotive navigation system: Refers to a system that provides real-time updates and presents appropriate driving routes and information to the driver.

[1448] Program processing explanation

[1449] When the system is operating, video information captured by the terminal's camera means and GPS location information are sent to the server via the communication means. Based on this information, the server's analysis means analyzes road conditions and traffic signs, and the data storage means stores the results in a database. At the same time, the server uses the selection means to select facility information likely to be of interest to the user, and generates the information in short sentence format using the text generation means.

[1450] The analysis results and selected information are sent back to the terminal via the communication means and displayed to the user via the display means. This makes it possible to provide useful information to the driver in real time. In addition, the driver's driving skills can be evaluated using the evaluation means, and driving risks can be calculated.

[1451] Specific examples

[1452] For example, if a user frequently searches for cafes, the information displayed might look like this:

[1453] "Turn right at the next intersection and you'll find a popular cafe 50 meters away."

[1454] "The road is in good condition at this point, so please be careful."

[1455] Prompt Sentence Examples

[1456] "Users often search for cafes. Please generate a short sentence like, 'Turn right at the next intersection and there's a popular cafe 50 meters away.'"

[1457] In this way, the system of the present invention analyzes road conditions in real time through data communication between the terminal and the server, and provides useful information to drivers. In addition, by using a generative AI model, it is possible to provide highly accurate information for autonomous vehicles.

[1458] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1459] Step 1:

[1460] The terminal acquires video information of roads using a camera means. At the same time, it also acquires GPS location information using a positioning means. The input data obtained from this is video captured in real time and location information of the location where the video was taken. This video information and location information are sent to a server via a communication means.

[1461] Step 2:

[1462] The server receives the video information and location information sent from the device. It passes this received data to the analysis means. The analysis means analyzes the video information using a trained artificial intelligence model (for example, using the OpenCV library) and extracts information such as road conditions and traffic signs. The data is processed based on the analysis results and organized for storage in a database in a clear format.

[1463] Step 3:

[1464] The server's data storage means registers the organized analysis results in a database and updates the information as necessary. This process ensures that the latest road conditions and traffic sign data is always available for use in other processes.

[1465] Step 4:

[1466] The server uses a selection means to analyze the user's location information and interest information (e.g., past search history) and select facility information that is likely to be of interest to the user. The selected information becomes useful information tailored to the user's current location and driving route. This process uses the user's location information as input data and outputs a list of facilities of interest.

[1467] Step 5:

[1468] The server uses a text generation means to generate the information selected by the selection means and the analysis results in short sentence format. Utilizing a generative AI model, it creates a short sentence such as, "Turn right at the next intersection and you'll find a popular cafe 50 meters away." The input data for this generation process is the selected facility information and the analysis results, and the output data is short sentence information to be provided to the user.

[1469] Step 6:

[1470] The server then uses the communication means to send the generated short message back to the terminal. The terminal then displays the received message to the user via a display. For example, a message such as "Turn right at the next intersection and you'll find a popular cafe 50 meters away" may be displayed on the smartphone screen.

[1471] Step 7:

[1472] At the same time, the evaluation means in the server evaluates the user's driving skills and calculates the driving risk based on the information obtained by the analysis means. For the evaluation, the number of sudden brakings, the frequency of speeding, etc. are quantified, and the driving risk is calculated based on this. In this way, the user's driving behavior is evaluated as a number, and further output as a driving risk.

[1473] Step 8:

[1474] The server transmits the driving skill evaluation results and driving risk calculated by the evaluation means to the user terminal via communication means. The terminal provides these evaluation results and risk information to the user via display means. For example, the smartphone screen may display "Driving score: 85 (good), Risk: low."

[1475] Through these steps, the system analyzes road conditions in real time, provides useful information to drivers, and evaluates driving skills and calculates risks.

[1476] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1477] The system of the present invention is mainly composed of a terminal, a server, and a user, and also has an emotion engine built in. The program of this system and its processing will be explained in detail below.

[1478] 1. Data Acquisition and Transmission

[1479] Device:

[1480] The terminal activates the camera means and captures road images in real time through the windshield of the car. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[1481] Examples:

[1482] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[1483] 2. Video analysis and database update

[1484] server:

[1485] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[1486] Examples:

[1487] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[1488] 3. Selection of Interest Information

[1489] server:

[1490] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[1491] Examples:

[1492] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[1493] 4. Short sentence generation and sending

[1494] server:

[1495] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[1496] Examples:

[1497] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[1498] 5. Display of Information

[1499] Device:

[1500] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[1501] Examples:

[1502] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[1503] 6. Driving skill evaluation and risk calculation

[1504] server:

[1505] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[1506] Examples:

[1507] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[1508] 7. Display of evaluation results

[1509] Device:

[1510] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[1511] Examples:

[1512] The device screen displays the rating "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[1513] 8. User Emotion Recognition

[1514] Device:

[1515] The device captures a user's face in real time using a built-in or external camera. The emotion engine analyzes facial features and classifies the user's emotion. The classified emotion information is then sent to the server as a data packet.

[1516] Examples:

[1517] While the user is driving, a camera captures their face, and AI analyzes their emotions, such as "happiness," "anger," and "sadness," in real time. The results are then sent to a server.

[1518] 9. Emotion-based information regulation

[1519] server:

[1520] The server adjusts the content and display method of the information it provides based on the emotional information received from the emotion engine. For example, if the user is feeling stressed, it will prioritize providing information about places where they can relax or change their mood.

[1521] Examples:

[1522] If the server determines through emotion analysis that the user is tired, it provides information such as, "There is a parking area up ahead where you can rest."

[1523] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[1524] The processing flow will be explained below.

[1525] Step 1:

[1526] Device:

[1527] The camera means is activated and captures road images in real time through the windshield of the vehicle. The images are temporarily saved as video files. At the same time, GPS data is acquired using the location information acquisition means.

[1528] Step 2:

[1529] Device:

[1530] The captured video file and the acquired location information are combined into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmitted to a server using a transmission means.

[1531] Step 3:

[1532] server:

[1533] Receives data packets sent from the device. The received data packets are stored in storage, and video information and location information are separated and extracted.

[1534] Step 4:

[1535] server:

[1536] Video information is processed using analytical means, specifically, image recognition algorithms are used to analyze and extract information such as road surface conditions (wear, potholes), road width, and traffic signs (speed limits, caution signs).

[1537] Step 5:

[1538] server:

[1539] The analysis results obtained by the analysis means are registered and updated in an existing database by the database management means, and road conditions and related information corresponding to specific location information are stored.

[1540] Step 6:

[1541] server:

[1542] Based on the analyzed location information and past search history, the server uses a selection method to select information about stores and facilities that the user is likely to be interested in. The selected information is also reflected in the navigation system.

[1543] Step 7:

[1544] server:

[1545] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[1546] Step 8:

[1547] Device:

[1548] The text information is extracted from the received data packet and provided to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[1549] Step 9:

[1550] server:

[1551] The system evaluates the user's driving skills based on video and location information acquired using analytical methods. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[1552] Step 10:

[1553] server:

[1554] The driving skill value and driving risk assessment calculated by the assessment means are again compiled into a data packet as information to be provided to the user, and are transmitted to the user terminal using the transmission means.

[1555] Step 11:

[1556] Device:

[1557] The device provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[1558] Step 12:

[1559] Device:

[1560] The device then captures a real-time image of the user's face using a built-in or external camera. The emotion engine analyzes the facial features and classifies the user's emotion. This classified emotion information is then sent to the server as a data packet.

[1561] Step 13:

[1562] server:

[1563] The server analyzes the emotion information received from the emotion engine and determines the emotional state the user is in. For example, if the user is feeling stressed or tired, that emotion information is sent to the server.

[1564] Step 14:

[1565] server:

[1566] The emotion engine recognizes the user's emotions and adjusts the content and display method of the information provided accordingly. For example, if it determines that the user is feeling stressed, it will prioritize providing information about places to relax and rest areas.

[1567] Step 15:

[1568] Device:

[1569] The terminal again receives the personalized information sent from the server and provides it to the user using the display means, allowing the user to obtain optimal information in real time according to their current emotional state.

[1570] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[1571] Example 2

[1572] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1573] Conventional driving assistance systems have difficulty obtaining real-time road conditions and location information and providing specific navigation based on that information. They also lack sufficient capabilities for evaluating driving skills and calculating driving risks, and do not provide information that takes into account the user's emotional state. To solve these issues, a more advanced and integrated system is needed.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1575] In this invention, the server includes: a camera for capturing video information; a location information sensor for capturing location information; a communication device for transmitting information obtained from the camera and location information sensor to a central processing unit; an analysis device for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs; a database management device for registering and updating the information obtained by the analysis device in a database; a selection device for selecting location information likely to be of interest to a user based on the location information; a sentence generation device for generating the information selected by the selection device and the analysis results in short sentences; a communication device for transmitting the information generated by the sentence generation device to a user terminal; an evaluation device for evaluating driving skills and calculating driving risks based on the information obtained by the analysis device; a communication device for transmitting the results calculated by the evaluation device to a user terminal; a display device for displaying information to a user using the communication device; a processing device including a camera for recognizing user emotions and an emotion analysis engine; and an information adjustment device for appropriately adjusting information based on the emotions recognized by the emotion analysis engine. This enables real-time information provision, driving skill evaluation, and personalized information provision according to user emotions.

[1576] The "capture means" is a device for capturing images of the road from inside the vehicle in real time, and is usually a device including a camera.

[1577] The "location information sensor means" is a device for acquiring the current location of the vehicle, and is usually a device including a GPS sensor.

[1578] The "communication means" is an interface for transmitting acquired data to a destination, and is a device that includes a wired or wireless communication protocol.

[1579] The "analysis device" is a device that analyzes the received video data and extracts information such as road surface conditions, road width, and traffic signs.

[1580] The "database management device" is software or hardware for registering and updating information obtained by the analysis device in a database.

[1581] The "selection device" is a device for selecting location information that is likely to interest a user based on the user's location information and past history.

[1582] A "sentence generation device" is software or hardware for generating selected information and analysis results in short sentence format.

[1583] The "evaluation device" is a device for evaluating driving skills based on video data and location information and calculating driving risks.

[1584] A "display device" is a device for displaying information on a user terminal, typically including a display or screen.

[1585] An "emotion analysis engine" is software or hardware that analyzes the features of a user's face and classifies the user's emotions.

[1586] An "information adjustment device" is a device that adjusts the content and display method of information provided based on emotions recognized by an emotion analysis engine.

[1587] The system of the present invention is implemented by a combination of a terminal, a server, and a user. A specific embodiment of the system will be described in detail below.

[1588] Device behavior

[1589] The terminal is installed inside the vehicle. It performs the following operations using a camera and GPS sensor. The camera captures real-time images of the road through the vehicle's windshield, and the GPS sensor acquires the vehicle's current location. This allows real-time video and location information to be obtained.

[1590] Examples:

[1591] When a user launches the application in a vehicle, the device's camera starts capturing images of the road ahead, and the GPS sensor continuously acquires the vehicle's current location. This video and location information is then formed into a data packet and sent to the server.

[1592] Server Operation

[1593] The server acts as a central processing unit and receives data sent from the terminals, performs multiple analyses and data processing. Specifically, it performs the following operations using an analysis device, a database management device, a selection device, a sentence generation device, an evaluation device, and a sentiment analysis engine.

[1594] Video Analysis

[1595] The server's analysis device analyzes the received video data and extracts information such as road surface conditions, road width, traffic signs, etc. This information is stored in a database using a database management device and updated as necessary.

[1596] Examples:

[1597] By analyzing the video data, information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" is extracted and registered in a database.

[1598] Selection of information of interest

[1599] The selection device of the server selects information (for example, stores and facilities) that is likely to interest the user based on the user's location information and past history.

[1600] Examples:

[1601] If the user has a history of searching for cafes in the past, the server will select and provide information such as "There is a popular cafe 500 meters ahead."

[1602] Short sentence generation

[1603] The server's sentence generation device generates the selected information and analysis results in an easy-to-understand short sentence format and transmits them again to the terminal as a data packet.

[1604] Examples:

[1605] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's terminal.

[1606] Assessment and Risk Calculation

[1607] The server evaluates driving skills and calculates driving risk. The evaluation device calculates a driving score based on the number of sudden braking and speeding, and performs risk assessment.

[1608] Examples:

[1609] The evaluation result "Driving score 85 (good), low risk" is generated and sent to the user's terminal.

[1610] Emotion recognition and information regulation

[1611] The device's camera and emotion analysis engine are used to analyze the user's emotions in real time. The emotion analysis engine classifies the emotions and sends them to the server. The server then adjusts the content and display method of the information based on this emotion information.

[1612] Examples:

[1613] If the emotion analysis engine determines that the user is feeling tired, the server will provide information such as, "There is a parking area up ahead where you can take a rest."

[1614] Adjusted short sentence generation

[1615] The adjusted information is then converted back into short sentences and sent to the device, enabling personalized information to be provided according to the user's emotional state.

[1616] Prompt Sentence Examples

[1617] Below is an example of a prompt that the server might input to the generative AI model:

[1618] "If you turn right at the next intersection, there's a cafe I recommend."

[1619] "Driving score: 85 (good), risk: low"

[1620] "There's a parking area up ahead where you can rest."

[1621] In this way, the entire system works together to provide real-time information and driving evaluation, as well as personalized information based on emotions.

[1622] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1623] Processing Steps

[1624] Step 1: Data Acquisition and Transmission

[1625] Device:

[1626] The device activates the camera and GPS sensor to capture real-time road images through the vehicle's windshield. At the same time, the GPS sensor acquires the device's current location. Based on this, the device assembles the image data and location information into a data packet and sends it to the server.

[1627] Specific behavior:

[1628] 1. The device will activate the built-in camera.

[1629] 2. The camera captures the road scenery ahead.

[1630] 3. The device activates its built-in GPS sensor and acquires location information.

[1631] 4. The video data and location information are packaged in a packet format and sent to the server.

[1632] Input: Camera footage, GPS location information

[1633] Output: Data packet (video data + location information)

[1634] Step 2: Video analysis

[1635] server:

[1636] The server processes the received data packets (video data and location information) using an analysis device to extract necessary information such as road surface conditions, traffic signs, road width, etc. The analysis results are then registered in a database and updated as necessary.

[1637] Specific behavior:

[1638] 1. The server receives the data packet.

[1639] 2. The analysis device analyzes the video data and extracts data such as road wear conditions and traffic signs.

[1640] 3. The extracted data is registered and updated in the database by the database management device.

[1641] Input: Data packet (video data + location information)

[1642] Output: Extracted information (road conditions, traffic signs, etc.)

[1643] Step 3: Selecting information of interest

[1644] server:

[1645] The server uses the user's current location and past search history to select information that is likely to be of interest to the user.

[1646] Specific behavior:

[1647] 1. The server references the location information and the user's search history.

[1648] 2. The selection device applies an algorithm to select stores and facility information that may be of interest.

[1649] Input: User's current location, past search history

[1650] Output: Information that may be of interest to the user

[1651] Step 4: Create and send a short message

[1652] server:

[1653] The server generates text in short sentence format based on the selected information and the analysis results, and transmits it to the user terminal as a data packet.

[1654] Specific behavior:

[1655] 1. The server retrieves the selected information.

[1656] 2. A sentence generator generates text in short sentence format.

[1657] 3. The generated short sentence is sent to the user terminal as a data packet.

[1658] Input: Selection information, analysis results

[1659] Output: Short text

[1660] Step 5: Viewing information

[1661] Device:

[1662] The terminal displays the received data to the user.

[1663] Specific behavior:

[1664] 1. The terminal receives a data packet sent from the server.

[1665] 2. Decode the received information and display it on the screen.

[1666] Input: Data packet (short text format)

[1667] Output: The displayed information (display)

[1668] Step 6: Driving skill assessment and risk calculation

[1669] server:

[1670] The server uses video data and location information to evaluate driving skills and calculate driving risks.

[1671] Specific behavior:

[1672] 1. The server obtains the video data and location information.

[1673] 2. The evaluation device evaluates driving skills based on the number of sudden braking and speeding.

[1674] 3. The calculated driving risk is quantified and an evaluation result is generated.

[1675] Input: Video data, location information

[1676] Output: Driving skill evaluation results, driving risk

[1677] Step 7: View the evaluation results

[1678] Device:

[1679] The terminal displays the driving skill evaluation results sent from the server.

[1680] Specific behavior:

[1681] 1. The terminal receives the evaluation result.

[1682] 2. The received evaluation results are displayed on the screen.

[1683] Input: Evaluation result, driving risk

[1684] Output: Displayed evaluation results (display)

[1685] Step 8: Recognizing User Emotions

[1686] Device:

[1687] The device uses a built-in or external camera to capture a picture of the user's face and classifies the emotion using an emotion analysis engine.

[1688] Specific behavior:

[1689] 1. The device activates the camera and takes a picture of the user's face.

[1690] 2. The emotion analysis engine analyzes facial feature points and classifies emotions.

[1691] 3. The classified emotion information is sent to the server as a data packet.

[1692] Input: A photographed face image

[1693] Output: Emotion information (data packet)

[1694] Step 9: Emotionally regulated information

[1695] server:

[1696] The server adjusts the information provided based on the emotion information and provides the most suitable information to the user.

[1697] Specific behavior:

[1698] 1. The server receives emotion information.

[1699] 2. The sentiment analysis engine adjusts the information based on the analysis results.

[1700] 3. Convert the adjusted information into short-text format and send it to the terminal.

[1701] Input: Emotion information

[1702] Output: Adjusted information (sent to terminal in short form)

[1703] (Application example 2)

[1704] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1705] Modern self-driving vehicles require more accurate information, such as analysis of road conditions and traffic signs. It is also important to analyze passengers' emotional states in real time to provide a comfortable driving service. In this environment, a comprehensive system is needed to provide optimal information to users, evaluate driving skills, and calculate driving risks.

[1706] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a camera means for acquiring video information, a location information acquisition means for acquiring location information, and an emotion analysis means including an emotion engine for analyzing the user's emotional state. This makes it possible to analyze information on road conditions and traffic signs in real time, evaluate driving skills and calculate driving risks, and provide information according to the passenger's emotional state.

[1707] "Moving image information" is video data captured by a camera means.

[1708] The "camera means" is a device for photographing an object and acquiring video information.

[1709] "Location information" refers to coordinate data of the current location obtained using GPS or other technologies.

[1710] "Location information acquisition means" refers to a device or technology for acquiring location information.

[1711] "Transmission means" refers to a technique or device for transmitting acquired data to an external device such as a server.

[1712] "Analysis means" refers to techniques or devices for analyzing acquired data and extracting useful information.

[1713] "Road surface conditions" is information that indicates the condition of the road surface.

[1714] "Road width" is information indicating the width of the road.

[1715] "Traffic signs" refers to various signs installed on roads.

[1716] "Database management means" refers to the technology and devices that register, manage, and update acquired information in a database.

[1717] A "selection means" is a technique or device for selecting information based on specific conditions.

[1718] "Text generation means" refers to a technique or device for generating selected information or analysis results in text format.

[1719] "Evaluation means" refers to technology or equipment that evaluates driving skills based on data and calculates driving risks.

[1720] An "emotion engine" is a technology or device for analyzing and classifying a user's emotional state.

[1721] "Emotion analysis means" refers to a technique or device that uses an emotion engine to analyze the user's emotional state.

[1722] "Display means" refers to technology or devices for visually presenting acquired information and analysis results to the user.

[1723] A "user terminal" is a terminal device that allows a user to obtain information.

[1724] In this invention, by using a system having the following configuration, it is possible to provide information on road conditions and traffic signs in real time, as well as information based on an evaluation of driving skills and the emotional state of passengers.

[1725] 1. Data acquisition and transmission

[1726] Device:

[1727] The terminal is equipped with a camera means and a location information acquisition means. The camera means of the terminal acquires video information in real time, and the location information acquisition means acquires accurate location information using GPS. These data are transmitted to the server using a transmission means.

[1728] For example, a camera mounted on an autonomous vehicle captures the road ahead, and a GPS device acquires the vehicle's current location, which is then transmitted to a server via wireless communication.

[1729] 2.Video analysis and database updates

[1730] server:

[1731] The server analyzes the received video information and location information using an analysis means. The analysis means uses a trained generative AI model to extract information such as road surface conditions, road width, and traffic signs. The extracted information is registered in a database by a database management means and updated as necessary.

[1732] As a specific example, the server's analysis results include information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" that is registered in the database.

[1733] 3. Selection and provision of information of interest

[1734] server:

[1735] Based on the location information and past data, the server selects store and facility information that the user is likely to be interested in. This information is converted into short text using a sentence generation means and sent to the terminal.

[1736] As a specific example, if the user has frequently searched for cafes in the past, the server will generate information such as "There is a popular cafe 500 meters ahead" and send it to the terminal.

[1737] 4. Evaluation of driving skills and calculation of driving risks

[1738] server:

[1739] The server has an evaluation means for evaluating driving skills based on the acquired video information and location information, and calculating driving risk. The evaluation results are sent to the user's terminal.

[1740] As a specific example, the server measures the number of sudden brakings and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[1741] 5. Sentiment analysis and information provision adjustment

[1742] Device:

[1743] The device captures the user's face in real time using a built-in or external camera, analyzes the user's emotional state using an emotion engine, and transmits the analyzed emotional information to a server as a data packet.

[1744] server:

[1745] The server adjusts the information it provides based on the emotional information received from the emotion analysis means. For example, if the user is feeling stressed, it will prioritize providing information about places where the user can relax or change their mood.

[1746] As a specific example, if the server determines that the user is tired, it provides information such as, "There is a parking area up ahead where you can take a rest."

[1747] Prompt Sentence Examples

[1748] An example of a prompt to input to a generative AI model is as follows:

[1749] Prompt: Design an application that uses in-car cameras and GPS information to analyze road conditions and traffic signs in real time, suggest optimal routes, and use an emotion engine to assess passengers' emotional state and provide a comfortable ride.

[1750] In this way, the invention provides useful information to drivers and passengers in real time, helping to improve driving skills and ensure comfortable driving.

[1751] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1752] Step 1:

[1753] Terminal: The terminal's camera means is activated and video information is acquired in real time. GPS data is acquired using the location information acquisition means. These data (video information and location information) are sent to the server using the transmission means. The input is the video image from the camera and the location information from the GPS device, and the output is a data packet sent to the server. Through this process, real-time road conditions and location information are acquired.

[1754] Step 2:

[1755] Server: The server receives the video information and location information sent from the device. After receiving the video information, it uses analysis means to analyze it and extract important information such as road surface conditions, road width, and traffic signs. The input is the video information and location information sent from the device, and the output is the extracted road conditions and traffic signs. Specific operations include video analysis using a machine learning model.

[1756] Step 3:

[1757] Server: The analysis results are registered in a database by the database management means and updated as necessary. The input is the information obtained from the analysis means, and the output is an updated database. This ensures that the latest road information and traffic sign data is always maintained.

[1758] Step 4:

[1759] Server: The server selects information about stores and facilities that the user may be interested in based on their location information and past data. It uses a selection method to identify locations that the user may have chosen as their destination. The input is location information and past search history, and the output is store information that the user may be interested in. For example, if the user frequently searches for cafes, the server selects information about nearby cafes.

[1760] Step 5:

[1761] Server: The selected information and analysis results are converted into short sentences using a sentence generation tool. The input is the selected store information and analysis results, and the output is short text information. Specifically, a short sentence such as "There is a popular cafe 500 meters ahead" is generated.

[1762] Step 6:

[1763] Server: Sends short-form information to the user terminal using a transmission means. The input is short-form text information, and the output is the information sent to the user terminal. This allows the user to obtain the information they need in real time.

[1764] Step 7:

[1765] Terminal: The terminal provides the received information to the user using a display means. The input is short text information sent from the server, and the output is information displayed on the terminal's display. Specifically, it displays "Turn right at the next intersection and you'll find a recommended bakery."

[1766] Step 8:

[1767] Server: Using analytical means, the server evaluates the user's driving skills based on the acquired video information and location information, and calculates the driving risk. The evaluation means quantifies the driving skills and calculates the driving risk. The input is video information and location information, and the output is a driving skill score and driving risk assessment. Specific examples such as "driving score 85 (good), low risk" are generated.

[1768] Step 9:

[1769] Server: Sends the evaluation results to the user's device. The input is the driving skill score and driving risk assessment, and the output is the evaluation result sent to the user's device. This allows the user to objectively understand their own driving skills.

[1770] Step 10:

[1771] Device: The device captures the user's face in real time using a built-in or external camera, analyzes facial features using an emotion engine, and classifies the user's emotion. The input is the user's facial image, and the output is emotion classification data as the analysis result.

[1772] Step 11:

[1773] Server: Adjusts the content and display method of the information provided based on the emotional information received from the emotion engine. The input is emotional classification data, and the output is the adjusted information provided. For example, if the user is feeling stressed, "information about places to relax and change your mood" will be provided preferentially.

[1774] This allows the system to provide useful information to the driver in real time, supporting improved driving skills and safe driving. In addition, by incorporating an emotion engine, it becomes possible to provide information optimized for each individual user, further enhancing driving comfort.

[1775] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1776] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1777] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1778] [Fourth embodiment]

[1779] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1780] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1781] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1782] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1783] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1784] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1785] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1786] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1787] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1788] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1789] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1790] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1791] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1792] The system of the present invention is mainly composed of a terminal, a server, and a user. The programs and processes of this system will be specifically described below.

[1793] 1. Data Acquisition and Transmission

[1794] Device:

[1795] The terminal first acquires video information of the road using the camera means. This video information is captured in real time. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[1796] Examples:

[1797] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[1798] 2. Video analysis and database update

[1799] server:

[1800] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[1801] Examples:

[1802] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[1803] 3. Selection of Interest Information

[1804] server:

[1805] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[1806] Examples:

[1807] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[1808] 4. Short sentence generation and sending

[1809] server:

[1810] The selected information and the analysis results are generated in the form of short sentences using a sentence generation means, and after the short sentences are generated, the information is again transmitted to the user terminal as a data packet by a transmission means.

[1811] Examples:

[1812] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[1813] 5. Display of Information

[1814] Device:

[1815] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[1816] Examples:

[1817] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and users can follow that information to drive.

[1818] 6. Driving skill evaluation and risk calculation

[1819] server:

[1820] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[1821] Examples:

[1822] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[1823] 7. Display of evaluation results

[1824] Device:

[1825] The device displays the results of the driving skill evaluation and risk assessment to the user, allowing the user to objectively understand their own driving skills.

[1826] Examples:

[1827] The device screen will display a rating of "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[1828] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[1829] The processing flow will be explained below.

[1830] Step 1:

[1831] Device:

[1832] The device activates the camera means and captures road images in real time through the windshield of the vehicle. The camera means acquires high-resolution images and temporarily stores them as video files. At the same time, the device uses the location information acquisition means to acquire current location information from the GPS sensor.

[1833] Step 2:

[1834] Device:

[1835] The device combines the captured video file and the acquired location information into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmits this data packet to the server using a transmission means.

[1836] Step 3:

[1837] server:

[1838] The server receives the data packets sent from the device, stores them in storage, and separates and extracts the video information and location information.

[1839] Step 4:

[1840] server:

[1841] The server processes the video information using analytical means, specifically, image recognition algorithms to analyze and extract information such as road surface conditions (e.g., wear, potholes), road width, and traffic signs (e.g., speed limits, caution signs).

[1842] Step 5:

[1843] server:

[1844] The information analyzed and extracted by the analysis means is registered and updated in an existing database by the database management means. The database stores road conditions and related information corresponding to specific location information.

[1845] Step 6:

[1846] server:

[1847] The server uses a selection method to select information about stores and facilities that the user is likely to be interested in based on the analyzed location information and past search history. The selected information is also reflected in the navigation system.

[1848] Step 7:

[1849] server:

[1850] The selected information and analysis results are converted into user-friendly short-form text using a text generation means, which is optimized for intuitive understanding while the user is driving.

[1851] Step 8:

[1852] server:

[1853] The text generated in the short sentence format is again packed into a data packet and transmitted from the server to the user terminal using the transmission means.

[1854] Step 9:

[1855] Device:

[1856] The terminal extracts text information from the received data packets and provides it to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[1857] Step 10:

[1858] server:

[1859] The server then uses the analysis means to evaluate the user's driving skills based on the acquired video and location information. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[1860] Step 11:

[1861] server:

[1862] The driving skill value and driving risk assessment calculated by the assessment means are transmitted from the server to the terminal as information to be provided to the user.

[1863] Step 12:

[1864] Device:

[1865] The device provides the user with the results of the driving skill evaluation and driving risk using a display, allowing the user to visually understand their own driving skills and improve their driving style as necessary.

[1866] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving.

[1867] Example 1

[1868] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1869] In modern transportation systems, drivers have difficulty obtaining real-time road and traffic information. Furthermore, there is a lack of efficient means to objectively evaluate their own driving skills and collect and analyze data that can be used to promote safe driving. This poses a challenge in improving driver safety and providing efficient driving assistance.

[1870] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1871] In this invention, the server includes: a photographing means for acquiring video information; a positioning means for acquiring location information; a communication means for transmitting information obtained from the photographing means and the positioning means to an information processing device; an information analysis means for analyzing the video information and extracting information such as traffic conditions, road width, and signs; a data management means for registering and updating the information obtained by the information analysis means in a data structure; an information selection means for selecting location information that is likely to be of interest to a user based on the location information; an information generation means for generating the information selected by the information selection means and the analysis results in a short sentence format; a communication means for transmitting the information generated by the information generation means to a user terminal; an evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the information analysis means; a communication means for transmitting the results calculated by the evaluation means to the user terminal; and a display means for displaying information to a user using the communication means. This makes it possible to provide useful information to drivers in real time and support the improvement of driving skills and safe driving.

[1872] text

[1873] "Photographing means" refers to a device or equipment for acquiring video information.

[1874] "Positioning means" refers to a device or equipment for acquiring location information.

[1875] A "communication means" is a device or mechanism for transmitting acquired information to another device or system.

[1876] "Information analysis means" refers to equipment or mechanisms for analyzing acquired video information and extracting useful information.

[1877] A "data management means" is a device or mechanism for registering and updating acquired and analyzed information in a data structure.

[1878] An "information selection tool" is a device or mechanism for selecting information that may be of interest to a user based on location information or other data.

[1879] "Information generation means" refers to a device or mechanism for generating selected information or analytical results in short form.

[1880] "Assessment tools" are devices or mechanisms for assessing driving skills and calculating driving risk.

[1881] "Display means" refers to a device or equipment that provides acquired information, analysis results, etc. to users.

[1882] MODE FOR CARRYING OUT THE INVENTION

[1883] The system of the present invention comprises an imaging means, a positioning means, a communication means, an information analysis means, a data management means, an information selection means, an information generation means, an evaluation means, and a display means. A specific implementation method of the system using each means will be described below.

[1884] Photography and positioning methods

[1885] Device:

[1886] The device first acquires video information of the road using a camera as a means of capturing images. This camera can be a smartphone or an in-vehicle camera. At the same time, it acquires location information using a GPS sensor as a means of positioning. This makes it possible to collect video information and its location information in real time.

[1887] Examples:

[1888] While driving, the user points the smartphone camera at the windshield, which captures the road ahead. At the same time, the GPS sensor acquires the current location information and sends this data to a server in real time.

[1889] communication means

[1890] Device:

[1891] The acquired video information and location information are sent to a server as data packets via a communication method such as Wi-Fi or mobile data communication.

[1892] Information analysis means

[1893] server:

[1894] The server analyzes the received video and location information using an image analysis library such as OpenCV. The analysis extracts information such as road surface conditions, traffic signs, and road width.

[1895] Examples:

[1896] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only traffic light at the next intersection."

[1897] Data Management Measures

[1898] server:

[1899] The extracted information is stored in a database using a data management tool. The database system can be MySQL or MongoDB. The database is updated as needed.

[1900] Information selection means

[1901] server:

[1902] The server uses information selection methods based on the user's location information and past search history to select location information that the user is likely to be interested in. A machine learning model is used for the selection.

[1903] Examples:

[1904] If the user has frequently searched for cafes in the past, the server will select information such as "There is a popular cafe 500 meters ahead."

[1905] Information generation means

[1906] server:

[1907] The selected information and analysis results are generated in short sentence format using an information generation means (e.g., GPT-3).

[1908] Example prompt sentence:

[1909] "If you turn right at the next intersection, there's a bakery I recommend."

[1910] Communication and display means

[1911] Server and Device:

[1912] The generated short sentence information is transmitted to the user terminal using the communication means and is displayed on the display means of the terminal.

[1913] Examples:

[1914] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[1915] Evaluation methods

[1916] server:

[1917] The server uses a proprietary algorithm to evaluate the driving technique and calculate the driving risk based on the acquired video and location information.

[1918] Examples:

[1919] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[1920] Display means

[1921] Device:

[1922] The evaluation results are presented to the user using the display means of the terminal, allowing the user to objectively grasp their own driving skills.

[1923] Examples:

[1924] The evaluation results, "Driving score: 85 (good), risk: low," are displayed on the device screen, allowing users to visually check their own driving skills.

[1925] In this way, the system of the present invention can provide useful information to the driver in real time, and support the improvement of driving skills and safe driving.

[1926] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1927] System program processing flow

[1928] Step 1: Data Acquisition

[1929] Device:

[1930] The device uses a camera to capture video information of the road in real time, and also uses a GPS sensor to acquire location information, allowing it to collect video data and location information.

[1931] Input: None (activated by user action)

[1932] Processing: Record video with the camera and obtain location information with the GPS sensor

[1933] Output: Video data, location data

[1934] Specific behavior:

[1935] The user points the smartphone camera at the windshield and launches the app, which then captures video of the road and uses the GPS sensor to obtain the user's current location.

[1936] Step 2: Send data

[1937] Device:

[1938] The acquired video data and location data are sent to the server as data packets using wireless communication technology (Wi-Fi or mobile data communication).

[1939] Input: Video data, location data

[1940] Processing: Generate a data packet and send it to the server

[1941] Output: Data packet (including video data and location data)

[1942] Specific behavior:

[1943] The device combines the video data and GPS location information into a single data packet and sends it to the server.

[1944] Step 3: Data reception and analysis

[1945] server:

[1946] The server receives the data packets sent from the terminal, separates the video data and the location information data, and then analyzes the video data using information analysis means to extract information on road conditions and traffic signs as analysis results.

[1947] Input: Data packet

[1948] Processing: Breaking down the data packets and analyzing the video and location data

[1949] Output: Analysis results (road surface conditions, traffic sign information, etc.)

[1950] Specific behavior:

[1951] The video data received by the server is analyzed using an image analysis library such as OpenCV, and information such as the "road wear condition" and "location of traffic signs" is extracted.

[1952] Step 4: Update the database

[1953] server:

[1954] The server stores the parsed information in a database and updates the data as needed. Data management is performed using a database management system (e.g., MySQL or MongoDB).

[1955] Input: Analysis results

[1956] Processing: Register and update the analysis results in the database.

[1957] Output: Updated database

[1958] Specific behavior:

[1959] The server stores the analysis results in a database and updates the data when new information is added or existing information is changed.

[1960] Step 5: Selecting Interest Information

[1961] server:

[1962] The server analyzes location information and past search history and uses machine learning models to select location information that is likely to be of interest to the user.

[1963] Input: Location data, past search history

[1964] Processing: Analyzing information using machine learning models

[1965] Output: Selected interest information

[1966] Specific behavior:

[1967] The server selects information about nearby popular cafes based on the user's past cafe search history.

[1968] Step 6: Short sentence generation

[1969] server:

[1970] The selected information of interest and analysis results are generated in short sentence format using an AI model (e.g., GPT-3).

[1971] Input: Interest information, analysis results

[1972] Processing: Generate short sentences using AI models

[1973] Output: Short message

[1974] Specific behavior:

[1975] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery."

[1976] Step 7: Send a short message

[1977] Server and Device:

[1978] The generated short sentence is sent to the user terminal using a communication means and displayed on the terminal.

[1979] Input: A short message

[1980] Processing: Send the message as a data packet

[1981] Output: Messages displayed on the terminal

[1982] Specific behavior:

[1983] Short sentences are displayed on the device screen, allowing users to obtain useful information in real time.

[1984] Step 8: Driving Skills Assessment

[1985] server:

[1986] The server uses a proprietary algorithm to evaluate driving skills based on video data and location data, and calculates driving risk.

[1987] Input: Video data, location data

[1988] Processing: Evaluation and risk calculation using proprietary algorithms

[1989] Output: Evaluation results, risk calculation results

[1990] Specific behavior:

[1991] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation result of "driving score 85 (good), low risk," and sends it to the user's terminal.

[1992] Step 9: View the evaluation results

[1993] Device:

[1994] The device displays the evaluation results and risk calculation results on the screen, allowing the user to objectively understand their own driving skills.

[1995] Input: Evaluation results, risk calculation results

[1996] Action: Display on screen

[1997] Output: Rating information displayed on the terminal

[1998] Specific behavior:

[1999] The device screen will display "Driving score: 85 (good), Risk: low," allowing the user to check their driving skills.

[2000] (Application example 1)

[2001] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2002] Conventional driver assistance and navigation systems do not adequately assess changes in road conditions and driving skills in real time, and therefore do not provide sufficient information useful to drivers. In particular, autonomous vehicles require accurate and up-to-date road information and rapid calculation of driving risks, but current technology lacks a comprehensive system to solve this problem.

[2003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2004] In this invention, the server includes an analysis means, a data storage means, and a selection means, which enable real-time analysis of road conditions. Also, by including a means for evaluating driving behavior and providing real-time updates to the vehicle's navigation system, it is possible to provide the driver with quick and useful information.

[2005] "Photographing device means" refers to a device used to capture video information.

[2006] A "positioning means" is a device or system used to obtain location information.

[2007] The "communication means" is a means for transmitting information obtained from the photographing device means and the positioning means to the data processing device.

[2008] "Analysis means" refers to technology for analyzing video information and extracting information such as road surface conditions, road width, and traffic signs.

[2009] The "data storage means" is a system for registering and updating information obtained by the analysis means in a database.

[2010] The "selection means" is a technique for selecting facility information that is likely to interest the user based on location information.

[2011] The "text generation means" is a technique for generating the information and analysis results selected by the selection means in a short sentence format.

[2012] A "display means" is a device or system for displaying information to a user using a communication means.

[2013] The "evaluation means" is a technique for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means.

[2014] An "automobile navigation system" is a system that provides real-time updates and presents appropriate driving routes and information to drivers.

[2015] MODE FOR CARRYING OUT THE INVENTION

[2016] The present invention is a system that includes real-time analysis of road conditions and evaluation of driving skills, and detailed embodiments of the system are described below.

[2017] System Configuration

[2018] Terminal

[2019] The device used is a smartphone or tablet and includes the following elements:

[2020] Camera means: Video information of the road is acquired using a smartphone or tablet camera.

[2021] Positioning method: Location information is obtained using the built-in GPS module.

[2022] Communication means: A communication means for transmitting the acquired video information and location information to the data processing device.

[2023] server

[2024] On the server, the following elements are included:

[2025] Analysis method: Technology to analyze received video information and extract information such as road conditions and traffic signs. This uses a trained artificial intelligence model (for example, using the OpenCV library).

[2026] Data storage means: A system for registering and updating information obtained by the analysis means in a database.

[2027] Selection method: Technology for selecting facility information (e.g., cafes, bakeries, etc.) that may be of interest to the user based on location information.

[2028] Text generation means: A technology that generates the information selected by the selection means and the analysis results in short sentences. The short sentences are generated using a generative AI model.

[2029] Evaluation method: A technology that evaluates driving skills and calculates driving risks based on information obtained by analytical methods.

[2030] User

[2031] Users interact with the system using smartphones and tablets.

[2032] Display Means: A device or system for displaying information to a user using a communication means.

[2033] Automotive navigation system: Refers to a system that provides real-time updates and presents appropriate driving routes and information to the driver.

[2034] Program processing explanation

[2035] When the system is operating, video information captured by the terminal's camera means and GPS location information are sent to the server via the communication means. Based on this information, the server's analysis means analyzes road conditions and traffic signs, and the data storage means stores the results in a database. At the same time, the server uses the selection means to select facility information likely to be of interest to the user, and generates the information in short sentence format using the text generation means.

[2036] The analysis results and selected information are sent back to the terminal via the communication means and displayed to the user via the display means. This makes it possible to provide useful information to the driver in real time. In addition, the driver's driving skills can be evaluated using the evaluation means, and driving risks can be calculated.

[2037] Specific examples

[2038] For example, if a user frequently searches for cafes, the information displayed might look like this:

[2039] "Turn right at the next intersection and you'll find a popular cafe 50 meters away."

[2040] "The road is in good condition at this point, so please be careful."

[2041] Prompt Sentence Examples

[2042] "Users often search for cafes. Please generate a short sentence like, 'Turn right at the next intersection and there's a popular cafe 50 meters away.'"

[2043] In this way, the system of the present invention analyzes road conditions in real time through data communication between the terminal and the server, and provides useful information to drivers. In addition, by using a generative AI model, it is possible to provide highly accurate information for autonomous vehicles.

[2044] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2045] Step 1:

[2046] The terminal acquires video information of roads using a camera means. At the same time, it also acquires GPS location information using a positioning means. The input data obtained from this is video captured in real time and location information of the location where the video was taken. This video information and location information are sent to a server via a communication means.

[2047] Step 2:

[2048] The server receives the video information and location information sent from the device. It passes this received data to the analysis means. The analysis means analyzes the video information using a trained artificial intelligence model (for example, using the OpenCV library) and extracts information such as road conditions and traffic signs. The data is processed based on the analysis results and organized for storage in a database in a clear format.

[2049] Step 3:

[2050] The server's data storage means registers the organized analysis results in a database and updates the information as necessary. This process ensures that the latest road conditions and traffic sign data is always available for use in other processes.

[2051] Step 4:

[2052] The server uses a selection means to analyze the user's location information and interest information (e.g., past search history) and select facility information that is likely to be of interest to the user. The selected information becomes useful information tailored to the user's current location and driving route. This process uses the user's location information as input data and outputs a list of facilities of interest.

[2053] Step 5:

[2054] The server uses a text generation means to generate the information selected by the selection means and the analysis results in short sentence format. Utilizing a generative AI model, it creates a short sentence such as, "Turn right at the next intersection and you'll find a popular cafe 50 meters away." The input data for this generation process is the selected facility information and the analysis results, and the output data is short sentence information to be provided to the user.

[2055] Step 6:

[2056] The server then uses the communication means to send the generated short message back to the terminal. The terminal then displays the received message to the user via a display. For example, a message such as "Turn right at the next intersection and you'll find a popular cafe 50 meters away" may be displayed on the smartphone screen.

[2057] Step 7:

[2058] At the same time, the evaluation means in the server evaluates the user's driving skills and calculates the driving risk based on the information obtained by the analysis means. For the evaluation, the number of sudden brakings, the frequency of speeding, etc. are quantified, and the driving risk is calculated based on this. In this way, the user's driving behavior is evaluated as a number, and further output as a driving risk.

[2059] Step 8:

[2060] The server transmits the driving skill evaluation results and driving risk calculated by the evaluation means to the user terminal via communication means. The terminal provides these evaluation results and risk information to the user via display means. For example, the smartphone screen may display "Driving score: 85 (good), Risk: low."

[2061] Through these steps, the system analyzes road conditions in real time, provides useful information to drivers, and evaluates driving skills and calculates risks.

[2062] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2063] The system of the present invention is mainly composed of a terminal, a server, and a user, and also has an emotion engine built in. The program of this system and its processing will be explained in detail below.

[2064] 1. Data Acquisition and Transmission

[2065] Device:

[2066] The terminal activates the camera means and captures road images in real time through the windshield of the car. At the same time, it acquires GPS data using the location information acquisition means. This data is then transmitted to the server as a data packet by the transmission means.

[2067] Examples:

[2068] While driving, the user launches the app and points the smartphone camera at the windshield. The camera captures the road ahead, and the app sends the video in real time to a server. At the same time, the GPS sensor obtains the car's current location, and that information is also sent to the server.

[2069] 2. Video analysis and database update

[2070] server:

[2071] The server receives the transmitted video information and location information. The server analyzes the video information using analysis means and extracts information such as road surface conditions, road width, and traffic signs. The analysis results are stored in a database by database management means and updated as necessary.

[2072] Examples:

[2073] The server analyzes the video it receives and extracts information such as "The road at this point is worn out" or "There is a pedestrian-only signal at the next intersection." This information is recorded in a database and used for future navigation.

[2074] 3. Selection of Interest Information

[2075] server:

[2076] Next, the server uses a selection means to analyze the user's location information and past search history to select information on stores and facilities that the user may be interested in. This selected information is provided according to the user's driving route.

[2077] Examples:

[2078] If the user has frequently searched for cafes in the past, the server will provide information such as "There is a popular cafe 500 meters ahead."

[2079] 4. Short sentence generation and sending

[2080] server:

[2081] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[2082] Examples:

[2083] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's smartphone.

[2084] 5. Display of Information

[2085] Device:

[2086] The terminal provides the received data to the user using a display means, thereby enabling the user to obtain useful information in real time.

[2087] Examples:

[2088] The device screen will display a message saying, "Turn right at the next intersection and you'll find a recommended bakery," and the user can drive according to that information.

[2089] 6. Driving skill evaluation and risk calculation

[2090] server:

[2091] The server evaluates the user's driving skills based on the video and location information acquired using the analysis means. The evaluation means quantifies the driving skills and calculates the driving risk based on the quantified data. The evaluation results are sent to the user's device.

[2092] Examples:

[2093] The server measures the number of sudden braking and the frequency of speeding, generates an evaluation such as "Driving score 85 (good), low risk," and sends it to the user's terminal.

[2094] 7. Display of evaluation results

[2095] Device:

[2096] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[2097] Examples:

[2098] The device screen displays the rating "Driving score: 85 (good), risk: low," allowing users to visually check their driving skills.

[2099] 8. User Emotion Recognition

[2100] Device:

[2101] The device captures a user's face in real time using a built-in or external camera. The emotion engine analyzes facial features and classifies the user's emotion. The classified emotion information is then sent to the server as a data packet.

[2102] Examples:

[2103] While the user is driving, a camera captures their face, and AI analyzes their emotions, such as "happiness," "anger," and "sadness," in real time. The results are then sent to a server.

[2104] 9. Emotion-based information regulation

[2105] server:

[2106] The server adjusts the content and display method of the information it provides based on the emotional information received from the emotion engine. For example, if the user is feeling stressed, it will prioritize providing information about places where they can relax or change their mood.

[2107] Examples:

[2108] If the server determines through emotion analysis that the user is tired, it provides information such as, "There is a parking area up ahead where you can rest."

[2109] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[2110] The processing flow will be explained below.

[2111] Step 1:

[2112] Device:

[2113] The camera means is activated and captures road images in real time through the windshield of the vehicle. The images are temporarily saved as video files. At the same time, GPS data is acquired using the location information acquisition means.

[2114] Step 2:

[2115] Device:

[2116] The captured video file and the acquired location information are combined into a single data packet, which contains the video data with a timestamp and GPS coordinates, and then transmitted to a server using a transmission means.

[2117] Step 3:

[2118] server:

[2119] Receives data packets sent from the device. The received data packets are stored in storage, and video information and location information are separated and extracted.

[2120] Step 4:

[2121] server:

[2122] Video information is processed using analytical means, specifically, image recognition algorithms are used to analyze and extract information such as road surface conditions (wear, potholes), road width, and traffic signs (speed limits, caution signs).

[2123] Step 5:

[2124] server:

[2125] The analysis results obtained by the analysis means are registered and updated in an existing database by the database management means, and road conditions and related information corresponding to specific location information are stored.

[2126] Step 6:

[2127] server:

[2128] Based on the analyzed location information and past search history, the server uses a selection method to select information about stores and facilities that the user is likely to be interested in. The selected information is also reflected in the navigation system.

[2129] Step 7:

[2130] server:

[2131] The selected information and the analysis results are converted into text in a short sentence format that is easy for the user to understand using a sentence generation means. After generating the short sentences, the information is again transmitted to the user terminal as a data packet by a transmission means.

[2132] Step 8:

[2133] Device:

[2134] The text information is extracted from the received data packet and provided to the user using a display means, allowing the user to check the information displayed in real time on the application screen.

[2135] Step 9:

[2136] server:

[2137] The system evaluates the user's driving skills based on video and location information acquired using analytical tools. Specifically, it measures indicators such as speeding, sudden braking, and smooth driving, and quantifies the driving skills.

[2138] Step 10:

[2139] server:

[2140] The driving skill value and driving risk assessment calculated by the assessment means are again compiled into a data packet as information to be provided to the user, and are transmitted to the user terminal using the transmission means.

[2141] Step 11:

[2142] Device:

[2143] The terminal provides the user with the evaluation results of driving skills and driving risks using a display means, allowing the user to objectively grasp their own driving skills.

[2144] Step 12:

[2145] Device:

[2146] The device then captures a real-time image of the user's face using a built-in or external camera. The emotion engine analyzes the facial features and classifies the user's emotion. This classified emotion information is then sent to the server as a data packet.

[2147] Step 13:

[2148] server:

[2149] The server analyzes the emotion information received from the emotion engine and determines the emotional state the user is in. For example, if the user is feeling stressed or tired, that emotion information is sent to the server.

[2150] Step 14:

[2151] server:

[2152] The emotion engine recognizes the user's emotions and adjusts the content and display method of the information provided accordingly. For example, if it determines that the user is feeling stressed, it will prioritize providing information about places to relax and rest areas.

[2153] Step 15:

[2154] Device:

[2155] The terminal again receives the personalized information sent from the server and provides it to the user using the display means, allowing the user to obtain optimal information in real time according to their current emotional state.

[2156] In this way, the system of the present invention provides useful information to the driver in real time, supporting the improvement of driving skills and safe driving. Furthermore, by incorporating an emotion engine, it becomes possible to provide personalized information according to the user's emotional state, further enhancing driving comfort.

[2157] Example 2

[2158] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2159] Conventional driving assistance systems have difficulty obtaining real-time road conditions and location information and providing specific navigation based on that information. They also lack sufficient capabilities for evaluating driving skills and calculating driving risks, and do not provide information that takes into account the user's emotional state. To solve these issues, a more advanced and integrated system is needed.

[2160] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2161] In this invention, the server includes: a camera for capturing video information; a location information sensor for capturing location information; a communication device for transmitting information obtained from the camera and location information sensor to a central processing unit; an analysis device for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs; a database management device for registering and updating the information obtained by the analysis device in a database; a selection device for selecting location information likely to be of interest to a user based on the location information; a sentence generation device for generating the information selected by the selection device and the analysis results in short sentences; a communication device for transmitting the information generated by the sentence generation device to a user terminal; an evaluation device for evaluating driving skills and calculating driving risks based on the information obtained by the analysis device; a communication device for transmitting the results calculated by the evaluation device to a user terminal; a display device for displaying information to a user using the communication device; a processing device including a camera for recognizing user emotions and an emotion analysis engine; and an information adjustment device for appropriately adjusting information based on the emotions recognized by the emotion analysis engine. This enables real-time information provision, driving skill evaluation, and personalized information provision according to user emotions.

[2162] The "capture means" is a device for capturing images of the road from inside the vehicle in real time, and is usually a device including a camera.

[2163] The "location information sensor means" is a device for acquiring the current location of the vehicle, and is usually a device including a GPS sensor.

[2164] The "communication means" is an interface for transmitting acquired data to a destination, and is a device that includes a wired or wireless communication protocol.

[2165] The "analysis device" is a device that analyzes the received video data and extracts information such as road surface conditions, road width, and traffic signs.

[2166] The "database management device" is software or hardware for registering and updating information obtained by the analysis device in a database.

[2167] The "selection device" is a device for selecting location information that is likely to interest a user based on the user's location information and past history.

[2168] A "sentence generation device" is software or hardware for generating selected information and analysis results in short sentence format.

[2169] The "evaluation device" is a device for evaluating driving skills based on video data and location information and calculating driving risks.

[2170] A "display device" is a device for displaying information on a user terminal, typically including a display or screen.

[2171] An "emotion analysis engine" is software or hardware that analyzes the features of a user's face and classifies the user's emotions.

[2172] An "information adjustment device" is a device that adjusts the content and display method of information provided based on emotions recognized by an emotion analysis engine.

[2173] The system of the present invention is implemented by a combination of a terminal, a server, and a user. A specific embodiment of the system will be described in detail below.

[2174] Device behavior

[2175] The terminal is installed inside the vehicle. It performs the following operations using a camera and GPS sensor. The camera captures real-time images of the road through the vehicle's windshield, and the GPS sensor acquires the vehicle's current location. This allows real-time video and location information to be obtained.

[2176] Examples:

[2177] When a user launches the application in a vehicle, the device's camera starts capturing images of the road ahead, and the GPS sensor continuously acquires the vehicle's current location. This video and location information is then formed into a data packet and sent to the server.

[2178] Server Operation

[2179] The server acts as a central processing unit and receives data sent from the terminals, performs multiple analyses and data processing. Specifically, it performs the following operations using an analysis device, a database management device, a selection device, a sentence generation device, an evaluation device, and a sentiment analysis engine.

[2180] Video Analysis

[2181] The server's analysis device analyzes the received video data and extracts information such as road surface conditions, road width, traffic signs, etc. This information is stored in a database using a database management device and updated as necessary.

[2182] Examples:

[2183] By analyzing the video data, information such as "The road at this point is worn out" and "There is a pedestrian-only traffic light at the next intersection" is extracted and registered in a database.

[2184] Selection of information of interest

[2185] The selection device of the server selects information (for example, stores and facilities) that is likely to interest the user based on the user's location information and past history.

[2186] Examples:

[2187] If the user has a history of searching for cafes in the past, the server will select and provide information such as "There is a popular cafe 500 meters ahead."

[2188] Short sentence generation

[2189] The server's sentence generation device generates the selected information and analysis results in an easy-to-understand short sentence format and transmits them again to the terminal as a data packet.

[2190] Examples:

[2191] The server generates a short sentence such as "Turn right at the next intersection and you'll find a recommended bakery" and sends it to the user's terminal.

[2192] Assessment and Risk Calculation

[2193] The server evaluates driving skills and calculates driving risk. The evaluation device calculates a driving score based on the number of sudden braking and speeding, and performs risk assessment.

[2194] Examples:

[2195] The evaluation result "Driving score 85 (good), low risk" is generated and sent to the user's terminal.

[2196] Emotion recognition and information regulation

[2197] The device's camera and emotion analysis engine are used to analyze the user's emotions in real time. The emotion analysis engine classifies the emotions and sends them to the server. The server then adjusts the content and display method of the information based on this emotion information.

[2198] Examples:

[2199] If the emotion analysis engine determines that the user is feeling tired, the server will provide information such as, "There is a parking area up ahead where you can take a rest."

[2200] Adjusted short sentence generation

[2201] The adjusted information is then converted back into short sentences and sent to the device, enabling personalized information to be provided according to the user's emotional state.

[2202] Prompt Sentence Examples

[2203] Below is an example of a prompt that the server might input to the generative AI model:

[2204] "If you turn right at the next intersection, there's a cafe I recommend."

[2205] "Driving score: 85 (good), risk: low"

[2206] "There's a parking area up ahead where you can rest."

[2207] In this way, the entire system works together to provide real-time information and driving evaluation, as well as personalized information based on emotions.

[2208] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2209] Processing Steps

[2210] Step 1: Data Acquisition and Transmission

[2211] Device:

[2212] The device activates the camera and GPS sensor to capture real-time road images through the vehicle's windshield. At the same time, the GPS sensor acquires the device's current location. Based on this, the device assembles the image data and location information into a data packet and sends it to the server.

[2213] Specific behavior:

[2214] 1. The device will activate the built-in camera.

[2215] 2. The camera captures the road scenery ahead.

[2216] 3. The device activates its built-in GPS sensor and acquires location information.

[2217] 4. The video data and location information are packaged in a packet format and sent to the server.

[2218] Input: Camera footage, GPS location information

[2219] Output: Data packet (video data + location information)

[2220] Step 2: Video analysis

[2221] server:

[2222] The server processes the received data packets (video data and location information) using an analysis device to extract necessary information such as road surface conditions, traffic signs, road width, etc. The analysis results are then registered in a database and updated as necessary.

[2223] Specific behavior:

[2224] 1. The server receives the data packet.

[2225] 2. The analysis device analyzes the video data and extracts data such as road wear conditions and traffic signs.

[2226] 3. The extracted data is registered and updated in the database by the database management device.

[2227] Input: Data packet (video data + location information)

[2228] Output: Extracted information (road conditions, traffic signs, etc.)

[2229] Step 3: Selecting information of interest

[2230] server:

[2231] The server uses the user's current location and past search history to select information that is likely to be of interest to the user.

[2232] Specific behavior:

[2233] 1. The server references the location information and the user's search history.

[2234] 2. The selection device applies an algorithm to select stores and facility information that may be of interest.

[2235] Input: User's current location, past search history

[2236] Output: Information that may be of interest to the user

[2237] Step 4: Create and send a short message

[2238] server:

[2239] The server generates text in short sentence format based on the selected information and the analysis results, and transmits it to the user terminal as a data packet.

[2240] Specific behavior:

[2241] 1. The server retrieves the selected information.

[2242] 2. A sentence generator generates text in short sentence format.

[2243] 3. The generated short sentence is sent to the user terminal as a data packet.

[2244] Input: Selection information, analysis results

[2245] Output: Short text

[2246] Step 5: Viewing information

[2247] Device:

[2248] The terminal displays the received data to the user.

[2249] Specific behavior:

[2250] 1. The terminal receives a data packet sent from the server.

[2251] 2. Decode the received information and display it on the screen.

[2252] Input: Data packet (short text format)

[2253] Output: The displayed information (display)

[2254] Step 6: Driving skill assessment and risk calculation

[2255] server:

[2256] The server uses video data and location information to evaluate driving skills and calculate driving risks.

[2257] Specific behavior:

[2258] 1. The server obtains the video data and location information.

[2259] 2. The evaluation device evaluates driving skills based on the number of sudden braking and speeding.

[2260] 3. The calculated driving risk is quantified and an evaluation result is generated.

[2261] Input: Video data, location information

[2262] Output: Driving skill evaluation results, driving risk

[2263] Step 7: View the evaluation results

[2264] Device:

[2265] The terminal displays the driving skill evaluation results sent from the server.

[2266] Specific behavior:

[2267] 1. The terminal receives the evaluation result.

[2268] 2. The received evaluation results are displayed on the screen.

[2269] Input: Evaluation result, driving risk

[2270] Output: Displayed evaluation results (display)

[2271] Step 8: Recognizing User Emotions

[2272] Device:

[2273] The device uses a built-in or external camera to capture a picture of the user's face and classifies the emotion using an emotion analysis engine.

[2274] Specific behavior:

[2275] 1. The device activates the camera and takes a picture of the user's face.

[2276] 2. The emotion analysis engine analyzes facial feature points and classifies emotions. ...

Claims

1. camera means for acquiring video information; location information acquisition means for acquiring location information; a transmitting means for transmitting the information obtained from the camera means and the position information obtaining means to a server; an analysis means for analyzing the video information and extracting information such as road surface conditions, road width, and traffic signs; a database management means for registering and updating the information obtained by the analysis means in a database; a selection means for selecting store information that is likely to be of interest to the user based on the location information; a sentence generation means for generating the information and analysis results selected by the selection means in a short sentence format; a transmitting means for transmitting the information generated by the sentence generating means to a user terminal; evaluation means for evaluating driving skills and calculating driving risks based on the information obtained by the analysis means; a transmitting means for transmitting the result calculated by the evaluating means to a user terminal; a display means for displaying information to a user using the transmission means; A system including:

2. The system according to claim 1 , wherein the evaluation means includes an algorithm for quantifying driving skills and calculating driving risks.

3. The system according to claim 1 , wherein the analysis means uses a trained artificial intelligence model generated for analyzing video information.

Images