system

Abstract

Provide a system. [Solution] A means of collecting driving data on older drivers; means for analyzing the driving data to detect abnormal driving patterns; means for assessing driving risk based on the abnormal driving pattern; means for generating an alert based on the driving risk assessment; means for notifying a driver of the warning; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Create it along the mat.

[0005] As elderly people's driving abilities decline with age, they are unable to accurately assess their own driving risks, increasing the risk of traffic accidents. In particular, when family members or other concerned parties advise elderly drivers to surrender their licenses, the process often does not go smoothly due to emotional conflicts and a lack of understanding. Furthermore, due to a lack of means to quantitatively assess actual driving risks, it is difficult to appropriately determine the timing of license surrender. This threatens the safety of elderly drivers and other road users. [Means for solving the problem]

[0006] To solve the above-mentioned problems, the present invention provides the following means. Specifically, the system includes a means for collecting driving data of an elderly driver, a means for analyzing the driving data to detect abnormal driving patterns, a means for assessing driving risk based on the abnormal driving patterns, a means for generating a warning based on the driving risk assessment, and a means for notifying the driver of the warning. Furthermore, by including a means for setting a warning level and issuing different types of warnings for medium-risk and high-risk situations, the system can encourage the driver to drive appropriately. Furthermore, by including a means for collecting driving data in real time and transmitting the driving data to a server in real time, the system can ensure immediacy or promptness while driving. By further including a means for performing noise removal, normalization, and missing value imputation on the driving data using preprocessing means, the system can perform highly reliable data analysis. By including a means for generating monthly reports and providing past driving trends to the driver and their family, the system can perform long-term driving evaluation. Furthermore, by including a means for displaying a driving risk assessment in numerical form and notifying family members to assist elderly drivers in surrendering their licenses, the system can quantitatively and objectively determine the timing of license surrender.

[0007] Below are definitions of important terms included in the claims.

[0008] "Driving data" refers to information obtained while an older driver is driving, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0009] "Abnormal driving patterns" are specific patterns that differ from normal driving behavior and refer to driving behavior that may pose risks, such as unstable speeds on highways, frequent sudden braking, and erratic driving.

[0010] "Driving risk" is a calculated number or rating based on abnormal driving patterns and other driving data that indicates a driver's likelihood of causing an accident.

[0011] "Warnings" are notifications or alarms generated based on driving risk assessments and intended to prompt the driver's immediate attention.

[0012] "Risk assessment" is the process of analyzing collected driving data and quantifying driving risk using specific algorithms.

[0013] "Preprocessing" refers to the process of removing noise from collected raw data, normalizing it, filling in missing values, and performing other operations to improve the reliability and consistency of the data.

[0014] The "Monthly Report" is a report that compiles and analyzes driving data from the past month and provides driving trends and driving risk assessments to drivers and their families.

[0015] "License surrender" is the process by which older drivers voluntarily give up their driver's license based on their driving ability and driving risk assessment.

[0016] "Real-time data collection" is a procedure for instantly acquiring driving data while driving and transmitting it to a server in order to instantly monitor driving behavior.

[0017] The "warning level" is a level of caution that is set based on the results of an evaluation of driving risk, and there are different levels: low risk, medium risk, and high risk. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This system monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[0040] Server Operation

[0041] Data collection

[0042] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0043] Data Preprocessing

[0044] The server performs pre-processing on the received driving data, which includes the following operations:

[0045] Noise removal: Removing unwanted noise from collected data.

[0046] Normalization: Transforming data values ​​onto a uniform scale.

[0047] Missing Values: Properly impute missing data.

[0048] Data analysis

[0049] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0050] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0051] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0052] Risk assessment: Quantify the driving risk based on the analysis results.

[0053] Warning generation

[0054] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[0055] Device behavior

[0056] Real-time data collection and transmission

[0057] The terminal (on-board computer) collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors.

[0058] Warning display

[0059] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0060] User Actions

[0061] Check your driving rating

[0062] Users (drivers and their families) can check warnings and regular reports from their devices and servers, allowing them to accurately understand their own driving risks.

[0063] Behavioral adjustment

[0064] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0065] Considering returning your license

[0066] The user (family member) considers the timing of when the elderly driver should return their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license return.

[0067] Specific examples

[0068] Example 1: Elderly driver driving on the highway

[0069] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[0070] Example 2: Family members checking monthly reports

[0071] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[0072] This invention makes it possible to quantitatively evaluate the driving risk of elderly drivers and issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[0073] The processing flow will be explained below.

[0074] Understood. Below, the program processing will be explained in detail by dividing it into steps.

[0075] Server Processing

[0076] Step 1:

[0077] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0078] Step 2:

[0079] The server performs pre-processing on the received driving data, which includes the following operations:

[0080] Noise removal: Removing unwanted noise from collected data.

[0081] Normalization: Transforming data values ​​onto a uniform scale.

[0082] Missing Values: Properly impute missing data.

[0083] Step 3:

[0084] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0085] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0086] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0087] Risk assessment: Quantify the driving risk based on the analysis results.

[0088] Step 4:

[0089] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[0090] Step 5:

[0091] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[0092] Terminal handling

[0093] Step 1:

[0094] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0095] Step 2:

[0096] The device transmits the collected data to the server in real time.

[0097] Step 3:

[0098] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0099] User Action

[0100] Step 1:

[0101] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[0102] Step 2:

[0103] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0104] Step 3:

[0105] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the assessment results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license surrender.

[0106] The above are the specific processing steps of the program.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] The decline in driving ability of elderly drivers is a factor that increases the risk of traffic accidents. Current driving risk assessment systems lack preprocessing capabilities for high-precision analysis and have difficulties in collecting data and assessing risk in real time. Therefore, there is a need for a system that can accurately assess the driving behavior of elderly drivers and provide appropriate warnings according to their driving risk.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for collecting driving data of elderly drivers, means for preprocessing the driving data, means for analyzing the preprocessed driving data to detect abnormal driving patterns, means for assessing driving risks based on the abnormal driving patterns, means for generating warnings based on the driving risk assessment, and means for notifying the drivers of the warnings, thereby enabling real-time monitoring of elderly drivers' driving behaviors, evaluating driving risks, and providing appropriate warnings.

[0112] "Driving data" refers to information about older drivers' driving behavior and vehicle conditions, such as vehicle speed, braking, steering, lane keeping, and environmental data.

[0113] "Preprocessing" refers to a series of data processing operations carried out to make collected driving data easier to analyze, including, for example, noise removal, data normalization, and missing value completion.

[0114] "Abnormal driving patterns" refer to patterns that detect deviations from normal driving behavior, such as sudden braking or swerving.

[0115] "Driving risk" refers to a quantitative assessment of the degree of danger posed by a driver's driving behavior. For example, it is assessed on a three-level scale: low risk, medium risk, and high risk.

[0116] "Warning" refers to a message or alarm that notifies the driver of a driving risk and alerts them to the risk, including, for example, a text message or an audio alarm.

[0117] "Real-time" refers to data collection and processing occurring almost instantaneously, with no delay, allowing for immediate analysis and alerts.

[0118] "Server" refers to the computer system that receives, pre-processes, analyzes, assesses risk, and generates alerts on driving data.

[0119] "Terminal" refers to a device that collects driving data, transmits it to a server, and notifies the driver of warnings, including, for example, an in-vehicle computer.

[0120] "Driver" refers to an elderly person who drives a vehicle.

[0121] "Notification means" refers to the method of conveying a warning message or alarm to the driver, including, for example, display or audio output.

[0122] MODE FOR CARRYING OUT THE INVENTION

[0123] The present invention is a system that monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[0124] Server Operation

[0125] Data collection

[0126] The server receives real-time driving data sent from the device. Driving data includes vehicle speed, braking, steering, lane keeping, and environmental data. This information is collected from the device's speed, brake, and steering sensors and sent to the server via WiFi. Specifically, the server uses a web framework such as "Flask" to build an API endpoint to receive the data.

[0127] Data Preprocessing

[0128] The server converts the received driving data into a data frame format and performs the following pre-processing.

[0129] Noise removal: Filtering unwanted noise from data using the "SciPy" library.

[0130] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[0131] Missing value handling: Use Python's "pandas" library to impute missing data with the mean value.

[0132] This prepares the data in a form suitable for analysis.

[0133] Data analysis

[0134] The server uses the preprocessed data to perform the following analysis:

[0135] Anomaly detection: Uses an "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[0136] Pattern Recognition: Uses "k-means clustering" to compare and identify normal and abnormal driving patterns.

[0137] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[0138] These algorithms are implemented in the "scikit-learn" library.

[0139] Warning generation

[0140] The server generates appropriate warnings based on the driving risk assessment, with three levels of warning:

[0141] Low risk: Generates a text warning message.

[0142] Medium risk: Generates audio alarm instructions in addition to text messages.

[0143] High risk: Generates a loud audio alarm and an emergency stop instruction.

[0144] The generated alert is sent to the terminal.

[0145] Device behavior

[0146] Real-time data collection and transmission

[0147] The device collects driving data in real time and sends it to a server. Data is collected using an on-board computer such as a Raspberry Pi, and data is acquired through sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0148] Warning display

[0149] The terminal notifies the driver of the warnings received from the server. If the risk is medium, a warning message is displayed on the terminal's display, and if the risk is high, an audio alarm is sounded. The actual notification is performed using an LCD display and speaker, and these output devices are controlled using the GPIO pins of Arduino or Raspberry Pi.

[0150] User Actions

[0151] Check your driving rating

[0152] Users (drivers and their families) can accurately understand their own driving risks by checking warnings and regular reports from their devices and servers. The regular reports include past driving data and its analysis results.

[0153] Behavioral adjustment

[0154] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0155] Considering returning your license

[0156] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, a family meeting is held to encourage the driver to surrender their license.

[0157] Specific examples

[0158] Example 1: Elderly driver driving on the highway

[0159] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[0160] Example 2: Family members checking monthly reports

[0161] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[0162] Examples of prompt statements

[0163] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[0164] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0165] Server Operation

[0166] Step 1: Data collection

[0167] Input: Driving data sent from the device (vehicle speed, braking, steering, lane keeping, environmental data)

[0168] Processing: The server receives these data in real time.

[0169] Output: Raw driving data

[0170] Specific operation: The server uses a web framework such as "Flask" to build an API endpoint to receive data, and receives it from the device via WiFi.

[0171] Step 2: Data Preprocessing

[0172] Input: Raw driving data

[0173] Processing: Preprocess the data in the following ways:

[0174] Denoising: Filtering unwanted noise from data using the "SciPy" library.

[0175] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[0176] Missing value handling: Impute missing data with the mean value using the "pandas" library.

[0177] Output: Preprocessed data

[0178] Specific operation: The server converts the data into a data frame format and applies each preprocessing operation sequentially to prepare the data.

[0179] Step 3: Data analysis

[0180] Input: Preprocessed data

[0181] Processing: Analysis using machine learning algorithms:

[0182] Anomaly detection: Uses the "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[0183] Pattern recognition: Using "k-means clustering" to compare normal and abnormal driving patterns.

[0184] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[0185] Output: Driving risk assessment results

[0186] What it does: The server uses the "scikit-learn" library to implement these algorithms, analyze the data, and generate a risk assessment result.

[0187] Step 4: Generate alerts

[0188] Input: Driving risk assessment results

[0189] Action: Generate a warning message depending on the driving risk:

[0190] Low risk: Generates a text warning message

[0191] Medium risk: Generates audio alarm instructions in addition to text messages

[0192] High risk: Generates a loud audio alarm and an emergency stop instruction

[0193] Output: Warning message

[0194] Specific operation: Based on the evaluation result, the server generates an appropriate warning message and sends it to the terminal.

[0195] Device behavior

[0196] Step 1: Real-time data collection and transmission

[0197] Input: Data from vehicle speed sensor, brake sensor, and steering sensor

[0198] Processing: Data is collected and sent to a server via WiFi.

[0199] Output: Driving data to be transmitted

[0200] How it works: An onboard computer such as a Raspberry Pi collects data from various sensors and sends it to a server.

[0201] Step 2: Warning display

[0202] Input: The warning message sent by the server

[0203] Action: Notify the driver with a warning message:

[0204] Medium risk: A warning message appears on the display

[0205] High risk: sound an audio alarm

[0206] Output: Warnings notified to the driver

[0207] What it does: The device displays messages on the LCD display and plays audio alarms through the speaker. It uses the GPIO pins of Arduino and Raspberry Pi to control these devices.

[0208] User Actions

[0209] Step 1: Check your driving rating

[0210] Input: Alert messages and scheduled reports from terminals and servers

[0211] Action: Check the driving evaluation results

[0212] Output: Check the driving risk assessment results

[0213] Specific operation: The user checks the evaluation report on the device display or through the smartphone app.

[0214] Step 2: Adjust your behavior

[0215] Input: The warning message received

[0216] Action: Adjust driving behavior based on the warning

[0217] Output: Adjusted driving behavior

[0218] Specific actions: Drivers will immediately improve their driving behavior and drive more cautiously.

[0219] Step 3: Consider surrendering your license

[0220] Input: Monthly reports and historical driving data

[0221] Processing: Consider the timing of license surrender based on the evaluation results

[0222] Output: Decision to surrender license if necessary

[0223] Specific actions: Hold a family meeting and consider surrendering the driver's license if the high risk assessment continues.

[0224] Examples of prompt statements

[0225] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[0226] (Application example 1)

[0227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0228] Monitoring driving risks and ensuring safety for elderly drivers are important social issues. In particular, when elderly drivers use self-driving vehicles, systems that assess driving risks and provide warnings or driving assistance as necessary are needed. Conventional technologies have struggled to effectively assess these risks in real time and provide appropriate warnings or assistance. Furthermore, there has been insufficient coordination between advanced analysis using generative AI models and prompt sentences and the self-driving system. Therefore, it is necessary to develop a system that can accurately assess the driving risks of elderly drivers and provide appropriate warnings and assistance.

[0229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0230] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for assessing driving risk based on the abnormal driving patterns, means for generating a warning based on the driving risk assessment, means for notifying the driver of the warning, means for the automated driving system to take over driving operations as necessary based on the driving risk, means for analyzing the driving data using a generative AI model to assess risk and generate a warning, and means for configuring the driving risk and warning generation process based on prompt sentences. This makes it possible to assess the driving risk of elderly drivers in real time, detect abnormal driving patterns, and provide appropriate warnings and driving assistance.

[0231] An "elderly driver" refers to a driver who is over a certain age and for whom there are concerns about the risks associated with age when driving.

[0232] "Driving Data" refers to a range of data collected while driving, such as vehicle speed, braking usage, steering angle, lane position and environmental data.

[0233] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking or swerving.

[0234] "Driving risk" is an indicator that indicates the likelihood that a driver will cause an accident while driving, and includes an evaluation value that is quantified through data analysis.

[0235] "Warning" refers to a warning message or alarm that is sent to the driver when driving risk exceeds a certain level.

[0236] "Autonomous driving system" refers to technology that enables a vehicle to perform driving operations without driver intervention.

[0237] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and makes predictions and analyses on new data.

[0238] A "prompt" is a piece of text that serves as an instruction input to a generative AI model.

[0239] MODE FOR CARRYING OUT THE INVENTION

[0240] This paper describes an embodiment of an "elderly driver safety support system" for improving safety when elderly drivers use automated driving vehicles. The system of the present invention is mainly composed of four elements: a server, an in-vehicle terminal, an automated driving system, and a user.

[0241] Server Operation

[0242] Data collection

[0243] The server receives driving data transmitted in real time from the in-vehicle device. This driving data includes vehicle speed, braking, steering, lane position, and environmental data. For example, data is collected using vehicle speed sensors, brake sensors, steering sensors, cameras, and various environmental sensors.

[0244] Data Preprocessing

[0245] The server performs pre-processing on the received driving data, which includes the following operations:

[0246] Noise removal: Removing unwanted noise from collected data.

[0247] Normalization: Transforming data values ​​onto a uniform scale.

[0248] Missing Values: Properly impute missing data.

[0249] Data analysis

[0250] The server uses the generative AI model to analyze the preprocessed data. This analysis process includes the following steps:

[0251] Anomaly detection: Uses algorithms such as Isolation Forest to detect sudden behavior (hard braking, swerving, etc.).

[0252] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0253] Risk assessment: Quantify the driving risk based on the analysis results.

[0254] Warning generation

[0255] The server generates warnings based on the driving risk assessment. There are three warning levels: low risk, medium risk, and high risk. Different types of warnings are generated for each level. For example, a medium risk warning message is displayed on the screen, and a high risk warning sounds an audio alarm.

[0256] In-vehicle terminal operation

[0257] Real-time data collection and transmission

[0258] The in-vehicle device collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors. The data collected in real time is sent to the server using wireless communication technology (e.g., Wi-Fi or 5G).

[0259] Warning display

[0260] The in-vehicle device receives the risk assessment results from the server and notifies the driver. If the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0261] Autonomous driving system operation

[0262] Driving assistance measures

[0263] The automated driving system automatically performs driving operations based on the driving risk assessment received from the server. For example, if the risk is high, the system takes over control of the vehicle and drives safely.

[0264] User Actions

[0265] Check your driving rating

[0266] Users (drivers and their families) can check warnings and regular reports from the in-vehicle terminal or server, allowing them to accurately understand their own driving risks.

[0267] Behavioral adjustment

[0268] The user (driver) adjusts their driving behavior according to the warning. If they receive a medium-risk warning, they will drive more carefully, and if they receive a high-risk warning, they will refrain from driving. For example, the server can generate a warning message saying, "Be careful. Your speed is unstable," and the device can notify the driver of this.

[0269] Specific examples

[0270] An example of a prompt sentence to input to the generative AI model is as follows:

[0271] "Assess driving risks in real time based on driving data of elderly drivers. Data includes vehicle speed, braking, steering, lane keeping, and environmental data. Detect abnormal patterns, generate risk scores, and generate appropriate warning messages."

[0272] This system makes it possible to assess the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] Step 1:

[0275] The server receives driving data in real time from the in-vehicle terminal. The input driving data includes vehicle speed, braking, steering operation, lane position, and environmental data. This data is transmitted to the server using wireless communication technology (e.g., Wi-Fi or 5G). The received driving data is stored in data storage and used for subsequent processing.

[0276] Step 2:

[0277] The server performs preprocessing on the received driving data. The input for this step is the data collected in step 1. First, noise removal is performed, and then data normalization is performed. Noise removal involves filtering out sensor errors and unnecessary data points. Data normalization involves converting data values ​​to a unified scale to facilitate comparison between different sensors. Finally, missing data is imputed appropriately. Specifically, SimpleImputer is used to impute missing values ​​with the mean value. This preprocessing results in a dataset that can be analyzed.

[0278] Step 3:

[0279] The server uses the generative AI model to analyze the preprocessed data. The input for this step is the preprocessed data from step 2. First, anomaly detection is performed using the Isolation Forest algorithm, which detects abnormal driving patterns (e.g., sudden braking or swerving). Next, pattern recognition is used to identify anomalies by comparing them with general driving patterns. Finally, the server evaluates the driving risk based on the analysis results and generates a risk score. This risk score is obtained as the output.

[0280] Step 4:

[0281] The server generates a warning based on the generated driving risk score. The input for this step is the risk score obtained in step 3. The generated warnings are divided into three levels: low risk, medium risk, and high risk, and different types of warning messages are prepared for each level. Specifically, a text warning message is displayed for medium risk, and an audio alarm is sounded for high risk. This warning message is sent from the server to the in-vehicle terminal.

[0282] Step 5:

[0283] The in-vehicle terminal notifies the driver of the warning message received from the server. The input of this step is the warning message generated in step 4. If the risk is medium, a warning message is displayed on the in-vehicle terminal's display. If the risk is high, an audio alarm is used to inform the driver of the urgency. This warning notification prompts the driver to adjust their driving behavior. The output of this step is a warning notification to the driver, and the driver is expected to act in accordance with the warning content.

[0284] Step 6:

[0285] The automated driving system takes over driving operations as necessary based on the driving risk assessment results received from the server. The input to this step is the driving risk assessment result obtained in step 4. If it is deemed high risk, the automated driving system takes over control and performs appropriate driving operations, such as rapidly decelerating the vehicle or guiding it to a safe route. The output of this step is the execution of safe driving operations.

[0286] Through the above steps, the "elderly driver safety support system" of the present invention is able to evaluate the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[0287] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0288] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[0289] Server Operation

[0290] Data collection

[0291] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0292] Data Preprocessing

[0293] The server performs pre-processing on the received driving data, which includes the following operations:

[0294] Noise removal: Removing unwanted noise from collected data.

[0295] Normalization: Transforming data values ​​onto a uniform scale.

[0296] Missing Values: Properly impute missing data.

[0297] Data analysis

[0298] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0299] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0300] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0301] Risk assessment: Quantify the driving risk based on the analysis results.

[0302] Emotional Data Integration

[0303] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment, which generates a comprehensive driving risk report.

[0304] Warning generation

[0305] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[0306] Notifications and Reporting

[0307] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[0308] Device behavior

[0309] Real-time data collection and transmission

[0310] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0311] Warning display

[0312] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0313] Emotion Engine Operation

[0314] emotion recognition

[0315] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[0316] Sending emotional data

[0317] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[0318] User Actions

[0319] Check your driving rating

[0320] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[0321] Behavioral adjustment

[0322] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0323] Considering returning your license

[0324] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[0325] Specific examples

[0326] Example 1: Elderly driver driving on the highway

[0327] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[0328] Example 2: Family members checking monthly reports

[0329] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[0330] This invention quantitatively evaluates the driving risk of elderly drivers and takes into account the user's emotions, allowing for timely warnings to be issued, thereby improving the safety of elderly drivers themselves and other road users.

[0331] The processing flow will be explained below.

[0332] Server Processing

[0333] Step 1:

[0334] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0335] Step 2:

[0336] The server performs pre-processing on the received driving data, which includes the following operations:

[0337] Noise removal: Removing unwanted noise from collected data.

[0338] Normalization: Transforming data values ​​onto a uniform scale.

[0339] Missing Values: Properly impute missing data.

[0340] Step 3:

[0341] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0342] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0343] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0344] Risk assessment: Quantify the driving risk based on the analysis results.

[0345] Step 4:

[0346] The server receives the user's emotion data sent from the emotion engine and integrates it into a driving risk assessment.

[0347] Step 5:

[0348] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set as follows:

[0349] Low risk: No immediate action required.

[0350] Medium risk: Display a warning message on the form screen.

[0351] High Risk: Display audio and visual warnings.

[0352] Step 6:

[0353] The server sends the warning information to the terminal and also generates a monthly report to provide to the user (driver and family).

[0354] Terminal handling

[0355] Step 1:

[0356] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0357] Step 2:

[0358] The device transmits the collected data to the server in real time.

[0359] Step 3:

[0360] The device notifies the driver of the risk assessment results and warnings received from the server. The notification format is as follows:

[0361] Medium risk: Display a warning message on the form screen saying "Please be careful. Drive safely."

[0362] High Risk: An audio alarm will be issued and a warning message will be displayed saying "High Risk. Do not drive."

[0363] Emotion engine processing

[0364] Step 1:

[0365] The emotion engine recognizes the user's emotions in real time through facial recognition and voice analysis, collecting the user's facial expressions and tone of voice through the camera and microphone.

[0366] Step 2:

[0367] The emotion engine analyzes the collected data and assesses the user's emotional state (e.g., tension, anxiety, calmness, etc.) in real time.

[0368] Step 3:

[0369] The emotion engine sends the recognized emotion data to the server.

[0370] User Action

[0371] Step 1:

[0372] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[0373] Step 2:

[0374] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0375] Step 3:

[0376] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[0377] The above are the specific processing steps in a system that combines emotion engines.

[0378] Example 2

[0379] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] As older drivers age, their reaction time and attention span decline, increasing the risk of accidents. However, current driver assistance systems rely solely on driving data and do not take into account the driver's emotional state. This makes it difficult to properly assess the impact of driver emotions on driving behavior and issue appropriate warnings in a timely manner. Furthermore, some systems suffer from poor data preprocessing accuracy, which can lead to noise and missing values, reducing the accuracy of driving risk assessment.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0382] In this invention, the server includes a device for collecting driving data of elderly drivers, a device for analyzing the driving data to detect abnormal driving behavior, a device for evaluating driving risk based on the abnormal driving behavior, a data preprocessing device for performing noise removal, numerical normalization, and missing value completion, and a device for recognizing user emotion data and integrating it into the driving risk evaluation. This makes it possible to integrate driving data and emotion data to perform a comprehensive driving risk evaluation and issue appropriate warnings in a timely manner.

[0383] "Elderly drivers" generally refer to drivers who are older and whose driving skills and reaction times may be declining.

[0384] "Driving statistics" refers to various data collected while driving, such as vehicle speed, brake usage, steering angle, and lane keeping information.

[0385] "Equipment" means an entity of hardware or software designed and manufactured to perform a specific function.

[0386] "Abnormal driving behavior" refers to detecting driving behavior that deviates from normal driving patterns, such as sudden braking or swerving.

[0387] "Driving risk" is a quantitative or qualitative assessment of the risk level of driving behavior, and is an indicator of potential threats to safety.

[0388] A "warning" is a warning message issued to drivers that includes content encouraging safe driving.

[0389] "Denoising" refers to the process of removing unwanted noise from collected data.

[0390] "Numeric normalization" refers to the process of converting data from different scales or units to a unified scale.

[0391] "Missing value imputation" refers to the process of rationally filling in missing data for an incomplete dataset.

[0392] "Emotional values" are data that quantifies a user's emotional state and refer to information obtained through facial recognition and voice analysis.

[0393] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[0394] Server Operation

[0395] Data collection

[0396] The server receives real-time driving data from the device, including vehicle speed, brake usage, steering operation, lane keeping, and environmental data. This data is collected from various sensors (e.g., vehicle speed sensor, brake sensor, steering sensor) installed on the device.

[0397] Data Preprocessing

[0398] The server performs pre-processing on the received driving data, which includes the following operations:

[0399] Noise removal: Use the Python Pandas library to remove unwanted noise from the collected data.

[0400] Normalization: Using scaling techniques to convert data values ​​to a uniform scale.

[0401] Missing Value Support: Use Scikit-learn's SimpleImputer to properly impute missing data.

[0402] Data analysis

[0403] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0404] Anomaly detection: Use anomaly detection algorithms such as Isolation Forest to detect sudden behavior (sudden braking, swerving, etc.).

[0405] Pattern Recognition: Running deep learning models using Keras to identify anomalous driving patterns compared to typical driving patterns.

[0406] Risk assessment: Quantify the driving risk based on the analysis results.

[0407] Emotional Data Integration

[0408] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment, for example, by using OpenCV for facial expression recognition and Google® Cloud Speech-to-Text for voice analysis, and generates a comprehensive driving risk report based on this integration.

[0409] Warning generation

[0410] The server generates warnings based on driving risk assessment and emotion data. There are three warning levels: low risk, medium risk, and high risk. Different warning formats are generated for each level. For example, messages such as "Be careful," "There is a problem with your driving pattern," and "Consider an emergency stop" are generated.

[0411] Notifications and Reporting

[0412] The server sends warning information to the device and, if necessary, generates a monthly report and provides it to the user (driver and family members). For example, a warning message is displayed on the device, and an audio alarm is sounded if there is a high risk.

[0413] Device behavior

[0414] Real-time data collection and transmission

[0415] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as speed sensors, brake sensors, and steering sensors, and the collected data is sent to a server.

[0416] Warning display

[0417] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0418] Emotion Engine Operation

[0419] emotion recognition

[0420] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[0421] Sending emotional data

[0422] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[0423] User Actions

[0424] Check your driving rating

[0425] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[0426] Behavioral adjustment

[0427] The user (driver) adjusts their driving behavior based on the warnings and their own emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0428] Considering returning your license

[0429] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistently high risk, the family member will consult with the elderly driver and, taking emotional data into consideration, proceed with the license surrender.

[0430] Specific examples

[0431] Example 1: Elderly driver driving on the highway

[0432] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[0433] Example 2: Family members checking monthly reports

[0434] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[0435] Example prompt

[0436] Example of input prompt for generative AI model:

[0437] "Please explain in detail how the system works, where the server detects anomalies based on data collected from the device when an elderly driver is driving on a highway and generates a medium-risk warning."

[0438] Through these specific processes, the present invention can accurately assess the driving risks of elderly drivers and provide effective driving assistance.

[0439] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0440] Step 1: Data collection

[0441] The server receives driving data from the device in real time. The device is equipped with a vehicle speed sensor, brake sensor, steering sensor, etc., and data collected from these sensors while driving is sent to the server. Input data includes vehicle speed, brake usage, steering angle, lane keeping information, etc. The server receives this data and records it in a log. The output data is a file of the collected driving data.

[0442] Step 2: Data Preprocessing

[0443] The server performs preprocessing on the received driving data. The input data is the collected raw driving data, and the following preprocessing is performed:

[0444] Noise removal: Using the Python Pandas library, outliers and noise are removed. Specifically, this involves filtering out abnormally high vehicle speed data and inconsistent braking data.

[0445] Normalization: Use min-max scaling to convert data values ​​to a uniform scale, for example, scaling vehicle speed data to a range of 0 to 1.

[0446] Missing data handling: Using Scikit-learn's SimpleImputer, we impute missing data with the mean value. For example, we impute missing brake data with the mean value of that brake sensor.

[0447] The output data is pre-processed, clean driving data.

[0448] Step 3: Data analysis

[0449] The server uses machine learning algorithms to analyze the preprocessed data. The input data is the preprocessed driving data, and the following analysis is performed:

[0450] Anomaly Detection: Isolation Forest is used to detect anomalous behaviors such as sudden braking or erratic driving by identifying data points that deviate from normal driving patterns.

[0451] Pattern Recognition: Using Keras, we run deep learning models to analyze driving patterns, for example, learning and comparing driving patterns around sharp turns and on highways.

[0452] Risk Assessment: Generate a risk score based on the analysis results, for example, quantifying risk on a scale of 0 to 100 based on the frequency and severity of abnormal behavior.

[0453] The output data is the driving risk assessment result.

[0454] Step 4: Integrating Emotional Data

[0455] The server receives the user's emotion data recognized by the emotion engine. The input data is emotion data obtained by face recognition and voice analysis, and the following operations are performed:

[0456] Emotion Recognition: Facial expressions are analyzed using OpenCV, and voice tones are analyzed using Google Cloud Speech-to-Text. Specific operations include quantifying the user's emotions, such as tension or anxiety.

[0457] Emotional data integration: Integrate driving risk assessment results with emotional data to generate a comprehensive driving risk report. For example, reassess the overall risk by incorporating emotional scores into the risk score.

[0458] The output data is a comprehensive driving risk report.

[0459] Step 5: Generate warnings

[0460] The server generates an alert based on the comprehensive driving risk report. The input data is the comprehensive driving risk report, and the following operations are performed:

[0461] Warning level setting: Sets the warning level (low risk, medium risk, high risk) based on the risk score. Specifically, if the risk score is 50 or above, it is considered a medium risk, and if it is 80 or above, it is considered a high risk.

[0462] Warning message generation: Generates a warning message according to each risk level. For example, for low risk, it generates a message saying "Be careful," for medium risk, it generates a message saying "There is a problem with your driving pattern," and for high risk, it generates a message saying "Consider making an emergency stop."

[0463] The output data is a warning message.

[0464] Step 6: Notifications and Reporting

[0465] The server sends the generated warning messages to the terminal and also generates monthly reports if necessary. The input data are the warning messages and the analyzed driving data, and the following operations are performed:

[0466] Warning notification: A warning message is sent to the device, which then displays it to the driver. Specifically, the device displays the warning message on the screen and sounds an audio alarm if the risk is high.

[0467] Report Generation: Historical driving data is aggregated to generate monthly reports, including driving risk assessments and details of abnormal driving patterns.

[0468] The output data is a warning message and a monthly report.

[0469] Through each of the above steps, a system will be realized that evaluates the driving risks of elderly drivers and provides appropriate warnings and driving assistance.

[0470] (Application example 2)

[0471] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0472] There is a need to appropriately assess the driving risks of elderly drivers and support their safe driving. In particular, driving assistance that takes into account the impact of elderly drivers' emotional states on driving risks is necessary. However, conventional technologies mainly assess risks based solely on driving data, and approaches that take driver emotions into account have not been fully implemented. Therefore, a system that incorporates the emotional states of elderly drivers to provide more accurate driving risk assessments and appropriate warnings is needed.

[0473] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0474] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for evaluating driving risks based on the abnormal driving patterns, means for notifying the driver of the warning, means for collecting driver emotion data, means for integrating the emotion data with the driving risk evaluation, and means for adjusting the content and format of the warning based on the emotion data. This enables a comprehensive evaluation of the driving risks of elderly drivers that also takes emotion data into consideration, and makes it possible to provide more appropriate warnings and driving assistance.

[0475] "Driving data" refers to information about driving behavior and vehicle conditions, such as vehicle speed, steering, braking, lane keeping, and environmental data.

[0476] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking, swerving, and unstable speed.

[0477] "Driving risk" is an index of driving safety and danger assessed based on driving data and abnormal driving patterns.

[0478] "Emotional data" is information about the driver's emotional state analyzed from facial expressions, tone of voice, etc.

[0479] "Warnings" are messages containing cautions or instructions that are generated based on driving risk assessment and emotion data.

[0480] "Emotion recognition" is the process of using cameras and microphones to analyze the driver's facial expressions and tone of voice and recognize their emotional state in real time.

[0481] "Data collection means" refers to a device or method for acquiring driving data and emotion data using sensors, cameras, microphones, etc.

[0482] "Data analysis means" refers to algorithms or software that analyzes collected driving data and emotional data to assess abnormal driving patterns and driving risks.

[0483] "Warning generator" is a process that generates appropriate warning messages or reminders based on driving risk assessment and emotion data.

[0484] This invention is a system that generates warnings based on driving risk assessment and emotion data of elderly drivers. This system consists of four elements: a server, a terminal, an emotion recognition engine, and a user.

[0485] Server Operation

[0486] The server is responsible for analyzing driving data and emotion data. The hardware used includes a cloud server, and the software uses machine learning libraries such as TENSORFLOW (registered trademark).

[0487] Data collection

[0488] The server receives real-time driving and emotion data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0489] Data Preprocessing

[0490] The server performs preprocessing on the received driving data and emotion data, including noise removal, normalization, and missing value handling, to improve the accuracy of data analysis.

[0491] Data analysis

[0492] The preprocessed data is then analyzed using TensorFlow. This analysis process includes anomaly detection, pattern recognition, risk assessment, etc. The server detects abnormal driving patterns and quantifies driving risk.

[0493] Emotional Data Integration

[0494] The server integrates the emotion data received from the emotion recognition engine with the driving risk assessment, thereby generating a comprehensive driving risk report.

[0495] Warning generation

[0496] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[0497] Device behavior

[0498] The device (e.g., smart glasses) is responsible for collecting driving data in real time and sending warnings. The hardware used includes smart glasses, and the software is an application with cloud communication capabilities.

[0499] Real-time data collection and transmission

[0500] The device collects driving data in real time while driving and sends it to a cloud server using sensors such as vehicle speed, brake, and steering.

[0501] Warning display

[0502] The terminal notifies the driver of the risk assessment results received from the server: a visual warning message is displayed in the case of a medium risk, and an audio alarm is sounded in the case of a high risk.

[0503] How the emotion recognition engine works

[0504] The emotion recognition engine is responsible for collecting and analyzing driver emotion data, and uses software such as Microsoft® Azure® Cognitive Services.

[0505] emotion recognition

[0506] The emotion recognition engine uses cameras and microphones to analyze the driver's facial expressions and tone of voice in real time to recognize their emotional state.

[0507] Sending emotional data

[0508] The recognized emotion data is sent to a cloud server and integrated into driving risk assessment.

[0509] User Actions

[0510] Users (drivers and their families) can review the generated warnings and reports and adjust their driving behavior.

[0511] Check your driving rating

[0512] Users can view real-time alerts from their devices and comprehensive risk reports generated by the server.

[0513] Behavioral adjustment

[0514] The user (driver) adjusts their driving behavior based on the warnings and their emotional state.

[0515] Considering returning your license

[0516] Users (family members) consider the timing of when elderly drivers should return their licenses based on monthly reports and past driving data.

[0517] Specific examples

[0518] Example 1: Elderly driver driving on the highway

[0519] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. In addition, an emotion recognition engine detects driver tension, and the content and tone of the warning are adjusted accordingly.

[0520] Example 2: Family members checking monthly reports

[0521] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[0522] Example prompts to input to the generative AI model

[0523] "Develop a smart glasses application that monitors the driving ability of elderly drivers and provides driving assistance and appropriate warnings by integrating emotional data. This application will collect real-time driving data and analyze it on a cloud server. It will also use a camera and microphone to analyze the driver's facial expressions and tone of voice to obtain emotional data. Based on the analysis results, it will generate appropriate warnings and notify the driver."

[0524] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0525] Step 1:

[0526] Real-time data collection

[0527] The device collects driving data (vehicle speed, braking, steering, lane keeping, and environmental data) in real time from the vehicle's onboard computer. The data is acquired by various sensors (vehicle speed sensor, brake sensor, steering sensor, etc.) attached to the device and sent to a cloud server via Bluetooth or Wi-Fi.

[0528] Input: Driving data collected on the device

[0529] Output: Driving data sent to the cloud server

[0530] Step 2:

[0531] Collecting Emotional Data

[0532] The device's camera and microphone are used to capture the driver's facial expressions and tone of voice. The captured emotional data is analyzed in real time to determine the driver's emotional state (tension, relief, etc.). The collected emotional data is sent to a cloud server.

[0533] Input: Driver's facial expressions and voice data captured by camera and microphone

[0534] Output: Emotion data sent to the cloud server

[0535] Step 3:

[0536] Data Preprocessing

[0537] The server receives the driving data and emotion data and performs preprocessing such as noise removal, normalization, and missing value completion. This preprocessing improves the accuracy of the data, which in turn improves the accuracy of the analysis.

[0538] Input: Driving data and emotion data received by the server

[0539] Output: Preprocessed data

[0540] Step 4:

[0541] Data analysis

[0542] TensorFlow is used to analyze the preprocessed data, and machine learning algorithms are used to detect abnormal driving patterns and quantify driving risk assessments.

[0543] Input: Preprocessed driving and emotion data

[0544] Output: Abnormal driving patterns and driving risk assessment

[0545] Step 5:

[0546] Emotional Data Integration

[0547] The server integrates the emotion data and driving risk assessment based on the analysis results, thereby comprehensively assessing the driving risk and generating a comprehensive risk report.

[0548] Input: Abnormal driving patterns, driving risk assessment, emotional data

[0549] Output: Comprehensive risk report

[0550] Step 6:

[0551] Warning generation

[0552] The server generates alerts based on the integrated data, with three levels of alerts: low risk, medium risk, and high risk, and generates different warning messages and audio alarms for each risk level.

[0553] Input: Comprehensive Risk Report

[0554] Output: Warning message or audio alarm

[0555] Step 7:

[0556] Warning notice

[0557] The device receives warning notifications from the server and provides visual and audio warnings to the driver: in the case of a medium risk, a warning message is displayed on the device's display, and in the case of a high risk, an audio alarm is sounded.

[0558] Input: Warning notification from the server

[0559] Output: Visual and audio warnings for the driver

[0560] Step 8:

[0561] Report generation and delivery

[0562] A monthly driving risk assessment report is automatically generated on the cloud server and sent to users (drivers and their families) via email, allowing them to check past driving data and risk assessment details.

[0563] Input: Past driving risk assessment data

[0564] Output: Monthly driving risk report

[0565] The above is the flow of specific processing steps for carrying out the present invention.

[0566] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0567] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0568] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0569] [Second embodiment]

[0570] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0571] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0572] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0573] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0574] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0575] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0576] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0577] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0578] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0579] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0580] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0581] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0582] This system monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[0583] Server Operation

[0584] Data collection

[0585] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0586] Data Preprocessing

[0587] The server performs pre-processing on the received driving data, which includes the following operations:

[0588] Noise removal: Removing unwanted noise from collected data.

[0589] Normalization: Transforming data values ​​onto a uniform scale.

[0590] Missing Values: Properly impute missing data.

[0591] Data analysis

[0592] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0593] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0594] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0595] Risk assessment: Quantify the driving risk based on the analysis results.

[0596] Warning generation

[0597] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[0598] Device behavior

[0599] Real-time data collection and transmission

[0600] The terminal (on-board computer) collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors.

[0601] Warning display

[0602] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0603] User Actions

[0604] Check your driving rating

[0605] Users (drivers and their families) can check warnings and regular reports from their devices and servers, allowing them to accurately understand their own driving risks.

[0606] Behavioral adjustment

[0607] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0608] Considering returning your license

[0609] The user (family member) considers the timing of when the elderly driver should return their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license return.

[0610] Specific examples

[0611] Example 1: Elderly driver driving on the highway

[0612] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[0613] Example 2: Family members checking monthly reports

[0614] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[0615] This invention makes it possible to quantitatively evaluate the driving risk of elderly drivers and issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[0616] The processing flow will be explained below.

[0617] Understood. Below, the program processing will be explained in detail by dividing it into steps.

[0618] Server Processing

[0619] Step 1:

[0620] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0621] Step 2:

[0622] The server performs pre-processing on the received driving data, which includes the following operations:

[0623] Noise removal: Removing unwanted noise from collected data.

[0624] Normalization: Transforming data values ​​onto a uniform scale.

[0625] Missing Values: Properly impute missing data.

[0626] Step 3:

[0627] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0628] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0629] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0630] Risk assessment: Quantify the driving risk based on the analysis results.

[0631] Step 4:

[0632] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[0633] Step 5:

[0634] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[0635] Terminal handling

[0636] Step 1:

[0637] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0638] Step 2:

[0639] The device transmits the collected data to the server in real time.

[0640] Step 3:

[0641] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0642] User Action

[0643] Step 1:

[0644] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[0645] Step 2:

[0646] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0647] Step 3:

[0648] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the assessment results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license surrender.

[0649] The above are the specific processing steps of the program.

[0650] Example 1

[0651] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] The decline in driving ability of elderly drivers is a factor that increases the risk of traffic accidents. Current driving risk assessment systems lack preprocessing capabilities for high-precision analysis and have difficulties in collecting data and assessing risk in real time. Therefore, there is a need for a system that can accurately assess the driving behavior of elderly drivers and provide appropriate warnings according to their driving risk.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0654] In this invention, the server includes means for collecting driving data of elderly drivers, means for preprocessing the driving data, means for analyzing the preprocessed driving data to detect abnormal driving patterns, means for assessing driving risks based on the abnormal driving patterns, means for generating warnings based on the driving risk assessment, and means for notifying the drivers of the warnings, thereby enabling real-time monitoring of elderly drivers' driving behaviors, evaluating driving risks, and providing appropriate warnings.

[0655] "Driving data" refers to information about older drivers' driving behavior and vehicle conditions, such as vehicle speed, braking, steering, lane keeping, and environmental data.

[0656] "Preprocessing" refers to a series of data processing operations carried out to make collected driving data easier to analyze, including, for example, noise removal, data normalization, and missing value completion.

[0657] "Abnormal driving patterns" refer to patterns that detect deviations from normal driving behavior, such as sudden braking or swerving.

[0658] "Driving risk" refers to a quantitative assessment of the degree of danger posed by a driver's driving behavior. For example, it is assessed on a three-level scale: low risk, medium risk, and high risk.

[0659] "Warning" refers to a message or alarm that notifies the driver of a driving risk and alerts them to the risk, including, for example, a text message or an audio alarm.

[0660] "Real-time" refers to data collection and processing occurring almost instantaneously, with no delay, allowing for immediate analysis and alerts.

[0661] "Server" refers to the computer system that receives, pre-processes, analyzes, assesses risk, and generates alerts on driving data.

[0662] "Terminal" refers to a device that collects driving data, transmits it to a server, and notifies the driver of warnings, including, for example, an in-vehicle computer.

[0663] "Driver" refers to an elderly person who drives a vehicle.

[0664] "Notification means" refers to the method of conveying a warning message or alarm to the driver, including, for example, display or audio output.

[0665] MODE FOR CARRYING OUT THE INVENTION

[0666] The present invention is a system that monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[0667] Server Operation

[0668] Data collection

[0669] The server receives real-time driving data sent from the device. Driving data includes vehicle speed, braking, steering, lane keeping, and environmental data. This information is collected from the device's speed, brake, and steering sensors and sent to the server via WiFi. Specifically, the server uses a web framework such as "Flask" to build an API endpoint to receive the data.

[0670] Data Preprocessing

[0671] The server converts the received driving data into a data frame format and performs the following pre-processing.

[0672] Noise removal: Filtering unwanted noise from data using the "SciPy" library.

[0673] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[0674] Missing value handling: Use Python's "pandas" library to impute missing data with the mean value.

[0675] This prepares the data in a form suitable for analysis.

[0676] Data analysis

[0677] The server uses the preprocessed data to perform the following analysis:

[0678] Anomaly detection: Uses an "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[0679] Pattern Recognition: Uses "k-means clustering" to compare and identify normal and abnormal driving patterns.

[0680] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[0681] These algorithms are implemented in the "scikit-learn" library.

[0682] Warning generation

[0683] The server generates appropriate warnings based on the driving risk assessment, with three levels of warning:

[0684] Low risk: Generates a text warning message.

[0685] Medium risk: Generates audio alarm instructions in addition to text messages.

[0686] High risk: Generates a loud audio alarm and an emergency stop instruction.

[0687] The generated alert is sent to the terminal.

[0688] Device behavior

[0689] Real-time data collection and transmission

[0690] The device collects driving data in real time and sends it to a server. Data is collected using an on-board computer such as a Raspberry Pi, and data is acquired through sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0691] Warning display

[0692] The terminal notifies the driver of the warnings received from the server. If the risk is medium, a warning message is displayed on the terminal's display, and if the risk is high, an audio alarm is sounded. The actual notification is performed using an LCD display and speaker, and these output devices are controlled using the GPIO pins of Arduino or Raspberry Pi.

[0693] User Actions

[0694] Check your driving rating

[0695] Users (drivers and their families) can accurately understand their own driving risks by checking warnings and regular reports from their devices and servers. The regular reports include past driving data and its analysis results.

[0696] Behavioral adjustment

[0697] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[0698] Considering returning your license

[0699] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, a family meeting is held to encourage the driver to surrender their license.

[0700] Specific examples

[0701] Example 1: Elderly driver driving on the highway

[0702] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[0703] Example 2: Family members checking monthly reports

[0704] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[0705] Examples of prompt statements

[0706] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[0707] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0708] Server Operation

[0709] Step 1: Data collection

[0710] Input: Driving data sent from the device (vehicle speed, braking, steering, lane keeping, environmental data)

[0711] Processing: The server receives these data in real time.

[0712] Output: Raw driving data

[0713] Specific operation: The server uses a web framework such as "Flask" to build an API endpoint to receive data, and receives it from the device via WiFi.

[0714] Step 2: Data Preprocessing

[0715] Input: Raw driving data

[0716] Processing: Preprocess the data in the following ways:

[0717] Denoising: Filtering unwanted noise from data using the "SciPy" library.

[0718] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[0719] Missing value handling: Impute missing data with the mean value using the "pandas" library.

[0720] Output: Preprocessed data

[0721] Specific operation: The server converts the data into a data frame format and applies each preprocessing operation sequentially to prepare the data.

[0722] Step 3: Data analysis

[0723] Input: Preprocessed data

[0724] Processing: Analysis using machine learning algorithms:

[0725] Anomaly detection: Uses the "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[0726] Pattern recognition: Using "k-means clustering" to compare normal and abnormal driving patterns.

[0727] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[0728] Output: Driving risk assessment results

[0729] What it does: The server uses the "scikit-learn" library to implement these algorithms, analyze the data, and generate a risk assessment result.

[0730] Step 4: Generate alerts

[0731] Input: Driving risk assessment results

[0732] Action: Generate a warning message depending on the driving risk:

[0733] Low risk: Generates a text warning message

[0734] Medium risk: Generates audio alarm instructions in addition to text messages

[0735] High risk: Generates a loud audio alarm and an emergency stop instruction

[0736] Output: Warning message

[0737] Specific operation: Based on the evaluation result, the server generates an appropriate warning message and sends it to the terminal.

[0738] Device behavior

[0739] Step 1: Real-time data collection and transmission

[0740] Input: Data from vehicle speed sensor, brake sensor, and steering sensor

[0741] Processing: Data is collected and sent to a server via WiFi.

[0742] Output: Driving data to be transmitted

[0743] How it works: An onboard computer such as a Raspberry Pi collects data from various sensors and sends it to a server.

[0744] Step 2: Warning display

[0745] Input: The warning message sent by the server

[0746] Action: Notify the driver with a warning message:

[0747] Medium risk: A warning message appears on the display

[0748] High risk: sound an audio alarm

[0749] Output: Warnings notified to the driver

[0750] What it does: The device displays messages on the LCD display and plays audio alarms through the speaker. It uses the GPIO pins of Arduino and Raspberry Pi to control these devices.

[0751] User Actions

[0752] Step 1: Check your driving rating

[0753] Input: Alert messages and scheduled reports from terminals and servers

[0754] Action: Check the driving evaluation results

[0755] Output: Check the driving risk assessment results

[0756] Specific operation: The user checks the evaluation report on the device display or through the smartphone app.

[0757] Step 2: Adjust your behavior

[0758] Input: The warning message received

[0759] Action: Adjust driving behavior based on the warning

[0760] Output: Adjusted driving behavior

[0761] Specific actions: Drivers will immediately improve their driving behavior and drive more cautiously.

[0762] Step 3: Consider surrendering your license

[0763] Input: Monthly reports and historical driving data

[0764] Processing: Consider the timing of license surrender based on the evaluation results

[0765] Output: Decision to surrender license if necessary

[0766] Specific actions: Hold a family meeting and consider surrendering the driver's license if the high risk assessment continues.

[0767] Examples of prompt statements

[0768] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[0769] (Application example 1)

[0770] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0771] Monitoring driving risks and ensuring safety for elderly drivers are important social issues. In particular, when elderly drivers use self-driving vehicles, systems that assess driving risks and provide warnings or driving assistance as necessary are needed. Conventional technologies have struggled to effectively assess these risks in real time and provide appropriate warnings or assistance. Furthermore, there has been insufficient coordination between advanced analysis using generative AI models and prompt sentences and the self-driving system. Therefore, it is necessary to develop a system that can accurately assess the driving risks of elderly drivers and provide appropriate warnings and assistance.

[0772] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0773] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for assessing driving risk based on the abnormal driving patterns, means for generating a warning based on the driving risk assessment, means for notifying the driver of the warning, means for the automated driving system to take over driving operations as necessary based on the driving risk, means for analyzing the driving data using a generative AI model to assess risk and generate a warning, and means for configuring the driving risk and warning generation process based on prompt sentences. This makes it possible to assess the driving risk of elderly drivers in real time, detect abnormal driving patterns, and provide appropriate warnings and driving assistance.

[0774] An "elderly driver" refers to a driver who is over a certain age and for whom there are concerns about the risks associated with age when driving.

[0775] "Driving Data" refers to a range of data collected while driving, such as vehicle speed, braking usage, steering angle, lane position and environmental data.

[0776] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking or swerving.

[0777] "Driving risk" is an indicator that indicates the likelihood that a driver will cause an accident while driving, and includes an evaluation value that is quantified through data analysis.

[0778] "Warning" refers to a warning message or alarm that is sent to the driver when driving risk exceeds a certain level.

[0779] "Autonomous driving system" refers to technology that enables a vehicle to perform driving operations without driver intervention.

[0780] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and makes predictions and analyses on new data.

[0781] A "prompt" is a piece of text that serves as an instruction input to a generative AI model.

[0782] MODE FOR CARRYING OUT THE INVENTION

[0783] This paper describes an embodiment of an "elderly driver safety support system" for improving safety when elderly drivers use automated driving vehicles. The system of the present invention is mainly composed of four elements: a server, an in-vehicle terminal, an automated driving system, and a user.

[0784] Server Operation

[0785] Data collection

[0786] The server receives driving data transmitted in real time from the in-vehicle device. This driving data includes vehicle speed, braking, steering, lane position, and environmental data. For example, data is collected using vehicle speed sensors, brake sensors, steering sensors, cameras, and various environmental sensors.

[0787] Data Preprocessing

[0788] The server performs pre-processing on the received driving data, which includes the following operations:

[0789] Noise removal: Removing unwanted noise from collected data.

[0790] Normalization: Transforming data values ​​onto a uniform scale.

[0791] Missing Values: Properly impute missing data.

[0792] Data analysis

[0793] The server uses the generative AI model to analyze the preprocessed data. This analysis process includes the following steps:

[0794] Anomaly detection: Uses algorithms such as Isolation Forest to detect sudden behavior (hard braking, swerving, etc.).

[0795] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0796] Risk assessment: Quantify the driving risk based on the analysis results.

[0797] Warning generation

[0798] The server generates warnings based on the driving risk assessment. There are three warning levels: low risk, medium risk, and high risk. Different types of warnings are generated for each level. For example, a medium risk warning message is displayed on the screen, and a high risk warning sounds an audio alarm.

[0799] In-vehicle terminal operation

[0800] Real-time data collection and transmission

[0801] The in-vehicle device collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors. The data collected in real time is sent to the server using wireless communication technology (e.g., Wi-Fi or 5G).

[0802] Warning display

[0803] The in-vehicle device receives the risk assessment results from the server and notifies the driver. If the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0804] Autonomous driving system operation

[0805] Driving assistance measures

[0806] The automated driving system automatically performs driving operations based on the driving risk assessment received from the server. For example, if the risk is high, the system takes over control of the vehicle and drives safely.

[0807] User Actions

[0808] Check your driving rating

[0809] Users (drivers and their families) can check warnings and regular reports from the in-vehicle terminal or server, allowing them to accurately understand their own driving risks.

[0810] Behavioral adjustment

[0811] The user (driver) adjusts their driving behavior according to the warning. If they receive a medium-risk warning, they will drive more carefully, and if they receive a high-risk warning, they will refrain from driving. For example, the server can generate a warning message saying, "Be careful. Your speed is unstable," and the device can notify the driver of this.

[0812] Specific examples

[0813] An example of a prompt sentence to input to the generative AI model is as follows:

[0814] "Assess driving risks in real time based on driving data of elderly drivers. Data includes vehicle speed, braking, steering, lane keeping, and environmental data. Detect abnormal patterns, generate risk scores, and generate appropriate warning messages."

[0815] This system makes it possible to assess the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[0816] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0817] Step 1:

[0818] The server receives driving data in real time from the in-vehicle terminal. The input driving data includes vehicle speed, braking, steering operation, lane position, and environmental data. This data is transmitted to the server using wireless communication technology (e.g., Wi-Fi or 5G). The received driving data is stored in data storage and used for subsequent processing.

[0819] Step 2:

[0820] The server performs preprocessing on the received driving data. The input for this step is the data collected in step 1. First, noise removal is performed, and then data normalization is performed. Noise removal involves filtering out sensor errors and unnecessary data points. Data normalization involves converting data values ​​to a unified scale to facilitate comparison between different sensors. Finally, missing data is imputed appropriately. Specifically, SimpleImputer is used to impute missing values ​​with the mean value. This preprocessing results in a dataset that can be analyzed.

[0821] Step 3:

[0822] The server uses the generative AI model to analyze the preprocessed data. The input for this step is the preprocessed data from step 2. First, anomaly detection is performed using the Isolation Forest algorithm, which detects abnormal driving patterns (e.g., sudden braking or swerving). Next, pattern recognition is used to identify anomalies by comparing them with general driving patterns. Finally, the server evaluates the driving risk based on the analysis results and generates a risk score. This risk score is obtained as the output.

[0823] Step 4:

[0824] The server generates a warning based on the generated driving risk score. The input for this step is the risk score obtained in step 3. The generated warnings are divided into three levels: low risk, medium risk, and high risk, and different types of warning messages are prepared for each level. Specifically, a text warning message is displayed for medium risk, and an audio alarm is sounded for high risk. This warning message is sent from the server to the in-vehicle terminal.

[0825] Step 5:

[0826] The in-vehicle terminal notifies the driver of the warning message received from the server. The input of this step is the warning message generated in step 4. If the risk is medium, a warning message is displayed on the in-vehicle terminal's display. If the risk is high, an audio alarm is used to inform the driver of the urgency. This warning notification prompts the driver to adjust their driving behavior. The output of this step is a warning notification to the driver, and the driver is expected to act in accordance with the warning content.

[0827] Step 6:

[0828] The automated driving system takes over driving operations as necessary based on the driving risk assessment results received from the server. The input to this step is the driving risk assessment result obtained in step 4. If it is deemed high risk, the automated driving system takes over control and performs appropriate driving operations, such as rapidly decelerating the vehicle or guiding it to a safe route. The output of this step is the execution of safe driving operations.

[0829] Through the above steps, the "elderly driver safety support system" of the present invention is able to evaluate the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[0830] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0831] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[0832] Server Operation

[0833] Data collection

[0834] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0835] Data Preprocessing

[0836] The server performs pre-processing on the received driving data, which includes the following operations:

[0837] Noise removal: Removing unwanted noise from collected data.

[0838] Normalization: Transforming data values ​​onto a uniform scale.

[0839] Missing Values: Properly impute missing data.

[0840] Data analysis

[0841] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0842] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0843] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0844] Risk assessment: Quantify the driving risk based on the analysis results.

[0845] Emotional Data Integration

[0846] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment, which generates a comprehensive driving risk report.

[0847] Warning generation

[0848] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[0849] Notifications and Reporting

[0850] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[0851] Device behavior

[0852] Real-time data collection and transmission

[0853] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0854] Warning display

[0855] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0856] Emotion Engine Operation

[0857] emotion recognition

[0858] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[0859] Sending emotional data

[0860] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[0861] User Actions

[0862] Check your driving rating

[0863] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[0864] Behavioral adjustment

[0865] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0866] Considering returning your license

[0867] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[0868] Specific examples

[0869] Example 1: Elderly driver driving on the highway

[0870] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[0871] Example 2: Family members checking monthly reports

[0872] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[0873] This invention quantitatively evaluates the driving risk of elderly drivers and takes into account the user's emotions, enabling it to issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[0874] The processing flow will be explained below.

[0875] Server Processing

[0876] Step 1:

[0877] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[0878] Step 2:

[0879] The server performs pre-processing on the received driving data, which includes the following operations:

[0880] Noise removal: Removing unwanted noise from collected data.

[0881] Normalization: Transforming data values ​​onto a uniform scale.

[0882] Missing Values: Properly impute missing data.

[0883] Step 3:

[0884] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0885] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[0886] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[0887] Risk assessment: Quantify the driving risk based on the analysis results.

[0888] Step 4:

[0889] The server receives the user's emotion data sent from the emotion engine and integrates it into a driving risk assessment.

[0890] Step 5:

[0891] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set as follows:

[0892] Low risk: No immediate action required.

[0893] Medium risk: Display a warning message on the form screen.

[0894] High Risk: Display audio and visual warnings.

[0895] Step 6:

[0896] The server sends the warning information to the terminal and also generates a monthly report to provide to the user (driver and family).

[0897] Terminal handling

[0898] Step 1:

[0899] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[0900] Step 2:

[0901] The device transmits the collected data to the server in real time.

[0902] Step 3:

[0903] The device notifies the driver of the risk assessment results and warnings received from the server. The notification format is as follows:

[0904] Medium risk: Display a warning message on the form screen saying "Please be careful. Drive safely."

[0905] High Risk: An audio alarm will be issued and a warning message will be displayed saying "High Risk. Do not drive."

[0906] Emotion engine processing

[0907] Step 1:

[0908] The emotion engine recognizes the user's emotions in real time through facial recognition and voice analysis, collecting the user's facial expressions and tone of voice through the camera and microphone.

[0909] Step 2:

[0910] The emotion engine analyzes the collected data and assesses the user's emotional state (e.g., tension, anxiety, calmness, etc.) in real time.

[0911] Step 3:

[0912] The emotion engine sends the recognized emotion data to the server.

[0913] User Action

[0914] Step 1:

[0915] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[0916] Step 2:

[0917] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0918] Step 3:

[0919] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[0920] The above are the specific processing steps in a system that combines emotion engines.

[0921] Example 2

[0922] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0923] As older drivers age, their reaction time and attention span decline, increasing the risk of accidents. However, current driver assistance systems rely solely on driving data and do not take into account the driver's emotional state. This makes it difficult to properly assess the impact of driver emotions on driving behavior and issue appropriate warnings in a timely manner. Furthermore, some systems suffer from poor data preprocessing accuracy, which can lead to noise and missing values, reducing the accuracy of driving risk assessment.

[0924] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0925] In this invention, the server includes a device for collecting driving data of elderly drivers, a device for analyzing the driving data to detect abnormal driving behavior, a device for evaluating driving risk based on the abnormal driving behavior, a data preprocessing device for performing noise removal, numerical normalization, and missing value completion, and a device for recognizing user emotion data and integrating it into the driving risk evaluation. This makes it possible to integrate driving data and emotion data to perform a comprehensive driving risk evaluation and issue appropriate warnings in a timely manner.

[0926] "Elderly drivers" generally refer to drivers who are older and whose driving skills and reaction times may be declining.

[0927] "Driving statistics" refers to various data collected while driving, such as vehicle speed, brake usage, steering angle, and lane keeping information.

[0928] "Equipment" means an entity of hardware or software designed and manufactured to perform a specific function.

[0929] "Abnormal driving behavior" refers to detecting driving behavior that deviates from normal driving patterns, such as sudden braking or swerving.

[0930] "Driving risk" is a quantitative or qualitative assessment of the risk level of driving behavior, and is an indicator of potential threats to safety.

[0931] A "warning" is a warning message issued to drivers that includes content encouraging safe driving.

[0932] "Denoising" refers to the process of removing unwanted noise from collected data.

[0933] "Numeric normalization" refers to the process of converting data from different scales or units to a unified scale.

[0934] "Missing value imputation" refers to the process of rationally filling in missing data for an incomplete dataset.

[0935] "Emotional values" are data that quantifies a user's emotional state and refer to information obtained through facial recognition and voice analysis.

[0936] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[0937] Server Operation

[0938] Data collection

[0939] The server receives real-time driving data from the device, including vehicle speed, brake usage, steering operation, lane keeping, and environmental data. This data is collected from various sensors (e.g., vehicle speed sensor, brake sensor, steering sensor) installed on the device.

[0940] Data Preprocessing

[0941] The server performs pre-processing on the received driving data, which includes the following operations:

[0942] Noise removal: Use the Python Pandas library to remove unwanted noise from the collected data.

[0943] Normalization: Using scaling techniques to convert data values ​​to a uniform scale.

[0944] Missing Value Support: Use Scikit-learn's SimpleImputer to properly impute missing data.

[0945] Data analysis

[0946] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[0947] Anomaly detection: Use anomaly detection algorithms such as Isolation Forest to detect sudden behavior (sudden braking, swerving, etc.).

[0948] Pattern Recognition: Running deep learning models using Keras to identify anomalous driving patterns compared to typical driving patterns.

[0949] Risk assessment: Quantify the driving risk based on the analysis results.

[0950] Emotional Data Integration

[0951] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment. For example, OpenCV is used to recognize the driver's facial expressions and Google Cloud Speech-to-Text is used for voice analysis. This integration generates a comprehensive driving risk report.

[0952] Warning generation

[0953] The server generates warnings based on driving risk assessment and emotion data. There are three warning levels: low risk, medium risk, and high risk. Different warning formats are generated for each level. For example, messages such as "Be careful," "There is a problem with your driving pattern," and "Consider an emergency stop" are generated.

[0954] Notifications and Reporting

[0955] The server sends warning information to the device and, if necessary, generates a monthly report and provides it to the user (driver and family members). For example, a warning message is displayed on the device, and an audio alarm is sounded if there is a high risk.

[0956] Device behavior

[0957] Real-time data collection and transmission

[0958] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as speed sensors, brake sensors, and steering sensors, and the collected data is sent to a server.

[0959] Warning display

[0960] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[0961] Emotion Engine Operation

[0962] emotion recognition

[0963] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[0964] Sending emotional data

[0965] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[0966] User Actions

[0967] Check your driving rating

[0968] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[0969] Behavioral adjustment

[0970] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[0971] Considering returning your license

[0972] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[0973] Specific examples

[0974] Example 1: Elderly driver driving on the highway

[0975] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[0976] Example 2: Family members checking monthly reports

[0977] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[0978] Example prompt

[0979] Example of input prompt for generative AI model:

[0980] "Please explain in detail how the system works, where the server detects anomalies based on data collected from the device when an elderly driver is driving on a highway and generates a medium-risk warning."

[0981] Through these specific processes, the present invention can accurately assess the driving risks of elderly drivers and provide effective driving assistance.

[0982] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0983] Step 1: Data collection

[0984] The server receives driving data from the device in real time. The device is equipped with a vehicle speed sensor, brake sensor, steering sensor, etc., and data collected from these sensors while driving is sent to the server. Input data includes vehicle speed, brake usage, steering angle, lane keeping information, etc. The server receives this data and records it in a log. The output data is a file of the collected driving data.

[0985] Step 2: Data Preprocessing

[0986] The server performs preprocessing on the received driving data. The input data is the collected raw driving data, and the following preprocessing is performed:

[0987] Noise removal: Using the Python Pandas library, outliers and noise are removed. Specifically, this involves filtering out abnormally high vehicle speed data and inconsistent braking data.

[0988] Normalization: Use min-max scaling to convert data values ​​to a uniform scale, for example, scaling vehicle speed data to a range of 0 to 1.

[0989] Missing data handling: Using Scikit-learn's SimpleImputer, we impute missing data with the mean value. For example, we impute missing brake data with the mean value of that brake sensor.

[0990] The output data is pre-processed, clean driving data.

[0991] Step 3: Data analysis

[0992] The server uses machine learning algorithms to analyze the preprocessed data. The input data is the preprocessed driving data, and the following analysis is performed:

[0993] Anomaly Detection: Isolation Forest is used to detect anomalous behaviors such as sudden braking or erratic driving by identifying data points that deviate from normal driving patterns.

[0994] Pattern Recognition: Using Keras, we run deep learning models to analyze driving patterns, for example, learning and comparing driving patterns around sharp turns and on highways.

[0995] Risk Assessment: Generate a risk score based on the analysis results, for example, quantifying risk on a scale of 0 to 100 based on the frequency and severity of abnormal behavior.

[0996] The output data is the driving risk assessment result.

[0997] Step 4: Integrating Emotional Data

[0998] The server receives the user's emotion data recognized by the emotion engine. The input data is emotion data obtained by face recognition and voice analysis, and the following operations are performed:

[0999] Emotion Recognition: Facial expressions are analyzed using OpenCV, and voice tones are analyzed using Google Cloud Speech-to-Text. Specific operations include quantifying the user's emotions, such as tension or anxiety.

[1000] Emotional data integration: Integrate driving risk assessment results with emotional data to generate a comprehensive driving risk report. For example, reassess the overall risk by incorporating emotional scores into the risk score.

[1001] The output data is a comprehensive driving risk report.

[1002] Step 5: Generate warnings

[1003] The server generates an alert based on the comprehensive driving risk report. The input data is the comprehensive driving risk report, and the following operations are performed:

[1004] Warning level setting: Sets the warning level (low risk, medium risk, high risk) based on the risk score. Specifically, if the risk score is 50 or above, it is considered a medium risk, and if it is 80 or above, it is considered a high risk.

[1005] Warning message generation: Generates a warning message according to each risk level. For example, for low risk, it generates a message saying "Be careful," for medium risk, it generates a message saying "There is a problem with your driving pattern," and for high risk, it generates a message saying "Consider making an emergency stop."

[1006] The output data is a warning message.

[1007] Step 6: Notifications and Reporting

[1008] The server sends the generated warning messages to the terminal and also generates monthly reports if necessary. The input data are the warning messages and the analyzed driving data, and the following operations are performed:

[1009] Warning notification: A warning message is sent to the device, which then displays it to the driver. Specifically, the device displays the warning message on the screen and sounds an audio alarm if the risk is high.

[1010] Report Generation: Historical driving data is aggregated to generate monthly reports, including driving risk assessments and details of abnormal driving patterns.

[1011] The output data is a warning message and a monthly report.

[1012] Through each of the above steps, a system will be realized that evaluates the driving risks of elderly drivers and provides appropriate warnings and driving assistance.

[1013] (Application example 2)

[1014] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1015] There is a need to appropriately assess the driving risks of elderly drivers and support their safe driving. In particular, driving assistance that takes into account the impact of elderly drivers' emotional states on driving risks is necessary. However, conventional technologies mainly assess risks based solely on driving data, and approaches that take driver emotions into account have not been fully implemented. Therefore, a system that incorporates the emotional states of elderly drivers to provide more accurate driving risk assessments and appropriate warnings is needed.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1017] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for evaluating driving risks based on the abnormal driving patterns, means for notifying the driver of the warning, means for collecting driver emotion data, means for integrating the emotion data with the driving risk evaluation, and means for adjusting the content and format of the warning based on the emotion data. This enables a comprehensive evaluation of the driving risks of elderly drivers that also takes emotion data into consideration, and makes it possible to provide more appropriate warnings and driving assistance.

[1018] "Driving data" refers to information about driving behavior and vehicle conditions, such as vehicle speed, steering, braking, lane keeping, and environmental data.

[1019] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking, swerving, and unstable speed.

[1020] "Driving risk" is an index of driving safety and danger assessed based on driving data and abnormal driving patterns.

[1021] "Emotional data" is information about the driver's emotional state analyzed from facial expressions, tone of voice, etc.

[1022] "Warnings" are messages containing cautions or instructions that are generated based on driving risk assessment and emotion data.

[1023] "Emotion recognition" is the process of using cameras and microphones to analyze the driver's facial expressions and tone of voice and recognize their emotional state in real time.

[1024] "Data collection means" refers to a device or method for acquiring driving data and emotion data using sensors, cameras, microphones, etc.

[1025] "Data analysis means" refers to algorithms or software that analyzes collected driving data and emotional data to assess abnormal driving patterns and driving risks.

[1026] "Warning generator" is a process that generates appropriate warning messages or reminders based on driving risk assessment and emotion data.

[1027] This invention is a system that generates warnings based on driving risk assessment and emotion data of elderly drivers. This system consists of four elements: a server, a terminal, an emotion recognition engine, and a user.

[1028] Server Operation

[1029] The server is responsible for analyzing driving data and emotion data. The hardware used includes a cloud server, and the software uses machine learning libraries such as TensorFlow.

[1030] Data collection

[1031] The server receives real-time driving and emotion data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1032] Data Preprocessing

[1033] The server performs preprocessing on the received driving data and emotion data, including noise removal, normalization, and missing value handling, to improve the accuracy of data analysis.

[1034] Data analysis

[1035] The preprocessed data is then analyzed using TensorFlow. This analysis process includes anomaly detection, pattern recognition, risk assessment, etc. The server detects abnormal driving patterns and quantifies driving risk.

[1036] Emotional Data Integration

[1037] The server integrates the emotion data received from the emotion recognition engine with the driving risk assessment, thereby generating a comprehensive driving risk report.

[1038] Warning generation

[1039] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[1040] Device behavior

[1041] The device (e.g., smart glasses) is responsible for collecting driving data in real time and sending warnings. The hardware used includes smart glasses, and the software is an application with cloud communication capabilities.

[1042] Real-time data collection and transmission

[1043] The device collects driving data in real time while driving and sends it to a cloud server using sensors such as vehicle speed, brake, and steering.

[1044] Warning display

[1045] The terminal notifies the driver of the risk assessment results received from the server: a visual warning message is displayed in the case of a medium risk, and an audio alarm is sounded in the case of a high risk.

[1046] How the emotion recognition engine works

[1047] The emotion recognition engine is responsible for collecting and analyzing driver emotion data, and uses software such as Microsoft Azure Cognitive Services.

[1048] emotion recognition

[1049] The emotion recognition engine uses cameras and microphones to analyze the driver's facial expressions and tone of voice in real time to recognize their emotional state.

[1050] Sending emotional data

[1051] The recognized emotion data is sent to a cloud server and integrated into driving risk assessment.

[1052] User Actions

[1053] Users (drivers and their families) can review the generated warnings and reports and adjust their driving behavior.

[1054] Check your driving rating

[1055] Users can view real-time alerts from their devices and comprehensive risk reports generated by the server.

[1056] Behavioral adjustment

[1057] The user (driver) adjusts their driving behavior based on the warnings and their emotional state.

[1058] Considering returning your license

[1059] Users (family members) consider the timing of when elderly drivers should return their licenses based on monthly reports and past driving data.

[1060] Specific examples

[1061] Example 1: Elderly driver driving on the highway

[1062] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. In addition, an emotion recognition engine detects driver tension, and the content and tone of the warning are adjusted accordingly.

[1063] Example 2: Family members checking monthly reports

[1064] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[1065] Example prompts to input to the generative AI model

[1066] "Develop a smart glasses application that monitors the driving ability of elderly drivers and provides driving assistance and appropriate warnings by integrating emotional data. This application will collect real-time driving data and analyze it on a cloud server. It will also use a camera and microphone to analyze the driver's facial expressions and tone of voice to obtain emotional data. Based on the analysis results, it will generate appropriate warnings and notify the driver."

[1067] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1068] Step 1:

[1069] Real-time data collection

[1070] The device collects driving data (vehicle speed, braking, steering, lane keeping, and environmental data) in real time from the vehicle's onboard computer. The data is acquired by various sensors (vehicle speed sensor, brake sensor, steering sensor, etc.) attached to the device and sent to a cloud server via Bluetooth or Wi-Fi.

[1071] Input: Driving data collected on the device

[1072] Output: Driving data sent to the cloud server

[1073] Step 2:

[1074] Collecting Emotional Data

[1075] The device's camera and microphone are used to capture the driver's facial expressions and tone of voice. The captured emotional data is analyzed in real time to determine the driver's emotional state (tension, relief, etc.). The collected emotional data is sent to a cloud server.

[1076] Input: Driver's facial expressions and voice data captured by camera and microphone

[1077] Output: Emotion data sent to the cloud server

[1078] Step 3:

[1079] Data Preprocessing

[1080] The server receives the driving data and emotion data and performs preprocessing such as noise removal, normalization, and missing value completion. This preprocessing improves the accuracy of the data, which in turn improves the accuracy of the analysis.

[1081] Input: Driving data and emotion data received by the server

[1082] Output: Preprocessed data

[1083] Step 4:

[1084] Data analysis

[1085] TensorFlow is used to analyze the preprocessed data, and machine learning algorithms are used to detect abnormal driving patterns and quantify driving risk assessments.

[1086] Input: Preprocessed driving and emotion data

[1087] Output: Abnormal driving patterns and driving risk assessment

[1088] Step 5:

[1089] Emotional Data Integration

[1090] The server integrates the emotion data and driving risk assessment based on the analysis results, thereby comprehensively assessing the driving risk and generating a comprehensive risk report.

[1091] Input: Abnormal driving patterns, driving risk assessment, emotional data

[1092] Output: Comprehensive risk report

[1093] Step 6:

[1094] Warning generation

[1095] The server generates alerts based on the integrated data, with three levels of alerts: low risk, medium risk, and high risk, and generates different warning messages and audio alarms for each risk level.

[1096] Input: Comprehensive Risk Report

[1097] Output: Warning message or audio alarm

[1098] Step 7:

[1099] Warning notice

[1100] The device receives warning notifications from the server and provides visual and audio warnings to the driver: in the case of a medium risk, a warning message is displayed on the device's display, and in the case of a high risk, an audio alarm is sounded.

[1101] Input: Warning notification from the server

[1102] Output: Visual and audio warnings for the driver

[1103] Step 8:

[1104] Report generation and delivery

[1105] A monthly driving risk assessment report is automatically generated on the cloud server and sent to users (drivers and their families) via email, allowing them to check past driving data and risk assessment details.

[1106] Input: Past driving risk assessment data

[1107] Output: Monthly driving risk report

[1108] The above is the flow of specific processing steps for carrying out the present invention.

[1109] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1110] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1111] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1112] [Third embodiment]

[1113] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1114] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1115] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1116] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1117] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1118] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1119] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1120] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1121] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1122] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1123] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1124] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1125] This system monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[1126] Server Operation

[1127] Data collection

[1128] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1129] Data Preprocessing

[1130] The server performs pre-processing on the received driving data, which includes the following operations:

[1131] Noise removal: Removing unwanted noise from collected data.

[1132] Normalization: Transforming data values ​​onto a uniform scale.

[1133] Missing Values: Properly impute missing data.

[1134] Data analysis

[1135] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1136] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1137] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1138] Risk assessment: Quantify the driving risk based on the analysis results.

[1139] Warning generation

[1140] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[1141] Device behavior

[1142] Real-time data collection and transmission

[1143] The terminal (on-board computer) collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors.

[1144] Warning display

[1145] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1146] User Actions

[1147] Check your driving rating

[1148] Users (drivers and their families) can check warnings and regular reports from their devices and servers, allowing them to accurately understand their own driving risks.

[1149] Behavioral adjustment

[1150] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1151] Considering returning your license

[1152] The user (family member) considers the timing of when the elderly driver should return their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license return.

[1153] Specific examples

[1154] Example 1: Elderly driver driving on the highway

[1155] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[1156] Example 2: Family members checking monthly reports

[1157] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[1158] This invention makes it possible to quantitatively evaluate the driving risk of elderly drivers and issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[1159] The processing flow will be explained below.

[1160] Understood. Below, the program processing will be explained in detail by dividing it into steps.

[1161] Server Processing

[1162] Step 1:

[1163] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1164] Step 2:

[1165] The server performs pre-processing on the received driving data, which includes the following operations:

[1166] Noise removal: Removing unwanted noise from collected data.

[1167] Normalization: Transforming data values ​​onto a uniform scale.

[1168] Missing Values: Properly impute missing data.

[1169] Step 3:

[1170] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1171] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1172] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1173] Risk assessment: Quantify the driving risk based on the analysis results.

[1174] Step 4:

[1175] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[1176] Step 5:

[1177] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[1178] Terminal handling

[1179] Step 1:

[1180] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1181] Step 2:

[1182] The device transmits the collected data to the server in real time.

[1183] Step 3:

[1184] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1185] User Action

[1186] Step 1:

[1187] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[1188] Step 2:

[1189] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1190] Step 3:

[1191] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the assessment results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license surrender.

[1192] The above are the specific processing steps of the program.

[1193] Example 1

[1194] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1195] The decline in driving ability of elderly drivers is a factor that increases the risk of traffic accidents. Current driving risk assessment systems lack preprocessing capabilities for high-precision analysis and have difficulties in collecting data and assessing risk in real time. Therefore, there is a need for a system that can accurately assess the driving behavior of elderly drivers and provide appropriate warnings according to their driving risk.

[1196] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1197] In this invention, the server includes means for collecting driving data of elderly drivers, means for preprocessing the driving data, means for analyzing the preprocessed driving data to detect abnormal driving patterns, means for assessing driving risks based on the abnormal driving patterns, means for generating warnings based on the driving risk assessment, and means for notifying the drivers of the warnings, thereby enabling real-time monitoring of elderly drivers' driving behaviors, evaluating driving risks, and providing appropriate warnings.

[1198] "Driving data" refers to information about older drivers' driving behavior and vehicle conditions, such as vehicle speed, braking, steering, lane keeping, and environmental data.

[1199] "Preprocessing" refers to a series of data processing operations carried out to make collected driving data easier to analyze, including, for example, noise removal, data normalization, and missing value completion.

[1200] "Abnormal driving patterns" refer to patterns that detect deviations from normal driving behavior, such as sudden braking or swerving.

[1201] "Driving risk" refers to a quantitative assessment of the degree of danger posed by a driver's driving behavior. For example, it is assessed on a three-level scale: low risk, medium risk, and high risk.

[1202] "Warning" refers to a message or alarm that notifies the driver of a driving risk and alerts them to the risk, including, for example, a text message or an audio alarm.

[1203] "Real-time" refers to data collection and processing occurring almost instantaneously, with no delay, allowing for immediate analysis and alerts.

[1204] "Server" refers to the computer system that receives, pre-processes, analyzes, assesses risk, and generates alerts on driving data.

[1205] "Terminal" refers to a device that collects driving data, transmits it to a server, and notifies the driver of warnings, including, for example, an in-vehicle computer.

[1206] "Driver" refers to an elderly person who drives a vehicle.

[1207] "Notification means" refers to the method of conveying a warning message or alarm to the driver, including, for example, display or audio output.

[1208] MODE FOR CARRYING OUT THE INVENTION

[1209] The present invention is a system that monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[1210] Server Operation

[1211] Data collection

[1212] The server receives real-time driving data sent from the device. Driving data includes vehicle speed, braking, steering, lane keeping, and environmental data. This information is collected from the device's speed, brake, and steering sensors and sent to the server via WiFi. Specifically, the server uses a web framework such as "Flask" to build an API endpoint to receive the data.

[1213] Data Preprocessing

[1214] The server converts the received driving data into a data frame format and performs the following pre-processing.

[1215] Noise removal: Filtering unwanted noise from data using the "SciPy" library.

[1216] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[1217] Missing value handling: Use Python's "pandas" library to impute missing data with the mean value.

[1218] This prepares the data in a form suitable for analysis.

[1219] Data analysis

[1220] The server uses the preprocessed data to perform the following analysis:

[1221] Anomaly detection: Uses an "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[1222] Pattern Recognition: Uses "k-means clustering" to compare and identify normal and abnormal driving patterns.

[1223] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[1224] These algorithms are implemented in the "scikit-learn" library.

[1225] Warning generation

[1226] The server generates appropriate warnings based on the driving risk assessment, with three levels of warning:

[1227] Low risk: Generates a text warning message.

[1228] Medium risk: Generates audio alarm instructions in addition to text messages.

[1229] High risk: Generates a loud audio alarm and an emergency stop instruction.

[1230] The generated alert is sent to the terminal.

[1231] Device behavior

[1232] Real-time data collection and transmission

[1233] The device collects driving data in real time and sends it to a server. Data is collected using an on-board computer such as a Raspberry Pi, and data is acquired through sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1234] Warning display

[1235] The terminal notifies the driver of the warnings received from the server. If the risk is medium, a warning message is displayed on the terminal's display, and if the risk is high, an audio alarm is sounded. The actual notification is performed using an LCD display and speaker, and these output devices are controlled using the GPIO pins of Arduino or Raspberry Pi.

[1236] User Actions

[1237] Check your driving rating

[1238] Users (drivers and their families) can accurately understand their own driving risks by checking warnings and regular reports from their devices and servers. The regular reports include past driving data and its analysis results.

[1239] Behavioral adjustment

[1240] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1241] Considering returning your license

[1242] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, a family meeting is held to encourage the driver to surrender their license.

[1243] Specific examples

[1244] Example 1: Elderly driver driving on the highway

[1245] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[1246] Example 2: Family members checking monthly reports

[1247] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[1248] Examples of prompt statements

[1249] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[1250] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1251] Server Operation

[1252] Step 1: Data collection

[1253] Input: Driving data sent from the device (vehicle speed, braking, steering, lane keeping, environmental data)

[1254] Processing: The server receives these data in real time.

[1255] Output: Raw driving data

[1256] Specific operation: The server uses a web framework such as "Flask" to build an API endpoint to receive data, and receives it from the device via WiFi.

[1257] Step 2: Data Preprocessing

[1258] Input: Raw driving data

[1259] Processing: Preprocess the data in the following ways:

[1260] Denoising: Filtering unwanted noise from data using the "SciPy" library.

[1261] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[1262] Missing value handling: Impute missing data with the mean value using the "pandas" library.

[1263] Output: Preprocessed data

[1264] Specific operation: The server converts the data into a data frame format and applies each preprocessing operation sequentially to prepare the data.

[1265] Step 3: Data analysis

[1266] Input: Preprocessed data

[1267] Processing: Analysis using machine learning algorithms:

[1268] Anomaly detection: Uses the "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[1269] Pattern recognition: Using "k-means clustering" to compare normal and abnormal driving patterns.

[1270] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[1271] Output: Driving risk assessment results

[1272] What it does: The server uses the "scikit-learn" library to implement these algorithms, analyze the data, and generate a risk assessment result.

[1273] Step 4: Generate alerts

[1274] Input: Driving risk assessment results

[1275] Action: Generate a warning message depending on the driving risk:

[1276] Low risk: Generates a text warning message

[1277] Medium risk: Generates audio alarm instructions in addition to text messages

[1278] High risk: Generates a loud audio alarm and an emergency stop instruction

[1279] Output: Warning message

[1280] Specific operation: Based on the evaluation result, the server generates an appropriate warning message and sends it to the terminal.

[1281] Device behavior

[1282] Step 1: Real-time data collection and transmission

[1283] Input: Data from vehicle speed sensor, brake sensor, and steering sensor

[1284] Processing: Data is collected and sent to a server via WiFi.

[1285] Output: Driving data to be transmitted

[1286] How it works: An onboard computer such as a Raspberry Pi collects data from various sensors and sends it to a server.

[1287] Step 2: Warning display

[1288] Input: The warning message sent by the server

[1289] Action: Notify the driver with a warning message:

[1290] Medium risk: A warning message appears on the display

[1291] High risk: sound an audio alarm

[1292] Output: Warnings notified to the driver

[1293] What it does: The device displays messages on the LCD display and plays audio alarms through the speaker. It uses the GPIO pins of Arduino and Raspberry Pi to control these devices.

[1294] User Actions

[1295] Step 1: Check your driving rating

[1296] Input: Alert messages and scheduled reports from terminals and servers

[1297] Action: Check the driving evaluation results

[1298] Output: Check the driving risk assessment results

[1299] Specific operation: The user checks the evaluation report on the device display or through the smartphone app.

[1300] Step 2: Adjust your behavior

[1301] Input: The warning message received

[1302] Action: Adjust driving behavior based on the warning

[1303] Output: Adjusted driving behavior

[1304] Specific actions: Drivers will immediately improve their driving behavior and drive more cautiously.

[1305] Step 3: Consider surrendering your license

[1306] Input: Monthly reports and historical driving data

[1307] Processing: Consider the timing of license surrender based on the evaluation results

[1308] Output: Decision to surrender license if necessary

[1309] Specific actions: Hold a family meeting and consider surrendering the driver's license if the high risk assessment continues.

[1310] Examples of prompt statements

[1311] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[1312] (Application example 1)

[1313] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1314] Monitoring driving risks and ensuring safety for elderly drivers are important social issues. In particular, when elderly drivers use self-driving vehicles, systems that assess driving risks and provide warnings or driving assistance as necessary are needed. Conventional technologies have struggled to effectively assess these risks in real time and provide appropriate warnings or assistance. Furthermore, there has been insufficient coordination between advanced analysis using generative AI models and prompt sentences and the self-driving system. Therefore, it is necessary to develop a system that can accurately assess the driving risks of elderly drivers and provide appropriate warnings and assistance.

[1315] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1316] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for assessing driving risk based on the abnormal driving patterns, means for generating a warning based on the driving risk assessment, means for notifying the driver of the warning, means for the automated driving system to take over driving operations as necessary based on the driving risk, means for analyzing the driving data using a generative AI model to assess risk and generate a warning, and means for configuring the driving risk and warning generation process based on prompt sentences. This makes it possible to assess the driving risk of elderly drivers in real time, detect abnormal driving patterns, and provide appropriate warnings and driving assistance.

[1317] An "elderly driver" refers to a driver who is over a certain age and for whom there are concerns about the risks associated with age when driving.

[1318] "Driving Data" refers to a range of data collected while driving, such as vehicle speed, braking usage, steering angle, lane position and environmental data.

[1319] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking or swerving.

[1320] "Driving risk" is an indicator that indicates the likelihood that a driver will cause an accident while driving, and includes an evaluation value that is quantified through data analysis.

[1321] "Warning" refers to a warning message or alarm that is sent to the driver when driving risk exceeds a certain level.

[1322] "Autonomous driving system" refers to technology that enables a vehicle to perform driving operations without driver intervention.

[1323] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and makes predictions and analyses on new data.

[1324] A "prompt" is a piece of text that serves as an instruction input to a generative AI model.

[1325] MODE FOR CARRYING OUT THE INVENTION

[1326] This paper describes an embodiment of an "elderly driver safety support system" for improving safety when elderly drivers use automated driving vehicles. The system of the present invention is mainly composed of four elements: a server, an in-vehicle terminal, an automated driving system, and a user.

[1327] Server Operation

[1328] Data collection

[1329] The server receives driving data transmitted in real time from the in-vehicle device. This driving data includes vehicle speed, braking, steering, lane position, and environmental data. For example, data is collected using vehicle speed sensors, brake sensors, steering sensors, cameras, and various environmental sensors.

[1330] Data Preprocessing

[1331] The server performs pre-processing on the received driving data, which includes the following operations:

[1332] Noise removal: Removing unwanted noise from collected data.

[1333] Normalization: Transforming data values ​​onto a uniform scale.

[1334] Missing Values: Properly impute missing data.

[1335] Data analysis

[1336] The server uses the generative AI model to analyze the preprocessed data. This analysis process includes the following steps:

[1337] Anomaly detection: Uses algorithms such as Isolation Forest to detect sudden behavior (hard braking, swerving, etc.).

[1338] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1339] Risk assessment: Quantify the driving risk based on the analysis results.

[1340] Warning generation

[1341] The server generates warnings based on the driving risk assessment. There are three warning levels: low risk, medium risk, and high risk. Different types of warnings are generated for each level. For example, a medium risk warning message is displayed on the screen, and a high risk warning sounds an audio alarm.

[1342] In-vehicle terminal operation

[1343] Real-time data collection and transmission

[1344] The in-vehicle device collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors. The data collected in real time is sent to the server using wireless communication technology (e.g., Wi-Fi or 5G).

[1345] Warning display

[1346] The in-vehicle device receives the risk assessment results from the server and notifies the driver. If the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1347] Autonomous driving system operation

[1348] Driving assistance measures

[1349] The automated driving system automatically performs driving operations based on the driving risk assessment received from the server. For example, if the risk is high, the system takes over control of the vehicle and drives safely.

[1350] User Actions

[1351] Check your driving rating

[1352] Users (drivers and their families) can check warnings and regular reports from the in-vehicle terminal or server, allowing them to accurately understand their own driving risks.

[1353] Behavioral adjustment

[1354] The user (driver) adjusts their driving behavior according to the warning. If they receive a medium-risk warning, they will drive more carefully, and if they receive a high-risk warning, they will refrain from driving. For example, the server can generate a warning message saying, "Be careful. Your speed is unstable," and the device can notify the driver of this.

[1355] Specific examples

[1356] An example of a prompt sentence to input to the generative AI model is as follows:

[1357] "Assess driving risks in real time based on driving data of elderly drivers. Data includes vehicle speed, braking, steering, lane keeping, and environmental data. Detect abnormal patterns, generate risk scores, and generate appropriate warning messages."

[1358] This system makes it possible to assess the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[1359] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1360] Step 1:

[1361] The server receives driving data in real time from the in-vehicle terminal. The input driving data includes vehicle speed, braking, steering operation, lane position, and environmental data. This data is transmitted to the server using wireless communication technology (e.g., Wi-Fi or 5G). The received driving data is stored in data storage and used for subsequent processing.

[1362] Step 2:

[1363] The server performs preprocessing on the received driving data. The input for this step is the data collected in step 1. First, noise removal is performed, and then data normalization is performed. Noise removal involves filtering out sensor errors and unnecessary data points. Data normalization involves converting data values ​​to a unified scale to facilitate comparison between different sensors. Finally, missing data is imputed appropriately. Specifically, SimpleImputer is used to impute missing values ​​with the mean value. This preprocessing results in a dataset that can be analyzed.

[1364] Step 3:

[1365] The server uses the generative AI model to analyze the preprocessed data. The input for this step is the preprocessed data from step 2. First, anomaly detection is performed using the Isolation Forest algorithm, which detects abnormal driving patterns (e.g., sudden braking or swerving). Next, pattern recognition is used to identify anomalies by comparing them with general driving patterns. Finally, the server evaluates the driving risk based on the analysis results and generates a risk score. This risk score is obtained as the output.

[1366] Step 4:

[1367] The server generates a warning based on the generated driving risk score. The input for this step is the risk score obtained in step 3. The generated warnings are divided into three levels: low risk, medium risk, and high risk, and different types of warning messages are prepared for each level. Specifically, a text warning message is displayed for medium risk, and an audio alarm is sounded for high risk. This warning message is sent from the server to the in-vehicle terminal.

[1368] Step 5:

[1369] The in-vehicle terminal notifies the driver of the warning message received from the server. The input of this step is the warning message generated in step 4. If the risk is medium, a warning message is displayed on the in-vehicle terminal's display. If the risk is high, an audio alarm is used to inform the driver of the urgency. This warning notification prompts the driver to adjust their driving behavior. The output of this step is a warning notification to the driver, and the driver is expected to act in accordance with the warning content.

[1370] Step 6:

[1371] The automated driving system takes over driving operations as necessary based on the driving risk assessment results received from the server. The input to this step is the driving risk assessment result obtained in step 4. If it is deemed high risk, the automated driving system takes over control and performs appropriate driving operations, such as rapidly decelerating the vehicle or guiding it to a safe route. The output of this step is the execution of safe driving operations.

[1372] Through the above steps, the "elderly driver safety support system" of the present invention is able to evaluate the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[1373] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1374] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[1375] Server Operation

[1376] Data collection

[1377] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1378] Data Preprocessing

[1379] The server performs pre-processing on the received driving data, which includes the following operations:

[1380] Noise removal: Removing unwanted noise from collected data.

[1381] Normalization: Transforming data values ​​onto a uniform scale.

[1382] Missing Values: Properly impute missing data.

[1383] Data analysis

[1384] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1385] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1386] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1387] Risk assessment: Quantify the driving risk based on the analysis results.

[1388] Emotional Data Integration

[1389] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment, which generates a comprehensive driving risk report.

[1390] Warning generation

[1391] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[1392] Notifications and Reporting

[1393] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[1394] Device behavior

[1395] Real-time data collection and transmission

[1396] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1397] Warning display

[1398] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1399] Emotion Engine Operation

[1400] emotion recognition

[1401] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[1402] Sending emotional data

[1403] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[1404] User Actions

[1405] Check your driving rating

[1406] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[1407] Behavioral adjustment

[1408] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[1409] Considering returning your license

[1410] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[1411] Specific examples

[1412] Example 1: Elderly driver driving on the highway

[1413] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[1414] Example 2: Family members checking monthly reports

[1415] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[1416] This invention quantitatively evaluates the driving risk of elderly drivers and takes into account the user's emotions, enabling it to issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[1417] The processing flow will be explained below.

[1418] Server Processing

[1419] Step 1:

[1420] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1421] Step 2:

[1422] The server performs pre-processing on the received driving data, which includes the following operations:

[1423] Noise removal: Removing unwanted noise from collected data.

[1424] Normalization: Transforming data values ​​onto a uniform scale.

[1425] Missing Values: Properly impute missing data.

[1426] Step 3:

[1427] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1428] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1429] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1430] Risk assessment: Quantify the driving risk based on the analysis results.

[1431] Step 4:

[1432] The server receives the user's emotion data sent from the emotion engine and integrates it into a driving risk assessment.

[1433] Step 5:

[1434] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set as follows:

[1435] Low risk: No immediate action required.

[1436] Medium risk: Display a warning message on the form screen.

[1437] High Risk: Display audio and visual warnings.

[1438] Step 6:

[1439] The server sends the warning information to the terminal and also generates a monthly report to provide to the user (driver and family).

[1440] Terminal handling

[1441] Step 1:

[1442] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1443] Step 2:

[1444] The device transmits the collected data to the server in real time.

[1445] Step 3:

[1446] The device notifies the driver of the risk assessment results and warnings received from the server. The notification format is as follows:

[1447] Medium risk: Display a warning message on the form screen saying "Please be careful. Drive safely."

[1448] High Risk: An audio alarm will be issued and a warning message will be displayed saying "High Risk. Do not drive."

[1449] Emotion engine processing

[1450] Step 1:

[1451] The emotion engine recognizes the user's emotions in real time through facial recognition and voice analysis, collecting the user's facial expressions and tone of voice through the camera and microphone.

[1452] Step 2:

[1453] The emotion engine analyzes the collected data and assesses the user's emotional state (e.g., tension, anxiety, calmness, etc.) in real time.

[1454] Step 3:

[1455] The emotion engine sends the recognized emotion data to the server.

[1456] User Action

[1457] Step 1:

[1458] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[1459] Step 2:

[1460] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[1461] Step 3:

[1462] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[1463] The above are the specific processing steps in a system that combines emotion engines.

[1464] Example 2

[1465] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1466] As older drivers age, their reaction time and attention span decline, increasing the risk of accidents. However, current driver assistance systems rely solely on driving data and do not take into account the driver's emotional state. This makes it difficult to properly assess the impact of driver emotions on driving behavior and issue appropriate warnings in a timely manner. Furthermore, some systems suffer from poor data preprocessing accuracy, which can lead to noise and missing values, reducing the accuracy of driving risk assessment.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1468] In this invention, the server includes a device for collecting driving data of elderly drivers, a device for analyzing the driving data to detect abnormal driving behavior, a device for evaluating driving risk based on the abnormal driving behavior, a data preprocessing device for performing noise removal, numerical normalization, and missing value completion, and a device for recognizing user emotion data and integrating it into the driving risk evaluation. This makes it possible to integrate driving data and emotion data to perform a comprehensive driving risk evaluation and issue appropriate warnings in a timely manner.

[1469] "Elderly drivers" generally refer to drivers who are older and whose driving skills and reaction times may be declining.

[1470] "Driving statistics" refers to various data collected while driving, such as vehicle speed, brake usage, steering angle, and lane keeping information.

[1471] "Equipment" means an entity of hardware or software designed and manufactured to perform a specific function.

[1472] "Abnormal driving behavior" refers to detecting driving behavior that deviates from normal driving patterns, such as sudden braking or swerving.

[1473] "Driving risk" is a quantitative or qualitative assessment of the risk level of driving behavior, and is an indicator of potential threats to safety.

[1474] A "warning" is a warning message issued to drivers that includes content encouraging safe driving.

[1475] "Denoising" refers to the process of removing unwanted noise from collected data.

[1476] "Numeric normalization" refers to the process of converting data from different scales or units to a unified scale.

[1477] "Missing value imputation" refers to the process of rationally filling in missing data for an incomplete dataset.

[1478] "Emotional values" are data that quantifies a user's emotional state and refer to information obtained through facial recognition and voice analysis.

[1479] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[1480] Server Operation

[1481] Data collection

[1482] The server receives real-time driving data from the device, including vehicle speed, brake usage, steering operation, lane keeping, and environmental data. This data is collected from various sensors (e.g., vehicle speed sensor, brake sensor, steering sensor) installed on the device.

[1483] Data Preprocessing

[1484] The server performs pre-processing on the received driving data, which includes the following operations:

[1485] Noise removal: Use the Python Pandas library to remove unwanted noise from the collected data.

[1486] Normalization: Using scaling techniques to convert data values ​​to a uniform scale.

[1487] Missing Value Support: Use Scikit-learn's SimpleImputer to properly impute missing data.

[1488] Data analysis

[1489] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1490] Anomaly detection: Use anomaly detection algorithms such as Isolation Forest to detect sudden behavior (sudden braking, swerving, etc.).

[1491] Pattern Recognition: Running deep learning models using Keras to identify anomalous driving patterns compared to typical driving patterns.

[1492] Risk assessment: Quantify the driving risk based on the analysis results.

[1493] Emotional Data Integration

[1494] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment. For example, OpenCV is used to recognize the driver's facial expressions and Google Cloud Speech-to-Text is used for voice analysis. This integration generates a comprehensive driving risk report.

[1495] Warning generation

[1496] The server generates warnings based on driving risk assessment and emotion data. There are three warning levels: low risk, medium risk, and high risk. Different warning formats are generated for each level. For example, messages such as "Be careful," "There is a problem with your driving pattern," and "Consider an emergency stop" are generated.

[1497] Notifications and Reporting

[1498] The server sends warning information to the device and, if necessary, generates a monthly report and provides it to the user (driver and family members). For example, a warning message is displayed on the device, and an audio alarm is sounded if there is a high risk.

[1499] Device behavior

[1500] Real-time data collection and transmission

[1501] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as speed sensors, brake sensors, and steering sensors, and the collected data is sent to a server.

[1502] Warning display

[1503] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1504] Emotion Engine Operation

[1505] emotion recognition

[1506] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[1507] Sending emotional data

[1508] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[1509] User Actions

[1510] Check your driving rating

[1511] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[1512] Behavioral adjustment

[1513] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[1514] Considering returning your license

[1515] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[1516] Specific examples

[1517] Example 1: Elderly driver driving on the highway

[1518] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[1519] Example 2: Family members checking monthly reports

[1520] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[1521] Example prompt

[1522] Example of input prompt for generative AI model:

[1523] "Please explain in detail how the system works, where the server detects anomalies based on data collected from the device when an elderly driver is driving on a highway and generates a medium-risk warning."

[1524] Through these specific processes, the present invention can accurately assess the driving risks of elderly drivers and provide effective driving assistance.

[1525] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1526] Step 1: Data collection

[1527] The server receives driving data from the device in real time. The device is equipped with a vehicle speed sensor, brake sensor, steering sensor, etc., and data collected from these sensors while driving is sent to the server. Input data includes vehicle speed, brake usage, steering angle, lane keeping information, etc. The server receives this data and records it in a log. The output data is a file of the collected driving data.

[1528] Step 2: Data Preprocessing

[1529] The server performs preprocessing on the received driving data. The input data is the collected raw driving data, and the following preprocessing is performed:

[1530] Noise removal: Using the Python Pandas library, outliers and noise are removed. Specifically, this involves filtering out abnormally high vehicle speed data and inconsistent braking data.

[1531] Normalization: Use min-max scaling to convert data values ​​to a uniform scale, for example, scaling vehicle speed data to a range of 0 to 1.

[1532] Missing data handling: Using Scikit-learn's SimpleImputer, we impute missing data with the mean value. For example, we impute missing brake data with the mean value of that brake sensor.

[1533] The output data is pre-processed, clean driving data.

[1534] Step 3: Data analysis

[1535] The server uses machine learning algorithms to analyze the preprocessed data. The input data is the preprocessed driving data, and the following analysis is performed:

[1536] Anomaly Detection: Isolation Forest is used to detect anomalous behaviors such as sudden braking or erratic driving by identifying data points that deviate from normal driving patterns.

[1537] Pattern Recognition: Using Keras, we run deep learning models to analyze driving patterns, for example, learning and comparing driving patterns around sharp turns and on highways.

[1538] Risk Assessment: Generate a risk score based on the analysis results, for example, quantifying risk on a scale of 0 to 100 based on the frequency and severity of abnormal behavior.

[1539] The output data is the driving risk assessment result.

[1540] Step 4: Integrating Emotional Data

[1541] The server receives the user's emotion data recognized by the emotion engine. The input data is emotion data obtained by face recognition and voice analysis, and the following operations are performed:

[1542] Emotion Recognition: Facial expressions are analyzed using OpenCV, and voice tones are analyzed using Google Cloud Speech-to-Text. Specific operations include quantifying the user's emotions, such as tension or anxiety.

[1543] Emotional data integration: Integrate driving risk assessment results with emotional data to generate a comprehensive driving risk report. For example, reassess the overall risk by incorporating emotional scores into the risk score.

[1544] The output data is a comprehensive driving risk report.

[1545] Step 5: Generate warnings

[1546] The server generates an alert based on the comprehensive driving risk report. The input data is the comprehensive driving risk report, and the following operations are performed:

[1547] Warning level setting: Sets the warning level (low risk, medium risk, high risk) based on the risk score. Specifically, if the risk score is 50 or above, it is considered a medium risk, and if it is 80 or above, it is considered a high risk.

[1548] Warning message generation: Generates a warning message according to each risk level. For example, for low risk, it generates a message saying "Be careful," for medium risk, it generates a message saying "There is a problem with your driving pattern," and for high risk, it generates a message saying "Consider making an emergency stop."

[1549] The output data is a warning message.

[1550] Step 6: Notifications and Reporting

[1551] The server sends the generated warning messages to the terminal and also generates monthly reports if necessary. The input data are the warning messages and the analyzed driving data, and the following operations are performed:

[1552] Warning notification: A warning message is sent to the device, which then displays it to the driver. Specifically, the device displays the warning message on the screen and sounds an audio alarm if the risk is high.

[1553] Report Generation: Historical driving data is aggregated to generate monthly reports, including driving risk assessments and details of abnormal driving patterns.

[1554] The output data is a warning message and a monthly report.

[1555] Through each of the above steps, a system will be realized that evaluates the driving risks of elderly drivers and provides appropriate warnings and driving assistance.

[1556] (Application example 2)

[1557] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1558] There is a need to appropriately assess the driving risks of elderly drivers and support their safe driving. In particular, driving assistance that takes into account the impact of elderly drivers' emotional states on driving risks is necessary. However, conventional technologies mainly assess risks based solely on driving data, and approaches that take driver emotions into account have not been fully implemented. Therefore, a system that incorporates the emotional states of elderly drivers to provide more accurate driving risk assessments and appropriate warnings is needed.

[1559] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1560] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for evaluating driving risks based on the abnormal driving patterns, means for notifying the driver of the warning, means for collecting driver emotion data, means for integrating the emotion data with the driving risk evaluation, and means for adjusting the content and format of the warning based on the emotion data. This enables a comprehensive evaluation of the driving risks of elderly drivers that also takes emotion data into consideration, and makes it possible to provide more appropriate warnings and driving assistance.

[1561] "Driving data" refers to information about driving behavior and vehicle conditions, such as vehicle speed, steering, braking, lane keeping, and environmental data.

[1562] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking, swerving, and unstable speed.

[1563] "Driving risk" is an index of driving safety and danger assessed based on driving data and abnormal driving patterns.

[1564] "Emotional data" is information about the driver's emotional state analyzed from facial expressions, tone of voice, etc.

[1565] "Warnings" are messages containing cautions or instructions that are generated based on driving risk assessment and emotion data.

[1566] "Emotion recognition" is the process of using cameras and microphones to analyze the driver's facial expressions and tone of voice and recognize their emotional state in real time.

[1567] "Data collection means" refers to a device or method for acquiring driving data and emotion data using sensors, cameras, microphones, etc.

[1568] "Data analysis means" refers to algorithms or software that analyzes collected driving data and emotional data to assess abnormal driving patterns and driving risks.

[1569] "Warning generator" is a process that generates appropriate warning messages or reminders based on driving risk assessment and emotion data.

[1570] This invention is a system that generates warnings based on driving risk assessment and emotion data of elderly drivers. This system consists of four elements: a server, a terminal, an emotion recognition engine, and a user.

[1571] Server Operation

[1572] The server is responsible for analyzing driving data and emotion data. The hardware used includes a cloud server, and the software uses machine learning libraries such as TensorFlow.

[1573] Data collection

[1574] The server receives real-time driving and emotion data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1575] Data Preprocessing

[1576] The server performs preprocessing on the received driving data and emotion data, including noise removal, normalization, and missing value handling, to improve the accuracy of data analysis.

[1577] Data analysis

[1578] The preprocessed data is then analyzed using TensorFlow. This analysis process includes anomaly detection, pattern recognition, risk assessment, etc. The server detects abnormal driving patterns and quantifies driving risk.

[1579] Emotional Data Integration

[1580] The server integrates the emotion data received from the emotion recognition engine with the driving risk assessment, thereby generating a comprehensive driving risk report.

[1581] Warning generation

[1582] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[1583] Device behavior

[1584] The device (e.g., smart glasses) is responsible for collecting driving data in real time and sending warnings. The hardware used includes smart glasses, and the software is an application with cloud communication capabilities.

[1585] Real-time data collection and transmission

[1586] The device collects driving data in real time while driving and sends it to a cloud server using sensors such as vehicle speed, brake, and steering.

[1587] Warning display

[1588] The terminal notifies the driver of the risk assessment results received from the server: a visual warning message is displayed in the case of a medium risk, and an audio alarm is sounded in the case of a high risk.

[1589] How the emotion recognition engine works

[1590] The emotion recognition engine is responsible for collecting and analyzing driver emotion data, and uses software such as Microsoft Azure Cognitive Services.

[1591] emotion recognition

[1592] The emotion recognition engine uses cameras and microphones to analyze the driver's facial expressions and tone of voice in real time to recognize their emotional state.

[1593] Sending emotional data

[1594] The recognized emotion data is sent to a cloud server and integrated into driving risk assessment.

[1595] User Actions

[1596] Users (drivers and their families) can review the generated warnings and reports and adjust their driving behavior.

[1597] Check your driving rating

[1598] Users can view real-time alerts from their devices and comprehensive risk reports generated by the server.

[1599] Behavioral adjustment

[1600] The user (driver) adjusts their driving behavior based on the warnings and their emotional state.

[1601] Considering returning your license

[1602] Users (family members) consider the timing of when elderly drivers should return their licenses based on monthly reports and past driving data.

[1603] Specific examples

[1604] Example 1: Elderly driver driving on the highway

[1605] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. In addition, an emotion recognition engine detects driver tension, and the content and tone of the warning are adjusted accordingly.

[1606] Example 2: Family members checking monthly reports

[1607] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[1608] Example prompts to input to the generative AI model

[1609] "Develop a smart glasses application that monitors the driving ability of elderly drivers and provides driving assistance and appropriate warnings by integrating emotional data. This application will collect real-time driving data and analyze it on a cloud server. It will also use a camera and microphone to analyze the driver's facial expressions and tone of voice to obtain emotional data. Based on the analysis results, it will generate appropriate warnings and notify the driver."

[1610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1611] Step 1:

[1612] Real-time data collection

[1613] The device collects driving data (vehicle speed, braking, steering, lane keeping, and environmental data) in real time from the vehicle's onboard computer. The data is acquired by various sensors (vehicle speed sensor, brake sensor, steering sensor, etc.) attached to the device and sent to a cloud server via Bluetooth or Wi-Fi.

[1614] Input: Driving data collected on the device

[1615] Output: Driving data sent to the cloud server

[1616] Step 2:

[1617] Collecting Emotional Data

[1618] The device's camera and microphone are used to capture the driver's facial expressions and tone of voice. The captured emotional data is analyzed in real time to determine the driver's emotional state (tension, relief, etc.). The collected emotional data is sent to a cloud server.

[1619] Input: Driver's facial expressions and voice data captured by camera and microphone

[1620] Output: Emotion data sent to the cloud server

[1621] Step 3:

[1622] Data Preprocessing

[1623] The server receives the driving data and emotion data and performs preprocessing such as noise removal, normalization, and missing value completion. This preprocessing improves the accuracy of the data, which in turn improves the accuracy of the analysis.

[1624] Input: Driving data and emotion data received by the server

[1625] Output: Preprocessed data

[1626] Step 4:

[1627] Data analysis

[1628] TensorFlow is used to analyze the preprocessed data, and machine learning algorithms are used to detect abnormal driving patterns and quantify driving risk assessments.

[1629] Input: Preprocessed driving and emotion data

[1630] Output: Abnormal driving patterns and driving risk assessment

[1631] Step 5:

[1632] Emotional Data Integration

[1633] The server integrates the emotion data and driving risk assessment based on the analysis results, thereby comprehensively assessing the driving risk and generating a comprehensive risk report.

[1634] Input: Abnormal driving patterns, driving risk assessment, emotional data

[1635] Output: Comprehensive risk report

[1636] Step 6:

[1637] Warning generation

[1638] The server generates alerts based on the integrated data, with three levels of alerts: low risk, medium risk, and high risk, and generates different warning messages and audio alarms for each risk level.

[1639] Input: Comprehensive Risk Report

[1640] Output: Warning message or audio alarm

[1641] Step 7:

[1642] Warning notice

[1643] The device receives warning notifications from the server and provides visual and audio warnings to the driver: in the case of a medium risk, a warning message is displayed on the device's display, and in the case of a high risk, an audio alarm is sounded.

[1644] Input: Warning notification from the server

[1645] Output: Visual and audio warnings for the driver

[1646] Step 8:

[1647] Report generation and delivery

[1648] A monthly driving risk assessment report is automatically generated on the cloud server and sent to users (drivers and their families) via email, allowing them to check past driving data and risk assessment details.

[1649] Input: Past driving risk assessment data

[1650] Output: Monthly driving risk report

[1651] The above is the flow of specific processing steps for carrying out the present invention.

[1652] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1653] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1654] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1655] [Fourth embodiment]

[1656] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1657] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1658] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1659] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1660] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1661] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1662] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1663] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1664] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1665] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1666] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1667] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1668] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1669] This system monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[1670] Server Operation

[1671] Data collection

[1672] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1673] Data Preprocessing

[1674] The server performs pre-processing on the received driving data, which includes the following operations:

[1675] Noise removal: Removing unwanted noise from collected data.

[1676] Normalization: Transforming data values ​​onto a uniform scale.

[1677] Missing Values: Properly impute missing data.

[1678] Data analysis

[1679] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1680] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1681] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1682] Risk assessment: Quantify the driving risk based on the analysis results.

[1683] Warning generation

[1684] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[1685] Device behavior

[1686] Real-time data collection and transmission

[1687] The terminal (on-board computer) collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors.

[1688] Warning display

[1689] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1690] User Actions

[1691] Check your driving rating

[1692] Users (drivers and their families) can check warnings and regular reports from their devices and servers, allowing them to accurately understand their own driving risks.

[1693] Behavioral adjustment

[1694] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1695] Considering returning your license

[1696] The user (family member) considers the timing of when the elderly driver should return their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license return.

[1697] Specific examples

[1698] Example 1: Elderly driver driving on the highway

[1699] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[1700] Example 2: Family members checking monthly reports

[1701] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[1702] This invention makes it possible to quantitatively evaluate the driving risk of elderly drivers and issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[1703] The processing flow will be explained below.

[1704] Understood. Below, the program processing will be explained in detail by dividing it into steps.

[1705] Server Processing

[1706] Step 1:

[1707] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1708] Step 2:

[1709] The server performs pre-processing on the received driving data, which includes the following operations:

[1710] Noise removal: Removing unwanted noise from collected data.

[1711] Normalization: Transforming data values ​​onto a uniform scale.

[1712] Missing Values: Properly impute missing data.

[1713] Step 3:

[1714] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1715] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1716] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1717] Risk assessment: Quantify the driving risk based on the analysis results.

[1718] Step 4:

[1719] The server generates warnings based on the driving risk assessment, which can be set to three levels: low risk, medium risk, or high risk, with different types of warnings generated for each level.

[1720] Step 5:

[1721] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[1722] Terminal handling

[1723] Step 1:

[1724] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1725] Step 2:

[1726] The device transmits the collected data to the server in real time.

[1727] Step 3:

[1728] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1729] User Action

[1730] Step 1:

[1731] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[1732] Step 2:

[1733] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1734] Step 3:

[1735] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the assessment results indicate a persistent high risk, the family member will consult with the elderly driver and proceed with the license surrender.

[1736] The above are the specific processing steps of the program.

[1737] Example 1

[1738] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1739] The decline in driving ability of elderly drivers is a factor that increases the risk of traffic accidents. Current driving risk assessment systems lack preprocessing capabilities for high-precision analysis and have difficulties in collecting data and assessing risk in real time. Therefore, there is a need for a system that can accurately assess the driving behavior of elderly drivers and provide appropriate warnings according to their driving risk.

[1740] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1741] In this invention, the server includes means for collecting driving data of elderly drivers, means for preprocessing the driving data, means for analyzing the preprocessed driving data to detect abnormal driving patterns, means for assessing driving risks based on the abnormal driving patterns, means for generating warnings based on the driving risk assessment, and means for notifying the drivers of the warnings, thereby enabling real-time monitoring of elderly drivers' driving behaviors, evaluating driving risks, and providing appropriate warnings.

[1742] "Driving data" refers to information about older drivers' driving behavior and vehicle conditions, such as vehicle speed, braking, steering, lane keeping, and environmental data.

[1743] "Preprocessing" refers to a series of data processing operations carried out to make collected driving data easier to analyze, including, for example, noise removal, data normalization, and missing value completion.

[1744] "Abnormal driving patterns" refer to patterns that detect deviations from normal driving behavior, such as sudden braking or swerving.

[1745] "Driving risk" refers to a quantitative assessment of the degree of danger posed by a driver's driving behavior. For example, it is assessed on a three-level scale: low risk, medium risk, and high risk.

[1746] "Warning" refers to a message or alarm that notifies the driver of a driving risk and alerts them to the risk, including, for example, a text message or an audio alarm.

[1747] "Real-time" refers to data collection and processing occurring almost instantaneously, with no delay, allowing for immediate analysis and alerts.

[1748] "Server" refers to the computer system that receives, pre-processes, analyzes, assesses risk, and generates alerts on driving data.

[1749] "Terminal" refers to a device that collects driving data, transmits it to a server, and notifies the driver of warnings, including, for example, an in-vehicle computer.

[1750] "Driver" refers to an elderly person who drives a vehicle.

[1751] "Notification means" refers to the method of conveying a warning message or alarm to the driver, including, for example, display or audio output.

[1752] MODE FOR CARRYING OUT THE INVENTION

[1753] The present invention is a system that monitors the driving ability of elderly drivers, evaluates their driving risk, and provides driving assistance or warnings when necessary. This system mainly consists of three elements: a server, a terminal, and a user.

[1754] Server Operation

[1755] Data collection

[1756] The server receives real-time driving data sent from the device. Driving data includes vehicle speed, braking, steering, lane keeping, and environmental data. This information is collected from the device's speed, brake, and steering sensors and sent to the server via WiFi. Specifically, the server uses a web framework such as "Flask" to build an API endpoint to receive the data.

[1757] Data Preprocessing

[1758] The server converts the received driving data into a data frame format and performs the following pre-processing.

[1759] Noise removal: Filtering unwanted noise from data using the "SciPy" library.

[1760] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[1761] Missing value handling: Use Python's "pandas" library to impute missing data with the mean value.

[1762] This prepares the data in a form suitable for analysis.

[1763] Data analysis

[1764] The server uses the preprocessed data to perform the following analysis:

[1765] Anomaly detection: Uses an "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[1766] Pattern Recognition: Uses "k-means clustering" to compare and identify normal and abnormal driving patterns.

[1767] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[1768] These algorithms are implemented in the "scikit-learn" library.

[1769] Warning generation

[1770] The server generates appropriate warnings based on the driving risk assessment, with three levels of warning:

[1771] Low risk: Generates a text warning message.

[1772] Medium risk: Generates audio alarm instructions in addition to text messages.

[1773] High risk: Generates a loud audio alarm and an emergency stop instruction.

[1774] The generated alert is sent to the terminal.

[1775] Device behavior

[1776] Real-time data collection and transmission

[1777] The device collects driving data in real time and sends it to a server. Data is collected using an on-board computer such as a Raspberry Pi, and data is acquired through sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1778] Warning display

[1779] The terminal notifies the driver of the warnings received from the server. If the risk is medium, a warning message is displayed on the terminal's display, and if the risk is high, an audio alarm is sounded. The actual notification is performed using an LCD display and speaker, and these output devices are controlled using the GPIO pins of Arduino or Raspberry Pi.

[1780] User Actions

[1781] Check your driving rating

[1782] Users (drivers and their families) can accurately understand their own driving risks by checking warnings and regular reports from their devices and servers. The regular reports include past driving data and its analysis results.

[1783] Behavioral adjustment

[1784] The user (driver) adjusts their driving behavior according to the warnings: if they receive a medium-risk warning, they drive more carefully, and if they receive a high-risk warning, they avoid driving at all.

[1785] Considering returning your license

[1786] The user (family member) considers the timing of the elderly driver's license surrender based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, a family meeting is held to encourage the driver to surrender their license.

[1787] Specific examples

[1788] Example 1: Elderly driver driving on the highway

[1789] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Caution! Speed ​​is unstable." The device notifies the driver, who then acknowledges the message and adjusts their driving behavior to stabilize the speed.

[1790] Example 2: Family members checking monthly reports

[1791] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving status. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of surrendering the driver's license.

[1792] Examples of prompt statements

[1793] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[1794] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1795] Server Operation

[1796] Step 1: Data collection

[1797] Input: Driving data sent from the device (vehicle speed, braking, steering, lane keeping, environmental data)

[1798] Processing: The server receives these data in real time.

[1799] Output: Raw driving data

[1800] Specific operation: The server uses a web framework such as "Flask" to build an API endpoint to receive data, and receives it from the device via WiFi.

[1801] Step 2: Data Preprocessing

[1802] Input: Raw driving data

[1803] Processing: Preprocess the data in the following ways:

[1804] Denoising: Filtering unwanted noise from data using the "SciPy" library.

[1805] Normalization: Use the "scikit-learn" library to convert the data to a uniform scale.

[1806] Missing value handling: Impute missing data with the mean value using the "pandas" library.

[1807] Output: Preprocessed data

[1808] Specific operation: The server converts the data into a data frame format and applies each preprocessing operation sequentially to prepare the data.

[1809] Step 3: Data analysis

[1810] Input: Preprocessed data

[1811] Processing: Analysis using machine learning algorithms:

[1812] Anomaly detection: Uses the "Isolation Forest" algorithm to detect anomalous behavior such as sudden braking or swerving.

[1813] Pattern recognition: Using "k-means clustering" to compare normal and abnormal driving patterns.

[1814] Risk assessment: Based on the analysis results, driving risk is quantified and classified as "low risk," "medium risk," or "high risk."

[1815] Output: Driving risk assessment results

[1816] What it does: The server uses the "scikit-learn" library to implement these algorithms, analyze the data, and generate a risk assessment result.

[1817] Step 4: Generate alerts

[1818] Input: Driving risk assessment results

[1819] Action: Generate a warning message depending on the driving risk:

[1820] Low risk: Generates a text warning message

[1821] Medium risk: Generates audio alarm instructions in addition to text messages

[1822] High risk: Generates a loud audio alarm and an emergency stop instruction

[1823] Output: Warning message

[1824] Specific operation: Based on the evaluation result, the server generates an appropriate warning message and sends it to the terminal.

[1825] Device behavior

[1826] Step 1: Real-time data collection and transmission

[1827] Input: Data from vehicle speed sensor, brake sensor, and steering sensor

[1828] Processing: Data is collected and sent to a server via WiFi.

[1829] Output: Driving data to be transmitted

[1830] How it works: An onboard computer such as a Raspberry Pi collects data from various sensors and sends it to a server.

[1831] Step 2: Warning display

[1832] Input: The warning message sent by the server

[1833] Action: Notify the driver with a warning message:

[1834] Medium risk: A warning message appears on the display

[1835] High risk: sound an audio alarm

[1836] Output: Warnings notified to the driver

[1837] What it does: The device displays messages on the LCD display and plays audio alarms through the speaker. It uses the GPIO pins of Arduino and Raspberry Pi to control these devices.

[1838] User Actions

[1839] Step 1: Check your driving rating

[1840] Input: Alert messages and scheduled reports from terminals and servers

[1841] Action: Check the driving evaluation results

[1842] Output: Check the driving risk assessment results

[1843] Specific operation: The user checks the evaluation report on the device display or through the smartphone app.

[1844] Step 2: Adjust your behavior

[1845] Input: The warning message received

[1846] Action: Adjust driving behavior based on the warning

[1847] Output: Adjusted driving behavior

[1848] Specific actions: Drivers will immediately improve their driving behavior and drive more cautiously.

[1849] Step 3: Consider surrendering your license

[1850] Input: Monthly reports and historical driving data

[1851] Processing: Consider the timing of license surrender based on the evaluation results

[1852] Output: Decision to surrender license if necessary

[1853] Specific actions: Hold a family meeting and consider surrendering the driver's license if the high risk assessment continues.

[1854] Examples of prompt statements

[1855] "Collect real-time driving data of elderly drivers, detect abnormal behavior, and implement an algorithm to perform risk assessment. The following data will be available: vehicle speed, braking, steering, lane keeping, and environmental data. Preprocess each data and analyze it using a machine learning algorithm. Use Isolation Forest for anomaly detection and k-means clustering for pattern recognition. Generate a warning based on the risk assessment result and notify the device."

[1856] (Application example 1)

[1857] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1858] Monitoring driving risks and ensuring safety for elderly drivers are important social issues. In particular, when elderly drivers use self-driving vehicles, systems that assess driving risks and provide warnings or driving assistance as necessary are needed. Conventional technologies have struggled to effectively assess these risks in real time and provide appropriate warnings or assistance. Furthermore, there has been insufficient coordination between advanced analysis using generative AI models and prompt sentences and the self-driving system. Therefore, it is necessary to develop a system that can accurately assess the driving risks of elderly drivers and provide appropriate warnings and assistance.

[1859] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1860] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for assessing driving risk based on the abnormal driving patterns, means for generating a warning based on the driving risk assessment, means for notifying the driver of the warning, means for the automated driving system to take over driving operations as necessary based on the driving risk, means for analyzing the driving data using a generative AI model to assess risk and generate a warning, and means for configuring the driving risk and warning generation process based on prompt sentences. This makes it possible to assess the driving risk of elderly drivers in real time, detect abnormal driving patterns, and provide appropriate warnings and driving assistance.

[1861] An "elderly driver" refers to a driver who is over a certain age and for whom there are concerns about the risks associated with age when driving.

[1862] "Driving Data" refers to a range of data collected while driving, such as vehicle speed, braking usage, steering angle, lane position and environmental data.

[1863] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking or swerving.

[1864] "Driving risk" is an indicator that indicates the likelihood that a driver will cause an accident while driving, and includes an evaluation value that is quantified through data analysis.

[1865] "Warning" refers to a warning message or alarm that is sent to the driver when driving risk exceeds a certain level.

[1866] "Autonomous driving system" refers to technology that enables a vehicle to perform driving operations without driver intervention.

[1867] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and makes predictions and analyses on new data.

[1868] A "prompt" is a piece of text that serves as an instruction input to a generative AI model.

[1869] MODE FOR CARRYING OUT THE INVENTION

[1870] This paper describes an embodiment of an "elderly driver safety support system" for improving safety when elderly drivers use automated driving vehicles. The system of the present invention is mainly composed of four elements: a server, an in-vehicle terminal, an automated driving system, and a user.

[1871] Server Operation

[1872] Data collection

[1873] The server receives driving data transmitted in real time from the in-vehicle device. This driving data includes vehicle speed, braking, steering, lane position, and environmental data. For example, data is collected using vehicle speed sensors, brake sensors, steering sensors, cameras, and various environmental sensors.

[1874] Data Preprocessing

[1875] The server performs pre-processing on the received driving data, which includes the following operations:

[1876] Noise removal: Removing unwanted noise from collected data.

[1877] Normalization: Transforming data values ​​onto a uniform scale.

[1878] Missing Values: Properly impute missing data.

[1879] Data analysis

[1880] The server uses the generative AI model to analyze the preprocessed data. This analysis process includes the following steps:

[1881] Anomaly detection: Uses algorithms such as Isolation Forest to detect sudden behavior (hard braking, swerving, etc.).

[1882] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1883] Risk assessment: Quantify the driving risk based on the analysis results.

[1884] Warning generation

[1885] The server generates warnings based on the driving risk assessment. There are three warning levels: low risk, medium risk, and high risk. Different types of warnings are generated for each level. For example, a medium risk warning message is displayed on the screen, and a high risk warning sounds an audio alarm.

[1886] In-vehicle terminal operation

[1887] Real-time data collection and transmission

[1888] The in-vehicle device collects driving data in real time and sends it to a server. Data is collected using sensors such as vehicle speed sensors, brake sensors, and steering sensors. The data collected in real time is sent to the server using wireless communication technology (e.g., Wi-Fi or 5G).

[1889] Warning display

[1890] The in-vehicle device receives the risk assessment results from the server and notifies the driver. If the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1891] Autonomous driving system operation

[1892] Driving assistance measures

[1893] The automated driving system automatically performs driving operations based on the driving risk assessment received from the server. For example, if the risk is high, the system takes over control of the vehicle and drives safely.

[1894] User Actions

[1895] Check your driving rating

[1896] Users (drivers and their families) can check warnings and regular reports from the in-vehicle terminal or server, allowing them to accurately understand their own driving risks.

[1897] Behavioral adjustment

[1898] The user (driver) adjusts their driving behavior according to the warning. If they receive a medium-risk warning, they will drive more carefully, and if they receive a high-risk warning, they will refrain from driving. For example, the server can generate a warning message saying, "Be careful. Your speed is unstable," and the device can notify the driver of this.

[1899] Specific examples

[1900] An example of a prompt sentence to input to the generative AI model is as follows:

[1901] "Assess driving risks in real time based on driving data of elderly drivers. Data includes vehicle speed, braking, steering, lane keeping, and environmental data. Detect abnormal patterns, generate risk scores, and generate appropriate warning messages."

[1902] This system makes it possible to assess the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[1903] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1904] Step 1:

[1905] The server receives driving data in real time from the in-vehicle terminal. The input driving data includes vehicle speed, braking, steering operation, lane position, and environmental data. This data is transmitted to the server using wireless communication technology (e.g., Wi-Fi or 5G). The received driving data is stored in data storage and used for subsequent processing.

[1906] Step 2:

[1907] The server performs preprocessing on the received driving data. The input for this step is the data collected in step 1. First, noise removal is performed, and then data normalization is performed. Noise removal involves filtering out sensor errors and unnecessary data points. Data normalization involves converting data values ​​to a unified scale to facilitate comparison between different sensors. Finally, missing data is imputed appropriately. Specifically, SimpleImputer is used to impute missing values ​​with the mean value. This preprocessing results in a dataset that can be analyzed.

[1908] Step 3:

[1909] The server uses the generative AI model to analyze the preprocessed data. The input for this step is the preprocessed data from step 2. First, anomaly detection is performed using the Isolation Forest algorithm, which detects abnormal driving patterns (e.g., sudden braking or swerving). Next, pattern recognition is used to identify anomalies by comparing them with general driving patterns. Finally, the server evaluates the driving risk based on the analysis results and generates a risk score. This risk score is obtained as the output.

[1910] Step 4:

[1911] The server generates a warning based on the generated driving risk score. The input for this step is the risk score obtained in step 3. The generated warnings are divided into three levels: low risk, medium risk, and high risk, and different types of warning messages are prepared for each level. Specifically, a text warning message is displayed for medium risk, and an audio alarm is sounded for high risk. This warning message is sent from the server to the in-vehicle terminal.

[1912] Step 5:

[1913] The in-vehicle terminal notifies the driver of the warning message received from the server. The input of this step is the warning message generated in step 4. If the risk is medium, a warning message is displayed on the in-vehicle terminal's display. If the risk is high, an audio alarm is used to inform the driver of the urgency. This warning notification prompts the driver to adjust their driving behavior. The output of this step is a warning notification to the driver, and the driver is expected to act in accordance with the warning content.

[1914] Step 6:

[1915] The automated driving system takes over driving operations as necessary based on the driving risk assessment results received from the server. The input to this step is the driving risk assessment result obtained in step 4. If it is deemed high risk, the automated driving system takes over control and performs appropriate driving operations, such as rapidly decelerating the vehicle or guiding it to a safe route. The output of this step is the execution of safe driving operations.

[1916] Through the above steps, the "elderly driver safety support system" of the present invention is able to evaluate the driving risks of elderly drivers in real time and provide appropriate warnings and driving assistance.

[1917] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1918] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[1919] Server Operation

[1920] Data collection

[1921] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1922] Data Preprocessing

[1923] The server performs pre-processing on the received driving data, which includes the following operations:

[1924] Noise removal: Removing unwanted noise from collected data.

[1925] Normalization: Transforming data values ​​onto a uniform scale.

[1926] Missing Values: Properly impute missing data.

[1927] Data analysis

[1928] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1929] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1930] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1931] Risk assessment: Quantify the driving risk based on the analysis results.

[1932] Emotional Data Integration

[1933] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment, which generates a comprehensive driving risk report.

[1934] Warning generation

[1935] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[1936] Notifications and Reporting

[1937] The server sends warning information to the terminal and, if necessary, generates a monthly report to provide to the user (driver and family).

[1938] Device behavior

[1939] Real-time data collection and transmission

[1940] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1941] Warning display

[1942] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[1943] Emotion Engine Operation

[1944] emotion recognition

[1945] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[1946] Sending emotional data

[1947] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[1948] User Actions

[1949] Check your driving rating

[1950] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[1951] Behavioral adjustment

[1952] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[1953] Considering returning your license

[1954] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[1955] Specific examples

[1956] Example 1: Elderly driver driving on the highway

[1957] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[1958] Example 2: Family members checking monthly reports

[1959] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[1960] This invention quantitatively evaluates the driving risk of elderly drivers and takes into account the user's emotions, enabling it to issue warnings at appropriate times, thereby improving the safety of elderly drivers themselves and other road users.

[1961] The processing flow will be explained below.

[1962] Server Processing

[1963] Step 1:

[1964] The server receives real-time driving data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[1965] Step 2:

[1966] The server performs pre-processing on the received driving data, which includes the following operations:

[1967] Noise removal: Removing unwanted noise from collected data.

[1968] Normalization: Transforming data values ​​onto a uniform scale.

[1969] Missing Values: Properly impute missing data.

[1970] Step 3:

[1971] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[1972] Anomaly detection: Detects sudden behavior (sudden braking, swerving, etc.).

[1973] Pattern Recognition: Identifying abnormal driving patterns compared to typical driving patterns.

[1974] Risk assessment: Quantify the driving risk based on the analysis results.

[1975] Step 4:

[1976] The server receives the user's emotion data sent from the emotion engine and integrates it into a driving risk assessment.

[1977] Step 5:

[1978] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set as follows:

[1979] Low risk: No immediate action required.

[1980] Medium risk: Display a warning message on the form screen.

[1981] High Risk: Display audio and visual warnings.

[1982] Step 6:

[1983] The server sends the warning information to the terminal and also generates a monthly report to provide to the user (driver and family).

[1984] Terminal handling

[1985] Step 1:

[1986] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as the vehicle speed sensor, brake sensor, and steering sensor.

[1987] Step 2:

[1988] The device transmits the collected data to the server in real time.

[1989] Step 3:

[1990] The device notifies the driver of the risk assessment results and warnings received from the server. The notification format is as follows:

[1991] Medium risk: Display a warning message on the form screen saying "Please be careful. Drive safely."

[1992] High Risk: An audio alarm will be issued and a warning message will be displayed saying "High Risk. Do not drive."

[1993] Emotion engine processing

[1994] Step 1:

[1995] The emotion engine recognizes the user's emotions in real time through facial recognition and voice analysis, collecting the user's facial expressions and tone of voice through the camera and microphone.

[1996] Step 2:

[1997] The emotion engine analyzes the collected data and assesses the user's emotional state (e.g., tension, anxiety, calmness, etc.) in real time.

[1998] Step 3:

[1999] The emotion engine sends the recognized emotion data to the server.

[2000] User Action

[2001] Step 1:

[2002] Users (drivers and their families) receive real-time alerts from their devices and monthly reports generated by the server.

[2003] Step 2:

[2004] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[2005] Step 3:

[2006] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[2007] The above are the specific processing steps in a system that combines emotion engines.

[2008] Example 2

[2009] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2010] As older drivers age, their reaction time and attention span decline, increasing the risk of accidents. However, current driver assistance systems rely solely on driving data and do not take into account the driver's emotional state. This makes it difficult to properly assess the impact of driver emotions on driving behavior and issue appropriate warnings in a timely manner. Furthermore, some systems suffer from poor data preprocessing accuracy, which can lead to noise and missing values, reducing the accuracy of driving risk assessment.

[2011] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2012] In this invention, the server includes a device for collecting driving data of elderly drivers, a device for analyzing the driving data to detect abnormal driving behavior, a device for evaluating driving risk based on the abnormal driving behavior, a data preprocessing device for performing noise removal, numerical normalization, and missing value completion, and a device for recognizing user emotion data and integrating it into the driving risk evaluation. This makes it possible to integrate driving data and emotion data to perform a comprehensive driving risk evaluation and issue appropriate warnings in a timely manner.

[2013] "Elderly drivers" generally refer to drivers who are older and whose driving skills and reaction times may be declining.

[2014] "Driving statistics" refers to various data collected while driving, such as vehicle speed, brake usage, steering angle, and lane keeping information.

[2015] "Equipment" means an entity of hardware or software designed and manufactured to perform a specific function.

[2016] "Abnormal driving behavior" refers to detecting driving behavior that deviates from normal driving patterns, such as sudden braking or swerving.

[2017] "Driving risk" is a quantitative or qualitative assessment of the risk level of driving behavior, and is an indicator of potential threats to safety.

[2018] A "warning" is a warning message issued to drivers that includes content encouraging safe driving.

[2019] "Denoising" refers to the process of removing unwanted noise from collected data.

[2020] "Numeric normalization" refers to the process of converting data from different scales or units to a unified scale.

[2021] "Missing value imputation" refers to the process of rationally filling in missing data for an incomplete dataset.

[2022] "Emotional values" are data that quantifies a user's emotional state and refer to information obtained through facial recognition and voice analysis.

[2023] This invention is a system that monitors the driving ability of elderly drivers, evaluates driving risks, and provides driving assistance and appropriate warnings by combining an emotion engine that recognizes the user's emotions. This system mainly consists of four elements: a server, a terminal, an emotion engine, and a user.

[2024] Server Operation

[2025] Data collection

[2026] The server receives real-time driving data from the device, including vehicle speed, brake usage, steering operation, lane keeping, and environmental data. This data is collected from various sensors (e.g., vehicle speed sensor, brake sensor, steering sensor) installed on the device.

[2027] Data Preprocessing

[2028] The server performs pre-processing on the received driving data, which includes the following operations:

[2029] Noise removal: Use the Python Pandas library to remove unwanted noise from the collected data.

[2030] Normalization: Using scaling techniques to convert data values ​​to a uniform scale.

[2031] Missing Value Support: Use Scikit-learn's SimpleImputer to properly impute missing data.

[2032] Data analysis

[2033] The server uses machine learning algorithms to analyze the preprocessed data. The analysis process includes the following steps:

[2034] Anomaly detection: Use anomaly detection algorithms such as Isolation Forest to detect sudden behavior (sudden braking, swerving, etc.).

[2035] Pattern Recognition: Running deep learning models using Keras to identify anomalous driving patterns compared to typical driving patterns.

[2036] Risk assessment: Quantify the driving risk based on the analysis results.

[2037] Emotional Data Integration

[2038] The server receives the user's emotion data recognized by the emotion engine and integrates it with the driving risk assessment. For example, OpenCV is used to recognize the driver's facial expressions and Google Cloud Speech-to-Text is used for voice analysis. This integration generates a comprehensive driving risk report.

[2039] Warning generation

[2040] The server generates warnings based on driving risk assessment and emotion data. There are three warning levels: low risk, medium risk, and high risk. Different warning formats are generated for each level. For example, messages such as "Be careful," "There is a problem with your driving pattern," and "Consider an emergency stop" are generated.

[2041] Notifications and Reporting

[2042] The server sends warning information to the device and, if necessary, generates a monthly report and provides it to the user (driver and family members). For example, a warning message is displayed on the device, and an audio alarm is sounded if there is a high risk.

[2043] Device behavior

[2044] Real-time data collection and transmission

[2045] The terminal (on-board computer) collects driving data in real time while driving, using sensors such as speed sensors, brake sensors, and steering sensors, and the collected data is sent to a server.

[2046] Warning display

[2047] The device receives the risk assessment results from the server and notifies the driver: if the risk is medium, a warning message is displayed on the screen, and if the risk is high, an audio alarm is sounded.

[2048] Emotion Engine Operation

[2049] emotion recognition

[2050] The emotion engine recognizes emotions in real time through facial recognition and voice analysis, for example, by using a camera and microphone to analyze the user's facial expressions and tone of voice.

[2051] Sending emotional data

[2052] The emotion engine transmits the recognized emotion data to the server, which can then integrate the emotion data into driving risk assessment.

[2053] User Actions

[2054] Check your driving rating

[2055] Users (drivers and their families) receive real-time warnings from their devices and view comprehensive risk reports generated by the server.

[2056] Behavioral adjustment

[2057] The user (driver) adjusts their driving behavior based on the warnings and their emotional state: driving more cautiously if they receive a medium-risk warning, and refraining from driving if they receive a high-risk warning.

[2058] Considering returning your license

[2059] The user (family member) considers the timing of when the elderly driver should surrender their license based on monthly reports and past driving data. If the evaluation results indicate a persistent high risk, the family will consult with the elderly driver and, taking into account the emotional data, proceed with the license surrender.

[2060] Specific examples

[2061] Example 1: Elderly driver driving on the highway

[2062] The server receives data from the device and detects abnormal speed fluctuations. If the analysis results indicate a medium risk, the server generates a warning message saying, "Be careful. Your speed is unstable." The device notifies the driver of this message, who then acknowledges it and adjusts their driving behavior to stabilize their speed. The emotion engine also detects driver tension and adjusts the content and tone of the warning accordingly.

[2063] Example 2: Family members checking monthly reports

[2064] The server aggregates driving data from the past month and generates a monthly report. This report includes a driving risk assessment and details of abnormal driving patterns. The user (family member) checks this report to understand the driver's recent driving condition. If the assessment results indicate a high risk, a family meeting is held to discuss the timing of license surrender, taking into account emotional data.

[2065] Example prompt

[2066] Example of input prompt for generative AI model:

[2067] "Please explain in detail how the system works, where the server detects anomalies based on data collected from the device when an elderly driver is driving on a highway and generates a medium-risk warning."

[2068] Through these specific processes, the present invention can accurately assess the driving risks of elderly drivers and provide effective driving assistance.

[2069] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2070] Step 1: Data collection

[2071] The server receives driving data from the device in real time. The device is equipped with a vehicle speed sensor, brake sensor, steering sensor, etc., and data collected from these sensors while driving is sent to the server. Input data includes vehicle speed, brake usage, steering angle, lane keeping information, etc. The server receives this data and records it in a log. The output data is a file of the collected driving data.

[2072] Step 2: Data Preprocessing

[2073] The server performs preprocessing on the received driving data. The input data is the collected raw driving data, and the following preprocessing is performed:

[2074] Noise removal: Using the Python Pandas library, outliers and noise are removed. Specifically, this involves filtering out abnormally high vehicle speed data and inconsistent braking data.

[2075] Normalization: Use min-max scaling to convert data values ​​to a uniform scale, for example, scaling vehicle speed data to a range of 0 to 1.

[2076] Missing data handling: Using Scikit-learn's SimpleImputer, we impute missing data with the mean value. For example, we impute missing brake data with the mean value of that brake sensor.

[2077] The output data is pre-processed, clean driving data.

[2078] Step 3: Data analysis

[2079] The server uses machine learning algorithms to analyze the preprocessed data. The input data is the preprocessed driving data, and the following analysis is performed:

[2080] Anomaly Detection: Isolation Forest is used to detect anomalous behaviors such as sudden braking or erratic driving by identifying data points that deviate from normal driving patterns.

[2081] Pattern Recognition: Using Keras, we run deep learning models to analyze driving patterns, for example, learning and comparing driving patterns around sharp turns and on highways.

[2082] Risk Assessment: Generate a risk score based on the analysis results, for example, quantifying risk on a scale of 0 to 100 based on the frequency and severity of abnormal behavior.

[2083] The output data is the driving risk assessment result.

[2084] Step 4: Integrating Emotional Data

[2085] The server receives the user's emotion data recognized by the emotion engine. The input data is emotion data obtained by face recognition and voice analysis, and the following operations are performed:

[2086] Emotion Recognition: Facial expressions are analyzed using OpenCV, and voice tones are analyzed using Google Cloud Speech-to-Text. Specific operations include quantifying the user's emotions, such as tension or anxiety.

[2087] Emotional data integration: Integrate driving risk assessment results with emotional data to generate a comprehensive driving risk report. For example, reassess the overall risk by incorporating emotional scores into the risk score.

[2088] The output data is a comprehensive driving risk report.

[2089] Step 5: Generate warnings

[2090] The server generates an alert based on the comprehensive driving risk report. The input data is the comprehensive driving risk report, and the following operations are performed:

[2091] Warning level setting: Sets the warning level (low risk, medium risk, high risk) based on the risk score. Specifically, if the risk score is 50 or above, it is considered a medium risk, and if it is 80 or above, it is considered a high risk.

[2092] Warning message generation: Generates a warning message according to each risk level. For example, for low risk, it generates a message saying "Be careful," for medium risk, it generates a message saying "There is a problem with your driving pattern," and for high risk, it generates a message saying "Consider making an emergency stop."

[2093] The output data is a warning message.

[2094] Step 6: Notifications and Reporting

[2095] The server sends the generated warning messages to the terminal and also generates monthly reports if necessary. The input data are the warning messages and the analyzed driving data, and the following operations are performed:

[2096] Warning notification: A warning message is sent to the device, which then displays it to the driver. Specifically, the device displays the warning message on the screen and sounds an audio alarm if the risk is high.

[2097] Report Generation: Historical driving data is aggregated to generate monthly reports, including driving risk assessments and details of abnormal driving patterns.

[2098] The output data is a warning message and a monthly report.

[2099] Through each of the above steps, a system will be realized that evaluates the driving risks of elderly drivers and provides appropriate warnings and driving assistance.

[2100] (Application example 2)

[2101] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2102] There is a need to appropriately assess the driving risks of elderly drivers and support their safe driving. In particular, driving assistance that takes into account the impact of elderly drivers' emotional states on driving risks is necessary. However, conventional technologies mainly assess risks based solely on driving data, and approaches that take driver emotions into account have not been fully implemented. Therefore, a system that incorporates the emotional states of elderly drivers to provide more accurate driving risk assessments and appropriate warnings is needed.

[2103] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2104] In this invention, the server includes means for collecting driving data of elderly drivers, means for analyzing the driving data to detect abnormal driving patterns, means for evaluating driving risks based on the abnormal driving patterns, means for notifying the driver of the warning, means for collecting driver emotion data, means for integrating the emotion data with the driving risk evaluation, and means for adjusting the content and format of the warning based on the emotion data. This enables a comprehensive evaluation of the driving risks of elderly drivers that also takes emotion data into consideration, and makes it possible to provide more appropriate warnings and driving assistance.

[2105] "Driving data" refers to information about driving behavior and vehicle conditions, such as vehicle speed, steering, braking, lane keeping, and environmental data.

[2106] "Abnormal driving patterns" refer to driving behavior that deviates from normal driving behavior, such as sudden braking, swerving, and unstable speed.

[2107] "Driving risk" is an index of driving safety and danger assessed based on driving data and abnormal driving patterns.

[2108] "Emotional data" is information about the driver's emotional state analyzed from facial expressions, tone of voice, etc.

[2109] "Warnings" are messages containing cautions or instructions that are generated based on driving risk assessment and emotion data.

[2110] "Emotion recognition" is the process of using cameras and microphones to analyze the driver's facial expressions and tone of voice and recognize their emotional state in real time.

[2111] "Data collection means" refers to a device or method for acquiring driving data and emotion data using sensors, cameras, microphones, etc.

[2112] "Data analysis means" refers to algorithms or software that analyzes collected driving data and emotional data to assess abnormal driving patterns and driving risks.

[2113] "Warning generator" is a process that generates appropriate warning messages or reminders based on driving risk assessment and emotion data.

[2114] This invention is a system that generates warnings based on driving risk assessment and emotion data of elderly drivers. This system consists of four elements: a server, a terminal, an emotion recognition engine, and a user.

[2115] Server Operation

[2116] The server is responsible for analyzing driving data and emotion data. The hardware used includes a cloud server, and the software uses machine learning libraries such as TensorFlow.

[2117] Data collection

[2118] The server receives real-time driving and emotion data from the device, including vehicle speed, braking, steering, lane keeping, and environmental data.

[2119] Data Preprocessing

[2120] The server performs preprocessing on the received driving data and emotion data, including noise removal, normalization, and missing value handling, to improve the accuracy of data analysis.

[2121] Data analysis

[2122] The preprocessed data is then analyzed using TensorFlow. This analysis process includes anomaly detection, pattern recognition, risk assessment, etc. The server detects abnormal driving patterns and quantifies driving risk.

[2123] Emotional Data Integration

[2124] The server integrates the emotion data received from the emotion recognition engine with the driving risk assessment, thereby generating a comprehensive driving risk report.

[2125] Warning generation

[2126] The server generates warnings based on driving risk assessment and emotion data. The warning levels are set to three levels: low risk, medium risk, and high risk, and different types of warnings are generated for each level.

[2127] Device behavior

[2128] The device (e.g., smart glasses) is responsible for collecting driving data in real time and sending warnings. The hardware used includes smart glasses, and the software is an application with cloud communication capabilities.

[2129] Real-time data collection and transmission

[2130] The device collects driving data in real time while driving and sends it to a cloud server using sensors such as vehicle speed, brake, and steering.

[2131] Warning display

[2132] The terminal notifies the driver of the risk assessment results received from the server: a visual warning message is displayed in the case of a medium risk, and an audio alarm is sounded in the case of a high risk.

[2133] How the emotion recognition engine works

[2134] The emotion recognition engine is responsible for collecting and analyzing driver emotion data, and uses software such as Microsoft Azure Cognitive Services.

[2135] emotion recognition

[2136] The emotion recognition engine uses cameras and microphones to analyze the driver's facial expressions and tone of voice in real time to recognize their emotional state.

[2137] Sending emotional data

[2138] The recognized emotion data is sent to a cloud server and integrated into driving risk assessment.

[2139] User Actions

[2140] Users (drivers and their families) can review the generated warnings and reports and...

Claims

1. A means of collecting driving data on older drivers; means for analyzing the driving data to detect abnormal driving patterns; means for assessing driving risk based on the abnormal driving pattern; means for generating an alert based on the driving risk assessment; means for notifying a driver of the warning; A system including:

2. The system of claim 1 further comprising means for setting warning levels and issuing different types of warnings for medium and high risk.

3. The system of claim 1 , further comprising means for collecting driving data in real time and transmitting said driving data to a server in real time.

4. The system of claim 1 further comprising means for denoising, normalising and imputing missing values ​​of the driving data using pre-processing means.

5. The system of claim 1 further comprising means for generating monthly reports and providing historical driving trends to the driver and family members.

6. The system according to claim 1, further comprising means for displaying the driving risk assessment in numerical form and notifying a family member of the numerical value in order to assist the elderly driver in surrendering his / her driver's license.

Images