System

Abstract

A system is provided.SOLUTION: A system including a means for collecting sensor data acquired by an in-vehicle device, a means for analyzing the sensor data in real time and providing advice or an alert to a driver, a means for transmitting collected data to a cloud server and evaluating a driving behavior based on the data, a means for calculating a driving score based on the evaluation, and a means for feeding back the driving score to a driver and notifying an insurance company of the driving score and applying a discount to an insurance premium.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the increase in traffic accidents has become a social problem, with accidents caused by novice drivers and those who are not good at driving. This problem is caused by inexperienced drivers and a lack of proper judgment of road conditions. For this reason, there is an urgent need to provide appropriate driving instruction to these drivers in real time and help them develop safe driving habits. However, current driver assistance systems are inadequate in providing real-time instruction, evaluating driving behavior, and providing incentives to drive drivers. A system that can solve this issue and also provide concrete incentives such as insurance premium discounts is needed. [Means for solving the problem]

[0005] The present invention provides a system that collects sensor data acquired by in-vehicle devices. This allows the driver to receive real-time advice and warnings. The collected data is then sent to a cloud server to evaluate driving behavior. The evaluation is calculated as a driving score and fed back to the driver. This driving score is also notified to insurance companies, which apply insurance premium discounts. The present invention uses a driving evaluation method that incorporates game elements, and sets and provides goals and rewards to the driver based on the driving score. The driving score is also registered in the driver's account and provided to the driver as visual information via a smartphone app. This helps to continuously improve drivers' skills and encourages them to develop safe driving habits.

[0006] An "in-vehicle device" is a device installed in a vehicle that collects data such as speed, acceleration, position, and obstacles ahead.

[0007] "Sensor data" refers to information such as speed, acceleration, GPS information, and camera footage obtained from in-vehicle devices.

[0008] A "cloud server" is a remote server accessed via the Internet, and is a system that analyzes and stores driving data, calculates driving scores, and more.

[0009] "Driving behavior" refers to actions such as accelerating, decelerating, turning, and stopping when a driver operates a vehicle.

[0010] The "driving score" is a numerical evaluation of the driver's driving behavior, and is an index showing the degree of safe driving.

[0011] An "insurance company" is a company that provides automobile insurance and offers premium discounts and coverage to policyholders.

[0012] "Game elements" are entertainment elements such as achievement goals, rewards, and rankings that are used to increase driver motivation.

[0013] "Feedback" is the process of communicating the results of analysis and evaluation to the driver, providing information to help improve driving skills.

[0014] An "account" is identification information used to manage the data of a specific driver, and is linked to scores and evaluation results. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] MODE FOR CARRYING OUT THE INVENTION

[0037] This system is realized by linking an in-vehicle device, a cloud server, and a user device (smartphone app) installed in the vehicle. This system provides appropriate advice and warnings to the driver in real time, evaluates driving behavior to calculate a driving score, and notifies the score to the insurance company to apply discounts on insurance premiums.

[0038] 1. Data Collection

[0039] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously acquires data from these sensors.

[0040] Example: When the vehicle exceeds a certain speed, the device measures the speed information in real time and also acquires acceleration data from the G sensor and location data from the GPS sensor.

[0041] 2. Real-time analysis and user notification

[0042] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if the distance to the vehicle ahead is too close or if the speed limit is exceeded, the device will warn the driver with a voice message or a display.

[0043] Example: When the distance to the vehicle ahead falls below a certain level, the device notifies the user with a voice message saying, "The distance to the vehicle ahead is too short. Please reduce your speed."

[0044] 3. Data transmission and cloud analysis

[0045] The device periodically transmits the collected data to a cloud server, where it is integrated and analyzed in detail.

[0046] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0047] 4. DriveScore calculation and feedback

[0048] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." The calculated DriveScore is registered in the user's account and can be viewed from the user's device.

[0049] Example: The server calculates the DriveScore based on data such as the number of sudden braking attempts, fluctuations in acceleration, and speed compliance, and provides the results so that the user can check them on a smartphone app.

[0050] 5. Gamification and rewards

[0051] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a certain score is reached, the driver will be given a digital badge and displayed at the top of the weekly rankings.

[0052] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0053] 6. Insurance premium discounts

[0054] With the user's consent, the server provides the calculated DriveScore to the insurance company, which then applies a discount to the insurance premium based on the score.

[0055] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0056] This invention makes it easier for novice drivers and inexperienced drivers to develop safe driving habits, which is expected to reduce the risk of traffic accidents. In addition, by evaluating driving behavior, it is possible to provide specific incentives and increase user motivation.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] As soon as the vehicle's engine starts, the device begins collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information such as current speed, acceleration, position, and distance to obstacles ahead.

[0060] Step 2:

[0061] The device stores the collected sensor data in temporary memory and performs pre-processing such as noise removal and data filtering, which removes outliers and smooths the location data.

[0062] Step 3:

[0063] The device analyzes the pre-processed data in real time to assess the driving situation, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short.

[0064] Step 4:

[0065] The device will provide real-time advice and reminders to the driver as needed through voice messages and display alerts, such as a notification that says, "You are too close to the vehicle ahead. Please reduce your speed."

[0066] Step 5:

[0067] The device transfers the collected and pre-processed data to the cloud server in batches at regular intervals (e.g., every 30 minutes). This data includes information on speed, position, acceleration, and distance to obstacles ahead.

[0068] Step 6:

[0069] The server then combines the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking attempts, sudden acceleration attempts, etc.

[0070] Step 7:

[0071] Based on the integrated data, the server calculates a driving score (DriveScore) using parameters such as safe driving, smooth acceleration and deceleration, and speed compliance.

[0072] Step 8:

[0073] The server registers the calculated DriveScore in the driver's account and allows the driver to check the result on their smartphone app. For example, the server provides information such as "This week's DriveScore is 88 points."

[0074] Step 9:

[0075] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore are given rewards such as a digital badge or a place at the top of the weekly rankings.

[0076] Step 10:

[0077] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result. For example, the insurance company may notify the user that "Your DriveScore is 90 or higher, so you will receive a 10% discount on your insurance premium."

[0078] This will establish a complete process from collection to analysis, feedback, incentive provision, and insurance premium discount application.

[0079] Example 1

[0080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0081] In modern vehicle driving, systems that promote safe driving and evaluate driver behavior still face challenges. In particular, there is no well-established system that evaluates driving behavior in real time, provides appropriate feedback, and links that evaluation to insurance premium discounts. Furthermore, there is still a lack of gamification elements to increase driver motivation and methods to easily visualize driving scores.

[0082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0083] In this invention, the server includes: means for collecting multiple types of sensor data acquired by devices installed in the vehicle; means for analyzing the multiple sensor data in real time and providing advice and warnings to the driver; means for transmitting the collected data to a remote server via a network and evaluating driving behavior based on the data; means for calculating a driving score based on the evaluation; means for providing the driving score as feedback to the driver and notifying an external institution and applying a fee discount; means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements and providing them based on the driving score; and means for registering the driving score in the driver's account and providing it to the driver as visual information via a mobile device application. This promotes safe driving, appropriately evaluates the driver's behavior, increases motivation, and enables the application of insurance premium discounts.

[0084] A "vehicle-mounted device" is a hardware device that is placed inside a vehicle and uses multiple sensors and cameras to collect driving data.

[0085] "Sensor data" refers to information obtained from speed sensors, G sensors, GPS sensors, and on-board cameras, including vehicle speed, acceleration, position, and images.

[0086] "Real-time analysis" means that data is processed as soon as it is collected, providing immediate feedback to the driver.

[0087] "Advice and warnings" refers to providing suggestions and warnings to encourage improvement based on the driver's driving behavior in the form of voice messages or display on the screen.

[0088] "Transmitting to a remote server via a network" refers to transmitting the collected data to a server in a remote location via a communication network such as the Internet.

[0089] The "means for evaluating driving behavior" is software or an algorithm that analyzes the driver's driving behavior based on the collected data and makes an evaluation based on that analysis.

[0090] A "driving score" is an index that quantifies the evaluation results of driving behavior and allows drivers to understand their driving performance at a glance.

[0091] "External organizations" are third-party organizations that are expected to notify driving scores, and mainly refer to insurance companies and traffic safety organizations.

[0092] "Means for applying discounts" refers to a system that reduces insurance premiums and other fees for drivers based on their assessed driving score.

[0093] A "driving evaluation method incorporating game elements" is a method in which driving behavior is evaluated in a game format and goals and rewards are set in order to increase the driver's motivation.

[0094] "Goals and rewards" refers to goals and incentives given to drivers when they achieve certain driving behaviors, including digital badges and ranking displays.

[0095] "Mobile application" means a software application used on a mobile device, such as a smartphone or tablet, to display driving scores and evaluation results.

[0096] "Means for providing visual information" refers to a method for displaying driving scores and evaluation results to the driver in the form of graphs, charts, text, etc.

[0097] MODE FOR CARRYING OUT THE INVENTION

[0098] The present invention is a system that provides real-time driving evaluation and feedback through collaboration between a vehicle device installed in a vehicle, a cloud server, and a user device (e.g., a smartphone app). This system promotes safe driving, evaluates driving behavior, calculates a driving score, and notifies the score to insurance companies to apply discounts on insurance premiums.

[0099] Device configuration and data collection

[0100] The in-vehicle device is equipped with a speed sensor, G sensor, GPS sensor, and in-vehicle camera. These sensors continuously collect the vehicle's speed, acceleration, position, and forward video data. For example, if the vehicle's speed exceeds 100 km / h, the speed information is measured in real time, and acceleration data from the G sensor and position data from the GPS sensor are simultaneously collected.

[0101] Data analysis and user notification

[0102] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." In addition, since footage from the in-car camera is also used for analysis, the device has the function of automatically saving footage of sudden braking and later uploading it to the cloud.

[0103] Data transmission and cloud analysis

[0104] The device periodically transmits the collected data (e.g., every 30 minutes) to a cloud server. This transmission is performed using a secure communication protocol (e.g., HTTPS). The server then integrates the received data and performs a detailed analysis. This analysis uses a big data analysis tool (e.g., Apache (registered trademark) Hadoop) to evaluate the number of sudden braking incidents, the frequency of speeding, and fluctuations in acceleration.

[0105] DriveScore calculation and feedback

[0106] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." This calculation uses machine learning algorithms (e.g., support vector machines and deep learning). The calculated DriveScore is registered in the user's account and can be viewed as visual information on a smartphone app. When the user opens the smartphone app, they can view a detailed report.

[0107] Gamification and rewards

[0108] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a user scores 80 or more out of 100, they will be awarded a digital badge as the "Top Driver of the Week" and notified of this via the smartphone app. Weekly rankings and other information will also be displayed, providing a system that allows users to enjoy their driving behavior in a game-like format.

[0109] Insurance premium discounts

[0110] With the user's consent, the server provides the calculated DriveScore to the insurance company. The insurance company applies a discount on the insurance premium based on this information. For example, if a user receives a DriveScore of 90 or more, the data is sent to the insurance company and the insurance premium is discounted. The user is notified of this discount via a smartphone app.

[0111] Specific examples

[0112] For example, if the following situation occurs:

[0113] The terminal detects when the vehicle's speed exceeds 100 km / h and measures speed information in real time.

[0114] When the device detects sudden braking, it automatically saves five seconds of footage from the onboard camera and uploads it to a cloud server.

[0115] The server uses data from the past 30 days to calculate a driving score based on factors such as the number of sudden braking attempts and speed limits, and allows users to check this score via a smartphone app.

[0116] The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0117] Prompt Sentence Examples

[0118] Example prompts to be input to the generative AI model:

[0119] Please explain how the system connects in-car devices, cloud servers, and user devices to provide real-time driving ratings and insurance discounts. Please provide examples.

[0120] These steps promote safe driving and increase motivation by providing drivers with tangible incentives.

[0121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0122] Step 1: Data collection

[0123] The terminal collects data from the speed sensor, G sensor, GPS sensor, and on-board camera installed in the in-vehicle device. For example, the speed sensor measures the vehicle's speed once per second, the G sensor acquires acceleration data, and the GPS sensor acquires location data. The on-board camera captures video of the vehicle ahead. All of this data is collected in real time. The input data to the terminal are speed, acceleration, location, and video data, and an initial data set is generated by collecting these in real time.

[0124] Step 2: Real-time analysis

[0125] The device analyzes the data collected in step 1 in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." The input data is the collected sensor data, and the output data is a warning message to the driver.

[0126] Step 3: Send data

[0127] The device periodically (e.g., every 30 minutes) transmits data that has been collected and analyzed in real time to a cloud server. This transmission is performed using a secure communication protocol (e.g., HTTPS). The input data here is the analyzed sensor data, and the output data is the integrated data transmitted to the cloud server. Specifically, the device uploads speed, acceleration, position, and video data to the cloud server every 30 minutes.

[0128] Step 4: Cloud Solve

[0129] The server integrates the data received in step 3 and performs detailed analysis. The server processes large amounts of data using big data analysis tools (e.g., Apache Hadoop). It evaluates the number of sudden braking attempts, the frequency of speeding, and fluctuations in acceleration to analyze driving behavior. The input data is the integrated data sent to the cloud server, and the output data is the analyzed driving behavior data. Specifically, it analyzes the driver's driving patterns based on data from the past 30 days.

[0130] Step 5: Calculate your DriveScore

[0131] The server calculates a driving score, "DriveScore," based on the results of the cloud analysis. This calculation uses machine learning algorithms (such as support vector machines and deep learning). For example, the score is calculated taking into account the number of sudden braking attempts, speed compliance, and fluctuations in acceleration. The input data is the analyzed driving behavior data, and the output data is the calculated DriveScore. Specifically, the server tallies points for each score item based on the evaluation results.

[0132] Step 6: Provide feedback

[0133] The server registers the calculated DriveScore in the user's account and notifies them via a smartphone app. For example, a detailed driving evaluation report can be provided in PDF format. Furthermore, gamification elements can be set, such as goals and rewards. For example, a "Top Driver of the Week" badge can be awarded. The input data is the DriveScore and evaluation results, and the output data is feedback information to the driver.

[0134] Step 7: Apply for discount on premium

[0135] With the user's consent, the server provides the calculated DriveScore to the insurance company and applies a discount on the insurance premium. For example, the insurance company applies a discount of several percent to users with a DriveScore of 90 or more. The input data is the DriveScore, and the output data is a notification to the insurance company and information on the discount application. Specifically, the server sends the evaluation score to the insurance company and notifies the user of the discount application result.

[0136] Through these steps, the system can promote safe driving, specifically evaluate driver behavior, provide feedback, and even link insurance premium discounts.

[0137] (Application example 1)

[0138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0139] Conventional in-vehicle device systems were limited in the advice and warnings they could give drivers, and the feedback they provided in real time was insufficient. Furthermore, the evaluation of driving behavior and the provision of incentives based on that evaluation were also limited, and no mechanism was in place to increase driver motivation. This meant that the inability to provide appropriate feedback and incentives to drivers resulted in the system not being effective enough in promoting safe driving. Furthermore, there were security concerns regarding the provision of data to insurance companies.

[0140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0141] In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, means for feeding back the driving score to the driver and notifying an insurance company and applying a discount on insurance premiums, and means for sending a warning message regarding the driving score to a smartphone app, smart glasses, or an in-vehicle robot. This makes it possible to provide appropriate information to the driver in real time, increase the driver's motivation through a driving evaluation method incorporating game elements, and securely provide data to insurance companies.

[0142] An "in-vehicle device" is a device installed in a vehicle to collect data from speed sensors, G sensors, GPS sensors, cameras, etc.

[0143] "Sensor data" refers to data collected from various sensors, including vehicle speed, acceleration, location information, and video data acquired by on-board cameras.

[0144] "Real-time analysis" refers to instantly processing collected sensor data and making evaluations and decisions according to the situation.

[0145] "Advice and warnings" are information and warning messages about driving behavior provided to the driver.

[0146] A "cloud server" is a server used via the Internet, a remote server for storing and analyzing collected data.

[0147] "Means for evaluating driving behavior" refers to the process of analyzing and evaluating the driver's driving patterns and behavior based on collected sensor data.

[0148] The "driving score" is a numerical representation of the evaluation results of driving behavior, and serves as an indicator of the safety and smoothness of a driver's driving.

[0149] "Feedback" refers to conveying driving scores and warning messages to drivers.

[0150] "Notifying insurance company" means sending data such as driving scores to the insurance company in a secure manner.

[0151] "Premium discount" refers to a reduction in premiums applied to a policyholder based on their driving score.

[0152] A "smartphone app" is application software that runs on a smartphone and provides various notifications and feedback to the driver.

[0153] "Smart glasses" are glasses-type devices that have the function of overlaying information onto the driver's vision.

[0154] An "in-vehicle robot" is a robotic device that is installed inside a vehicle and provides warnings and advice to the driver through voice and screen displays.

[0155] "Providing appropriate information in real time" means providing drivers with advice and warnings that respond to their driving behavior immediately.

[0156] A "driving evaluation method incorporating game elements" is a method for evaluating a driver's driving behavior using playful elements such as rankings and badges, thereby increasing the driver's motivation.

[0157] "Providing data securely" refers to using technologies and protocols to ensure safety during the data transmission process.

[0158] This system promotes safe driving by providing real-time feedback to drivers through collaboration between in-vehicle devices, a cloud server, and user devices (such as smartphone apps and smart glasses). This system can also evaluate driving behavior, calculate a driving score, and apply insurance discounts.

[0159] 1. Data Collection

[0160] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. Data from these sensors is collected in real time. Specific hardware that can be used is an embedded Linux device such as a Raspberry Pi.

[0161] 2. Real-time analysis

[0162] The collected data is analyzed in real time using machine learning libraries such as TENSORFLOW (registered trademark) and PyTorch, which run in a Python environment. Based on the analysis results, the driver's driving behavior is evaluated and appropriate advice or warning messages are provided. For example, if the distance to the vehicle ahead is too close, a voice message will be issued saying, "The distance to the vehicle ahead is too close. Please reduce your speed."

[0163] 3. Data transmission and cloud analysis

[0164] The collected data is sent to a cloud server at regular intervals, where it is analyzed using cloud platforms such as AWS (registered trademark) and Google (registered trademark) Cloud, with services such as Amazon S3 and AWS Lambda performing big data analysis.

[0165] 4. DriveScore calculation and feedback

[0166] The cloud server integrates and analyzes the collected data, evaluates driving behavior, and calculates a driving score, "DriveScore," based on various factors. This DriveScore is provided as feedback to the user's device (smartphone app or smart glasses).

[0167] 5. Gamification and rewards

[0168] The cloud server sets goals and rewards for drivers based on the calculated DriveScore. User progress is managed using databases such as Firebase and MongoDB. For example, drivers with a DriveScore of 80 or higher are awarded a digital badge called "Top Driver of the Week" and notified via a smartphone app.

[0169] 6. Insurance premium discounts

[0170] The server then provides the driving score to insurance companies using a secure API, which then applies discounts to insurance premiums based on the data. The data is transmitted using a security protocol such as OAuth 2.0.

[0171] Specific examples

[0172] For example, if a driver suddenly brakes, a notification will appear on the smart glasses saying, "Sudden braking has been detected. This will affect your driving score." The data will also be sent to a cloud server for further analysis.

[0173] Prompt Sentence Examples

[0174] "Generate an algorithm that will display a warning message about your driving score if it detects sudden braking. Also include example code to send the data to the cloud for further analysis."

[0175] With the above configuration, the present invention can provide appropriate feedback to drivers in real time, evaluate their driving behavior, and improve their motivation based on the evaluation.In addition, data can be securely provided to insurance companies, allowing them to apply discounts on insurance premiums.

[0176] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0177] Step 1:

[0178] The in-vehicle device (terminal) collects data from the speed sensor, G sensor, GPS sensor, and on-board camera. This includes the vehicle's speed, acceleration, location information, and forward video. The input is raw data from each sensor, and the output is integrated sensor data. This collected data is sent to the next processing stage in real time.

[0179] Step 2:

[0180] The device analyzes the collected data in real time. It processes and analyzes the data using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. For example, it analyzes fluctuations in speed and acceleration and generates a warning message if sudden braking or acceleration is detected. The input is the integrated sensor data, and the output is the analysis result and whether or not a warning is issued. This warning message is immediately notified to the driver.

[0181] Step 3:

[0182] The device periodically transmits the analyzed data to the cloud server. This transmission occurs, for example, every 30 minutes, in preparation for the integration and analysis of large amounts of data in the cloud. The input is the analyzed sensor data, and the output is the data transmitted to the cloud server.

[0183] Step 4:

[0184] The cloud server integrates the received data and performs detailed analysis. This analysis is performed using cloud platform services (e.g., AWS Lambda and Amazon S3) such as AWS and Google Cloud. The input is the data sent to the cloud, and the output is the evaluation results of the driver's driving behavior. This evaluation is multidimensional, and each element of driving behavior (such as speed limits and the number of sudden braking attempts) is evaluated.

[0185] Step 5:

[0186] The server calculates a driving score (DriveScore) based on the evaluation results. Each element of driving behavior is calculated using an algorithm and expressed as an overall score. The input is the evaluation result of driving behavior, and the output is the DriveScore. This score is registered in the user's account.

[0187] Step 6:

[0188] The server provides feedback to the user's device based on the calculated DriveScore. The score and warning messages are displayed on the smartphone app or smart glasses. The input is the DriveScore, and the output is the feedback information displayed on the smartphone app or smart glasses.

[0189] Step 7:

[0190] The server sets goals and rewards for the driver based on the acquired DriveScore. The user's progress is managed using a database such as Firebase or MongoDB. The input is the DriveScore, and the output is the goal and reward information sent to the user's device.

[0191] Step 8:

[0192] The server provides the DriveScore of consenting users to the insurance company using a secure API (e.g., OAuth 2.0), which then applies premium discounts based on the DriveScore. The input is the DriveScore, and the output is the data sent to the insurance company and the applied premium discount.

[0193] As described above, a real-time safe driving feedback system is realized by processing and analyzing the data obtained at each step and proceeding with the process sequentially.

[0194] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0195] MODE FOR CARRYING OUT THE INVENTION

[0196] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The emotion engine can recognize the driver's emotional state in real time and provide appropriate advice and warnings based on that information.

[0197] 1. Data Collection

[0198] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously collects data from these sensors. The emotion engine also analyzes the driver's facial expressions and tone of voice to obtain emotional data.

[0199] Example: The device collects real-time data on the vehicle's speed, acceleration, and location, as well as changes in the driver's facial expressions and voice.

[0200] 2. Real-time analysis and user notification

[0201] The device analyzes sensor data and emotion data in real time to evaluate the driver's driving situation and emotional state. The emotion engine detects the driver's emotions such as stress, impatience, and fatigue, and provides advice and warnings at the appropriate time.

[0202] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0203] 3. Data transmission and cloud analysis

[0204] The device periodically transfers the collected and pre-processed data (speed, acceleration, position, and emotion data) in batches to a cloud server.

[0205] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0206] 4. DriveScore calculation and feedback

[0207] The server evaluates the driving behavior and emotional data based on the integrated data to calculate a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation. The calculated DriveScore is registered in the user's account and can be viewed on the user's device.

[0208] Example: The server calculates the DriveScore based on the number of sudden braking incidents, fluctuations in acceleration, and the driver's emotional data, and provides the results to the user so that they can be checked on a smartphone app.

[0209] 5. Gamification and rewards

[0210] The server sets goals and rewards for users based on the calculated DriveScore. For example, users who reach a certain score will receive a digital badge or be ranked high in the weekly rankings.

[0211] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0212] 6. Insurance premium discounts

[0213] With the user's consent, the server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[0214] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0215] This system not only helps novice and inexperienced drivers develop safe driving habits, but also significantly reduces driving risks by taking their emotional state into account. Furthermore, by providing specific incentives and feedback to users, it can highly motivate them to drive safely.

[0216] The processing flow will be explained below.

[0217] Step 1:

[0218] As soon as the vehicle's engine starts, the device starts collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information on the current speed, acceleration, position, distance to obstacles ahead, etc. The emotion engine also analyzes the driver's facial expressions and tone of voice to collect emotion data.

[0219] Step 2:

[0220] The device temporarily stores the collected sensor data and emotion data, and performs preprocessing such as noise reduction and data filtering to remove outliers and smooth the location data. The device also filters out ambiguous emotion determinations from the emotion data.

[0221] Step 3:

[0222] The device analyzes the pre-processed data in real time to assess the driving situation and the driver's emotional state, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short, and assessing the driver's emotional state if it indicates impatience or stress.

[0223] Step 4:

[0224] The device provides real-time advice and warnings to the driver as needed through voice messages and display, and the emotion engine changes the content and tone of the advice based on the driver's emotional state at the time.

[0225] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0226] Step 5:

[0227] The device transfers the collected and pre-processed data (speed, acceleration, location, emotion data) to a cloud server in batches at regular intervals (e.g., every 30 minutes), which includes all data points during driving.

[0228] Step 6:

[0229] The server then integrates the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking and accelerations, as well as emotional fluctuations.

[0230] Step 7:

[0231] The server evaluates the driver's driving safety, acceleration and deceleration smoothness, speed adherence rate, and emotional state based on the integrated data, and calculates a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation.

[0232] Step 8:

[0233] The server registers the calculated DriveScore in the driver's account, allowing the driver to check the results on a smartphone app.

[0234] Example: The server displays the information "This week's DriveScore is 88" on the user's smartphone app.

[0235] Step 9:

[0236] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore will be rewarded with a digital badge or a high ranking in the weekly rankings.

[0237] Example: The server notifies drivers with a DriveScore of 80 or above with a "Top Driver of the Week" badge on their smartphone app.

[0238] Step 10:

[0239] With the user's consent, the server provides the DriveScore to the insurance company, which applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result.

[0240] Example: The server provides data on users whose DriveScore is 90 or higher to an insurance company, and the insurance company applies discounts to insurance premiums based on this data. For example, it sends a notification saying, "Your insurance premium will be discounted by 10%."

[0241] In this way, a system can be built that uses collected data to perform detailed analysis of driving behavior and emotional states, providing drivers with specific feedback and incentives to promote safe driving and even offer discounts on insurance premiums.

[0242] Example 2

[0243] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0244] Conventional driving evaluation systems for in-vehicle devices rely primarily on driving behavior data and do not take into account the driver's emotional state, making it difficult to accurately evaluate driving conditions. Furthermore, they lack the ability to provide real-time feedback and appropriate advice to drivers, making it difficult to promote safe driving. Furthermore, incentive systems and insurance discounts that utilize driving evaluation results have limitations because they do not include emotional data.

[0245] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0246] In this invention, the server includes means for collecting sensor data and emotional data acquired by an in-vehicle device, means for analyzing the sensor data and emotional data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating the driving behavior and emotional state based on the data, means for calculating a driving score that takes into account the driving score and emotional state based on the evaluation, and means for feeding back the driving score to the driver and notifying the insurance company of the score and applying a discount on the insurance premium. This enables more accurate driving evaluation that takes into account not only the driver's driving behavior but also their emotional state, leading to the promotion of safe driving and the provision of appropriate incentives.

[0247] An "on-board device" is a device that is installed in a vehicle and includes multiple sensors such as a speed sensor, a G sensor, a GPS sensor, and an on-board camera.

[0248] "Sensor data" refers to driving-related data such as speed, acceleration, and location, and is information collected from in-vehicle devices.

[0249] "Emotion data" refers to data relating to the driver's emotional state, obtained by analyzing the driver's facial expressions and tone of voice.

[0250] An "emotion engine" is a software or hardware device that analyzes the driver's facial expressions and tone of voice and generates emotion data.

[0251] "Cloud server" refers to a remote server that stores and analyzes data over the Internet, and is the place where collected data is processed and stored.

[0252] The "driving score" is a numerical value that indicates the evaluation result of the driver's driving behavior, calculated by analyzing sensor data and emotional data.

[0253] "Advice and Reminders" refers to suggestions and warning messages provided based on the driver's driving situation and emotional state.

[0254] "Feedback" refers to notifying the driver of the calculated driving score and warnings, and is primarily done through the user's device.

[0255] "Gamification" is a method of incorporating game elements to evaluate a driver's driving behavior and increase motivation by setting goals and rewards.

[0256] "User device" means a device on which a driver checks their driving score and feedback, including, for example, a smartphone or tablet.

[0257] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The specific configuration and operation of this system will be described below.

[0258] In-vehicle devices

[0259] The in-vehicle device is equipped with a speed sensor, a G-sensor, a GPS sensor, and an on-board camera. The device continuously collects data from these sensors. It also uses an emotion engine to analyze the driver's facial expressions and tone of voice to collect emotional data. For example, the vehicle's speed and acceleration are acquired every second while driving, and the driver's facial expressions captured by the on-board camera are analyzed using image recognition software.

[0260] Cloud Server

[0261] Data collected by the device is periodically sent in batches to a cloud server. The cloud server then integrates the received data and prepares it for analysis. The server then analyzes speed, acceleration, position, and emotional data to evaluate the driver's driving behavior and emotional state. This information is then used to calculate a driving score (DriveScore). By including emotional data in addition to sensor data, driving evaluation can be performed with greater accuracy than before.

[0262] Feedback and User Devices

[0263] The server registers the calculated driving score in the user's account and notifies the result via the user's device (e.g., smartphone). For example, a message such as "Your DriveScore is 85" is displayed on the user's device. In addition, real-time warnings and advice are also provided through the user's device. For example, while driving, the user can receive a voice message saying, "Take your time and slow down."

[0264] Gamification and rewards

[0265] The server sets goals and rewards for users based on their driving scores. When a certain score is reached, rewards such as digital badges and weekly rankings are provided. For example, a "Top Driver of the Week" badge is awarded, and a notification appears on the user's smartphone saying, "Congratulations! You're the Top Driver of the Week."

[0266] Insurance premium discounts

[0267] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount on insurance premiums based on the driving score. For example, if a user's DriveScore is 90 or higher, the server provides the data to the insurance company, and the user receives a 10% discount on insurance premiums.

[0268] This system can evaluate the driver's driving behavior and emotional state in detail, promoting safe driving and improving motivation. As a specific example, the following prompt sentences can be input into the generative AI model:

[0269] Example prompt sentence:

[0270] 1. "Describe a system that analyzes the driver's emotional state in real time based on data collected by an in-car device and provides appropriate advice."

[0271] 2. "Please explain in detail how the device will notify the user with a warning message if the user becomes impatient while driving."

[0272] 3. "Please explain what DriveScore is, how it is calculated, and what feedback it provides to users."

[0273] 4. "Can you give us a concrete example of how insurance discounts based on driving data work?"

[0274] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0275] Step 1:

[0276] The terminal collects data in real time from each sensor of the in-vehicle device (speed sensor, G sensor, GPS sensor, and in-vehicle camera). This collected data includes speed, acceleration, location, and the driver's facial expression and tone of voice. An emotion engine is used to analyze this data and generate driver emotion data. The input sensor data and facial expression data become real-time driving data and emotion data, and this data is stored in the terminal.

[0277] Step 2:

[0278] The device analyzes the collected sensor data and emotional data in real time. Specifically, it evaluates the driver's driving behavior and emotional state. This allows it to automatically generate appropriate advice or warnings if the driver appears to be impatient, tired, or stressed. Using the real-time data and emotional data as input, the device generates driving situation evaluation data and emotional evaluation data through analysis.

[0279] Step 3:

[0280] The device then provides the driver with advice or warning messages based on the analysis results. For example, if the driver is in a hurry and speeding up, the device will notify the user with a voice message saying, "Please stay calm and slow down." Based on the evaluation data input, a specific message is generated and notified to the user as voice or display output.

[0281] Step 4:

[0282] The device periodically transfers the collected data (speed, acceleration, position, and emotion data) to the cloud server in batches. Specifically, all data is packaged at regular intervals and sent to the cloud server via a communication line. The accumulated data as input is converted into a batch package and sent to the cloud server.

[0283] Step 5:

[0284] The server consolidates the data entered on the cloud and prepares it for analysis, specifically storing it in a literal database and performing any necessary preprocessing (e.g., data cleansing and format conversion). It receives batch data as input, stores and preprocesses it, and outputs it in an analyzable format.

[0285] Step 6:

[0286] The server evaluates driving behavior and emotional state based on the integrated data and calculates a driving score (DriveScore). Specifically, it analyzes the number of sudden braking attempts, fluctuations in acceleration, and the driver's emotional data to generate a quantified driving score. It evaluates the integrated data as input and outputs a driving score.

[0287] Step 7:

[0288] The server registers the calculated driving score to the user's account and provides it as visual information via the user's device. Specifically, it sends a notification to the user via a smartphone app saying, "Your DriveScore is 85." Using the driving score data as input, it generates a visual feedback message and outputs it to the user's device.

[0289] Step 8:

[0290] The server sets goals and rewards based on the driving score. For example, it can award a "Top Driver of the Week" badge to drivers who reach a certain score and notify them via a smartphone app. The server uses the driving score as input to set rewards and award badges, and notifies the user of the reward information as output.

[0291] Step 9:

[0292] After obtaining the user's consent, the server provides the DriveScore to the insurance company, which then applies a discount to the insurance premium. Specifically, if the user's DriveScore is 90 or higher, the server sends the user's data via API, and the insurance company notifies and applies the premium discount. The server sends the score data as input and receives the result of the premium discount application as output.

[0293] (Application example 2)

[0294] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0295] Conventional automated driving systems perform driving operations without taking into account the driver's emotional state, which means they are unable to effectively reduce driver stress and fatigue. Furthermore, the lack of technology to appropriately adjust automated driving parameters based on the driver's emotional state has prevented sufficient improvements in safety.

[0296] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for analyzing the driver's emotional state and adjusting automated driving parameters based on the data, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, and means for providing feedback about the driving score to the driver, notifying the insurance company, and applying a discount on insurance premiums. This makes it possible to recognize the driver's emotional state in real time and adjust appropriate automated driving parameters based on the driving score. Furthermore, by providing feedback and rewards to the driver, it is possible to encourage improvement in driving behavior and improve safety.

[0297] An "in-vehicle device" is a device that is installed in a vehicle and includes various sensors and cameras for acquiring driving data and environmental data.

[0298] "Sensor data" is a general term for data acquired by in-vehicle devices, such as speed, acceleration, GPS location information, the driver's facial expressions, and voice tone.

[0299] "Advice and Caution" refers to appropriate instructions or warnings provided to the driver based on sensor data and emotional state.

[0300] A "cloud server" is a remote server that stores collected data and performs integrated analysis and evaluation.

[0301] The "means for evaluating driving behavior" is a mechanism that analyzes the driver's driving behavior based on collected sensor data and emotional state, and generates an evaluation result.

[0302] The "driving score" is a numerical index that comprehensively evaluates driving behavior and emotional state.

[0303] "Feedback" is a means of providing information to drivers by notifying them of their driving score, advice, and warnings, encouraging them to improve their driving behavior.

[0304] "Insurance premium discounts" are premium reduction benefits applied by insurance companies based on driving scores.

[0305] "Emotional state" is information that indicates the psychological state of the driver, such as stress, fatigue, or impatience.

[0306] "Autonomous driving parameters" are control elements such as speed, following distance, and braking sensitivity that are set when the autonomous driving system performs driving operations.

[0307] "Game elements" are fun, competitive elements that set goals and rewards to encourage improved driving behavior.

[0308] "Smartphone app" means a software application for a mobile device used to provide visual driving scores and feedback to the driver.

[0309] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. Using the emotion engine, it is possible to recognize the driver's emotional state in real time and adjust autonomous driving parameters based on that. This system is configured by combining the following hardware and software.

[0310] Hardware used

[0311] 1. In-vehicle devices:

[0312] This includes an in-car camera, microphone, speed sensor, G-sensor, and GPS sensor.

[0313] Data is obtained from these sensors in real time.

[0314] 2. Cloud Server:

[0315] A remote server for storing and analyzing data.

[0316] Examples: AWS (Amazon Web Services), Google Cloud, etc.

[0317] 3. User Device:

[0318] Mobile devices such as smartphones and tablets.

[0319] Used to review data and provide feedback.

[0320] Software used

[0321] 1. Emotion Engine:

[0322] An AI model for analyzing facial expressions and tone of voice.

[0323] Examples: OpenAI(R), GOOGLE TENSOR(R) Flow.

[0324] 2. Data analysis platform:

[0325] A tool for integrating and analyzing data.

[0326] For example: Apache Spark, Hadoop.

[0327] 3. Smartphone App:

[0328] An application that displays and notifies user data.

[0329] System Operation

[0330] Data collection

[0331] The in-vehicle device collects real-time sensor data such as speed, acceleration, GPS location, the driver's facial expressions and voice tone, allowing for accurate understanding of the driving situation and the driver's emotional state.

[0332] Real-time analysis and parameter adjustment

[0333] The terminal (in-vehicle device) analyzes the acquired data in real time. The emotion engine analyzes the driver's facial expressions and tone of voice to identify the driver's emotional state, such as stress, fatigue, or impatience. Based on this data, the autonomous driving system's parameters (speed, following distance, braking sensitivity, etc.) are adjusted appropriately.

[0334] Data transmission and cloud analysis

[0335] The device periodically transmits the acquired data to a cloud server, which then comprehensively analyzes the data and evaluates the driver's driving behavior and emotional state.

[0336] DriveScore calculation and feedback

[0337] The cloud server calculates a driving score (DriveScore) based on the integrated data. This score is the result of a comprehensive evaluation of driving behavior and emotional state. The calculated DriveScore is provided as feedback to the driver via the user's device.

[0338] Gamification and rewards

[0339] The cloud server sets goals and rewards for drivers based on their DriveScore. For example, achieving a certain score will earn them a digital badge or a spot at the top of the weekly rankings.

[0340] Insurance premium discounts

[0341] With the user's consent, the cloud server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[0342] Specific examples

[0343] For example, if the in-car device detects that the driver is in an "anger" state from their facial expression, the system will automatically slow down the vehicle and give a voice notification to the driver saying "Please relax." In addition, the collected data is sent to a cloud server every 30 minutes, and a driving score (DriveScore) is calculated through integrated analysis. Users can check this score on their smartphone app.

[0344] Prompt Sentence Examples

[0345] "Perform real-time emotion analysis using driver facial expression data and vehicle driving data, and generate Python code to set appropriate autonomous driving parameters."

[0346] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0347] Step 1: Data collection

[0348] The terminal (in-vehicle device) uses an in-vehicle camera, microphone, speed sensor, G sensor, and GPS sensor to capture the driver's facial expression, tone of voice, vehicle speed, acceleration, and location information in real time. The input data are facial expression data, voice data, speed data, G sensor data, and GPS data, which are sent to the analysis unit after initial processing. The output is initially processed emotion data and driving data.

[0349] Step 2: Sentiment Analysis

[0350] The device uses an emotion engine to analyze the collected facial expression data and voice data in real time. This analysis identifies the driver's emotional state (e.g., stress, fatigue, anger, etc.). The input is initially processed facial expression data and voice data, which are then fed into the emotion engine, which outputs the driver's emotional state.

[0351] Step 3: Real-time analysis and autonomous driving parameter adjustment

[0352] The device combines emotional state and driving data for real-time analysis and adjusts autonomous driving parameters (speed, following distance, braking sensitivity, etc.). The input is driving data and analyzed emotional data, and based on this data, it calculates and outputs autonomous driving control parameters. Specifically, if the driver's emotion is "anger," the device will slow down the vehicle and give a voice notification.

[0353] Step 4: Send data to the cloud server

[0354] The device periodically (e.g., every 30 minutes) sends the collected and preprocessed data to the cloud server in batch format. The input is the preprocessed driving data and emotion data, and the cloud server receives these data. The output is the integrated data sent to the cloud server.

[0355] Step 5: Integrated analysis and DriveScore calculation

[0356] The server (cloud server) comprehensively analyzes the transmitted data, evaluates driving behavior and emotional state, and calculates a DriveScore. The input is collected historical driving data and emotional data, which is used to apply statistical analysis and machine learning models. The output is the driver's DriveScore. A generative AI model is used to comprehensively evaluate emotions and driving behavior.

[0357] Step 6: Provide feedback

[0358] The server provides the calculated DriveScore as feedback to the user's smartphone app. The input is the calculated DriveScore, which is reported to the user device. The output is feedback information displayed on the user device, such as a "Top Driver of the Week" badge.

[0359] Step 7: Gamify and reward

[0360] The server sets goals and rewards based on the DriveScore and provides them to the user. The input is the DriveScore and user profile data, and based on this, the server sets goals and determines rewards. The output is a digital badge and weekly ranking display provided to the user.

[0361] Step 8: Provide data to insurance companies and apply discounts

[0362] With the user's consent, the server provides the insurance company with the user's DriveScore, and the insurance company applies a discount on the premium based on the score. The input is the user's DriveScore and consent information, which are sent to the insurance company. The output is a notification from the insurance company that the discount has been applied.

[0363] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0364] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0365] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0366] [Second embodiment]

[0367] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0368] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0369] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0370] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0371] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0372] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0373] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0374] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0375] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0376] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0377] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0378] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0379] MODE FOR CARRYING OUT THE INVENTION

[0380] This system is realized by linking an in-vehicle device, a cloud server, and a user device (smartphone app) installed in the vehicle. This system provides appropriate advice and warnings to the driver in real time, evaluates driving behavior to calculate a driving score, and notifies the score to the insurance company to apply discounts on insurance premiums.

[0381] 1. Data Collection

[0382] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously acquires data from these sensors.

[0383] Example: When the vehicle exceeds a certain speed, the device measures the speed information in real time and also acquires acceleration data from the G sensor and location data from the GPS sensor.

[0384] 2. Real-time analysis and user notification

[0385] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if the distance to the vehicle ahead is too close or if the speed limit is exceeded, the device will warn the driver with a voice message or a display.

[0386] Example: When the distance to the vehicle ahead falls below a certain level, the device notifies the user with a voice message saying, "The distance to the vehicle ahead is too short. Please reduce your speed."

[0387] 3. Data transmission and cloud analysis

[0388] The device periodically transmits the collected data to a cloud server, where it is integrated and analyzed in detail.

[0389] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0390] 4. DriveScore calculation and feedback

[0391] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." The calculated DriveScore is registered in the user's account and can be viewed from the user's device.

[0392] Example: The server calculates the DriveScore based on data such as the number of sudden braking attempts, fluctuations in acceleration, and speed compliance, and provides the results so that the user can check them on a smartphone app.

[0393] 5. Gamification and rewards

[0394] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a certain score is reached, the driver will be given a digital badge and displayed at the top of the weekly rankings.

[0395] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0396] 6. Insurance premium discounts

[0397] With the user's consent, the server provides the calculated DriveScore to the insurance company, which then applies a discount to the insurance premium based on the score.

[0398] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0399] This invention makes it easier for novice drivers and inexperienced drivers to develop safe driving habits, which is expected to reduce the risk of traffic accidents. In addition, by evaluating driving behavior, it is possible to provide specific incentives and increase user motivation.

[0400] The processing flow will be explained below.

[0401] Step 1:

[0402] As soon as the vehicle's engine starts, the device begins collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information such as current speed, acceleration, position, and distance to obstacles ahead.

[0403] Step 2:

[0404] The device stores the collected sensor data in temporary memory and performs pre-processing such as noise removal and data filtering, which removes outliers and smooths the location data.

[0405] Step 3:

[0406] The device analyzes the pre-processed data in real time to assess the driving situation, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short.

[0407] Step 4:

[0408] The device will provide real-time advice and reminders to the driver as needed through voice messages and display alerts, such as a notification that says, "You are too close to the vehicle ahead. Please reduce your speed."

[0409] Step 5:

[0410] The device transfers the collected and pre-processed data to the cloud server in batches at regular intervals (e.g., every 30 minutes). This data includes information on speed, position, acceleration, and distance to obstacles ahead.

[0411] Step 6:

[0412] The server then combines the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking attempts, sudden acceleration attempts, etc.

[0413] Step 7:

[0414] Based on the integrated data, the server calculates a driving score (DriveScore) using parameters such as safe driving, smooth acceleration and deceleration, and speed compliance.

[0415] Step 8:

[0416] The server registers the calculated DriveScore in the driver's account and allows the driver to check the result on their smartphone app. For example, the server provides information such as "This week's DriveScore is 88 points."

[0417] Step 9:

[0418] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore are given rewards such as a digital badge or a place at the top of the weekly rankings.

[0419] Step 10:

[0420] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result. For example, the insurance company may notify the user that "Your DriveScore is 90 or higher, so you will receive a 10% discount on your insurance premium."

[0421] This will establish a complete process from collection to analysis, feedback, incentive provision, and insurance premium discount application.

[0422] Example 1

[0423] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0424] In modern vehicle driving, systems that promote safe driving and evaluate driver behavior still face challenges. In particular, there is no well-established system that evaluates driving behavior in real time, provides appropriate feedback, and links that evaluation to insurance premium discounts. Furthermore, there is still a lack of gamification elements to increase driver motivation and methods to easily visualize driving scores.

[0425] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0426] In this invention, the server includes: means for collecting multiple types of sensor data acquired by devices installed in the vehicle; means for analyzing the multiple sensor data in real time and providing advice and warnings to the driver; means for transmitting the collected data to a remote server via a network and evaluating driving behavior based on the data; means for calculating a driving score based on the evaluation; means for providing the driving score as feedback to the driver and notifying an external institution and applying a fee discount; means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements and providing them based on the driving score; and means for registering the driving score in the driver's account and providing it to the driver as visual information via a mobile device application. This promotes safe driving, appropriately evaluates the driver's behavior, increases motivation, and enables the application of insurance premium discounts.

[0427] A "vehicle-mounted device" is a hardware device that is placed inside a vehicle and uses multiple sensors and cameras to collect driving data.

[0428] "Sensor data" refers to information obtained from speed sensors, G sensors, GPS sensors, and on-board cameras, including vehicle speed, acceleration, position, and images.

[0429] "Real-time analysis" means that data is processed as soon as it is collected, providing immediate feedback to the driver.

[0430] "Advice and warnings" refers to providing suggestions and warnings to encourage improvement based on the driver's driving behavior in the form of voice messages or display on the screen.

[0431] "Transmitting to a remote server via a network" refers to transmitting the collected data to a server in a remote location via a communication network such as the Internet.

[0432] The "means for evaluating driving behavior" is software or an algorithm that analyzes the driver's driving behavior based on the collected data and makes an evaluation based on that analysis.

[0433] A "driving score" is an index that quantifies the evaluation results of driving behavior and allows drivers to understand their driving performance at a glance.

[0434] "External organizations" are third-party organizations that are expected to notify driving scores, and mainly refer to insurance companies and traffic safety organizations.

[0435] "Means for applying discounts" refers to a system that reduces insurance premiums and other fees for drivers based on their assessed driving score.

[0436] A "driving evaluation method incorporating game elements" is a method in which driving behavior is evaluated in a game format and goals and rewards are set in order to increase the driver's motivation.

[0437] "Goals and rewards" refers to goals and incentives given to drivers when they achieve certain driving behaviors, including digital badges and ranking displays.

[0438] "Mobile application" means a software application used on a mobile device, such as a smartphone or tablet, to display driving scores and evaluation results.

[0439] "Means for providing visual information" refers to a method for displaying driving scores and evaluation results to the driver in the form of graphs, charts, text, etc.

[0440] MODE FOR CARRYING OUT THE INVENTION

[0441] The present invention is a system that provides real-time driving evaluation and feedback through collaboration between a vehicle device installed in a vehicle, a cloud server, and a user device (e.g., a smartphone app). This system promotes safe driving, evaluates driving behavior, calculates a driving score, and notifies the score to insurance companies to apply discounts on insurance premiums.

[0442] Device configuration and data collection

[0443] The in-vehicle device is equipped with a speed sensor, G sensor, GPS sensor, and in-vehicle camera. These sensors continuously collect the vehicle's speed, acceleration, position, and forward video data. For example, if the vehicle's speed exceeds 100 km / h, the speed information is measured in real time, and acceleration data from the G sensor and position data from the GPS sensor are simultaneously collected.

[0444] Data analysis and user notification

[0445] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." In addition, since footage from the in-car camera is also used for analysis, the device has the function of automatically saving footage of sudden braking and later uploading it to the cloud.

[0446] Data transmission and cloud analysis

[0447] The device periodically transmits the collected data (for example, every 30 minutes) to a cloud server. This transmission is performed using a secure communication protocol (for example, HTTPS). The server then integrates the received data and performs a detailed analysis. This analysis uses a big data analysis tool (for example, Apache Hadoop) to evaluate the number of sudden braking incidents, the frequency of speeding, and fluctuations in acceleration.

[0448] DriveScore calculation and feedback

[0449] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." This calculation uses machine learning algorithms (e.g., support vector machines and deep learning). The calculated DriveScore is registered in the user's account and can be viewed as visual information on a smartphone app. When the user opens the smartphone app, they can view a detailed report.

[0450] Gamification and rewards

[0451] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a user scores 80 or more out of 100, they will be awarded a digital badge as the "Top Driver of the Week" and notified of this via the smartphone app. Weekly rankings and other information will also be displayed, providing a system that allows users to enjoy their driving behavior in a game-like format.

[0452] Insurance premium discounts

[0453] With the user's consent, the server provides the calculated DriveScore to the insurance company. The insurance company applies a discount on the insurance premium based on this information. For example, if a user receives a DriveScore of 90 or more, the data is sent to the insurance company and the insurance premium is discounted. The user is notified of this discount via a smartphone app.

[0454] Specific examples

[0455] For example, if the following situation occurs:

[0456] The terminal detects when the vehicle's speed exceeds 100 km / h and measures speed information in real time.

[0457] When the device detects sudden braking, it automatically saves five seconds of footage from the onboard camera and uploads it to a cloud server.

[0458] The server uses data from the past 30 days to calculate a driving score based on factors such as the number of sudden braking attempts and speed limits, and allows users to check this score via a smartphone app.

[0459] The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0460] Prompt Sentence Examples

[0461] Example prompts to be input to the generative AI model:

[0462] Please explain how the system connects in-car devices, cloud servers, and user devices to provide real-time driving ratings and insurance discounts. Please provide examples.

[0463] These steps promote safe driving and increase motivation by providing drivers with tangible incentives.

[0464] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0465] Step 1: Data collection

[0466] The terminal collects data from the speed sensor, G sensor, GPS sensor, and on-board camera installed in the in-vehicle device. For example, the speed sensor measures the vehicle's speed once per second, the G sensor acquires acceleration data, and the GPS sensor acquires location data. The on-board camera captures video of the vehicle ahead. All of this data is collected in real time. The input data to the terminal are speed, acceleration, location, and video data, and an initial data set is generated by collecting these in real time.

[0467] Step 2: Real-time analysis

[0468] The device analyzes the data collected in step 1 in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." The input data is the collected sensor data, and the output data is a warning message to the driver.

[0469] Step 3: Send data

[0470] The device periodically (e.g., every 30 minutes) transmits data that has been collected and analyzed in real time to a cloud server. This transmission is performed using a secure communication protocol (e.g., HTTPS). The input data here is the analyzed sensor data, and the output data is the integrated data transmitted to the cloud server. Specifically, the device uploads speed, acceleration, position, and video data to the cloud server every 30 minutes.

[0471] Step 4: Cloud Solve

[0472] The server integrates the data received in step 3 and performs detailed analysis. The server processes large amounts of data using big data analysis tools (e.g., Apache Hadoop). It evaluates the number of sudden braking attempts, the frequency of speeding, and fluctuations in acceleration to analyze driving behavior. The input data is the integrated data sent to the cloud server, and the output data is the analyzed driving behavior data. Specifically, it analyzes the driver's driving patterns based on data from the past 30 days.

[0473] Step 5: Calculate your DriveScore

[0474] The server calculates a driving score, "DriveScore," based on the results of the cloud analysis. This calculation uses machine learning algorithms (such as support vector machines and deep learning). For example, the score is calculated taking into account the number of sudden braking attempts, speed compliance, and fluctuations in acceleration. The input data is the analyzed driving behavior data, and the output data is the calculated DriveScore. Specifically, the server tallies points for each score item based on the evaluation results.

[0475] Step 6: Provide feedback

[0476] The server registers the calculated DriveScore in the user's account and notifies them via a smartphone app. For example, a detailed driving evaluation report can be provided in PDF format. Furthermore, gamification elements can be set, such as goals and rewards. For example, a "Top Driver of the Week" badge can be awarded. The input data is the DriveScore and evaluation results, and the output data is feedback information to the driver.

[0477] Step 7: Apply for discount on premium

[0478] With the user's consent, the server provides the calculated DriveScore to the insurance company and applies a discount on the insurance premium. For example, the insurance company applies a discount of several percent to users with a DriveScore of 90 or more. The input data is the DriveScore, and the output data is a notification to the insurance company and information on the discount application. Specifically, the server sends the evaluation score to the insurance company and notifies the user of the discount application result.

[0479] Through these steps, the system can promote safe driving, specifically evaluate driver behavior, provide feedback, and even link insurance premium discounts.

[0480] (Application example 1)

[0481] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0482] Conventional in-vehicle device systems were limited in the advice and warnings they could give drivers, and the feedback they provided in real time was insufficient. Furthermore, the evaluation of driving behavior and the provision of incentives based on that evaluation were also limited, and no mechanism was in place to increase driver motivation. This meant that the inability to provide appropriate feedback and incentives to drivers resulted in the system not being effective enough in promoting safe driving. Furthermore, there were security concerns regarding the provision of data to insurance companies.

[0483] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0484] In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, means for feeding back the driving score to the driver and notifying an insurance company and applying a discount on insurance premiums, and means for sending a warning message regarding the driving score to a smartphone app, smart glasses, or an in-vehicle robot. This makes it possible to provide appropriate information to the driver in real time, increase the driver's motivation through a driving evaluation method incorporating game elements, and securely provide data to insurance companies.

[0485] An "in-vehicle device" is a device installed in a vehicle to collect data from speed sensors, G sensors, GPS sensors, cameras, etc.

[0486] "Sensor data" refers to data collected from various sensors, including vehicle speed, acceleration, location information, and video data acquired by on-board cameras.

[0487] "Real-time analysis" refers to instantly processing collected sensor data and making evaluations and decisions according to the situation.

[0488] "Advice and warnings" are information and warning messages about driving behavior provided to the driver.

[0489] A "cloud server" is a server used via the Internet, a remote server for storing and analyzing collected data.

[0490] "Means for evaluating driving behavior" refers to the process of analyzing and evaluating the driver's driving patterns and behavior based on collected sensor data.

[0491] The "driving score" is a numerical representation of the evaluation results of driving behavior, and serves as an indicator of the safety and smoothness of a driver's driving.

[0492] "Feedback" refers to conveying driving scores and warning messages to drivers.

[0493] "Notifying insurance company" means sending data such as driving scores to the insurance company in a secure manner.

[0494] "Premium discount" refers to a reduction in premiums applied to a policyholder based on their driving score.

[0495] A "smartphone app" is application software that runs on a smartphone and provides various notifications and feedback to the driver.

[0496] "Smart glasses" are glasses-type devices that have the function of overlaying information onto the driver's vision.

[0497] An "in-vehicle robot" is a robotic device that is installed inside a vehicle and provides warnings and advice to the driver through voice and screen displays.

[0498] "Providing appropriate information in real time" means providing drivers with advice and warnings that respond to their driving behavior immediately.

[0499] A "driving evaluation method incorporating game elements" is a method for evaluating a driver's driving behavior using playful elements such as rankings and badges, thereby increasing the driver's motivation.

[0500] "Providing data securely" refers to using technologies and protocols to ensure safety during the data transmission process.

[0501] This system promotes safe driving by providing real-time feedback to drivers through collaboration between in-vehicle devices, a cloud server, and user devices (such as smartphone apps and smart glasses). This system can also evaluate driving behavior, calculate a driving score, and apply insurance discounts.

[0502] 1. Data Collection

[0503] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. Data from these sensors is collected in real time. Specific hardware that can be used is an embedded Linux device such as a Raspberry Pi.

[0504] 2. Real-time analysis

[0505] The collected data is analyzed in real time using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. Based on the analysis results, the driver's driving behavior is evaluated and appropriate advice or warning messages are provided. For example, if the distance to the vehicle ahead is too close, a voice message will be issued saying, "The distance to the vehicle ahead is too close. Please reduce your speed."

[0506] 3. Data transmission and cloud analysis

[0507] The collected data is sent to a cloud server at regular intervals, where it is analyzed using cloud platforms such as AWS and Google Cloud, and services such as Amazon S3 and AWS Lambda.

[0508] 4. DriveScore calculation and feedback

[0509] The cloud server integrates and analyzes the collected data, evaluates driving behavior, and calculates a driving score, "DriveScore," based on various factors. This DriveScore is provided as feedback to the user's device (smartphone app or smart glasses).

[0510] 5. Gamification and rewards

[0511] The cloud server sets goals and rewards for drivers based on the calculated DriveScore. User progress is managed using databases such as Firebase and MongoDB. For example, drivers with a DriveScore of 80 or higher are awarded a digital badge called "Top Driver of the Week" and notified via a smartphone app.

[0512] 6. Insurance premium discounts

[0513] The server then provides the driving score to insurance companies using a secure API, which then applies discounts to insurance premiums based on the data. The data is transmitted using a security protocol such as OAuth 2.0.

[0514] Specific examples

[0515] For example, if a driver suddenly brakes, a notification will appear on the smart glasses saying, "Sudden braking has been detected. This will affect your driving score." The data will also be sent to a cloud server for further analysis.

[0516] Prompt Sentence Examples

[0517] "Generate an algorithm that will display a warning message about your driving score if it detects sudden braking. Also include example code to send the data to the cloud for further analysis."

[0518] With the above configuration, the present invention can provide appropriate feedback to drivers in real time, evaluate their driving behavior, and improve their motivation based on the evaluation.In addition, data can be securely provided to insurance companies, allowing them to apply discounts on insurance premiums.

[0519] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0520] Step 1:

[0521] The in-vehicle device (terminal) collects data from the speed sensor, G sensor, GPS sensor, and on-board camera. This includes the vehicle's speed, acceleration, location information, and forward video. The input is raw data from each sensor, and the output is integrated sensor data. This collected data is sent to the next processing stage in real time.

[0522] Step 2:

[0523] The device analyzes the collected data in real time. It processes and analyzes the data using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. For example, it analyzes fluctuations in speed and acceleration and generates a warning message if sudden braking or acceleration is detected. The input is the integrated sensor data, and the output is the analysis result and whether or not a warning is issued. This warning message is immediately notified to the driver.

[0524] Step 3:

[0525] The device periodically transmits the analyzed data to the cloud server. This transmission occurs, for example, every 30 minutes, in preparation for the integration and analysis of large amounts of data in the cloud. The input is the analyzed sensor data, and the output is the data transmitted to the cloud server.

[0526] Step 4:

[0527] The cloud server integrates the received data and performs detailed analysis. This analysis is performed using cloud platform services (e.g., AWS Lambda and Amazon S3) such as AWS and Google Cloud. The input is the data sent to the cloud, and the output is the evaluation results of the driver's driving behavior. This evaluation is multidimensional, and each element of driving behavior (such as speed limits and the number of sudden braking attempts) is evaluated.

[0528] Step 5:

[0529] The server calculates a driving score (DriveScore) based on the evaluation results. Each element of driving behavior is calculated using an algorithm and expressed as an overall score. The input is the evaluation result of driving behavior, and the output is the DriveScore. This score is registered in the user's account.

[0530] Step 6:

[0531] The server provides feedback to the user's device based on the calculated DriveScore. The score and warning messages are displayed on the smartphone app or smart glasses. The input is the DriveScore, and the output is the feedback information displayed on the smartphone app or smart glasses.

[0532] Step 7:

[0533] The server sets goals and rewards for the driver based on the acquired DriveScore. The user's progress is managed using a database such as Firebase or MongoDB. The input is the DriveScore, and the output is the goal and reward information sent to the user's device.

[0534] Step 8:

[0535] The server provides the DriveScore of consenting users to the insurance company using a secure API (e.g., OAuth 2.0), which then applies premium discounts based on the DriveScore. The input is the DriveScore, and the output is the data sent to the insurance company and the applied premium discount.

[0536] As described above, a real-time safe driving feedback system is realized by processing and analyzing the data obtained at each step and proceeding with the process sequentially.

[0537] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0538] MODE FOR CARRYING OUT THE INVENTION

[0539] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The emotion engine can recognize the driver's emotional state in real time and provide appropriate advice and warnings based on that information.

[0540] 1. Data Collection

[0541] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously collects data from these sensors. The emotion engine also analyzes the driver's facial expressions and tone of voice to obtain emotional data.

[0542] Example: The device collects real-time data on the vehicle's speed, acceleration, and location, as well as changes in the driver's facial expressions and voice.

[0543] 2. Real-time analysis and user notification

[0544] The device analyzes sensor data and emotion data in real time to evaluate the driver's driving situation and emotional state. The emotion engine detects the driver's emotions such as stress, impatience, and fatigue, and provides advice and warnings at the appropriate time.

[0545] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0546] 3. Data transmission and cloud analysis

[0547] The device periodically transfers the collected and pre-processed data (speed, acceleration, position, and emotion data) in batches to a cloud server.

[0548] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0549] 4. DriveScore calculation and feedback

[0550] The server evaluates the driving behavior and emotional data based on the integrated data to calculate a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation. The calculated DriveScore is registered in the user's account and can be viewed on the user's device.

[0551] Example: The server calculates the DriveScore based on the number of sudden braking incidents, fluctuations in acceleration, and the driver's emotional data, and provides the results to the user so that they can be checked on a smartphone app.

[0552] 5. Gamification and rewards

[0553] The server sets goals and rewards for users based on the calculated DriveScore. For example, users who reach a certain score will receive a digital badge or be ranked high in the weekly rankings.

[0554] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0555] 6. Insurance premium discounts

[0556] With the user's consent, the server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[0557] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0558] This system not only helps novice and inexperienced drivers develop safe driving habits, but also significantly reduces driving risks by taking their emotional state into account. Furthermore, by providing specific incentives and feedback to users, it can highly motivate them to drive safely.

[0559] The processing flow will be explained below.

[0560] Step 1:

[0561] As soon as the vehicle's engine starts, the device starts collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information on the current speed, acceleration, position, distance to obstacles ahead, etc. The emotion engine also analyzes the driver's facial expressions and tone of voice to collect emotion data.

[0562] Step 2:

[0563] The device temporarily stores the collected sensor data and emotion data, and performs preprocessing such as noise reduction and data filtering to remove outliers and smooth the location data. The device also filters out ambiguous emotion determinations from the emotion data.

[0564] Step 3:

[0565] The device analyzes the pre-processed data in real time to assess the driving situation and the driver's emotional state, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short, and assessing the driver's emotional state if it indicates impatience or stress.

[0566] Step 4:

[0567] The device provides real-time advice and warnings to the driver as needed through voice messages and display, and the emotion engine changes the content and tone of the advice based on the driver's emotional state at the time.

[0568] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0569] Step 5:

[0570] The device transfers the collected and pre-processed data (speed, acceleration, location, emotion data) to a cloud server in batches at regular intervals (e.g., every 30 minutes), which includes all data points during driving.

[0571] Step 6:

[0572] The server then integrates the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking and accelerations, as well as emotional fluctuations.

[0573] Step 7:

[0574] The server evaluates the driver's driving safety, acceleration and deceleration smoothness, speed adherence rate, and emotional state based on the integrated data, and calculates a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation.

[0575] Step 8:

[0576] The server registers the calculated DriveScore in the driver's account, allowing the driver to check the results on a smartphone app.

[0577] Example: The server displays the information "This week's DriveScore is 88" on the user's smartphone app.

[0578] Step 9:

[0579] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore will be rewarded with a digital badge or a high ranking in the weekly rankings.

[0580] Example: The server notifies drivers with a DriveScore of 80 or above with a "Top Driver of the Week" badge on their smartphone app.

[0581] Step 10:

[0582] With the user's consent, the server provides the DriveScore to the insurance company, which applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result.

[0583] Example: The server provides data on users whose DriveScore is 90 or higher to an insurance company, and the insurance company applies discounts to insurance premiums based on this data. For example, it sends a notification saying, "Your insurance premium will be discounted by 10%."

[0584] In this way, a system can be built that uses collected data to perform detailed analysis of driving behavior and emotional states, providing drivers with specific feedback and incentives to promote safe driving and even offer discounts on insurance premiums.

[0585] Example 2

[0586] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0587] Conventional driving evaluation systems for in-vehicle devices rely primarily on driving behavior data and do not take into account the driver's emotional state, making it difficult to accurately evaluate driving conditions. Furthermore, they lack the ability to provide real-time feedback and appropriate advice to drivers, making it difficult to promote safe driving. Furthermore, incentive systems and insurance discounts that utilize driving evaluation results have limitations because they do not include emotional data.

[0588] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0589] In this invention, the server includes means for collecting sensor data and emotional data acquired by an in-vehicle device, means for analyzing the sensor data and emotional data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating the driving behavior and emotional state based on the data, means for calculating a driving score that takes into account the driving score and emotional state based on the evaluation, and means for feeding back the driving score to the driver and notifying the insurance company of the score and applying a discount on the insurance premium. This enables more accurate driving evaluation that takes into account not only the driver's driving behavior but also their emotional state, leading to the promotion of safe driving and the provision of appropriate incentives.

[0590] An "on-board device" is a device that is installed in a vehicle and includes multiple sensors such as a speed sensor, a G sensor, a GPS sensor, and an on-board camera.

[0591] "Sensor data" refers to driving-related data such as speed, acceleration, and location, and is information collected from in-vehicle devices.

[0592] "Emotion data" refers to data relating to the driver's emotional state, obtained by analyzing the driver's facial expressions and tone of voice.

[0593] An "emotion engine" is a software or hardware device that analyzes the driver's facial expressions and tone of voice and generates emotion data.

[0594] "Cloud server" refers to a remote server that stores and analyzes data over the Internet, and is the place where collected data is processed and stored.

[0595] The "driving score" is a numerical value that indicates the evaluation result of the driver's driving behavior, calculated by analyzing sensor data and emotional data.

[0596] "Advice and Reminders" refers to suggestions and warning messages provided based on the driver's driving situation and emotional state.

[0597] "Feedback" refers to notifying the driver of the calculated driving score and warnings, and is primarily done through the user's device.

[0598] "Gamification" is a method of incorporating game elements to evaluate a driver's driving behavior and increase motivation by setting goals and rewards.

[0599] "User device" means a device on which a driver checks their driving score and feedback, including, for example, a smartphone or tablet.

[0600] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The specific configuration and operation of this system will be described below.

[0601] In-vehicle devices

[0602] The in-vehicle device is equipped with a speed sensor, a G-sensor, a GPS sensor, and an on-board camera. The device continuously collects data from these sensors. It also uses an emotion engine to analyze the driver's facial expressions and tone of voice to collect emotional data. For example, the vehicle's speed and acceleration are acquired every second while driving, and the driver's facial expressions captured by the on-board camera are analyzed using image recognition software.

[0603] Cloud Server

[0604] Data collected by the device is periodically sent in batches to a cloud server. The cloud server then integrates the received data and prepares it for analysis. The server then analyzes speed, acceleration, position, and emotional data to evaluate the driver's driving behavior and emotional state. This information is then used to calculate a driving score (DriveScore). By including emotional data in addition to sensor data, driving evaluation can be performed with greater accuracy than before.

[0605] Feedback and User Devices

[0606] The server registers the calculated driving score in the user's account and notifies the result via the user's device (e.g., smartphone). For example, a message such as "Your DriveScore is 85" is displayed on the user's device. In addition, real-time warnings and advice are also provided through the user's device. For example, while driving, the user can receive a voice message saying, "Take your time and slow down."

[0607] Gamification and rewards

[0608] The server sets goals and rewards for users based on their driving scores. When a certain score is reached, rewards such as digital badges and weekly rankings are provided. For example, a "Top Driver of the Week" badge is awarded, and a notification appears on the user's smartphone saying, "Congratulations! You're the Top Driver of the Week."

[0609] Insurance premium discounts

[0610] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount on insurance premiums based on the driving score. For example, if a user's DriveScore is 90 or higher, the server provides the data to the insurance company, and the user receives a 10% discount on insurance premiums.

[0611] This system can evaluate the driver's driving behavior and emotional state in detail, promoting safe driving and improving motivation. As a specific example, the following prompt sentences can be input into the generative AI model:

[0612] Example prompt sentence:

[0613] 1. "Describe a system that analyzes the driver's emotional state in real time based on data collected by an in-car device and provides appropriate advice."

[0614] 2. "Please explain in detail how the device will notify the user with a warning message if the user becomes impatient while driving."

[0615] 3. "Please explain what DriveScore is, how it is calculated, and what feedback it provides to users."

[0616] 4. "Can you give us a concrete example of how insurance discounts based on driving data work?"

[0617] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0618] Step 1:

[0619] The terminal collects data in real time from each sensor of the in-vehicle device (speed sensor, G sensor, GPS sensor, and in-vehicle camera). This collected data includes speed, acceleration, location, and the driver's facial expression and tone of voice. An emotion engine is used to analyze this data and generate driver emotion data. The input sensor data and facial expression data become real-time driving data and emotion data, and this data is stored in the terminal.

[0620] Step 2:

[0621] The device analyzes the collected sensor data and emotional data in real time. Specifically, it evaluates the driver's driving behavior and emotional state. This allows it to automatically generate appropriate advice or warnings if the driver appears to be impatient, tired, or stressed. Using the real-time data and emotional data as input, the device generates driving situation evaluation data and emotional evaluation data through analysis.

[0622] Step 3:

[0623] The device then provides the driver with advice or warning messages based on the analysis results. For example, if the driver is in a hurry and speeding up, the device will notify the user with a voice message saying, "Please stay calm and slow down." Based on the evaluation data input, a specific message is generated and notified to the user as voice or display output.

[0624] Step 4:

[0625] The device periodically transfers the collected data (speed, acceleration, position, and emotion data) to the cloud server in batches. Specifically, all data is packaged at regular intervals and sent to the cloud server via a communication line. The accumulated data as input is converted into a batch package and sent to the cloud server.

[0626] Step 5:

[0627] The server consolidates the data entered on the cloud and prepares it for analysis, specifically storing it in a literal database and performing any necessary preprocessing (e.g., data cleansing and format conversion). It receives batch data as input, stores and preprocesses it, and outputs it in an analyzable format.

[0628] Step 6:

[0629] The server evaluates driving behavior and emotional state based on the integrated data and calculates a driving score (DriveScore). Specifically, it analyzes the number of sudden braking attempts, fluctuations in acceleration, and the driver's emotional data to generate a quantified driving score. It evaluates the integrated data as input and outputs a driving score.

[0630] Step 7:

[0631] The server registers the calculated driving score to the user's account and provides it as visual information via the user's device. Specifically, it sends a notification to the user via a smartphone app saying, "Your DriveScore is 85." Using the driving score data as input, it generates a visual feedback message and outputs it to the user's device.

[0632] Step 8:

[0633] The server sets goals and rewards based on the driving score. For example, it can award a "Top Driver of the Week" badge to drivers who reach a certain score and notify them via a smartphone app. The server uses the driving score as input to set rewards and award badges, and notifies the user of the reward information as output.

[0634] Step 9:

[0635] After obtaining the user's consent, the server provides the DriveScore to the insurance company, which then applies a discount to the insurance premium. Specifically, if the user's DriveScore is 90 or higher, the server sends the user's data via API, and the insurance company notifies and applies the premium discount. The server sends the score data as input and receives the result of the premium discount application as output.

[0636] (Application example 2)

[0637] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0638] Conventional automated driving systems perform driving operations without taking into account the driver's emotional state, which means they are unable to effectively reduce driver stress and fatigue. Furthermore, the lack of technology to appropriately adjust automated driving parameters based on the driver's emotional state has prevented sufficient improvements in safety.

[0639] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for analyzing the driver's emotional state and adjusting automated driving parameters based on the data, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, and means for providing feedback about the driving score to the driver, notifying the insurance company, and applying a discount on insurance premiums. This makes it possible to recognize the driver's emotional state in real time and adjust appropriate automated driving parameters based on the driving score. Furthermore, by providing feedback and rewards to the driver, it is possible to encourage improvement in driving behavior and improve safety.

[0640] An "in-vehicle device" is a device that is installed in a vehicle and includes various sensors and cameras for acquiring driving data and environmental data.

[0641] "Sensor data" is a general term for data acquired by in-vehicle devices, such as speed, acceleration, GPS location information, the driver's facial expressions, and voice tone.

[0642] "Advice and Caution" refers to appropriate instructions or warnings provided to the driver based on sensor data and emotional state.

[0643] A "cloud server" is a remote server that stores collected data and performs integrated analysis and evaluation.

[0644] The "means for evaluating driving behavior" is a mechanism that analyzes the driver's driving behavior based on collected sensor data and emotional state, and generates an evaluation result.

[0645] The "driving score" is a numerical index that comprehensively evaluates driving behavior and emotional state.

[0646] "Feedback" is a means of providing information to drivers by notifying them of their driving score, advice, and warnings, encouraging them to improve their driving behavior.

[0647] "Insurance premium discounts" are premium reduction benefits applied by insurance companies based on driving scores.

[0648] "Emotional state" is information that indicates the psychological state of the driver, such as stress, fatigue, or impatience.

[0649] "Autonomous driving parameters" are control elements such as speed, following distance, and braking sensitivity that are set when the autonomous driving system performs driving operations.

[0650] "Game elements" are fun, competitive elements that set goals and rewards to encourage improved driving behavior.

[0651] "Smartphone app" means a software application for a mobile device used to provide visual driving scores and feedback to the driver.

[0652] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. Using the emotion engine, it is possible to recognize the driver's emotional state in real time and adjust autonomous driving parameters based on that. This system is configured by combining the following hardware and software.

[0653] Hardware used

[0654] 1. In-vehicle devices:

[0655] This includes an in-car camera, microphone, speed sensor, G-sensor, and GPS sensor.

[0656] Data is obtained from these sensors in real time.

[0657] 2. Cloud Server:

[0658] A remote server for storing and analyzing data.

[0659] Examples: AWS (Amazon Web Services), Google Cloud, etc.

[0660] 3. User Device:

[0661] Mobile devices such as smartphones and tablets.

[0662] Used to review data and provide feedback.

[0663] Software used

[0664] 1. Emotion Engine:

[0665] An AI model for analyzing facial expressions and tone of voice.

[0666] Examples: OpenAI, Google TensorFlow.

[0667] 2. Data analysis platform:

[0668] A tool for integrating and analyzing data.

[0669] For example: Apache Spark, Hadoop.

[0670] 3. Smartphone App:

[0671] An application that displays and notifies user data.

[0672] System Operation

[0673] Data collection

[0674] The in-vehicle device collects real-time sensor data such as speed, acceleration, GPS location, the driver's facial expressions and voice tone, allowing for accurate understanding of the driving situation and the driver's emotional state.

[0675] Real-time analysis and parameter adjustment

[0676] The terminal (in-vehicle device) analyzes the acquired data in real time. The emotion engine analyzes the driver's facial expressions and tone of voice to identify the driver's emotional state, such as stress, fatigue, or impatience. Based on this data, the autonomous driving system's parameters (speed, following distance, braking sensitivity, etc.) are adjusted appropriately.

[0677] Data transmission and cloud analysis

[0678] The device periodically transmits the acquired data to a cloud server, which then comprehensively analyzes the data and evaluates the driver's driving behavior and emotional state.

[0679] DriveScore calculation and feedback

[0680] The cloud server calculates a driving score (DriveScore) based on the integrated data. This score is the result of a comprehensive evaluation of driving behavior and emotional state. The calculated DriveScore is provided as feedback to the driver via the user's device.

[0681] Gamification and rewards

[0682] The cloud server sets goals and rewards for drivers based on their DriveScore. For example, achieving a certain score will earn them a digital badge or a spot at the top of the weekly rankings.

[0683] Insurance premium discounts

[0684] With the user's consent, the cloud server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[0685] Specific examples

[0686] For example, if the in-car device detects that the driver is in an "anger" state from their facial expression, the system will automatically slow down the vehicle and give a voice notification to the driver saying "Please relax." In addition, the collected data is sent to a cloud server every 30 minutes, and a driving score (DriveScore) is calculated through integrated analysis. Users can check this score on their smartphone app.

[0687] Prompt Sentence Examples

[0688] "Perform real-time emotion analysis using driver facial expression data and vehicle driving data, and generate Python code to set appropriate autonomous driving parameters."

[0689] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0690] Step 1: Data collection

[0691] The terminal (in-vehicle device) uses an in-vehicle camera, microphone, speed sensor, G sensor, and GPS sensor to capture the driver's facial expression, tone of voice, vehicle speed, acceleration, and location information in real time. The input data are facial expression data, voice data, speed data, G sensor data, and GPS data, which are sent to the analysis unit after initial processing. The output is initially processed emotion data and driving data.

[0692] Step 2: Sentiment Analysis

[0693] The device uses an emotion engine to analyze the collected facial expression data and voice data in real time. This analysis identifies the driver's emotional state (e.g., stress, fatigue, anger, etc.). The input is initially processed facial expression data and voice data, which are then fed into the emotion engine, which outputs the driver's emotional state.

[0694] Step 3: Real-time analysis and autonomous driving parameter adjustment

[0695] The device combines emotional state and driving data for real-time analysis and adjusts autonomous driving parameters (speed, following distance, braking sensitivity, etc.). The input is driving data and analyzed emotional data, and based on this data, it calculates and outputs autonomous driving control parameters. Specifically, if the driver's emotion is "anger," the device will slow down the vehicle and give a voice notification.

[0696] Step 4: Send data to the cloud server

[0697] The device periodically (e.g., every 30 minutes) sends the collected and preprocessed data to the cloud server in batch format. The input is the preprocessed driving data and emotion data, and the cloud server receives these data. The output is the integrated data sent to the cloud server.

[0698] Step 5: Integrated analysis and DriveScore calculation

[0699] The server (cloud server) comprehensively analyzes the transmitted data, evaluates driving behavior and emotional state, and calculates a DriveScore. The input is collected historical driving data and emotional data, which is used to apply statistical analysis and machine learning models. The output is the driver's DriveScore. A generative AI model is used to comprehensively evaluate emotions and driving behavior.

[0700] Step 6: Provide feedback

[0701] The server provides the calculated DriveScore as feedback to the user's smartphone app. The input is the calculated DriveScore, which is reported to the user device. The output is feedback information displayed on the user device, such as a "Top Driver of the Week" badge.

[0702] Step 7: Gamify and reward

[0703] The server sets goals and rewards based on the DriveScore and provides them to the user. The input is the DriveScore and user profile data, and based on this, the server sets goals and determines rewards. The output is a digital badge and weekly ranking display provided to the user.

[0704] Step 8: Provide data to insurance companies and apply discounts

[0705] With the user's consent, the server provides the insurance company with the user's DriveScore, and the insurance company applies a discount on the premium based on the score. The input is the user's DriveScore and consent information, which are sent to the insurance company. The output is a notification from the insurance company that the discount has been applied.

[0706] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0708] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0709] [Third embodiment]

[0710] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0711] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0713] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0717] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0718] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0719] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0720] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0721] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0722] MODE FOR CARRYING OUT THE INVENTION

[0723] This system is realized by linking an in-vehicle device, a cloud server, and a user device (smartphone app) installed in the vehicle. This system provides appropriate advice and warnings to the driver in real time, evaluates driving behavior to calculate a driving score, and notifies the score to the insurance company to apply discounts on insurance premiums.

[0724] 1. Data Collection

[0725] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously acquires data from these sensors.

[0726] Example: When the vehicle exceeds a certain speed, the device measures the speed information in real time and also acquires acceleration data from the G sensor and location data from the GPS sensor.

[0727] 2. Real-time analysis and user notification

[0728] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if the distance to the vehicle ahead is too close or if the speed limit is exceeded, the device will warn the driver with a voice message or a display.

[0729] Example: When the distance to the vehicle ahead falls below a certain level, the device notifies the user with a voice message saying, "The distance to the vehicle ahead is too short. Please reduce your speed."

[0730] 3. Data transmission and cloud analysis

[0731] The device periodically transmits the collected data to a cloud server, where it is integrated and analyzed in detail.

[0732] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0733] 4. DriveScore calculation and feedback

[0734] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." The calculated DriveScore is registered in the user's account and can be viewed from the user's device.

[0735] Example: The server calculates the DriveScore based on data such as the number of sudden braking attempts, fluctuations in acceleration, and speed compliance, and provides the results so that the user can check them on a smartphone app.

[0736] 5. Gamification and rewards

[0737] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a certain score is reached, the driver will be given a digital badge and displayed at the top of the weekly rankings.

[0738] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0739] 6. Insurance premium discounts

[0740] With the user's consent, the server provides the calculated DriveScore to the insurance company, which then applies a discount to the insurance premium based on the score.

[0741] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0742] This invention makes it easier for novice drivers and inexperienced drivers to develop safe driving habits, which is expected to reduce the risk of traffic accidents. In addition, by evaluating driving behavior, it is possible to provide specific incentives and increase user motivation.

[0743] The processing flow will be explained below.

[0744] Step 1:

[0745] As soon as the vehicle's engine starts, the device begins collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information such as current speed, acceleration, position, and distance to obstacles ahead.

[0746] Step 2:

[0747] The device stores the collected sensor data in temporary memory and performs pre-processing such as noise removal and data filtering, which removes outliers and smooths the location data.

[0748] Step 3:

[0749] The device analyzes the pre-processed data in real time to assess the driving situation, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short.

[0750] Step 4:

[0751] The device will provide real-time advice and reminders to the driver as needed through voice messages and display alerts, such as a notification that says, "You are too close to the vehicle ahead. Please reduce your speed."

[0752] Step 5:

[0753] The device transfers the collected and pre-processed data to the cloud server in batches at regular intervals (e.g., every 30 minutes). This data includes information on speed, position, acceleration, and distance to obstacles ahead.

[0754] Step 6:

[0755] The server then combines the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking attempts, sudden acceleration attempts, etc.

[0756] Step 7:

[0757] Based on the integrated data, the server calculates a driving score (DriveScore) using parameters such as safe driving, smooth acceleration and deceleration, and speed compliance.

[0758] Step 8:

[0759] The server registers the calculated DriveScore in the driver's account and allows the driver to check the result on their smartphone app. For example, the server provides information such as "This week's DriveScore is 88 points."

[0760] Step 9:

[0761] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore are given rewards such as a digital badge or a place at the top of the weekly rankings.

[0762] Step 10:

[0763] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result. For example, the insurance company may notify the user that "Your DriveScore is 90 or higher, so you will receive a 10% discount on your insurance premium."

[0764] This will establish a complete process from collection to analysis, feedback, incentive provision, and insurance premium discount application.

[0765] Example 1

[0766] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0767] In modern vehicle driving, systems that promote safe driving and evaluate driver behavior still face challenges. In particular, there is no well-established system that evaluates driving behavior in real time, provides appropriate feedback, and links that evaluation to insurance premium discounts. Furthermore, there is still a lack of gamification elements to increase driver motivation and methods to easily visualize driving scores.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0769] In this invention, the server includes: means for collecting multiple types of sensor data acquired by devices installed in the vehicle; means for analyzing the multiple sensor data in real time and providing advice and warnings to the driver; means for transmitting the collected data to a remote server via a network and evaluating driving behavior based on the data; means for calculating a driving score based on the evaluation; means for providing the driving score as feedback to the driver and notifying an external institution and applying a fee discount; means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements and providing them based on the driving score; and means for registering the driving score in the driver's account and providing it to the driver as visual information via a mobile device application. This promotes safe driving, appropriately evaluates the driver's behavior, increases motivation, and enables the application of insurance premium discounts.

[0770] A "vehicle-mounted device" is a hardware device that is placed inside a vehicle and uses multiple sensors and cameras to collect driving data.

[0771] "Sensor data" refers to information obtained from speed sensors, G sensors, GPS sensors, and on-board cameras, including vehicle speed, acceleration, position, and images.

[0772] "Real-time analysis" means that data is processed as soon as it is collected, providing immediate feedback to the driver.

[0773] "Advice and warnings" refers to providing suggestions and warnings to encourage improvement based on the driver's driving behavior in the form of voice messages or display on the screen.

[0774] "Transmitting to a remote server via a network" refers to transmitting the collected data to a server in a remote location via a communication network such as the Internet.

[0775] The "means for evaluating driving behavior" is software or an algorithm that analyzes the driver's driving behavior based on the collected data and makes an evaluation based on that analysis.

[0776] A "driving score" is an index that quantifies the evaluation results of driving behavior and allows drivers to understand their driving performance at a glance.

[0777] "External organizations" are third-party organizations that are expected to notify driving scores, and mainly refer to insurance companies and traffic safety organizations.

[0778] "Means for applying discounts" refers to a system that reduces insurance premiums and other fees for drivers based on their assessed driving score.

[0779] A "driving evaluation method incorporating game elements" is a method in which driving behavior is evaluated in a game format and goals and rewards are set in order to increase the driver's motivation.

[0780] "Goals and rewards" refers to goals and incentives given to drivers when they achieve certain driving behaviors, including digital badges and ranking displays.

[0781] "Mobile application" means a software application used on a mobile device, such as a smartphone or tablet, to display driving scores and evaluation results.

[0782] "Means for providing visual information" refers to a method for displaying driving scores and evaluation results to the driver in the form of graphs, charts, text, etc.

[0783] MODE FOR CARRYING OUT THE INVENTION

[0784] The present invention is a system that provides real-time driving evaluation and feedback through collaboration between a vehicle device installed in a vehicle, a cloud server, and a user device (e.g., a smartphone app). This system promotes safe driving, evaluates driving behavior, calculates a driving score, and notifies the score to insurance companies to apply discounts on insurance premiums.

[0785] Device configuration and data collection

[0786] The in-vehicle device is equipped with a speed sensor, G sensor, GPS sensor, and in-vehicle camera. These sensors continuously collect the vehicle's speed, acceleration, position, and forward video data. For example, if the vehicle's speed exceeds 100 km / h, the speed information is measured in real time, and acceleration data from the G sensor and position data from the GPS sensor are simultaneously collected.

[0787] Data analysis and user notification

[0788] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." In addition, since footage from the in-car camera is also used for analysis, the device has the function of automatically saving footage of sudden braking and later uploading it to the cloud.

[0789] Data transmission and cloud analysis

[0790] The device periodically transmits the collected data (for example, every 30 minutes) to a cloud server. This transmission is performed using a secure communication protocol (for example, HTTPS). The server then integrates the received data and performs a detailed analysis. This analysis uses a big data analysis tool (for example, Apache Hadoop) to evaluate the number of sudden braking incidents, the frequency of speeding, and fluctuations in acceleration.

[0791] DriveScore calculation and feedback

[0792] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." This calculation uses machine learning algorithms (e.g., support vector machines and deep learning). The calculated DriveScore is registered in the user's account and can be viewed as visual information on a smartphone app. When the user opens the smartphone app, they can view a detailed report.

[0793] Gamification and rewards

[0794] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a user scores 80 or more out of 100, they will be awarded a digital badge as the "Top Driver of the Week" and notified of this via the smartphone app. Weekly rankings and other information will also be displayed, providing a system that allows users to enjoy their driving behavior in a game-like format.

[0795] Insurance premium discounts

[0796] With the user's consent, the server provides the calculated DriveScore to the insurance company. The insurance company applies a discount on the insurance premium based on this information. For example, if a user receives a DriveScore of 90 or more, the data is sent to the insurance company and the insurance premium is discounted. The user is notified of this discount via a smartphone app.

[0797] Specific examples

[0798] For example, if the following situation occurs:

[0799] The terminal detects when the vehicle's speed exceeds 100 km / h and measures speed information in real time.

[0800] When the device detects sudden braking, it automatically saves five seconds of footage from the onboard camera and uploads it to a cloud server.

[0801] The server uses data from the past 30 days to calculate a driving score based on factors such as the number of sudden braking attempts and speed limits, and allows users to check this score via a smartphone app.

[0802] The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0803] Prompt Sentence Examples

[0804] Example prompts to be input to the generative AI model:

[0805] Please explain how the system connects in-car devices, cloud servers, and user devices to provide real-time driving ratings and insurance discounts. Please provide examples.

[0806] These steps promote safe driving and increase motivation by providing drivers with tangible incentives.

[0807] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0808] Step 1: Data collection

[0809] The terminal collects data from the speed sensor, G sensor, GPS sensor, and on-board camera installed in the in-vehicle device. For example, the speed sensor measures the vehicle's speed once per second, the G sensor acquires acceleration data, and the GPS sensor acquires location data. The on-board camera captures video of the vehicle ahead. All of this data is collected in real time. The input data to the terminal are speed, acceleration, location, and video data, and an initial data set is generated by collecting these in real time.

[0810] Step 2: Real-time analysis

[0811] The device analyzes the data collected in step 1 in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." The input data is the collected sensor data, and the output data is a warning message to the driver.

[0812] Step 3: Send data

[0813] The device periodically (e.g., every 30 minutes) transmits data that has been collected and analyzed in real time to a cloud server. This transmission is performed using a secure communication protocol (e.g., HTTPS). The input data here is the analyzed sensor data, and the output data is the integrated data transmitted to the cloud server. Specifically, the device uploads speed, acceleration, position, and video data to the cloud server every 30 minutes.

[0814] Step 4: Cloud Solve

[0815] The server integrates the data received in step 3 and performs detailed analysis. The server processes large amounts of data using big data analysis tools (e.g., Apache Hadoop). It evaluates the number of sudden braking attempts, the frequency of speeding, and fluctuations in acceleration to analyze driving behavior. The input data is the integrated data sent to the cloud server, and the output data is the analyzed driving behavior data. Specifically, it analyzes the driver's driving patterns based on data from the past 30 days.

[0816] Step 5: Calculate your DriveScore

[0817] The server calculates a driving score, "DriveScore," based on the results of the cloud analysis. This calculation uses machine learning algorithms (such as support vector machines and deep learning). For example, the score is calculated taking into account the number of sudden braking attempts, speed compliance, and fluctuations in acceleration. The input data is the analyzed driving behavior data, and the output data is the calculated DriveScore. Specifically, the server tallies points for each score item based on the evaluation results.

[0818] Step 6: Provide feedback

[0819] The server registers the calculated DriveScore in the user's account and notifies them via a smartphone app. For example, a detailed driving evaluation report can be provided in PDF format. Furthermore, gamification elements can be set, such as goals and rewards. For example, a "Top Driver of the Week" badge can be awarded. The input data is the DriveScore and evaluation results, and the output data is feedback information to the driver.

[0820] Step 7: Apply for discount on premium

[0821] With the user's consent, the server provides the calculated DriveScore to the insurance company and applies a discount on the insurance premium. For example, the insurance company applies a discount of several percent to users with a DriveScore of 90 or more. The input data is the DriveScore, and the output data is a notification to the insurance company and information on the discount application. Specifically, the server sends the evaluation score to the insurance company and notifies the user of the discount application result.

[0822] Through these steps, the system can promote safe driving, specifically evaluate driver behavior, provide feedback, and even link insurance premium discounts.

[0823] (Application example 1)

[0824] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0825] Conventional in-vehicle device systems were limited in the advice and warnings they could give drivers, and the feedback they provided in real time was insufficient. Furthermore, the evaluation of driving behavior and the provision of incentives based on that evaluation were also limited, and no mechanism was in place to increase driver motivation. This meant that the inability to provide appropriate feedback and incentives to drivers resulted in the system not being effective enough in promoting safe driving. Furthermore, there were security concerns regarding the provision of data to insurance companies.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0827] In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, means for feeding back the driving score to the driver and notifying an insurance company and applying a discount on insurance premiums, and means for sending a warning message regarding the driving score to a smartphone app, smart glasses, or an in-vehicle robot. This makes it possible to provide appropriate information to the driver in real time, increase the driver's motivation through a driving evaluation method incorporating game elements, and securely provide data to insurance companies.

[0828] An "in-vehicle device" is a device installed in a vehicle to collect data from speed sensors, G sensors, GPS sensors, cameras, etc.

[0829] "Sensor data" refers to data collected from various sensors, including vehicle speed, acceleration, location information, and video data acquired by on-board cameras.

[0830] "Real-time analysis" refers to instantly processing collected sensor data and making evaluations and decisions according to the situation.

[0831] "Advice and warnings" are information and warning messages about driving behavior provided to the driver.

[0832] A "cloud server" is a server used via the Internet, a remote server for storing and analyzing collected data.

[0833] "Means for evaluating driving behavior" refers to the process of analyzing and evaluating the driver's driving patterns and behavior based on collected sensor data.

[0834] The "driving score" is a numerical representation of the evaluation results of driving behavior, and serves as an indicator of the safety and smoothness of a driver's driving.

[0835] "Feedback" refers to conveying driving scores and warning messages to drivers.

[0836] "Notifying insurance company" means sending data such as driving scores to the insurance company in a secure manner.

[0837] "Premium discount" refers to a reduction in premiums applied to a policyholder based on their driving score.

[0838] A "smartphone app" is application software that runs on a smartphone and provides various notifications and feedback to the driver.

[0839] "Smart glasses" are glasses-type devices that have the function of overlaying information onto the driver's vision.

[0840] An "in-vehicle robot" is a robotic device that is installed inside a vehicle and provides warnings and advice to the driver through voice and screen displays.

[0841] "Providing appropriate information in real time" means providing drivers with advice and warnings that respond to their driving behavior immediately.

[0842] A "driving evaluation method incorporating game elements" is a method for evaluating a driver's driving behavior using playful elements such as rankings and badges, thereby increasing the driver's motivation.

[0843] "Providing data securely" refers to using technologies and protocols to ensure safety during the data transmission process.

[0844] This system promotes safe driving by providing real-time feedback to drivers through collaboration between in-vehicle devices, a cloud server, and user devices (such as smartphone apps and smart glasses). This system can also evaluate driving behavior, calculate a driving score, and apply insurance discounts.

[0845] 1. Data Collection

[0846] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. Data from these sensors is collected in real time. Specific hardware that can be used is an embedded Linux device such as a Raspberry Pi.

[0847] 2. Real-time analysis

[0848] The collected data is analyzed in real time using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. Based on the analysis results, the driver's driving behavior is evaluated and appropriate advice or warning messages are provided. For example, if the distance to the vehicle ahead is too close, a voice message will be issued saying, "The distance to the vehicle ahead is too close. Please reduce your speed."

[0849] 3. Data transmission and cloud analysis

[0850] The collected data is sent to a cloud server at regular intervals, where it is analyzed using cloud platforms such as AWS and Google Cloud, and services such as Amazon S3 and AWS Lambda.

[0851] 4. DriveScore calculation and feedback

[0852] The cloud server integrates and analyzes the collected data, evaluates driving behavior, and calculates a driving score, "DriveScore," based on various factors. This DriveScore is provided as feedback to the user's device (smartphone app or smart glasses).

[0853] 5. Gamification and rewards

[0854] The cloud server sets goals and rewards for drivers based on the calculated DriveScore. User progress is managed using databases such as Firebase and MongoDB. For example, drivers with a DriveScore of 80 or higher are awarded a digital badge called "Top Driver of the Week" and notified via a smartphone app.

[0855] 6. Insurance premium discounts

[0856] The server then provides the driving score to insurance companies using a secure API, which then applies discounts to insurance premiums based on the data. The data is transmitted using a security protocol such as OAuth 2.0.

[0857] Specific examples

[0858] For example, if a driver suddenly brakes, a notification will appear on the smart glasses saying, "Sudden braking has been detected. This will affect your driving score." The data will also be sent to a cloud server for further analysis.

[0859] Prompt Sentence Examples

[0860] "Generate an algorithm that will display a warning message about your driving score if it detects sudden braking. Also include example code to send the data to the cloud for further analysis."

[0861] With the above configuration, the present invention can provide appropriate feedback to drivers in real time, evaluate their driving behavior, and improve their motivation based on the evaluation.In addition, data can be securely provided to insurance companies, allowing them to apply discounts on insurance premiums.

[0862] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0863] Step 1:

[0864] The in-vehicle device (terminal) collects data from the speed sensor, G sensor, GPS sensor, and on-board camera. This includes the vehicle's speed, acceleration, location information, and forward video. The input is raw data from each sensor, and the output is integrated sensor data. This collected data is sent to the next processing stage in real time.

[0865] Step 2:

[0866] The device analyzes the collected data in real time. It processes and analyzes the data using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. For example, it analyzes fluctuations in speed and acceleration and generates a warning message if sudden braking or acceleration is detected. The input is the integrated sensor data, and the output is the analysis result and whether or not a warning is issued. This warning message is immediately notified to the driver.

[0867] Step 3:

[0868] The device periodically transmits the analyzed data to the cloud server. This transmission occurs, for example, every 30 minutes, in preparation for the integration and analysis of large amounts of data in the cloud. The input is the analyzed sensor data, and the output is the data transmitted to the cloud server.

[0869] Step 4:

[0870] The cloud server integrates the received data and performs detailed analysis. This analysis is performed using cloud platform services (e.g., AWS Lambda and Amazon S3) such as AWS and Google Cloud. The input is the data sent to the cloud, and the output is the evaluation results of the driver's driving behavior. This evaluation is multidimensional, and each element of driving behavior (such as speed limits and the number of sudden braking attempts) is evaluated.

[0871] Step 5:

[0872] The server calculates a driving score (DriveScore) based on the evaluation results. Each element of driving behavior is calculated using an algorithm and expressed as an overall score. The input is the evaluation result of driving behavior, and the output is the DriveScore. This score is registered in the user's account.

[0873] Step 6:

[0874] The server provides feedback to the user's device based on the calculated DriveScore. The score and warning messages are displayed on the smartphone app or smart glasses. The input is the DriveScore, and the output is the feedback information displayed on the smartphone app or smart glasses.

[0875] Step 7:

[0876] The server sets goals and rewards for the driver based on the acquired DriveScore. The user's progress is managed using a database such as Firebase or MongoDB. The input is the DriveScore, and the output is the goal and reward information sent to the user's device.

[0877] Step 8:

[0878] The server provides the DriveScore of consenting users to the insurance company using a secure API (e.g., OAuth 2.0), which then applies premium discounts based on the DriveScore. The input is the DriveScore, and the output is the data sent to the insurance company and the applied premium discount.

[0879] As described above, a real-time safe driving feedback system is realized by processing and analyzing the data obtained at each step and proceeding with the process sequentially.

[0880] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0881] MODE FOR CARRYING OUT THE INVENTION

[0882] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The emotion engine can recognize the driver's emotional state in real time and provide appropriate advice and warnings based on that information.

[0883] 1. Data Collection

[0884] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously collects data from these sensors. The emotion engine also analyzes the driver's facial expressions and tone of voice to obtain emotional data.

[0885] Example: The device collects real-time data on the vehicle's speed, acceleration, and location, as well as changes in the driver's facial expressions and voice.

[0886] 2. Real-time analysis and user notification

[0887] The device analyzes sensor data and emotion data in real time to evaluate the driver's driving situation and emotional state. The emotion engine detects the driver's emotions such as stress, impatience, and fatigue, and provides advice and warnings at the appropriate time.

[0888] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0889] 3. Data transmission and cloud analysis

[0890] The device periodically transfers the collected and pre-processed data (speed, acceleration, position, and emotion data) in batches to a cloud server.

[0891] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[0892] 4. DriveScore calculation and feedback

[0893] The server evaluates the driving behavior and emotional data based on the integrated data to calculate a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation. The calculated DriveScore is registered in the user's account and can be viewed on the user's device.

[0894] Example: The server calculates the DriveScore based on the number of sudden braking incidents, fluctuations in acceleration, and the driver's emotional data, and provides the results to the user so that they can be checked on a smartphone app.

[0895] 5. Gamification and rewards

[0896] The server sets goals and rewards for users based on the calculated DriveScore. For example, users who reach a certain score will receive a digital badge or be ranked high in the weekly rankings.

[0897] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[0898] 6. Insurance premium discounts

[0899] With the user's consent, the server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[0900] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[0901] This system not only helps novice and inexperienced drivers develop safe driving habits, but also significantly reduces driving risks by taking their emotional state into account. Furthermore, by providing specific incentives and feedback to users, it can highly motivate them to drive safely.

[0902] The processing flow will be explained below.

[0903] Step 1:

[0904] As soon as the vehicle's engine starts, the device starts collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information on the current speed, acceleration, position, distance to obstacles ahead, etc. The emotion engine also analyzes the driver's facial expressions and tone of voice to collect emotion data.

[0905] Step 2:

[0906] The device temporarily stores the collected sensor data and emotion data, and performs preprocessing such as noise reduction and data filtering to remove outliers and smooth the location data. The device also filters out ambiguous emotion determinations from the emotion data.

[0907] Step 3:

[0908] The device analyzes the pre-processed data in real time to assess the driving situation and the driver's emotional state, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short, and assessing the driver's emotional state if it indicates impatience or stress.

[0909] Step 4:

[0910] The device provides real-time advice and warnings to the driver as needed through voice messages and display, and the emotion engine changes the content and tone of the advice based on the driver's emotional state at the time.

[0911] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[0912] Step 5:

[0913] The device transfers the collected and pre-processed data (speed, acceleration, location, emotion data) to a cloud server in batches at regular intervals (e.g., every 30 minutes), which includes all data points during driving.

[0914] Step 6:

[0915] The server then integrates the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking and accelerations, as well as emotional fluctuations.

[0916] Step 7:

[0917] The server evaluates the driver's driving safety, acceleration and deceleration smoothness, speed adherence rate, and emotional state based on the integrated data, and calculates a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation.

[0918] Step 8:

[0919] The server registers the calculated DriveScore in the driver's account, allowing the driver to check the results on a smartphone app.

[0920] Example: The server displays the information "This week's DriveScore is 88" on the user's smartphone app.

[0921] Step 9:

[0922] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore will be rewarded with a digital badge or a high ranking in the weekly rankings.

[0923] Example: The server notifies drivers with a DriveScore of 80 or above with a "Top Driver of the Week" badge on their smartphone app.

[0924] Step 10:

[0925] With the user's consent, the server provides the DriveScore to the insurance company, which applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result.

[0926] Example: The server provides data on users whose DriveScore is 90 or higher to an insurance company, and the insurance company applies discounts to insurance premiums based on this data. For example, it sends a notification saying, "Your insurance premium will be discounted by 10%."

[0927] In this way, a system can be built that uses collected data to perform detailed analysis of driving behavior and emotional states, providing drivers with specific feedback and incentives to promote safe driving and even offer discounts on insurance premiums.

[0928] Example 2

[0929] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0930] Conventional driving evaluation systems for in-vehicle devices rely primarily on driving behavior data and do not take into account the driver's emotional state, making it difficult to accurately evaluate driving conditions. Furthermore, they lack the ability to provide real-time feedback and appropriate advice to drivers, making it difficult to promote safe driving. Furthermore, incentive systems and insurance discounts that utilize driving evaluation results have limitations because they do not include emotional data.

[0931] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0932] In this invention, the server includes means for collecting sensor data and emotional data acquired by an in-vehicle device, means for analyzing the sensor data and emotional data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating the driving behavior and emotional state based on the data, means for calculating a driving score that takes into account the driving score and emotional state based on the evaluation, and means for feeding back the driving score to the driver and notifying the insurance company of the score and applying a discount on the insurance premium. This enables more accurate driving evaluation that takes into account not only the driver's driving behavior but also their emotional state, leading to the promotion of safe driving and the provision of appropriate incentives.

[0933] An "on-board device" is a device that is installed in a vehicle and includes multiple sensors such as a speed sensor, a G sensor, a GPS sensor, and an on-board camera.

[0934] "Sensor data" refers to driving-related data such as speed, acceleration, and location, and is information collected from in-vehicle devices.

[0935] "Emotion data" refers to data relating to the driver's emotional state, obtained by analyzing the driver's facial expressions and tone of voice.

[0936] An "emotion engine" is a software or hardware device that analyzes the driver's facial expressions and tone of voice and generates emotion data.

[0937] "Cloud server" refers to a remote server that stores and analyzes data over the Internet, and is the place where collected data is processed and stored.

[0938] The "driving score" is a numerical value that indicates the evaluation result of the driver's driving behavior, calculated by analyzing sensor data and emotional data.

[0939] "Advice and Reminders" refers to suggestions and warning messages provided based on the driver's driving situation and emotional state.

[0940] "Feedback" refers to notifying the driver of the calculated driving score and warnings, and is primarily done through the user's device.

[0941] "Gamification" is a method of incorporating game elements to evaluate a driver's driving behavior and increase motivation by setting goals and rewards.

[0942] "User device" means a device on which a driver checks their driving score and feedback, including, for example, a smartphone or tablet.

[0943] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The specific configuration and operation of this system will be described below.

[0944] In-vehicle devices

[0945] The in-vehicle device is equipped with a speed sensor, a G-sensor, a GPS sensor, and an on-board camera. The device continuously collects data from these sensors. It also uses an emotion engine to analyze the driver's facial expressions and tone of voice to collect emotional data. For example, the vehicle's speed and acceleration are acquired every second while driving, and the driver's facial expressions captured by the on-board camera are analyzed using image recognition software.

[0946] Cloud Server

[0947] Data collected by the device is periodically sent in batches to a cloud server. The cloud server then integrates the received data and prepares it for analysis. The server then analyzes speed, acceleration, position, and emotional data to evaluate the driver's driving behavior and emotional state. This information is then used to calculate a driving score (DriveScore). By including emotional data in addition to sensor data, driving evaluation can be performed with greater accuracy than before.

[0948] Feedback and User Devices

[0949] The server registers the calculated driving score in the user's account and notifies the result via the user's device (e.g., smartphone). For example, a message such as "Your DriveScore is 85" is displayed on the user's device. In addition, real-time warnings and advice are also provided through the user's device. For example, while driving, the user can receive a voice message saying, "Take your time and slow down."

[0950] Gamification and rewards

[0951] The server sets goals and rewards for users based on their driving scores. When a certain score is reached, rewards such as digital badges and weekly rankings are provided. For example, a "Top Driver of the Week" badge is awarded, and a notification appears on the user's smartphone saying, "Congratulations! You're the Top Driver of the Week."

[0952] Insurance premium discounts

[0953] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount on insurance premiums based on the driving score. For example, if a user's DriveScore is 90 or higher, the server provides the data to the insurance company, and the user receives a 10% discount on insurance premiums.

[0954] This system can evaluate the driver's driving behavior and emotional state in detail, promoting safe driving and improving motivation. As a specific example, the following prompt sentences can be input into the generative AI model:

[0955] Example prompt sentence:

[0956] 1. "Describe a system that analyzes the driver's emotional state in real time based on data collected by an in-car device and provides appropriate advice."

[0957] 2. "Please explain in detail how the device will notify the user with a warning message if the user becomes impatient while driving."

[0958] 3. "Please explain what DriveScore is, how it is calculated, and what feedback it provides to users."

[0959] 4. "Can you give us a concrete example of how insurance discounts based on driving data work?"

[0960] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0961] Step 1:

[0962] The terminal collects data in real time from each sensor of the in-vehicle device (speed sensor, G sensor, GPS sensor, and in-vehicle camera). This collected data includes speed, acceleration, location, and the driver's facial expression and tone of voice. An emotion engine is used to analyze this data and generate driver emotion data. The input sensor data and facial expression data become real-time driving data and emotion data, and this data is stored in the terminal.

[0963] Step 2:

[0964] The device analyzes the collected sensor data and emotional data in real time. Specifically, it evaluates the driver's driving behavior and emotional state. This allows it to automatically generate appropriate advice or warnings if the driver appears to be impatient, tired, or stressed. Using the real-time data and emotional data as input, the device generates driving situation evaluation data and emotional evaluation data through analysis.

[0965] Step 3:

[0966] The device then provides the driver with advice or warning messages based on the analysis results. For example, if the driver is in a hurry and speeding up, the device will notify the user with a voice message saying, "Please stay calm and slow down." Based on the evaluation data input, a specific message is generated and notified to the user as voice or display output.

[0967] Step 4:

[0968] The device periodically transfers the collected data (speed, acceleration, position, and emotion data) to the cloud server in batches. Specifically, all data is packaged at regular intervals and sent to the cloud server via a communication line. The accumulated data as input is converted into a batch package and sent to the cloud server.

[0969] Step 5:

[0970] The server consolidates the data entered on the cloud and prepares it for analysis, specifically storing it in a literal database and performing any necessary preprocessing (e.g., data cleansing and format conversion). It receives batch data as input, stores and preprocesses it, and outputs it in an analyzable format.

[0971] Step 6:

[0972] The server evaluates driving behavior and emotional state based on the integrated data and calculates a driving score (DriveScore). Specifically, it analyzes the number of sudden braking attempts, fluctuations in acceleration, and the driver's emotional data to generate a quantified driving score. It evaluates the integrated data as input and outputs a driving score.

[0973] Step 7:

[0974] The server registers the calculated driving score to the user's account and provides it as visual information via the user's device. Specifically, it sends a notification to the user via a smartphone app saying, "Your DriveScore is 85." Using the driving score data as input, it generates a visual feedback message and outputs it to the user's device.

[0975] Step 8:

[0976] The server sets goals and rewards based on the driving score. For example, it can award a "Top Driver of the Week" badge to drivers who reach a certain score and notify them via a smartphone app. The server uses the driving score as input to set rewards and award badges, and notifies the user of the reward information as output.

[0977] Step 9:

[0978] After obtaining the user's consent, the server provides the DriveScore to the insurance company, which then applies a discount to the insurance premium. Specifically, if the user's DriveScore is 90 or higher, the server sends the user's data via API, and the insurance company notifies and applies the premium discount. The server sends the score data as input and receives the result of the premium discount application as output.

[0979] (Application example 2)

[0980] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0981] Conventional automated driving systems perform driving operations without taking into account the driver's emotional state, which means they are unable to effectively reduce driver stress and fatigue. Furthermore, the lack of technology to appropriately adjust automated driving parameters based on the driver's emotional state has prevented sufficient improvements in safety.

[0982] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for analyzing the driver's emotional state and adjusting automated driving parameters based on the data, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, and means for providing feedback about the driving score to the driver, notifying the insurance company, and applying a discount on insurance premiums. This makes it possible to recognize the driver's emotional state in real time and adjust appropriate automated driving parameters based on the driving score. Furthermore, by providing feedback and rewards to the driver, it is possible to encourage improvement in driving behavior and improve safety.

[0983] An "in-vehicle device" is a device that is installed in a vehicle and includes various sensors and cameras for acquiring driving data and environmental data.

[0984] "Sensor data" is a general term for data acquired by in-vehicle devices, such as speed, acceleration, GPS location information, the driver's facial expressions, and voice tone.

[0985] "Advice and Caution" refers to appropriate instructions or warnings provided to the driver based on sensor data and emotional state.

[0986] A "cloud server" is a remote server that stores collected data and performs integrated analysis and evaluation.

[0987] The "means for evaluating driving behavior" is a mechanism that analyzes the driver's driving behavior based on collected sensor data and emotional state, and generates an evaluation result.

[0988] The "driving score" is a numerical index that comprehensively evaluates driving behavior and emotional state.

[0989] "Feedback" is a means of providing information to drivers by notifying them of their driving score, advice, and warnings, encouraging them to improve their driving behavior.

[0990] "Insurance premium discounts" are premium reduction benefits applied by insurance companies based on driving scores.

[0991] "Emotional state" is information that indicates the psychological state of the driver, such as stress, fatigue, or impatience.

[0992] "Autonomous driving parameters" are control elements such as speed, following distance, and braking sensitivity that are set when the autonomous driving system performs driving operations.

[0993] "Game elements" are fun, competitive elements that set goals and rewards to encourage improved driving behavior.

[0994] "Smartphone app" means a software application for a mobile device used to provide visual driving scores and feedback to the driver.

[0995] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. Using the emotion engine, it is possible to recognize the driver's emotional state in real time and adjust autonomous driving parameters based on that. This system is configured by combining the following hardware and software.

[0996] Hardware used

[0997] 1. In-vehicle devices:

[0998] This includes an in-car camera, microphone, speed sensor, G-sensor, and GPS sensor.

[0999] Data is obtained from these sensors in real time.

[1000] 2. Cloud Server:

[1001] A remote server for storing and analyzing data.

[1002] Examples: AWS (Amazon Web Services), Google Cloud, etc.

[1003] 3. User Device:

[1004] Mobile devices such as smartphones and tablets.

[1005] Used to review data and provide feedback.

[1006] Software used

[1007] 1. Emotion Engine:

[1008] An AI model for analyzing facial expressions and tone of voice.

[1009] Examples: OpenAI, Google TensorFlow.

[1010] 2. Data analysis platform:

[1011] A tool for integrating and analyzing data.

[1012] For example: Apache Spark, Hadoop.

[1013] 3. Smartphone App:

[1014] An application that displays and notifies user data.

[1015] System Operation

[1016] Data collection

[1017] The in-vehicle device collects real-time sensor data such as speed, acceleration, GPS location, the driver's facial expressions and voice tone, allowing for accurate understanding of the driving situation and the driver's emotional state.

[1018] Real-time analysis and parameter adjustment

[1019] The terminal (in-vehicle device) analyzes the acquired data in real time. The emotion engine analyzes the driver's facial expressions and tone of voice to identify the driver's emotional state, such as stress, fatigue, or impatience. Based on this data, the autonomous driving system's parameters (speed, following distance, braking sensitivity, etc.) are adjusted appropriately.

[1020] Data transmission and cloud analysis

[1021] The device periodically transmits the acquired data to a cloud server, which then comprehensively analyzes the data and evaluates the driver's driving behavior and emotional state.

[1022] DriveScore calculation and feedback

[1023] The cloud server calculates a driving score (DriveScore) based on the integrated data. This score is the result of a comprehensive evaluation of driving behavior and emotional state. The calculated DriveScore is provided as feedback to the driver via the user's device.

[1024] Gamification and rewards

[1025] The cloud server sets goals and rewards for drivers based on their DriveScore. For example, achieving a certain score will earn them a digital badge or a spot at the top of the weekly rankings.

[1026] Insurance premium discounts

[1027] With the user's consent, the cloud server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[1028] Specific examples

[1029] For example, if the in-car device detects that the driver is in an "anger" state from their facial expression, the system will automatically slow down the vehicle and give a voice notification to the driver saying "Please relax." In addition, the collected data is sent to a cloud server every 30 minutes, and a driving score (DriveScore) is calculated through integrated analysis. Users can check this score on their smartphone app.

[1030] Prompt Sentence Examples

[1031] "Perform real-time emotion analysis using driver facial expression data and vehicle driving data, and generate Python code to set appropriate autonomous driving parameters."

[1032] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1033] Step 1: Data collection

[1034] The terminal (in-vehicle device) uses an in-vehicle camera, microphone, speed sensor, G sensor, and GPS sensor to capture the driver's facial expression, tone of voice, vehicle speed, acceleration, and location information in real time. The input data are facial expression data, voice data, speed data, G sensor data, and GPS data, which are sent to the analysis unit after initial processing. The output is initially processed emotion data and driving data.

[1035] Step 2: Sentiment Analysis

[1036] The device uses an emotion engine to analyze the collected facial expression data and voice data in real time. This analysis identifies the driver's emotional state (e.g., stress, fatigue, anger, etc.). The input is initially processed facial expression data and voice data, which are then fed into the emotion engine, which outputs the driver's emotional state.

[1037] Step 3: Real-time analysis and autonomous driving parameter adjustment

[1038] The device combines emotional state and driving data for real-time analysis and adjusts autonomous driving parameters (speed, following distance, braking sensitivity, etc.). The input is driving data and analyzed emotional data, and based on this data, it calculates and outputs autonomous driving control parameters. Specifically, if the driver's emotion is "anger," the device will slow down the vehicle and give a voice notification.

[1039] Step 4: Send data to the cloud server

[1040] The device periodically (e.g., every 30 minutes) sends the collected and preprocessed data to the cloud server in batch format. The input is the preprocessed driving data and emotion data, and the cloud server receives these data. The output is the integrated data sent to the cloud server.

[1041] Step 5: Integrated analysis and DriveScore calculation

[1042] The server (cloud server) comprehensively analyzes the transmitted data, evaluates driving behavior and emotional state, and calculates a DriveScore. The input is collected historical driving data and emotional data, which is used to apply statistical analysis and machine learning models. The output is the driver's DriveScore. A generative AI model is used to comprehensively evaluate emotions and driving behavior.

[1043] Step 6: Provide feedback

[1044] The server provides the calculated DriveScore as feedback to the user's smartphone app. The input is the calculated DriveScore, which is reported to the user device. The output is feedback information displayed on the user device, such as a "Top Driver of the Week" badge.

[1045] Step 7: Gamify and reward

[1046] The server sets goals and rewards based on the DriveScore and provides them to the user. The input is the DriveScore and user profile data, and based on this, the server sets goals and determines rewards. The output is a digital badge and weekly ranking display provided to the user.

[1047] Step 8: Provide data to insurance companies and apply discounts

[1048] With the user's consent, the server provides the insurance company with the user's DriveScore, and the insurance company applies a discount on the premium based on the score. The input is the user's DriveScore and consent information, which are sent to the insurance company. The output is a notification from the insurance company that the discount has been applied.

[1049] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1050] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1051] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1052] [Fourth embodiment]

[1053] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1054] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1055] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1056] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1057] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1058] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1059] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1060] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1061] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1062] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1063] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1064] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1065] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1066] MODE FOR CARRYING OUT THE INVENTION

[1067] This system is realized by linking an in-vehicle device, a cloud server, and a user device (smartphone app) installed in the vehicle. This system provides appropriate advice and warnings to the driver in real time, evaluates driving behavior to calculate a driving score, and notifies the score to the insurance company to apply discounts on insurance premiums.

[1068] 1. Data Collection

[1069] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously acquires data from these sensors.

[1070] Example: When the vehicle exceeds a certain speed, the device measures the speed information in real time and also acquires acceleration data from the G sensor and location data from the GPS sensor.

[1071] 2. Real-time analysis and user notification

[1072] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if the distance to the vehicle ahead is too close or if the speed limit is exceeded, the device will warn the driver with a voice message or a display.

[1073] Example: When the distance to the vehicle ahead falls below a certain level, the device notifies the user with a voice message saying, "The distance to the vehicle ahead is too short. Please reduce your speed."

[1074] 3. Data transmission and cloud analysis

[1075] The device periodically transmits the collected data to a cloud server, where it is integrated and analyzed in detail.

[1076] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[1077] 4. DriveScore calculation and feedback

[1078] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." The calculated DriveScore is registered in the user's account and can be viewed from the user's device.

[1079] Example: The server calculates the DriveScore based on data such as the number of sudden braking attempts, fluctuations in acceleration, and speed compliance, and provides the results so that the user can check them on a smartphone app.

[1080] 5. Gamification and rewards

[1081] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a certain score is reached, the driver will be given a digital badge and displayed at the top of the weekly rankings.

[1082] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[1083] 6. Insurance premium discounts

[1084] With the user's consent, the server provides the calculated DriveScore to the insurance company, which then applies a discount to the insurance premium based on the score.

[1085] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[1086] This invention makes it easier for novice drivers and inexperienced drivers to develop safe driving habits, which is expected to reduce the risk of traffic accidents. In addition, by evaluating driving behavior, it is possible to provide specific incentives and increase user motivation.

[1087] The processing flow will be explained below.

[1088] Step 1:

[1089] As soon as the vehicle's engine starts, the device begins collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information such as current speed, acceleration, position, and distance to obstacles ahead.

[1090] Step 2:

[1091] The device stores the collected sensor data in temporary memory and performs pre-processing such as noise removal and data filtering, which removes outliers and smooths the location data.

[1092] Step 3:

[1093] The device analyzes the pre-processed data in real time to assess the driving situation, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short.

[1094] Step 4:

[1095] The device will provide real-time advice and reminders to the driver as needed through voice messages and display alerts, such as a notification that says, "You are too close to the vehicle ahead. Please reduce your speed."

[1096] Step 5:

[1097] The device transfers the collected and pre-processed data to the cloud server in batches at regular intervals (e.g., every 30 minutes). This data includes information on speed, position, acceleration, and distance to obstacles ahead.

[1098] Step 6:

[1099] The server then combines the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking attempts, sudden acceleration attempts, etc.

[1100] Step 7:

[1101] Based on the integrated data, the server calculates a driving score (DriveScore) using parameters such as safe driving, smooth acceleration and deceleration, and speed compliance.

[1102] Step 8:

[1103] The server registers the calculated DriveScore in the driver's account and allows the driver to check the result on their smartphone app. For example, the server provides information such as "This week's DriveScore is 88 points."

[1104] Step 9:

[1105] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore are given rewards such as a digital badge or a place at the top of the weekly rankings.

[1106] Step 10:

[1107] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result. For example, the insurance company may notify the user that "Your DriveScore is 90 or higher, so you will receive a 10% discount on your insurance premium."

[1108] This will establish a complete process from collection to analysis, feedback, incentive provision, and insurance premium discount application.

[1109] Example 1

[1110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1111] In modern vehicle driving, systems that promote safe driving and evaluate driver behavior still face challenges. In particular, there is no well-established system that evaluates driving behavior in real time, provides appropriate feedback, and links that evaluation to insurance premium discounts. Furthermore, there is still a lack of gamification elements to increase driver motivation and methods to easily visualize driving scores.

[1112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1113] In this invention, the server includes: means for collecting multiple types of sensor data acquired by devices installed in the vehicle; means for analyzing the multiple sensor data in real time and providing advice and warnings to the driver; means for transmitting the collected data to a remote server via a network and evaluating driving behavior based on the data; means for calculating a driving score based on the evaluation; means for providing the driving score as feedback to the driver and notifying an external institution and applying a fee discount; means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements and providing them based on the driving score; and means for registering the driving score in the driver's account and providing it to the driver as visual information via a mobile device application. This promotes safe driving, appropriately evaluates the driver's behavior, increases motivation, and enables the application of insurance premium discounts.

[1114] A "vehicle-mounted device" is a hardware device that is placed inside a vehicle and uses multiple sensors and cameras to collect driving data.

[1115] "Sensor data" refers to information obtained from speed sensors, G sensors, GPS sensors, and on-board cameras, including vehicle speed, acceleration, position, and images.

[1116] "Real-time analysis" means that data is processed as soon as it is collected, providing immediate feedback to the driver.

[1117] "Advice and warnings" refers to providing suggestions and warnings to encourage improvement based on the driver's driving behavior in the form of voice messages or display on the screen.

[1118] "Transmitting to a remote server via a network" refers to transmitting the collected data to a server in a remote location via a communication network such as the Internet.

[1119] The "means for evaluating driving behavior" is software or an algorithm that analyzes the driver's driving behavior based on the collected data and makes an evaluation based on that analysis.

[1120] A "driving score" is an index that quantifies the evaluation results of driving behavior and allows drivers to understand their driving performance at a glance.

[1121] "External organizations" are third-party organizations that are expected to notify driving scores, and mainly refer to insurance companies and traffic safety organizations.

[1122] "Means for applying discounts" refers to a system that reduces insurance premiums and other fees for drivers based on their assessed driving score.

[1123] A "driving evaluation method incorporating game elements" is a method in which driving behavior is evaluated in a game format and goals and rewards are set in order to increase the driver's motivation.

[1124] "Goals and rewards" refers to goals and incentives given to drivers when they achieve certain driving behaviors, including digital badges and ranking displays.

[1125] "Mobile application" means a software application used on a mobile device, such as a smartphone or tablet, to display driving scores and evaluation results.

[1126] "Means for providing visual information" refers to a method for displaying driving scores and evaluation results to the driver in the form of graphs, charts, text, etc.

[1127] MODE FOR CARRYING OUT THE INVENTION

[1128] The present invention is a system that provides real-time driving evaluation and feedback through collaboration between a vehicle device installed in a vehicle, a cloud server, and a user device (e.g., a smartphone app). This system promotes safe driving, evaluates driving behavior, calculates a driving score, and notifies the score to insurance companies to apply discounts on insurance premiums.

[1129] Device configuration and data collection

[1130] The in-vehicle device is equipped with a speed sensor, G sensor, GPS sensor, and in-vehicle camera. These sensors continuously collect the vehicle's speed, acceleration, position, and forward video data. For example, if the vehicle's speed exceeds 100 km / h, the speed information is measured in real time, and acceleration data from the G sensor and position data from the GPS sensor are simultaneously collected.

[1131] Data analysis and user notification

[1132] The device analyzes the collected data in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." In addition, since footage from the in-car camera is also used for analysis, the device has the function of automatically saving footage of sudden braking and later uploading it to the cloud.

[1133] Data transmission and cloud analysis

[1134] The device periodically transmits the collected data (for example, every 30 minutes) to a cloud server. This transmission is performed using a secure communication protocol (for example, HTTPS). The server then integrates the received data and performs a detailed analysis. This analysis uses a big data analysis tool (for example, Apache Hadoop) to evaluate the number of sudden braking incidents, the frequency of speeding, and fluctuations in acceleration.

[1135] DriveScore calculation and feedback

[1136] The server evaluates driving behavior based on the integrated data and calculates a driving score called "DriveScore." This calculation uses machine learning algorithms (e.g., support vector machines and deep learning). The calculated DriveScore is registered in the user's account and can be viewed as visual information on a smartphone app. When the user opens the smartphone app, they can view a detailed report.

[1137] Gamification and rewards

[1138] The server sets goals and rewards for drivers based on the calculated DriveScore. For example, if a user scores 80 or more out of 100, they will be awarded a digital badge as the "Top Driver of the Week" and notified of this via the smartphone app. Weekly rankings and other information will also be displayed, providing a system that allows users to enjoy their driving behavior in a game-like format.

[1139] Insurance premium discounts

[1140] With the user's consent, the server provides the calculated DriveScore to the insurance company. The insurance company applies a discount on the insurance premium based on this information. For example, if a user receives a DriveScore of 90 or more, the data is sent to the insurance company and the insurance premium is discounted. The user is notified of this discount via a smartphone app.

[1141] Specific examples

[1142] For example, if the following situation occurs:

[1143] The terminal detects when the vehicle's speed exceeds 100 km / h and measures speed information in real time.

[1144] When the device detects sudden braking, it automatically saves five seconds of footage from the onboard camera and uploads it to a cloud server.

[1145] The server uses data from the past 30 days to calculate a driving score based on factors such as the number of sudden braking attempts and speed limits, and allows users to check this score via a smartphone app.

[1146] The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[1147] Prompt Sentence Examples

[1148] Example prompts to be input to the generative AI model:

[1149] Please explain how the system connects in-car devices, cloud servers, and user devices to provide real-time driving ratings and insurance discounts. Please provide examples.

[1150] These steps promote safe driving and increase motivation by providing drivers with tangible incentives.

[1151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1152] Step 1: Data collection

[1153] The terminal collects data from the speed sensor, G sensor, GPS sensor, and on-board camera installed in the in-vehicle device. For example, the speed sensor measures the vehicle's speed once per second, the G sensor acquires acceleration data, and the GPS sensor acquires location data. The on-board camera captures video of the vehicle ahead. All of this data is collected in real time. The input data to the terminal are speed, acceleration, location, and video data, and an initial data set is generated by collecting these in real time.

[1154] Step 2: Real-time analysis

[1155] The device analyzes the data collected in step 1 in real time and evaluates the driver's driving behavior. For example, if sudden braking is detected or the distance to the vehicle ahead is too close, the device analyzes the data and warns the driver. Specifically, it notifies the user with a voice message saying, "The distance to the vehicle ahead is too close. Please reduce your speed." The input data is the collected sensor data, and the output data is a warning message to the driver.

[1156] Step 3: Send data

[1157] The device periodically (e.g., every 30 minutes) transmits data that has been collected and analyzed in real time to a cloud server. This transmission is performed using a secure communication protocol (e.g., HTTPS). The input data here is the analyzed sensor data, and the output data is the integrated data transmitted to the cloud server. Specifically, the device uploads speed, acceleration, position, and video data to the cloud server every 30 minutes.

[1158] Step 4: Cloud Solve

[1159] The server integrates the data received in step 3 and performs detailed analysis. The server processes large amounts of data using big data analysis tools (e.g., Apache Hadoop). It evaluates the number of sudden braking attempts, the frequency of speeding, and fluctuations in acceleration to analyze driving behavior. The input data is the integrated data sent to the cloud server, and the output data is the analyzed driving behavior data. Specifically, it analyzes the driver's driving patterns based on data from the past 30 days.

[1160] Step 5: Calculate your DriveScore

[1161] The server calculates a driving score, "DriveScore," based on the results of the cloud analysis. This calculation uses machine learning algorithms (such as support vector machines and deep learning). For example, the score is calculated taking into account the number of sudden braking attempts, speed compliance, and fluctuations in acceleration. The input data is the analyzed driving behavior data, and the output data is the calculated DriveScore. Specifically, the server tallies points for each score item based on the evaluation results.

[1162] Step 6: Provide feedback

[1163] The server registers the calculated DriveScore in the user's account and notifies them via a smartphone app. For example, a detailed driving evaluation report can be provided in PDF format. Furthermore, gamification elements can be set, such as goals and rewards. For example, a "Top Driver of the Week" badge can be awarded. The input data is the DriveScore and evaluation results, and the output data is feedback information to the driver.

[1164] Step 7: Apply for discount on premium

[1165] With the user's consent, the server provides the calculated DriveScore to the insurance company and applies a discount on the insurance premium. For example, the insurance company applies a discount of several percent to users with a DriveScore of 90 or more. The input data is the DriveScore, and the output data is a notification to the insurance company and information on the discount application. Specifically, the server sends the evaluation score to the insurance company and notifies the user of the discount application result.

[1166] Through these steps, the system can promote safe driving, specifically evaluate driver behavior, provide feedback, and even link insurance premium discounts.

[1167] (Application example 1)

[1168] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1169] Conventional in-vehicle device systems were limited in the advice and warnings they could give drivers, and the feedback they provided in real time was insufficient. Furthermore, the evaluation of driving behavior and the provision of incentives based on that evaluation were also limited, and no mechanism was in place to increase driver motivation. This meant that the inability to provide appropriate feedback and incentives to drivers resulted in the system not being effective enough in promoting safe driving. Furthermore, there were security concerns regarding the provision of data to insurance companies.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1171] In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, means for feeding back the driving score to the driver and notifying an insurance company and applying a discount on insurance premiums, and means for sending a warning message regarding the driving score to a smartphone app, smart glasses, or an in-vehicle robot. This makes it possible to provide appropriate information to the driver in real time, increase the driver's motivation through a driving evaluation method incorporating game elements, and securely provide data to insurance companies.

[1172] An "in-vehicle device" is a device installed in a vehicle to collect data from speed sensors, G sensors, GPS sensors, cameras, etc.

[1173] "Sensor data" refers to data collected from various sensors, including vehicle speed, acceleration, location information, and video data acquired by on-board cameras.

[1174] "Real-time analysis" refers to instantly processing collected sensor data and making evaluations and decisions according to the situation.

[1175] "Advice and warnings" are information and warning messages about driving behavior provided to the driver.

[1176] A "cloud server" is a server used via the Internet, a remote server for storing and analyzing collected data.

[1177] "Means for evaluating driving behavior" refers to the process of analyzing and evaluating the driver's driving patterns and behavior based on collected sensor data.

[1178] The "driving score" is a numerical representation of the evaluation results of driving behavior, and serves as an indicator of the safety and smoothness of a driver's driving.

[1179] "Feedback" refers to conveying driving scores and warning messages to drivers.

[1180] "Notifying insurance company" means sending data such as driving scores to the insurance company in a secure manner.

[1181] "Premium discount" refers to a reduction in premiums applied to a policyholder based on their driving score.

[1182] A "smartphone app" is application software that runs on a smartphone and provides various notifications and feedback to the driver.

[1183] "Smart glasses" are glasses-type devices that have the function of overlaying information onto the driver's vision.

[1184] An "in-vehicle robot" is a robotic device that is installed inside a vehicle and provides warnings and advice to the driver through voice and screen displays.

[1185] "Providing appropriate information in real time" means providing drivers with advice and warnings that respond to their driving behavior immediately.

[1186] A "driving evaluation method incorporating game elements" is a method for evaluating a driver's driving behavior using playful elements such as rankings and badges, thereby increasing the driver's motivation.

[1187] "Providing data securely" refers to using technologies and protocols to ensure safety during the data transmission process.

[1188] This system promotes safe driving by providing real-time feedback to drivers through collaboration between in-vehicle devices, a cloud server, and user devices (such as smartphone apps and smart glasses). This system can also evaluate driving behavior, calculate a driving score, and apply insurance discounts.

[1189] 1. Data Collection

[1190] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. Data from these sensors is collected in real time. Specific hardware that can be used is an embedded Linux device such as a Raspberry Pi.

[1191] 2. Real-time analysis

[1192] The collected data is analyzed in real time using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. Based on the analysis results, the driver's driving behavior is evaluated and appropriate advice or warning messages are provided. For example, if the distance to the vehicle ahead is too close, a voice message will be issued saying, "The distance to the vehicle ahead is too close. Please reduce your speed."

[1193] 3. Data transmission and cloud analysis

[1194] The collected data is sent to a cloud server at regular intervals, where it is analyzed using cloud platforms such as AWS and Google Cloud, and services such as Amazon S3 and AWS Lambda.

[1195] 4. DriveScore calculation and feedback

[1196] The cloud server integrates and analyzes the collected data, evaluates driving behavior, and calculates a driving score, "DriveScore," based on various factors. This DriveScore is provided as feedback to the user's device (smartphone app or smart glasses).

[1197] 5. Gamification and rewards

[1198] The cloud server sets goals and rewards for drivers based on the calculated DriveScore. User progress is managed using databases such as Firebase and MongoDB. For example, drivers with a DriveScore of 80 or higher are awarded a digital badge called "Top Driver of the Week" and notified via a smartphone app.

[1199] 6. Insurance premium discounts

[1200] The server then provides the driving score to insurance companies using a secure API, which then applies discounts to insurance premiums based on the data. The data is transmitted using a security protocol such as OAuth 2.0.

[1201] Specific examples

[1202] For example, if a driver suddenly brakes, a notification will appear on the smart glasses saying, "Sudden braking has been detected. This will affect your driving score." The data will also be sent to a cloud server for further analysis.

[1203] Prompt Sentence Examples

[1204] "Generate an algorithm that will display a warning message about your driving score if it detects sudden braking. Also include example code to send the data to the cloud for further analysis."

[1205] With the above configuration, the present invention can provide appropriate feedback to drivers in real time, evaluate their driving behavior, and improve their motivation based on the evaluation.In addition, data can be securely provided to insurance companies, allowing them to apply discounts on insurance premiums.

[1206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1207] Step 1:

[1208] The in-vehicle device (terminal) collects data from the speed sensor, G sensor, GPS sensor, and on-board camera. This includes the vehicle's speed, acceleration, location information, and forward video. The input is raw data from each sensor, and the output is integrated sensor data. This collected data is sent to the next processing stage in real time.

[1209] Step 2:

[1210] The device analyzes the collected data in real time. It processes and analyzes the data using machine learning libraries such as TensorFlow and PyTorch, which run in a Python environment. For example, it analyzes fluctuations in speed and acceleration and generates a warning message if sudden braking or acceleration is detected. The input is the integrated sensor data, and the output is the analysis result and whether or not a warning is issued. This warning message is immediately notified to the driver.

[1211] Step 3:

[1212] The device periodically transmits the analyzed data to the cloud server. This transmission occurs, for example, every 30 minutes, in preparation for the integration and analysis of large amounts of data in the cloud. The input is the analyzed sensor data, and the output is the data transmitted to the cloud server.

[1213] Step 4:

[1214] The cloud server integrates the received data and performs detailed analysis. This analysis is performed using cloud platform services (e.g., AWS Lambda and Amazon S3) such as AWS and Google Cloud. The input is the data sent to the cloud, and the output is the evaluation results of the driver's driving behavior. This evaluation is multidimensional, and each element of driving behavior (such as speed limits and the number of sudden braking attempts) is evaluated.

[1215] Step 5:

[1216] The server calculates a driving score (DriveScore) based on the evaluation results. Each element of driving behavior is calculated using an algorithm and expressed as an overall score. The input is the evaluation result of driving behavior, and the output is the DriveScore. This score is registered in the user's account.

[1217] Step 6:

[1218] The server provides feedback to the user's device based on the calculated DriveScore. The score and warning messages are displayed on the smartphone app or smart glasses. The input is the DriveScore, and the output is the feedback information displayed on the smartphone app or smart glasses.

[1219] Step 7:

[1220] The server sets goals and rewards for the driver based on the acquired DriveScore. The user's progress is managed using a database such as Firebase or MongoDB. The input is the DriveScore, and the output is the goal and reward information sent to the user's device.

[1221] Step 8:

[1222] The server provides the DriveScore of consenting users to the insurance company using a secure API (e.g., OAuth 2.0), which then applies premium discounts based on the DriveScore. The input is the DriveScore, and the output is the data sent to the insurance company and the applied premium discount.

[1223] As described above, a real-time safe driving feedback system is realized by processing and analyzing the data obtained at each step and proceeding with the process sequentially.

[1224] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1225] MODE FOR CARRYING OUT THE INVENTION

[1226] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The emotion engine can recognize the driver's emotional state in real time and provide appropriate advice and warnings based on that information.

[1227] 1. Data Collection

[1228] The in-vehicle device is equipped with a speed sensor, a G sensor, a GPS sensor, and an in-vehicle camera. The in-vehicle device, which is a terminal, continuously collects data from these sensors. The emotion engine also analyzes the driver's facial expressions and tone of voice to obtain emotional data.

[1229] Example: The device collects real-time data on the vehicle's speed, acceleration, and location, as well as changes in the driver's facial expressions and voice.

[1230] 2. Real-time analysis and user notification

[1231] The device analyzes sensor data and emotion data in real time to evaluate the driver's driving situation and emotional state. The emotion engine detects the driver's emotions such as stress, impatience, and fatigue, and provides advice and warnings at the appropriate time.

[1232] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[1233] 3. Data transmission and cloud analysis

[1234] The device periodically transfers the collected and pre-processed data (speed, acceleration, position, and emotion data) in batches to a cloud server.

[1235] Example: The device sends all collected data every 30 minutes to a cloud server, which receives it and performs integrated analysis.

[1236] 4. DriveScore calculation and feedback

[1237] The server evaluates the driving behavior and emotional data based on the integrated data to calculate a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation. The calculated DriveScore is registered in the user's account and can be viewed on the user's device.

[1238] Example: The server calculates the DriveScore based on the number of sudden braking incidents, fluctuations in acceleration, and the driver's emotional data, and provides the results to the user so that they can be checked on a smartphone app.

[1239] 5. Gamification and rewards

[1240] The server sets goals and rewards for users based on the calculated DriveScore. For example, users who reach a certain score will receive a digital badge or be ranked high in the weekly rankings.

[1241] Example: The server awards a "Top Driver of the Week" badge to drivers with a DriveScore of 80 or above and notifies them on a smartphone app.

[1242] 6. Insurance premium discounts

[1243] With the user's consent, the server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[1244] Example: A server provides data on users with a DriveScore of 90 or higher to an insurance company, which then applies discounts to insurance premiums based on this data.

[1245] This system not only helps novice and inexperienced drivers develop safe driving habits, but also significantly reduces driving risks by taking their emotional state into account. Furthermore, by providing specific incentives and feedback to users, it can highly motivate them to drive safely.

[1246] The processing flow will be explained below.

[1247] Step 1:

[1248] As soon as the vehicle's engine starts, the device starts collecting data from the speed sensor, G sensor, GPS sensor, and on-board camera, including information on the current speed, acceleration, position, distance to obstacles ahead, etc. The emotion engine also analyzes the driver's facial expressions and tone of voice to collect emotion data.

[1249] Step 2:

[1250] The device temporarily stores the collected sensor data and emotion data, and performs preprocessing such as noise reduction and data filtering to remove outliers and smooth the location data. The device also filters out ambiguous emotion determinations from the emotion data.

[1251] Step 3:

[1252] The device analyzes the pre-processed data in real time to assess the driving situation and the driver's emotional state, for example, checking whether the vehicle is traveling faster than the speed limit or whether the distance to the vehicle ahead is too short, and assessing the driver's emotional state if it indicates impatience or stress.

[1253] Step 4:

[1254] The device provides real-time advice and warnings to the driver as needed through voice messages and display, and the emotion engine changes the content and tone of the advice based on the driver's emotional state at the time.

[1255] Example: If the driver is in a hurry and speeding up, the device notifies the user with a voice message saying, "Please stay calm and slow down."

[1256] Step 5:

[1257] The device transfers the collected and pre-processed data (speed, acceleration, location, emotion data) to a cloud server in batches at regular intervals (e.g., every 30 minutes), which includes all data points during driving.

[1258] Step 6:

[1259] The server then integrates the received data and performs a detailed analysis based on multiple driving sessions, assessing overall driving behavior and counting the number of sudden braking and accelerations, as well as emotional fluctuations.

[1260] Step 7:

[1261] The server evaluates the driver's driving safety, acceleration and deceleration smoothness, speed adherence rate, and emotional state based on the integrated data, and calculates a driving score (DriveScore). Taking emotional state into account enables more accurate evaluation.

[1262] Step 8:

[1263] The server registers the calculated DriveScore in the driver's account, allowing the driver to check the results on a smartphone app.

[1264] Example: The server displays the information "This week's DriveScore is 88" on the user's smartphone app.

[1265] Step 9:

[1266] The server sets goals and rewards for drivers based on their DriveScore. For example, drivers who achieve a certain DriveScore will be rewarded with a digital badge or a high ranking in the weekly rankings.

[1267] Example: The server notifies drivers with a DriveScore of 80 or above with a "Top Driver of the Week" badge on their smartphone app.

[1268] Step 10:

[1269] With the user's consent, the server provides the DriveScore to the insurance company, which applies a discount to the insurance premium based on the provided DriveScore and notifies the user of the result.

[1270] Example: The server provides data on users whose DriveScore is 90 or higher to an insurance company, and the insurance company applies discounts to insurance premiums based on this data. For example, it sends a notification saying, "Your insurance premium will be discounted by 10%."

[1271] In this way, a system can be built that uses collected data to perform detailed analysis of driving behavior and emotional states, providing drivers with specific feedback and incentives to promote safe driving and even offer discounts on insurance premiums.

[1272] Example 2

[1273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1274] Conventional driving evaluation systems for in-vehicle devices rely primarily on driving behavior data and do not take into account the driver's emotional state, making it difficult to accurately evaluate driving conditions. Furthermore, they lack the ability to provide real-time feedback and appropriate advice to drivers, making it difficult to promote safe driving. Furthermore, incentive systems and insurance discounts that utilize driving evaluation results have limitations because they do not include emotional data.

[1275] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1276] In this invention, the server includes means for collecting sensor data and emotional data acquired by an in-vehicle device, means for analyzing the sensor data and emotional data in real time and providing advice and warnings to the driver, means for transmitting the collected data to a cloud server and evaluating the driving behavior and emotional state based on the data, means for calculating a driving score that takes into account the driving score and emotional state based on the evaluation, and means for feeding back the driving score to the driver and notifying the insurance company of the score and applying a discount on the insurance premium. This enables more accurate driving evaluation that takes into account not only the driver's driving behavior but also their emotional state, leading to the promotion of safe driving and the provision of appropriate incentives.

[1277] An "on-board device" is a device that is installed in a vehicle and includes multiple sensors such as a speed sensor, a G sensor, a GPS sensor, and an on-board camera.

[1278] "Sensor data" refers to driving-related data such as speed, acceleration, and location, and is information collected from in-vehicle devices.

[1279] "Emotion data" refers to data relating to the driver's emotional state, obtained by analyzing the driver's facial expressions and tone of voice.

[1280] An "emotion engine" is a software or hardware device that analyzes the driver's facial expressions and tone of voice and generates emotion data.

[1281] "Cloud server" refers to a remote server that stores and analyzes data over the Internet, and is the place where collected data is processed and stored.

[1282] The "driving score" is a numerical value that indicates the evaluation result of the driver's driving behavior, calculated by analyzing sensor data and emotional data.

[1283] "Advice and Reminders" refers to suggestions and warning messages provided based on the driver's driving situation and emotional state.

[1284] "Feedback" refers to notifying the driver of the calculated driving score and warnings, and is primarily done through the user's device.

[1285] "Gamification" is a method of incorporating game elements to evaluate a driver's driving behavior and increase motivation by setting goals and rewards.

[1286] "User device" means a device on which a driver checks their driving score and feedback, including, for example, a smartphone or tablet.

[1287] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. The specific configuration and operation of this system will be described below.

[1288] In-vehicle devices

[1289] The in-vehicle device is equipped with a speed sensor, a G-sensor, a GPS sensor, and an on-board camera. The device continuously collects data from these sensors. It also uses an emotion engine to analyze the driver's facial expressions and tone of voice to collect emotional data. For example, the vehicle's speed and acceleration are acquired every second while driving, and the driver's facial expressions captured by the on-board camera are analyzed using image recognition software.

[1290] Cloud Server

[1291] Data collected by the device is periodically sent in batches to a cloud server. The cloud server then integrates the received data and prepares it for analysis. The server then analyzes speed, acceleration, position, and emotional data to evaluate the driver's driving behavior and emotional state. This information is then used to calculate a driving score (DriveScore). By including emotional data in addition to sensor data, driving evaluation can be performed with greater accuracy than before.

[1292] Feedback and User Devices

[1293] The server registers the calculated driving score in the user's account and notifies the result via the user's device (e.g., smartphone). For example, a message such as "Your DriveScore is 85" is displayed on the user's device. In addition, real-time warnings and advice are also provided through the user's device. For example, while driving, the user can receive a voice message saying, "Take your time and slow down."

[1294] Gamification and rewards

[1295] The server sets goals and rewards for users based on their driving scores. When a certain score is reached, rewards such as digital badges and weekly rankings are provided. For example, a "Top Driver of the Week" badge is awarded, and a notification appears on the user's smartphone saying, "Congratulations! You're the Top Driver of the Week."

[1296] Insurance premium discounts

[1297] With the user's consent, the server provides the DriveScore to the insurance company. The insurance company applies a discount on insurance premiums based on the driving score. For example, if a user's DriveScore is 90 or higher, the server provides the data to the insurance company, and the user receives a 10% discount on insurance premiums.

[1298] This system can evaluate the driver's driving behavior and emotional state in detail, promoting safe driving and improving motivation. As a specific example, the following prompt sentences can be input into the generative AI model:

[1299] Example prompt sentence:

[1300] 1. "Describe a system that analyzes the driver's emotional state in real time based on data collected by an in-car device and provides appropriate advice."

[1301] 2. "Please explain in detail how the device will notify the user with a warning message if the user becomes impatient while driving."

[1302] 3. "Please explain what DriveScore is, how it is calculated, and what feedback it provides to users."

[1303] 4. "Can you give us a concrete example of how insurance discounts based on driving data work?"

[1304] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1305] Step 1:

[1306] The terminal collects data in real time from each sensor of the in-vehicle device (speed sensor, G sensor, GPS sensor, and in-vehicle camera). This collected data includes speed, acceleration, location, and the driver's facial expression and tone of voice. An emotion engine is used to analyze this data and generate driver emotion data. The input sensor data and facial expression data become real-time driving data and emotion data, and this data is stored in the terminal.

[1307] Step 2:

[1308] The device analyzes the collected sensor data and emotional data in real time. Specifically, it evaluates the driver's driving behavior and emotional state. This allows it to automatically generate appropriate advice or warnings if the driver appears to be impatient, tired, or stressed. Using the real-time data and emotional data as input, the device generates driving situation evaluation data and emotional evaluation data through analysis.

[1309] Step 3:

[1310] The device then provides the driver with advice or warning messages based on the analysis results. For example, if the driver is in a hurry and speeding up, the device will notify the user with a voice message saying, "Please stay calm and slow down." Based on the evaluation data input, a specific message is generated and notified to the user as voice or display output.

[1311] Step 4:

[1312] The device periodically transfers the collected data (speed, acceleration, position, and emotion data) to the cloud server in batches. Specifically, all data is packaged at regular intervals and sent to the cloud server via a communication line. The accumulated data as input is converted into a batch package and sent to the cloud server.

[1313] Step 5:

[1314] The server consolidates the data entered on the cloud and prepares it for analysis, specifically storing it in a literal database and performing any necessary preprocessing (e.g., data cleansing and format conversion). It receives batch data as input, stores and preprocesses it, and outputs it in an analyzable format.

[1315] Step 6:

[1316] The server evaluates driving behavior and emotional state based on the integrated data and calculates a driving score (DriveScore). Specifically, it analyzes the number of sudden braking attempts, fluctuations in acceleration, and the driver's emotional data to generate a quantified driving score. It evaluates the integrated data as input and outputs a driving score.

[1317] Step 7:

[1318] The server registers the calculated driving score to the user's account and provides it as visual information via the user's device. Specifically, it sends a notification to the user via a smartphone app saying, "Your DriveScore is 85." Using the driving score data as input, it generates a visual feedback message and outputs it to the user's device.

[1319] Step 8:

[1320] The server sets goals and rewards based on the driving score. For example, it can award a "Top Driver of the Week" badge to drivers who reach a certain score and notify them via a smartphone app. The server uses the driving score as input to set rewards and award badges, and notifies the user of the reward information as output.

[1321] Step 9:

[1322] After obtaining the user's consent, the server provides the DriveScore to the insurance company, which then applies a discount to the insurance premium. Specifically, if the user's DriveScore is 90 or higher, the server sends the user's data via API, and the insurance company notifies and applies the premium discount. The server sends the score data as input and receives the result of the premium discount application as output.

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1325] Conventional automated driving systems perform driving operations without taking into account the driver's emotional state, which means they are unable to effectively reduce driver stress and fatigue. Furthermore, the lack of technology to appropriately adjust automated driving parameters based on the driver's emotional state has prevented sufficient improvements in safety.

[1326] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting sensor data acquired by an in-vehicle device, means for analyzing the sensor data in real time and providing advice and warnings to the driver, means for analyzing the driver's emotional state and adjusting automated driving parameters based on the data, means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data, means for calculating a driving score based on the evaluation, and means for providing feedback about the driving score to the driver, notifying the insurance company, and applying a discount on insurance premiums. This makes it possible to recognize the driver's emotional state in real time and adjust appropriate automated driving parameters based on the driving score. Furthermore, by providing feedback and rewards to the driver, it is possible to encourage improvement in driving behavior and improve safety.

[1327] An "in-vehicle device" is a device that is installed in a vehicle and includes various sensors and cameras for acquiring driving data and environmental data.

[1328] "Sensor data" is a general term for data acquired by in-vehicle devices, such as speed, acceleration, GPS location information, the driver's facial expressions, and voice tone.

[1329] "Advice and Caution" refers to appropriate instructions or warnings provided to the driver based on sensor data and emotional state.

[1330] A "cloud server" is a remote server that stores collected data and performs integrated analysis and evaluation.

[1331] The "means for evaluating driving behavior" is a mechanism that analyzes the driver's driving behavior based on collected sensor data and emotional state, and generates an evaluation result.

[1332] The "driving score" is a numerical index that comprehensively evaluates driving behavior and emotional state.

[1333] "Feedback" is a means of providing information to drivers by notifying them of their driving score, advice, and warnings, encouraging them to improve their driving behavior.

[1334] "Insurance premium discounts" are premium reduction benefits applied by insurance companies based on driving scores.

[1335] "Emotional state" is information that indicates the psychological state of the driver, such as stress, fatigue, or impatience.

[1336] "Autonomous driving parameters" are control elements such as speed, following distance, and braking sensitivity that are set when the autonomous driving system performs driving operations.

[1337] "Game elements" are fun, competitive elements that set goals and rewards to encourage improved driving behavior.

[1338] "Smartphone app" means a software application for a mobile device used to provide visual driving scores and feedback to the driver.

[1339] The present invention is realized by a system that combines an in-vehicle device, a cloud server, a user device, and an emotion engine. Using the emotion engine, it is possible to recognize the driver's emotional state in real time and adjust autonomous driving parameters based on that. This system is configured by combining the following hardware and software.

[1340] Hardware used

[1341] 1. In-vehicle devices:

[1342] This includes an in-car camera, microphone, speed sensor, G-sensor, and GPS sensor.

[1343] Data is obtained from these sensors in real time.

[1344] 2. Cloud Server:

[1345] A remote server for storing and analyzing data.

[1346] Examples: AWS (Amazon Web Services), Google Cloud, etc.

[1347] 3. User Device:

[1348] Mobile devices such as smartphones and tablets.

[1349] Used to review data and provide feedback.

[1350] Software used

[1351] 1. Emotion Engine:

[1352] An AI model for analyzing facial expressions and tone of voice.

[1353] Examples: OpenAI, Google TensorFlow.

[1354] 2. Data analysis platform:

[1355] A tool for integrating and analyzing data.

[1356] For example: Apache Spark, Hadoop.

[1357] 3. Smartphone App:

[1358] An application that displays and notifies user data.

[1359] System Operation

[1360] Data collection

[1361] The in-vehicle device collects real-time sensor data such as speed, acceleration, GPS location, the driver's facial expressions and voice tone, allowing for accurate understanding of the driving situation and the driver's emotional state.

[1362] Real-time analysis and parameter adjustment

[1363] The terminal (in-vehicle device) analyzes the acquired data in real time. The emotion engine analyzes the driver's facial expressions and tone of voice to identify the driver's emotional state, such as stress, fatigue, or impatience. Based on this data, the autonomous driving system's parameters (speed, following distance, braking sensitivity, etc.) are adjusted appropriately.

[1364] Data transmission and cloud analysis

[1365] The device periodically transmits the acquired data to a cloud server, which then comprehensively analyzes the data and evaluates the driver's driving behavior and emotional state.

[1366] DriveScore calculation and feedback

[1367] The cloud server calculates a driving score (DriveScore) based on the integrated data. This score is the result of a comprehensive evaluation of driving behavior and emotional state. The calculated DriveScore is provided as feedback to the driver via the user's device.

[1368] Gamification and rewards

[1369] The cloud server sets goals and rewards for drivers based on their DriveScore. For example, achieving a certain score will earn them a digital badge or a spot at the top of the weekly rankings.

[1370] Insurance premium discounts

[1371] With the user's consent, the cloud server provides the DriveScore to insurance companies, who then apply discounts to insurance premiums based on the score.

[1372] Specific examples

[1373] For example, if the in-car device detects that the driver is in an "anger" state from their facial expression, the system will automatically slow down the vehicle and give a voice notification to the driver saying "Please relax." In addition, the collected data is sent to a cloud server every 30 minutes, and a driving score (DriveScore) is calculated through integrated analysis. Users can check this score on their smartphone app.

[1374] Prompt Sentence Examples

[1375] "Perform real-time emotion analysis using driver facial expression data and vehicle driving data, and generate Python code to set appropriate autonomous driving parameters."

[1376] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1377] Step 1: Data collection

[1378] The terminal (in-vehicle device) uses an in-vehicle camera, microphone, speed sensor, G sensor, and GPS sensor to capture the driver's facial expression, tone of voice, vehicle speed, acceleration, and location information in real time. The input data are facial expression data, voice data, speed data, G sensor data, and GPS data, which are sent to the analysis unit after initial processing. The output is initially processed emotion data and driving data.

[1379] Step 2: Sentiment Analysis

[1380] The device uses an emotion engine to analyze the collected facial expression data and voice data in real time. This analysis identifies the driver's emotional state (e.g., stress, fatigue, anger, etc.). The input is initially processed facial expression data and voice data, which are then fed into the emotion engine, which outputs the driver's emotional state.

[1381] Step 3: Real-time analysis and autonomous driving parameter adjustment

[1382] The device combines emotional state and driving data for real-time analysis and adjusts autonomous driving parameters (speed, following distance, braking sensitivity, etc.). The input is driving data and analyzed emotional data, and based on this data, it calculates and outputs autonomous driving control parameters. Specifically, if the driver's emotion is "anger," the device will slow down the vehicle and give a voice notification.

[1383] Step 4: Send data to the cloud server

[1384] The device periodically (e.g., every 30 minutes) sends the collected and preprocessed data to the cloud server in batch format. The input is the preprocessed driving data and emotion data, and the cloud server receives these data. The output is the integrated data sent to the cloud server.

[1385] Step 5: Integrated analysis and DriveScore calculation

[1386] The server (cloud server) comprehensively analyzes the transmitted data, evaluates driving behavior and emotional state, and calculates a DriveScore. The input is collected historical driving data and emotional data, which is used to apply statistical analysis and machine learning models. The output is the driver's DriveScore. A generative AI model is used to comprehensively evaluate emotions and driving behavior.

[1387] Step 6: Provide feedback

[1388] The server provides the calculated DriveScore as feedback to the user's smartphone app. The input is the calculated DriveScore, which is reported to the user device. The output is feedback information displayed on the user device, such as a "Top Driver of the Week" badge.

[1389] Step 7: Gamify and reward

[1390] The server sets goals and rewards based on the DriveScore and provides them to the user. The input is the DriveScore and user profile data, and based on this, the server sets goals and determines rewards. The output is a digital badge and weekly ranking display provided to the user.

[1391] Step 8: Provide data to insurance companies and apply discounts

[1392] With the user's consent, the server provides the insurance company with the user's DriveScore, and the insurance company applies a discount on the premium based on the score. The input is the user's DriveScore and consent information, which are sent to the insurance company. The output is a notification from the insurance company that the discount has been applied.

[1393] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1394] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1395] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1396] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1397] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1398] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1399] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1400] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1401] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1402] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1403] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1404] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1405] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1406] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1407] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1408] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1409] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1410] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1411] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1412] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1413] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1414] The following is further disclosed regarding the above embodiment.

[1415] (Claim 1)

[1416] means for collecting sensor data acquired by an in-vehicle device;

[1417] A means for analyzing the sensor data in real time and providing advice or warnings to the driver;

[1418] A means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data;

[1419] means for calculating a driving score based on the evaluation;

[1420] A means for feeding back the driving score to the driver and notifying the insurance company of the driving score and applying a discount on the insurance premium;

[1421] A system including:

[1422] (Claim 2)

[1423] 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements, and providing them based on the driving score.

[1424] (Claim 3)

[1425] 10. The system of claim 1, further comprising means for registering the driving score in the driver's account and providing it as visual information to the driver via a smartphone app.

[1426] "Example 1"

[1427] (Claim 1)

[1428] A means for collecting multiple types of sensor data acquired by devices installed in a vehicle;

[1429] a means for analyzing the plurality of sensor data in real time and providing advice or warnings to the driver;

[1430] a means for transmitting the collected data to a remote server via a network and evaluating the driving behavior based on the data;

[1431] means for calculating a driving score based on the evaluation;

[1432] A means for feeding back the driving score to the driver and notifying an external organization of the driving score and applying a discount on the fee;

[1433] A system including:

[1434] (Claim 2)

[1435] 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements, and providing them based on the driving score.

[1436] (Claim 3)

[1437] 10. The system of claim 1, further comprising means for registering the driving score in a driver's account and providing the driving score as visual information to the driver via a mobile device application.

[1438] "Application Example 1"

[1439] (Claim 1)

[1440] means for collecting sensor data acquired by an in-vehicle device;

[1441] A means for analyzing the sensor data in real time and providing advice or warnings to the driver;

[1442] A means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data;

[1443] means for calculating a driving score based on the evaluation;

[1444] A means for feeding back the driving score to the driver and notifying the insurance company of the driving score and applying a discount on the insurance premium;

[1445] a means for sending a warning message about the driving score to a smartphone app, smart glasses, or an in-car robot;

[1446] A system including:

[1447] (Claim 2)

[1448] 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements, and providing them based on the driving score.

[1449] (Claim 3)

[1450] 10. The system of claim 1, further comprising means for registering the driving score in the driver's account and providing it as visual information to the driver via a smartphone app.

[1451] (Claim 4)

[1452] 10. The system of claim 1, further comprising means for providing data to insurance companies using a secure API.

[1453] "Example 2: Combining Emotion Engines"

[1454] (Claim 1)

[1455] means for collecting sensor data and emotion data acquired by an in-vehicle device;

[1456] a means for analyzing the sensor data and emotion data in real time and providing advice or warnings to the driver;

[1457] a means for transmitting the collected data to a cloud server and evaluating the driving behavior and emotional state based on the data;

[1458] a means for calculating a driving score based on the evaluation and taking into account the emotional state;

[1459] A means for feeding back the driving score to the driver and notifying the insurance company of the driving score and applying a discount on the insurance premium;

[1460] A system including:

[1461] (Claim 2)

[1462] 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements, and providing them based on the driving score.

[1463] (Claim 3)

[1464] 10. The system of claim 1, further comprising means for registering the driving score to a driver's account and providing the driving score as visual information to the driver via a user device.

[1465] "Application example 2 when combining emotion engines"

[1466] (Claim 1)

[1467] means for collecting sensor data acquired by an in-vehicle device;

[1468] A means for analyzing the sensor data in real time and providing advice or warnings to the driver;

[1469] A means for analyzing the driver's emotional state and adjusting the parameters of the automated driving system based on that data;

[1470] A means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data;

[1471] means for calculating a driving score based on the evaluation;

[1472] A means for feeding back the driving score to the driver and notifying the insurance company of the driving score and applying a discount on the insurance premium;

[1473] A system including:

[1474] (Claim 2)

[1475] 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver using a driving evaluation method incorporating game elements, and providing them based on the driving score.

[1476] (Claim 3)

[1477] 10. The system of claim 1, further comprising means for registering the driving score in the driver's account and providing it as visual information to the driver via a smartphone app. [Explanation of symbols]

[1478] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting sensor data acquired by an in-vehicle device; A means for analyzing the sensor data in real time and providing advice or warnings to the driver; A means for transmitting the collected data to a cloud server and evaluating driving behavior based on the data; means for calculating a driving score based on the evaluation; A means for feeding back the driving score to the driver and notifying the insurance company of the driving score and applying a discount on the insurance premium; A system including:

2. 2. The system according to claim 1, further comprising means for setting goals and rewards for the driver by a driving evaluation method incorporating game elements, and providing them based on the driving score.

3. The system of claim 1 further comprising means for registering the driving score in the driver's account and providing the driving score as visual information to the driver via a smartphone app.

Images