System

Abstract

A system is provided.SOLUTION: An electronic control unit equipped with a generative artificial intelligence model, the system comprising: means for collecting real-time data from a plurality of subsystems of a vehicle; means for normalizing and filtering the collected data and inputting the data into the generative artificial intelligence model; and means for outputting a control command to each subsystem based on a result analyzed by the generative artificial intelligence model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The electronic control units (ECUs) installed in conventional vehicles independently control each subsystem, but lack the ability to integrate and analyze these functions, which prevents the vehicle's overall efficiency, safety, and user experience from being maximized. Furthermore, when users operate various vehicle functions, the operation tends to be cumbersome, which can distract them from driving. Furthermore, the lack of real-time safety monitoring and immediate response increases the risk of accidents and breakdowns. [Means for solving the problem]

[0005] The present invention provides an electronic control unit (ECU) that integrates a generative artificial intelligence (AI) model to comprehensively manage each vehicle subsystem, analyze data in real time, and generate optimal control commands. Specifically, the ECU includes a generative AI model and means for collecting, normalizing, and filtering real-time data from multiple vehicle subsystems. It also provides means for outputting control commands to each subsystem based on the analyzed data. It also includes means for analyzing user voice commands using a voice recognition engine and generating and transmitting appropriate control commands based on the analysis results. It also includes means for constantly monitoring real-time vehicle data and, if a safety-related abnormality is detected, for issuing a warning to the user and automatically issuing the necessary control commands. This significantly improves vehicle efficiency, safety, and user experience.

[0006] A "generative artificial intelligence model" is a model that uses artificial intelligence technology to analyze various vehicle data in real time and generate optimal control commands.

[0007] An "electronic control unit (ECU)" is a device that electronically controls and monitors each subsystem in a vehicle, such as the engine, brakes, navigation, and entertainment.

[0008] "Subsystem" refers to a system within a vehicle that has a specific function, such as an engine control system, braking system, navigation system, or entertainment system.

[0009] "Real-time data" refers to data that changes instantly and is obtained from each subsystem and sensor of the vehicle.

[0010] "Normalization" refers to the process of conforming collected data to certain standards and converting it into a form suitable for analysis.

[0011] "Filtering" refers to the process of removing noise and outliers from collected data to make it suitable for analysis.

[0012] A "control command" is a command that indicates a specific operation or setting to be performed for each subsystem based on the analysis results.

[0013] A "voice recognition engine" is a device or software that converts a user's speech into text and analyzes its content.

[0014] "Analysis results" refers to the information and instructions obtained by the generative artificial intelligence model when it analyzes input data.

[0015] "Real-time monitoring" refers to the process of constantly monitoring data obtained from each vehicle subsystem and sensor to immediately detect abnormalities or abnormal situations.

[0016] "Anomaly detection" refers to detecting deviations from the vehicle's normal state or dangerous conditions based on collected data and real-time monitoring. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention provides a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[0039] 1. Initialization and input data capture

[0040] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0041] Examples:

[0042] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0043] 2. Data analysis and model application

[0044] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0045] Examples:

[0046] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0047] 3. Control command output

[0048] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0049] Examples:

[0050] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0051] 4. Receiving and processing voice commands

[0052] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0053] Examples:

[0054] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0055] 5. Safety and Feedback Management

[0056] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[0057] Examples:

[0058] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0059] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] System startup

[0063] The server detects when the vehicle's power is turned on and starts the system.

[0064] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[0065] The server initializes each subsystem and puts it into a ready state.

[0066] Step 2:

[0067] Collecting data from sensors

[0068] The server collects a variety of data in real time, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0069] This data is sent to a server and stored in a database.

[0070] Step 3:

[0071] Data Preprocessing

[0072] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[0073] The server detects noise and outliers and performs filtering to remove them.

[0074] Step 4:

[0075] Application of generative artificial intelligence models

[0076] The server inputs the preprocessed data into a generative artificial intelligence model for analysis.

[0077] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[0078] Step 5:

[0079] Control command generation

[0080] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[0081] Control commands are provided to the engine control module, braking system, navigation system, etc.

[0082] Step 6:

[0083] Sending commands to each control system

[0084] The terminal transmits the generated control commands to each subsystem.

[0085] For example, it sends a command to the engine control module to adjust the fuel supply, and a command to the brake system to adjust the brake pressure.

[0086] Step 7:

[0087] Receiving voice instructions

[0088] The terminal collects the user's voice instructions through a microphone.

[0089] The voice instructions are sent to the server.

[0090] Step 8:

[0091] Analysis and processing of voice instructions

[0092] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[0093] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0094] Step 9:

[0095] Real-time monitoring

[0096] The server continuously monitors all real-time data and detects safety anomalies.

[0097] If an abnormality is detected, an alert is sent to the user.

[0098] Step 10:

[0099] Anomaly detection and response

[0100] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[0101] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[0102] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Modern vehicles have multiple subsystems, such as engine control and brake control, which operate independently, limiting the improvement of efficiency and safety. Furthermore, it is difficult to respond quickly to user instructions, and real-time data analysis and safety monitoring are not adequately performed. This creates a demand for improved user experience, such as improved fuel efficiency and safer driving support.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative AI model, means for outputting control commands to each subsystem based on the results of analysis by the generative AI model, means for automatically starting up the entire system and initializing all control modules when the vehicle is powered on, means for receiving user voice instructions, generating control commands based on the instructions and sending them to the subsystems, and means for monitoring the real-time data, detecting safety-related anomalies, and issuing necessary control commands, thereby enabling improvements in vehicle efficiency, safety, and user experience.

[0108] "Multiple subsystems" refer to multiple modules within a vehicle that operate independently and are responsible for specific functions.

[0109] "Real-time data" refers to data that instantly collects and analyzes current conditions and information.

[0110] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes collected data and generates optimal control commands.

[0111] "Data normalization" is the process of converting collected data into a consistent format that is easier to analyze.

[0112] "Data filtering" refers to the process of removing noise and outliers from collected data.

[0113] "Control commands" are operational instructions sent to each subsystem based on the analysis results.

[0114] "Automatic system startup" refers to the process of automatically starting all control modules as soon as the vehicle is powered on.

[0115] "Voice instructions" refer to operational instructions given by the user to the system using voice.

[0116] A "safety-related abnormality" is an abnormal condition that may pose a risk to the operation or performance of a vehicle.

[0117] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. Specific aspects of this system are described below.

[0118] 1. Initialization and input data capture

[0119] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0120] Examples:

[0121] The server starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed: 2000 rpm, brake pressure: 50 psi, fuel tank: 80% full or more, tire pressure: 2.5 bar, GPS location: latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0122] 2. Data analysis and model application

[0123] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0124] Examples:

[0125] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0126] 3. Control command output

[0127] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0128] Examples:

[0129] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0130] 4. Receiving and processing voice commands

[0131] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0132] Examples:

[0133] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0134] 5. Safety and Feedback Management

[0135] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[0136] Examples:

[0137] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0138] Example prompt sentence:

[0139] "Please explain the role of a generative AI model that collects data from sensors when a car's engine starts and generates control commands for the cooling fan based on that data."

[0140] Through the above process, the present invention is a system that can significantly improve vehicle efficiency, safety, and user experience.

[0141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0142] Step 1:

[0143] Initialization and input data capture

[0144] The server automatically starts the system and initializes all control systems as soon as the vehicle is turned on. Specifically, the server detects the power status via the vehicle's main ECU and starts each module, such as engine control, brake control, and entertainment system.

[0145] Input: Vehicle power on signal.

[0146] Output: System startup and initialization of each module completed.

[0147] Step 2:

[0148] Collecting data from sensors

[0149] The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0150] Inputs: engine start, braking, fuel usage, tire condition, GPS location, entertainment controls, etc.

[0151] Output: Real-time data collection.

[0152] What it does: The server continuously monitors and collects data on engine RPM, brake pressure, fuel tank level, tire pressure, GPS location, and entertainment system status.

[0153] Step 3:

[0154] Data normalization

[0155] The raw data acquired by the server is converted into a format that is easy to analyze. By converting into a unified data format, consistency in analysis is ensured.

[0156] Input: Raw data (engine RPM, brake pressure, etc.).

[0157] Output: Normalized data.

[0158] Specific operation: For example, if the temperature data is mixed in Celsius and Fahrenheit, the server converts it all to Celsius.

[0159] Step 4:

[0160] Data filtering

[0161] The server removes noise and outliers from the normalized data, improving the accuracy of the analysis.

[0162] Input: Normalized data.

[0163] Output: filtered clean data.

[0164] Specific operation: The server uses statistical methods and filtering algorithms to remove obviously abnormal values ​​(e.g., momentary sensor misses).

[0165] Step 5:

[0166] Data input to generative AI models

[0167] The server feeds the normalized and filtered data into a generative AI model.

[0168] Input: Filtered clean data.

[0169] Output: The data input to the AI ​​model.

[0170] Specific operation: The server inputs clean data from various sensors into the AI ​​model and performs real-time analysis.

[0171] Step 6:

[0172] Analysis and control command generation by AI model

[0173] The server analyzes the data using a generative AI model and generates optimal control commands.

[0174] Input: The data fed into the AI ​​model.

[0175] Output: The generated control commands.

[0176] Specific operations: For example, generating a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, or to adjust the engine speed to optimize fuel efficiency.

[0177] Step 7:

[0178] Output of control commands to each subsystem

[0179] The server generates control commands and sends them to each subsystem to adjust the vehicle's operation.

[0180] Input: The generated control command.

[0181] Output: Sends commands to each subsystem.

[0182] Specific actions: For example, it sends a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, and it sends a command to the brake system to adjust the required brake pressure to 50 psi.

[0183] Step 8:

[0184] Receiving voice instructions

[0185] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine.

[0186] Input: User's voice command.

[0187] Output: Parsed audio data.

[0188] Specific operation: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the device captures the voice and sends it to the server for analysis.

[0189] Step 9:

[0190] Generation of control commands based on voice instructions

[0191] The server uses a speech recognition engine to analyze the voice instructions and generate appropriate control commands based on them.

[0192] Input: Parsed audio data.

[0193] Output: Control command.

[0194] Specific operation: Based on the results of voice analysis, the server generates and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0195] Step 10:

[0196] Safety monitoring and abnormality response

[0197] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it alerts the user and issues necessary control commands.

[0198] Input: Real-time data.

[0199] Output: Warnings and control commands.

[0200] Specific operation: If the server detects a sudden drop in tire pressure, it will notify the user with an audible warning and issue a command to reduce engine power to safely slow down the vehicle.

[0201] Through these steps, the present invention can significantly improve vehicle efficiency, safety, and user experience.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] Rather than simply controlling each vehicle subsystem individually, it is necessary to manage and optimize the entire vehicle in an integrated manner to significantly improve vehicle safety, efficiency, and user experience. Furthermore, the ability to detect abnormalities in real time and automatically implement appropriate countermeasures is required. Current systems lack overall optimization because these functions are not implemented as a single integrated system.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for acquiring data from various sensors in the vehicle via the server through an API, means for analyzing the acquired data and providing optimal driving advice, and means for collecting voice instructions and sending control commands to each subsystem based on the analysis results, thereby enabling optimal management and control of the entire vehicle and realizing improved safety, efficiency, and user experience.

[0207] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes collected data and generates appropriate control commands.

[0208] An "electronic control unit" is a device that electronically controls and manages various systems in a vehicle.

[0209] "Real-time data" refers to data that instantly captures current information from sensors and other monitoring devices.

[0210] "Normalization" refers to the process of converting acquired data into a form suitable for analysis.

[0211] "Filtering" refers to the process of removing noise and outliers from data.

[0212] "Subsystem" refers to an independent partial system that performs various vehicle functions.

[0213] A "control command" is a command that includes instructions for performing a specific vehicle operation or function.

[0214] A "voice recognition engine" is a software or hardware system that analyzes a user's voice instructions and converts them into text data.

[0215] "Anomaly detection" refers to the process of detecting phenomena that cause a system to deviate from its normal operating range.

[0216] An "API" is an interface that allows a software component to be used by other software.

[0217] "Driving advice" refers to optimal driving instructions and suggestions provided to improve vehicle performance and safety.

[0218] "Data analysis" is the process of analyzing collected data in detail and extracting useful information and patterns.

[0219] "Voice instruction" refers to commands or requests given by a user to a system using voice.

[0220] The present invention is a system that uses an electronic control unit equipped with a generative artificial intelligence model to comprehensively manage and control multiple subsystems of a vehicle. Specific means for realizing this invention are described below.

[0221] Hardware and Software Configuration

[0222] Hardware:

[0223] 1. Sensors: Collect real-time data from multiple sensors installed in the vehicle (e.g., GPS sensors, tire pressure sensors, engine sensors, etc.).

[0224] 2. Electronic Control Unit (ECU): A central device that manages and controls each subsystem within a vehicle.

[0225] 3. Server: Hosts the generative AI model and performs data analysis.

[0226] software:

[0227] 1. Generative AI model: Analyzes collected data and generates appropriate control commands. Based on the analysis results, the server sends control commands to each subsystem.

[0228] 2. API endpoints: Used to retrieve data and send control commands.

[0229] 3. Speech recognition engine: Analyzes the user's voice instructions and generates control commands based on the results.

[0230] Specific examples of implementation

[0231] Data collection and analysis

[0232] The server collects real-time data from each vehicle subsystem (e.g., engine, brakes, etc.), normalizes and filters the data, and feeds it into a generative artificial intelligence model to generate appropriate control commands.

[0233] Server Roles and Operations

[0234] The server normalizes and filters the collected raw data. This process removes noise and outliers and converts the data into a format suitable for analysis. The normalized data is then input into a generative artificial intelligence model, which analyzes the real-time data. Based on the analysis results, the server generates optimal control commands for each subsystem.

[0235] Processing voice commands

[0236] The device collects voice instructions from the user and sends them to the server. The server uses a voice recognition engine to analyze the voice data and generate control commands based on the results. For example, if a user commands "set the air conditioner to 25 degrees," the server will send a command to the air conditioning system to set it to 25 degrees based on the analysis results.

[0237] Safety monitoring and response

[0238] The server constantly monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will warn the user and automatically issue necessary control commands. For example, if tire pressure suddenly drops, the server will sound an alarm to the user and issue a command to reduce engine power to safely slow down the vehicle.

[0239] Examples of prompt statements

[0240] An example of a prompt sent to a generative artificial intelligence model is "Input data: engine speed 2000 rpm, brake pressure 50 psi, fuel level 80%, tire pressure 2.5 bar, GPS location latitude 35.6895 degrees, longitude 139.6917 degrees, entertainment system ON."

[0241] An example of a prompt sentence for the speech recognition engine is a specific voice instruction such as "Set the air conditioner to 25 degrees."

[0242] The above configuration enables optimal management and control of the entire vehicle, improving safety, efficiency, and user experience.

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] As soon as the vehicle is powered on, the server automatically starts the system and collects real-time data from multiple vehicle subsystems. Data collected from sensors includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. These sensor data are taken as input data, and the raw data is stored in the server as the first output.

[0246] Step 2:

[0247] The server then performs normalization and filtering on the collected raw data. Normalization transforms the data into a form suitable for analysis, while filtering removes noise and outliers. The input to this process is the raw data collected in step 1, and the output is clean data ready for analysis.

[0248] Step 3:

[0249] The server inputs the normalized and filtered data into the generative artificial intelligence model, which analyzes the real-time data and generates optimal control commands. The input to this step is the normalized and filtered data from step 2, and the output is control commands for each subsystem.

[0250] Step 4:

[0251] The server sends the control commands obtained from the generative AI model to each subsystem. The input of this process is the control command obtained in step 3, and the output is the appropriate control execution for each subsystem. For example, it may send a command to the engine control module to adjust the engine speed, or it may instruct the braking system on the required brake pressure.

[0252] Step 5:

[0253] When a user inputs voice commands via a terminal, this voice data is sent to the server via a microphone. The server analyzes the voice data using a voice recognition engine and generates control commands based on the analysis results. The input is the user's voice data, and the output is control commands based on the analysis results and transmission to each subsystem.

[0254] Step 6:

[0255] When an abnormality occurs, the server continues to monitor real-time data and detects it. For example, if tire pressure suddenly drops, the server detects this data and immediately issues a warning to the user and a control command to ensure safety. The input of this step is the continuously collected real-time data, and the output is a warning to the user and the issuance of a corresponding control command.

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model and an emotion engine to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[0258] 1. Initialization and input data capture

[0259] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0260] Examples:

[0261] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0262] 2. Data analysis and model application

[0263] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0264] Examples:

[0265] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0266] 3. Control command output

[0267] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0268] Examples:

[0269] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0270] 4. Receiving and processing voice commands

[0271] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0272] Examples:

[0273] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0274] 5. Emotion Recognition by Emotion Engine

[0275] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0276] Examples:

[0277] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0278] 6. Emotion-based control commands

[0279] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to alleviate that stress.

[0280] Examples:

[0281] If the user is showing negative emotions, the server instructs the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0282] 7. Safety and Feedback Management

[0283] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0284] Examples:

[0285] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0286] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, more flexible control according to the user's emotional state is possible, creating a more comfortable and safe driving environment.

[0287] The processing flow will be explained below.

[0288] Step 1:

[0289] System startup

[0290] The server detects when the vehicle's power is turned on and starts the system.

[0291] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[0292] The server initializes each subsystem and puts it into a ready state.

[0293] Step 2:

[0294] Collecting data from sensors

[0295] The server collects real-time data such as engine RPM, brake pressure, fuel level, tire pressure, GPS location, entertainment system status, and user voice and facial expression data.

[0296] This data is sent to a server and stored in a database.

[0297] Step 3:

[0298] Data Preprocessing

[0299] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[0300] The server detects noise and outliers and performs filtering to remove them.

[0301] Step 4:

[0302] Application of generative artificial intelligence models

[0303] The server inputs the preprocessed data into a generative artificial intelligence model and performs data analysis.

[0304] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[0305] Step 5:

[0306] Control command generation

[0307] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[0308] For example, an engine control module may be provided with a command to adjust the fuel supply, and a braking system may be provided with a command to adjust the brake pressure.

[0309] Step 6:

[0310] Sending commands to each control system

[0311] The terminal transmits the generated control commands to each subsystem.

[0312] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0313] Step 7:

[0314] Receiving voice instructions

[0315] The terminal collects the user's voice instructions through a microphone.

[0316] The voice instructions are sent to the server and input into a voice recognition engine.

[0317] Step 8:

[0318] Analysis and processing of voice instructions

[0319] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[0320] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0321] For example, if a user commands "set the air conditioner to 25 degrees," the server recognizes this and sends a command to the air conditioning system to set it to 25 degrees.

[0322] Step 9:

[0323] Emotion recognition by emotion engine

[0324] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine.

[0325] The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0326] For example, if the emotion engine detects that the user is feeling stressed from the user's tone of voice or facial expression, it sends that information to the server.

[0327] Step 10:

[0328] Emotion-based control commands

[0329] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine.

[0330] For example, if a user is feeling stressed or angry, commands to alleviate that stress can be generated and sent to an entertainment system or air conditioning system.

[0331] It commands the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0332] Step 11:

[0333] Real-time monitoring

[0334] The server continuously monitors real-time data and detects safety anomalies.

[0335] If an abnormality is detected, an alert is sent to the user immediately.

[0336] Step 12:

[0337] Anomaly detection and response

[0338] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[0339] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[0340] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience. In addition, by combining it with an emotion engine, flexible control according to the user's emotional state becomes possible, realizing a more comfortable and safe driving environment.

[0341] Example 2

[0342] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0343] To improve safety and comfort when driving a vehicle, it is necessary to monitor the vehicle's status in real time and implement optimal control. However, conventional systems do not adequately collect and analyze accurate data, especially control that takes into account voice instructions and emotional states, and therefore do not fully achieve user satisfaction or safety.

[0344] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0345] In this invention, the server includes means for collecting information in real time from multiple subsystems of the vehicle, means for normalizing and filtering the collected information and inputting it into a generative artificial intelligence model, means for outputting optimal control commands to each subsystem based on the analysis results of the generative artificial intelligence model, means for collecting user voice and analyzing it using a voice recognition engine, means for analyzing user emotions using an emotion engine and generating control commands based on the results, and means for monitoring safety-related abnormalities in real time, issuing an alarm when an abnormality is detected, and automatically generating control commands, thereby enabling efficient vehicle operation, high safety, and providing a comfortable in-vehicle environment for the user.

[0346] "Vehicle subsystems" are groups of functional parts within a vehicle, such as the engine control module, braking system, fuel system, tire pressure sensors, GPS system, and entertainment system.

[0347] "Means for collecting information in real time" refers to a system that uses sensors and data collection devices to instantly grasp the current status of each subsystem while the vehicle is in operation.

[0348] "Normalization and filtering" refers to the process of processing the collected raw data, converting it into a format suitable for analysis, and removing noise and outliers to improve the accuracy of the data.

[0349] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates optimal control commands for a vehicle in real time.

[0350] A "voice recognition engine" is a program that analyzes a user's voice instructions, converts the content into text format, and analyzes it.

[0351] An "emotion engine" is an algorithm or program that analyzes a user's voice and facial expression data to recognize the user's emotional state.

[0352] A "control command" is an instruction or command issued to each subsystem of a vehicle, which adjusts each subsystem to operate appropriately.

[0353] "Means for monitoring safety-related abnormalities" refers to a mechanism for monitoring data from each vehicle subsystem in real time to detect abnormalities or dangerous conditions.

[0354] "Means for issuing warnings and automatically generating control commands" refers to the process of issuing a sound or display to warn the user when an abnormality is detected, and generating and issuing a control command to automatically respond.

[0355] A "comfortable in-car environment" is an environment that optimizes the temperature, music, lighting, etc. inside the car according to the user's emotional state and voice instructions.

[0356] The present invention provides a system for improving vehicle safety, efficiency, and user experience. The system utilizes a generative artificial intelligence model and an emotion engine to achieve comprehensive management and analysis. Specific embodiments for implementing the present invention are described below.

[0357] Initialization and input data capture

[0358] The server automatically starts the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0359] Examples:

[0360] The unit activates when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0361] Data analysis and model application

[0362] The server normalizes and filters the collected raw data and inputs it into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0363] Examples:

[0364] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0365] Control command output

[0366] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0367] Examples:

[0368] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0369] Receiving and processing voice commands

[0370] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0371] Examples:

[0372] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0373] Emotion recognition by emotion engine

[0374] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0375] Examples:

[0376] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0377] Emotion-based control command generation

[0378] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to reduce that stress.

[0379] Examples:

[0380] If the user is showing negative emotions, the server will instruct the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0381] Safety and Feedback Management

[0382] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0383] Examples:

[0384] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0385] Prompt Sentence Examples

[0386] Here are some example prompts you can provide to a generative AI model:

[0387] Prompt: "The vehicle's current position is 35.6895 degrees latitude and 139.6917 degrees longitude, the engine speed is 2000 RPM, and the fuel tank is 80% full. Analyze this data and generate optimal control commands."

[0388] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, flexible control according to the user's emotional state is possible, realizing a more comfortable and safe driving environment.

[0389] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0390] Step 1: System initialization and data collection

[0391] Input: Vehicle power-on signal

[0392] Output: Each control system initialized and the initial data collected.

[0393] Specific processing:

[0394] The server starts the system when the vehicle is powered on, initializing the generative AI model and all control systems, while collecting initial data from multiple subsystems such as the engine control module, braking system, and fuel system.

[0395] Example: When a vehicle engine starts, the server receives an initialization signal, which starts collecting data from all subsystems, such as the engine speed of 2000 rpm and the GPS location information of 35.6895 degrees latitude and 139.6917 degrees longitude.

[0396] Step 2: Normalize and filter the data

[0397] Input: Raw data collected after initialization

[0398] Output: Normalized and filtered data

[0399] Specific processing:

[0400] After initialization, the server normalizes and filters the collected raw data to prepare it for analysis. Normalization transforms the data into a consistent format, while filtering improves the accuracy of the data by removing noise and outliers.

[0401] Example: A server processes raw data, for example, converting engine RPM data to an appropriate scale and filtering out abnormally high RPMs or sudden pressure fluctuations.

[0402] Step 3: Input and analyze data into the generative AI model

[0403] Input: Normalized and filtered data

[0404] Output: Analysis results from the generative AI model

[0405] Specific processing:

[0406] The server inputs the normalized and filtered data into a generative AI model, which analyzes the data in real time. The model generates optimal control commands for each subsystem based on the collected data.

[0407] Example: If the engine temperature exceeds 90 degrees, the model generates a command to turn on the cooling fan. It also generates a command to adjust the engine speed appropriately for fuel efficiency.

[0408] Step 4: Output and execution of control commands

[0409] Input: Analysis results from a generative AI model

[0410] Output: Control commands to each subsystem

[0411] Specific processing:

[0412] Based on the analysis results obtained from the generated AI model, the server generates and immediately transmits optimal control commands for each subsystem, allowing each subsystem to adjust its operation.

[0413] Example: Send a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, or send a command to the braking system to adjust the required brake pressure to 50 psi.

[0414] Step 5: Receiving and processing voice commands

[0415] Input: Voice command from the user

[0416] Output: Control commands based on the results of voice analysis

[0417] Specific processing:

[0418] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the subsystem.

[0419] Example: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0420] Step 6: Emotion recognition and response with the emotion engine

[0421] Input: User voice and facial expression data

[0422] Output: Emotion analysis results and response instructions

[0423] Specific processing:

[0424] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes the data using an emotion engine. Based on the analysis results, it generates and sends appropriate control commands.

[0425] Example: An emotion engine detects from the user's tone of voice and facial expression that the user is stressed. The server uses this information to send a command to the entertainment system to play relaxing music.

[0426] Step 7: Safety monitoring and feedback

[0427] Input: Real-time data from each subsystem

[0428] Output: Safety warnings and control commands

[0429] Specific processing:

[0430] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0431] Example: If a sudden drop in tire pressure is detected, the system will sound an audible warning to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0432] Through these processing steps, the present invention provides a system that significantly improves vehicle efficiency, safety, and user experience. The combination of a generative AI model and an emotion engine enables flexible control according to the user's emotional state, creating a more comfortable and safe driving environment.

[0433] (Application example 2)

[0434] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0435] Conventional vehicle control systems often lack safety and comfort due to insufficient coordination between subsystems or flexible control based on user emotions. Furthermore, the efficiency of real-time data analysis and control command issuance is limited, making it difficult to achieve an advanced user experience.

[0436] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it to a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for collecting user emotion data, analyzing it using an emotion engine, and outputting control commands to each subsystem based on the analysis results, means for analyzing user voice instructions using a voice recognition engine, generating control commands based on the analysis results, and transmitting them to the vehicle subsystems, means for monitoring the vehicle's real-time data, issuing an alert if a safety abnormality is detected, and automatically issuing necessary control commands, and means for automatically adjusting the entertainment system and air conditioning system according to the user's emotional state. This enables improvements in vehicle efficiency, safety, and user experience.

[0437] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates optimal control commands based on collected data.

[0438] An "electronic control unit" is an electronic circuit board that comprehensively manages and controls each subsystem of a vehicle.

[0439] A "subsystem" is a separate functional unit in a vehicle, such as the engine, brakes, or entertainment system.

[0440] "Normalization" is the process of converting data into a form suitable for analysis.

[0441] "Filtering" is the process of cleaning data by removing noise and outliers.

[0442] An "emotion engine" is an algorithm that analyzes data such as the user's voice and facial expressions to recognize their emotional state.

[0443] A "voice recognition engine" is an algorithm that analyzes a user's voice, understands its content, and generates corresponding commands.

[0444] "Real-time data" refers to the latest data collected and analyzed at the current time.

[0445] A "control command" is an instruction to cause a vehicle subsystem to perform a specific operation.

[0446] "Emotional data" refers to data that indicates the user's emotional state, and includes voice, facial expression, heart rate, and the like.

[0447] An "entertainment system" is a system that provides content such as music and video in a vehicle.

[0448] An "air conditioning system" is a system that adjusts the temperature and humidity inside a vehicle.

[0449] "Abnormality detection" is the process of determining abnormalities when the vehicle's operation or state is different from normal.

[0450] This invention is a system that is equipped with a generative artificial intelligence model and an emotion engine, and that comprehensively manages and analyzes each subsystem of a vehicle in real time. Specific means and processes for implementing this invention will be described.

[0451] 1. System Initialization and Data Collection

[0452] When the engine of a vehicle equipped with an electronic control unit (ECU) starts, the server automatically starts the system and initializes all control systems. The server collects real-time information from multiple sensors in the vehicle, such as engine speed, brake pressure, fuel level, tire pressure, GPS location information, and entertainment system status.

[0453] Examples:

[0454] The unit activates when the vehicle's engine is started and collects information such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0455] 2. Data analysis and control command generation

[0456] The server normalizes and filters the collected raw data and inputs it into a generative AI model. The generative AI model analyzes the real-time data and generates optimal control commands. Based on the results, optimal control commands are output for each subsystem.

[0457] Examples:

[0458] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0459] 3. Collecting and analyzing user emotion data

[0460] The device collects user emotional data through a microphone and camera. The server uses an emotion engine to analyze this data and recognize the user's emotions in real time. Based on the results, it generates appropriate control commands for each subsystem.

[0461] Examples:

[0462] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0463] 4. Receiving and processing voice commands

[0464] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0465] Examples:

[0466] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0467] 5. Safety monitoring and control

[0468] The server continuously monitors real-time data to detect safety-related anomalies, immediately notifying the user with an alert and automatically issuing necessary control commands.

[0469] Examples:

[0470] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0471] The specific hardware and software used

[0472] Hardware:

[0473] Smartphone (camera, microphone)

[0474] Various sensors inside the vehicle (engine, brake, fuel, etc.)

[0475] software:

[0476] OpenCV (camera image processing)

[0477] Keras (emotion recognition model)

[0478] Pyautogui (system control)

[0479] Speech Recognition Engine

[0480] Example prompt sentence:

[0481] "Capture the user's face with a camera, and if the emotion recognition model detects 'sadness,' play relaxing music."

[0482] This will enable improvements in vehicle efficiency, safety and user experience.

[0483] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0484] Step 1:

[0485] The server automatically starts the system when the vehicle engine is started and collects real-time data from multiple sensors. Input data includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. The collected raw data is stored on the server.

[0486] Step 2:

[0487] The server normalizes and filters the collected raw data. During normalization, the data values ​​are standardized and converted into a format suitable for analysis. During filtering, noise and outliers are removed. The resulting clean data is fed into the generative artificial intelligence model.

[0488] Step 3:

[0489] The server inputs clean data into a generative AI model and analyzes the real-time data. The model generates optimal control commands as a result of the analysis and returns them to the server. For example, it generates commands to adjust engine speed or turn cooling fans on and off.

[0490] Step 4:

[0491] The server sends the generated control commands to each subsystem to adjust the vehicle's operation. Based on the input control commands, specific operations are performed on the engine control module and the brake system. For example, a command to reduce fuel supply is sent to the engine control module.

[0492] Step 5:

[0493] The device collects the user's emotional data, using a microphone to collect voice data and a camera to capture facial expression data, which is then sent to a server in real time.

[0494] Step 6:

[0495] The server inputs the collected voice and facial expression data into the emotion engine to analyze the user's emotional state. The emotion engine processes this data and determines the emotion the user is feeling. For example, it may detect that the user is feeling stressed.

[0496] Step 7:

[0497] Based on the analysis results of the emotion engine, the server generates and sends optimal control commands to each subsystem, such as a command to play relaxing music to the entertainment system or an adjustment command to the air conditioning system.

[0498] Step 8:

[0499] The device collects the user's voice instructions through a microphone and sends them to the server. The server then analyzes the voice instructions using a voice recognition engine and generates control commands based on the analysis results. For example, if the user says, "Set the air conditioner to 25 degrees," a command to set the temperature to 25 degrees is sent to the air conditioning system.

[0500] Step 9:

[0501] The server monitors real-time vehicle data and detects safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with a warning and automatically issues necessary control commands. For example, if tire pressure suddenly drops, a warning will sound and a command will be issued to reduce engine output to safely decelerate.

[0502] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0504] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0505] [Second embodiment]

[0506] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0507] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0508] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0509] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0510] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0511] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0512] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0513] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0514] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0515] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0516] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0517] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0518] The present invention provides a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[0519] 1. Initialization and input data capture

[0520] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0521] Examples:

[0522] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0523] 2. Data analysis and model application

[0524] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0525] Examples:

[0526] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0527] 3. Control command output

[0528] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0529] Examples:

[0530] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0531] 4. Receiving and processing voice commands

[0532] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0533] Examples:

[0534] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0535] 5. Safety and Feedback Management

[0536] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[0537] Examples:

[0538] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0539] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience.

[0540] The processing flow will be explained below.

[0541] Step 1:

[0542] System startup

[0543] The server detects when the vehicle's power is turned on and starts the system.

[0544] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[0545] The server initializes each subsystem and puts it into a ready state.

[0546] Step 2:

[0547] Collecting data from sensors

[0548] The server collects a variety of data in real time, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0549] This data is sent to a server and stored in a database.

[0550] Step 3:

[0551] Data Preprocessing

[0552] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[0553] The server detects noise and outliers and performs filtering to remove them.

[0554] Step 4:

[0555] Application of generative artificial intelligence models

[0556] The server inputs the preprocessed data into a generative artificial intelligence model for analysis.

[0557] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[0558] Step 5:

[0559] Control command generation

[0560] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[0561] Control commands are provided to the engine control module, braking system, navigation system, etc.

[0562] Step 6:

[0563] Sending commands to each control system

[0564] The terminal transmits the generated control commands to each subsystem.

[0565] For example, it sends a command to the engine control module to adjust the fuel supply, and a command to the brake system to adjust the brake pressure.

[0566] Step 7:

[0567] Receiving voice instructions

[0568] The terminal collects the user's voice instructions through a microphone.

[0569] The voice instructions are sent to the server.

[0570] Step 8:

[0571] Analysis and processing of voice instructions

[0572] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[0573] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0574] Step 9:

[0575] Real-time monitoring

[0576] The server continuously monitors all real-time data and detects safety anomalies.

[0577] If an abnormality is detected, an alert is sent to the user.

[0578] Step 10:

[0579] Anomaly detection and response

[0580] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[0581] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[0582] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience.

[0583] Example 1

[0584] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0585] Modern vehicles have multiple subsystems, such as engine control and brake control, which operate independently, limiting the improvement of efficiency and safety. Furthermore, it is difficult to respond quickly to user instructions, and real-time data analysis and safety monitoring are not adequately performed. This creates a demand for improved user experience, such as improved fuel efficiency and safer driving support.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0587] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative AI model, means for outputting control commands to each subsystem based on the results of analysis by the generative AI model, means for automatically starting up the entire system and initializing all control modules when the vehicle is powered on, means for receiving user voice instructions, generating control commands based on the instructions and sending them to the subsystems, and means for monitoring the real-time data, detecting safety-related anomalies, and issuing necessary control commands, thereby enabling improvements in vehicle efficiency, safety, and user experience.

[0588] "Multiple subsystems" refer to multiple modules within a vehicle that operate independently and are responsible for specific functions.

[0589] "Real-time data" refers to data that instantly collects and analyzes current conditions and information.

[0590] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes collected data and generates optimal control commands.

[0591] "Data normalization" is the process of converting collected data into a consistent format that is easier to analyze.

[0592] "Data filtering" refers to the process of removing noise and outliers from collected data.

[0593] "Control commands" are operational instructions sent to each subsystem based on the analysis results.

[0594] "Automatic system startup" refers to the process of automatically starting all control modules as soon as the vehicle is powered on.

[0595] "Voice instructions" refer to operational instructions given by the user to the system using voice.

[0596] A "safety-related abnormality" is an abnormal condition that may pose a risk to the operation or performance of a vehicle.

[0597] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. Specific aspects of this system are described below.

[0598] 1. Initialization and input data capture

[0599] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0600] Examples:

[0601] The server starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed: 2000 rpm, brake pressure: 50 psi, fuel tank: 80% full or more, tire pressure: 2.5 bar, GPS location: latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0602] 2. Data analysis and model application

[0603] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0604] Examples:

[0605] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0606] 3. Control command output

[0607] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0608] Examples:

[0609] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0610] 4. Receiving and processing voice commands

[0611] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0612] Examples:

[0613] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0614] 5. Safety and Feedback Management

[0615] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[0616] Examples:

[0617] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0618] Example prompt sentence:

[0619] "Please explain the role of a generative AI model that collects data from sensors when a car's engine starts and generates control commands for the cooling fan based on that data."

[0620] Through the above process, the present invention is a system that can significantly improve vehicle efficiency, safety, and user experience.

[0621] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0622] Step 1:

[0623] Initialization and input data capture

[0624] The server automatically starts the system and initializes all control systems as soon as the vehicle is turned on. Specifically, the server detects the power status via the vehicle's main ECU and starts each module, such as engine control, brake control, and entertainment system.

[0625] Input: Vehicle power on signal.

[0626] Output: System startup and initialization of each module completed.

[0627] Step 2:

[0628] Collecting data from sensors

[0629] The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0630] Inputs: engine start, braking, fuel usage, tire condition, GPS location, entertainment controls, etc.

[0631] Output: Real-time data collection.

[0632] What it does: The server continuously monitors and collects data on engine RPM, brake pressure, fuel tank level, tire pressure, GPS location, and entertainment system status.

[0633] Step 3:

[0634] Data normalization

[0635] The raw data acquired by the server is converted into a format that is easy to analyze. By converting into a unified data format, consistency in analysis is ensured.

[0636] Input: Raw data (engine RPM, brake pressure, etc.).

[0637] Output: Normalized data.

[0638] Specific operation: For example, if the temperature data is mixed in Celsius and Fahrenheit, the server converts it all to Celsius.

[0639] Step 4:

[0640] Data filtering

[0641] The server removes noise and outliers from the normalized data, improving the accuracy of the analysis.

[0642] Input: Normalized data.

[0643] Output: filtered clean data.

[0644] Specific operation: The server uses statistical methods and filtering algorithms to remove obviously abnormal values ​​(e.g., momentary sensor misses).

[0645] Step 5:

[0646] Data input to generative AI models

[0647] The server feeds the normalized and filtered data into a generative AI model.

[0648] Input: Filtered clean data.

[0649] Output: The data input to the AI ​​model.

[0650] Specific operation: The server inputs clean data from various sensors into the AI ​​model and performs real-time analysis.

[0651] Step 6:

[0652] Analysis and control command generation by AI model

[0653] The server analyzes the data using a generative AI model and generates optimal control commands.

[0654] Input: The data fed into the AI ​​model.

[0655] Output: The generated control commands.

[0656] Specific operations: For example, generating a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, or to adjust the engine speed to optimize fuel efficiency.

[0657] Step 7:

[0658] Output of control commands to each subsystem

[0659] The server generates control commands and sends them to each subsystem to adjust the vehicle's operation.

[0660] Input: The generated control command.

[0661] Output: Sends commands to each subsystem.

[0662] Specific actions: For example, it sends a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, and it sends a command to the brake system to adjust the required brake pressure to 50 psi.

[0663] Step 8:

[0664] Receiving voice instructions

[0665] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine.

[0666] Input: User's voice command.

[0667] Output: Parsed audio data.

[0668] Specific operation: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the device captures the voice and sends it to the server for analysis.

[0669] Step 9:

[0670] Generation of control commands based on voice instructions

[0671] The server uses a speech recognition engine to analyze the voice instructions and generate appropriate control commands based on them.

[0672] Input: Parsed audio data.

[0673] Output: Control command.

[0674] Specific operation: Based on the results of voice analysis, the server generates and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0675] Step 10:

[0676] Safety monitoring and abnormality response

[0677] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it alerts the user and issues necessary control commands.

[0678] Input: Real-time data.

[0679] Output: Warnings and control commands.

[0680] Specific operation: If the server detects a sudden drop in tire pressure, it will notify the user with an audible warning and issue a command to reduce engine power to safely slow down the vehicle.

[0681] Through these steps, the present invention can significantly improve vehicle efficiency, safety, and user experience.

[0682] (Application example 1)

[0683] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0684] Rather than simply controlling each vehicle subsystem individually, it is necessary to manage and optimize the entire vehicle in an integrated manner to significantly improve vehicle safety, efficiency, and user experience. Furthermore, the ability to detect abnormalities in real time and automatically implement appropriate countermeasures is required. Current systems lack overall optimization because these functions are not implemented as a single integrated system.

[0685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0686] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for acquiring data from various sensors in the vehicle via the server through an API, means for analyzing the acquired data and providing optimal driving advice, and means for collecting voice instructions and sending control commands to each subsystem based on the analysis results, thereby enabling optimal management and control of the entire vehicle and realizing improved safety, efficiency, and user experience.

[0687] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes collected data and generates appropriate control commands.

[0688] An "electronic control unit" is a device that electronically controls and manages various systems in a vehicle.

[0689] "Real-time data" refers to data that instantly captures current information from sensors and other monitoring devices.

[0690] "Normalization" refers to the process of converting acquired data into a form suitable for analysis.

[0691] "Filtering" refers to the process of removing noise and outliers from data.

[0692] "Subsystem" refers to an independent partial system that performs various vehicle functions.

[0693] A "control command" is a command that includes instructions for performing a specific vehicle operation or function.

[0694] A "voice recognition engine" is a software or hardware system that analyzes a user's voice instructions and converts them into text data.

[0695] "Anomaly detection" refers to the process of detecting phenomena that cause a system to deviate from its normal operating range.

[0696] An "API" is an interface that allows a software component to be used by other software.

[0697] "Driving advice" refers to optimal driving instructions and suggestions provided to improve vehicle performance and safety.

[0698] "Data analysis" is the process of analyzing collected data in detail and extracting useful information and patterns.

[0699] "Voice instruction" refers to commands or requests given by a user to a system using voice.

[0700] The present invention is a system that uses an electronic control unit equipped with a generative artificial intelligence model to comprehensively manage and control multiple subsystems of a vehicle. Specific means for realizing this invention are described below.

[0701] Hardware and Software Configuration

[0702] Hardware:

[0703] 1. Sensors: Collect real-time data from multiple sensors installed in the vehicle (e.g., GPS sensors, tire pressure sensors, engine sensors, etc.).

[0704] 2. Electronic Control Unit (ECU): A central device that manages and controls each subsystem within a vehicle.

[0705] 3. Server: Hosts the generative AI model and performs data analysis.

[0706] software:

[0707] 1. Generative AI model: Analyzes collected data and generates appropriate control commands. Based on the analysis results, the server sends control commands to each subsystem.

[0708] 2. API endpoints: Used to retrieve data and send control commands.

[0709] 3. Speech recognition engine: Analyzes the user's voice instructions and generates control commands based on the results.

[0710] Specific examples of implementation

[0711] Data collection and analysis

[0712] The server collects real-time data from each vehicle subsystem (e.g., engine, brakes, etc.), normalizes and filters the data, and feeds it into a generative artificial intelligence model to generate appropriate control commands.

[0713] Server Roles and Operations

[0714] The server normalizes and filters the collected raw data. This process removes noise and outliers and converts the data into a format suitable for analysis. The normalized data is then input into a generative artificial intelligence model, which analyzes the real-time data. Based on the analysis results, the server generates optimal control commands for each subsystem.

[0715] Processing voice commands

[0716] The device collects voice instructions from the user and sends them to the server. The server uses a voice recognition engine to analyze the voice data and generate control commands based on the results. For example, if a user commands "set the air conditioner to 25 degrees," the server will send a command to the air conditioning system to set it to 25 degrees based on the analysis results.

[0717] Safety monitoring and response

[0718] The server constantly monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will warn the user and automatically issue necessary control commands. For example, if tire pressure suddenly drops, the server will sound an alarm to the user and issue a command to reduce engine power to safely slow down the vehicle.

[0719] Examples of prompt statements

[0720] An example of a prompt sent to a generative artificial intelligence model is "Input data: engine speed 2000 rpm, brake pressure 50 psi, fuel level 80%, tire pressure 2.5 bar, GPS location latitude 35.6895 degrees, longitude 139.6917 degrees, entertainment system ON."

[0721] An example of a prompt sentence for the speech recognition engine is a specific voice instruction such as "Set the air conditioner to 25 degrees."

[0722] The above configuration enables optimal management and control of the entire vehicle, improving safety, efficiency, and user experience.

[0723] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0724] Step 1:

[0725] As soon as the vehicle is powered on, the server automatically starts the system and collects real-time data from multiple vehicle subsystems. Data collected from sensors includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. These sensor data are taken as input data, and the raw data is stored in the server as the first output.

[0726] Step 2:

[0727] The server then performs normalization and filtering on the collected raw data. Normalization transforms the data into a form suitable for analysis, while filtering removes noise and outliers. The input to this process is the raw data collected in step 1, and the output is clean data ready for analysis.

[0728] Step 3:

[0729] The server inputs the normalized and filtered data into the generative artificial intelligence model, which analyzes the real-time data and generates optimal control commands. The input to this step is the normalized and filtered data from step 2, and the output is control commands for each subsystem.

[0730] Step 4:

[0731] The server sends the control commands obtained from the generative AI model to each subsystem. The input of this process is the control command obtained in step 3, and the output is the appropriate control execution for each subsystem. For example, it may send a command to the engine control module to adjust the engine speed, or it may instruct the braking system on the required brake pressure.

[0732] Step 5:

[0733] When a user inputs voice commands via a terminal, this voice data is sent to the server via a microphone. The server analyzes the voice data using a voice recognition engine and generates control commands based on the analysis results. The input is the user's voice data, and the output is control commands based on the analysis results and transmission to each subsystem.

[0734] Step 6:

[0735] When an abnormality occurs, the server continues to monitor real-time data and detects it. For example, if tire pressure suddenly drops, the server detects this data and immediately issues a warning to the user and a control command to ensure safety. The input of this step is the continuously collected real-time data, and the output is a warning to the user and the issuance of a corresponding control command.

[0736] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0737] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model and an emotion engine to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[0738] 1. Initialization and input data capture

[0739] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0740] Examples:

[0741] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0742] 2. Data analysis and model application

[0743] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0744] Examples:

[0745] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0746] 3. Control command output

[0747] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0748] Examples:

[0749] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0750] 4. Receiving and processing voice commands

[0751] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0752] Examples:

[0753] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[0754] 5. Emotion Recognition by Emotion Engine

[0755] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0756] Examples:

[0757] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0758] 6. Emotion-based control commands

[0759] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to alleviate that stress.

[0760] Examples:

[0761] If the user is showing negative emotions, the server instructs the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0762] 7. Safety and Feedback Management

[0763] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0764] Examples:

[0765] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[0766] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, more flexible control according to the user's emotional state is possible, creating a more comfortable and safe driving environment.

[0767] The processing flow will be explained below.

[0768] Step 1:

[0769] System startup

[0770] The server detects when the vehicle's power is turned on and starts the system.

[0771] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[0772] The server initializes each subsystem and puts it into a ready state.

[0773] Step 2:

[0774] Collecting data from sensors

[0775] The server collects real-time data such as engine RPM, brake pressure, fuel level, tire pressure, GPS location, entertainment system status, and user voice and facial expression data.

[0776] This data is sent to a server and stored in a database.

[0777] Step 3:

[0778] Data Preprocessing

[0779] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[0780] The server detects noise and outliers and performs filtering to remove them.

[0781] Step 4:

[0782] Application of generative artificial intelligence models

[0783] The server inputs the preprocessed data into a generative artificial intelligence model and performs data analysis.

[0784] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[0785] Step 5:

[0786] Control command generation

[0787] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[0788] For example, an engine control module may be provided with a command to adjust the fuel supply, and a braking system may be provided with a command to adjust the brake pressure.

[0789] Step 6:

[0790] Sending commands to each control system

[0791] The terminal transmits the generated control commands to each subsystem.

[0792] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0793] Step 7:

[0794] Receiving voice instructions

[0795] The terminal collects the user's voice instructions through a microphone.

[0796] The voice instructions are sent to the server and input into a voice recognition engine.

[0797] Step 8:

[0798] Analysis and processing of voice instructions

[0799] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[0800] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0801] For example, if a user commands "set the air conditioner to 25 degrees," the server recognizes this and sends a command to the air conditioning system to set it to 25 degrees.

[0802] Step 9:

[0803] Emotion recognition by emotion engine

[0804] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine.

[0805] The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0806] For example, if the emotion engine detects that the user is feeling stressed from the user's tone of voice or facial expression, it sends that information to the server.

[0807] Step 10:

[0808] Emotion-based control commands

[0809] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine.

[0810] For example, if a user is feeling stressed or angry, commands to alleviate that stress can be generated and sent to an entertainment system or air conditioning system.

[0811] It commands the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0812] Step 11:

[0813] Real-time monitoring

[0814] The server continuously monitors real-time data and detects safety anomalies.

[0815] If an abnormality is detected, an alert is sent to the user immediately.

[0816] Step 12:

[0817] Anomaly detection and response

[0818] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[0819] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[0820] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience. In addition, by combining it with an emotion engine, flexible control according to the user's emotional state becomes possible, realizing a more comfortable and safe driving environment.

[0821] Example 2

[0822] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0823] To improve safety and comfort when driving a vehicle, it is necessary to monitor the vehicle's status in real time and implement optimal control. However, conventional systems do not adequately collect and analyze accurate data, especially control that takes into account voice instructions and emotional states, and therefore do not fully achieve user satisfaction or safety.

[0824] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0825] In this invention, the server includes means for collecting information in real time from multiple subsystems of the vehicle, means for normalizing and filtering the collected information and inputting it into a generative artificial intelligence model, means for outputting optimal control commands to each subsystem based on the analysis results of the generative artificial intelligence model, means for collecting user voice and analyzing it using a voice recognition engine, means for analyzing user emotions using an emotion engine and generating control commands based on the results, and means for monitoring safety-related abnormalities in real time, issuing an alarm when an abnormality is detected, and automatically generating control commands, thereby enabling efficient vehicle operation, high safety, and providing a comfortable in-vehicle environment for the user.

[0826] "Vehicle subsystems" are groups of functional parts within a vehicle, such as the engine control module, braking system, fuel system, tire pressure sensors, GPS system, and entertainment system.

[0827] "Means for collecting information in real time" refers to a system that uses sensors and data collection devices to instantly grasp the current status of each subsystem while the vehicle is in operation.

[0828] "Normalization and filtering" refers to the process of processing the collected raw data, converting it into a format suitable for analysis, and removing noise and outliers to improve the accuracy of the data.

[0829] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates optimal control commands for a vehicle in real time.

[0830] A "voice recognition engine" is a program that analyzes a user's voice instructions, converts the content into text format, and analyzes it.

[0831] An "emotion engine" is an algorithm or program that analyzes a user's voice and facial expression data to recognize the user's emotional state.

[0832] A "control command" is an instruction or command issued to each subsystem of a vehicle, which adjusts each subsystem to operate appropriately.

[0833] "Means for monitoring safety-related abnormalities" refers to a mechanism for monitoring data from each vehicle subsystem in real time to detect abnormalities or dangerous conditions.

[0834] "Means for issuing warnings and automatically generating control commands" refers to the process of issuing a sound or display to warn the user when an abnormality is detected, and generating and issuing a control command to automatically respond.

[0835] A "comfortable in-car environment" is an environment that optimizes the temperature, music, lighting, etc. inside the car according to the user's emotional state and voice instructions.

[0836] The present invention provides a system for improving vehicle safety, efficiency, and user experience. The system utilizes a generative artificial intelligence model and an emotion engine to achieve comprehensive management and analysis. Specific embodiments for implementing the present invention are described below.

[0837] Initialization and input data capture

[0838] The server automatically starts the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[0839] Examples:

[0840] The unit activates when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0841] Data analysis and model application

[0842] The server normalizes and filters the collected raw data and inputs it into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[0843] Examples:

[0844] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0845] Control command output

[0846] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[0847] Examples:

[0848] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[0849] Receiving and processing voice commands

[0850] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0851] Examples:

[0852] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0853] Emotion recognition by emotion engine

[0854] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[0855] Examples:

[0856] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0857] Emotion-based control command generation

[0858] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to reduce that stress.

[0859] Examples:

[0860] If the user is showing negative emotions, the server will instruct the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[0861] Safety and Feedback Management

[0862] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0863] Examples:

[0864] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0865] Prompt Sentence Examples

[0866] Here are some example prompts you can provide to a generative AI model:

[0867] Prompt: "The vehicle's current position is 35.6895 degrees latitude and 139.6917 degrees longitude, the engine speed is 2000 RPM, and the fuel tank is 80% full. Analyze this data and generate optimal control commands."

[0868] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, flexible control according to the user's emotional state is possible, realizing a more comfortable and safe driving environment.

[0869] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0870] Step 1: System initialization and data collection

[0871] Input: Vehicle power-on signal

[0872] Output: Each control system initialized and the initial data collected.

[0873] Specific processing:

[0874] The server starts the system when the vehicle is powered on, initializing the generative AI model and all control systems, while collecting initial data from multiple subsystems such as the engine control module, braking system, and fuel system.

[0875] Example: When a vehicle engine starts, the server receives an initialization signal, which starts collecting data from all subsystems, such as the engine speed of 2000 rpm and the GPS location information of 35.6895 degrees latitude and 139.6917 degrees longitude.

[0876] Step 2: Normalize and filter the data

[0877] Input: Raw data collected after initialization

[0878] Output: Normalized and filtered data

[0879] Specific processing:

[0880] After initialization, the server normalizes and filters the collected raw data to prepare it for analysis. Normalization transforms the data into a consistent format, while filtering improves the accuracy of the data by removing noise and outliers.

[0881] Example: A server processes raw data, for example, converting engine RPM data to an appropriate scale and filtering out abnormally high RPMs or sudden pressure fluctuations.

[0882] Step 3: Input and analyze data into the generative AI model

[0883] Input: Normalized and filtered data

[0884] Output: Analysis results from the generative AI model

[0885] Specific processing:

[0886] The server inputs the normalized and filtered data into a generative AI model, which analyzes the data in real time. The model generates optimal control commands for each subsystem based on the collected data.

[0887] Example: If the engine temperature exceeds 90 degrees, the model generates a command to turn on the cooling fan. It also generates a command to adjust the engine speed appropriately for fuel efficiency.

[0888] Step 4: Output and execution of control commands

[0889] Input: Analysis results from a generative AI model

[0890] Output: Control commands to each subsystem

[0891] Specific processing:

[0892] Based on the analysis results obtained from the generated AI model, the server generates and immediately transmits optimal control commands for each subsystem, allowing each subsystem to adjust its operation.

[0893] Example: Send a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, or send a command to the braking system to adjust the required brake pressure to 50 psi.

[0894] Step 5: Receiving and processing voice commands

[0895] Input: Voice command from the user

[0896] Output: Control commands based on the results of voice analysis

[0897] Specific processing:

[0898] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the subsystem.

[0899] Example: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0900] Step 6: Emotion recognition and response with the emotion engine

[0901] Input: User voice and facial expression data

[0902] Output: Emotion analysis results and response instructions

[0903] Specific processing:

[0904] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes the data using an emotion engine. Based on the analysis results, it generates and sends appropriate control commands.

[0905] Example: An emotion engine detects from the user's tone of voice and facial expression that the user is stressed. The server uses this information to send a command to the entertainment system to play relaxing music.

[0906] Step 7: Safety monitoring and feedback

[0907] Input: Real-time data from each subsystem

[0908] Output: Safety warnings and control commands

[0909] Specific processing:

[0910] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[0911] Example: If a sudden drop in tire pressure is detected, the system will sound an audible warning to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0912] Through these processing steps, the present invention provides a system that significantly improves vehicle efficiency, safety, and user experience. The combination of a generative AI model and an emotion engine enables flexible control according to the user's emotional state, creating a more comfortable and safe driving environment.

[0913] (Application example 2)

[0914] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0915] Conventional vehicle control systems often lack safety and comfort due to insufficient coordination between subsystems or flexible control based on user emotions. Furthermore, the efficiency of real-time data analysis and control command issuance is limited, making it difficult to achieve an advanced user experience.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it to a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for collecting user emotion data, analyzing it using an emotion engine, and outputting control commands to each subsystem based on the analysis results, means for analyzing user voice instructions using a voice recognition engine, generating control commands based on the analysis results, and transmitting them to the vehicle subsystems, means for monitoring the vehicle's real-time data, issuing an alert if a safety abnormality is detected, and automatically issuing necessary control commands, and means for automatically adjusting the entertainment system and air conditioning system according to the user's emotional state. This enables improvements in vehicle efficiency, safety, and user experience.

[0917] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates optimal control commands based on collected data.

[0918] An "electronic control unit" is an electronic circuit board that comprehensively manages and controls each subsystem of a vehicle.

[0919] A "subsystem" is a separate functional unit in a vehicle, such as the engine, brakes, or entertainment system.

[0920] "Normalization" is the process of converting data into a form suitable for analysis.

[0921] "Filtering" is the process of cleaning data by removing noise and outliers.

[0922] An "emotion engine" is an algorithm that analyzes data such as the user's voice and facial expressions to recognize their emotional state.

[0923] A "voice recognition engine" is an algorithm that analyzes a user's voice, understands its content, and generates corresponding commands.

[0924] "Real-time data" refers to the latest data collected and analyzed at the current time.

[0925] A "control command" is an instruction to cause a vehicle subsystem to perform a specific operation.

[0926] "Emotional data" refers to data that indicates the user's emotional state, and includes voice, facial expression, heart rate, and the like.

[0927] An "entertainment system" is a system that provides content such as music and video in a vehicle.

[0928] An "air conditioning system" is a system that adjusts the temperature and humidity inside a vehicle.

[0929] "Abnormality detection" is the process of determining abnormalities when the vehicle's operation or state is different from normal.

[0930] This invention is a system that is equipped with a generative artificial intelligence model and an emotion engine, and that comprehensively manages and analyzes each subsystem of a vehicle in real time. Specific means and processes for implementing this invention will be described.

[0931] 1. System Initialization and Data Collection

[0932] When the engine of a vehicle equipped with an electronic control unit (ECU) starts, the server automatically starts the system and initializes all control systems. The server collects real-time information from multiple sensors in the vehicle, such as engine speed, brake pressure, fuel level, tire pressure, GPS location information, and entertainment system status.

[0933] Examples:

[0934] The unit activates when the vehicle's engine is started and collects information such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[0935] 2. Data analysis and control command generation

[0936] The server normalizes and filters the collected raw data and inputs it into a generative AI model. The generative AI model analyzes the real-time data and generates optimal control commands. Based on the results, optimal control commands are output for each subsystem.

[0937] Examples:

[0938] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[0939] 3. Collecting and analyzing user emotion data

[0940] The device collects user emotional data through a microphone and camera. The server uses an emotion engine to analyze this data and recognize the user's emotions in real time. Based on the results, it generates appropriate control commands for each subsystem.

[0941] Examples:

[0942] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[0943] 4. Receiving and processing voice commands

[0944] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[0945] Examples:

[0946] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[0947] 5. Safety monitoring and control

[0948] The server continuously monitors real-time data to detect safety-related anomalies, immediately notifying the user with an alert and automatically issuing necessary control commands.

[0949] Examples:

[0950] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[0951] The specific hardware and software used

[0952] Hardware:

[0953] Smartphone (camera, microphone)

[0954] Various sensors inside the vehicle (engine, brake, fuel, etc.)

[0955] software:

[0956] OpenCV (camera image processing)

[0957] Keras (emotion recognition model)

[0958] Pyautogui (system control)

[0959] Speech Recognition Engine

[0960] Example prompt sentence:

[0961] "Capture the user's face with a camera, and if the emotion recognition model detects 'sadness,' play relaxing music."

[0962] This will enable improvements in vehicle efficiency, safety and user experience.

[0963] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0964] Step 1:

[0965] The server automatically starts the system when the vehicle engine is started and collects real-time data from multiple sensors. Input data includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. The collected raw data is stored on the server.

[0966] Step 2:

[0967] The server normalizes and filters the collected raw data. During normalization, the data values ​​are standardized and converted into a format suitable for analysis. During filtering, noise and outliers are removed. The resulting clean data is fed into the generative artificial intelligence model.

[0968] Step 3:

[0969] The server inputs clean data into a generative AI model and analyzes the real-time data. The model generates optimal control commands as a result of the analysis and returns them to the server. For example, it generates commands to adjust engine speed or turn cooling fans on and off.

[0970] Step 4:

[0971] The server sends the generated control commands to each subsystem to adjust the vehicle's operation. Based on the input control commands, specific operations are performed on the engine control module and the brake system. For example, a command to reduce fuel supply is sent to the engine control module.

[0972] Step 5:

[0973] The device collects the user's emotional data, using a microphone to collect voice data and a camera to capture facial expression data, which is then sent to a server in real time.

[0974] Step 6:

[0975] The server inputs the collected voice and facial expression data into the emotion engine to analyze the user's emotional state. The emotion engine processes this data and determines the emotion the user is feeling. For example, it may detect that the user is feeling stressed.

[0976] Step 7:

[0977] Based on the analysis results of the emotion engine, the server generates and sends optimal control commands to each subsystem, such as a command to play relaxing music to the entertainment system or an adjustment command to the air conditioning system.

[0978] Step 8:

[0979] The device collects the user's voice instructions through a microphone and sends them to the server. The server then analyzes the voice instructions using a voice recognition engine and generates control commands based on the analysis results. For example, if the user says, "Set the air conditioner to 25 degrees," a command to set the temperature to 25 degrees is sent to the air conditioning system.

[0980] Step 9:

[0981] The server monitors real-time vehicle data and detects safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with a warning and automatically issues necessary control commands. For example, if tire pressure suddenly drops, a warning will sound and a command will be issued to reduce engine output to safely decelerate.

[0982] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0983] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0984] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0985] [Third embodiment]

[0986] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0987] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0988] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0989] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0990] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0991] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0992] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0993] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0994] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0995] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0996] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0997] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0998] The present invention provides a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[0999] 1. Initialization and input data capture

[1000] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1001] Examples:

[1002] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1003] 2. Data analysis and model application

[1004] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1005] Examples:

[1006] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1007] 3. Control command output

[1008] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1009] Examples:

[1010] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1011] 4. Receiving and processing voice commands

[1012] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1013] Examples:

[1014] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1015] 5. Safety and Feedback Management

[1016] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[1017] Examples:

[1018] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1019] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience.

[1020] The processing flow will be explained below.

[1021] Step 1:

[1022] System startup

[1023] The server detects when the vehicle's power is turned on and starts the system.

[1024] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[1025] The server initializes each subsystem and puts it into a ready state.

[1026] Step 2:

[1027] Collecting data from sensors

[1028] The server collects a variety of data in real time, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1029] This data is sent to a server and stored in a database.

[1030] Step 3:

[1031] Data Preprocessing

[1032] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[1033] The server detects noise and outliers and performs filtering to remove them.

[1034] Step 4:

[1035] Application of generative artificial intelligence models

[1036] The server inputs the preprocessed data into a generative artificial intelligence model for analysis.

[1037] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[1038] Step 5:

[1039] Control command generation

[1040] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[1041] Control commands are provided to the engine control module, braking system, navigation system, etc.

[1042] Step 6:

[1043] Sending commands to each control system

[1044] The terminal transmits the generated control commands to each subsystem.

[1045] For example, it sends a command to the engine control module to adjust the fuel supply, and a command to the brake system to adjust the brake pressure.

[1046] Step 7:

[1047] Receiving voice instructions

[1048] The terminal collects the user's voice instructions through a microphone.

[1049] The voice instructions are sent to the server.

[1050] Step 8:

[1051] Analysis and processing of voice instructions

[1052] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[1053] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1054] Step 9:

[1055] Real-time monitoring

[1056] The server continuously monitors all real-time data and detects safety anomalies.

[1057] If an abnormality is detected, an alert is sent to the user.

[1058] Step 10:

[1059] Anomaly detection and response

[1060] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[1061] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[1062] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience.

[1063] Example 1

[1064] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] Modern vehicles have multiple subsystems, such as engine control and brake control, which operate independently, limiting the improvement of efficiency and safety. Furthermore, it is difficult to respond quickly to user instructions, and real-time data analysis and safety monitoring are not adequately performed. This creates a demand for improved user experience, such as improved fuel efficiency and safer driving support.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1067] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative AI model, means for outputting control commands to each subsystem based on the results of analysis by the generative AI model, means for automatically starting up the entire system and initializing all control modules when the vehicle is powered on, means for receiving user voice instructions, generating control commands based on the instructions and sending them to the subsystems, and means for monitoring the real-time data, detecting safety-related anomalies, and issuing necessary control commands, thereby enabling improvements in vehicle efficiency, safety, and user experience.

[1068] "Multiple subsystems" refer to multiple modules within a vehicle that operate independently and are responsible for specific functions.

[1069] "Real-time data" refers to data that instantly collects and analyzes current conditions and information.

[1070] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes collected data and generates optimal control commands.

[1071] "Data normalization" is the process of converting collected data into a consistent format that is easier to analyze.

[1072] "Data filtering" refers to the process of removing noise and outliers from collected data.

[1073] "Control commands" are operational instructions sent to each subsystem based on the analysis results.

[1074] "Automatic system startup" refers to the process of automatically starting all control modules as soon as the vehicle is powered on.

[1075] "Voice instructions" refer to operational instructions given by the user to the system using voice.

[1076] A "safety-related abnormality" is an abnormal condition that may pose a risk to the operation or performance of a vehicle.

[1077] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. Specific aspects of this system are described below.

[1078] 1. Initialization and input data capture

[1079] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1080] Examples:

[1081] The server starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed: 2000 rpm, brake pressure: 50 psi, fuel tank: 80% full or more, tire pressure: 2.5 bar, GPS location: latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1082] 2. Data analysis and model application

[1083] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1084] Examples:

[1085] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1086] 3. Control command output

[1087] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1088] Examples:

[1089] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1090] 4. Receiving and processing voice commands

[1091] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1092] Examples:

[1093] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1094] 5. Safety and Feedback Management

[1095] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[1096] Examples:

[1097] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1098] Example prompt sentence:

[1099] "Please explain the role of a generative AI model that collects data from sensors when a car's engine starts and generates control commands for the cooling fan based on that data."

[1100] Through the above process, the present invention is a system that can significantly improve vehicle efficiency, safety, and user experience.

[1101] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1102] Step 1:

[1103] Initialization and input data capture

[1104] The server automatically starts the system and initializes all control systems as soon as the vehicle is turned on. Specifically, the server detects the power status via the vehicle's main ECU and starts each module, such as engine control, brake control, and entertainment system.

[1105] Input: Vehicle power on signal.

[1106] Output: System startup and initialization of each module completed.

[1107] Step 2:

[1108] Collecting data from sensors

[1109] The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1110] Inputs: engine start, braking, fuel usage, tire condition, GPS location, entertainment controls, etc.

[1111] Output: Real-time data collection.

[1112] What it does: The server continuously monitors and collects data on engine RPM, brake pressure, fuel tank level, tire pressure, GPS location, and entertainment system status.

[1113] Step 3:

[1114] Data normalization

[1115] The raw data acquired by the server is converted into a format that is easy to analyze. By converting into a unified data format, consistency in analysis is ensured.

[1116] Input: Raw data (engine RPM, brake pressure, etc.).

[1117] Output: Normalized data.

[1118] Specific operation: For example, if the temperature data is mixed in Celsius and Fahrenheit, the server converts it all to Celsius.

[1119] Step 4:

[1120] Data filtering

[1121] The server removes noise and outliers from the normalized data, improving the accuracy of the analysis.

[1122] Input: Normalized data.

[1123] Output: filtered clean data.

[1124] Specific operation: The server uses statistical methods and filtering algorithms to remove obviously abnormal values ​​(e.g., momentary sensor misses).

[1125] Step 5:

[1126] Data input to generative AI models

[1127] The server feeds the normalized and filtered data into a generative AI model.

[1128] Input: Filtered clean data.

[1129] Output: The data input to the AI ​​model.

[1130] Specific operation: The server inputs clean data from various sensors into the AI ​​model and performs real-time analysis.

[1131] Step 6:

[1132] Analysis and control command generation by AI model

[1133] The server analyzes the data using a generative AI model and generates optimal control commands.

[1134] Input: The data fed into the AI ​​model.

[1135] Output: The generated control commands.

[1136] Specific operations: For example, generating a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, or to adjust the engine speed to optimize fuel efficiency.

[1137] Step 7:

[1138] Output of control commands to each subsystem

[1139] The server generates control commands and sends them to each subsystem to adjust the vehicle's operation.

[1140] Input: The generated control command.

[1141] Output: Sends commands to each subsystem.

[1142] Specific actions: For example, it sends a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, and it sends a command to the brake system to adjust the required brake pressure to 50 psi.

[1143] Step 8:

[1144] Receiving voice instructions

[1145] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine.

[1146] Input: User's voice command.

[1147] Output: Parsed audio data.

[1148] Specific operation: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the device captures the voice and sends it to the server for analysis.

[1149] Step 9:

[1150] Generation of control commands based on voice instructions

[1151] The server uses a speech recognition engine to analyze the voice instructions and generate appropriate control commands based on them.

[1152] Input: Parsed audio data.

[1153] Output: Control command.

[1154] Specific operation: Based on the results of voice analysis, the server generates and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1155] Step 10:

[1156] Safety monitoring and abnormality response

[1157] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it alerts the user and issues necessary control commands.

[1158] Input: Real-time data.

[1159] Output: Warnings and control commands.

[1160] Specific operation: If the server detects a sudden drop in tire pressure, it will notify the user with an audible warning and issue a command to reduce engine power to safely slow down the vehicle.

[1161] Through these steps, the present invention can significantly improve vehicle efficiency, safety, and user experience.

[1162] (Application example 1)

[1163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1164] Rather than simply controlling each vehicle subsystem individually, it is necessary to manage and optimize the entire vehicle in an integrated manner to significantly improve vehicle safety, efficiency, and user experience. Furthermore, the ability to detect abnormalities in real time and automatically implement appropriate countermeasures is required. Current systems lack overall optimization because these functions are not implemented as a single integrated system.

[1165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1166] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for acquiring data from various sensors in the vehicle via the server through an API, means for analyzing the acquired data and providing optimal driving advice, and means for collecting voice instructions and sending control commands to each subsystem based on the analysis results, thereby enabling optimal management and control of the entire vehicle and realizing improved safety, efficiency, and user experience.

[1167] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes collected data and generates appropriate control commands.

[1168] An "electronic control unit" is a device that electronically controls and manages various systems in a vehicle.

[1169] "Real-time data" refers to data that instantly captures current information from sensors and other monitoring devices.

[1170] "Normalization" refers to the process of converting acquired data into a form suitable for analysis.

[1171] "Filtering" refers to the process of removing noise and outliers from data.

[1172] "Subsystem" refers to an independent partial system that performs various vehicle functions.

[1173] A "control command" is a command that includes instructions for performing a specific vehicle operation or function.

[1174] A "voice recognition engine" is a software or hardware system that analyzes a user's voice instructions and converts them into text data.

[1175] "Anomaly detection" refers to the process of detecting phenomena that cause a system to deviate from its normal operating range.

[1176] An "API" is an interface that allows a software component to be used by other software.

[1177] "Driving advice" refers to optimal driving instructions and suggestions provided to improve vehicle performance and safety.

[1178] "Data analysis" is the process of analyzing collected data in detail and extracting useful information and patterns.

[1179] "Voice instruction" refers to commands or requests given by a user to a system using voice.

[1180] The present invention is a system that uses an electronic control unit equipped with a generative artificial intelligence model to comprehensively manage and control multiple subsystems of a vehicle. Specific means for realizing this invention are described below.

[1181] Hardware and Software Configuration

[1182] Hardware:

[1183] 1. Sensors: Collect real-time data from multiple sensors installed in the vehicle (e.g., GPS sensors, tire pressure sensors, engine sensors, etc.).

[1184] 2. Electronic Control Unit (ECU): A central device that manages and controls each subsystem within a vehicle.

[1185] 3. Server: Hosts the generative AI model and performs data analysis.

[1186] software:

[1187] 1. Generative AI model: Analyzes collected data and generates appropriate control commands. Based on the analysis results, the server sends control commands to each subsystem.

[1188] 2. API endpoints: Used to retrieve data and send control commands.

[1189] 3. Speech recognition engine: Analyzes the user's voice instructions and generates control commands based on the results.

[1190] Specific examples of implementation

[1191] Data collection and analysis

[1192] The server collects real-time data from each vehicle subsystem (e.g., engine, brakes, etc.), normalizes and filters the data, and feeds it into a generative artificial intelligence model to generate appropriate control commands.

[1193] Server Roles and Operations

[1194] The server normalizes and filters the collected raw data. This process removes noise and outliers and converts the data into a format suitable for analysis. The normalized data is then input into a generative artificial intelligence model, which analyzes the real-time data. Based on the analysis results, the server generates optimal control commands for each subsystem.

[1195] Processing voice commands

[1196] The device collects voice instructions from the user and sends them to the server. The server uses a voice recognition engine to analyze the voice data and generate control commands based on the results. For example, if a user commands "set the air conditioner to 25 degrees," the server will send a command to the air conditioning system to set it to 25 degrees based on the analysis results.

[1197] Safety monitoring and response

[1198] The server constantly monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will warn the user and automatically issue necessary control commands. For example, if tire pressure suddenly drops, the server will sound an alarm to the user and issue a command to reduce engine power to safely slow down the vehicle.

[1199] Examples of prompt statements

[1200] An example of a prompt sent to a generative artificial intelligence model is "Input data: engine speed 2000 rpm, brake pressure 50 psi, fuel level 80%, tire pressure 2.5 bar, GPS location latitude 35.6895 degrees, longitude 139.6917 degrees, entertainment system ON."

[1201] An example of a prompt sentence for the speech recognition engine is a specific voice instruction such as "Set the air conditioner to 25 degrees."

[1202] The above configuration enables optimal management and control of the entire vehicle, improving safety, efficiency, and user experience.

[1203] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1204] Step 1:

[1205] As soon as the vehicle is powered on, the server automatically starts the system and collects real-time data from multiple vehicle subsystems. Data collected from sensors includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. These sensor data are taken as input data, and the raw data is stored in the server as the first output.

[1206] Step 2:

[1207] The server then performs normalization and filtering on the collected raw data. Normalization transforms the data into a form suitable for analysis, while filtering removes noise and outliers. The input to this process is the raw data collected in step 1, and the output is clean data ready for analysis.

[1208] Step 3:

[1209] The server inputs the normalized and filtered data into the generative artificial intelligence model, which analyzes the real-time data and generates optimal control commands. The input to this step is the normalized and filtered data from step 2, and the output is control commands for each subsystem.

[1210] Step 4:

[1211] The server sends the control commands obtained from the generative AI model to each subsystem. The input of this process is the control command obtained in step 3, and the output is the appropriate control execution for each subsystem. For example, it may send a command to the engine control module to adjust the engine speed, or it may instruct the braking system on the required brake pressure.

[1212] Step 5:

[1213] When a user inputs voice commands via a terminal, this voice data is sent to the server via a microphone. The server analyzes the voice data using a voice recognition engine and generates control commands based on the analysis results. The input is the user's voice data, and the output is control commands based on the analysis results and transmission to each subsystem.

[1214] Step 6:

[1215] When an abnormality occurs, the server continues to monitor real-time data and detects it. For example, if tire pressure suddenly drops, the server detects this data and immediately issues a warning to the user and a control command to ensure safety. The input of this step is the continuously collected real-time data, and the output is a warning to the user and the issuance of a corresponding control command.

[1216] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1217] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model and an emotion engine to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[1218] 1. Initialization and input data capture

[1219] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1220] Examples:

[1221] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1222] 2. Data analysis and model application

[1223] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1224] Examples:

[1225] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1226] 3. Control command output

[1227] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1228] Examples:

[1229] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1230] 4. Receiving and processing voice commands

[1231] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1232] Examples:

[1233] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1234] 5. Emotion Recognition by Emotion Engine

[1235] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1236] Examples:

[1237] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1238] 6. Emotion-based control commands

[1239] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to alleviate that stress.

[1240] Examples:

[1241] If the user is showing negative emotions, the server instructs the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1242] 7. Safety and Feedback Management

[1243] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1244] Examples:

[1245] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1246] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, more flexible control according to the user's emotional state is possible, creating a more comfortable and safe driving environment.

[1247] The processing flow will be explained below.

[1248] Step 1:

[1249] System startup

[1250] The server detects when the vehicle's power is turned on and starts the system.

[1251] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[1252] The server initializes each subsystem and puts it into a ready state.

[1253] Step 2:

[1254] Collecting data from sensors

[1255] The server collects real-time data such as engine RPM, brake pressure, fuel level, tire pressure, GPS location, entertainment system status, and user voice and facial expression data.

[1256] This data is sent to a server and stored in a database.

[1257] Step 3:

[1258] Data Preprocessing

[1259] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[1260] The server detects noise and outliers and performs filtering to remove them.

[1261] Step 4:

[1262] Application of generative artificial intelligence models

[1263] The server inputs the preprocessed data into a generative artificial intelligence model and performs data analysis.

[1264] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[1265] Step 5:

[1266] Control command generation

[1267] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[1268] For example, an engine control module may be provided with a command to adjust the fuel supply, and a braking system may be provided with a command to adjust the brake pressure.

[1269] Step 6:

[1270] Sending commands to each control system

[1271] The terminal transmits the generated control commands to each subsystem.

[1272] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1273] Step 7:

[1274] Receiving voice instructions

[1275] The terminal collects the user's voice instructions through a microphone.

[1276] The voice instructions are sent to the server and input into a voice recognition engine.

[1277] Step 8:

[1278] Analysis and processing of voice instructions

[1279] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[1280] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1281] For example, if a user commands "set the air conditioner to 25 degrees," the server recognizes this and sends a command to the air conditioning system to set it to 25 degrees.

[1282] Step 9:

[1283] Emotion recognition by emotion engine

[1284] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine.

[1285] The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1286] For example, if the emotion engine detects that the user is feeling stressed from the user's tone of voice or facial expression, it sends that information to the server.

[1287] Step 10:

[1288] Emotion-based control commands

[1289] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine.

[1290] For example, if a user is feeling stressed or angry, commands to alleviate that stress can be generated and sent to an entertainment system or air conditioning system.

[1291] It commands the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1292] Step 11:

[1293] Real-time monitoring

[1294] The server continuously monitors real-time data and detects safety anomalies.

[1295] If an abnormality is detected, an alert is sent to the user immediately.

[1296] Step 12:

[1297] Anomaly detection and response

[1298] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[1299] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[1300] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience. In addition, by combining it with an emotion engine, flexible control according to the user's emotional state becomes possible, realizing a more comfortable and safe driving environment.

[1301] Example 2

[1302] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1303] To improve safety and comfort when driving a vehicle, it is necessary to monitor the vehicle's status in real time and implement optimal control. However, conventional systems do not adequately collect and analyze accurate data, especially control that takes into account voice instructions and emotional states, and therefore do not fully achieve user satisfaction or safety.

[1304] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1305] In this invention, the server includes means for collecting information in real time from multiple subsystems of the vehicle, means for normalizing and filtering the collected information and inputting it into a generative artificial intelligence model, means for outputting optimal control commands to each subsystem based on the analysis results of the generative artificial intelligence model, means for collecting user voice and analyzing it using a voice recognition engine, means for analyzing user emotions using an emotion engine and generating control commands based on the results, and means for monitoring safety-related abnormalities in real time, issuing an alarm when an abnormality is detected, and automatically generating control commands, thereby enabling efficient vehicle operation, high safety, and providing a comfortable in-vehicle environment for the user.

[1306] "Vehicle subsystems" are groups of functional parts within a vehicle, such as the engine control module, braking system, fuel system, tire pressure sensors, GPS system, and entertainment system.

[1307] "Means for collecting information in real time" refers to a system that uses sensors and data collection devices to instantly grasp the current status of each subsystem while the vehicle is in operation.

[1308] "Normalization and filtering" refers to the process of processing the collected raw data, converting it into a format suitable for analysis, and removing noise and outliers to improve the accuracy of the data.

[1309] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates optimal control commands for a vehicle in real time.

[1310] A "voice recognition engine" is a program that analyzes a user's voice instructions, converts the content into text format, and analyzes it.

[1311] An "emotion engine" is an algorithm or program that analyzes a user's voice and facial expression data to recognize the user's emotional state.

[1312] A "control command" is an instruction or command issued to each subsystem of a vehicle, which adjusts each subsystem to operate appropriately.

[1313] "Means for monitoring safety-related abnormalities" refers to a mechanism for monitoring data from each vehicle subsystem in real time to detect abnormalities or dangerous conditions.

[1314] "Means for issuing warnings and automatically generating control commands" refers to the process of issuing a sound or display to warn the user when an abnormality is detected, and generating and issuing a control command to automatically respond.

[1315] A "comfortable in-car environment" is an environment that optimizes the temperature, music, lighting, etc. inside the car according to the user's emotional state and voice instructions.

[1316] The present invention provides a system for improving vehicle safety, efficiency, and user experience. The system utilizes a generative artificial intelligence model and an emotion engine to achieve comprehensive management and analysis. Specific embodiments for implementing the present invention are described below.

[1317] Initialization and input data capture

[1318] The server automatically starts the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1319] Examples:

[1320] The unit activates when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1321] Data analysis and model application

[1322] The server normalizes and filters the collected raw data and inputs it into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1323] Examples:

[1324] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1325] Control command output

[1326] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1327] Examples:

[1328] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1329] Receiving and processing voice commands

[1330] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1331] Examples:

[1332] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1333] Emotion recognition by emotion engine

[1334] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1335] Examples:

[1336] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1337] Emotion-based control command generation

[1338] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to reduce that stress.

[1339] Examples:

[1340] If the user is showing negative emotions, the server will instruct the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1341] Safety and Feedback Management

[1342] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1343] Examples:

[1344] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1345] Prompt Sentence Examples

[1346] Here are some example prompts you can provide to a generative AI model:

[1347] Prompt: "The vehicle's current position is 35.6895 degrees latitude and 139.6917 degrees longitude, the engine speed is 2000 RPM, and the fuel tank is 80% full. Analyze this data and generate optimal control commands."

[1348] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, flexible control according to the user's emotional state is possible, realizing a more comfortable and safe driving environment.

[1349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1350] Step 1: System initialization and data collection

[1351] Input: Vehicle power-on signal

[1352] Output: Each control system initialized and the initial data collected.

[1353] Specific processing:

[1354] The server starts the system when the vehicle is powered on, initializing the generative AI model and all control systems, while collecting initial data from multiple subsystems such as the engine control module, braking system, and fuel system.

[1355] Example: When a vehicle engine starts, the server receives an initialization signal, which starts collecting data from all subsystems, such as the engine speed of 2000 rpm and the GPS location information of 35.6895 degrees latitude and 139.6917 degrees longitude.

[1356] Step 2: Normalize and filter the data

[1357] Input: Raw data collected after initialization

[1358] Output: Normalized and filtered data

[1359] Specific processing:

[1360] After initialization, the server normalizes and filters the collected raw data to prepare it for analysis. Normalization transforms the data into a consistent format, while filtering improves the accuracy of the data by removing noise and outliers.

[1361] Example: A server processes raw data, for example, converting engine RPM data to an appropriate scale and filtering out abnormally high RPMs or sudden pressure fluctuations.

[1362] Step 3: Input and analyze data into the generative AI model

[1363] Input: Normalized and filtered data

[1364] Output: Analysis results from the generative AI model

[1365] Specific processing:

[1366] The server inputs the normalized and filtered data into a generative AI model, which analyzes the data in real time. The model generates optimal control commands for each subsystem based on the collected data.

[1367] Example: If the engine temperature exceeds 90 degrees, the model generates a command to turn on the cooling fan. It also generates a command to adjust the engine speed appropriately for fuel efficiency.

[1368] Step 4: Output and execution of control commands

[1369] Input: Analysis results from a generative AI model

[1370] Output: Control commands to each subsystem

[1371] Specific processing:

[1372] Based on the analysis results obtained from the generated AI model, the server generates and immediately transmits optimal control commands for each subsystem, allowing each subsystem to adjust its operation.

[1373] Example: Send a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, or send a command to the braking system to adjust the required brake pressure to 50 psi.

[1374] Step 5: Receiving and processing voice commands

[1375] Input: Voice command from the user

[1376] Output: Control commands based on the results of voice analysis

[1377] Specific processing:

[1378] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the subsystem.

[1379] Example: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1380] Step 6: Emotion recognition and response with the emotion engine

[1381] Input: User voice and facial expression data

[1382] Output: Emotion analysis results and response instructions

[1383] Specific processing:

[1384] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes the data using an emotion engine. Based on the analysis results, it generates and sends appropriate control commands.

[1385] Example: An emotion engine detects from the user's tone of voice and facial expression that the user is stressed. The server uses this information to send a command to the entertainment system to play relaxing music.

[1386] Step 7: Safety monitoring and feedback

[1387] Input: Real-time data from each subsystem

[1388] Output: Safety warnings and control commands

[1389] Specific processing:

[1390] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1391] Example: If a sudden drop in tire pressure is detected, the system will sound an audible warning to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1392] Through these processing steps, the present invention provides a system that significantly improves vehicle efficiency, safety, and user experience. The combination of a generative AI model and an emotion engine enables flexible control according to the user's emotional state, creating a more comfortable and safe driving environment.

[1393] (Application example 2)

[1394] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1395] Conventional vehicle control systems often lack safety and comfort due to insufficient coordination between subsystems or flexible control based on user emotions. Furthermore, the efficiency of real-time data analysis and control command issuance is limited, making it difficult to achieve an advanced user experience.

[1396] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it to a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for collecting user emotion data, analyzing it using an emotion engine, and outputting control commands to each subsystem based on the analysis results, means for analyzing user voice instructions using a voice recognition engine, generating control commands based on the analysis results, and transmitting them to the vehicle subsystems, means for monitoring the vehicle's real-time data, issuing an alert if a safety abnormality is detected, and automatically issuing necessary control commands, and means for automatically adjusting the entertainment system and air conditioning system according to the user's emotional state. This enables improvements in vehicle efficiency, safety, and user experience.

[1397] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates optimal control commands based on collected data.

[1398] An "electronic control unit" is an electronic circuit board that comprehensively manages and controls each subsystem of a vehicle.

[1399] A "subsystem" is a separate functional unit in a vehicle, such as the engine, brakes, or entertainment system.

[1400] "Normalization" is the process of converting data into a form suitable for analysis.

[1401] "Filtering" is the process of cleaning data by removing noise and outliers.

[1402] An "emotion engine" is an algorithm that analyzes data such as the user's voice and facial expressions to recognize their emotional state.

[1403] A "voice recognition engine" is an algorithm that analyzes a user's voice, understands its content, and generates corresponding commands.

[1404] "Real-time data" refers to the latest data collected and analyzed at the current time.

[1405] A "control command" is an instruction to cause a vehicle subsystem to perform a specific operation.

[1406] "Emotional data" refers to data that indicates the user's emotional state, and includes voice, facial expression, heart rate, and the like.

[1407] An "entertainment system" is a system that provides content such as music and video in a vehicle.

[1408] An "air conditioning system" is a system that adjusts the temperature and humidity inside a vehicle.

[1409] "Abnormality detection" is the process of determining abnormalities when the vehicle's operation or state is different from normal.

[1410] This invention is a system that is equipped with a generative artificial intelligence model and an emotion engine, and that comprehensively manages and analyzes each subsystem of a vehicle in real time. Specific means and processes for implementing this invention will be described.

[1411] 1. System Initialization and Data Collection

[1412] When the engine of a vehicle equipped with an electronic control unit (ECU) starts, the server automatically starts the system and initializes all control systems. The server collects real-time information from multiple sensors in the vehicle, such as engine speed, brake pressure, fuel level, tire pressure, GPS location information, and entertainment system status.

[1413] Examples:

[1414] The unit activates when the vehicle's engine is started and collects information such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1415] 2. Data analysis and control command generation

[1416] The server normalizes and filters the collected raw data and inputs it into a generative AI model. The generative AI model analyzes the real-time data and generates optimal control commands. Based on the results, optimal control commands are output for each subsystem.

[1417] Examples:

[1418] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1419] 3. Collecting and analyzing user emotion data

[1420] The device collects user emotional data through a microphone and camera. The server uses an emotion engine to analyze this data and recognize the user's emotions in real time. Based on the results, it generates appropriate control commands for each subsystem.

[1421] Examples:

[1422] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1423] 4. Receiving and processing voice commands

[1424] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1425] Examples:

[1426] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1427] 5. Safety monitoring and control

[1428] The server continuously monitors real-time data to detect safety-related anomalies, immediately notifying the user with an alert and automatically issuing necessary control commands.

[1429] Examples:

[1430] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1431] The specific hardware and software used

[1432] Hardware:

[1433] Smartphone (camera, microphone)

[1434] Various sensors inside the vehicle (engine, brake, fuel, etc.)

[1435] software:

[1436] OpenCV (camera image processing)

[1437] Keras (emotion recognition model)

[1438] Pyautogui (system control)

[1439] Speech Recognition Engine

[1440] Example prompt sentence:

[1441] "Capture the user's face with a camera, and if the emotion recognition model detects 'sadness,' play relaxing music."

[1442] This will enable improvements in vehicle efficiency, safety and user experience.

[1443] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1444] Step 1:

[1445] The server automatically starts the system when the vehicle engine is started and collects real-time data from multiple sensors. Input data includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. The collected raw data is stored on the server.

[1446] Step 2:

[1447] The server normalizes and filters the collected raw data. During normalization, the data values ​​are standardized and converted into a format suitable for analysis. During filtering, noise and outliers are removed. The resulting clean data is fed into the generative artificial intelligence model.

[1448] Step 3:

[1449] The server inputs clean data into a generative AI model and analyzes the real-time data. The model generates optimal control commands as a result of the analysis and returns them to the server. For example, it generates commands to adjust engine speed or turn cooling fans on and off.

[1450] Step 4:

[1451] The server sends the generated control commands to each subsystem to adjust the vehicle's operation. Based on the input control commands, specific operations are performed on the engine control module and the brake system. For example, a command to reduce fuel supply is sent to the engine control module.

[1452] Step 5:

[1453] The device collects the user's emotional data, using a microphone to collect voice data and a camera to capture facial expression data, which is then sent to a server in real time.

[1454] Step 6:

[1455] The server inputs the collected voice and facial expression data into the emotion engine to analyze the user's emotional state. The emotion engine processes this data and determines the emotion the user is feeling. For example, it may detect that the user is feeling stressed.

[1456] Step 7:

[1457] Based on the analysis results of the emotion engine, the server generates and sends optimal control commands to each subsystem, such as a command to play relaxing music to the entertainment system or an adjustment command to the air conditioning system.

[1458] Step 8:

[1459] The device collects the user's voice instructions through a microphone and sends them to the server. The server then analyzes the voice instructions using a voice recognition engine and generates control commands based on the analysis results. For example, if the user says, "Set the air conditioner to 25 degrees," a command to set the temperature to 25 degrees is sent to the air conditioning system.

[1460] Step 9:

[1461] The server monitors real-time vehicle data and detects safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with a warning and automatically issues necessary control commands. For example, if tire pressure suddenly drops, a warning will sound and a command will be issued to reduce engine output to safely decelerate.

[1462] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1464] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1465] [Fourth embodiment]

[1466] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1467] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1468] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1469] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1470] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1471] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1472] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1473] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1474] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1475] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1476] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1477] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1478] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1479] The present invention provides a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[1480] 1. Initialization and input data capture

[1481] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1482] Examples:

[1483] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1484] 2. Data analysis and model application

[1485] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1486] Examples:

[1487] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1488] 3. Control command output

[1489] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1490] Examples:

[1491] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1492] 4. Receiving and processing voice commands

[1493] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1494] Examples:

[1495] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1496] 5. Safety and Feedback Management

[1497] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[1498] Examples:

[1499] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1500] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience.

[1501] The processing flow will be explained below.

[1502] Step 1:

[1503] System startup

[1504] The server detects when the vehicle's power is turned on and starts the system.

[1505] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[1506] The server initializes each subsystem and puts it into a ready state.

[1507] Step 2:

[1508] Collecting data from sensors

[1509] The server collects a variety of data in real time, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1510] This data is sent to a server and stored in a database.

[1511] Step 3:

[1512] Data Preprocessing

[1513] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[1514] The server detects noise and outliers and performs filtering to remove them.

[1515] Step 4:

[1516] Application of generative artificial intelligence models

[1517] The server inputs the preprocessed data into a generative artificial intelligence model for analysis.

[1518] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[1519] Step 5:

[1520] Control command generation

[1521] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[1522] Control commands are provided to the engine control module, braking system, navigation system, etc.

[1523] Step 6:

[1524] Sending commands to each control system

[1525] The terminal transmits the generated control commands to each subsystem.

[1526] For example, it sends a command to the engine control module to adjust the fuel supply, and a command to the brake system to adjust the brake pressure.

[1527] Step 7:

[1528] Receiving voice instructions

[1529] The terminal collects the user's voice instructions through a microphone.

[1530] The voice instructions are sent to the server.

[1531] Step 8:

[1532] Analysis and processing of voice instructions

[1533] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[1534] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1535] Step 9:

[1536] Real-time monitoring

[1537] The server continuously monitors all real-time data and detects safety anomalies.

[1538] If an abnormality is detected, an alert is sent to the user.

[1539] Step 10:

[1540] Anomaly detection and response

[1541] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[1542] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[1543] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience.

[1544] Example 1

[1545] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1546] Modern vehicles have multiple subsystems, such as engine control and brake control, which operate independently, limiting the improvement of efficiency and safety. Furthermore, it is difficult to respond quickly to user instructions, and real-time data analysis and safety monitoring are not adequately performed. This creates a demand for improved user experience, such as improved fuel efficiency and safer driving support.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1548] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative AI model, means for outputting control commands to each subsystem based on the results of analysis by the generative AI model, means for automatically starting up the entire system and initializing all control modules when the vehicle is powered on, means for receiving user voice instructions, generating control commands based on the instructions and sending them to the subsystems, and means for monitoring the real-time data, detecting safety-related anomalies, and issuing necessary control commands, thereby enabling improvements in vehicle efficiency, safety, and user experience.

[1549] "Multiple subsystems" refer to multiple modules within a vehicle that operate independently and are responsible for specific functions.

[1550] "Real-time data" refers to data that instantly collects and analyzes current conditions and information.

[1551] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes collected data and generates optimal control commands.

[1552] "Data normalization" is the process of converting collected data into a consistent format that is easier to analyze.

[1553] "Data filtering" refers to the process of removing noise and outliers from collected data.

[1554] "Control commands" are operational instructions sent to each subsystem based on the analysis results.

[1555] "Automatic system startup" refers to the process of automatically starting all control modules as soon as the vehicle is powered on.

[1556] "Voice instructions" refer to operational instructions given by the user to the system using voice.

[1557] A "safety-related abnormality" is an abnormal condition that may pose a risk to the operation or performance of a vehicle.

[1558] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. Specific aspects of this system are described below.

[1559] 1. Initialization and input data capture

[1560] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1561] Examples:

[1562] The server starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed: 2000 rpm, brake pressure: 50 psi, fuel tank: 80% full or more, tire pressure: 2.5 bar, GPS location: latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1563] 2. Data analysis and model application

[1564] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1565] Examples:

[1566] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1567] 3. Control command output

[1568] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1569] Examples:

[1570] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1571] 4. Receiving and processing voice commands

[1572] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1573] Examples:

[1574] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1575] 5. Safety and Feedback Management

[1576] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will alert the user and simultaneously issue control commands to ensure safety.

[1577] Examples:

[1578] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1579] Example prompt sentence:

[1580] "Please explain the role of a generative AI model that collects data from sensors when a car's engine starts and generates control commands for the cooling fan based on that data."

[1581] Through the above process, the present invention is a system that can significantly improve vehicle efficiency, safety, and user experience.

[1582] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1583] Step 1:

[1584] Initialization and input data capture

[1585] The server automatically starts the system and initializes all control systems as soon as the vehicle is turned on. Specifically, the server detects the power status via the vehicle's main ECU and starts each module, such as engine control, brake control, and entertainment system.

[1586] Input: Vehicle power on signal.

[1587] Output: System startup and initialization of each module completed.

[1588] Step 2:

[1589] Collecting data from sensors

[1590] The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1591] Inputs: engine start, braking, fuel usage, tire condition, GPS location, entertainment controls, etc.

[1592] Output: Real-time data collection.

[1593] What it does: The server continuously monitors and collects data on engine RPM, brake pressure, fuel tank level, tire pressure, GPS location, and entertainment system status.

[1594] Step 3:

[1595] Data normalization

[1596] The raw data acquired by the server is converted into a format that is easy to analyze. By converting into a unified data format, consistency in analysis is ensured.

[1597] Input: Raw data (engine RPM, brake pressure, etc.).

[1598] Output: Normalized data.

[1599] Specific operation: For example, if the temperature data is mixed in Celsius and Fahrenheit, the server converts it all to Celsius.

[1600] Step 4:

[1601] Data filtering

[1602] The server removes noise and outliers from the normalized data, improving the accuracy of the analysis.

[1603] Input: Normalized data.

[1604] Output: filtered clean data.

[1605] Specific operation: The server uses statistical methods and filtering algorithms to remove obviously abnormal values ​​(e.g., momentary sensor misses).

[1606] Step 5:

[1607] Data input to generative AI models

[1608] The server feeds the normalized and filtered data into a generative AI model.

[1609] Input: Filtered clean data.

[1610] Output: The data input to the AI ​​model.

[1611] Specific operation: The server inputs clean data from various sensors into the AI ​​model and performs real-time analysis.

[1612] Step 6:

[1613] Analysis and control command generation by AI model

[1614] The server analyzes the data using a generative AI model and generates optimal control commands.

[1615] Input: The data fed into the AI ​​model.

[1616] Output: The generated control commands.

[1617] Specific operations: For example, generating a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, or to adjust the engine speed to optimize fuel efficiency.

[1618] Step 7:

[1619] Output of control commands to each subsystem

[1620] The server generates control commands and sends them to each subsystem to adjust the vehicle's operation.

[1621] Input: The generated control command.

[1622] Output: Sends commands to each subsystem.

[1623] Specific actions: For example, it sends a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, and it sends a command to the brake system to adjust the required brake pressure to 50 psi.

[1624] Step 8:

[1625] Receiving voice instructions

[1626] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine.

[1627] Input: User's voice command.

[1628] Output: Parsed audio data.

[1629] Specific operation: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the device captures the voice and sends it to the server for analysis.

[1630] Step 9:

[1631] Generation of control commands based on voice instructions

[1632] The server uses a speech recognition engine to analyze the voice instructions and generate appropriate control commands based on them.

[1633] Input: Parsed audio data.

[1634] Output: Control command.

[1635] Specific operation: Based on the results of voice analysis, the server generates and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1636] Step 10:

[1637] Safety monitoring and abnormality response

[1638] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it alerts the user and issues necessary control commands.

[1639] Input: Real-time data.

[1640] Output: Warnings and control commands.

[1641] Specific operation: If the server detects a sudden drop in tire pressure, it will notify the user with an audible warning and issue a command to reduce engine power to safely slow down the vehicle.

[1642] Through these steps, the present invention can significantly improve vehicle efficiency, safety, and user experience.

[1643] (Application example 1)

[1644] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1645] Rather than simply controlling each vehicle subsystem individually, it is necessary to manage and optimize the entire vehicle in an integrated manner to significantly improve vehicle safety, efficiency, and user experience. Furthermore, the ability to detect abnormalities in real time and automatically implement appropriate countermeasures is required. Current systems lack overall optimization because these functions are not implemented as a single integrated system.

[1646] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1647] In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for acquiring data from various sensors in the vehicle via the server through an API, means for analyzing the acquired data and providing optimal driving advice, and means for collecting voice instructions and sending control commands to each subsystem based on the analysis results, thereby enabling optimal management and control of the entire vehicle and realizing improved safety, efficiency, and user experience.

[1648] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that analyzes collected data and generates appropriate control commands.

[1649] An "electronic control unit" is a device that electronically controls and manages various systems in a vehicle.

[1650] "Real-time data" refers to data that instantly captures current information from sensors and other monitoring devices.

[1651] "Normalization" refers to the process of converting acquired data into a form suitable for analysis.

[1652] "Filtering" refers to the process of removing noise and outliers from data.

[1653] "Subsystem" refers to an independent partial system that performs various vehicle functions.

[1654] A "control command" is a command that includes instructions for performing a specific vehicle operation or function.

[1655] A "voice recognition engine" is a software or hardware system that analyzes a user's voice instructions and converts them into text data.

[1656] "Anomaly detection" refers to the process of detecting phenomena that cause a system to deviate from its normal operating range.

[1657] An "API" is an interface that allows a software component to be used by other software.

[1658] "Driving advice" refers to optimal driving instructions and suggestions provided to improve vehicle performance and safety.

[1659] "Data analysis" is the process of analyzing collected data in detail and extracting useful information and patterns.

[1660] "Voice instruction" refers to commands or requests given by a user to a system using voice.

[1661] The present invention is a system that uses an electronic control unit equipped with a generative artificial intelligence model to comprehensively manage and control multiple subsystems of a vehicle. Specific means for realizing this invention are described below.

[1662] Hardware and Software Configuration

[1663] Hardware:

[1664] 1. Sensors: Collect real-time data from multiple sensors installed in the vehicle (e.g., GPS sensors, tire pressure sensors, engine sensors, etc.).

[1665] 2. Electronic Control Unit (ECU): A central device that manages and controls each subsystem within a vehicle.

[1666] 3. Server: Hosts the generative AI model and performs data analysis.

[1667] software:

[1668] 1. Generative AI model: Analyzes collected data and generates appropriate control commands. Based on the analysis results, the server sends control commands to each subsystem.

[1669] 2. API endpoints: Used to retrieve data and send control commands.

[1670] 3. Speech recognition engine: Analyzes the user's voice instructions and generates control commands based on the results.

[1671] Specific examples of implementation

[1672] Data collection and analysis

[1673] The server collects real-time data from each vehicle subsystem (e.g., engine, brakes, etc.), normalizes and filters the data, and feeds it into a generative artificial intelligence model to generate appropriate control commands.

[1674] Server Roles and Operations

[1675] The server normalizes and filters the collected raw data. This process removes noise and outliers and converts the data into a format suitable for analysis. The normalized data is then input into a generative artificial intelligence model, which analyzes the real-time data. Based on the analysis results, the server generates optimal control commands for each subsystem.

[1676] Processing voice commands

[1677] The device collects voice instructions from the user and sends them to the server. The server uses a voice recognition engine to analyze the voice data and generate control commands based on the results. For example, if a user commands "set the air conditioner to 25 degrees," the server will send a command to the air conditioning system to set it to 25 degrees based on the analysis results.

[1678] Safety monitoring and response

[1679] The server constantly monitors real-time data and detects safety-related anomalies. If an anomaly is detected, the server will warn the user and automatically issue necessary control commands. For example, if tire pressure suddenly drops, the server will sound an alarm to the user and issue a command to reduce engine power to safely slow down the vehicle.

[1680] Examples of prompt statements

[1681] An example of a prompt sent to a generative artificial intelligence model is "Input data: engine speed 2000 rpm, brake pressure 50 psi, fuel level 80%, tire pressure 2.5 bar, GPS location latitude 35.6895 degrees, longitude 139.6917 degrees, entertainment system ON."

[1682] An example of a prompt sentence for the speech recognition engine is a specific voice instruction such as "Set the air conditioner to 25 degrees."

[1683] The above configuration enables optimal management and control of the entire vehicle, improving safety, efficiency, and user experience.

[1684] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1685] Step 1:

[1686] As soon as the vehicle is powered on, the server automatically starts the system and collects real-time data from multiple vehicle subsystems. Data collected from sensors includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. These sensor data are taken as input data, and the raw data is stored in the server as the first output.

[1687] Step 2:

[1688] The server then performs normalization and filtering on the collected raw data. Normalization transforms the data into a form suitable for analysis, while filtering removes noise and outliers. The input to this process is the raw data collected in step 1, and the output is clean data ready for analysis.

[1689] Step 3:

[1690] The server inputs the normalized and filtered data into the generative artificial intelligence model, which analyzes the real-time data and generates optimal control commands. The input to this step is the normalized and filtered data from step 2, and the output is control commands for each subsystem.

[1691] Step 4:

[1692] The server sends the control commands obtained from the generative AI model to each subsystem. The input of this process is the control command obtained in step 3, and the output is the appropriate control execution for each subsystem. For example, it may send a command to the engine control module to adjust the engine speed, or it may instruct the braking system on the required brake pressure.

[1693] Step 5:

[1694] When a user inputs voice commands via a terminal, this voice data is sent to the server via a microphone. The server analyzes the voice data using a voice recognition engine and generates control commands based on the analysis results. The input is the user's voice data, and the output is control commands based on the analysis results and transmission to each subsystem.

[1695] Step 6:

[1696] When an abnormality occurs, the server continues to monitor real-time data and detects it. For example, if tire pressure suddenly drops, the server detects this data and immediately issues a warning to the user and a control command to ensure safety. The input of this step is the continuously collected real-time data, and the output is a warning to the user and the issuance of a corresponding control command.

[1697] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1698] The present invention is a system that improves safety, efficiency, and user experience by applying an electronic control unit (ECU) equipped with a generative artificial intelligence model and an emotion engine to a vehicle and comprehensively managing and analyzing multiple vehicle subsystems. As a specific embodiment for implementing the present invention, the program processing and specific examples thereof will be described in detail below.

[1699] 1. Initialization and input data capture

[1700] The server automatically boots up the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1701] Examples:

[1702] The unit starts up when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1703] 2. Data analysis and model application

[1704] The raw data collected by the server is normalized and filtered, and then input into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1705] Examples:

[1706] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1707] 3. Control command output

[1708] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1709] Examples:

[1710] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1711] 4. Receiving and processing voice commands

[1712] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1713] Examples:

[1714] When a user issues a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set the temperature to 25 degrees.

[1715] 5. Emotion Recognition by Emotion Engine

[1716] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1717] Examples:

[1718] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1719] 6. Emotion-based control commands

[1720] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to alleviate that stress.

[1721] Examples:

[1722] If the user is showing negative emotions, the server instructs the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1723] 7. Safety and Feedback Management

[1724] The server continuously monitors real-time data and detects safety-related anomalies. If an anomaly is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1725] Examples:

[1726] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine output to safely slow down the vehicle.

[1727] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, more flexible control according to the user's emotional state is possible, creating a more comfortable and safe driving environment.

[1728] The processing flow will be explained below.

[1729] Step 1:

[1730] System startup

[1731] The server detects when the vehicle's power is turned on and starts the system.

[1732] The server starts communicating with all sensors connected to the ECU (Electronic Control Unit).

[1733] The server initializes each subsystem and puts it into a ready state.

[1734] Step 2:

[1735] Collecting data from sensors

[1736] The server collects real-time data such as engine RPM, brake pressure, fuel level, tire pressure, GPS location, entertainment system status, and user voice and facial expression data.

[1737] This data is sent to a server and stored in a database.

[1738] Step 3:

[1739] Data Preprocessing

[1740] The server normalizes the collected raw data and converts it into a format suitable for analysis.

[1741] The server detects noise and outliers and performs filtering to remove them.

[1742] Step 4:

[1743] Application of generative artificial intelligence models

[1744] The server inputs the preprocessed data into a generative artificial intelligence model and performs data analysis.

[1745] The artificial intelligence model generates optimal control commands based on real-time and historical data.

[1746] Step 5:

[1747] Control command generation

[1748] Based on the analysis results output by the artificial intelligence model, the server generates specific control commands to be sent to each subsystem.

[1749] For example, an engine control module may be provided with a command to adjust the fuel supply, and a braking system may be provided with a command to adjust the brake pressure.

[1750] Step 6:

[1751] Sending commands to each control system

[1752] The terminal transmits the generated control commands to each subsystem.

[1753] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1754] Step 7:

[1755] Receiving voice instructions

[1756] The terminal collects the user's voice instructions through a microphone.

[1757] The voice instructions are sent to the server and input into a voice recognition engine.

[1758] Step 8:

[1759] Analysis and processing of voice instructions

[1760] The server uses a speech recognition engine to convert the voice instructions into text and analyzes the content.

[1761] Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1762] For example, if a user commands "set the air conditioner to 25 degrees," the server recognizes this and sends a command to the air conditioning system to set it to 25 degrees.

[1763] Step 9:

[1764] Emotion recognition by emotion engine

[1765] The device collects the user's voice and facial expression data, and the server analyzes this data using an emotion engine.

[1766] The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1767] For example, if the emotion engine detects that the user is feeling stressed from the user's tone of voice or facial expression, it sends that information to the server.

[1768] Step 10:

[1769] Emotion-based control commands

[1770] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine.

[1771] For example, if a user is feeling stressed or angry, commands to alleviate that stress can be generated and sent to an entertainment system or air conditioning system.

[1772] It commands the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1773] Step 11:

[1774] Real-time monitoring

[1775] The server continuously monitors real-time data and detects safety anomalies.

[1776] If an abnormality is detected, an alert is sent to the user immediately.

[1777] Step 12:

[1778] Anomaly detection and response

[1779] When the server detects an abnormality, it automatically generates the necessary control commands and sends them to each subsystem.

[1780] For example, if tire pressure suddenly drops, a command is issued to reduce engine output and safely decelerate.

[1781] Through the above processing steps, the system of the present invention can dramatically improve vehicle safety, efficiency, and user experience. In addition, by combining it with an emotion engine, flexible control according to the user's emotional state becomes possible, realizing a more comfortable and safe driving environment.

[1782] Example 2

[1783] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1784] To improve safety and comfort when driving a vehicle, it is necessary to monitor the vehicle's status in real time and implement optimal control. However, conventional systems do not adequately collect and analyze accurate data, especially control that takes into account voice instructions and emotional states, and therefore do not fully achieve user satisfaction or safety.

[1785] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1786] In this invention, the server includes means for collecting information in real time from multiple subsystems of the vehicle, means for normalizing and filtering the collected information and inputting it into a generative artificial intelligence model, means for outputting optimal control commands to each subsystem based on the analysis results of the generative artificial intelligence model, means for collecting user voice and analyzing it using a voice recognition engine, means for analyzing user emotions using an emotion engine and generating control commands based on the results, and means for monitoring safety-related abnormalities in real time, issuing an alarm when an abnormality is detected, and automatically generating control commands, thereby enabling efficient vehicle operation, high safety, and providing a comfortable in-vehicle environment for the user.

[1787] "Vehicle subsystems" are groups of functional parts within a vehicle, such as the engine control module, braking system, fuel system, tire pressure sensors, GPS system, and entertainment system.

[1788] "Means for collecting information in real time" refers to a system that uses sensors and data collection devices to instantly grasp the current status of each subsystem while the vehicle is in operation.

[1789] "Normalization and filtering" refers to the process of processing the collected raw data, converting it into a format suitable for analysis, and removing noise and outliers to improve the accuracy of the data.

[1790] A "generative artificial intelligence model" is an algorithm or program that learns from large amounts of data and generates optimal control commands for a vehicle in real time.

[1791] A "voice recognition engine" is a program that analyzes a user's voice instructions, converts the content into text format, and analyzes it.

[1792] An "emotion engine" is an algorithm or program that analyzes a user's voice and facial expression data to recognize the user's emotional state.

[1793] A "control command" is an instruction or command issued to each subsystem of a vehicle, which adjusts each subsystem to operate appropriately.

[1794] "Means for monitoring safety-related abnormalities" refers to a mechanism for monitoring data from each vehicle subsystem in real time to detect abnormalities or dangerous conditions.

[1795] "Means for issuing warnings and automatically generating control commands" refers to the process of issuing a sound or display to warn the user when an abnormality is detected, and generating and issuing a control command to automatically respond.

[1796] A "comfortable in-car environment" is an environment that optimizes the temperature, music, lighting, etc. inside the car according to the user's emotional state and voice instructions.

[1797] The present invention provides a system for improving vehicle safety, efficiency, and user experience. The system utilizes a generative artificial intelligence model and an emotion engine to achieve comprehensive management and analysis. Specific embodiments for implementing the present invention are described below.

[1798] Initialization and input data capture

[1799] The server automatically starts the system and initializes all control systems when the vehicle is powered on. The server collects real-time data from multiple sensors, including engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status.

[1800] Examples:

[1801] The unit activates when the vehicle's engine is started and begins collecting data from all sensors, such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees and longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1802] Data analysis and model application

[1803] The server normalizes and filters the collected raw data and inputs it into a generative AI model. Normalization is the process of converting data into a format suitable for analysis, and filtering is the process of removing noise and outliers. The generative AI model analyzes the real-time data and generates optimal control commands based on it.

[1804] Examples:

[1805] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1806] Control command output

[1807] Based on the analysis results obtained from the generative AI model, the server generates optimal control commands for each subsystem. These commands are immediately sent to each subsystem to adjust the vehicle's operation.

[1808] Examples:

[1809] For example, it can send a command to the engine control module to reduce fuel delivery by 10% to improve fuel economy, or it can send a command to the braking system to adjust the required brake pressure to 50 psi.

[1810] Receiving and processing voice commands

[1811] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1812] Examples:

[1813] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1814] Emotion recognition by emotion engine

[1815] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes this data using an emotion engine. The emotion engine recognizes the user's emotions in real time and sends the results to the server.

[1816] Examples:

[1817] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1818] Emotion-based control command generation

[1819] The server generates appropriate control commands for each subsystem based on the emotion data from the emotion engine. For example, if the user is feeling stressed or angry, it generates commands to reduce that stress.

[1820] Examples:

[1821] If the user is showing negative emotions, the server will instruct the entertainment system to play relaxing music and the air conditioning system to adjust the temperature to a comfortable level.

[1822] Safety and Feedback Management

[1823] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1824] Examples:

[1825] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1826] Prompt Sentence Examples

[1827] Here are some example prompts you can provide to a generative AI model:

[1828] Prompt: "The vehicle's current position is 35.6895 degrees latitude and 139.6917 degrees longitude, the engine speed is 2000 RPM, and the fuel tank is 80% full. Analyze this data and generate optimal control commands."

[1829] Through the above process, the present invention provides a system that can significantly improve vehicle efficiency, safety, and user experience. By combining it with an emotion engine, flexible control according to the user's emotional state is possible, realizing a more comfortable and safe driving environment.

[1830] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1831] Step 1: System initialization and data collection

[1832] Input: Vehicle power-on signal

[1833] Output: Each control system initialized and the initial data collected.

[1834] Specific processing:

[1835] The server starts the system when the vehicle is powered on, initializing the generative AI model and all control systems, while collecting initial data from multiple subsystems such as the engine control module, braking system, and fuel system.

[1836] Example: When a vehicle engine starts, the server receives an initialization signal, which starts collecting data from all subsystems, such as the engine speed of 2000 rpm and the GPS location information of 35.6895 degrees latitude and 139.6917 degrees longitude.

[1837] Step 2: Normalize and filter the data

[1838] Input: Raw data collected after initialization

[1839] Output: Normalized and filtered data

[1840] Specific processing:

[1841] After initialization, the server normalizes and filters the collected raw data to prepare it for analysis. Normalization transforms the data into a consistent format, while filtering improves the accuracy of the data by removing noise and outliers.

[1842] Example: A server processes raw data, for example, converting engine RPM data to an appropriate scale and filtering out abnormally high RPMs or sudden pressure fluctuations.

[1843] Step 3: Input and analyze data into the generative AI model

[1844] Input: Normalized and filtered data

[1845] Output: Analysis results from the generative AI model

[1846] Specific processing:

[1847] The server inputs the normalized and filtered data into a generative AI model, which analyzes the data in real time. The model generates optimal control commands for each subsystem based on the collected data.

[1848] Example: If the engine temperature exceeds 90 degrees, the model generates a command to turn on the cooling fan. It also generates a command to adjust the engine speed appropriately for fuel efficiency.

[1849] Step 4: Output and execution of control commands

[1850] Input: Analysis results from a generative AI model

[1851] Output: Control commands to each subsystem

[1852] Specific processing:

[1853] Based on the analysis results obtained from the generated AI model, the server generates and immediately transmits optimal control commands for each subsystem, allowing each subsystem to adjust its operation.

[1854] Example: Send a command to the engine control module to reduce fuel supply by 10% to improve fuel economy, or send a command to the braking system to adjust the required brake pressure to 50 psi.

[1855] Step 5: Receiving and processing voice commands

[1856] Input: Voice command from the user

[1857] Output: Control commands based on the results of voice analysis

[1858] Specific processing:

[1859] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the subsystem.

[1860] Example: When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1861] Step 6: Emotion recognition and response with the emotion engine

[1862] Input: User voice and facial expression data

[1863] Output: Emotion analysis results and response instructions

[1864] Specific processing:

[1865] The device collects the user's voice and facial expression data through a camera and microphone, and the server analyzes the data using an emotion engine. Based on the analysis results, it generates and sends appropriate control commands.

[1866] Example: An emotion engine detects from the user's tone of voice and facial expression that the user is stressed. The server uses this information to send a command to the entertainment system to play relaxing music.

[1867] Step 7: Safety monitoring and feedback

[1868] Input: Real-time data from each subsystem

[1869] Output: Safety warnings and control commands

[1870] Specific processing:

[1871] The server continuously monitors real-time data and detects any safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with an alert and automatically issues the necessary control commands.

[1872] Example: If a sudden drop in tire pressure is detected, the system will sound an audible warning to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1873] Through these processing steps, the present invention provides a system that significantly improves vehicle efficiency, safety, and user experience. The combination of a generative AI model and an emotion engine enables flexible control according to the user's emotional state, creating a more comfortable and safe driving environment.

[1874] (Application example 2)

[1875] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1876] Conventional vehicle control systems often lack safety and comfort due to insufficient coordination between subsystems or flexible control based on user emotions. Furthermore, the efficiency of real-time data analysis and control command issuance is limited, making it difficult to achieve an advanced user experience.

[1877] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting real-time data from multiple subsystems of the vehicle, means for normalizing and filtering the collected data and inputting it to a generative artificial intelligence model, means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model, means for collecting user emotion data, analyzing it using an emotion engine, and outputting control commands to each subsystem based on the analysis results, means for analyzing user voice instructions using a voice recognition engine, generating control commands based on the analysis results, and transmitting them to the vehicle subsystems, means for monitoring the vehicle's real-time data, issuing an alert if a safety abnormality is detected, and automatically issuing necessary control commands, and means for automatically adjusting the entertainment system and air conditioning system according to the user's emotional state. This enables improvements in vehicle efficiency, safety, and user experience.

[1878] A "generative artificial intelligence model" is an artificial intelligence algorithm that generates optimal control commands based on collected data.

[1879] An "electronic control unit" is an electronic circuit board that comprehensively manages and controls each subsystem of a vehicle.

[1880] A "subsystem" is a separate functional unit in a vehicle, such as the engine, brakes, or entertainment system.

[1881] "Normalization" is the process of converting data into a form suitable for analysis.

[1882] "Filtering" is the process of cleaning data by removing noise and outliers.

[1883] An "emotion engine" is an algorithm that analyzes data such as the user's voice and facial expressions to recognize their emotional state.

[1884] A "voice recognition engine" is an algorithm that analyzes a user's voice, understands its content, and generates corresponding commands.

[1885] "Real-time data" refers to the latest data collected and analyzed at the current time.

[1886] A "control command" is an instruction to cause a vehicle subsystem to perform a specific operation.

[1887] "Emotional data" refers to data that indicates the user's emotional state, and includes voice, facial expression, heart rate, and the like.

[1888] An "entertainment system" is a system that provides content such as music and video in a vehicle.

[1889] An "air conditioning system" is a system that adjusts the temperature and humidity inside a vehicle.

[1890] "Abnormality detection" is the process of determining abnormalities when the vehicle's operation or state is different from normal.

[1891] This invention is a system that is equipped with a generative artificial intelligence model and an emotion engine, and that comprehensively manages and analyzes each subsystem of a vehicle in real time. Specific means and processes for implementing this invention will be described.

[1892] 1. System Initialization and Data Collection

[1893] When the engine of a vehicle equipped with an electronic control unit (ECU) starts, the server automatically starts the system and initializes all control systems. The server collects real-time information from multiple sensors in the vehicle, such as engine speed, brake pressure, fuel level, tire pressure, GPS location information, and entertainment system status.

[1894] Examples:

[1895] The unit activates when the vehicle's engine is started and collects information such as engine speed at 2000 rpm, brake pressure at 50 psi, fuel tank 80% full, tire pressure at 2.5 bar, GPS location at latitude 35.6895 degrees, longitude 139.6917 degrees, and whether the radio is on in the entertainment system.

[1896] 2. Data analysis and control command generation

[1897] The server normalizes and filters the collected raw data and inputs it into a generative AI model. The generative AI model analyzes the real-time data and generates optimal control commands. Based on the results, optimal control commands are output for each subsystem.

[1898] Examples:

[1899] The server generates a command to turn on the cooling fan if the engine temperature exceeds 90 degrees, and also outputs specific control commands such as adjusting engine speed to optimize fuel efficiency.

[1900] 3. Collecting and analyzing user emotion data

[1901] The device collects user emotional data through a microphone and camera. The server uses an emotion engine to analyze this data and recognize the user's emotions in real time. Based on the results, it generates appropriate control commands for each subsystem.

[1902] Examples:

[1903] The emotion engine detects when a user is stressed from their tone of voice and facial expression, and the server uses this information to send commands to the entertainment system to play relaxing music.

[1904] 4. Receiving and processing voice commands

[1905] The terminal collects the user's voice instructions through a microphone, and the server analyzes them using a voice recognition engine. Based on the analysis results, the server generates appropriate control commands and sends them to the corresponding subsystems.

[1906] Examples:

[1907] When a user gives a voice command such as "Set the air conditioner to 25 degrees," the server analyzes the voice and sends a command to the air conditioning system to set it to 25 degrees.

[1908] 5. Safety monitoring and control

[1909] The server continuously monitors real-time data to detect safety-related anomalies, immediately notifying the user with an alert and automatically issuing necessary control commands.

[1910] Examples:

[1911] If the server detects a sudden drop in tire pressure, it will sound an alarm to notify the user and issue a command to reduce engine power to safely slow down the vehicle.

[1912] The specific hardware and software used

[1913] Hardware:

[1914] Smartphone (camera, microphone)

[1915] Various sensors inside the vehicle (engine, brake, fuel, etc.)

[1916] software:

[1917] OpenCV (camera image processing)

[1918] Keras (emotion recognition model)

[1919] Pyautogui (system control)

[1920] Speech Recognition Engine

[1921] Example prompt sentence:

[1922] "Capture the user's face with a camera, and if the emotion recognition model detects 'sadness,' play relaxing music."

[1923] This will enable improvements in vehicle efficiency, safety and user experience.

[1924] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1925] Step 1:

[1926] The server automatically starts the system when the vehicle engine is started and collects real-time data from multiple sensors. Input data includes engine RPM, brake pressure, fuel level, tire pressure, GPS location, and entertainment system status. The collected raw data is stored on the server.

[1927] Step 2:

[1928] The server normalizes and filters the collected raw data. During normalization, the data values ​​are standardized and converted into a format suitable for analysis. During filtering, noise and outliers are removed. The resulting clean data is fed into the generative artificial intelligence model.

[1929] Step 3:

[1930] The server inputs clean data into a generative AI model and analyzes the real-time data. The model generates optimal control commands as a result of the analysis and returns them to the server. For example, it generates commands to adjust engine speed or turn cooling fans on and off.

[1931] Step 4:

[1932] The server sends the generated control commands to each subsystem to adjust the vehicle's operation. Based on the input control commands, specific operations are performed on the engine control module and the brake system. For example, a command to reduce fuel supply is sent to the engine control module.

[1933] Step 5:

[1934] The device collects the user's emotional data, using a microphone to collect voice data and a camera to capture facial expression data, which is then sent to a server in real time.

[1935] Step 6:

[1936] The server inputs the collected voice and facial expression data into the emotion engine to analyze the user's emotional state. The emotion engine processes this data and determines the emotion the user is feeling. For example, it may detect that the user is feeling stressed.

[1937] Step 7:

[1938] Based on the analysis results of the emotion engine, the server generates and sends optimal control commands to each subsystem, such as a command to play relaxing music to the entertainment system or an adjustment command to the air conditioning system.

[1939] Step 8:

[1940] The device collects the user's voice instructions through a microphone and sends them to the server. The server then analyzes the voice instructions using a voice recognition engine and generates control commands based on the analysis results. For example, if the user says, "Set the air conditioner to 25 degrees," a command to set the temperature to 25 degrees is sent to the air conditioning system.

[1941] Step 9:

[1942] The server monitors real-time vehicle data and detects safety-related abnormalities. If an abnormality is detected, it immediately notifies the user with a warning and automatically issues necessary control commands. For example, if tire pressure suddenly drops, a warning will sound and a command will be issued to reduce engine output to safely decelerate.

[1943] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1944] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1945] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1946] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1947] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1948] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1949] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1950] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1951] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1952] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1953] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1954] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1955] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1956] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1957] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1958] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1959] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1960] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1961] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1962] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1963] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1964] The following is further disclosed regarding the above embodiment.

[1965] (Claim 1)

[1966] an electronic control unit equipped with a generative artificial intelligence model and having means for collecting real-time data from multiple subsystems of a vehicle;

[1967] a means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model;

[1968] a means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model;

[1969] A system including:

[1970] (Claim 2)

[1971] 10. The system of claim 1, further comprising means for analyzing the user's voice instructions through a voice recognition engine, generating control commands based on the analysis results, and transmitting the control commands to the vehicle subsystems.

[1972] (Claim 3)

[1973] 10. The system of claim 1, further comprising means for monitoring real-time data of the vehicle, issuing an alert when a safety-related abnormality is detected, and automatically issuing necessary control commands.

[1974] "Example 1"

[1975] (Claim 1)

[1976] means for collecting real-time data from a plurality of subsystems of the vehicle;

[1977] a means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model;

[1978] a means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model;

[1979] means for automatically starting the entire system and initializing all control modules when the vehicle is powered on;

[1980] means for receiving a voice instruction from a user, generating a control command based on the instruction, and transmitting the control command to the subsystem;

[1981] A means for monitoring real-time data, detecting safety abnormalities, and issuing necessary control commands;

[1982] A system including:

[1983] (Claim 2)

[1984] 10. The system of claim 1, further comprising means for analyzing the user's voice instructions through a voice recognition engine, generating control commands based on the analysis results, and transmitting the control commands to the vehicle subsystems.

[1985] (Claim 3)

[1986] 10. The system of claim 1, further comprising means for monitoring real-time data of the vehicle, issuing an alert when a safety-related abnormality is detected, and automatically issuing necessary control commands.

[1987] "Application Example 1"

[1988] (Claim 1)

[1989] an electronic control unit equipped with a generative artificial intelligence model and having means for collecting real-time data from multiple subsystems of a vehicle;

[1990] a means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model;

[1991] a means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model;

[1992] A means of acquiring data from various sensors inside the vehicle via an API via a server,

[1993] A means for analyzing the acquired data and providing optimal driving advice;

[1994] means for collecting voice instructions and transmitting control commands to each subsystem based on the analysis results;

[1995] A system including:

[1996] (Claim 2)

[1997] 10. The system of claim 1, further comprising means for analyzing the user's voice instructions through a voice recognition engine, generating control commands based on the analysis results, and transmitting the control commands to the vehicle subsystems.

[1998] (Claim 3)

[1999] 10. The system of claim 1, further comprising means for monitoring real-time vehicle data and issuing an alert and automatically adjusting deceleration or fueling upon detecting a safety-related anomaly.

[2000] "Example 2: Combining Emotion Engines"

[2001] (Claim 1)

[2002] a means for collecting information in real time from multiple subsystems of the vehicle;

[2003] a means for normalizing and filtering the collected information and inputting it into a generative artificial intelligence model;

[2004] means for outputting optimal control commands to each subsystem based on the analysis results of the generative artificial intelligence model;

[2005] A means for collecting and analyzing a user's voice using a speech recognition engine;

[2006] a means for analyzing a user's emotions using an emotion engine and generating control commands based on the results of the analysis;

[2007] a means for monitoring safety-related anomalies in real time, issuing an alert when an anomaly is detected, and automatically generating control commands;

[2008] A system including:

[2009] (Claim 2)

[2010] 10. The system of claim ...

Claims

1. an electronic control unit equipped with a generative artificial intelligence model and having means for collecting real-time data from multiple subsystems of a vehicle; a means for normalizing and filtering the collected data and inputting it into a generative artificial intelligence model; a means for outputting control commands to each subsystem based on the results of analysis by the generative artificial intelligence model; A system including:

2. 10. The system according to claim 1, further comprising means for analyzing the user's voice instructions through a voice recognition engine, generating control commands based on the analysis results, and transmitting the control commands to the vehicle subsystems.

3. 10. The system according to claim 1, further comprising means for monitoring real-time data of the vehicle, issuing a warning when a safety-related abnormality is detected, and automatically issuing a necessary control command.

Images