system

Abstract

This system connects traffic lights and vehicles nationwide via a communication network, automatically disabling vehicle acceleration when the traffic light is red. [Solution] A means for acquiring signal information from traffic lights, A system comprising: means for transmitting the aforementioned signal information to a terminal installed in the vehicle; means for disabling the vehicle's accelerator control based on the aforementioned signal information; means for monitoring the vehicle's driving pattern; and means for slowing down or stopping the vehicle when an abnormal driving pattern is detected.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a transportation society, traffic accidents caused by driver human errors still maintain a high incidence rate. The main causes include reverse driving, overlooking red lights, misstepping on the accelerator and brake, drowsy driving, etc. Since these accidents cause serious problems directly related to human lives, conventional driving assistance systems and warning systems are not sufficient. Also, incorrect operations by the elderly and novice drivers remain an issue, and real-time accident prevention measures are required.

Means for Solving the Problems

[0005] This invention provides a system that connects traffic lights and vehicles nationwide via a communication network and automatically disables vehicle acceleration when the traffic light is red. The system includes means for acquiring signal information from traffic lights and transmitting this information to the vehicle's terminal, and further has a function to remotely disable acceleration control based on the received signal information. Vehicles are also equipped with means for continuously monitoring driving patterns, and if an abnormal driving pattern is detected using a generated AI model, the vehicle can be automatically decelerated or stopped. This contributes to accident prevention and improved safety. Furthermore, a special mechanism is provided to disable signal control for emergency vehicles, and the system also has a warning function. This multi-layered system makes it possible to prevent traffic accidents while enhancing driver safety.

[0006] A "traffic light" is a device on a road that serves as a traffic signal to direct vehicles and pedestrians to proceed, stop, or exercise caution.

[0007] "Signal information" refers to data that represents the state of the signal transmitted from a traffic light, and usually includes signal states such as red, yellow, and blue.

[0008] A "terminal" is a communication module installed in a vehicle to receive and analyze signal information, and it works in conjunction with the vehicle's electronic control system.

[0009] "Accelerator control" is a function that allows the vehicle's accelerator pedal to adjust the engine speed and vehicle speed.

[0010] A "driving pattern" is data that shows the behavior of a vehicle while it is in motion, and is based on information from multiple sensors, such as acceleration, vehicle speed, and direction.

[0011] A "generative AI model" is an algorithm designed to analyze driving patterns from large amounts of data using artificial intelligence and detect abnormal behavior.

[0012] "Emergency vehicles" are vehicles such as fire trucks, ambulances, and police vehicles that are permitted to operate under special circumstances and are exempt from normal traffic regulations by having specific identification signals. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that improves vehicle safety by utilizing signal information from traffic lights. To implement this invention, the operation of traffic lights, a communication terminal mounted on the vehicle, a server, and a generated AI model must be coordinated.

[0035] System configuration and operation

[0036] Acquisition of signal information

[0037] The server is connected to traffic lights nationwide and retrieves real-time signal information (red, yellow, and blue) transmitted from each traffic light. The retrieved signal information, along with the location information of the traffic lights, is stored in a database.

[0038] Notification to vehicle terminal

[0039] The server transmits signal information to the terminals of vehicles within the area affected by the signal, based on the location information of the traffic lights. The terminals receive this information and control the vehicles based on its contents.

[0040] Disabling accelerator control

[0041] If the terminal receives a red signal, it issues a command to the in-vehicle control system to disable accelerator control. This prevents unintended acceleration by the driver and ensures safety.

[0042] Monitoring and analysis of driving patterns

[0043] The terminal collects data in real time from various sensors installed in the vehicle (accelerometer, vehicle speed sensor, etc.) and transmits it to a generated AI model. The AI ​​model analyzes this data and detects abnormal driving patterns.

[0044] Warning and control of abnormal operation

[0045] If the generated AI model detects abnormal driving behavior (for example, frequent swerving or sudden acceleration), the server will issue a warning to the terminal of the vehicle in question and, if necessary, instruct it to slow down or stop.

[0046] Emergency vehicle response

[0047] Users (emergency vehicle operators) can operate without signal control by using special identification information. This information is sent from the terminal to the server, and once its legitimacy is confirmed, the restrictions are temporarily lifted.

[0048] Specific example

[0049] Example 1: Disabling a red light

[0050] The server acquires red light information from traffic lights in urban areas and transmits it to all vehicles within a 500-meter radius. The terminal receives this information and sends a command to the vehicle's in-vehicle system to disable the vehicle's accelerator.

[0051] Example 2: Detection of drowsy driving

[0052] The terminal uses a generated AI model to detect irregular weaving on straight roads. The server issues a warning, and if the driver is unaware, it commands the vehicle to slow down or stop.

[0053] This system is designed with the goal of preventing traffic accidents and ensuring driver safety, as described above. The implementation of this invention is easily adaptable to current traffic systems and has the potential to dramatically improve public safety.

[0054] The following describes the processing flow.

[0055] Step 1:

[0056] The server acquires signal information from traffic lights in real time. The signal status (red, yellow, blue) and geographical location data for each traffic light are recorded in a database, enabling immediate monitoring of signal changes.

[0057] Step 2:

[0058] The server sets a specific area of ​​influence based on the collected signal information and sends the relevant signal information to the terminals of all vehicles within that area. This determines which traffic signals a vehicle in a particular location should be affected by.

[0059] Step 3:

[0060] The terminal receives signal information transmitted from the server. If the information indicates a "red light," it issues a command to the vehicle's control system to disable accelerator operation, preparing to prevent driver error.

[0061] Step 4:

[0062] The terminal continuously acquires data from various sensors installed in the vehicle (accelerometer, direction sensor, etc.) and monitors the driving pattern based on that data. This information is sent to a generating AI model.

[0063] Step 5:

[0064] The server analyzes the received driving patterns through a generating AI model, and if abnormal driving behavior (e.g., frequent weaving or abnormal acceleration / deceleration) is detected, it issues a warning signal or sends an instruction to the vehicle's terminal to slow down or stop.

[0065] Step 6:

[0066] By entering specific authentication information into a terminal, the user (emergency vehicle operator) is excluded from normal traffic signal control. This allows emergency vehicles to operate smoothly without being affected by traffic signals.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] It is necessary to control vehicle operations based on information from signaling devices to improve safety. Furthermore, it is essential to detect abnormal driving, provide effective warnings, and take control as needed. Additionally, special vehicles require means to operate quickly and safely while avoiding normal signaling control.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes means for acquiring and transmitting data from a signaling device, means for disabling the vehicle's propulsion control, and means for analyzing abnormal driving patterns using an electronic model and issuing warnings as necessary. This enables safe control of vehicle operation based on signaling information, allows for early detection and appropriate response to abnormal driving, and supports the smooth operation of special vehicles.

[0072] "Data" refers to information obtained from signaling devices to control vehicle operations.

[0073] A "traffic signal device" is a device that displays traffic signals and provides information about them.

[0074] A "terminal device" is a device installed in a vehicle that receives and processes data from a server.

[0075] "Propulsion control" refers to the operation of the vehicle's accelerator pedal, and is a control system that adjusts the power output.

[0076] An "operation pattern" is a series of data that indicates the driving behavior of a vehicle.

[0077] An "electronic model" is an algorithm that analyzes driving data and detects anomalies.

[0078] A "special vehicle" refers to a vehicle that does not comply with normal traffic regulations, such as one used for responding to emergencies.

[0079] A "warning" is a signal issued to alert the user when an abnormal operating pattern is detected.

[0080] This invention provides a system that improves vehicle safety by utilizing data obtained from signaling devices. Implementation of the system requires the cooperation of a server, terminal devices mounted on vehicles, signaling devices, and a generating AI model. The server accesses the signaling device network and acquires signaling information in real time. This information, along with the location information of the signaling devices, is stored in a database.

[0081] Based on the acquired signal information, the server transmits the information to terminal devices installed in vehicles within the affected area. The terminal devices receive the signal information and perform actions such as disabling propulsion control and monitoring vehicle operation patterns. Furthermore, the terminal devices transmit data obtained from sensors installed in the vehicles to a generating AI model to detect abnormal operation patterns. The generating AI model analyzes this data, and if an anomaly is detected, it sends a warning to the vehicle via the server and slows down or stops the vehicle as necessary.

[0082] As a concrete example, a server acquires red light information from traffic signals in urban areas and transmits this information to all vehicles within a 500-meter radius. This disables accelerator operation in terminal devices, reducing the risk of accidents caused by accidental operation. In addition, a generated AI model analyzes driving data and detects irregular weaving that may indicate drowsy driving. In this case, the server issues a warning, and if the driver does not respond, it automatically slows down or stops.

[0083] For example, a prompt such as "Is the current vehicle speed exceeding the normal range?" is sent to the generating AI model, and the analysis result is obtained immediately. This allows the system to contribute to improved safety.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The server accesses the signaling device network to acquire data. It receives real-time red, yellow, and blue signal information from each signaling device as input, and combines this information with the signaling device's location information to store it in a database as output. This data processing geographically organizes the signaling information, preparing it for subsequent steps.

[0087] Step 2:

[0088] The server analyzes the acquired signal information and transmits it to terminal devices within the affected area. It uses the location of the signal device and vehicle information within a 500-meter radius as input, and transmits the signal information to the terminal devices of the relevant vehicles as output. The server distributes this information via wireless communication, allowing the vehicle's terminal to recognize the current signal status.

[0089] Step 3:

[0090] The terminal uses the received signal information to adjust the propulsion control within the vehicle. It uses the signal information received from the server as input and issues a command to the vehicle's engine control unit (ECU) to disable accelerator control as output. This operation prevents erroneous operation at red lights, improving safety.

[0091] Step 4:

[0092] The terminal collects operation pattern data from sensors installed in the vehicle. It receives real-time data from vehicle speed and acceleration sensors as input, and sends this data to a generating AI model as output. This allows the terminal to analyze detailed data on driving conditions.

[0093] Step 5:

[0094] The generative AI model analyzes the transmitted data and detects abnormal operating patterns. It uses driving data transmitted from the terminal as input and sends the results of the abnormal pattern detection to the server as output. The generative AI model utilizes advanced algorithms to analyze, for example, the possibility of drowsy driving.

[0095] Step 6:

[0096] The server receives analysis results from the generated AI model and sends warnings to terminals as needed. It receives notifications of abnormal pattern detection as input and distributes warning messages to the terminals of the relevant vehicles as output. Furthermore, the server takes immediate safety measures by issuing commands to slow down or stop, depending on the situation.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In recent years, with the proliferation of automobiles, the risk of traffic accidents has increased, and among these, accidents caused by running red lights and reckless driving have become a serious problem. Conventional warning systems only alert drivers, lacking specific vehicle control and real-time signal response. Therefore, there is a need for effective methods that utilize signal information to ensure the safe operation of vehicles.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for acquiring signal information, means for transmitting the signal information to an information processing device mounted on the vehicle, and means for disabling the vehicle's power system control based on the signal information. This makes it possible to prevent dangerous behaviors such as running red lights and to create a safe traffic environment.

[0102] "Signal information" refers to data about the status of each red, yellow, and blue signal transmitted from traffic lights, and is used to control the movement of vehicles.

[0103] An "information processing device" is a terminal installed in a vehicle that receives signal information and contributes to subsequent vehicle control.

[0104] "Means of disabling power system control" refers to a function that executes commands to temporarily stop the vehicle's acceleration or speed maintenance based on signal information.

[0105] "Driving characteristics" refer to the patterns of movement such as acceleration, deceleration, and direction changes that a vehicle exhibits while in operation.

[0106] "Abnormal driving characteristics" refer to irregular operations that differ from normal driving, sudden acceleration and deceleration, erratic driving, etc., and the resulting vehicle behavior that carries risks.

[0107] "Means for slowing down or stopping a vehicle" refers to a system that operates to reduce the speed of a vehicle or bring it to a complete stop when abnormal driving characteristics are detected.

[0108] "Location information" refers to the current geographical location of a vehicle, obtained using technologies such as GPS.

[0109] A "communication network" refers to a network system, such as the internet or dedicated lines, that enables external data communication.

[0110] The system realizing this invention comprehensively performs tasks such as acquiring signal information, notifying vehicles of this information, monitoring driving characteristics, detecting abnormal driving behavior, and controlling the vehicle. The aim is to significantly improve traffic safety.

[0111] The server first acquires real-time signal information from traffic lights in various locations and stores it in a database. Next, based on the location information, it transmits the signal information to information processing devices of vehicles within the range affected by the signal information. When these information processing devices receive the signal information, if it is a red light, they immediately disable the vehicle's power system control to prevent unintended acceleration by the driver.

[0112] Furthermore, the terminal constantly monitors the vehicle's driving characteristics. It acquires location information, acceleration, and other data based on GPS and sensor data, and sends it to a generated AI model. This AI model analyzes the collected data, and if abnormal driving characteristics are detected, the server immediately sends a command to the terminal to slow down or stop the vehicle.

[0113] For example, when a user approaches an intersection in their car, the server informs the terminal in advance of the red light, allowing the vehicle to automatically slow down and stop safely and smoothly. Furthermore, if a user becomes distracted due to fatigue during long-distance driving and exhibits abnormal driving characteristics, the AI ​​model detects this and automatically issues a warning.

[0114] An example of a prompt message is: "Based on the latest sensor data, detect abnormal driving patterns. Compare this with past data and set the system to issue a warning if erratic driving is detected." This can enhance driver safety and reduce the risk of traffic accidents.

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The server acquires traffic signal information in real time from traffic lights nationwide. It receives data from traffic lights as input, outputs this data as traffic signal status (red, yellow, green) and location information, and stores it in a database. This ensures that the traffic signal information is always up-to-date.

[0118] Step 2:

[0119] The server identifies vehicles within the affected area based on the location information of traffic signals. It uses vehicle location information as input and performs data calculations to identify vehicles within the area. As output, it generates a list of vehicles within the affected area and prepares to send the signal information to the terminal.

[0120] Step 3:

[0121] The terminal receives signal information transmitted from the server. It takes the signal information as input, and if the signal is red, it generates a command to disable the vehicle's power system control as output and notifies the in-vehicle control system. This action allows the vehicle to stop safely.

[0122] Step 4:

[0123] The terminal collects driving characteristics in real time from various sensors mounted on the vehicle. It acquires sensor data as input, aggregates the obtained data such as speed, acceleration, and direction, and outputs it. This data is sent to a generated AI model. This collected data is useful for safety verification.

[0124] Step 5:

[0125] The server uses a generated AI model to analyze sensor data and check for abnormal driving characteristics. It receives sensor data as input and determines whether or not abnormal driving is present as output. If an abnormal pattern (e.g., erratic driving) is detected through this analysis, the system can immediately proceed to the next action.

[0126] Step 6:

[0127] If abnormal operation is detected, the server sends a warning signal to the terminal of the vehicle in question. It takes the detection result of the abnormal pattern as input and sends warning information to the terminal as output. This immediately alerts the user and, if necessary, allows them to slow down or stop the vehicle.

[0128] Step 7:

[0129] If the user does not respond to the warning, the terminal takes control of the vehicle and returns it to a safe state. If data indicating a user response cannot be obtained as input, the terminal automatically executes a deceleration or stop command as output. This further enhances user safety.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] This invention is characterized by its inclusion of an emotion engine that recognizes user emotions, in addition to conventional systems that acquire signal information from traffic lights and provide driving assistance for vehicles. In implementing this invention, flexible driving assistance based on emotion recognition is introduced in addition to vehicle control based on signal information.

[0132] System configuration and operation

[0133] Acquisition and notification of signal information

[0134] The server acquires signal information from traffic lights in real time. Based on the signal information, it transmits it to vehicle terminals within the affected area. This disables the vehicle's accelerator control when the traffic light is red.

[0135] Recognition of user emotions by an emotion engine

[0136] The device uses cameras and sensors installed inside the vehicle to collect information such as the user's facial expressions, heart rate, and voice tone, and transmits this information to an emotion engine. This engine analyzes and recognizes the user's emotional state (e.g., stress, fatigue, relaxation).

[0137] Emotional state-based driving assistance

[0138] The device controls the vehicle's operation based on the recognized emotional state of the user. For example, if a high stress level is detected, it will play relaxing music or display a visual alert. Furthermore, if significant fatigue is detected, the system will consider automatically stopping the vehicle.

[0139] Monitoring of abnormal operation

[0140] The server monitors and analyzes driving patterns and takes action if abnormal driving is detected. This includes slowing down or stopping the vehicle and issuing warnings to the driver.

[0141] Specific example

[0142] Example 1: Driving assistance under high stress conditions

[0143] The device's emotion engine detected that the user was experiencing high levels of stress while driving. In response, it played music to help the user relax and displayed driving advice in a soft tone on the screen.

[0144] Example 2: Emergency shutdown due to emotion

[0145] Based on the user's facial expressions and physical reactions, the terminal's emotion engine determined that the user was experiencing extreme fatigue. Therefore, the server instructed the vehicle to stop at a safe location and encouraged the user to rest.

[0146] This system aims to prevent traffic accidents and ensure driver safety. The introduction of an emotional engine enables flexible support tailored to the individual user's condition, complementing conventional mechanical control and further improving safety.

[0147] The following describes the processing flow.

[0148] Step 1:

[0149] The server acquires signal information from traffic lights in real time. It receives signal status (red, yellow, blue) and traffic light location data, and records it in the relevant control database.

[0150] Step 2:

[0151] The server transmits the signal information to a vehicle terminal with specific location information, based on the signal information. This ensures that the vehicle properly receives real-time signal status.

[0152] Step 3:

[0153] The terminal analyzes the received signal information and, if the signal is red, sends a command to the vehicle's engine control system to disable accelerator control. This prevents unintended acceleration by the driver.

[0154] Step 4:

[0155] An emotion engine installed in the device monitors the user's state using in-car cameras and biosensors. This includes facial recognition, voice analysis, and even heart rate monitoring.

[0156] Step 5:

[0157] The emotion engine analyzes the user's emotional state and determines levels of stress, fatigue, and other factors. Based on the analysis, if it determines that the emotional state is affecting driving, it sends appropriate driving assistance or warnings to the device.

[0158] Step 6:

[0159] The device follows instructions from the emotion engine and performs actions according to the user's emotional state. This includes playing music to reduce stress and automatically stopping the vehicle if fatigue reaches its extreme.

[0160] Step 7:

[0161] The server continuously monitors the vehicle's driving patterns through a generated AI model. When abnormal driving behavior is detected, it issues a warning to the driver or provides control instructions to adjust the vehicle's speed or bring it to a complete stop.

[0162] This integrates the user's emotions with the vehicle's driving conditions, reducing the risk of accidents and providing a safe and comfortable driving environment.

[0163] (Example 2)

[0164] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0165] In modern transportation systems, there is a need to effectively mitigate the risk of traffic accidents caused by the driver's emotional state and driving environment. However, conventional systems are limited to mechanical control using signal information and cannot take into account the emotions, stress, or fatigue levels of individual drivers, making it difficult to provide flexible driving assistance that responds to diverse driving situations. Furthermore, it has been difficult to detect abnormal driving patterns early and take prompt corrective measures.

[0166] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0167] In this invention, the server includes means for acquiring signal data from a signaling device, means for transmitting the signal data to an information display device installed on the transport equipment, and means for analyzing the driver's emotional state using an emotion analysis device. This enables flexible driving support that takes into account the individual emotional state of the driver, and allows for the implementation of appropriate safety measures according to the driving environment. Furthermore, it makes it possible to detect abnormal driving at an early stage and improve traffic safety with appropriate countermeasures.

[0168] A "traffic signal system" is a system that uses light, sound, or other means to display signals and control the flow of traffic.

[0169] "Signal data" refers to information acquired from signaling equipment, including data on the status and fluctuations of signals.

[0170] "Transportation equipment" is a general term for vehicles and machinery used to transport people or goods.

[0171] An "information display device" is a display or monitor device installed in transportation equipment to provide visual information and instructions to the operator.

[0172] "Power control" refers to a control mechanism used to adjust the driving force of engines, motors, and other components of transportation equipment.

[0173] A "movement pattern" is a series of motion data that shows the trajectory, speed, and acceleration of a transport device.

[0174] An "abnormal movement pattern" is a pattern that includes sudden movements or unpredictable actions that are judged to be abnormal compared to normal movement.

[0175] An "emotion analysis device" is a system that recognizes and analyzes data such as facial expressions, voice, and vital signs in order to analyze the emotional state of the pilot.

[0176] "External stimuli" refer to environmental factors that affect the operator, such as music, lighting, and visual displays inside the transport vehicle.

[0177] This system is designed to provide advanced support for the operation of transportation equipment in traffic environments. Specifically, it utilizes signal data from traffic signaling devices and an emotion analysis device to understand the driver's emotional state, thereby providing flexible driving assistance.

[0178] The server acquires signal data from signaling equipment in real time. In this process, data is acquired via communication devices installed on traffic signals, using protocol XYZ. The server also transmits the signal data to information display devices installed on transportation equipment, thereby supporting safe driving by appropriately adjusting the power control of the transportation equipment.

[0179] The terminal uses cameras and sensors installed within the transport vehicle to collect emotional data such as the operator's facial expressions, heart rate, and voice tone. This data is sent to an emotion analysis device, where a generative AI model is used to analyze the operator's emotional state. Based on the analysis results, the terminal controls external stimuli (music, lighting, etc.) within the transport vehicle to reduce operator stress and promote safety.

[0180] For example, if the emotion analysis device determines that the driver is under high stress, the terminal will play relaxing classical music to create a calmer environment inside the car. Furthermore, if fatigue from prolonged driving is detected, an alert function will be used to encourage rest, and the vehicle can be automatically stopped if necessary.

[0181] An example of a prompt based on a generated AI model is, "Analyze the driver's emotional state while driving and suggest appropriate driving assistance measures."

[0182] By using such a system, we can contribute to preventing traffic accidents and improving driver safety, while also achieving advanced support that surpasses conventional driver assistance systems.

[0183] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0184] Step 1:

[0185] The server acquires signal data in real time from the signaling equipment. As input, it receives signal data (signal color, remaining time, etc.) transmitted from communication devices installed on the traffic signals. This data is acquired using protocol XYZ and stored internally by the server as the current state of the signal. As output, the signal data is sent to the next processing step.

[0186] Step 2:

[0187] The server transmits the acquired signal data to an information display device mounted on the transport vehicle. The input is the signal data obtained in step 1. The server analyzes the data and compares the results with the vehicle's location information. As output, detailed signal status information is transmitted to the information display device, and the signal status is displayed on the transport vehicle's display to inform the operator.

[0188] Step 3:

[0189] The terminal collects data using cameras and sensors installed within the transport vehicle to understand the operator's emotional state. Inputs include biometric data such as the operator's facial expressions, heart rate, and voice tone. An emotion analysis device and a generative AI model analyze this data to identify the operator's emotions (stress, fatigue, relaxation, etc.). The output is a report of the analyzed emotional state, which is then sent to the next step.

[0190] Step 4:

[0191] The terminal controls the transport equipment based on the analyzed emotional state. It receives an emotional state report obtained in step 3 as input. The terminal selects and implements appropriate external stimuli (music, lighting, etc.) to adjust environmental factors. Specifically, if stress is detected, it plays relaxing music and adjusts the lighting. The output is an environment designed to help the operator relax.

[0192] Step 5:

[0193] The server monitors the movement patterns transmitted from the transport equipment and detects abnormal movements. Inputs include driving data such as vehicle speed, acceleration, and location information. The server compares this data to baseline values ​​and automatically generates a warning if an anomaly is detected. As output, a warning message is sent to the transport equipment's terminal, and power control is performed as needed.

[0194] (Application Example 2)

[0195] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0196] Conventional mobile vehicle control systems are limited to simple motion control based on the acquisition of signal data, and lack adaptive support that takes into account the emotional state of the occupant. As a result, there was a problem in that the risk of accidents due to occupant stress and fatigue could not be reduced. The present invention aims to provide safer and more comfortable travel by realizing flexible support that takes into account the emotional state of the occupant.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0198] In this invention, the server includes means for acquiring signal data from a signal display device, means for transmitting the signal data to a device mounted on the mobile body, means for monitoring the operating pattern of the mobile body, means for recognizing the emotional state of the occupant, means for adjusting the internal environment of the mobile body according to the emotional state, and means for suggesting rest based on the emotional state. This enables safe control of the mobile body's operation and provides support tailored to the occupant's emotional state.

[0199] A "traffic signal display device" is a device that transmits signal data and is installed to regulate the flow of traffic.

[0200] "Signal data" refers to traffic control information transmitted from a signal display device, including data indicating proceed, stop, or caution.

[0201] "Mobile vehicles" refer to all vehicles and rideboats that travel on roads, primarily those that serve the function of transporting people or goods.

[0202] "Device" refers to various equipment and systems attached to a mobile object, primarily used for data processing and analysis.

[0203] "Propulsion control" refers to a control function used to adjust the speed and direction of a moving object.

[0204] A "motion pattern" refers to the tendency of a series of movements or behaviors that a moving object takes within a certain period of time.

[0205] An "abnormal operating pattern" refers to the movement or unnatural behavior of a moving object that differs from the normal operation and may cause problems for safe operation.

[0206] "Emotional state" refers to the mental and physiological state of the crew, and includes various emotions such as stress and relaxation.

[0207] "The internal environment of a mobile vehicle" refers to the physical or acoustic conditions inside a mobile vehicle and is a factor that affects the comfort of the occupants.

[0208] "Means for suggesting rest" refers to a function that informs the crew of their need for rest and suggests the most suitable method for doing so.

[0209] A system implementing this invention includes means for acquiring signal data from a signal display device and transmitting it to a vehicle to perform propulsion control of the moving object. A server within the system receives signal data from the signal display device in real time. The received data is sent to a device mounted on the moving object to disable propulsion control such as the accelerator. This ensures that the moving object operates safely in accordance with the signals.

[0210] Furthermore, the device uses in-car sensors and cameras to analyze the emotional state of the occupants. This analysis particularly utilizes facial expression analysis and heart rate monitoring, and the acquired data is sent to the emotion engine. The emotion engine uses this data to assess the occupants' stress and fatigue levels.

[0211] The device automatically adjusts the in-vehicle environment based on the occupant's emotional state. For example, if it detects high stress levels, it automatically plays relaxation music. Furthermore, if the emotional state indicates excessive fatigue, it suggests the next available resting point, encouraging the occupant to rest.

[0212] For example, if a passenger is experiencing stress during a long drive, the system will adjust the lighting in the car along with the music to create a comfortable environment. Furthermore, an example of a prompt for the generated AI model is: "Please analyze the current emotional state of the passenger using the camera feed and heart rate data. If high stress or fatigue is detected, initiate appropriate relaxation measures or suggest taking a rest break."

[0213] Thus, the present invention has specific embodiments that directly address the emotional state of the occupants and enhance comfort and safety within a mobile vehicle.

[0214] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0215] Step 1:

[0216] The server receives signal data transmitted from the signal display device in real time. It takes data from the signal display device as input and generates information indicating the current state of the signal as output. This information shows how the signal is displayed on the moving object.

[0217] Step 2:

[0218] The terminal receives signal data transmitted from the server and controls the acceleration of the moving object. The input is signal data from the server, and the output is a command for propulsion control. As a result, the speed of the moving object is controlled based on the signal.

[0219] Step 3:

[0220] The terminal collects occupant information using in-vehicle sensors and cameras. Inputs include facial expressions, heart rate, and other physiological data. Outputs generate analytical data sent to the emotion engine, which indicates the occupant's emotional state.

[0221] Step 4:

[0222] The emotion engine analyzes the emotional state of the occupants using data received from the terminal. Data from sensors and cameras is used as input. The output is an evaluation of the occupants' emotional state. Based on this result, the level of stress and fatigue is determined.

[0223] Step 5:

[0224] The device adjusts the environment within the mobile vehicle based on results from the emotion engine. It takes emotion evaluation results as input and generates environmental control signals as output. These signals are used to control actions such as playing relaxation music and adjusting lighting.

[0225] Step 6:

[0226] If the device determines that the user is fatigued, it will suggest taking a break. The fatigue level assessment result is used as input. The output is a notification prompting the user to rest. Specifically, it shows the user information about locations where they can rest.

[0227] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0228] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0229] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0230] [Second Embodiment]

[0231] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0232] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0233] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0234] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0235] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0236] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0237] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0238] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0239] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0240] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0241] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0242] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0243] This invention is a system that improves vehicle safety by utilizing signal information from traffic lights. To implement this invention, the operation of traffic lights, a communication terminal mounted on the vehicle, a server, and a generated AI model must be coordinated.

[0244] System configuration and operation

[0245] Acquisition of signal information

[0246] The server is connected to traffic lights nationwide and retrieves real-time signal information (red, yellow, and blue) transmitted from each traffic light. The retrieved signal information, along with the location information of the traffic lights, is stored in a database.

[0247] Notification to vehicle terminal

[0248] The server uses the location information of the traffic lights to transmit signal information to the terminals of vehicles within the affected area. The terminals receive this information and control the vehicles based on its contents.

[0249] Disabling accelerator control

[0250] If the terminal receives a red signal, it issues a command to the in-vehicle control system to disable accelerator control. This prevents unintended acceleration by the driver and ensures safety.

[0251] Monitoring and analysis of driving patterns

[0252] The terminal collects data in real time from various sensors installed in the vehicle (accelerometer, vehicle speed sensor, etc.) and transmits it to a generated AI model. The AI ​​model analyzes this data and detects abnormal driving patterns.

[0253] Warning and control of abnormal operation

[0254] If the generated AI model detects abnormal driving behavior (for example, frequent swerving or sudden acceleration), the server will issue a warning to the terminal of the vehicle in question and, if necessary, instruct it to slow down or stop.

[0255] Emergency vehicle response

[0256] Users (emergency vehicle operators) can operate without signal control by using special identification information. This information is sent from the terminal to the server, and once its legitimacy is confirmed, the restrictions are temporarily lifted.

[0257] Specific example

[0258] Example 1: Disabling a red light

[0259] The server acquires red light information from traffic lights in urban areas and transmits it to all vehicles within a 500-meter radius. The terminal receives this information and sends a command to the vehicle's in-vehicle system to disable the vehicle's accelerator.

[0260] Example 2: Detection of drowsy driving

[0261] The terminal uses a generated AI model to detect irregular weaving on straight roads. The server issues a warning, and if the driver is unaware, it commands the vehicle to slow down or stop.

[0262] This system is designed with the goal of preventing traffic accidents and ensuring driver safety, as described above. The implementation of this invention is easily adaptable to current traffic systems and has the potential to dramatically improve public safety.

[0263] The following describes the processing flow.

[0264] Step 1:

[0265] The server acquires signal information from traffic lights in real time. The signal status (red, yellow, blue) and geographical location data for each traffic light are recorded in a database, enabling immediate monitoring of signal changes.

[0266] Step 2:

[0267] The server sets a specific area of ​​influence based on the collected signal information and sends the relevant signal information to the terminals of all vehicles within that area. This determines which traffic signals a vehicle in a particular location should be affected by.

[0268] Step 3:

[0269] The terminal receives signal information transmitted from the server. If the information indicates a "red light," it issues a command to the vehicle's control system to disable accelerator operation, preparing to prevent driver error.

[0270] Step 4:

[0271] The terminal continuously acquires data from various sensors installed in the vehicle (accelerometer, direction sensor, etc.) and monitors the driving pattern based on that data. This information is sent to a generating AI model.

[0272] Step 5:

[0273] The server analyzes the received driving patterns through a generating AI model, and if abnormal driving behavior (e.g., frequent weaving or abnormal acceleration / deceleration) is detected, it issues a warning signal or sends an instruction to the vehicle's terminal to slow down or stop.

[0274] Step 6:

[0275] By entering specific authentication information into a terminal, the user (emergency vehicle operator) is excluded from normal traffic signal control. This allows emergency vehicles to operate smoothly without being affected by traffic signals.

[0276] (Example 1)

[0277] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0278] It is necessary to control vehicle operations based on information from signaling devices to improve safety. Furthermore, it is essential to detect abnormal driving, provide effective warnings, and take control as needed. Additionally, special vehicles require means to operate quickly and safely while avoiding normal signaling control.

[0279] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0280] In this invention, the server includes means for acquiring and transmitting data from a signal device, means for disabling the propulsion control of a vehicle, and means for analyzing an abnormal driving pattern with an electronic model and issuing a warning as necessary. As a result, safe control of vehicle operations based on signal information becomes possible, early detection of abnormal driving and appropriate responses can be achieved, and smooth operation of special vehicles can also be supported.

[0281] "Data" is information for controlling the operation of a vehicle acquired from a signal device.

[0282] "Signal device" is a device that displays traffic signals and provides that information.

[0283] "Terminal device" is a device mounted on a vehicle that receives and processes data from a server.

[0284] "Propulsion control" refers to the accelerator operation of a vehicle and is control for adjusting power.

[0285] "Operation pattern" is a series of data indicating the driving behavior of a vehicle.

[0286] "Electronic model" is an algorithm for analyzing driving data and detecting abnormalities.

[0287] "Special vehicle" refers to a vehicle that does not follow normal traffic rules for emergency response, etc.

[0288] "Warning" is a signal for raising awareness issued when an abnormal operation pattern is detected.

[0289] This invention provides a system for improving vehicle safety by utilizing data obtained from signal devices. For the implementation of the system, cooperation with a server, a terminal device mounted on a vehicle, a signal device, and a generated AI model is essential. The server accesses the signal device network and acquires signal information in real time. This information is stored in a database together with the position information of the signal device.

[0290] Based on the acquired signal information, the server transmits the information to terminal devices installed in vehicles within the affected area. The terminal devices receive the signal information and perform actions such as disabling propulsion control and monitoring vehicle operation patterns. Furthermore, the terminal devices transmit data obtained from sensors installed in the vehicles to a generating AI model to detect abnormal operation patterns. The generating AI model analyzes this data, and if an anomaly is detected, it sends a warning to the vehicle via the server and slows down or stops the vehicle as necessary.

[0291] As a concrete example, a server acquires red light information from traffic signals in urban areas and transmits this information to all vehicles within a 500-meter radius. This disables accelerator operation in terminal devices, reducing the risk of accidents caused by accidental operation. In addition, a generated AI model analyzes driving data and detects irregular weaving that may indicate drowsy driving. In this case, the server issues a warning, and if the driver does not respond, it automatically slows down or stops.

[0292] For example, a prompt such as "Is the current vehicle speed exceeding the normal range?" is sent to the generating AI model, and the analysis result is obtained immediately. This allows the system to contribute to improved safety.

[0293] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0294] Step 1:

[0295] The server accesses the signaling device network to acquire data. It receives real-time red, yellow, and blue signal information from each signaling device as input, and combines this information with the signaling device's location information to store it in a database as output. This data processing geographically organizes the signaling information, preparing it for subsequent steps.

[0296] Step 2:

[0297] The server analyzes the acquired signal information and transmits it to terminal devices within the affected area. It uses the location of the signal device and vehicle information within a 500-meter radius as input, and transmits the signal information to the terminal devices of the relevant vehicles as output. The server distributes this information via wireless communication, allowing the vehicle's terminal to recognize the current signal status.

[0298] Step 3:

[0299] The terminal uses the received signal information to adjust the propulsion control within the vehicle. It uses the signal information received from the server as input and issues a command to the vehicle's engine control unit (ECU) to disable accelerator control as output. This operation prevents erroneous operation at red lights, improving safety.

[0300] Step 4:

[0301] The terminal collects operation pattern data from sensors installed in the vehicle. It receives real-time data from vehicle speed and acceleration sensors as input, and sends this data to a generating AI model as output. This allows the terminal to analyze detailed data on driving conditions.

[0302] Step 5:

[0303] The generative AI model analyzes the transmitted data and detects abnormal operating patterns. It uses driving data transmitted from the terminal as input and sends the results of the abnormal pattern detection to the server as output. The generative AI model utilizes advanced algorithms to analyze, for example, the possibility of drowsy driving.

[0304] Step 6:

[0305] The server receives analysis results from the generated AI model and sends warnings to terminals as needed. It receives notifications of abnormal pattern detection as input and distributes warning messages to the terminals of the relevant vehicles as output. Furthermore, the server takes immediate safety measures by issuing commands to slow down or stop, depending on the situation.

[0306] (Application Example 1)

[0307] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0308] In recent years, with the popularization of automobiles, the risk of traffic accidents has been increasing. Among them, accidents caused by ignoring signals and abnormal driving have become serious problems. Conventional warning systems only prompt the driver's attention and lack specific vehicle control and real-time signal response. Therefore, an effective method for utilizing signal information to ensure the safe operation of vehicles is required.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0310] In this invention, the server includes means for acquiring signal information, means for transmitting the signal information to an information processing device mounted on a vehicle, and means for invalidating the power train control of the vehicle based on the signal information. Thereby, dangerous behaviors such as ignoring signals can be prevented, and a safe traffic environment can be constructed.

[0311] "Signal information" is data regarding the states of the red, yellow, and blue signals transmitted from a traffic signal, and is used to control the progress of a vehicle.

[0312] "Information processing device" is a terminal mounted on a vehicle that receives signal information and contributes to subsequent vehicle control.

[0313] "Means for invalidating the power train control" refers to a function for executing an instruction to temporarily stop the acceleration or speed maintenance of a vehicle based on signal information.

[0314] "Driving characteristics" refer to the operation patterns such as acceleration, deceleration, and direction change shown by a vehicle during operation.

[0315] "Abnormal driving characteristics" refer to irregular operations that differ from normal driving, sudden acceleration and deceleration, erratic driving, etc., and the resulting vehicle behavior that carries risks.

[0316] "Means for slowing down or stopping a vehicle" refers to a system that operates to reduce the speed of a vehicle or bring it to a complete stop when abnormal driving characteristics are detected.

[0317] "Location information" refers to the current geographical location of a vehicle, obtained using technologies such as GPS.

[0318] A "communication network" refers to a network system, such as the internet or dedicated lines, that enables external data communication.

[0319] The system realizing this invention comprehensively performs tasks such as acquiring signal information, notifying vehicles of this information, monitoring driving characteristics, detecting abnormal driving behavior, and controlling the vehicle. The aim is to significantly improve traffic safety.

[0320] The server first acquires real-time signal information from traffic lights in various locations and stores it in a database. Next, based on the location information, it transmits the signal information to information processing devices of vehicles within the range affected by the signal information. When these information processing devices receive the signal information, if it is a red light, they immediately disable the vehicle's power system control to prevent unintended acceleration by the driver.

[0321] Furthermore, the terminal constantly monitors the vehicle's driving characteristics. It acquires location information, acceleration, and other data based on GPS and sensor data, and sends it to a generated AI model. This AI model analyzes the collected data, and if abnormal driving characteristics are detected, the server immediately sends a command to the terminal to slow down or stop the vehicle.

[0322] For example, when a user approaches an intersection in their car, the server informs the terminal in advance of the red light, allowing the vehicle to automatically slow down and stop safely and smoothly. Furthermore, if a user becomes distracted due to fatigue during long-distance driving and exhibits abnormal driving characteristics, the AI ​​model detects this and automatically issues a warning.

[0323] An example of a prompt message is: "Based on the latest sensor data, detect abnormal driving patterns. Compare this with past data and set the system to issue a warning if erratic driving is detected." This can enhance driver safety and reduce the risk of traffic accidents.

[0324] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0325] Step 1:

[0326] The server acquires traffic signal information in real time from traffic lights nationwide. It receives data from traffic lights as input, outputs this data as traffic signal status (red, yellow, green) and location information, and stores it in a database. This ensures that the traffic signal information is always up-to-date.

[0327] Step 2:

[0328] The server identifies vehicles within the affected area based on the location information of traffic signals. It uses vehicle location information as input and performs data calculations to identify vehicles within the area. As output, it generates a list of vehicles within the affected area and prepares to send the signal information to the terminal.

[0329] Step 3:

[0330] The terminal receives signal information transmitted from the server. It takes the signal information as input, and if the signal is red, it generates a command to disable the vehicle's power system control as output and notifies the in-vehicle control system. This action allows the vehicle to stop safely.

[0331] Step 4:

[0332] The terminal collects driving characteristics in real time from various sensors mounted on the vehicle. It acquires sensor data as input, aggregates the obtained data such as speed, acceleration, and direction, and outputs it. This data is sent to a generated AI model. This collected data is useful for safety verification.

[0333] Step 5:

[0334] The server uses a generated AI model to analyze sensor data and check for abnormal driving characteristics. It receives sensor data as input and determines whether or not abnormal driving is present as output. If an abnormal pattern (e.g., erratic driving) is detected through this analysis, the system can immediately proceed to the next action.

[0335] Step 6:

[0336] If abnormal operation is detected, the server sends a warning signal to the terminal of the vehicle in question. It takes the detection result of the abnormal pattern as input and sends warning information to the terminal as output. This immediately alerts the user and, if necessary, allows them to slow down or stop the vehicle.

[0337] Step 7:

[0338] If the user does not respond to the warning, the terminal takes control of the vehicle and returns it to a safe state. If data indicating a user response cannot be obtained as input, the terminal automatically executes a deceleration or stop command as output. This further enhances user safety.

[0339] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0340] This invention is characterized by its inclusion of an emotion engine that recognizes user emotions, in addition to conventional systems that acquire signal information from traffic lights and provide driving assistance for vehicles. In implementing this invention, flexible driving assistance based on emotion recognition is introduced in addition to vehicle control based on signal information.

[0341] System configuration and operation

[0342] Acquisition and notification of signal information

[0343] The server acquires signal information from traffic lights in real time. Based on the signal information, it transmits it to vehicle terminals within the affected area. This disables the vehicle's accelerator control when the traffic light is red.

[0344] Recognition of user emotions by an emotion engine

[0345] The device uses cameras and sensors installed inside the vehicle to collect information such as the user's facial expressions, heart rate, and voice tone, and transmits this information to an emotion engine. This engine analyzes and recognizes the user's emotional state (e.g., stress, fatigue, relaxation).

[0346] Emotional state-based driving assistance

[0347] The device controls the vehicle's operation based on the recognized emotional state of the user. For example, if a high stress level is detected, it will play relaxing music or display a visual alert. Furthermore, if significant fatigue is detected, the system will consider automatically stopping the vehicle.

[0348] Monitoring of abnormal operation

[0349] The server monitors and analyzes driving patterns and takes action if abnormal driving is detected. This includes slowing down or stopping the vehicle and issuing warnings to the driver.

[0350] Specific example

[0351] Example 1: Driving assistance under high stress conditions

[0352] The device's emotion engine detected that the user was experiencing high levels of stress while driving. In response, it played music to help the user relax and displayed driving advice in a soft tone on the screen.

[0353] Example 2: Emergency shutdown due to emotion

[0354] Based on the user's facial expressions and physical reactions, the terminal's emotion engine determined that the user was experiencing extreme fatigue. Therefore, the server instructed the vehicle to stop at a safe location and encouraged the user to rest.

[0355] This system aims to prevent traffic accidents and ensure driver safety. The introduction of an emotional engine enables flexible support tailored to the individual user's condition, complementing conventional mechanical control and further improving safety.

[0356] The following describes the processing flow.

[0357] Step 1:

[0358] The server acquires signal information from traffic lights in real time. It receives signal status (red, yellow, blue) and traffic light location data, and records it in the relevant control database.

[0359] Step 2:

[0360] The server transmits the signal information to a vehicle terminal with specific location information, based on the signal information. This ensures that the vehicle properly receives real-time signal status.

[0361] Step 3:

[0362] The terminal analyzes the received signal information and, if the signal is red, sends a command to the vehicle's engine control system to disable accelerator control. This prevents unintended acceleration by the driver.

[0363] Step 4:

[0364] An emotion engine installed in the device monitors the user's state using in-car cameras and biosensors. This includes facial recognition, voice analysis, and even heart rate monitoring.

[0365] Step 5:

[0366] The emotion engine analyzes the user's emotional state and determines levels of stress, fatigue, and other factors. Based on the analysis, if it determines that the emotional state is affecting driving, it sends appropriate driving assistance or warnings to the device.

[0367] Step 6:

[0368] The device follows instructions from the emotion engine and performs actions according to the user's emotional state. This includes playing music to reduce stress and automatically stopping the vehicle if fatigue reaches its extreme.

[0369] Step 7:

[0370] The server continuously monitors the vehicle's driving patterns through a generated AI model. When abnormal driving behavior is detected, it issues a warning to the driver or provides control instructions to adjust the vehicle's speed or bring it to a complete stop.

[0371] This integrates the user's emotions with the vehicle's driving conditions, reducing the risk of accidents and providing a safe and comfortable driving environment.

[0372] (Example 2)

[0373] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0374] In modern transportation systems, there is a need to effectively mitigate the risk of traffic accidents caused by the driver's emotional state and driving environment. However, conventional systems are limited to mechanical control using signal information and cannot take into account the emotions, stress, or fatigue levels of individual drivers, making it difficult to provide flexible driving assistance that responds to diverse driving situations. Furthermore, it has been difficult to detect abnormal driving patterns early and take prompt corrective measures.

[0375] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0376] In this invention, the server includes means for acquiring signal data from a signaling device, means for transmitting the signal data to an information display device installed on the transport equipment, and means for analyzing the driver's emotional state using an emotion analysis device. This enables flexible driving support that takes into account the individual emotional state of the driver, and allows for the implementation of appropriate safety measures according to the driving environment. Furthermore, it makes it possible to detect abnormal driving at an early stage and improve traffic safety with appropriate countermeasures.

[0377] A "traffic signal system" is a system that uses light, sound, or other means to display signals and control the flow of traffic.

[0378] "Signal data" refers to information acquired from signaling equipment, including data on the status and fluctuations of signals.

[0379] "Transportation equipment" is a general term for vehicles and machinery used to transport people or goods.

[0380] An "information display device" is a display or monitor device installed in transportation equipment to provide visual information and instructions to the operator.

[0381] "Power control" refers to a control mechanism used to adjust the driving force of engines, motors, and other components of transportation equipment.

[0382] A "movement pattern" is a series of motion data that shows the trajectory, speed, and acceleration of a transport device.

[0383] An "abnormal movement pattern" is a pattern that includes sudden movements or unpredictable actions that are judged to be abnormal compared to normal movement.

[0384] An "emotion analysis device" is a system that recognizes and analyzes data such as facial expressions, voice, and vital signs in order to analyze the emotional state of the pilot.

[0385] "External stimuli" refer to environmental factors that affect the operator, such as music, lighting, and visual displays inside the transport vehicle.

[0386] This system is designed to provide advanced support for the operation of transportation equipment in traffic environments. Specifically, it utilizes signal data from traffic signaling devices and an emotion analysis device to understand the driver's emotional state, thereby providing flexible driving assistance.

[0387] The server acquires signal data from signaling equipment in real time. In this process, data is acquired via communication devices installed on traffic signals, using protocol XYZ. The server also transmits the signal data to information display devices installed on transportation equipment, thereby supporting safe driving by appropriately adjusting the power control of the transportation equipment.

[0388] The terminal uses cameras and sensors installed within the transport vehicle to collect emotional data such as the operator's facial expressions, heart rate, and voice tone. This data is sent to an emotion analysis device, where a generative AI model is used to analyze the operator's emotional state. Based on the analysis results, the terminal controls external stimuli (music, lighting, etc.) within the transport vehicle to reduce operator stress and promote safety.

[0389] For example, if the emotion analysis device determines that the driver is under high stress, the terminal will play relaxing classical music to create a calmer environment inside the car. Furthermore, if fatigue from prolonged driving is detected, an alert function will be used to encourage rest, and the vehicle can be automatically stopped if necessary.

[0390] An example of a prompt based on a generated AI model is, "Analyze the driver's emotional state while driving and suggest appropriate driving assistance measures."

[0391] By using such a system, we can contribute to preventing traffic accidents and improving driver safety, while also achieving advanced support that surpasses conventional driver assistance systems.

[0392] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0393] Step 1:

[0394] The server acquires signal data in real time from the signaling equipment. As input, it receives signal data (signal color, remaining time, etc.) transmitted from communication devices installed on the traffic signals. This data is acquired using protocol XYZ and stored internally by the server as the current state of the signal. As output, the signal data is sent to the next processing step.

[0395] Step 2:

[0396] The server transmits the acquired signal data to an information display device mounted on the transport vehicle. The input is the signal data obtained in step 1. The server analyzes the data and compares the results with the vehicle's location information. As output, detailed signal status information is transmitted to the information display device, and the signal status is displayed on the transport vehicle's display to inform the operator.

[0397] Step 3:

[0398] The terminal collects data using cameras and sensors installed within the transport vehicle to understand the operator's emotional state. Inputs include biometric data such as the operator's facial expressions, heart rate, and voice tone. An emotion analysis device and a generative AI model analyze this data to identify the operator's emotions (stress, fatigue, relaxation, etc.). The output is a report of the analyzed emotional state, which is then sent to the next step.

[0399] Step 4:

[0400] The terminal controls the transport equipment based on the analyzed emotional state. It receives an emotional state report obtained in step 3 as input. The terminal selects and implements appropriate external stimuli (music, lighting, etc.) to adjust environmental factors. Specifically, if stress is detected, it plays relaxing music and adjusts the lighting. The output is an environment designed to help the operator relax.

[0401] Step 5:

[0402] The server monitors the movement patterns transmitted from the transport equipment and detects abnormal movements. Inputs include driving data such as vehicle speed, acceleration, and location information. The server compares this data to baseline values ​​and automatically generates a warning if an anomaly is detected. As output, a warning message is sent to the transport equipment's terminal, and power control is performed as needed.

[0403] (Application Example 2)

[0404] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0405] Conventional mobile vehicle control systems are limited to simple motion control based on the acquisition of signal data, and lack adaptive support that takes into account the emotional state of the occupant. As a result, there was a problem in that the risk of accidents due to occupant stress and fatigue could not be reduced. The present invention aims to provide safer and more comfortable travel by realizing flexible support that takes into account the emotional state of the occupant.

[0406] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0407] In this invention, the server includes means for acquiring signal data from a signal display device, means for transmitting the signal data to a device mounted on the mobile body, means for monitoring the operating pattern of the mobile body, means for recognizing the emotional state of the occupant, means for adjusting the internal environment of the mobile body according to the emotional state, and means for suggesting rest based on the emotional state. This enables safe control of the mobile body's operation and provides support tailored to the occupant's emotional state.

[0408] A "traffic signal display device" is a device that transmits signal data and is installed to regulate the flow of traffic.

[0409] "Signal data" refers to traffic control information transmitted from a signal display device, including data indicating proceed, stop, or caution.

[0410] "Mobile vehicles" refer to all vehicles and rideboats that travel on roads, primarily those that serve the function of transporting people or goods.

[0411] "Device" refers to various equipment and systems attached to a mobile object, primarily used for data processing and analysis.

[0412] "Propulsion control" refers to a control function used to adjust the speed and direction of a moving object.

[0413] A "motion pattern" refers to the tendency of a series of movements or behaviors that a moving object takes within a certain period of time.

[0414] An "abnormal operating pattern" refers to the movement or unnatural behavior of a moving object that differs from the normal operation and may cause problems for safe operation.

[0415] "Emotional state" refers to the mental and physiological state of the crew, and includes various emotions such as stress and relaxation.

[0416] "The internal environment of a mobile vehicle" refers to the physical or acoustic conditions inside a mobile vehicle and is a factor that affects the comfort of the occupants.

[0417] "Means for suggesting rest" refers to a function that informs the crew of their need for rest and suggests the most suitable method for doing so.

[0418] A system implementing this invention includes means for acquiring signal data from a signal display device and transmitting it to a vehicle to perform propulsion control of the moving object. A server within the system receives signal data from the signal display device in real time. The received data is sent to a device mounted on the moving object to disable propulsion control such as the accelerator. This ensures that the moving object operates safely in accordance with the signals.

[0419] Furthermore, the device uses in-car sensors and cameras to analyze the emotional state of the occupants. This analysis particularly utilizes facial expression analysis and heart rate monitoring, and the acquired data is sent to the emotion engine. The emotion engine uses this data to assess the occupants' stress and fatigue levels.

[0420] The device automatically adjusts the in-vehicle environment based on the occupant's emotional state. For example, if it detects high stress levels, it automatically plays relaxation music. Furthermore, if the emotional state indicates excessive fatigue, it suggests the next available resting point, encouraging the occupant to rest.

[0421] For example, if a passenger is experiencing stress during a long drive, the system will adjust the lighting in the car along with the music to create a comfortable environment. Furthermore, an example of a prompt for the generated AI model is: "Please analyze the current emotional state of the passenger using the camera feed and heart rate data. If high stress or fatigue is detected, initiate appropriate relaxation measures or suggest taking a rest break."

[0422] Thus, the present invention has specific embodiments that directly address the emotional state of the occupants and enhance comfort and safety within a mobile vehicle.

[0423] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0424] Step 1:

[0425] The server receives signal data transmitted from the signal display device in real time. It takes data from the signal display device as input and generates information indicating the current state of the signal as output. This information shows how the signal is displayed on the moving object.

[0426] Step 2:

[0427] The terminal receives signal data transmitted from the server and controls the acceleration of the moving object. The input is signal data from the server, and the output is a command for propulsion control. As a result, the speed of the moving object is controlled based on the signal.

[0428] Step 3:

[0429] The terminal collects occupant information using in-vehicle sensors and cameras. Inputs include facial expressions, heart rate, and other physiological data. Outputs generate analytical data sent to the emotion engine, which indicates the occupant's emotional state.

[0430] Step 4:

[0431] The emotion engine analyzes the emotional state of the occupants using data received from the terminal. Data from sensors and cameras is used as input. The output is an evaluation of the occupants' emotional state. Based on this result, the level of stress and fatigue is determined.

[0432] Step 5:

[0433] The device adjusts the environment within the mobile vehicle based on results from the emotion engine. It takes emotion evaluation results as input and generates environmental control signals as output. These signals are used to control actions such as playing relaxation music and adjusting lighting.

[0434] Step 6:

[0435] If the device determines that the user is fatigued, it will suggest taking a break. The fatigue level assessment result is used as input. The output is a notification prompting the user to rest. Specifically, it shows the user information about locations where they can rest.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0437] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0439] [Third Embodiment]

[0440] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0441] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0442] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0444] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0446] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0447] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0448] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0451] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0452] This invention is a system that improves vehicle safety by utilizing signal information from traffic lights. To implement this invention, the operation of traffic lights, a communication terminal mounted on the vehicle, a server, and a generated AI model must be coordinated.

[0453] System configuration and operation

[0454] Acquisition of signal information

[0455] The server is connected to traffic lights nationwide and retrieves real-time signal information (red, yellow, and blue) transmitted from each traffic light. The retrieved signal information, along with the location information of the traffic lights, is stored in a database.

[0456] Notification to vehicle terminal

[0457] The server uses the location information of the traffic lights to transmit signal information to the terminals of vehicles within the affected area. The terminals receive this information and control the vehicles based on its contents.

[0458] Disabling accelerator control

[0459] If the terminal receives a red signal, it issues a command to the in-vehicle control system to disable accelerator control. This prevents unintended acceleration by the driver and ensures safety.

[0460] Monitoring and analysis of driving patterns

[0461] The terminal collects data in real time from various sensors installed in the vehicle (accelerometer, vehicle speed sensor, etc.) and transmits it to a generated AI model. The AI ​​model analyzes this data and detects abnormal driving patterns.

[0462] Warning and control of abnormal operation

[0463] If the generated AI model detects abnormal driving behavior (for example, frequent swerving or sudden acceleration), the server will issue a warning to the terminal of the vehicle in question and, if necessary, instruct it to slow down or stop.

[0464] Emergency vehicle response

[0465] Users (emergency vehicle operators) can operate without signal control by using special identification information. This information is sent from the terminal to the server, and once its legitimacy is confirmed, the restrictions are temporarily lifted.

[0466] Specific example

[0467] Example 1: Disabling a red light

[0468] The server acquires red light information from traffic lights in urban areas and transmits it to all vehicles within a 500-meter radius. The terminal receives this information and sends a command to the vehicle's in-vehicle system to disable the vehicle's accelerator.

[0469] Example 2: Detection of drowsy driving

[0470] The terminal uses a generated AI model to detect irregular weaving on straight roads. The server issues a warning, and if the driver is unaware, it commands the vehicle to slow down or stop.

[0471] This system is designed with the goal of preventing traffic accidents and ensuring driver safety, as described above. The implementation of this invention is easily adaptable to current traffic systems and has the potential to dramatically improve public safety.

[0472] The following describes the processing flow.

[0473] Step 1:

[0474] The server acquires signal information from traffic lights in real time. The signal status (red, yellow, blue) and geographical location data for each traffic light are recorded in a database, enabling immediate monitoring of signal changes.

[0475] Step 2:

[0476] The server sets a specific area of ​​influence based on the collected signal information and sends the relevant signal information to the terminals of all vehicles within that area. This determines which traffic signals a vehicle in a particular location should be affected by.

[0477] Step 3:

[0478] The terminal receives signal information transmitted from the server. If the information indicates a "red light," it issues a command to the vehicle's control system to disable accelerator operation, preparing to prevent driver error.

[0479] Step 4:

[0480] The terminal continuously acquires data from various sensors installed in the vehicle (accelerometer, direction sensor, etc.) and monitors the driving pattern based on that data. This information is sent to a generating AI model.

[0481] Step 5:

[0482] The server analyzes the received driving patterns through a generating AI model, and if abnormal driving behavior (e.g., frequent weaving or abnormal acceleration / deceleration) is detected, it issues a warning signal or sends an instruction to the vehicle's terminal to slow down or stop.

[0483] Step 6:

[0484] By entering specific authentication information into a terminal, the user (emergency vehicle operator) is excluded from normal traffic signal control. This allows emergency vehicles to operate smoothly without being affected by traffic signals.

[0485] (Example 1)

[0486] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0487] It is necessary to control vehicle operations based on information from signaling devices to improve safety. Furthermore, it is essential to detect abnormal driving, provide effective warnings, and take control as needed. Additionally, special vehicles require means to operate quickly and safely while avoiding normal signaling control.

[0488] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0489] In this invention, the server includes means for acquiring and transmitting data from a signaling device, means for disabling the vehicle's propulsion control, and means for analyzing abnormal driving patterns using an electronic model and issuing warnings as necessary. This enables safe control of vehicle operation based on signaling information, allows for early detection and appropriate response to abnormal driving, and supports the smooth operation of special vehicles.

[0490] "Data" refers to information obtained from signaling devices to control vehicle operations.

[0491] A "traffic signal device" is a device that displays traffic signals and provides information about them.

[0492] A "terminal device" is a device installed in a vehicle that receives and processes data from a server.

[0493] "Propulsion control" refers to the operation of the vehicle's accelerator pedal, and is a control system that adjusts the power output.

[0494] An "operation pattern" is a series of data that indicates the driving behavior of a vehicle.

[0495] An "electronic model" is an algorithm that analyzes driving data and detects anomalies.

[0496] A "special vehicle" refers to a vehicle that does not comply with normal traffic regulations, such as one used for responding to emergencies.

[0497] A "warning" is a signal issued to alert the user when an abnormal operating pattern is detected.

[0498] This invention provides a system that improves vehicle safety by utilizing data obtained from signaling devices. Implementation of the system requires the cooperation of a server, terminal devices mounted on vehicles, signaling devices, and a generating AI model. The server accesses the signaling device network and acquires signaling information in real time. This information, along with the location information of the signaling devices, is stored in a database.

[0499] Based on the acquired signal information, the server transmits the information to terminal devices installed in vehicles within the affected area. The terminal devices receive the signal information and perform actions such as disabling propulsion control and monitoring vehicle operation patterns. Furthermore, the terminal devices transmit data obtained from sensors installed in the vehicles to a generating AI model to detect abnormal operation patterns. The generating AI model analyzes this data, and if an anomaly is detected, it sends a warning to the vehicle via the server and slows down or stops the vehicle as necessary.

[0500] As a concrete example, a server acquires red light information from traffic signals in urban areas and transmits this information to all vehicles within a 500-meter radius. This disables accelerator operation in terminal devices, reducing the risk of accidents caused by accidental operation. In addition, a generated AI model analyzes driving data and detects irregular weaving that may indicate drowsy driving. In this case, the server issues a warning, and if the driver does not respond, it automatically slows down or stops.

[0501] For example, a prompt such as "Is the current vehicle speed exceeding the normal range?" is sent to the generating AI model, and the analysis result is obtained immediately. This allows the system to contribute to improved safety.

[0502] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0503] Step 1:

[0504] The server accesses the signaling device network to acquire data. It receives real-time red, yellow, and blue signal information from each signaling device as input, and combines this information with the signaling device's location information to store it in a database as output. This data processing geographically organizes the signaling information, preparing it for subsequent steps.

[0505] Step 2:

[0506] The server analyzes the acquired signal information and transmits it to terminal devices within the affected area. It uses the location of the signal device and vehicle information within a 500-meter radius as input, and transmits the signal information to the terminal devices of the relevant vehicles as output. The server distributes this information via wireless communication, allowing the vehicle's terminal to recognize the current signal status.

[0507] Step 3:

[0508] The terminal uses the received signal information to adjust the propulsion control within the vehicle. It uses the signal information received from the server as input and issues a command to the vehicle's engine control unit (ECU) to disable accelerator control as output. This operation prevents erroneous operation at red lights, improving safety.

[0509] Step 4:

[0510] The terminal collects operation pattern data from sensors installed in the vehicle. It receives real-time data from vehicle speed and acceleration sensors as input, and sends this data to a generating AI model as output. This allows the terminal to analyze detailed data on driving conditions.

[0511] Step 5:

[0512] The generative AI model analyzes the transmitted data and detects abnormal operating patterns. It uses driving data transmitted from the terminal as input and sends the results of the abnormal pattern detection to the server as output. The generative AI model utilizes advanced algorithms to analyze, for example, the possibility of drowsy driving.

[0513] Step 6:

[0514] The server receives analysis results from the generated AI model and sends warnings to terminals as needed. It receives notifications of abnormal pattern detection as input and distributes warning messages to the terminals of the relevant vehicles as output. Furthermore, the server takes immediate safety measures by issuing commands to slow down or stop, depending on the situation.

[0515] (Application Example 1)

[0516] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0517] In recent years, with the proliferation of automobiles, the risk of traffic accidents has increased, and among these, accidents caused by running red lights and reckless driving have become a serious problem. Conventional warning systems only alert drivers, lacking specific vehicle control and real-time signal response. Therefore, there is a need for effective methods that utilize signal information to ensure the safe operation of vehicles.

[0518] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0519] In this invention, the server includes means for acquiring signal information, means for transmitting the signal information to an information processing device mounted on the vehicle, and means for disabling the vehicle's power system control based on the signal information. This makes it possible to prevent dangerous behaviors such as running red lights and to create a safe traffic environment.

[0520] "Signal information" refers to data about the status of each red, yellow, and blue signal transmitted from traffic lights, and is used to control the movement of vehicles.

[0521] An "information processing device" is a terminal installed in a vehicle that receives signal information and contributes to subsequent vehicle control.

[0522] "Means of disabling power system control" refers to a function that executes commands to temporarily stop the vehicle's acceleration or speed maintenance based on signal information.

[0523] "Driving characteristics" refer to the patterns of movement such as acceleration, deceleration, and direction changes that a vehicle exhibits while in operation.

[0524] "Abnormal driving characteristics" refer to irregular operations that differ from normal driving, sudden acceleration and deceleration, erratic driving, etc., and the resulting vehicle behavior that carries risks.

[0525] "Means for slowing down or stopping a vehicle" refers to a system that operates to reduce the speed of a vehicle or bring it to a complete stop when abnormal driving characteristics are detected.

[0526] "Location information" refers to the current geographical location of a vehicle, obtained using technologies such as GPS.

[0527] A "communication network" refers to a network system, such as the internet or dedicated lines, that enables external data communication.

[0528] The system realizing this invention comprehensively performs tasks such as acquiring signal information, notifying vehicles of this information, monitoring driving characteristics, detecting abnormal driving behavior, and controlling the vehicle. The aim is to significantly improve traffic safety.

[0529] The server first acquires real-time signal information from traffic lights in various locations and stores it in a database. Next, based on the location information, it transmits the signal information to information processing devices of vehicles within the range affected by the signal information. When these information processing devices receive the signal information, if it is a red light, they immediately disable the vehicle's power system control to prevent unintended acceleration by the driver.

[0530] Furthermore, the terminal constantly monitors the vehicle's driving characteristics. It acquires location information, acceleration, and other data based on GPS and sensor data, and sends it to a generated AI model. This AI model analyzes the collected data, and if abnormal driving characteristics are detected, the server immediately sends a command to the terminal to slow down or stop the vehicle.

[0531] For example, when a user approaches an intersection in their car, the server informs the terminal in advance of the red light, allowing the vehicle to automatically slow down and stop safely and smoothly. Furthermore, if a user becomes distracted due to fatigue during long-distance driving and exhibits abnormal driving characteristics, the AI ​​model detects this and automatically issues a warning.

[0532] An example of a prompt message is: "Based on the latest sensor data, detect abnormal driving patterns. Compare this with past data and set the system to issue a warning if erratic driving is detected." This can enhance driver safety and reduce the risk of traffic accidents.

[0533] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0534] Step 1:

[0535] The server acquires traffic signal information in real time from traffic lights nationwide. It receives data from traffic lights as input, outputs this data as traffic signal status (red, yellow, green) and location information, and stores it in a database. This ensures that the traffic signal information is always up-to-date.

[0536] Step 2:

[0537] The server identifies vehicles within the affected area based on the location information of traffic signals. It uses vehicle location information as input and performs data calculations to identify vehicles within the area. As output, it generates a list of vehicles within the affected area and prepares to send the signal information to the terminal.

[0538] Step 3:

[0539] The terminal receives signal information transmitted from the server. It takes the signal information as input, and if the signal is red, it generates a command to disable the vehicle's power system control as output and notifies the in-vehicle control system. This action allows the vehicle to stop safely.

[0540] Step 4:

[0541] The terminal collects driving characteristics in real time from various sensors mounted on the vehicle. It acquires sensor data as input, aggregates the obtained data such as speed, acceleration, and direction, and outputs it. This data is sent to a generated AI model. This collected data is useful for safety verification.

[0542] Step 5:

[0543] The server uses a generated AI model to analyze sensor data and check for abnormal driving characteristics. It receives sensor data as input and determines whether or not abnormal driving is present as output. If an abnormal pattern (e.g., erratic driving) is detected through this analysis, the system can immediately proceed to the next action.

[0544] Step 6:

[0545] If abnormal operation is detected, the server sends a warning signal to the terminal of the vehicle in question. It takes the detection result of the abnormal pattern as input and sends warning information to the terminal as output. This immediately alerts the user and, if necessary, allows them to slow down or stop the vehicle.

[0546] Step 7:

[0547] If the user does not respond to the warning, the terminal takes control of the vehicle and returns it to a safe state. If data indicating a user response cannot be obtained as input, the terminal automatically executes a deceleration or stop command as output. This further enhances user safety.

[0548] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0549] This invention is characterized by its inclusion of an emotion engine that recognizes user emotions, in addition to conventional systems that acquire signal information from traffic lights and provide driving assistance for vehicles. In implementing this invention, flexible driving assistance based on emotion recognition is introduced in addition to vehicle control based on signal information.

[0550] System configuration and operation

[0551] Acquisition and notification of signal information

[0552] The server acquires signal information from traffic lights in real time. Based on the signal information, it transmits it to vehicle terminals within the affected area. This disables the vehicle's accelerator control when the traffic light is red.

[0553] Recognition of user emotions by an emotion engine

[0554] The device uses cameras and sensors installed inside the vehicle to collect information such as the user's facial expressions, heart rate, and voice tone, and transmits this information to an emotion engine. This engine analyzes and recognizes the user's emotional state (e.g., stress, fatigue, relaxation).

[0555] Emotional state-based driving assistance

[0556] The device controls the vehicle's operation based on the recognized emotional state of the user. For example, if a high stress level is detected, it will play relaxing music or display a visual alert. Furthermore, if significant fatigue is detected, the system will consider automatically stopping the vehicle.

[0557] Monitoring of abnormal operation

[0558] The server monitors and analyzes driving patterns and takes action if abnormal driving is detected. This includes slowing down or stopping the vehicle and issuing warnings to the driver.

[0559] Specific example

[0560] Example 1: Driving assistance under high stress conditions

[0561] The device's emotion engine detected that the user was experiencing high levels of stress while driving. In response, it played music to help the user relax and displayed driving advice in a soft tone on the screen.

[0562] Example 2: Emergency shutdown due to emotion

[0563] Based on the user's facial expressions and physical reactions, the terminal's emotion engine determined that the user was experiencing extreme fatigue. Therefore, the server instructed the vehicle to stop at a safe location and encouraged the user to rest.

[0564] This system aims to prevent traffic accidents and ensure driver safety. The introduction of an emotional engine enables flexible support tailored to the individual user's condition, complementing conventional mechanical control and further improving safety.

[0565] The following describes the processing flow.

[0566] Step 1:

[0567] The server acquires signal information from traffic lights in real time. It receives signal status (red, yellow, blue) and traffic light location data, and records it in the relevant control database.

[0568] Step 2:

[0569] The server transmits the signal information to a vehicle terminal with specific location information, based on the signal information. This ensures that the vehicle properly receives real-time signal status.

[0570] Step 3:

[0571] The terminal analyzes the received signal information and, if the signal is red, sends a command to the vehicle's engine control system to disable accelerator control. This prevents unintended acceleration by the driver.

[0572] Step 4:

[0573] An emotion engine installed in the device monitors the user's state using in-car cameras and biosensors. This includes facial recognition, voice analysis, and even heart rate monitoring.

[0574] Step 5:

[0575] The emotion engine analyzes the user's emotional state and determines levels of stress, fatigue, and other factors. Based on the analysis, if it determines that the emotional state is affecting driving, it sends appropriate driving assistance or warnings to the device.

[0576] Step 6:

[0577] The device follows instructions from the emotion engine and performs actions according to the user's emotional state. This includes playing music to reduce stress and automatically stopping the vehicle if fatigue reaches its extreme.

[0578] Step 7:

[0579] The server continuously monitors the vehicle's driving patterns through a generated AI model. When abnormal driving behavior is detected, it issues a warning to the driver or provides control instructions to adjust the vehicle's speed or bring it to a complete stop.

[0580] This integrates the user's emotions with the vehicle's driving conditions, reducing the risk of accidents and providing a safe and comfortable driving environment.

[0581] (Example 2)

[0582] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0583] In modern transportation systems, there is a need to effectively mitigate the risk of traffic accidents caused by the driver's emotional state and driving environment. However, conventional systems are limited to mechanical control using signal information and cannot take into account the emotions, stress, or fatigue levels of individual drivers, making it difficult to provide flexible driving assistance that responds to diverse driving situations. Furthermore, it has been difficult to detect abnormal driving patterns early and take prompt corrective measures.

[0584] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0585] In this invention, the server includes means for acquiring signal data from a signaling device, means for transmitting the signal data to an information display device installed on the transport equipment, and means for analyzing the driver's emotional state using an emotion analysis device. This enables flexible driving support that takes into account the individual emotional state of the driver, and allows for the implementation of appropriate safety measures according to the driving environment. Furthermore, it makes it possible to detect abnormal driving at an early stage and improve traffic safety with appropriate countermeasures.

[0586] A "traffic signal system" is a system that uses light, sound, or other means to display signals and control the flow of traffic.

[0587] "Signal data" refers to information acquired from signaling equipment, including data on the status and fluctuations of signals.

[0588] "Transportation equipment" is a general term for vehicles and machinery used to transport people or goods.

[0589] An "information display device" is a display or monitor device installed in transportation equipment to provide visual information and instructions to the operator.

[0590] "Power control" refers to a control mechanism used to adjust the driving force of engines, motors, and other components of transportation equipment.

[0591] A "movement pattern" is a series of motion data that shows the trajectory, speed, and acceleration of a transport device.

[0592] An "abnormal movement pattern" is a pattern that includes sudden movements or unpredictable actions that are judged to be abnormal compared to normal movement.

[0593] An "emotion analysis device" is a system that recognizes and analyzes data such as facial expressions, voice, and vital signs in order to analyze the emotional state of the pilot.

[0594] "External stimuli" refer to environmental factors that affect the operator, such as music, lighting, and visual displays inside the transport vehicle.

[0595] This system is designed to provide advanced support for the operation of transportation equipment in traffic environments. Specifically, it utilizes signal data from traffic signaling devices and an emotion analysis device to understand the driver's emotional state, thereby providing flexible driving assistance.

[0596] The server acquires signal data from signaling equipment in real time. In this process, data is acquired via communication devices installed on traffic signals, using protocol XYZ. The server also transmits the signal data to information display devices installed on transportation equipment, thereby supporting safe driving by appropriately adjusting the power control of the transportation equipment.

[0597] The terminal uses cameras and sensors installed within the transport vehicle to collect emotional data such as the operator's facial expressions, heart rate, and voice tone. This data is sent to an emotion analysis device, where a generative AI model is used to analyze the operator's emotional state. Based on the analysis results, the terminal controls external stimuli (music, lighting, etc.) within the transport vehicle to reduce operator stress and promote safety.

[0598] For example, if the emotion analysis device determines that the driver is under high stress, the terminal will play relaxing classical music to create a calmer environment inside the car. Furthermore, if fatigue from prolonged driving is detected, an alert function will be used to encourage rest, and the vehicle can be automatically stopped if necessary.

[0599] An example of a prompt based on a generated AI model is, "Analyze the driver's emotional state while driving and suggest appropriate driving assistance measures."

[0600] By using such a system, we can contribute to preventing traffic accidents and improving driver safety, while also achieving advanced support that surpasses conventional driver assistance systems.

[0601] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0602] Step 1:

[0603] The server acquires signal data in real time from the signaling equipment. As input, it receives signal data (signal color, remaining time, etc.) transmitted from communication devices installed on the traffic signals. This data is acquired using protocol XYZ and stored internally by the server as the current state of the signal. As output, the signal data is sent to the next processing step.

[0604] Step 2:

[0605] The server transmits the acquired signal data to an information display device mounted on the transport vehicle. The input is the signal data obtained in step 1. The server analyzes the data and compares the results with the vehicle's location information. As output, detailed signal status information is transmitted to the information display device, and the signal status is displayed on the transport vehicle's display to inform the operator.

[0606] Step 3:

[0607] The terminal collects data using cameras and sensors installed within the transport vehicle to understand the operator's emotional state. Inputs include biometric data such as the operator's facial expressions, heart rate, and voice tone. An emotion analysis device and a generative AI model analyze this data to identify the operator's emotions (stress, fatigue, relaxation, etc.). The output is a report of the analyzed emotional state, which is then sent to the next step.

[0608] Step 4:

[0609] The terminal controls the transport equipment based on the analyzed emotional state. It receives an emotional state report obtained in step 3 as input. The terminal selects and implements appropriate external stimuli (music, lighting, etc.) to adjust environmental factors. Specifically, if stress is detected, it plays relaxing music and adjusts the lighting. The output is an environment designed to help the operator relax.

[0610] Step 5:

[0611] The server monitors the movement patterns transmitted from the transport equipment and detects abnormal movements. Inputs include driving data such as vehicle speed, acceleration, and location information. The server compares this data to baseline values ​​and automatically generates a warning if an anomaly is detected. As output, a warning message is sent to the transport equipment's terminal, and power control is performed as needed.

[0612] (Application Example 2)

[0613] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0614] Conventional mobile vehicle control systems are limited to simple motion control based on the acquisition of signal data, and lack adaptive support that takes into account the emotional state of the occupant. As a result, there was a problem in that the risk of accidents due to occupant stress and fatigue could not be reduced. The present invention aims to provide safer and more comfortable travel by realizing flexible support that takes into account the emotional state of the occupant.

[0615] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0616] In this invention, the server includes means for acquiring signal data from a signal display device, means for transmitting the signal data to a device mounted on the mobile body, means for monitoring the operating pattern of the mobile body, means for recognizing the emotional state of the occupant, means for adjusting the internal environment of the mobile body according to the emotional state, and means for suggesting rest based on the emotional state. This enables safe control of the mobile body's operation and provides support tailored to the occupant's emotional state.

[0617] A "traffic signal display device" is a device that transmits signal data and is installed to regulate the flow of traffic.

[0618] "Signal data" refers to traffic control information transmitted from a signal display device, including data indicating proceed, stop, or caution.

[0619] "Mobile vehicles" refer to all vehicles and rideboats that travel on roads, primarily those that serve the function of transporting people or goods.

[0620] "Device" refers to various equipment and systems attached to a mobile object, primarily used for data processing and analysis.

[0621] "Propulsion control" refers to a control function used to adjust the speed and direction of a moving object.

[0622] A "motion pattern" refers to the tendency of a series of movements or behaviors that a moving object takes within a certain period of time.

[0623] An "abnormal operating pattern" refers to the movement or unnatural behavior of a moving object that differs from the normal operation and may cause problems for safe operation.

[0624] "Emotional state" refers to the mental and physiological state of the crew, and includes various emotions such as stress and relaxation.

[0625] "The internal environment of a mobile vehicle" refers to the physical or acoustic conditions inside a mobile vehicle and is a factor that affects the comfort of the occupants.

[0626] "Means for suggesting rest" refers to a function that informs the crew of their need for rest and suggests the most suitable method for doing so.

[0627] A system implementing this invention includes means for acquiring signal data from a signal display device and transmitting it to a vehicle to perform propulsion control of the moving object. A server within the system receives signal data from the signal display device in real time. The received data is sent to a device mounted on the moving object to disable propulsion control such as the accelerator. This ensures that the moving object operates safely in accordance with the signals.

[0628] Furthermore, the device uses in-car sensors and cameras to analyze the emotional state of the occupants. This analysis particularly utilizes facial expression analysis and heart rate monitoring, and the acquired data is sent to the emotion engine. The emotion engine uses this data to assess the occupants' stress and fatigue levels.

[0629] The device automatically adjusts the in-vehicle environment based on the occupant's emotional state. For example, if it detects high stress levels, it automatically plays relaxation music. Furthermore, if the emotional state indicates excessive fatigue, it suggests the next available resting point, encouraging the occupant to rest.

[0630] For example, if a passenger is experiencing stress during a long drive, the system will adjust the lighting in the car along with the music to create a comfortable environment. Furthermore, an example of a prompt for the generated AI model is: "Please analyze the current emotional state of the passenger using the camera feed and heart rate data. If high stress or fatigue is detected, initiate appropriate relaxation measures or suggest taking a rest break."

[0631] Thus, the present invention has specific embodiments that directly address the emotional state of the occupants and enhance comfort and safety within a mobile vehicle.

[0632] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0633] Step 1:

[0634] The server receives signal data transmitted from the signal display device in real time. It takes data from the signal display device as input and generates information indicating the current state of the signal as output. This information shows how the signal is displayed on the moving object.

[0635] Step 2:

[0636] The terminal receives signal data transmitted from the server and controls the acceleration of the moving object. The input is signal data from the server, and the output is a command for propulsion control. As a result, the speed of the moving object is controlled based on the signal.

[0637] Step 3:

[0638] The terminal collects occupant information using in-vehicle sensors and cameras. Inputs include facial expressions, heart rate, and other physiological data. Outputs generate analytical data sent to the emotion engine, which indicates the occupant's emotional state.

[0639] Step 4:

[0640] The emotion engine analyzes the emotional state of the occupants using data received from the terminal. Data from sensors and cameras is used as input. The output is an evaluation of the occupants' emotional state. Based on this result, the level of stress and fatigue is determined.

[0641] Step 5:

[0642] The device adjusts the environment within the mobile vehicle based on results from the emotion engine. It takes emotion evaluation results as input and generates environmental control signals as output. These signals are used to control actions such as playing relaxation music and adjusting lighting.

[0643] Step 6:

[0644] If the device determines that the user is fatigued, it will suggest taking a break. The fatigue level assessment result is used as input. The output is a notification prompting the user to rest. Specifically, it shows the user information about locations where they can rest.

[0645] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0646] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0647] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0648] [Fourth Embodiment]

[0649] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0650] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0651] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0652] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0653] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0654] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0655] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0656] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0657] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0658] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0659] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0660] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0661] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0662] This invention is a system that improves vehicle safety by utilizing signal information from traffic lights. To implement this invention, the operation of traffic lights, a communication terminal mounted on the vehicle, a server, and a generated AI model must be coordinated.

[0663] System configuration and operation

[0664] Acquisition of signal information

[0665] The server is connected to traffic lights nationwide and retrieves real-time signal information (red, yellow, and blue) transmitted from each traffic light. The retrieved signal information, along with the location information of the traffic lights, is stored in a database.

[0666] Notification to vehicle terminal

[0667] The server uses the location information of the traffic lights to transmit signal information to the terminals of vehicles within the affected area. The terminals receive this information and control the vehicles based on its contents.

[0668] Disabling accelerator control

[0669] If the terminal receives a red signal, it issues a command to the in-vehicle control system to disable accelerator control. This prevents unintended acceleration by the driver and ensures safety.

[0670] Monitoring and analysis of driving patterns

[0671] The terminal collects data in real time from various sensors installed in the vehicle (accelerometer, vehicle speed sensor, etc.) and transmits it to a generated AI model. The AI ​​model analyzes this data and detects abnormal driving patterns.

[0672] Warning and control of abnormal operation

[0673] If the generated AI model detects abnormal driving behavior (for example, frequent swerving or sudden acceleration), the server will issue a warning to the terminal of the vehicle in question and, if necessary, instruct it to slow down or stop.

[0674] Emergency vehicle response

[0675] Users (emergency vehicle operators) can operate without signal control by using special identification information. This information is sent from the terminal to the server, and once its legitimacy is confirmed, the restrictions are temporarily lifted.

[0676] Specific example

[0677] Example 1: Disabling a red light

[0678] The server acquires red light information from traffic lights in urban areas and transmits it to all vehicles within a 500-meter radius. The terminal receives this information and sends a command to the vehicle's in-vehicle system to disable the vehicle's accelerator.

[0679] Example 2: Detection of drowsy driving

[0680] The terminal uses a generated AI model to detect irregular weaving on straight roads. The server issues a warning, and if the driver is unaware, it commands the vehicle to slow down or stop.

[0681] This system is designed with the goal of preventing traffic accidents and ensuring driver safety, as described above. The implementation of this invention is easily adaptable to current traffic systems and has the potential to dramatically improve public safety.

[0682] The following describes the processing flow.

[0683] Step 1:

[0684] The server acquires signal information from traffic lights in real time. The signal status (red, yellow, blue) and geographical location data for each traffic light are recorded in a database, enabling immediate monitoring of signal changes.

[0685] Step 2:

[0686] The server sets a specific area of ​​influence based on the collected signal information and sends the relevant signal information to the terminals of all vehicles within that area. This determines which traffic signals a vehicle in a particular location should be affected by.

[0687] Step 3:

[0688] The terminal receives signal information transmitted from the server. If the information indicates a "red light," it issues a command to the vehicle's control system to disable accelerator operation, preparing to prevent driver error.

[0689] Step 4:

[0690] The terminal continuously acquires data from various sensors installed in the vehicle (accelerometer, direction sensor, etc.) and monitors the driving pattern based on that data. This information is sent to a generating AI model.

[0691] Step 5:

[0692] The server analyzes the received driving patterns through a generating AI model, and if abnormal driving behavior (e.g., frequent weaving or abnormal acceleration / deceleration) is detected, it issues a warning signal or sends an instruction to the vehicle's terminal to slow down or stop.

[0693] Step 6:

[0694] By entering specific authentication information into a terminal, the user (emergency vehicle operator) is excluded from normal traffic signal control. This allows emergency vehicles to operate smoothly without being affected by traffic signals.

[0695] (Example 1)

[0696] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0697] It is necessary to control vehicle operations based on information from signaling devices to improve safety. Furthermore, it is essential to detect abnormal driving, provide effective warnings, and take control as needed. Additionally, special vehicles require means to operate quickly and safely while avoiding normal signaling control.

[0698] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0699] In this invention, the server includes means for acquiring and transmitting data from a signaling device, means for disabling the vehicle's propulsion control, and means for analyzing abnormal driving patterns using an electronic model and issuing warnings as necessary. This enables safe control of vehicle operation based on signaling information, allows for early detection and appropriate response to abnormal driving, and supports the smooth operation of special vehicles.

[0700] "Data" refers to information obtained from signaling devices to control vehicle operations.

[0701] A "traffic signal device" is a device that displays traffic signals and provides information about them.

[0702] A "terminal device" is a device installed in a vehicle that receives and processes data from a server.

[0703] "Propulsion control" refers to the operation of the vehicle's accelerator pedal, and is a control system that adjusts the power output.

[0704] An "operation pattern" is a series of data that indicates the driving behavior of a vehicle.

[0705] An "electronic model" is an algorithm that analyzes driving data and detects anomalies.

[0706] A "special vehicle" refers to a vehicle that does not comply with normal traffic regulations, such as one used for responding to emergencies.

[0707] A "warning" is a signal issued to alert the user when an abnormal operating pattern is detected.

[0708] This invention provides a system that improves vehicle safety by utilizing data obtained from signaling devices. Implementation of the system requires the cooperation of a server, terminal devices mounted on vehicles, signaling devices, and a generating AI model. The server accesses the signaling device network and acquires signaling information in real time. This information, along with the location information of the signaling devices, is stored in a database.

[0709] Based on the acquired signal information, the server transmits the information to terminal devices installed in vehicles within the affected area. The terminal devices receive the signal information and perform actions such as disabling propulsion control and monitoring vehicle operation patterns. Furthermore, the terminal devices transmit data obtained from sensors installed in the vehicles to a generating AI model to detect abnormal operation patterns. The generating AI model analyzes this data, and if an anomaly is detected, it sends a warning to the vehicle via the server and slows down or stops the vehicle as necessary.

[0710] As a concrete example, a server acquires red light information from traffic signals in urban areas and transmits this information to all vehicles within a 500-meter radius. This disables accelerator operation in terminal devices, reducing the risk of accidents caused by accidental operation. In addition, a generated AI model analyzes driving data and detects irregular weaving that may indicate drowsy driving. In this case, the server issues a warning, and if the driver does not respond, it automatically slows down or stops.

[0711] For example, a prompt such as "Is the current vehicle speed exceeding the normal range?" is sent to the generating AI model, and the analysis result is obtained immediately. This allows the system to contribute to improved safety.

[0712] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0713] Step 1:

[0714] The server accesses the signaling device network to acquire data. It receives real-time red, yellow, and blue signal information from each signaling device as input, and combines this information with the signaling device's location information to store it in a database as output. This data processing geographically organizes the signaling information, preparing it for subsequent steps.

[0715] Step 2:

[0716] The server analyzes the acquired signal information and transmits it to terminal devices within the affected area. It uses the location of the signal device and vehicle information within a 500-meter radius as input, and transmits the signal information to the terminal devices of the relevant vehicles as output. The server distributes this information via wireless communication, allowing the vehicle's terminal to recognize the current signal status.

[0717] Step 3:

[0718] The terminal uses the received signal information to adjust the propulsion control within the vehicle. It uses the signal information received from the server as input and issues a command to the vehicle's engine control unit (ECU) to disable accelerator control as output. This operation prevents erroneous operation at red lights, improving safety.

[0719] Step 4:

[0720] The terminal collects operation pattern data from sensors installed in the vehicle. It receives real-time data from vehicle speed and acceleration sensors as input, and sends this data to a generating AI model as output. This allows the terminal to analyze detailed data on driving conditions.

[0721] Step 5:

[0722] The generative AI model analyzes the transmitted data and detects abnormal operating patterns. It uses driving data transmitted from the terminal as input and sends the results of the abnormal pattern detection to the server as output. The generative AI model utilizes advanced algorithms to analyze, for example, the possibility of drowsy driving.

[0723] Step 6:

[0724] The server receives analysis results from the generated AI model and sends warnings to terminals as needed. It receives notifications of abnormal pattern detection as input and distributes warning messages to the terminals of the relevant vehicles as output. Furthermore, the server takes immediate safety measures by issuing commands to slow down or stop, depending on the situation.

[0725] (Application Example 1)

[0726] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0727] In recent years, with the proliferation of automobiles, the risk of traffic accidents has increased, and among these, accidents caused by running red lights and reckless driving have become a serious problem. Conventional warning systems only alert drivers, lacking specific vehicle control and real-time signal response. Therefore, there is a need for effective methods that utilize signal information to ensure the safe operation of vehicles.

[0728] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0729] In this invention, the server includes means for acquiring signal information, means for transmitting the signal information to an information processing device mounted on the vehicle, and means for disabling the vehicle's power system control based on the signal information. This makes it possible to prevent dangerous behaviors such as running red lights and to create a safe traffic environment.

[0730] "Signal information" refers to data about the status of each red, yellow, and blue signal transmitted from traffic lights, and is used to control the movement of vehicles.

[0731] An "information processing device" is a terminal installed in a vehicle that receives signal information and contributes to subsequent vehicle control.

[0732] "Means of disabling power system control" refers to a function that executes commands to temporarily stop the vehicle's acceleration or speed maintenance based on signal information.

[0733] "Driving characteristics" refer to the patterns of movement such as acceleration, deceleration, and direction changes that a vehicle exhibits while in operation.

[0734] "Abnormal driving characteristics" refer to irregular operations that differ from normal driving, sudden acceleration and deceleration, erratic driving, etc., and the resulting vehicle behavior that carries risks.

[0735] "Means for slowing down or stopping a vehicle" refers to a system that operates to reduce the speed of a vehicle or bring it to a complete stop when abnormal driving characteristics are detected.

[0736] "Location information" refers to the current geographical location of a vehicle, obtained using technologies such as GPS.

[0737] A "communication network" refers to a network system, such as the internet or dedicated lines, that enables external data communication.

[0738] The system realizing this invention comprehensively performs tasks such as acquiring signal information, notifying vehicles of this information, monitoring driving characteristics, detecting abnormal driving behavior, and controlling the vehicle. The aim is to significantly improve traffic safety.

[0739] The server first acquires real-time signal information from traffic lights in various locations and stores it in a database. Next, based on the location information, it transmits the signal information to information processing devices of vehicles within the range affected by the signal information. When these information processing devices receive the signal information, if it is a red light, they immediately disable the vehicle's power system control to prevent unintended acceleration by the driver.

[0740] Furthermore, the terminal constantly monitors the vehicle's driving characteristics. It acquires location information, acceleration, and other data based on GPS and sensor data, and sends it to a generated AI model. This AI model analyzes the collected data, and if abnormal driving characteristics are detected, the server immediately sends a command to the terminal to slow down or stop the vehicle.

[0741] For example, when a user approaches an intersection in their car, the server informs the terminal in advance of the red light, allowing the vehicle to automatically slow down and stop safely and smoothly. Furthermore, if a user becomes distracted due to fatigue during long-distance driving and exhibits abnormal driving characteristics, the AI ​​model detects this and automatically issues a warning.

[0742] An example of a prompt message is: "Based on the latest sensor data, detect abnormal driving patterns. Compare this with past data and set the system to issue a warning if erratic driving is detected." This can enhance driver safety and reduce the risk of traffic accidents.

[0743] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0744] Step 1:

[0745] The server acquires traffic signal information in real time from traffic lights nationwide. It receives data from traffic lights as input, outputs this data as traffic signal status (red, yellow, green) and location information, and stores it in a database. This ensures that the traffic signal information is always up-to-date.

[0746] Step 2:

[0747] The server identifies vehicles within the affected area based on the location information of traffic signals. It uses vehicle location information as input and performs data calculations to identify vehicles within the area. As output, it generates a list of vehicles within the affected area and prepares to send the signal information to the terminal.

[0748] Step 3:

[0749] The terminal receives signal information transmitted from the server. It takes the signal information as input, and if the signal is red, it generates a command to disable the vehicle's power system control as output and notifies the in-vehicle control system. This action allows the vehicle to stop safely.

[0750] Step 4:

[0751] The terminal collects driving characteristics in real time from various sensors mounted on the vehicle. It acquires sensor data as input, aggregates the obtained data such as speed, acceleration, and direction, and outputs it. This data is sent to a generated AI model. This collected data is useful for safety verification.

[0752] Step 5:

[0753] The server uses a generated AI model to analyze sensor data and check for abnormal driving characteristics. It receives sensor data as input and determines whether or not abnormal driving is present as output. If an abnormal pattern (e.g., erratic driving) is detected through this analysis, the system can immediately proceed to the next action.

[0754] Step 6:

[0755] If abnormal operation is detected, the server sends a warning signal to the terminal of the vehicle in question. It takes the detection result of the abnormal pattern as input and sends warning information to the terminal as output. This immediately alerts the user and, if necessary, allows them to slow down or stop the vehicle.

[0756] Step 7:

[0757] If the user does not respond to the warning, the terminal takes control of the vehicle and returns it to a safe state. If data indicating a user response cannot be obtained as input, the terminal automatically executes a deceleration or stop command as output. This further enhances user safety.

[0758] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0759] This invention is characterized by its inclusion of an emotion engine that recognizes user emotions, in addition to conventional systems that acquire signal information from traffic lights and provide driving assistance for vehicles. In implementing this invention, flexible driving assistance based on emotion recognition is introduced in addition to vehicle control based on signal information.

[0760] System configuration and operation

[0761] Acquisition and notification of signal information

[0762] The server acquires signal information from traffic lights in real time. Based on the signal information, it transmits it to vehicle terminals within the affected area. This disables the vehicle's accelerator control when the traffic light is red.

[0763] Recognition of user emotions by an emotion engine

[0764] The device uses cameras and sensors installed inside the vehicle to collect information such as the user's facial expressions, heart rate, and voice tone, and transmits this information to an emotion engine. This engine analyzes and recognizes the user's emotional state (e.g., stress, fatigue, relaxation).

[0765] Emotional state-based driving assistance

[0766] The device controls the vehicle's operation based on the recognized emotional state of the user. For example, if a high stress level is detected, it will play relaxing music or display a visual alert. Furthermore, if significant fatigue is detected, the system will consider automatically stopping the vehicle.

[0767] Monitoring of abnormal operation

[0768] The server monitors and analyzes driving patterns and takes action if abnormal driving is detected. This includes slowing down or stopping the vehicle and issuing warnings to the driver.

[0769] Specific example

[0770] Example 1: Driving assistance under high stress conditions

[0771] The device's emotion engine detected that the user was experiencing high levels of stress while driving. In response, it played music to help the user relax and displayed driving advice in a soft tone on the screen.

[0772] Example 2: Emergency shutdown due to emotion

[0773] Based on the user's facial expressions and physical reactions, the terminal's emotion engine determined that the user was experiencing extreme fatigue. Therefore, the server instructed the vehicle to stop at a safe location and encouraged the user to rest.

[0774] This system aims to prevent traffic accidents and ensure driver safety. The introduction of an emotional engine enables flexible support tailored to the individual user's condition, complementing conventional mechanical control and further improving safety.

[0775] The following describes the processing flow.

[0776] Step 1:

[0777] The server acquires signal information from traffic lights in real time. It receives signal status (red, yellow, blue) and traffic light location data, and records it in the relevant control database.

[0778] Step 2:

[0779] The server transmits the signal information to a vehicle terminal with specific location information, based on the signal information. This ensures that the vehicle properly receives real-time signal status.

[0780] Step 3:

[0781] The terminal analyzes the received signal information and, if the signal is red, sends a command to the vehicle's engine control system to disable accelerator control. This prevents unintended acceleration by the driver.

[0782] Step 4:

[0783] An emotion engine installed in the device monitors the user's state using in-car cameras and biosensors. This includes facial recognition, voice analysis, and even heart rate monitoring.

[0784] Step 5:

[0785] The emotion engine analyzes the user's emotional state and determines levels of stress, fatigue, and other factors. Based on the analysis, if it determines that the emotional state is affecting driving, it sends appropriate driving assistance or warnings to the device.

[0786] Step 6:

[0787] The device follows instructions from the emotion engine and performs actions according to the user's emotional state. This includes playing music to reduce stress and automatically stopping the vehicle if fatigue reaches its extreme.

[0788] Step 7:

[0789] The server continuously monitors the vehicle's driving patterns through a generated AI model. When abnormal driving behavior is detected, it issues a warning to the driver or provides control instructions to adjust the vehicle's speed or bring it to a complete stop.

[0790] This integrates the user's emotions with the vehicle's driving conditions, reducing the risk of accidents and providing a safe and comfortable driving environment.

[0791] (Example 2)

[0792] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0793] In modern transportation systems, there is a need to effectively mitigate the risk of traffic accidents caused by the driver's emotional state and driving environment. However, conventional systems are limited to mechanical control using signal information and cannot take into account the emotions, stress, or fatigue levels of individual drivers, making it difficult to provide flexible driving assistance that responds to diverse driving situations. Furthermore, it has been difficult to detect abnormal driving patterns early and take prompt corrective measures.

[0794] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0795] In this invention, the server includes means for acquiring signal data from a signaling device, means for transmitting the signal data to an information display device installed on the transport equipment, and means for analyzing the driver's emotional state using an emotion analysis device. This enables flexible driving support that takes into account the individual emotional state of the driver, and allows for the implementation of appropriate safety measures according to the driving environment. Furthermore, it makes it possible to detect abnormal driving at an early stage and improve traffic safety with appropriate countermeasures.

[0796] A "traffic signal system" is a system that uses light, sound, or other means to display signals and control the flow of traffic.

[0797] "Signal data" refers to information acquired from signaling equipment, including data on the status and fluctuations of signals.

[0798] "Transportation equipment" is a general term for vehicles and machinery used to transport people or goods.

[0799] An "information display device" is a display or monitor device installed in transportation equipment to provide visual information and instructions to the operator.

[0800] "Power control" refers to a control mechanism used to adjust the driving force of engines, motors, and other components of transportation equipment.

[0801] A "movement pattern" is a series of motion data that shows the trajectory, speed, and acceleration of a transport device.

[0802] An "abnormal movement pattern" is a pattern that includes sudden movements or unpredictable actions that are judged to be abnormal compared to normal movement.

[0803] An "emotion analysis device" is a system that recognizes and analyzes data such as facial expressions, voice, and vital signs in order to analyze the emotional state of the pilot.

[0804] "External stimuli" refer to environmental factors that affect the operator, such as music, lighting, and visual displays inside the transport vehicle.

[0805] This system is designed to provide advanced support for the operation of transportation equipment in traffic environments. Specifically, it utilizes signal data from traffic signaling devices and an emotion analysis device to understand the driver's emotional state, thereby providing flexible driving assistance.

[0806] The server acquires signal data from signaling equipment in real time. In this process, data is acquired via communication devices installed on traffic signals, using protocol XYZ. The server also transmits the signal data to information display devices installed on transportation equipment, thereby supporting safe driving by appropriately adjusting the power control of the transportation equipment.

[0807] The terminal uses cameras and sensors installed within the transport vehicle to collect emotional data such as the operator's facial expressions, heart rate, and voice tone. This data is sent to an emotion analysis device, where a generative AI model is used to analyze the operator's emotional state. Based on the analysis results, the terminal controls external stimuli (music, lighting, etc.) within the transport vehicle to reduce operator stress and promote safety.

[0808] For example, if the emotion analysis device determines that the driver is under high stress, the terminal will play relaxing classical music to create a calmer environment inside the car. Furthermore, if fatigue from prolonged driving is detected, an alert function will be used to encourage rest, and the vehicle can be automatically stopped if necessary.

[0809] An example of a prompt based on a generated AI model is, "Analyze the driver's emotional state while driving and suggest appropriate driving assistance measures."

[0810] By using such a system, we can contribute to preventing traffic accidents and improving driver safety, while also achieving advanced support that surpasses conventional driver assistance systems.

[0811] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0812] Step 1:

[0813] The server acquires signal data in real time from the signaling equipment. As input, it receives signal data (signal color, remaining time, etc.) transmitted from communication devices installed on the traffic signals. This data is acquired using protocol XYZ and stored internally by the server as the current state of the signal. As output, the signal data is sent to the next processing step.

[0814] Step 2:

[0815] The server transmits the acquired signal data to an information display device mounted on the transport vehicle. The input is the signal data obtained in step 1. The server analyzes the data and compares the results with the vehicle's location information. As output, detailed signal status information is transmitted to the information display device, and the signal status is displayed on the transport vehicle's display to inform the operator.

[0816] Step 3:

[0817] The terminal collects data using cameras and sensors installed within the transport vehicle to understand the operator's emotional state. Inputs include biometric data such as the operator's facial expressions, heart rate, and voice tone. An emotion analysis device and a generative AI model analyze this data to identify the operator's emotions (stress, fatigue, relaxation, etc.). The output is a report of the analyzed emotional state, which is then sent to the next step.

[0818] Step 4:

[0819] The terminal controls the transport equipment based on the analyzed emotional state. It receives an emotional state report obtained in step 3 as input. The terminal selects and implements appropriate external stimuli (music, lighting, etc.) to adjust environmental factors. Specifically, if stress is detected, it plays relaxing music and adjusts the lighting. The output is an environment designed to help the operator relax.

[0820] Step 5:

[0821] The server monitors the movement patterns transmitted from the transport equipment and detects abnormal movements. Inputs include driving data such as vehicle speed, acceleration, and location information. The server compares this data to baseline values ​​and automatically generates a warning if an anomaly is detected. As output, a warning message is sent to the transport equipment's terminal, and power control is performed as needed.

[0822] (Application Example 2)

[0823] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0824] Conventional mobile vehicle control systems are limited to simple motion control based on the acquisition of signal data, and lack adaptive support that takes into account the emotional state of the occupant. As a result, there was a problem in that the risk of accidents due to occupant stress and fatigue could not be reduced. The present invention aims to provide safer and more comfortable travel by realizing flexible support that takes into account the emotional state of the occupant.

[0825] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0826] In this invention, the server includes means for acquiring signal data from a signal display device, means for transmitting the signal data to a device mounted on the mobile body, means for monitoring the operating pattern of the mobile body, means for recognizing the emotional state of the occupant, means for adjusting the internal environment of the mobile body according to the emotional state, and means for suggesting rest based on the emotional state. This enables safe control of the mobile body's operation and provides support tailored to the occupant's emotional state.

[0827] A "traffic signal display device" is a device that transmits signal data and is installed to regulate the flow of traffic.

[0828] "Signal data" refers to traffic control information transmitted from a signal display device, including data indicating proceed, stop, or caution.

[0829] "Mobile vehicles" refer to all vehicles and rideboats that travel on roads, primarily those that serve the function of transporting people or goods.

[0830] "Device" refers to various equipment and systems attached to a mobile object, primarily used for data processing and analysis.

[0831] "Propulsion control" refers to a control function used to adjust the speed and direction of a moving object.

[0832] A "motion pattern" refers to the tendency of a series of movements or behaviors that a moving object takes within a certain period of time.

[0833] An "abnormal operating pattern" refers to the movement or unnatural behavior of a moving object that differs from the normal operation and may cause problems for safe operation.

[0834] "Emotional state" refers to the mental and physiological state of the crew, and includes various emotions such as stress and relaxation.

[0835] "The internal environment of a mobile vehicle" refers to the physical or acoustic conditions inside a mobile vehicle and is a factor that affects the comfort of the occupants.

[0836] "Means for suggesting rest" refers to a function that informs the crew of their need for rest and suggests the most suitable method for doing so.

[0837] A system implementing this invention includes means for acquiring signal data from a signal display device and transmitting it to a vehicle to perform propulsion control of the moving object. A server within the system receives signal data from the signal display device in real time. The received data is sent to a device mounted on the moving object to disable propulsion control such as the accelerator. This ensures that the moving object operates safely in accordance with the signals.

[0838] Furthermore, the device uses in-car sensors and cameras to analyze the emotional state of the occupants. This analysis particularly utilizes facial expression analysis and heart rate monitoring, and the acquired data is sent to the emotion engine. The emotion engine uses this data to assess the occupants' stress and fatigue levels.

[0839] The device automatically adjusts the in-vehicle environment based on the occupant's emotional state. For example, if it detects high stress levels, it automatically plays relaxation music. Furthermore, if the emotional state indicates excessive fatigue, it suggests the next available resting point, encouraging the occupant to rest.

[0840] For example, if a passenger is experiencing stress during a long drive, the system will adjust the lighting in the car along with the music to create a comfortable environment. Furthermore, an example of a prompt for the generated AI model is: "Please analyze the current emotional state of the passenger using the camera feed and heart rate data. If high stress or fatigue is detected, initiate appropriate relaxation measures or suggest taking a rest break."

[0841] Thus, the present invention has specific embodiments that directly address the emotional state of the occupants and enhance comfort and safety within a mobile vehicle.

[0842] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0843] Step 1:

[0844] The server receives signal data transmitted from the signal display device in real time. It takes data from the signal display device as input and generates information indicating the current state of the signal as output. This information shows how the signal is displayed on the moving object.

[0845] Step 2:

[0846] The terminal receives signal data transmitted from the server and controls the acceleration of the moving object. The input is signal data from the server, and the output is a command for propulsion control. As a result, the speed of the moving object is controlled based on the signal.

[0847] Step 3:

[0848] The terminal collects occupant information using in-vehicle sensors and cameras. Inputs include facial expressions, heart rate, and other physiological data. Outputs generate analytical data sent to the emotion engine, which indicates the occupant's emotional state.

[0849] Step 4:

[0850] The emotion engine analyzes the emotional state of the occupants using data received from the terminal. Data from sensors and cameras is used as input. The output is an evaluation of the occupants' emotional state. Based on this result, the level of stress and fatigue is determined.

[0851] Step 5:

[0852] The device adjusts the environment within the mobile vehicle based on results from the emotion engine. It takes emotion evaluation results as input and generates environmental control signals as output. These signals are used to control actions such as playing relaxation music and adjusting lighting.

[0853] Step 6:

[0854] If the device determines that the user is fatigued, it will suggest taking a break. The fatigue level assessment result is used as input. The output is a notification prompting the user to rest. Specifically, it shows the user information about locations where they can rest.

[0855] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0856] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0857] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0858] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0859] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0860] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0861] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0862] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0863] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0864] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0865] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0866] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0867] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0868] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0869] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0870] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0871] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0872] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0873] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0874] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0875] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0876] The following is further disclosed regarding the embodiments described above.

[0877] (Claim 1)

[0878] A means of obtaining signal information from traffic lights,

[0879] Means for transmitting the aforementioned signal information to a terminal installed in the vehicle,

[0880] A means for disabling the vehicle's accelerator control based on the aforementioned signal information,

[0881] Means for monitoring the vehicle's driving pattern,

[0882] A means for slowing down or stopping a vehicle when an abnormal driving pattern is detected,

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, further comprising means for preventing emergency vehicles from being affected by signal control.

[0886] (Claim 3)

[0887] The system according to claim 1, further comprising means for issuing a warning based on generated driving pattern data.

[0888] "Example 1"

[0889] (Claim 1)

[0890] Means for acquiring data from a signaling device,

[0891] Means for transmitting the aforementioned data to a terminal device installed in the vehicle,

[0892] Based on the aforementioned data, means for disabling the vehicle's propulsion control,

[0893] Means for monitoring the vehicle's operating patterns,

[0894] A means for slowing down or stopping the vehicle when an abnormal operating pattern is detected,

[0895] A means for analyzing the operation pattern using an electronic model,

[0896] A means of issuing a warning when an abnormal operating pattern is detected,

[0897] A system that includes this.

[0898] (Claim 2)

[0899] The system according to claim 1, further comprising means for avoiding the influence of signal control only on special vehicles.

[0900] (Claim 3)

[0901] The system according to claim 1, comprising means for transmitting driving data to an electronic model for analysis and issuing a warning.

[0902] "Application Example 1"

[0903] (Claim 1)

[0904] Means for acquiring signal information,

[0905] means for transmitting the signal information to an information processing device mounted on the vehicle,

[0906] A means for disabling the vehicle's power system control based on the aforementioned signal information,

[0907] Means for monitoring driving characteristics,

[0908] A means for decelerating or stopping a vehicle when abnormal driving characteristics are detected,

[0909] A means of acquiring location information and analyzing it in combination with signal information,

[0910] A means of receiving signal information in real time via an external communication network and issuing notifications regarding the vehicle's direction of travel,

[0911] A system that includes this.

[0912] (Claim 2)

[0913] The system according to claim 1, further comprising means for ensuring that only vehicles with specific permission are not affected by signal control.

[0914] (Claim 3)

[0915] The system according to claim 1, further comprising means for notifying the user of an abnormality based on generated driving characteristic data.

[0916] "Example 2 of combining an emotion engine"

[0917] (Claim 1)

[0918] A means for acquiring signal data from a signaling device,

[0919] Means for transmitting the aforementioned signal data to an information display device installed in the transport equipment,

[0920] A means for disabling the power control of the transport equipment based on the aforementioned signal data,

[0921] Means for monitoring the movement patterns of transport equipment,

[0922] A means for slowing down or stopping transport equipment when an abnormal movement pattern is detected,

[0923] A means of analyzing the emotional state of the pilot using an emotion analysis device,

[0924] A means for controlling external stimuli within the transport device based on the aforementioned emotional state,

[0925] A system that includes this.

[0926] (Claim 2)

[0927] The system according to claim 1, further comprising means for preventing only emergency transport equipment from being affected by signal control.

[0928] (Claim 3)

[0929] The system according to claim 1, further comprising means for issuing a warning based on generated movement pattern data.

[0930] "Application example 2 when combining with an emotional engine"

[0931] (Claim 1)

[0932] A means for acquiring signal data from a signal display device,

[0933] Means for transmitting the aforementioned signal data to a device mounted on a mobile body,

[0934] A means for disabling the propulsion control of the moving body based on the aforementioned signal data,

[0935] Means for monitoring the movement patterns of a moving object,

[0936] A means for slowing down or stopping a moving object when an abnormal operating pattern is detected,

[0937] Means for recognizing the emotional state of the crew,

[0938] Means for adjusting the internal environment of a mobile device according to the aforementioned emotional state,

[0939] Means for suggesting rest based on emotional state,

[0940] A system that includes this.

[0941] (Claim 2)

[0942] The system according to claim 1, further comprising means for preventing emergency mobile objects from being affected by signal control.

[0943] (Claim 3)

[0944] The system according to claim 1, further comprising means for issuing a warning based on generated operation pattern data. [Explanation of Symbols]

[0945] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining signal information from traffic lights, Means for transmitting the aforementioned signal information to a terminal installed in the vehicle, A means for disabling the vehicle's accelerator control based on the aforementioned signal information, Means for monitoring the vehicle's driving pattern, A means for slowing down or stopping a vehicle when an abnormal driving pattern is detected, A system that includes this.

2. The system according to claim 1, further comprising means for preventing emergency vehicles from being affected by signal control.

3. The system according to claim 1, further comprising means for issuing a warning based on generated driving pattern data.

Images