System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving video data obtained from a local monitoring device; means for analyzing the video data using a machine learning algorithm to detect the presence of a child; and means for sending a notification to a driver's communication terminal if the presence of a child is detected.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, the number of traffic accidents involving children has been increasing, and the causes include driver inattention and children suddenly running out into the street. Therefore, current traffic safety measures alone are not enough to ensure the safety of children, and new technological solutions are needed. In particular, accidents are occurring frequently on the way to and from school and on crosswalks, and an effective solution to these problems is needed. [Means for solving the problem]

[0005] This invention provides a system that receives video data acquired from a local surveillance device and uses a machine learning algorithm to detect the presence of children in real time. The system then notifies the driver of the detected information via their communication device, thereby alerting the driver and ensuring the safety of children. It also includes a means for collecting and analyzing driver behavior data and providing feedback to support safe driving. This improves driver behavior and prevents traffic accidents. It also provides a means for analyzing traffic conditions across an entire area and optimizing traffic safety measures.

[0006] Specific means for solving this problem include the following: a means for receiving video data acquired from a local surveillance device, a means for analyzing the video data with a machine learning algorithm to detect the presence of a child, a means for sending a notification to the driver's communication device when the presence of a child is detected, a means for collecting and analyzing data on the driver's behavior, and a means for providing feedback based on the collected data.

[0007] "Area monitoring devices" are devices that include cameras and sensors installed to monitor traffic conditions and the surrounding environment in real time.

[0008] "Video data" refers to information on moving or still images collected by a monitoring device or a user terminal.

[0009] A "machine learning algorithm" is a computational method for recognizing and predicting specific patterns and features based on data.

[0010] "Means for detecting the presence of a child" refers to software and hardware that analyzes video data and recognizes the appearance and behavior of a child from that data.

[0011] A "driver's communication device" is a communication-enabled device such as a smartphone or tablet carried by the driver.

[0012] The "means for sending notifications" is a function for sending messages and warnings from the system to the driver's communication terminal.

[0013] "Driver behavior data" refers to data related to driving behavior such as speed, braking, and steering while driving.

[0014] "Feedback to support safe driving" refers to guidance information such as advice and warning messages provided to the driver based on the driver's behavioral data.

[0015] The "means for analyzing traffic conditions" refers to software and hardware that analyzes data obtained from multiple monitoring devices and identifies traffic flow and dangerous spots throughout the area.

[0016] "Means for optimizing traffic safety measures" is a function that uses the analysis results to derive specific measures and proposals to improve traffic safety throughout the region. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is an AI system for preventing traffic accidents involving children using a community monitoring device or a user terminal. Specific embodiments for carrying out the present invention will be described in detail below.

[0039] Server Roles

[0040] 1. Video data collection

[0041] The server receives video data in real time from local surveillance devices and user terminals. For example, surveillance cameras installed near an elementary school send video footage of children on their way to and from school.

[0042] 2. Analysis of video data

[0043] The server pre-processes the received video data and uses machine learning algorithms to detect the presence of children, specifically using models that can distinguish between different age groups and accurately distinguish between pedestrians and children.

[0044] 3. Risk Assessment and Notification Generation

[0045] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a car approaches while a child is crossing the street, the risk is deemed high. In this case, a notification is immediately sent to the driver's communication device, displaying a warning such as "There is a child ahead. Please slow down."

[0046] 4. Collecting and analyzing operational data

[0047] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.), providing real-time feedback to support safe driving. For example, if a driver frequently brakes suddenly, the system sends an alert to the driver saying, "Please try to brake more gently."

[0048] 5. Analysis of traffic conditions across the region

[0049] The server integrates data collected from multiple monitoring devices and analyzes traffic conditions across the entire region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if a particular intersection is experiencing frequent traffic accidents, specific recommendations can be made, such as installing traffic lights or relocating pedestrian crossings.

[0050] Device Role

[0051] 1. Receiving and viewing notifications

[0052] The device receives notifications sent from the server in real time and displays them to the user (driver). The notifications are delivered not only as visual displays but also as audio alerts, so drivers can immediately pay attention while driving.

[0053] 2. Sending driving data

[0054] The device collects driver behavior data and transmits it to a server, including GPS data and data from the vehicle's acceleration sensor. The data is sent periodically and is useful for analysis on the server side.

[0055] User (driver) behavior

[0056] 1. Check notifications and drive safely

[0057] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[0058] 2. Providing Feedback

[0059] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[0060] Specific examples

[0061] Let's say a camera monitoring an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device saying, "Children are at the intersection. Please slow down." The driver sees the notification, immediately reduces their speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[0062] In this way, the system of the present invention detects the presence of children in real time and notifies drivers, thereby preventing traffic accidents involving children and strengthening traffic safety measures throughout the region.

[0063] The processing flow will be explained below.

[0064] Server Processing

[0065] Step 1: Receiving video data

[0066] The server receives video data in real time from the area monitoring devices and user terminals using HTTP streaming or RTSP protocols, and the received data is temporarily stored in a buffer.

[0067] Step 2: Preprocessing the video data

[0068] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0069] Step 3: Detect child presence

[0070] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[0071] Step 4: Risk assessment

[0072] The server evaluates the risk by combining the child's presence data with surrounding traffic information, taking into account the child's location, speed, and the movement of surrounding vehicles, among other factors, to calculate the risk level.

[0073] Step 5: Generate notifications

[0074] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, including details of the risk (e.g., "Child ahead. Be careful.").

[0075] Step 6: Sending notifications

[0076] The server sends the generated notification to the driver's communication device using a push notification service. The notification is formatted in JSON format and sent to the target device.

[0077] Step 7: Collect driving data

[0078] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[0079] Step 8: Data analysis and feedback generation

[0080] The server analyzes the collected driving data and generates feedback to support safe driving, such as a message saying, "You braked too often. Please try to brake more gently."

[0081] Terminal handling

[0082] Step 1: Receive notifications

[0083] The device receives notifications from the server in real time, which are not only displayed visually but also delivered to the driver as audio alerts.

[0084] Step 2: View notifications

[0085] The device analyzes the content of the received notification and displays it to the driver. Specifically, it displays the message "There is a child ahead. Please slow down" on the screen and issues a voice alert.

[0086] Step 3: Collect driving data

[0087] The device collects driver behavior data using the built-in GPS and acceleration sensors, and periodically stores the data in a buffer.

[0088] Step 4: Sending data

[0089] The device periodically transmits the collected driving data to a server using the HTTPS protocol, ensuring that the data is delivered securely to the server.

[0090] User (driver) behavior

[0091] Step 1: Check notifications

[0092] The user checks the notification from the device, and not only does it display a visual notification but also an audio alert to alert the driver.

[0093] Step 2: Practice safe driving

[0094] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and check their surroundings.

[0095] Step 3: Provide feedback

[0096] Users have the ability to provide feedback through the application, for example, by recording and sending feedback such as "the notification was appropriate" or "there was a false alarm."

[0097] Through the above processing steps, the present invention prevents traffic accidents involving children and provides a safe driving environment.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] In recent years, traffic accidents involving children have been increasing in urban and suburban areas. In particular, around schools and residential areas, drivers are unable to predict children's movements, resulting in unexpected accidents. To prevent such accidents, there is a need for a system that detects the presence of children and notifies drivers in real time. Furthermore, there is a need to reduce the risk of traffic accidents by collecting driver behavior data and providing feedback to promote safe driving. The development of a new system to solve these issues is required.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for receiving video data from a local monitoring device or a user terminal, means for preprocessing the video data and detecting the presence of a child using a machine learning algorithm, means for performing risk assessment based on the detected child's location information and sending a notification to the driver's communication terminal, means for collecting the driver's behavioral data and sending it to the server, and means for analyzing the driver's behavioral data and surrounding traffic information and providing feedback to support safe driving, thereby enabling real-time notifications to prevent traffic accidents involving children and supporting safe driving for the driver.

[0103] "Area monitoring devices" are devices such as cameras and sensors installed in specific areas to collect video data and other information.

[0104] A "user terminal" is a device used by a driver, such as a mobile phone, tablet, or car navigation system, that receives notifications from the server and displays them to the driver.

[0105] "Video data" refers to video or still image data acquired from a local monitoring device or a user terminal, and is information to be analyzed.

[0106] "Preprocessing" refers to processing performed to make video data easier to analyze, and includes operations such as noise removal, image correction, and frame extraction.

[0107] A "machine learning algorithm" is an algorithm that automatically learns data patterns and analyzes new data based on those patterns.

[0108] A "risk assessment" is an assessment of the dangers of a particular situation, based on the child's location and surrounding traffic conditions.

[0109] A "notification" is a message sent from the server to the driver's communication terminal, and is intended to warn or alert the driver visually or audibly.

[0110] "Driver behavior data" is information related to the driver's driving behavior, including speed, acceleration, GPS data, and the like.

[0111] "Feedback" refers to advice and warnings provided to assist safe driving based on collected driver behavior data.

[0112] "Traffic conditions" refers to information that indicates the state of traffic flow, congestion, accidents, etc. in a specific area.

[0113] "Integration" refers to the process of bringing together information obtained from multiple data sources, and is a necessary operation for analysis and evaluation.

[0114] "Traffic safety measures" are policies and measures taken to prevent traffic accidents and ensure traffic safety throughout the region.

[0115] The "feedback function" allows the driver to send their evaluations and opinions to the system, which will be used to improve the system.

[0116] The present invention is an AI system for preventing traffic accidents involving children, using a community monitoring device or a user terminal. Specific embodiments of the present invention will be described in detail below.

[0117] System hardware configuration

[0118] The system of the present invention is composed of the following main hardware components:

[0119] 1. Community surveillance devices: Surveillance cameras installed near elementary schools and residential areas.

[0120] 2. User devices: Mobile phones, tablets, and car navigation systems used by drivers.

[0121] 3. Server: A high-performance computer located in the cloud or a data center that performs data analysis.

[0122] System software configuration

[0123] The system of the present invention comprises the following software components:

[0124] 1. Data collection module: Software for receiving video data from area monitoring devices or user terminals.

[0125] 2. Pre-processing module: Software that removes noise from the received video data and corrects the image.

[0126] 3. Machine Learning Module: Software that detects the presence of children using machine learning algorithms such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector).

[0127] 4. Risk Assessment Module: Software that assesses risk based on the child's location and generates notification content.

[0128] 5. Notification sending module: Software for sending notifications to the driver's communication device.

[0129] 6. Data analysis module: Software that analyzes driver behavior data and surrounding traffic information, and provides feedback to support safe driving.

[0130] 7. Integrated Data Analysis Module: Software for analyzing traffic conditions across an entire region and optimizing traffic safety measures.

[0131] 8. Feedback Collection Module: Software that collects feedback from users and helps improve the system.

[0132] Example of operation

[0133] Suppose a surveillance camera captures a group of children crossing a street in front of an elementary school. The server receives the video data from the camera via the RTSP protocol. The pre-processing module removes noise and enhances the image, and the machine learning module detects the presence of children.

[0134] Next, the risk assessment module performs a risk assessment based on the detected child's location information and surrounding traffic information, and sends a notification to the driver's device saying, "A child is at the intersection. Please reduce your speed." The driver confirms this notification, reduces their speed, and continues driving safely.

[0135] Additionally, driving behavior data (GPS data and acceleration data) is sent from the device to a server. The server analyzes this data and provides real-time feedback to promote safe driving. For example, it sends an alert such as, "You are braking frequently. Please brake gently."

[0136] The server integrates data from the monitoring devices and analyzes traffic conditions across the entire region, and if accidents occur frequently at a particular intersection, it will suggest specific countermeasures such as installing traffic lights or relocating pedestrian crossings.

[0137] Example prompts for generative AI models

[0138] Prompt: "A surveillance camera captures a child crossing a street in front of an elementary school. Explain the specific steps a server takes in this footage, uses a machine learning model to detect the child, and then sends a notification to the driver."

[0139] As described above, the system of the present invention analyzes data from local monitoring devices and user terminals in real time and immediately notifies drivers, thereby preventing traffic accidents involving children and enhancing traffic safety throughout the entire region.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1:

[0142] Video data collection

[0143] The server receives video data in real time from the local monitoring device and user terminals. The input is the video stream sent from the local monitoring device, and the output is the video data buffered in the server. Specifically, the server acquires the video data using the RTSP protocol and stores it in a buffer at a fixed frame rate (e.g., 30 frames per second).

[0144] Step 2:

[0145] Video data preprocessing

[0146] The server preprocesses the received video data. The input is the buffered video data, and the output is the corrected video data with noise removed. Specifically, it applies noise removal algorithms and inter-frame compensation algorithms to improve the image quality.

[0147] Step 3:

[0148] Child Detection

[0149] The server analyzes the preprocessed video data using machine learning algorithms (e.g., YOLO or SSD) to detect the presence of children. The input is the preprocessed video data, and the output is the location information of the detected child. Specifically, it uses a machine learning model to detect people and applies a classification algorithm to distinguish between children and other pedestrians.

[0150] Step 4:

[0151] Risk assessment and notification generation

[0152] The server performs a risk assessment based on the child's location information and surrounding traffic information, and generates a notification. The input is the detected child's location information and traffic information, and the output is the notification content sent to the driver's device. Specifically, it calculates the risk level using a certain risk assessment algorithm (for example, an evaluation function based on distance and speed), and generates a notification content such as "There is a child ahead. Please slow down."

[0153] Step 5:

[0154] Receiving and Viewing Notifications

[0155] The terminal receives notifications sent from the server and displays them to the user (driver). The input is the notification sent from the server, and the output is the warning message displayed to the driver. Specifically, the terminal waits for notifications from the server in the background, and when it receives one, it displays a pop-up message on the screen and plays an audio alert at the same time.

[0156] Step 6:

[0157] Safe driving practices

[0158] The user checks the notification from the device and drives safely. The input is the notification message displayed on the device, and the output is a driving maneuver to safely reduce speed. Specifically, the user acknowledges the notification, applies the brakes, slows down, and ensures the safety of the child while checking the surrounding traffic conditions.

[0159] Step 7:

[0160] Collecting and transmitting driving data

[0161] The terminal collects driver behavior data (GPS data and acceleration data) and sends it to a server. The input is behavior data acquired from the terminal's sensors, and the output is collected data sent to the server. Specifically, the terminal acquires data from the GPS sensor and acceleration sensor, aggregates the data at regular time intervals (for example, every second), and sends it to the server.

[0162] Step 8:

[0163] Providing Feedback

[0164] The server analyzes the driver's behavioral data and surrounding traffic information, and provides feedback to support safe driving. The input is the collected driving data and traffic information, and the output is feedback provided to the driver. Specifically, the server analyzes the behavioral data, generates advice and warnings to support safe driving (e.g., "You are braking frequently. Please brake gently"), and sends them to the driver's device.

[0165] (Application example 1)

[0166] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0167] This invention relates to a system for preventing traffic accidents involving children and improving the safety of autonomous vehicles. Specifically, the purpose is to reduce the risk of traffic accidents by detecting the presence of children in real time using a local monitoring device and a control system for autonomous vehicles and taking appropriate measures. Furthermore, the invention aims to implement comprehensive traffic safety measures by analyzing driver behavior data and traffic conditions throughout the region.

[0168] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0169] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal or the control system of the autonomous vehicle when the presence of a child is detected, and means for stopping or slowing down the vehicle based on the notification, thereby enabling the presence of a child to be detected in real time and appropriate driving operations to be performed.

[0170] An "area monitoring device" is a device that monitors a specific area or location and collects video data in real time.

[0171] "Video data" refers to data of moving images or still images acquired using equipment such as cameras or sensors.

[0172] A "machine learning algorithm" is an algorithm that automatically identifies specific patterns and features by having a computer learn from data.

[0173] "Means for detecting the presence of a child" refers to a system or method for analyzing video data to identify the child's appearance and confirm their location information.

[0174] A "driver's communication device" is an electronic device such as a smartphone or tablet carried by the driver that receives notifications and alerts.

[0175] An "autonomous vehicle control system" is a system for automatically operating and controlling a vehicle.

[0176] "Means for sending notification" refers to a system or method for sending detected information to the driver's communication terminal or the control system of the autonomous vehicle.

[0177] "Means for stopping or slowing a vehicle" means a system or method for automatically adjusting the speed of a vehicle to stop or slow down based on the notification.

[0178] "Driver behavior data" refers to data such as the speed, acceleration, and frequency of sudden braking when the driver operates the vehicle.

[0179] A "means for providing feedback" is a system or method for analyzing driver behavior data and providing advice or warnings to the driver for safe driving.

[0180] The "means for integrating video data" is a system or method for centrally consolidating video data acquired from multiple monitoring devices.

[0181] A "means for analyzing traffic conditions across an entire region" is a system or method for analyzing the integrated video data and identifying traffic conditions and problem areas in a specific region.

[0182] "Means for optimizing traffic safety measures" refers to a system or method for proposing and implementing appropriate traffic safety measures for each region based on the analysis results.

[0183] "Child presence information detected by the vehicle control system" refers to safety-related information such as the location of a child detected in real time by the vehicle control system.

[0184] The present invention relates to a system for preventing traffic accidents involving children, using a regional monitoring device and a control system for an autonomous vehicle. Specific embodiments for carrying out the invention will be described below.

[0185] Server Roles

[0186] 1. Video data collection

[0187] The server receives video data in real time from local surveillance devices and in-vehicle cameras. For example, video is transmitted from surveillance cameras installed around elementary schools and residential areas, or from cameras mounted on autonomous vehicles.

[0188] 2. Analysis of video data

[0189] The server preprocesses the received video data and uses machine learning algorithms to detect the presence of children. This uses a pre-trained generative AI model using a machine learning framework such as TensorFlow. The model can distinguish between different age groups and accurately distinguish between pedestrians and children.

[0190] 3. Risk Assessment and Notification Generation

[0191] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a vehicle approaches while a child is crossing the street, it is deemed to be at high risk. In this case, a notification is immediately sent to the driver's communication device and the autonomous vehicle's control system, issuing a warning such as "There is a child ahead. Please slow down."

[0192] 4. Collecting and analyzing operational data

[0193] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and autonomous vehicle control data. This provides real-time feedback to support safe driving. For example, a driver who frequently brakes suddenly may receive an alert saying, "Please try to brake more gently."

[0194] 5. Analysis of traffic conditions across the region

[0195] The server integrates data obtained from multiple monitoring devices and analyzes traffic conditions throughout the region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if traffic accidents frequently occur at a particular intersection, specific recommendations will be made, such as installing traffic lights or relocating pedestrian crossings. Furthermore, information on the presence of children detected by autonomous vehicles can also be reflected in safety measures for the entire region.

[0196] Device Role

[0197] 1. Receiving and viewing notifications

[0198] The device receives notifications sent from the server in real time and displays them to the driver. The notifications are delivered not only as visual displays but also as audio alerts, allowing drivers to immediately pay attention while driving. Furthermore, the autonomous vehicle control system automatically adjusts the vehicle's speed upon receiving the notification.

[0199] 2. Sending driving data

[0200] The device collects driver behavior data and transmits it to a server. This includes GPS data and data from the vehicle's acceleration sensor. In the case of autonomous vehicles, control information is also transmitted. The data is transmitted periodically and is useful for analysis on the server side.

[0201] User (driver) behavior

[0202] 1. Check notifications and drive safely

[0203] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[0204] 2. Providing Feedback

[0205] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[0206] Specific examples

[0207] Let's say a camera monitoring an area in front of an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device or the autonomous vehicle's control system saying, "Children are at the intersection. Please slow down." The driver or autonomous vehicle then checks the notification, immediately reduces speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[0208] Example prompt for a generative AI model:

[0209] "Suddenly, children start crossing the street. How should the AI ​​detection system react?"

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] The server receives video data in real time from area surveillance devices and on-board cameras of autonomous vehicles. This video data is transmitted from each camera through a dedicated communication channel and received by the server. The input is the video data from the surveillance cameras, and the output is the storage of this data on the server. The server temporarily stores the received video data in a buffer.

[0213] Step 2:

[0214] The server preprocesses the received video data. Specifically, it removes noise from the video data and normalizes it for each frame. It also resizes the video data as needed. The input is the video data stored on the server, and the output is the preprocessed video data. This allows the machine learning model to analyze the data efficiently.

[0215] Step 3:

[0216] The server sends the preprocessed video data to a machine learning algorithm to detect the presence of a child. This uses a generative AI model from TensorFlow. The input is the preprocessed video data, and the output is a determination of whether a child is present. The server determines the presence of a child for each frame based on the output of the AI ​​model.

[0217] Step 4:

[0218] The server performs a risk assessment based on the detected child's location and surrounding traffic information. The inputs are the judgment results from the AI ​​model and surrounding traffic information (e.g., the speed and distance of nearby vehicles), and the output is the risk assessment result. The server performs the risk assessment by taking into account multiple factors, and if the risk is high, it determines the necessary measures.

[0219] Step 5:

[0220] Based on the results of the risk assessment, the server sends a notification to the driver's communication device or the autonomous vehicle's control system. The input is the result of the risk assessment, and the output is a notification message. For example, a message saying "There is a child ahead. Please slow down." The notification is sent in real time, allowing the driver or vehicle system to respond immediately.

[0221] Step 6:

[0222] The driver's communication terminal or the autonomous vehicle's control system receives the notification from the server. The input is the notification message from the server, and the output is an alert display for the driver or speed adjustment by the vehicle control system. Specifically, a visual warning display or an audio alert is sounded, and the autonomous vehicle automatically adjusts its speed.

[0223] Step 7:

[0224] The terminal collects behavioral data of the driver or autonomous vehicle and sends it to the server. The input is the behavioral data acquired by the terminal, and the output is the data sent to the server. This behavioral data includes information on speed, acceleration, braking operation, etc. The data is periodically sent to the server and used for further analysis.

[0225] Step 8:

[0226] The server analyzes the received behavioral data and provides feedback to support safe driving. The input is behavioral data sent from the driver or the autonomous vehicle, and the output is a feedback message. For example, advice such as "Please do not brake suddenly too often" is provided to the driver.

[0227] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0228] The present invention combines an AI system using a community monitoring device or user terminal to prevent traffic accidents involving children with an emotion engine. This system recognizes the driver's emotional state and promotes safe driving by adjusting risk assessments and notification content. Specific embodiments for implementing the present invention are described in detail below.

[0229] Server Roles

[0230] 1. Video data collection

[0231] The server receives real-time video data from local surveillance devices and user terminals. For example, surveillance cameras installed around a school transmit images of children going to and from school.

[0232] 2. Preprocessing of video data

[0233] The server performs pre-processing on the received video data, which includes noise removal, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0234] 3. Child presence detection

[0235] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[0236] 4. Risk Assessment and Notification Generation

[0237] The server combines data on the child's presence with surrounding traffic information to assess risk and generates a notification if the risk level is high. For example, if the server detects an approaching car while a child is crossing the street, it sends a warning to the driver's device saying, "There is a child ahead. Please slow down."

[0238] 5. Collecting and analyzing operational data

[0239] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and provides feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive an alert saying, "Please try to brake more gently."

[0240] 6. Operation of the Emotion Engine

[0241] The server analyzes the driver's facial expressions and tone of voice and recognizes their emotional state using an emotion engine, an algorithm for performing emotion analysis based on camera footage and audio data.

[0242] 7. Emotion-Based Risk Assessment

[0243] The server performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards are applied to further promote safe driving.

[0244] 8. Long-term emotional data analysis

[0245] The server monitors the driver's emotional state over the long term and analyzes the data to provide optimal feedback to each individual driver, for example, providing stress management advice to drivers who frequently feel stressed.

[0246] Device Role

[0247] 1. Receiving and viewing notifications

[0248] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[0249] 2. Collecting Emotional Data

[0250] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and sends this data to the emotion engine, which uses it to monitor the driver's emotional state in real time while driving.

[0251] 3. Sending driving data

[0252] The device continuously collects driver behavior data and periodically transmits it to a server, including GPS data and data from the vehicle's acceleration sensor.

[0253] User (driver) behavior

[0254] 1. Check notifications

[0255] The user checks the notification from the device, and the system not only displays a visual notification but also an audio alert to attract the driver's attention.

[0256] 2. Safe driving

[0257] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[0258] 3. Providing Feedback

[0259] Users have the ability to provide feedback through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded and sent to the server.

[0260] Specific examples

[0261] For example, on a weekday morning, while drivers are commuting to work and children are on their way to school, a community monitoring device may detect the presence of a child crossing a street. After the information is sent to the server, the server immediately confirms the presence of the child and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the server determines that the driver is stressed, it may also provide additional feedback such as, "Please drive calmly." In this way, the system of the present invention can detect the presence of a child in real time, grasp the driver's emotional state, and provide appropriate notifications, thereby preventing traffic accidents involving children and providing a safe driving environment.

[0262] The processing flow will be explained below.

[0263] Server Processing

[0264] Step 1: Receiving video data

[0265] The server receives real-time video data from local surveillance devices and user terminals, such as cameras monitoring children on their way to and from school.

[0266] Step 2: Preprocessing the video data

[0267] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0268] Step 3: Detect child presence

[0269] The server feeds the pre-processed video data into a machine learning algorithm to detect the presence of children, which then identifies people in the video and applies an age estimation model to identify children among them.

[0270] Step 4: Risk assessment

[0271] The server combines the child's presence data with surrounding traffic information to assess risk. For example, when a child is about to cross the road, it calculates the risk level based on the speed and location of nearby vehicles.

[0272] Step 5: Generate notifications

[0273] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, for example, a specific warning message such as "There is a child ahead, please slow down."

[0274] Step 6: Sending notifications

[0275] The server then sends the generated notification to the driver's communication device using a push notification service.

[0276] Step 7: Collect driving data

[0277] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[0278] Step 8: Data analysis and feedback generation

[0279] The server analyzes the collected driving data and provides feedback to support safe driving, such as generating a message like, "You braked too often. Please brake more gently."

[0280] Step 9: Collect emotion data

[0281] The server receives the driver's facial expressions and voice data and analyzes them with an emotion engine, thereby recognizing the driver's emotional state in real time.

[0282] Step 10: Emotion-Based Risk Assessment

[0283] The server performs a risk assessment based on the perceived emotional state: for example, if the driver is angry, it sets a higher risk level and applies stricter notification criteria.

[0284] Step 11: Analyze Long-Term Sentiment Data

[0285] The server monitors the driver's emotional state over a long period of time, accumulating and analyzing the data, thereby identifying the driver's individual stress patterns and fatigue levels and providing optimal feedback.

[0286] Terminal handling

[0287] Step 1: Receive notifications

[0288] The device receives notifications sent from the server in real time and warns the driver using a combination of visual and audio alerts.

[0289] Step 2: View notifications

[0290] The device analyzes the received notification and displays it to the driver. For example, a notification such as "There is a child ahead. Please slow down" may be displayed on the screen and read aloud.

[0291] Step 3: Collecting emotion data

[0292] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and passes this data to the emotion engine, which uses it to monitor the driver's emotional state in real time.

[0293] Step 4: Collect driving data

[0294] The device collects driving data using GPS and acceleration sensors, which are temporarily stored in a buffer and periodically sent to a server.

[0295] Step 5: Sending data

[0296] The device transmits the collected driving and emotion data to a server using a secure protocol (e.g., HTTPS).

[0297] User (driver) behavior

[0298] Step 1: Check notifications

[0299] The user checks the notification from the device, receives it both visually and audibly, and understands its content.

[0300] Step 2: Practice safe driving

[0301] The user can take appropriate driving actions based on the notification content. For example, if the user receives a notification that there is a child ahead, the user can slow down and check the surrounding area for safety.

[0302] Step 3: Emotional Management

[0303] Users are expected to pay attention to their emotional state and take appropriate driving behaviors, and the system provides feedback to help them manage stress and fatigue.

[0304] Step 4: Provide feedback

[0305] Users provide feedback to the system through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded within the app and sent to the server.

[0306] By implementing the above processing steps, the system of the present invention can prevent traffic accidents involving children and provide a safe driving environment. By taking the driver's emotional state into consideration, even more effective safe driving support can be achieved.

[0307] Example 2

[0308] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0309] Conventional area monitoring systems and safe driving support systems do not take into account the driver's psychological state, and therefore do not provide appropriate feedback or notifications according to the situation, resulting in limited effectiveness in preventing traffic accidents. Furthermore, there is no system that not only notifies drivers of the presence of children but also comprehensively evaluates the driver's emotional and behavioral data, which creates problems with risk management during parking. This does not promote safe driving behavior among drivers, and the risk of traffic accidents remains.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0311] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication device when the presence of a child is detected, and means for adjusting the content of the notification based on the driver's emotional state using an emotion engine that analyzes the driver's emotional state, thereby enabling real-time risk assessment and promotion of safe driving that takes the driver's emotional state into consideration.

[0312] "Area monitoring device" refers to a hardware device that is deployed in a specific area to collect video and other data in real time.

[0313] "Video data" refers to video information acquired from a local monitoring device or a user terminal, and is digital data to be analyzed.

[0314] A "machine learning algorithm" is a computational method for analyzing large amounts of data and learning patterns from that data, and is used, for example, for person detection and age estimation.

[0315] The "means for detecting the presence of a child" is a system element for analyzing video data and identifying a child from within the video data.

[0316] The "driver's communication terminal" refers to a communication device such as a mobile phone, smartphone, or in-car device carried by the driver.

[0317] "Means for sending notification" refers to a communication method or device for immediately conveying detected information and risk assessment results to the driver.

[0318] The "emotion engine" is a system component that analyzes the driver's facial expressions and voice data to recognize their emotional state.

[0319] "Driver behavior data" refers to data that indicates the driving behavior of the driver, including speed, acceleration, use of sudden braking, and the like.

[0320] The "means for providing feedback" is a method or device that provides information or advice to the driver to support safe driving based on the analysis results.

[0321] The "integration means" is a system function that centrally analyzes video data acquired from multiple monitoring devices and grasps the overall situation.

[0322] "Means for optimizing road safety measures" are system elements for planning and implementing local measures to promote safe driving based on integrated data and driver emotion data.

[0323] This invention combines an emotion engine with an AI system that aims to prevent traffic accidents involving children using community monitoring devices and user terminals. The system recognizes the driver's emotional state and adjusts risk assessment and notification content to promote safe driving.

[0324] Hardware Configuration

[0325] The main hardware components of the system include area monitoring devices, servers, and user terminals. Area monitoring devices can be CCTV cameras or other video capture devices. The servers use high-performance computers and cloud services to perform data analysis and processing. User terminals are mainly smartphones or communication devices in cars.

[0326] Software Configuration

[0327] The main software used in the system is as follows:

[0328] OpenCV library: Used for preprocessing video data.

[0329] YOLO, SSD: Person detection as a machine learning algorithm.

[0330] Google Cloud Speech-to-Text API: Used to analyze audio data.

[0331] dlib library: Used for facial expression recognition.

[0332] Data processing flow

[0333] The server receives video data from the area surveillance devices in real time and preprocesses the received data to remove noise and adjust the resolution. The preprocessed video data is then fed into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. If a child is detected, the server assesses the risk and, if necessary, sends a notification to the driver's communication device. The notification is provided in audio and visual formats.

[0334] The server also collects and analyzes driver behavior data from GPS and vehicle sensors to provide feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive advice such as "Try to brake more gently."

[0335] Emotion Engine Operation

[0336] The server collects camera footage and audio data from the user's device and uses an emotion engine to recognize the driver's emotional state. Based on this emotional state, the server adjusts risk assessment and notification content. For example, if the driver is angry, the notification standards will be tightened and a warning such as "Please drive calmly" will be sent.

[0337] Specific examples

[0338] For example, on a weekday morning, while a driver is commuting to work and children are on their way to school, a community monitoring device detects the presence of a child crossing a crosswalk. This information is sent to a server, which immediately verifies it and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the system determines that the driver is stressed, it may provide additional feedback such as, "Please drive calmly." In this way, by detecting the presence of a child in real time and providing appropriate notifications while understanding the driver's emotional state, the system can prevent traffic accidents and provide a safe driving environment.

[0339] Prompt Sentence Examples

[0340] "Please explain the role of an AI system aimed at preventing traffic accidents involving children. The system incorporates an emotion engine to recognize the driver's emotional state and perform risk assessments. It also adjusts notifications to promote safe driving."

[0341] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0342] Step 1: Collect video data

[0343] The server receives video data from the area surveillance devices in real time. The input is the video data sent from the area surveillance devices, and the output is the raw video data processed within the server. Specifically, it acquires video streams from the surveillance cameras and transfers the data securely using the TCP / IP protocol.

[0344] Step 2: Preprocessing the video data

[0345] The server performs preprocessing on the received video data, such as noise removal, resolution adjustment, and background removal. The input is raw video data, and the output is preprocessed video data. Specifically, it uses the OpenCV library to apply background subtraction to remove noise and maintain a constant video resolution.

[0346] Step 3: Detect child presence

[0347] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is information on whether the presence of a child was detected. As a concrete example, a video frame is input into the YOLO model, the detection results are saved in JSON format, and the coordinate information where the child was identified is extracted.

[0348] Step 4: Risk assessment and notification generation

[0349] The server assesses risk by combining data on the child's presence with information on surrounding traffic. The inputs are data on the child's presence and traffic information, and the output is a notification message. For example, it obtains traffic information from a real-time traffic API and calculates risk by comparing the timing of a child crossing the street with the speed of the car. If the risk level is high, it sends a notification to the driver's device saying, "There is a child ahead. Please slow down."

[0350] Step 5: Collect and analyze driving data

[0351] The device continuously collects driver behavior data and periodically transmits it to the server. The input is GPS data and vehicle sensor data, and the output is driving data sent to the server. Specifically, the device collects vehicle speed and acceleration data via the smartphone's GPS sensor and OBD-II interface.

[0352] Step 6: Emotion Engine in Action

[0353] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and sends them to the server. The input is camera footage and audio data, and the output is the recognition result of the driver's emotional state. The server analyzes this data, performs facial expression recognition using the dlib library, and converts the audio data into text using the Google Cloud Speech-to-Text API to perform emotion analysis.

[0354] Step 7: Emotion-Based Risk Assessment

[0355] The server performs risk assessment based on the emotion recognition results. The input is the emotion recognition results, and the output is a risk level and notification content based on the emotion. For example, if it determines that the driver is angry, it applies stricter notification standards than usual and generates a warning message saying, "Please drive calmly."

[0356] Step 8: Analyze Long-Term Sentiment Data

[0357] The server monitors the driver's emotional state over the long term and analyzes the data. The input is past emotional data, and the output is long-term trend analysis and feedback. The emotional data is aggregated monthly and stress management advice is provided based on the analysis results. For example, relaxation suggestions are made to drivers who frequently feel stressed.

[0358] In this way, the system of the present invention is designed to prevent traffic accidents by detecting the presence of children in real time through multiple processing steps and sending appropriate notifications while taking into account the driver's emotional state.

[0359] (Application example 2)

[0360] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0361] The objective of this invention is to provide a system for preventing traffic accidents involving children in local areas and promoting safe driving. In particular, we aim to build a system that provides appropriate feedback on the driver's emotional state and performs real-time risk assessment to increase the driver's attention and support safe driving.

[0362] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data acquired from a local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal when the presence of a child is detected, means for analyzing image and audio data to recognize the driver's emotional state, and means for performing a risk assessment based on the driver's emotional state and adjusting the content of the notification. This makes it possible to detect the presence of a child in real time and send an appropriate notification according to the driver's emotional state.

[0363] A "regional monitoring device" is a device that is installed in a specific region and is used to collect video data.

[0364] "Video data" refers to image information acquired by an imaging device such as a camera.

[0365] A "machine learning algorithm" is a mathematical model or calculation method for automatically analyzing data and recognizing patterns.

[0366] "Means for detecting the presence of children" refers to a system that analyzes video data to determine whether a child is in the image.

[0367] "Driver" means a person operating and driving a vehicle.

[0368] A "communication terminal" is an electronic device capable of sending and receiving information. Specifically, it refers to a smartphone or similar device.

[0369] The "means for sending notification" is a mechanism for sending specific information to the driver's communication terminal.

[0370] "Emotional state" refers to the driver's mental state and emotional movements.

[0371] "Image and audio data" refers to visual and audio information acquired through a camera or microphone.

[0372] A "means for risk assessment" is a system for determining the degree of danger based on collected data.

[0373] The "means for adjusting notification content" is a mechanism for optimizing the content of notifications sent based on the driver's emotional state and detected risks.

[0374] The present invention provides a system for preventing traffic accidents involving children, using video data from local surveillance devices and the driver's communication terminal. The system has the function of recognizing the driver's emotional state and adjusting risk assessment and notification content based on that state.

[0375] Server Roles

[0376] 1. Video data collection

[0377] The server receives real-time video data from local surveillance devices, such as cameras installed around a school, capturing images of children walking to and from school. The received video data is analyzed according to the processing steps described below.

[0378] 2. Preprocessing of video data

[0379] The server performs preprocessing on the received video data, such as removing noise, adjusting resolution, and cutting out the background, and the preprocessed data is used as input data for machine learning algorithms.

[0380] 3. Child presence detection

[0381] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and then identifies children using an age estimation model.

[0382] 4. Risk Assessment and Notification Generation

[0383] The server combines the child presence data with traffic information to assess the risk, and if the risk level is high, generates and sends a notification to the driver's communication device, such as "There is a child ahead. Please slow down."

[0384] 5. Recognizing the driver's emotional state

[0385] The server analyzes the image and audio data collected from the communication device to recognize the driver's emotional state: facial expression recognition algorithms are applied to the image data, and voice analysis algorithms are applied to the audio data.

[0386] 6. Emotion-Based Risk Assessment

[0387] The server then performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards will be applied.

[0388] 7. Long-term emotional data analysis

[0389] The server monitors the driver's emotional state over the long term and analyzes the data, providing stress management advice to drivers who frequently experience stress.

[0390] Device Role

[0391] 1. Receiving and viewing notifications

[0392] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[0393] 2. Collecting Emotional Data

[0394] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and transmits the data to a server, which uses it to monitor the driver's emotional state in real time.

[0395] 3. Sending driving data

[0396] The device continuously collects driver behavior data (e.g., speed, acceleration, sudden braking, etc.) and periodically transmits it to the server, which provides feedback for safe driving.

[0397] User (driver) behavior

[0398] 1. Check notifications

[0399] Users check notifications from their devices, and audio alerts as well as visual notifications are used to draw attention.

[0400] 2. Safe driving

[0401] The user is then advised to take appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[0402] 3. Providing Feedback

[0403] Users have the ability to provide feedback through the application. For example, feedback such as "the notification was appropriate" or "there was a false alarm" can be recorded and sent to the server for future improvements.

[0404] Specific examples

[0405] For example, during the morning commute, a driver gets into their vehicle and launches the smartphone app. The app automatically receives video data from local surveillance devices and detects the presence of children. The driver's face is captured by a camera and their emotional state is analyzed using a facial recognition algorithm. If the app determines that the driver is stressed, a notification will appear saying, "There is a child ahead. Please drive calmly." Upon seeing this notification, the driver will slow down and take action to check their surroundings.

[0406] Example prompts to input to a generative AI model:

[0407] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[0408] As a result, the present invention provides a safe driving support system that takes into account the emotional state of the driver, thereby contributing to the prevention of traffic accidents involving children.

[0409] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0410] Step 1:

[0411] The server receives video data in real time from the local surveillance equipment. The input is raw video data from the local surveillance equipment, and the output is video data for preprocessing. Specifically, the video of children going to and from school is transmitted from the surveillance cameras installed around the school.

[0412] Step 2:

[0413] The server performs preprocessing on the received video data. The input is raw video data, and the output is preprocessed video data that has undergone noise removal, resolution adjustment, background removal, etc. This prepares the data so that the analysis algorithm can operate accurately.

[0414] Step 3:

[0415] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is data indicating the child's location. The algorithm identifies people in the video and uses an age estimation model to identify the child.

[0416] Step 4:

[0417] The server combines the presence data of the child and traffic information to perform risk assessment. The input is the presence data of the child and surrounding traffic information, and the output is assessment data indicating the risk level. If the risk level is high, a notification is generated.

[0418] Step 5:

[0419] The server sends a notification based on the risk assessment results to the driver's communication device. The input is risk level assessment data, and the output is notification information such as "There is a child ahead. Please slow down." This allows the driver to recognize danger early.

[0420] Step 6:

[0421] The server recognizes the driver's emotional state using image and voice data acquired from the driver's communication device. The input is the driver's facial image and voice tone data, and the output is the emotion recognition result. Facial expression recognition algorithms and voice analysis algorithms are used.

[0422] Step 7:

[0423] The server re-performs risk assessment based on the emotion recognition results. The inputs are the emotion recognition results and existing risk assessment data, and the output is adjusted risk assessment data based on the emotional state. For example, if the driver is angry, a more severe notification is generated.

[0424] Step 8:

[0425] The terminal receives notifications sent from the server in real time and displays them to the driver. The input is notification data, and the output is visual and audio notification displays. This allows the driver to immediately prepare to take appropriate driving actions.

[0426] Step 9:

[0427] The device continuously collects the driver's emotional data using a camera and microphone and sends it to the server. The input is the driver's facial expression and tone of voice data, and the output is sent to the server as emotional data. This data is used for subsequent emotional analysis.

[0428] Step 10:

[0429] The server analyzes the driver's behavioral data (speed, acceleration, sudden braking, etc.) and provides long-term feedback. The input is behavioral data, and the output is advice such as "Try to brake more gently."

[0430] Step 11:

[0431] The user checks the notification from the device and performs safe driving behavior. The input is the notification information from the device, and the output is the action taken by the driver. For example, upon receiving a notification that "there is a child ahead," the user slows down and checks their surroundings.

[0432] Example prompt sentence:

[0433] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[0434] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0435] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0436] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0437] [Second embodiment]

[0438] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0439] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0440] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0441] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0442] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0444] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0445] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0446] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0447] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0448] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0449] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0450] The present invention is an AI system for preventing traffic accidents involving children using a community monitoring device or a user terminal. Specific embodiments for carrying out the present invention will be described in detail below.

[0451] Server Roles

[0452] 1. Video data collection

[0453] The server receives video data in real time from local surveillance devices and user terminals. For example, surveillance cameras installed near an elementary school send video footage of children on their way to and from school.

[0454] 2. Analysis of video data

[0455] The server pre-processes the received video data and uses machine learning algorithms to detect the presence of children, specifically using models that can distinguish between different age groups and accurately distinguish between pedestrians and children.

[0456] 3. Risk Assessment and Notification Generation

[0457] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a car approaches while a child is crossing the street, the risk is deemed high. In this case, a notification is immediately sent to the driver's communication device, displaying a warning such as "There is a child ahead. Please slow down."

[0458] 4. Collecting and analyzing operational data

[0459] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.), providing real-time feedback to support safe driving. For example, if a driver frequently brakes suddenly, the system sends an alert to the driver saying, "Please try to brake more gently."

[0460] 5. Analysis of traffic conditions across the region

[0461] The server integrates data collected from multiple monitoring devices and analyzes traffic conditions across the entire region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if a particular intersection is experiencing frequent traffic accidents, specific recommendations can be made, such as installing traffic lights or relocating pedestrian crossings.

[0462] Device Role

[0463] 1. Receiving and viewing notifications

[0464] The device receives notifications sent from the server in real time and displays them to the user (driver). The notifications are delivered not only as visual displays but also as audio alerts, so drivers can immediately pay attention while driving.

[0465] 2. Sending driving data

[0466] The device collects driver behavior data and transmits it to a server, including GPS data and data from the vehicle's acceleration sensor. The data is sent periodically and is useful for analysis on the server side.

[0467] User (driver) behavior

[0468] 1. Check notifications and drive safely

[0469] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[0470] 2. Providing Feedback

[0471] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[0472] Specific examples

[0473] Let's say a camera monitoring an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device saying, "Children are at the intersection. Please slow down." The driver sees the notification, immediately reduces their speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[0474] In this way, the system of the present invention detects the presence of children in real time and notifies drivers, thereby preventing traffic accidents involving children and strengthening traffic safety measures throughout the region.

[0475] The processing flow will be explained below.

[0476] Server Processing

[0477] Step 1: Receiving video data

[0478] The server receives video data in real time from the area monitoring devices and user terminals using HTTP streaming or RTSP protocols, and the received data is temporarily stored in a buffer.

[0479] Step 2: Preprocessing the video data

[0480] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0481] Step 3: Detect child presence

[0482] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[0483] Step 4: Risk assessment

[0484] The server evaluates the risk by combining the child's presence data with surrounding traffic information, taking into account the child's location, speed, and the movement of surrounding vehicles, among other factors, to calculate the risk level.

[0485] Step 5: Generate notifications

[0486] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, including details of the risk (e.g., "Child ahead. Be careful.").

[0487] Step 6: Sending notifications

[0488] The server sends the generated notification to the driver's communication device using a push notification service. The notification is formatted in JSON format and sent to the target device.

[0489] Step 7: Collect driving data

[0490] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[0491] Step 8: Data analysis and feedback generation

[0492] The server analyzes the collected driving data and generates feedback to support safe driving, such as a message saying, "You braked too often. Please try to brake more gently."

[0493] Terminal handling

[0494] Step 1: Receive notifications

[0495] The device receives notifications from the server in real time, which are not only displayed visually but also delivered to the driver as audio alerts.

[0496] Step 2: View notifications

[0497] The device analyzes the content of the received notification and displays it to the driver. Specifically, it displays the message "There is a child ahead. Please slow down" on the screen and issues a voice alert.

[0498] Step 3: Collect driving data

[0499] The device collects driver behavior data using the built-in GPS and acceleration sensors, and periodically stores the data in a buffer.

[0500] Step 4: Sending data

[0501] The device periodically transmits the collected driving data to a server using the HTTPS protocol, ensuring that the data is delivered securely to the server.

[0502] User (driver) behavior

[0503] Step 1: Check notifications

[0504] The user checks the notification from the device, and not only does it display a visual notification but also an audio alert to alert the driver.

[0505] Step 2: Practice safe driving

[0506] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and check their surroundings.

[0507] Step 3: Provide feedback

[0508] Users have the ability to provide feedback through the application, for example, by recording and sending feedback such as "the notification was appropriate" or "there was a false alarm."

[0509] Through the above processing steps, the present invention prevents traffic accidents involving children and provides a safe driving environment.

[0510] Example 1

[0511] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0512] In recent years, traffic accidents involving children have been increasing in urban and suburban areas. In particular, around schools and residential areas, drivers are unable to predict children's movements, resulting in unexpected accidents. To prevent such accidents, there is a need for a system that detects the presence of children and notifies drivers in real time. Furthermore, there is a need to reduce the risk of traffic accidents by collecting driver behavior data and providing feedback to promote safe driving. The development of a new system to solve these issues is required.

[0513] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0514] In this invention, the server includes means for receiving video data from a local monitoring device or a user terminal, means for preprocessing the video data and detecting the presence of a child using a machine learning algorithm, means for performing risk assessment based on the detected child's location information and sending a notification to the driver's communication terminal, means for collecting the driver's behavioral data and sending it to the server, and means for analyzing the driver's behavioral data and surrounding traffic information and providing feedback to support safe driving, thereby enabling real-time notifications to prevent traffic accidents involving children and supporting safe driving for the driver.

[0515] "Area monitoring devices" are devices such as cameras and sensors installed in specific areas to collect video data and other information.

[0516] A "user terminal" is a device used by a driver, such as a mobile phone, tablet, or car navigation system, that receives notifications from the server and displays them to the driver.

[0517] "Video data" refers to video or still image data acquired from a local monitoring device or a user terminal, and is information to be analyzed.

[0518] "Preprocessing" refers to processing performed to make video data easier to analyze, and includes operations such as noise removal, image correction, and frame extraction.

[0519] A "machine learning algorithm" is an algorithm that automatically learns data patterns and analyzes new data based on those patterns.

[0520] A "risk assessment" is an assessment of the dangers of a particular situation, based on the child's location and surrounding traffic conditions.

[0521] A "notification" is a message sent from the server to the driver's communication terminal, and is intended to warn or alert the driver visually or audibly.

[0522] "Driver behavior data" is information related to the driver's driving behavior, including speed, acceleration, GPS data, and the like.

[0523] "Feedback" refers to advice and warnings provided to assist safe driving based on collected driver behavior data.

[0524] "Traffic conditions" refers to information that indicates the state of traffic flow, congestion, accidents, etc. in a specific area.

[0525] "Integration" refers to the process of bringing together information obtained from multiple data sources, and is a necessary operation for analysis and evaluation.

[0526] "Traffic safety measures" are policies and measures taken to prevent traffic accidents and ensure traffic safety throughout the region.

[0527] The "feedback function" allows the driver to send their evaluations and opinions to the system, which will be used to improve the system.

[0528] The present invention is an AI system for preventing traffic accidents involving children, using a community monitoring device or a user terminal. Specific embodiments of the present invention will be described in detail below.

[0529] System hardware configuration

[0530] The system of the present invention is composed of the following main hardware components:

[0531] 1. Community surveillance devices: Surveillance cameras installed near elementary schools and residential areas.

[0532] 2. User devices: Mobile phones, tablets, and car navigation systems used by drivers.

[0533] 3. Server: A high-performance computer located in the cloud or a data center that performs data analysis.

[0534] System software configuration

[0535] The system of the present invention comprises the following software components:

[0536] 1. Data collection module: Software for receiving video data from area monitoring devices or user terminals.

[0537] 2. Pre-processing module: Software that removes noise from the received video data and corrects the image.

[0538] 3. Machine Learning Module: Software that detects the presence of children using machine learning algorithms such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector).

[0539] 4. Risk Assessment Module: Software that assesses risk based on the child's location and generates notification content.

[0540] 5. Notification sending module: Software for sending notifications to the driver's communication device.

[0541] 6. Data analysis module: Software that analyzes driver behavior data and surrounding traffic information, and provides feedback to support safe driving.

[0542] 7. Integrated Data Analysis Module: Software for analyzing traffic conditions across an entire region and optimizing traffic safety measures.

[0543] 8. Feedback Collection Module: Software that collects feedback from users and helps improve the system.

[0544] Example of operation

[0545] Suppose a surveillance camera captures a group of children crossing a street in front of an elementary school. The server receives the video data from the camera via the RTSP protocol. The pre-processing module removes noise and enhances the image, and the machine learning module detects the presence of children.

[0546] Next, the risk assessment module performs a risk assessment based on the detected child's location information and surrounding traffic information, and sends a notification to the driver's device saying, "A child is at the intersection. Please reduce your speed." The driver confirms this notification, reduces their speed, and continues driving safely.

[0547] Additionally, driving behavior data (GPS data and acceleration data) is sent from the device to a server. The server analyzes this data and provides real-time feedback to promote safe driving. For example, it sends an alert such as, "You are braking frequently. Please brake gently."

[0548] The server integrates data from the monitoring devices and analyzes traffic conditions across the entire region, and if accidents occur frequently at a particular intersection, it will suggest specific countermeasures such as installing traffic lights or relocating pedestrian crossings.

[0549] Example prompts for generative AI models

[0550] Prompt: "A surveillance camera captures a child crossing a street in front of an elementary school. Explain the specific steps a server takes in this footage, uses a machine learning model to detect the child, and then sends a notification to the driver."

[0551] As described above, the system of the present invention analyzes data from local monitoring devices and user terminals in real time and immediately notifies drivers, thereby preventing traffic accidents involving children and enhancing traffic safety throughout the entire region.

[0552] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0553] Step 1:

[0554] Video data collection

[0555] The server receives video data in real time from the local monitoring device and user terminals. The input is the video stream sent from the local monitoring device, and the output is the video data buffered in the server. Specifically, the server acquires the video data using the RTSP protocol and stores it in a buffer at a fixed frame rate (e.g., 30 frames per second).

[0556] Step 2:

[0557] Video data preprocessing

[0558] The server preprocesses the received video data. The input is the buffered video data, and the output is the corrected video data with noise removed. Specifically, it applies noise removal algorithms and inter-frame compensation algorithms to improve the image quality.

[0559] Step 3:

[0560] Child Detection

[0561] The server analyzes the preprocessed video data using machine learning algorithms (e.g., YOLO or SSD) to detect the presence of children. The input is the preprocessed video data, and the output is the location information of the detected child. Specifically, it uses a machine learning model to detect people and applies a classification algorithm to distinguish between children and other pedestrians.

[0562] Step 4:

[0563] Risk assessment and notification generation

[0564] The server performs a risk assessment based on the child's location information and surrounding traffic information, and generates a notification. The input is the detected child's location information and traffic information, and the output is the notification content sent to the driver's device. Specifically, it calculates the risk level using a certain risk assessment algorithm (for example, an evaluation function based on distance and speed), and generates a notification content such as "There is a child ahead. Please slow down."

[0565] Step 5:

[0566] Receiving and Viewing Notifications

[0567] The terminal receives notifications sent from the server and displays them to the user (driver). The input is the notification sent from the server, and the output is the warning message displayed to the driver. Specifically, the terminal waits for notifications from the server in the background, and when it receives one, it displays a pop-up message on the screen and plays an audio alert at the same time.

[0568] Step 6:

[0569] Safe driving practices

[0570] The user checks the notification from the device and drives safely. The input is the notification message displayed on the device, and the output is a driving maneuver to safely reduce speed. Specifically, the user acknowledges the notification, applies the brakes, slows down, and ensures the safety of the child while checking the surrounding traffic conditions.

[0571] Step 7:

[0572] Collecting and transmitting driving data

[0573] The terminal collects driver behavior data (GPS data and acceleration data) and sends it to a server. The input is behavior data acquired from the terminal's sensors, and the output is collected data sent to the server. Specifically, the terminal acquires data from the GPS sensor and acceleration sensor, aggregates the data at regular time intervals (for example, every second), and sends it to the server.

[0574] Step 8:

[0575] Providing Feedback

[0576] The server analyzes the driver's behavioral data and surrounding traffic information, and provides feedback to support safe driving. The input is the collected driving data and traffic information, and the output is feedback provided to the driver. Specifically, the server analyzes the behavioral data, generates advice and warnings to support safe driving (e.g., "You are braking frequently. Please brake gently"), and sends them to the driver's device.

[0577] (Application example 1)

[0578] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0579] This invention relates to a system for preventing traffic accidents involving children and improving the safety of autonomous vehicles. Specifically, the purpose is to reduce the risk of traffic accidents by detecting the presence of children in real time using a local monitoring device and a control system for autonomous vehicles and taking appropriate measures. Furthermore, the invention aims to implement comprehensive traffic safety measures by analyzing driver behavior data and traffic conditions throughout the region.

[0580] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0581] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal or the control system of the autonomous vehicle when the presence of a child is detected, and means for stopping or slowing down the vehicle based on the notification, thereby enabling the presence of a child to be detected in real time and appropriate driving operations to be performed.

[0582] An "area monitoring device" is a device that monitors a specific area or location and collects video data in real time.

[0583] "Video data" refers to data of moving images or still images acquired using equipment such as cameras or sensors.

[0584] A "machine learning algorithm" is an algorithm that automatically identifies specific patterns and features by having a computer learn from data.

[0585] "Means for detecting the presence of a child" refers to a system or method for analyzing video data to identify the child's appearance and confirm their location information.

[0586] A "driver's communication device" is an electronic device such as a smartphone or tablet carried by the driver that receives notifications and alerts.

[0587] An "autonomous vehicle control system" is a system for automatically operating and controlling a vehicle.

[0588] "Means for sending notification" refers to a system or method for sending detected information to the driver's communication terminal or the control system of the autonomous vehicle.

[0589] "Means for stopping or slowing a vehicle" means a system or method for automatically adjusting the speed of a vehicle to stop or slow down based on the notification.

[0590] "Driver behavior data" refers to data such as the speed, acceleration, and frequency of sudden braking when the driver operates the vehicle.

[0591] A "means for providing feedback" is a system or method for analyzing driver behavior data and providing advice or warnings to the driver for safe driving.

[0592] The "means for integrating video data" is a system or method for centrally consolidating video data acquired from multiple monitoring devices.

[0593] A "means for analyzing traffic conditions across an entire region" is a system or method for analyzing the integrated video data and identifying traffic conditions and problem areas in a specific region.

[0594] "Means for optimizing traffic safety measures" refers to a system or method for proposing and implementing appropriate traffic safety measures for each region based on the analysis results.

[0595] "Child presence information detected by the vehicle control system" refers to safety-related information such as the location of a child detected in real time by the vehicle control system.

[0596] The present invention relates to a system for preventing traffic accidents involving children, using a regional monitoring device and a control system for an autonomous vehicle. Specific embodiments for carrying out the invention will be described below.

[0597] Server Roles

[0598] 1. Video data collection

[0599] The server receives video data in real time from local surveillance devices and in-vehicle cameras. For example, video is transmitted from surveillance cameras installed around elementary schools and residential areas, or from cameras mounted on autonomous vehicles.

[0600] 2. Analysis of video data

[0601] The server preprocesses the received video data and uses machine learning algorithms to detect the presence of children. This uses a pre-trained generative AI model using a machine learning framework such as TensorFlow. The model can distinguish between different age groups and accurately distinguish between pedestrians and children.

[0602] 3. Risk Assessment and Notification Generation

[0603] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a vehicle approaches while a child is crossing the street, it is deemed to be at high risk. In this case, a notification is immediately sent to the driver's communication device and the autonomous vehicle's control system, issuing a warning such as "There is a child ahead. Please slow down."

[0604] 4. Collecting and analyzing operational data

[0605] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and autonomous vehicle control data. This provides real-time feedback to support safe driving. For example, a driver who frequently brakes suddenly may receive an alert saying, "Please try to brake more gently."

[0606] 5. Analysis of traffic conditions across the region

[0607] The server integrates data obtained from multiple monitoring devices and analyzes traffic conditions throughout the region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if traffic accidents frequently occur at a particular intersection, specific recommendations will be made, such as installing traffic lights or relocating pedestrian crossings. Furthermore, information on the presence of children detected by autonomous vehicles can also be reflected in safety measures for the entire region.

[0608] Device Role

[0609] 1. Receiving and viewing notifications

[0610] The device receives notifications sent from the server in real time and displays them to the driver. The notifications are delivered not only as visual displays but also as audio alerts, allowing drivers to immediately pay attention while driving. Furthermore, the autonomous vehicle control system automatically adjusts the vehicle's speed upon receiving the notification.

[0611] 2. Sending driving data

[0612] The device collects driver behavior data and transmits it to a server. This includes GPS data and data from the vehicle's acceleration sensor. In the case of autonomous vehicles, control information is also transmitted. The data is transmitted periodically and is useful for analysis on the server side.

[0613] User (driver) behavior

[0614] 1. Check notifications and drive safely

[0615] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[0616] 2. Providing Feedback

[0617] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[0618] Specific examples

[0619] Let's say a camera monitoring an area in front of an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device or the autonomous vehicle's control system saying, "Children are at the intersection. Please slow down." The driver or autonomous vehicle then checks the notification, immediately reduces speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[0620] Example prompt for a generative AI model:

[0621] "Suddenly, children start crossing the street. How should the AI ​​detection system react?"

[0622] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0623] Step 1:

[0624] The server receives video data in real time from area surveillance devices and on-board cameras of autonomous vehicles. This video data is transmitted from each camera through a dedicated communication channel and received by the server. The input is the video data from the surveillance cameras, and the output is the storage of this data on the server. The server temporarily stores the received video data in a buffer.

[0625] Step 2:

[0626] The server preprocesses the received video data. Specifically, it removes noise from the video data and normalizes it for each frame. It also resizes the video data as needed. The input is the video data stored on the server, and the output is the preprocessed video data. This allows the machine learning model to analyze the data efficiently.

[0627] Step 3:

[0628] The server sends the preprocessed video data to a machine learning algorithm to detect the presence of a child. This uses a generative AI model from TensorFlow. The input is the preprocessed video data, and the output is a determination of whether a child is present. The server determines the presence of a child for each frame based on the output of the AI ​​model.

[0629] Step 4:

[0630] The server performs a risk assessment based on the detected child's location and surrounding traffic information. The inputs are the judgment results from the AI ​​model and surrounding traffic information (e.g., the speed and distance of nearby vehicles), and the output is the risk assessment result. The server performs the risk assessment by taking into account multiple factors, and if the risk is high, it determines the necessary measures.

[0631] Step 5:

[0632] Based on the results of the risk assessment, the server sends a notification to the driver's communication device or the autonomous vehicle's control system. The input is the result of the risk assessment, and the output is a notification message. For example, a message saying "There is a child ahead. Please slow down." The notification is sent in real time, allowing the driver or vehicle system to respond immediately.

[0633] Step 6:

[0634] The driver's communication terminal or the autonomous vehicle's control system receives the notification from the server. The input is the notification message from the server, and the output is an alert display for the driver or speed adjustment by the vehicle control system. Specifically, a visual warning display or an audio alert is sounded, and the autonomous vehicle automatically adjusts its speed.

[0635] Step 7:

[0636] The terminal collects behavioral data of the driver or autonomous vehicle and sends it to the server. The input is the behavioral data acquired by the terminal, and the output is the data sent to the server. This behavioral data includes information on speed, acceleration, braking operation, etc. The data is periodically sent to the server and used for further analysis.

[0637] Step 8:

[0638] The server analyzes the received behavioral data and provides feedback to support safe driving. The input is behavioral data sent from the driver or the autonomous vehicle, and the output is a feedback message. For example, advice such as "Please do not brake suddenly too often" is provided to the driver.

[0639] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0640] The present invention combines an AI system using a community monitoring device or user terminal to prevent traffic accidents involving children with an emotion engine. This system recognizes the driver's emotional state and promotes safe driving by adjusting risk assessments and notification content. Specific embodiments for implementing the present invention are described in detail below.

[0641] Server Roles

[0642] 1. Video data collection

[0643] The server receives real-time video data from local surveillance devices and user terminals. For example, surveillance cameras installed around a school transmit images of children going to and from school.

[0644] 2. Preprocessing of video data

[0645] The server performs pre-processing on the received video data, which includes noise removal, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0646] 3. Child presence detection

[0647] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[0648] 4. Risk Assessment and Notification Generation

[0649] The server combines data on the child's presence with surrounding traffic information to assess risk and generates a notification if the risk level is high. For example, if the server detects an approaching car while a child is crossing the street, it sends a warning to the driver's device saying, "There is a child ahead. Please slow down."

[0650] 5. Collecting and analyzing operational data

[0651] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and provides feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive an alert saying, "Please try to brake more gently."

[0652] 6. Operation of the Emotion Engine

[0653] The server analyzes the driver's facial expressions and tone of voice and recognizes their emotional state using an emotion engine, an algorithm for performing emotion analysis based on camera footage and audio data.

[0654] 7. Emotion-Based Risk Assessment

[0655] The server performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards are applied to further promote safe driving.

[0656] 8. Long-term emotional data analysis

[0657] The server monitors the driver's emotional state over the long term and analyzes the data to provide optimal feedback to each individual driver, for example, providing stress management advice to drivers who frequently feel stressed.

[0658] Device Role

[0659] 1. Receiving and viewing notifications

[0660] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[0661] 2. Collecting Emotional Data

[0662] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and sends this data to the emotion engine, which uses it to monitor the driver's emotional state in real time while driving.

[0663] 3. Sending driving data

[0664] The device continuously collects driver behavior data and periodically transmits it to a server, including GPS data and data from the vehicle's acceleration sensor.

[0665] User (driver) behavior

[0666] 1. Check notifications

[0667] The user checks the notification from the device, and the system not only displays a visual notification but also an audio alert to attract the driver's attention.

[0668] 2. Safe driving

[0669] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[0670] 3. Providing Feedback

[0671] Users have the ability to provide feedback through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded and sent to the server.

[0672] Specific examples

[0673] For example, on a weekday morning, while drivers are commuting to work and children are on their way to school, a community monitoring device may detect the presence of a child crossing a street. After the information is sent to the server, the server immediately confirms the presence of the child and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the server determines that the driver is stressed, it may also provide additional feedback such as, "Please drive calmly." In this way, the system of the present invention can detect the presence of a child in real time, grasp the driver's emotional state, and provide appropriate notifications, thereby preventing traffic accidents involving children and providing a safe driving environment.

[0674] The processing flow will be explained below.

[0675] Server Processing

[0676] Step 1: Receiving video data

[0677] The server receives real-time video data from local surveillance devices and user terminals, such as cameras monitoring children on their way to and from school.

[0678] Step 2: Preprocessing the video data

[0679] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0680] Step 3: Detect child presence

[0681] The server feeds the pre-processed video data into a machine learning algorithm to detect the presence of children, which then identifies people in the video and applies an age estimation model to identify children among them.

[0682] Step 4: Risk assessment

[0683] The server combines the child's presence data with surrounding traffic information to assess risk. For example, when a child is about to cross the road, it calculates the risk level based on the speed and location of nearby vehicles.

[0684] Step 5: Generate notifications

[0685] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, for example, a specific warning message such as "There is a child ahead, please slow down."

[0686] Step 6: Sending notifications

[0687] The server then sends the generated notification to the driver's communication device using a push notification service.

[0688] Step 7: Collect driving data

[0689] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[0690] Step 8: Data analysis and feedback generation

[0691] The server analyzes the collected driving data and provides feedback to support safe driving, such as generating a message like, "You braked too often. Please brake more gently."

[0692] Step 9: Collect emotion data

[0693] The server receives the driver's facial expressions and voice data and analyzes them with an emotion engine, thereby recognizing the driver's emotional state in real time.

[0694] Step 10: Emotion-Based Risk Assessment

[0695] The server performs a risk assessment based on the perceived emotional state: for example, if the driver is angry, it sets a higher risk level and applies stricter notification criteria.

[0696] Step 11: Analyze Long-Term Sentiment Data

[0697] The server monitors the driver's emotional state over a long period of time, accumulating and analyzing the data, thereby identifying the driver's individual stress patterns and fatigue levels and providing optimal feedback.

[0698] Terminal handling

[0699] Step 1: Receive notifications

[0700] The device receives notifications sent from the server in real time and warns the driver using a combination of visual and audio alerts.

[0701] Step 2: View notifications

[0702] The device analyzes the received notification and displays it to the driver. For example, a notification such as "There is a child ahead. Please slow down" may be displayed on the screen and read aloud.

[0703] Step 3: Collecting emotion data

[0704] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and passes this data to the emotion engine, which uses it to monitor the driver's emotional state in real time.

[0705] Step 4: Collect driving data

[0706] The device collects driving data using GPS and acceleration sensors, which are temporarily stored in a buffer and periodically sent to a server.

[0707] Step 5: Sending data

[0708] The device transmits the collected driving and emotion data to a server using a secure protocol (e.g., HTTPS).

[0709] User (driver) behavior

[0710] Step 1: Check notifications

[0711] The user checks the notification from the device, receives it both visually and audibly, and understands its content.

[0712] Step 2: Practice safe driving

[0713] The user can take appropriate driving actions based on the notification content. For example, if the user receives a notification that there is a child ahead, the user can slow down and check the surrounding area for safety.

[0714] Step 3: Emotional Management

[0715] Users are expected to pay attention to their emotional state and take appropriate driving behaviors, and the system provides feedback to help them manage stress and fatigue.

[0716] Step 4: Provide feedback

[0717] Users provide feedback to the system through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded within the app and sent to the server.

[0718] By implementing the above processing steps, the system of the present invention can prevent traffic accidents involving children and provide a safe driving environment. By taking the driver's emotional state into consideration, even more effective safe driving support can be achieved.

[0719] Example 2

[0720] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0721] Conventional area monitoring systems and safe driving support systems do not take into account the driver's psychological state, and therefore do not provide appropriate feedback or notifications according to the situation, resulting in limited effectiveness in preventing traffic accidents. Furthermore, there is no system that not only notifies drivers of the presence of children but also comprehensively evaluates the driver's emotional and behavioral data, which creates problems with risk management during parking. This does not promote safe driving behavior among drivers, and the risk of traffic accidents remains.

[0722] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0723] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication device when the presence of a child is detected, and means for adjusting the content of the notification based on the driver's emotional state using an emotion engine that analyzes the driver's emotional state, thereby enabling real-time risk assessment and promotion of safe driving that takes the driver's emotional state into consideration.

[0724] "Area monitoring device" refers to a hardware device that is deployed in a specific area to collect video and other data in real time.

[0725] "Video data" refers to video information acquired from a local monitoring device or a user terminal, and is digital data to be analyzed.

[0726] A "machine learning algorithm" is a computational method for analyzing large amounts of data and learning patterns from that data, and is used, for example, for person detection and age estimation.

[0727] The "means for detecting the presence of a child" is a system element for analyzing video data and identifying a child from within the video data.

[0728] The "driver's communication terminal" refers to a communication device such as a mobile phone, smartphone, or in-car device carried by the driver.

[0729] "Means for sending notification" refers to a communication method or device for immediately conveying detected information and risk assessment results to the driver.

[0730] The "emotion engine" is a system component that analyzes the driver's facial expressions and voice data to recognize their emotional state.

[0731] "Driver behavior data" refers to data that indicates the driving behavior of the driver, including speed, acceleration, use of sudden braking, and the like.

[0732] The "means for providing feedback" is a method or device that provides information or advice to the driver to support safe driving based on the analysis results.

[0733] The "integration means" is a system function that centrally analyzes video data acquired from multiple monitoring devices and grasps the overall situation.

[0734] "Means for optimizing road safety measures" are system elements for planning and implementing local measures to promote safe driving based on integrated data and driver emotion data.

[0735] This invention combines an emotion engine with an AI system that aims to prevent traffic accidents involving children using community monitoring devices and user terminals. The system recognizes the driver's emotional state and adjusts risk assessment and notification content to promote safe driving.

[0736] Hardware Configuration

[0737] The main hardware components of the system include area monitoring devices, servers, and user terminals. Area monitoring devices can be CCTV cameras or other video capture devices. The servers use high-performance computers and cloud services to perform data analysis and processing. User terminals are mainly smartphones or communication devices in cars.

[0738] Software Configuration

[0739] The main software used in the system is as follows:

[0740] OpenCV library: Used for preprocessing video data.

[0741] YOLO, SSD: Person detection as a machine learning algorithm.

[0742] Google Cloud Speech-to-Text API: Used to analyze audio data.

[0743] dlib library: Used for facial expression recognition.

[0744] Data processing flow

[0745] The server receives video data from the area surveillance devices in real time and preprocesses the received data to remove noise and adjust the resolution. The preprocessed video data is then fed into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. If a child is detected, the server assesses the risk and, if necessary, sends a notification to the driver's communication device. The notification is provided in audio and visual formats.

[0746] The server also collects and analyzes driver behavior data from GPS and vehicle sensors to provide feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive advice such as "Try to brake more gently."

[0747] Emotion Engine Operation

[0748] The server collects camera footage and audio data from the user's device and uses an emotion engine to recognize the driver's emotional state. Based on this emotional state, the server adjusts risk assessment and notification content. For example, if the driver is angry, the notification standards will be tightened and a warning such as "Please drive calmly" will be sent.

[0749] Specific examples

[0750] For example, on a weekday morning, while a driver is commuting to work and children are on their way to school, a community monitoring device detects the presence of a child crossing a crosswalk. This information is sent to a server, which immediately verifies it and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the system determines that the driver is stressed, it may provide additional feedback such as, "Please drive calmly." In this way, by detecting the presence of a child in real time and providing appropriate notifications while understanding the driver's emotional state, the system can prevent traffic accidents and provide a safe driving environment.

[0751] Prompt Sentence Examples

[0752] "Please explain the role of an AI system aimed at preventing traffic accidents involving children. The system incorporates an emotion engine to recognize the driver's emotional state and perform risk assessments. It also adjusts notifications to promote safe driving."

[0753] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0754] Step 1: Collect video data

[0755] The server receives video data from the area surveillance devices in real time. The input is the video data sent from the area surveillance devices, and the output is the raw video data processed within the server. Specifically, it acquires video streams from the surveillance cameras and transfers the data securely using the TCP / IP protocol.

[0756] Step 2: Preprocessing the video data

[0757] The server performs preprocessing on the received video data, such as noise removal, resolution adjustment, and background removal. The input is raw video data, and the output is preprocessed video data. Specifically, it uses the OpenCV library to apply background subtraction to remove noise and maintain a constant video resolution.

[0758] Step 3: Detect child presence

[0759] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is information on whether the presence of a child was detected. As a concrete example, a video frame is input into the YOLO model, the detection results are saved in JSON format, and the coordinate information where the child was identified is extracted.

[0760] Step 4: Risk assessment and notification generation

[0761] The server assesses risk by combining data on the child's presence with information on surrounding traffic. The inputs are data on the child's presence and traffic information, and the output is a notification message. For example, it obtains traffic information from a real-time traffic API and calculates risk by comparing the timing of a child crossing the street with the speed of the car. If the risk level is high, it sends a notification to the driver's device saying, "There is a child ahead. Please slow down."

[0762] Step 5: Collect and analyze driving data

[0763] The device continuously collects driver behavior data and periodically transmits it to the server. The input is GPS data and vehicle sensor data, and the output is driving data sent to the server. Specifically, the device collects vehicle speed and acceleration data via the smartphone's GPS sensor and OBD-II interface.

[0764] Step 6: Emotion Engine in Action

[0765] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and sends them to the server. The input is camera footage and audio data, and the output is the recognition result of the driver's emotional state. The server analyzes this data, performs facial expression recognition using the dlib library, and converts the audio data into text using the Google Cloud Speech-to-Text API to perform emotion analysis.

[0766] Step 7: Emotion-Based Risk Assessment

[0767] The server performs risk assessment based on the emotion recognition results. The input is the emotion recognition results, and the output is a risk level and notification content based on the emotion. For example, if it determines that the driver is angry, it applies stricter notification standards than usual and generates a warning message saying, "Please drive calmly."

[0768] Step 8: Analyze Long-Term Sentiment Data

[0769] The server monitors the driver's emotional state over the long term and analyzes the data. The input is past emotional data, and the output is long-term trend analysis and feedback. The emotional data is aggregated monthly and stress management advice is provided based on the analysis results. For example, relaxation suggestions are made to drivers who frequently feel stressed.

[0770] In this way, the system of the present invention is designed to prevent traffic accidents by detecting the presence of children in real time through multiple processing steps and sending appropriate notifications while taking into account the driver's emotional state.

[0771] (Application example 2)

[0772] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0773] The objective of this invention is to provide a system for preventing traffic accidents involving children in local areas and promoting safe driving. In particular, we aim to build a system that provides appropriate feedback on the driver's emotional state and performs real-time risk assessment to increase the driver's attention and support safe driving.

[0774] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data acquired from a local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal when the presence of a child is detected, means for analyzing image and audio data to recognize the driver's emotional state, and means for performing a risk assessment based on the driver's emotional state and adjusting the content of the notification. This makes it possible to detect the presence of a child in real time and send an appropriate notification according to the driver's emotional state.

[0775] A "regional monitoring device" is a device that is installed in a specific region and is used to collect video data.

[0776] "Video data" refers to image information acquired by an imaging device such as a camera.

[0777] A "machine learning algorithm" is a mathematical model or calculation method for automatically analyzing data and recognizing patterns.

[0778] "Means for detecting the presence of children" refers to a system that analyzes video data to determine whether a child is in the image.

[0779] "Driver" means a person operating and driving a vehicle.

[0780] A "communication terminal" is an electronic device capable of sending and receiving information. Specifically, it refers to a smartphone or similar device.

[0781] The "means for sending notification" is a mechanism for sending specific information to the driver's communication terminal.

[0782] "Emotional state" refers to the driver's mental state and emotional movements.

[0783] "Image and audio data" refers to visual and audio information acquired through a camera or microphone.

[0784] A "means for risk assessment" is a system for determining the degree of danger based on collected data.

[0785] The "means for adjusting notification content" is a mechanism for optimizing the content of notifications sent based on the driver's emotional state and detected risks.

[0786] The present invention provides a system for preventing traffic accidents involving children, using video data from local surveillance devices and the driver's communication terminal. The system has the function of recognizing the driver's emotional state and adjusting risk assessment and notification content based on that state.

[0787] Server Roles

[0788] 1. Video data collection

[0789] The server receives real-time video data from local surveillance devices, such as cameras installed around a school, capturing images of children walking to and from school. The received video data is analyzed according to the processing steps described below.

[0790] 2. Preprocessing of video data

[0791] The server performs preprocessing on the received video data, such as removing noise, adjusting resolution, and cutting out the background, and the preprocessed data is used as input data for machine learning algorithms.

[0792] 3. Child presence detection

[0793] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and then identifies children using an age estimation model.

[0794] 4. Risk Assessment and Notification Generation

[0795] The server combines the child presence data with traffic information to assess the risk, and if the risk level is high, generates and sends a notification to the driver's communication device, such as "There is a child ahead. Please slow down."

[0796] 5. Recognizing the driver's emotional state

[0797] The server analyzes the image and audio data collected from the communication device to recognize the driver's emotional state: facial expression recognition algorithms are applied to the image data, and voice analysis algorithms are applied to the audio data.

[0798] 6. Emotion-Based Risk Assessment

[0799] The server then performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards will be applied.

[0800] 7. Long-term emotional data analysis

[0801] The server monitors the driver's emotional state over the long term and analyzes the data, providing stress management advice to drivers who frequently experience stress.

[0802] Device Role

[0803] 1. Receiving and viewing notifications

[0804] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[0805] 2. Collecting Emotional Data

[0806] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and transmits the data to a server, which uses it to monitor the driver's emotional state in real time.

[0807] 3. Sending driving data

[0808] The device continuously collects driver behavior data (e.g., speed, acceleration, sudden braking, etc.) and periodically transmits it to the server, which provides feedback for safe driving.

[0809] User (driver) behavior

[0810] 1. Check notifications

[0811] Users check notifications from their devices, and audio alerts as well as visual notifications are used to draw attention.

[0812] 2. Safe driving

[0813] The user is then advised to take appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[0814] 3. Providing Feedback

[0815] Users have the ability to provide feedback through the application. For example, feedback such as "the notification was appropriate" or "there was a false alarm" can be recorded and sent to the server for future improvements.

[0816] Specific examples

[0817] For example, during the morning commute, a driver gets into their vehicle and launches the smartphone app. The app automatically receives video data from local surveillance devices and detects the presence of children. The driver's face is captured by a camera and their emotional state is analyzed using a facial recognition algorithm. If the app determines that the driver is stressed, a notification will appear saying, "There is a child ahead. Please drive calmly." Upon seeing this notification, the driver will slow down and take action to check their surroundings.

[0818] Example prompts to input to a generative AI model:

[0819] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[0820] As a result, the present invention provides a safe driving support system that takes into account the emotional state of the driver, thereby contributing to the prevention of traffic accidents involving children.

[0821] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0822] Step 1:

[0823] The server receives video data in real time from the local surveillance equipment. The input is raw video data from the local surveillance equipment, and the output is video data for preprocessing. Specifically, the video of children going to and from school is transmitted from the surveillance cameras installed around the school.

[0824] Step 2:

[0825] The server performs preprocessing on the received video data. The input is raw video data, and the output is preprocessed video data that has undergone noise removal, resolution adjustment, background removal, etc. This prepares the data so that the analysis algorithm can operate accurately.

[0826] Step 3:

[0827] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is data indicating the child's location. The algorithm identifies people in the video and uses an age estimation model to identify the child.

[0828] Step 4:

[0829] The server combines the presence data of the child and traffic information to perform risk assessment. The input is the presence data of the child and surrounding traffic information, and the output is assessment data indicating the risk level. If the risk level is high, a notification is generated.

[0830] Step 5:

[0831] The server sends a notification based on the risk assessment results to the driver's communication device. The input is risk level assessment data, and the output is notification information such as "There is a child ahead. Please slow down." This allows the driver to recognize danger early.

[0832] Step 6:

[0833] The server recognizes the driver's emotional state using image and voice data acquired from the driver's communication device. The input is the driver's facial image and voice tone data, and the output is the emotion recognition result. Facial expression recognition algorithms and voice analysis algorithms are used.

[0834] Step 7:

[0835] The server re-performs risk assessment based on the emotion recognition results. The inputs are the emotion recognition results and existing risk assessment data, and the output is adjusted risk assessment data based on the emotional state. For example, if the driver is angry, a more severe notification is generated.

[0836] Step 8:

[0837] The terminal receives notifications sent from the server in real time and displays them to the driver. The input is notification data, and the output is visual and audio notification displays. This allows the driver to immediately prepare to take appropriate driving actions.

[0838] Step 9:

[0839] The device continuously collects the driver's emotional data using a camera and microphone and sends it to the server. The input is the driver's facial expression and tone of voice data, and the output is sent to the server as emotional data. This data is used for subsequent emotional analysis.

[0840] Step 10:

[0841] The server analyzes the driver's behavioral data (speed, acceleration, sudden braking, etc.) and provides long-term feedback. The input is behavioral data, and the output is advice such as "Try to brake more gently."

[0842] Step 11:

[0843] The user checks the notification from the device and performs safe driving behavior. The input is the notification information from the device, and the output is the action taken by the driver. For example, upon receiving a notification that "there is a child ahead," the user slows down and checks their surroundings.

[0844] Example prompt sentence:

[0845] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[0846] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0847] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0848] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0849] [Third embodiment]

[0850] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0851] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0852] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0853] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0854] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0855] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0856] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0857] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0858] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0859] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0860] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0861] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0862] The present invention is an AI system for preventing traffic accidents involving children using a community monitoring device or a user terminal. Specific embodiments for carrying out the present invention will be described in detail below.

[0863] Server Roles

[0864] 1. Video data collection

[0865] The server receives video data in real time from local surveillance devices and user terminals. For example, surveillance cameras installed near an elementary school send video footage of children on their way to and from school.

[0866] 2. Analysis of video data

[0867] The server pre-processes the received video data and uses machine learning algorithms to detect the presence of children, specifically using models that can distinguish between different age groups and accurately distinguish between pedestrians and children.

[0868] 3. Risk Assessment and Notification Generation

[0869] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a car approaches while a child is crossing the street, the risk is deemed high. In this case, a notification is immediately sent to the driver's communication device, displaying a warning such as "There is a child ahead. Please slow down."

[0870] 4. Collecting and analyzing operational data

[0871] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.), providing real-time feedback to support safe driving. For example, if a driver frequently brakes suddenly, the system sends an alert to the driver saying, "Please try to brake more gently."

[0872] 5. Analysis of traffic conditions across the region

[0873] The server integrates data collected from multiple monitoring devices and analyzes traffic conditions across the entire region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if a particular intersection is experiencing frequent traffic accidents, specific recommendations can be made, such as installing traffic lights or relocating pedestrian crossings.

[0874] Device Role

[0875] 1. Receiving and viewing notifications

[0876] The device receives notifications sent from the server in real time and displays them to the user (driver). The notifications are delivered not only as visual displays but also as audio alerts, so drivers can immediately pay attention while driving.

[0877] 2. Sending driving data

[0878] The device collects driver behavior data and transmits it to a server, including GPS data and data from the vehicle's acceleration sensor. The data is sent periodically and is useful for analysis on the server side.

[0879] User (driver) behavior

[0880] 1. Check notifications and drive safely

[0881] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[0882] 2. Providing Feedback

[0883] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[0884] Specific examples

[0885] Let's say a camera monitoring an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device saying, "Children are at the intersection. Please slow down." The driver sees the notification, immediately reduces their speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[0886] In this way, the system of the present invention detects the presence of children in real time and notifies drivers, thereby preventing traffic accidents involving children and strengthening traffic safety measures throughout the region.

[0887] The processing flow will be explained below.

[0888] Server Processing

[0889] Step 1: Receiving video data

[0890] The server receives video data in real time from the area monitoring devices and user terminals using HTTP streaming or RTSP protocols, and the received data is temporarily stored in a buffer.

[0891] Step 2: Preprocessing the video data

[0892] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[0893] Step 3: Detect child presence

[0894] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[0895] Step 4: Risk assessment

[0896] The server evaluates the risk by combining the child's presence data with surrounding traffic information, taking into account the child's location, speed, and the movement of surrounding vehicles, among other factors, to calculate the risk level.

[0897] Step 5: Generate notifications

[0898] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, including details of the risk (e.g., "Child ahead. Be careful.").

[0899] Step 6: Sending notifications

[0900] The server sends the generated notification to the driver's communication device using a push notification service. The notification is formatted in JSON format and sent to the target device.

[0901] Step 7: Collect driving data

[0902] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[0903] Step 8: Data analysis and feedback generation

[0904] The server analyzes the collected driving data and generates feedback to support safe driving, such as a message saying, "You braked too often. Please try to brake more gently."

[0905] Terminal handling

[0906] Step 1: Receive notifications

[0907] The device receives notifications from the server in real time, which are not only displayed visually but also delivered to the driver as audio alerts.

[0908] Step 2: View notifications

[0909] The device analyzes the content of the received notification and displays it to the driver. Specifically, it displays the message "There is a child ahead. Please slow down" on the screen and issues a voice alert.

[0910] Step 3: Collect driving data

[0911] The device collects driver behavior data using the built-in GPS and acceleration sensors, and periodically stores the data in a buffer.

[0912] Step 4: Sending data

[0913] The device periodically transmits the collected driving data to a server using the HTTPS protocol, ensuring that the data is delivered securely to the server.

[0914] User (driver) behavior

[0915] Step 1: Check notifications

[0916] The user checks the notification from the device, and not only does it display a visual notification but also an audio alert to alert the driver.

[0917] Step 2: Practice safe driving

[0918] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and check their surroundings.

[0919] Step 3: Provide feedback

[0920] Users have the ability to provide feedback through the application, for example, by recording and sending feedback such as "the notification was appropriate" or "there was a false alarm."

[0921] Through the above processing steps, the present invention prevents traffic accidents involving children and provides a safe driving environment.

[0922] Example 1

[0923] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0924] In recent years, traffic accidents involving children have been increasing in urban and suburban areas. In particular, around schools and residential areas, drivers are unable to predict children's movements, resulting in unexpected accidents. To prevent such accidents, there is a need for a system that detects the presence of children and notifies drivers in real time. Furthermore, there is a need to reduce the risk of traffic accidents by collecting driver behavior data and providing feedback to promote safe driving. The development of a new system to solve these issues is required.

[0925] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0926] In this invention, the server includes means for receiving video data from a local monitoring device or a user terminal, means for preprocessing the video data and detecting the presence of a child using a machine learning algorithm, means for performing risk assessment based on the detected child's location information and sending a notification to the driver's communication terminal, means for collecting the driver's behavioral data and sending it to the server, and means for analyzing the driver's behavioral data and surrounding traffic information and providing feedback to support safe driving, thereby enabling real-time notifications to prevent traffic accidents involving children and supporting safe driving for the driver.

[0927] "Area monitoring devices" are devices such as cameras and sensors installed in specific areas to collect video data and other information.

[0928] A "user terminal" is a device used by a driver, such as a mobile phone, tablet, or car navigation system, that receives notifications from the server and displays them to the driver.

[0929] "Video data" refers to video or still image data acquired from a local monitoring device or a user terminal, and is information to be analyzed.

[0930] "Preprocessing" refers to processing performed to make video data easier to analyze, and includes operations such as noise removal, image correction, and frame extraction.

[0931] A "machine learning algorithm" is an algorithm that automatically learns data patterns and analyzes new data based on those patterns.

[0932] A "risk assessment" is an assessment of the dangers of a particular situation, based on the child's location and surrounding traffic conditions.

[0933] A "notification" is a message sent from the server to the driver's communication terminal, and is intended to warn or alert the driver visually or audibly.

[0934] "Driver behavior data" is information related to the driver's driving behavior, including speed, acceleration, GPS data, and the like.

[0935] "Feedback" refers to advice and warnings provided to assist safe driving based on collected driver behavior data.

[0936] "Traffic conditions" refers to information that indicates the state of traffic flow, congestion, accidents, etc. in a specific area.

[0937] "Integration" refers to the process of bringing together information obtained from multiple data sources, and is a necessary operation for analysis and evaluation.

[0938] "Traffic safety measures" are policies and measures taken to prevent traffic accidents and ensure traffic safety throughout the region.

[0939] The "feedback function" allows the driver to send their evaluations and opinions to the system, which will be used to improve the system.

[0940] The present invention is an AI system for preventing traffic accidents involving children, using a community monitoring device or a user terminal. Specific embodiments of the present invention will be described in detail below.

[0941] System hardware configuration

[0942] The system of the present invention is composed of the following main hardware components:

[0943] 1. Community surveillance devices: Surveillance cameras installed near elementary schools and residential areas.

[0944] 2. User devices: Mobile phones, tablets, and car navigation systems used by drivers.

[0945] 3. Server: A high-performance computer located in the cloud or a data center that performs data analysis.

[0946] System software configuration

[0947] The system of the present invention comprises the following software components:

[0948] 1. Data collection module: Software for receiving video data from area monitoring devices or user terminals.

[0949] 2. Pre-processing module: Software that removes noise from the received video data and corrects the image.

[0950] 3. Machine Learning Module: Software that detects the presence of children using machine learning algorithms such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector).

[0951] 4. Risk Assessment Module: Software that assesses risk based on the child's location and generates notification content.

[0952] 5. Notification sending module: Software for sending notifications to the driver's communication device.

[0953] 6. Data analysis module: Software that analyzes driver behavior data and surrounding traffic information, and provides feedback to support safe driving.

[0954] 7. Integrated Data Analysis Module: Software for analyzing traffic conditions across an entire region and optimizing traffic safety measures.

[0955] 8. Feedback Collection Module: Software that collects feedback from users and helps improve the system.

[0956] Example of operation

[0957] Suppose a surveillance camera captures a group of children crossing a street in front of an elementary school. The server receives the video data from the camera via the RTSP protocol. The pre-processing module removes noise and enhances the image, and the machine learning module detects the presence of children.

[0958] Next, the risk assessment module performs a risk assessment based on the detected child's location information and surrounding traffic information, and sends a notification to the driver's device saying, "A child is at the intersection. Please reduce your speed." The driver confirms this notification, reduces their speed, and continues driving safely.

[0959] Additionally, driving behavior data (GPS data and acceleration data) is sent from the device to a server. The server analyzes this data and provides real-time feedback to promote safe driving. For example, it sends an alert such as, "You are braking frequently. Please brake gently."

[0960] The server integrates data from the monitoring devices and analyzes traffic conditions across the entire region, and if accidents occur frequently at a particular intersection, it will suggest specific countermeasures such as installing traffic lights or relocating pedestrian crossings.

[0961] Example prompts for generative AI models

[0962] Prompt: "A surveillance camera captures a child crossing a street in front of an elementary school. Explain the specific steps a server takes in this footage, uses a machine learning model to detect the child, and then sends a notification to the driver."

[0963] As described above, the system of the present invention analyzes data from local monitoring devices and user terminals in real time and immediately notifies drivers, thereby preventing traffic accidents involving children and enhancing traffic safety throughout the entire region.

[0964] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0965] Step 1:

[0966] Video data collection

[0967] The server receives video data in real time from the local monitoring device and user terminals. The input is the video stream sent from the local monitoring device, and the output is the video data buffered in the server. Specifically, the server acquires the video data using the RTSP protocol and stores it in a buffer at a fixed frame rate (e.g., 30 frames per second).

[0968] Step 2:

[0969] Video data preprocessing

[0970] The server preprocesses the received video data. The input is the buffered video data, and the output is the corrected video data with noise removed. Specifically, it applies noise removal algorithms and inter-frame compensation algorithms to improve the image quality.

[0971] Step 3:

[0972] Child Detection

[0973] The server analyzes the preprocessed video data using machine learning algorithms (e.g., YOLO or SSD) to detect the presence of children. The input is the preprocessed video data, and the output is the location information of the detected child. Specifically, it uses a machine learning model to detect people and applies a classification algorithm to distinguish between children and other pedestrians.

[0974] Step 4:

[0975] Risk assessment and notification generation

[0976] The server performs a risk assessment based on the child's location information and surrounding traffic information, and generates a notification. The input is the detected child's location information and traffic information, and the output is the notification content sent to the driver's device. Specifically, it calculates the risk level using a certain risk assessment algorithm (for example, an evaluation function based on distance and speed), and generates a notification content such as "There is a child ahead. Please slow down."

[0977] Step 5:

[0978] Receiving and Viewing Notifications

[0979] The terminal receives notifications sent from the server and displays them to the user (driver). The input is the notification sent from the server, and the output is the warning message displayed to the driver. Specifically, the terminal waits for notifications from the server in the background, and when it receives one, it displays a pop-up message on the screen and plays an audio alert at the same time.

[0980] Step 6:

[0981] Safe driving practices

[0982] The user checks the notification from the device and drives safely. The input is the notification message displayed on the device, and the output is a driving maneuver to safely reduce speed. Specifically, the user acknowledges the notification, applies the brakes, slows down, and ensures the safety of the child while checking the surrounding traffic conditions.

[0983] Step 7:

[0984] Collecting and transmitting driving data

[0985] The terminal collects driver behavior data (GPS data and acceleration data) and sends it to a server. The input is behavior data acquired from the terminal's sensors, and the output is collected data sent to the server. Specifically, the terminal acquires data from the GPS sensor and acceleration sensor, aggregates the data at regular time intervals (for example, every second), and sends it to the server.

[0986] Step 8:

[0987] Providing Feedback

[0988] The server analyzes the driver's behavioral data and surrounding traffic information, and provides feedback to support safe driving. The input is the collected driving data and traffic information, and the output is feedback provided to the driver. Specifically, the server analyzes the behavioral data, generates advice and warnings to support safe driving (e.g., "You are braking frequently. Please brake gently"), and sends them to the driver's device.

[0989] (Application example 1)

[0990] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0991] This invention relates to a system for preventing traffic accidents involving children and improving the safety of autonomous vehicles. Specifically, the purpose is to reduce the risk of traffic accidents by detecting the presence of children in real time using a local monitoring device and a control system for autonomous vehicles and taking appropriate measures. Furthermore, the invention aims to implement comprehensive traffic safety measures by analyzing driver behavior data and traffic conditions throughout the region.

[0992] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0993] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal or the control system of the autonomous vehicle when the presence of a child is detected, and means for stopping or slowing down the vehicle based on the notification, thereby enabling the presence of a child to be detected in real time and appropriate driving operations to be performed.

[0994] An "area monitoring device" is a device that monitors a specific area or location and collects video data in real time.

[0995] "Video data" refers to data of moving images or still images acquired using equipment such as cameras or sensors.

[0996] A "machine learning algorithm" is an algorithm that automatically identifies specific patterns and features by having a computer learn from data.

[0997] "Means for detecting the presence of a child" refers to a system or method for analyzing video data to identify the child's appearance and confirm their location information.

[0998] A "driver's communication device" is an electronic device such as a smartphone or tablet carried by the driver that receives notifications and alerts.

[0999] An "autonomous vehicle control system" is a system for automatically operating and controlling a vehicle.

[1000] "Means for sending notification" refers to a system or method for sending detected information to the driver's communication terminal or the control system of the autonomous vehicle.

[1001] "Means for stopping or slowing a vehicle" means a system or method for automatically adjusting the speed of a vehicle to stop or slow down based on the notification.

[1002] "Driver behavior data" refers to data such as the speed, acceleration, and frequency of sudden braking when the driver operates the vehicle.

[1003] A "means for providing feedback" is a system or method for analyzing driver behavior data and providing advice or warnings to the driver for safe driving.

[1004] The "means for integrating video data" is a system or method for centrally consolidating video data acquired from multiple monitoring devices.

[1005] A "means for analyzing traffic conditions across an entire region" is a system or method for analyzing the integrated video data and identifying traffic conditions and problem areas in a specific region.

[1006] "Means for optimizing traffic safety measures" refers to a system or method for proposing and implementing appropriate traffic safety measures for each region based on the analysis results.

[1007] "Child presence information detected by the vehicle control system" refers to safety-related information such as the location of a child detected in real time by the vehicle control system.

[1008] The present invention relates to a system for preventing traffic accidents involving children, using a regional monitoring device and a control system for an autonomous vehicle. Specific embodiments for carrying out the invention will be described below.

[1009] Server Roles

[1010] 1. Video data collection

[1011] The server receives video data in real time from local surveillance devices and in-vehicle cameras. For example, video is transmitted from surveillance cameras installed around elementary schools and residential areas, or from cameras mounted on autonomous vehicles.

[1012] 2. Analysis of video data

[1013] The server preprocesses the received video data and uses machine learning algorithms to detect the presence of children. This uses a pre-trained generative AI model using a machine learning framework such as TensorFlow. The model can distinguish between different age groups and accurately distinguish between pedestrians and children.

[1014] 3. Risk Assessment and Notification Generation

[1015] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a vehicle approaches while a child is crossing the street, it is deemed to be at high risk. In this case, a notification is immediately sent to the driver's communication device and the autonomous vehicle's control system, issuing a warning such as "There is a child ahead. Please slow down."

[1016] 4. Collecting and analyzing operational data

[1017] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and autonomous vehicle control data. This provides real-time feedback to support safe driving. For example, a driver who frequently brakes suddenly may receive an alert saying, "Please try to brake more gently."

[1018] 5. Analysis of traffic conditions across the region

[1019] The server integrates data obtained from multiple monitoring devices and analyzes traffic conditions throughout the region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if traffic accidents frequently occur at a particular intersection, specific recommendations will be made, such as installing traffic lights or relocating pedestrian crossings. Furthermore, information on the presence of children detected by autonomous vehicles can also be reflected in safety measures for the entire region.

[1020] Device Role

[1021] 1. Receiving and viewing notifications

[1022] The device receives notifications sent from the server in real time and displays them to the driver. The notifications are delivered not only as visual displays but also as audio alerts, allowing drivers to immediately pay attention while driving. Furthermore, the autonomous vehicle control system automatically adjusts the vehicle's speed upon receiving the notification.

[1023] 2. Sending driving data

[1024] The device collects driver behavior data and transmits it to a server. This includes GPS data and data from the vehicle's acceleration sensor. In the case of autonomous vehicles, control information is also transmitted. The data is transmitted periodically and is useful for analysis on the server side.

[1025] User (driver) behavior

[1026] 1. Check notifications and drive safely

[1027] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[1028] 2. Providing Feedback

[1029] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[1030] Specific examples

[1031] Let's say a camera monitoring an area in front of an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device or the autonomous vehicle's control system saying, "Children are at the intersection. Please slow down." The driver or autonomous vehicle then checks the notification, immediately reduces speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[1032] Example prompt for a generative AI model:

[1033] "Suddenly, children start crossing the street. How should the AI ​​detection system react?"

[1034] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1035] Step 1:

[1036] The server receives video data in real time from area surveillance devices and on-board cameras of autonomous vehicles. This video data is transmitted from each camera through a dedicated communication channel and received by the server. The input is the video data from the surveillance cameras, and the output is the storage of this data on the server. The server temporarily stores the received video data in a buffer.

[1037] Step 2:

[1038] The server preprocesses the received video data. Specifically, it removes noise from the video data and normalizes it for each frame. It also resizes the video data as needed. The input is the video data stored on the server, and the output is the preprocessed video data. This allows the machine learning model to analyze the data efficiently.

[1039] Step 3:

[1040] The server sends the preprocessed video data to a machine learning algorithm to detect the presence of a child. This uses a generative AI model from TensorFlow. The input is the preprocessed video data, and the output is a determination of whether a child is present. The server determines the presence of a child for each frame based on the output of the AI ​​model.

[1041] Step 4:

[1042] The server performs a risk assessment based on the detected child's location and surrounding traffic information. The inputs are the judgment results from the AI ​​model and surrounding traffic information (e.g., the speed and distance of nearby vehicles), and the output is the risk assessment result. The server performs the risk assessment by taking into account multiple factors, and if the risk is high, it determines the necessary measures.

[1043] Step 5:

[1044] Based on the results of the risk assessment, the server sends a notification to the driver's communication device or the autonomous vehicle's control system. The input is the result of the risk assessment, and the output is a notification message. For example, a message saying "There is a child ahead. Please slow down." The notification is sent in real time, allowing the driver or vehicle system to respond immediately.

[1045] Step 6:

[1046] The driver's communication terminal or the autonomous vehicle's control system receives the notification from the server. The input is the notification message from the server, and the output is an alert display for the driver or speed adjustment by the vehicle control system. Specifically, a visual warning display or an audio alert is sounded, and the autonomous vehicle automatically adjusts its speed.

[1047] Step 7:

[1048] The terminal collects behavioral data of the driver or autonomous vehicle and sends it to the server. The input is the behavioral data acquired by the terminal, and the output is the data sent to the server. This behavioral data includes information on speed, acceleration, braking operation, etc. The data is periodically sent to the server and used for further analysis.

[1049] Step 8:

[1050] The server analyzes the received behavioral data and provides feedback to support safe driving. The input is behavioral data sent from the driver or the autonomous vehicle, and the output is a feedback message. For example, advice such as "Please do not brake suddenly too often" is provided to the driver.

[1051] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1052] The present invention combines an AI system using a community monitoring device or user terminal to prevent traffic accidents involving children with an emotion engine. This system recognizes the driver's emotional state and promotes safe driving by adjusting risk assessments and notification content. Specific embodiments for implementing the present invention are described in detail below.

[1053] Server Roles

[1054] 1. Video data collection

[1055] The server receives real-time video data from local surveillance devices and user terminals. For example, surveillance cameras installed around a school transmit images of children going to and from school.

[1056] 2. Preprocessing of video data

[1057] The server performs pre-processing on the received video data, which includes noise removal, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[1058] 3. Child presence detection

[1059] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[1060] 4. Risk Assessment and Notification Generation

[1061] The server combines data on the child's presence with surrounding traffic information to assess risk and generates a notification if the risk level is high. For example, if the server detects an approaching car while a child is crossing the street, it sends a warning to the driver's device saying, "There is a child ahead. Please slow down."

[1062] 5. Collecting and analyzing operational data

[1063] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and provides feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive an alert saying, "Please try to brake more gently."

[1064] 6. Operation of the Emotion Engine

[1065] The server analyzes the driver's facial expressions and tone of voice and recognizes their emotional state using an emotion engine, an algorithm for performing emotion analysis based on camera footage and audio data.

[1066] 7. Emotion-Based Risk Assessment

[1067] The server performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards are applied to further promote safe driving.

[1068] 8. Long-term emotional data analysis

[1069] The server monitors the driver's emotional state over the long term and analyzes the data to provide optimal feedback to each individual driver, for example, providing stress management advice to drivers who frequently feel stressed.

[1070] Device Role

[1071] 1. Receiving and viewing notifications

[1072] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[1073] 2. Collecting Emotional Data

[1074] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and sends this data to the emotion engine, which uses it to monitor the driver's emotional state in real time while driving.

[1075] 3. Sending driving data

[1076] The device continuously collects driver behavior data and periodically transmits it to a server, including GPS data and data from the vehicle's acceleration sensor.

[1077] User (driver) behavior

[1078] 1. Check notifications

[1079] The user checks the notification from the device, and the system not only displays a visual notification but also an audio alert to attract the driver's attention.

[1080] 2. Safe driving

[1081] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[1082] 3. Providing Feedback

[1083] Users have the ability to provide feedback through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded and sent to the server.

[1084] Specific examples

[1085] For example, on a weekday morning, while drivers are commuting to work and children are on their way to school, a community monitoring device may detect the presence of a child crossing a street. After the information is sent to the server, the server immediately confirms the presence of the child and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the server determines that the driver is stressed, it may also provide additional feedback such as, "Please drive calmly." In this way, the system of the present invention can detect the presence of a child in real time, grasp the driver's emotional state, and provide appropriate notifications, thereby preventing traffic accidents involving children and providing a safe driving environment.

[1086] The processing flow will be explained below.

[1087] Server Processing

[1088] Step 1: Receiving video data

[1089] The server receives real-time video data from local surveillance devices and user terminals, such as cameras monitoring children on their way to and from school.

[1090] Step 2: Preprocessing the video data

[1091] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[1092] Step 3: Detect child presence

[1093] The server feeds the pre-processed video data into a machine learning algorithm to detect the presence of children, which then identifies people in the video and applies an age estimation model to identify children among them.

[1094] Step 4: Risk assessment

[1095] The server combines the child's presence data with surrounding traffic information to assess risk. For example, when a child is about to cross the road, it calculates the risk level based on the speed and location of nearby vehicles.

[1096] Step 5: Generate notifications

[1097] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, for example, a specific warning message such as "There is a child ahead, please slow down."

[1098] Step 6: Sending notifications

[1099] The server then sends the generated notification to the driver's communication device using a push notification service.

[1100] Step 7: Collect driving data

[1101] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[1102] Step 8: Data analysis and feedback generation

[1103] The server analyzes the collected driving data and provides feedback to support safe driving, such as generating a message like, "You braked too often. Please brake more gently."

[1104] Step 9: Collect emotion data

[1105] The server receives the driver's facial expressions and voice data and analyzes them with an emotion engine, thereby recognizing the driver's emotional state in real time.

[1106] Step 10: Emotion-Based Risk Assessment

[1107] The server performs a risk assessment based on the perceived emotional state: for example, if the driver is angry, it sets a higher risk level and applies stricter notification criteria.

[1108] Step 11: Analyze Long-Term Sentiment Data

[1109] The server monitors the driver's emotional state over a long period of time, accumulating and analyzing the data, thereby identifying the driver's individual stress patterns and fatigue levels and providing optimal feedback.

[1110] Terminal handling

[1111] Step 1: Receive notifications

[1112] The device receives notifications sent from the server in real time and warns the driver using a combination of visual and audio alerts.

[1113] Step 2: View notifications

[1114] The device analyzes the received notification and displays it to the driver. For example, a notification such as "There is a child ahead. Please slow down" may be displayed on the screen and read aloud.

[1115] Step 3: Collecting emotion data

[1116] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and passes this data to the emotion engine, which uses it to monitor the driver's emotional state in real time.

[1117] Step 4: Collect driving data

[1118] The device collects driving data using GPS and acceleration sensors, which are temporarily stored in a buffer and periodically sent to a server.

[1119] Step 5: Sending data

[1120] The device transmits the collected driving and emotion data to a server using a secure protocol (e.g., HTTPS).

[1121] User (driver) behavior

[1122] Step 1: Check notifications

[1123] The user checks the notification from the device, receives it both visually and audibly, and understands its content.

[1124] Step 2: Practice safe driving

[1125] The user can take appropriate driving actions based on the notification content. For example, if the user receives a notification that there is a child ahead, the user can slow down and check the surrounding area for safety.

[1126] Step 3: Emotional Management

[1127] Users are expected to pay attention to their emotional state and take appropriate driving behaviors, and the system provides feedback to help them manage stress and fatigue.

[1128] Step 4: Provide feedback

[1129] Users provide feedback to the system through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded within the app and sent to the server.

[1130] By implementing the above processing steps, the system of the present invention can prevent traffic accidents involving children and provide a safe driving environment. By taking the driver's emotional state into consideration, even more effective safe driving support can be achieved.

[1131] Example 2

[1132] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] Conventional area monitoring systems and safe driving support systems do not take into account the driver's psychological state, and therefore do not provide appropriate feedback or notifications according to the situation, resulting in limited effectiveness in preventing traffic accidents. Furthermore, there is no system that not only notifies drivers of the presence of children but also comprehensively evaluates the driver's emotional and behavioral data, which creates problems with risk management during parking. This does not promote safe driving behavior among drivers, and the risk of traffic accidents remains.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1135] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication device when the presence of a child is detected, and means for adjusting the content of the notification based on the driver's emotional state using an emotion engine that analyzes the driver's emotional state, thereby enabling real-time risk assessment and promotion of safe driving that takes the driver's emotional state into consideration.

[1136] "Area monitoring device" refers to a hardware device that is deployed in a specific area to collect video and other data in real time.

[1137] "Video data" refers to video information acquired from a local monitoring device or a user terminal, and is digital data to be analyzed.

[1138] A "machine learning algorithm" is a computational method for analyzing large amounts of data and learning patterns from that data, and is used, for example, for person detection and age estimation.

[1139] The "means for detecting the presence of a child" is a system element for analyzing video data and identifying a child from within the video data.

[1140] The "driver's communication terminal" refers to a communication device such as a mobile phone, smartphone, or in-car device carried by the driver.

[1141] "Means for sending notification" refers to a communication method or device for immediately conveying detected information and risk assessment results to the driver.

[1142] The "emotion engine" is a system component that analyzes the driver's facial expressions and voice data to recognize their emotional state.

[1143] "Driver behavior data" refers to data that indicates the driving behavior of the driver, including speed, acceleration, use of sudden braking, and the like.

[1144] The "means for providing feedback" is a method or device that provides information or advice to the driver to support safe driving based on the analysis results.

[1145] The "integration means" is a system function that centrally analyzes video data acquired from multiple monitoring devices and grasps the overall situation.

[1146] "Means for optimizing road safety measures" are system elements for planning and implementing local measures to promote safe driving based on integrated data and driver emotion data.

[1147] This invention combines an emotion engine with an AI system that aims to prevent traffic accidents involving children using community monitoring devices and user terminals. The system recognizes the driver's emotional state and adjusts risk assessment and notification content to promote safe driving.

[1148] Hardware Configuration

[1149] The main hardware components of the system include area monitoring devices, servers, and user terminals. Area monitoring devices can be CCTV cameras or other video capture devices. The servers use high-performance computers and cloud services to perform data analysis and processing. User terminals are mainly smartphones or communication devices in cars.

[1150] Software Configuration

[1151] The main software used in the system is as follows:

[1152] OpenCV library: Used for preprocessing video data.

[1153] YOLO, SSD: Person detection as a machine learning algorithm.

[1154] Google Cloud Speech-to-Text API: Used to analyze audio data.

[1155] dlib library: Used for facial expression recognition.

[1156] Data processing flow

[1157] The server receives video data from the area surveillance devices in real time and preprocesses the received data to remove noise and adjust the resolution. The preprocessed video data is then fed into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. If a child is detected, the server assesses the risk and, if necessary, sends a notification to the driver's communication device. The notification is provided in audio and visual formats.

[1158] The server also collects and analyzes driver behavior data from GPS and vehicle sensors to provide feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive advice such as "Try to brake more gently."

[1159] Emotion Engine Operation

[1160] The server collects camera footage and audio data from the user's device and uses an emotion engine to recognize the driver's emotional state. Based on this emotional state, the server adjusts risk assessment and notification content. For example, if the driver is angry, the notification standards will be tightened and a warning such as "Please drive calmly" will be sent.

[1161] Specific examples

[1162] For example, on a weekday morning, while a driver is commuting to work and children are on their way to school, a community monitoring device detects the presence of a child crossing a crosswalk. This information is sent to a server, which immediately verifies it and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the system determines that the driver is stressed, it may provide additional feedback such as, "Please drive calmly." In this way, by detecting the presence of a child in real time and providing appropriate notifications while understanding the driver's emotional state, the system can prevent traffic accidents and provide a safe driving environment.

[1163] Prompt Sentence Examples

[1164] "Please explain the role of an AI system aimed at preventing traffic accidents involving children. The system incorporates an emotion engine to recognize the driver's emotional state and perform risk assessments. It also adjusts notifications to promote safe driving."

[1165] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1166] Step 1: Collect video data

[1167] The server receives video data from the area surveillance devices in real time. The input is the video data sent from the area surveillance devices, and the output is the raw video data processed within the server. Specifically, it acquires video streams from the surveillance cameras and transfers the data securely using the TCP / IP protocol.

[1168] Step 2: Preprocessing the video data

[1169] The server performs preprocessing on the received video data, such as noise removal, resolution adjustment, and background removal. The input is raw video data, and the output is preprocessed video data. Specifically, it uses the OpenCV library to apply background subtraction to remove noise and maintain a constant video resolution.

[1170] Step 3: Detect child presence

[1171] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is information on whether the presence of a child was detected. As a concrete example, a video frame is input into the YOLO model, the detection results are saved in JSON format, and the coordinate information where the child was identified is extracted.

[1172] Step 4: Risk assessment and notification generation

[1173] The server assesses risk by combining data on the child's presence with information on surrounding traffic. The inputs are data on the child's presence and traffic information, and the output is a notification message. For example, it obtains traffic information from a real-time traffic API and calculates risk by comparing the timing of a child crossing the street with the speed of the car. If the risk level is high, it sends a notification to the driver's device saying, "There is a child ahead. Please slow down."

[1174] Step 5: Collect and analyze driving data

[1175] The device continuously collects driver behavior data and periodically transmits it to the server. The input is GPS data and vehicle sensor data, and the output is driving data sent to the server. Specifically, the device collects vehicle speed and acceleration data via the smartphone's GPS sensor and OBD-II interface.

[1176] Step 6: Emotion Engine in Action

[1177] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and sends them to the server. The input is camera footage and audio data, and the output is the recognition result of the driver's emotional state. The server analyzes this data, performs facial expression recognition using the dlib library, and converts the audio data into text using the Google Cloud Speech-to-Text API to perform emotion analysis.

[1178] Step 7: Emotion-Based Risk Assessment

[1179] The server performs risk assessment based on the emotion recognition results. The input is the emotion recognition results, and the output is a risk level and notification content based on the emotion. For example, if it determines that the driver is angry, it applies stricter notification standards than usual and generates a warning message saying, "Please drive calmly."

[1180] Step 8: Analyze Long-Term Sentiment Data

[1181] The server monitors the driver's emotional state over the long term and analyzes the data. The input is past emotional data, and the output is long-term trend analysis and feedback. The emotional data is aggregated monthly and stress management advice is provided based on the analysis results. For example, relaxation suggestions are made to drivers who frequently feel stressed.

[1182] In this way, the system of the present invention is designed to prevent traffic accidents by detecting the presence of children in real time through multiple processing steps and sending appropriate notifications while taking into account the driver's emotional state.

[1183] (Application example 2)

[1184] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] The objective of this invention is to provide a system for preventing traffic accidents involving children in local areas and promoting safe driving. In particular, we aim to build a system that provides appropriate feedback on the driver's emotional state and performs real-time risk assessment to increase the driver's attention and support safe driving.

[1186] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data acquired from a local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal when the presence of a child is detected, means for analyzing image and audio data to recognize the driver's emotional state, and means for performing a risk assessment based on the driver's emotional state and adjusting the content of the notification. This makes it possible to detect the presence of a child in real time and send an appropriate notification according to the driver's emotional state.

[1187] A "regional monitoring device" is a device that is installed in a specific region and is used to collect video data.

[1188] "Video data" refers to image information acquired by an imaging device such as a camera.

[1189] A "machine learning algorithm" is a mathematical model or calculation method for automatically analyzing data and recognizing patterns.

[1190] "Means for detecting the presence of children" refers to a system that analyzes video data to determine whether a child is in the image.

[1191] "Driver" means a person operating and driving a vehicle.

[1192] A "communication terminal" is an electronic device capable of sending and receiving information. Specifically, it refers to a smartphone or similar device.

[1193] The "means for sending notification" is a mechanism for sending specific information to the driver's communication terminal.

[1194] "Emotional state" refers to the driver's mental state and emotional movements.

[1195] "Image and audio data" refers to visual and audio information acquired through a camera or microphone.

[1196] A "means for risk assessment" is a system for determining the degree of danger based on collected data.

[1197] The "means for adjusting notification content" is a mechanism for optimizing the content of notifications sent based on the driver's emotional state and detected risks.

[1198] The present invention provides a system for preventing traffic accidents involving children, using video data from local surveillance devices and the driver's communication terminal. The system has the function of recognizing the driver's emotional state and adjusting risk assessment and notification content based on that state.

[1199] Server Roles

[1200] 1. Video data collection

[1201] The server receives real-time video data from local surveillance devices, such as cameras installed around a school, capturing images of children walking to and from school. The received video data is analyzed according to the processing steps described below.

[1202] 2. Preprocessing of video data

[1203] The server performs preprocessing on the received video data, such as removing noise, adjusting resolution, and cutting out the background, and the preprocessed data is used as input data for machine learning algorithms.

[1204] 3. Child presence detection

[1205] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and then identifies children using an age estimation model.

[1206] 4. Risk Assessment and Notification Generation

[1207] The server combines the child presence data with traffic information to assess the risk, and if the risk level is high, generates and sends a notification to the driver's communication device, such as "There is a child ahead. Please slow down."

[1208] 5. Recognizing the driver's emotional state

[1209] The server analyzes the image and audio data collected from the communication device to recognize the driver's emotional state: facial expression recognition algorithms are applied to the image data, and voice analysis algorithms are applied to the audio data.

[1210] 6. Emotion-Based Risk Assessment

[1211] The server then performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards will be applied.

[1212] 7. Long-term emotional data analysis

[1213] The server monitors the driver's emotional state over the long term and analyzes the data, providing stress management advice to drivers who frequently experience stress.

[1214] Device Role

[1215] 1. Receiving and viewing notifications

[1216] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[1217] 2. Collecting Emotional Data

[1218] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and transmits the data to a server, which uses it to monitor the driver's emotional state in real time.

[1219] 3. Sending driving data

[1220] The device continuously collects driver behavior data (e.g., speed, acceleration, sudden braking, etc.) and periodically transmits it to the server, which provides feedback for safe driving.

[1221] User (driver) behavior

[1222] 1. Check notifications

[1223] Users check notifications from their devices, and audio alerts as well as visual notifications are used to draw attention.

[1224] 2. Safe driving

[1225] The user is then advised to take appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[1226] 3. Providing Feedback

[1227] Users have the ability to provide feedback through the application. For example, feedback such as "the notification was appropriate" or "there was a false alarm" can be recorded and sent to the server for future improvements.

[1228] Specific examples

[1229] For example, during the morning commute, a driver gets into their vehicle and launches the smartphone app. The app automatically receives video data from local surveillance devices and detects the presence of children. The driver's face is captured by a camera and their emotional state is analyzed using a facial recognition algorithm. If the app determines that the driver is stressed, a notification will appear saying, "There is a child ahead. Please drive calmly." Upon seeing this notification, the driver will slow down and take action to check their surroundings.

[1230] Example prompts to input to a generative AI model:

[1231] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[1232] As a result, the present invention provides a safe driving support system that takes into account the emotional state of the driver, thereby contributing to the prevention of traffic accidents involving children.

[1233] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1234] Step 1:

[1235] The server receives video data in real time from the local surveillance equipment. The input is raw video data from the local surveillance equipment, and the output is video data for preprocessing. Specifically, the video of children going to and from school is transmitted from the surveillance cameras installed around the school.

[1236] Step 2:

[1237] The server performs preprocessing on the received video data. The input is raw video data, and the output is preprocessed video data that has undergone noise removal, resolution adjustment, background removal, etc. This prepares the data so that the analysis algorithm can operate accurately.

[1238] Step 3:

[1239] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is data indicating the child's location. The algorithm identifies people in the video and uses an age estimation model to identify the child.

[1240] Step 4:

[1241] The server combines the presence data of the child and traffic information to perform risk assessment. The input is the presence data of the child and surrounding traffic information, and the output is assessment data indicating the risk level. If the risk level is high, a notification is generated.

[1242] Step 5:

[1243] The server sends a notification based on the risk assessment results to the driver's communication device. The input is risk level assessment data, and the output is notification information such as "There is a child ahead. Please slow down." This allows the driver to recognize danger early.

[1244] Step 6:

[1245] The server recognizes the driver's emotional state using image and voice data acquired from the driver's communication device. The input is the driver's facial image and voice tone data, and the output is the emotion recognition result. Facial expression recognition algorithms and voice analysis algorithms are used.

[1246] Step 7:

[1247] The server re-performs risk assessment based on the emotion recognition results. The inputs are the emotion recognition results and existing risk assessment data, and the output is adjusted risk assessment data based on the emotional state. For example, if the driver is angry, a more severe notification is generated.

[1248] Step 8:

[1249] The terminal receives notifications sent from the server in real time and displays them to the driver. The input is notification data, and the output is visual and audio notification displays. This allows the driver to immediately prepare to take appropriate driving actions.

[1250] Step 9:

[1251] The device continuously collects the driver's emotional data using a camera and microphone and sends it to the server. The input is the driver's facial expression and tone of voice data, and the output is sent to the server as emotional data. This data is used for subsequent emotional analysis.

[1252] Step 10:

[1253] The server analyzes the driver's behavioral data (speed, acceleration, sudden braking, etc.) and provides long-term feedback. The input is behavioral data, and the output is advice such as "Try to brake more gently."

[1254] Step 11:

[1255] The user checks the notification from the device and performs safe driving behavior. The input is the notification information from the device, and the output is the action taken by the driver. For example, upon receiving a notification that "there is a child ahead," the user slows down and checks their surroundings.

[1256] Example prompt sentence:

[1257] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[1258] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1259] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1260] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1261] [Fourth embodiment]

[1262] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1263] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1264] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1265] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1266] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1267] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1268] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1269] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1270] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1271] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1272] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1273] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1274] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1275] The present invention is an AI system for preventing traffic accidents involving children using a community monitoring device or a user terminal. Specific embodiments for carrying out the present invention will be described in detail below.

[1276] Server Roles

[1277] 1. Video data collection

[1278] The server receives video data in real time from local surveillance devices and user terminals. For example, surveillance cameras installed near an elementary school send video footage of children on their way to and from school.

[1279] 2. Analysis of video data

[1280] The server pre-processes the received video data and uses machine learning algorithms to detect the presence of children, specifically using models that can distinguish between different age groups and accurately distinguish between pedestrians and children.

[1281] 3. Risk Assessment and Notification Generation

[1282] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a car approaches while a child is crossing the street, the risk is deemed high. In this case, a notification is immediately sent to the driver's communication device, displaying a warning such as "There is a child ahead. Please slow down."

[1283] 4. Collecting and analyzing operational data

[1284] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.), providing real-time feedback to support safe driving. For example, if a driver frequently brakes suddenly, the system sends an alert to the driver saying, "Please try to brake more gently."

[1285] 5. Analysis of traffic conditions across the region

[1286] The server integrates data collected from multiple monitoring devices and analyzes traffic conditions across the entire region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if a particular intersection is experiencing frequent traffic accidents, specific recommendations can be made, such as installing traffic lights or relocating pedestrian crossings.

[1287] Device Role

[1288] 1. Receiving and viewing notifications

[1289] The device receives notifications sent from the server in real time and displays them to the user (driver). The notifications are delivered not only as visual displays but also as audio alerts, so drivers can immediately pay attention while driving.

[1290] 2. Sending driving data

[1291] The device collects driver behavior data and transmits it to a server, including GPS data and data from the vehicle's acceleration sensor. The data is sent periodically and is useful for analysis on the server side.

[1292] User (driver) behavior

[1293] 1. Check notifications and drive safely

[1294] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[1295] 2. Providing Feedback

[1296] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[1297] Specific examples

[1298] Let's say a camera monitoring an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device saying, "Children are at the intersection. Please slow down." The driver sees the notification, immediately reduces their speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[1299] In this way, the system of the present invention detects the presence of children in real time and notifies drivers, thereby preventing traffic accidents involving children and strengthening traffic safety measures throughout the region.

[1300] The processing flow will be explained below.

[1301] Server Processing

[1302] Step 1: Receiving video data

[1303] The server receives video data in real time from the area monitoring devices and user terminals using HTTP streaming or RTSP protocols, and the received data is temporarily stored in a buffer.

[1304] Step 2: Preprocessing the video data

[1305] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[1306] Step 3: Detect child presence

[1307] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[1308] Step 4: Risk assessment

[1309] The server evaluates the risk by combining the child's presence data with surrounding traffic information, taking into account the child's location, speed, and the movement of surrounding vehicles, among other factors, to calculate the risk level.

[1310] Step 5: Generate notifications

[1311] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, including details of the risk (e.g., "Child ahead. Be careful.").

[1312] Step 6: Sending notifications

[1313] The server sends the generated notification to the driver's communication device using a push notification service. The notification is formatted in JSON format and sent to the target device.

[1314] Step 7: Collect driving data

[1315] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[1316] Step 8: Data analysis and feedback generation

[1317] The server analyzes the collected driving data and generates feedback to support safe driving, such as a message saying, "You braked too often. Please try to brake more gently."

[1318] Terminal handling

[1319] Step 1: Receive notifications

[1320] The device receives notifications from the server in real time, which are not only displayed visually but also delivered to the driver as audio alerts.

[1321] Step 2: View notifications

[1322] The device analyzes the content of the received notification and displays it to the driver. Specifically, it displays the message "There is a child ahead. Please slow down" on the screen and issues a voice alert.

[1323] Step 3: Collect driving data

[1324] The device collects driver behavior data using the built-in GPS and acceleration sensors, and periodically stores the data in a buffer.

[1325] Step 4: Sending data

[1326] The device periodically transmits the collected driving data to a server using the HTTPS protocol, ensuring that the data is delivered securely to the server.

[1327] User (driver) behavior

[1328] Step 1: Check notifications

[1329] The user checks the notification from the device, and not only does it display a visual notification but also an audio alert to alert the driver.

[1330] Step 2: Practice safe driving

[1331] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and check their surroundings.

[1332] Step 3: Provide feedback

[1333] Users have the ability to provide feedback through the application, for example, by recording and sending feedback such as "the notification was appropriate" or "there was a false alarm."

[1334] Through the above processing steps, the present invention prevents traffic accidents involving children and provides a safe driving environment.

[1335] Example 1

[1336] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1337] In recent years, traffic accidents involving children have been increasing in urban and suburban areas. In particular, around schools and residential areas, drivers are unable to predict children's movements, resulting in unexpected accidents. To prevent such accidents, there is a need for a system that detects the presence of children and notifies drivers in real time. Furthermore, there is a need to reduce the risk of traffic accidents by collecting driver behavior data and providing feedback to promote safe driving. The development of a new system to solve these issues is required.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1339] In this invention, the server includes means for receiving video data from a local monitoring device or a user terminal, means for preprocessing the video data and detecting the presence of a child using a machine learning algorithm, means for performing risk assessment based on the detected child's location information and sending a notification to the driver's communication terminal, means for collecting the driver's behavioral data and sending it to the server, and means for analyzing the driver's behavioral data and surrounding traffic information and providing feedback to support safe driving, thereby enabling real-time notifications to prevent traffic accidents involving children and supporting safe driving for the driver.

[1340] "Area monitoring devices" are devices such as cameras and sensors installed in specific areas to collect video data and other information.

[1341] A "user terminal" is a device used by a driver, such as a mobile phone, tablet, or car navigation system, that receives notifications from the server and displays them to the driver.

[1342] "Video data" refers to video or still image data acquired from a local monitoring device or a user terminal, and is information to be analyzed.

[1343] "Preprocessing" refers to processing performed to make video data easier to analyze, and includes operations such as noise removal, image correction, and frame extraction.

[1344] A "machine learning algorithm" is an algorithm that automatically learns data patterns and analyzes new data based on those patterns.

[1345] A "risk assessment" is an assessment of the dangers of a particular situation, based on the child's location and surrounding traffic conditions.

[1346] A "notification" is a message sent from the server to the driver's communication terminal, and is intended to warn or alert the driver visually or audibly.

[1347] "Driver behavior data" is information related to the driver's driving behavior, including speed, acceleration, GPS data, and the like.

[1348] "Feedback" refers to advice and warnings provided to assist safe driving based on collected driver behavior data.

[1349] "Traffic conditions" refers to information that indicates the state of traffic flow, congestion, accidents, etc. in a specific area.

[1350] "Integration" refers to the process of bringing together information obtained from multiple data sources, and is a necessary operation for analysis and evaluation.

[1351] "Traffic safety measures" are policies and measures taken to prevent traffic accidents and ensure traffic safety throughout the region.

[1352] The "feedback function" allows the driver to send their evaluations and opinions to the system, which will be used to improve the system.

[1353] The present invention is an AI system for preventing traffic accidents involving children, using a community monitoring device or a user terminal. Specific embodiments of the present invention will be described in detail below.

[1354] System hardware configuration

[1355] The system of the present invention is composed of the following main hardware components:

[1356] 1. Community surveillance devices: Surveillance cameras installed near elementary schools and residential areas.

[1357] 2. User devices: Mobile phones, tablets, and car navigation systems used by drivers.

[1358] 3. Server: A high-performance computer located in the cloud or a data center that performs data analysis.

[1359] System software configuration

[1360] The system of the present invention comprises the following software components:

[1361] 1. Data collection module: Software for receiving video data from area monitoring devices or user terminals.

[1362] 2. Pre-processing module: Software that removes noise from the received video data and corrects the image.

[1363] 3. Machine Learning Module: Software that detects the presence of children using machine learning algorithms such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector).

[1364] 4. Risk Assessment Module: Software that assesses risk based on the child's location and generates notification content.

[1365] 5. Notification sending module: Software for sending notifications to the driver's communication device.

[1366] 6. Data analysis module: Software that analyzes driver behavior data and surrounding traffic information, and provides feedback to support safe driving.

[1367] 7. Integrated Data Analysis Module: Software for analyzing traffic conditions across an entire region and optimizing traffic safety measures.

[1368] 8. Feedback Collection Module: Software that collects feedback from users and helps improve the system.

[1369] Example of operation

[1370] Suppose a surveillance camera captures a group of children crossing a street in front of an elementary school. The server receives the video data from the camera via the RTSP protocol. The pre-processing module removes noise and enhances the image, and the machine learning module detects the presence of children.

[1371] Next, the risk assessment module performs a risk assessment based on the detected child's location information and surrounding traffic information, and sends a notification to the driver's device saying, "A child is at the intersection. Please reduce your speed." The driver confirms this notification, reduces their speed, and continues driving safely.

[1372] Additionally, driving behavior data (GPS data and acceleration data) is sent from the device to a server. The server analyzes this data and provides real-time feedback to promote safe driving. For example, it sends an alert such as, "You are braking frequently. Please brake gently."

[1373] The server integrates data from the monitoring devices and analyzes traffic conditions across the entire region, and if accidents occur frequently at a particular intersection, it will suggest specific countermeasures such as installing traffic lights or relocating pedestrian crossings.

[1374] Example prompts for generative AI models

[1375] Prompt: "A surveillance camera captures a child crossing a street in front of an elementary school. Explain the specific steps a server takes in this footage, uses a machine learning model to detect the child, and then sends a notification to the driver."

[1376] As described above, the system of the present invention analyzes data from local monitoring devices and user terminals in real time and immediately notifies drivers, thereby preventing traffic accidents involving children and enhancing traffic safety throughout the entire region.

[1377] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1378] Step 1:

[1379] Video data collection

[1380] The server receives video data in real time from the local monitoring device and user terminals. The input is the video stream sent from the local monitoring device, and the output is the video data buffered in the server. Specifically, the server acquires the video data using the RTSP protocol and stores it in a buffer at a fixed frame rate (e.g., 30 frames per second).

[1381] Step 2:

[1382] Video data preprocessing

[1383] The server preprocesses the received video data. The input is the buffered video data, and the output is the corrected video data with noise removed. Specifically, it applies noise removal algorithms and inter-frame compensation algorithms to improve the image quality.

[1384] Step 3:

[1385] Child Detection

[1386] The server analyzes the preprocessed video data using machine learning algorithms (e.g., YOLO or SSD) to detect the presence of children. The input is the preprocessed video data, and the output is the location information of the detected child. Specifically, it uses a machine learning model to detect people and applies a classification algorithm to distinguish between children and other pedestrians.

[1387] Step 4:

[1388] Risk assessment and notification generation

[1389] The server performs a risk assessment based on the child's location information and surrounding traffic information, and generates a notification. The input is the detected child's location information and traffic information, and the output is the notification content sent to the driver's device. Specifically, it calculates the risk level using a certain risk assessment algorithm (for example, an evaluation function based on distance and speed), and generates a notification content such as "There is a child ahead. Please slow down."

[1390] Step 5:

[1391] Receiving and Viewing Notifications

[1392] The terminal receives notifications sent from the server and displays them to the user (driver). The input is the notification sent from the server, and the output is the warning message displayed to the driver. Specifically, the terminal waits for notifications from the server in the background, and when it receives one, it displays a pop-up message on the screen and plays an audio alert at the same time.

[1393] Step 6:

[1394] Safe driving practices

[1395] The user checks the notification from the device and drives safely. The input is the notification message displayed on the device, and the output is a driving maneuver to safely reduce speed. Specifically, the user acknowledges the notification, applies the brakes, slows down, and ensures the safety of the child while checking the surrounding traffic conditions.

[1396] Step 7:

[1397] Collecting and transmitting driving data

[1398] The terminal collects driver behavior data (GPS data and acceleration data) and sends it to a server. The input is behavior data acquired from the terminal's sensors, and the output is collected data sent to the server. Specifically, the terminal acquires data from the GPS sensor and acceleration sensor, aggregates the data at regular time intervals (for example, every second), and sends it to the server.

[1399] Step 8:

[1400] Providing Feedback

[1401] The server analyzes the driver's behavioral data and surrounding traffic information, and provides feedback to support safe driving. The input is the collected driving data and traffic information, and the output is feedback provided to the driver. Specifically, the server analyzes the behavioral data, generates advice and warnings to support safe driving (e.g., "You are braking frequently. Please brake gently"), and sends them to the driver's device.

[1402] (Application example 1)

[1403] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] This invention relates to a system for preventing traffic accidents involving children and improving the safety of autonomous vehicles. Specifically, the purpose is to reduce the risk of traffic accidents by detecting the presence of children in real time using a local monitoring device and a control system for autonomous vehicles and taking appropriate measures. Furthermore, the invention aims to implement comprehensive traffic safety measures by analyzing driver behavior data and traffic conditions throughout the region.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1406] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal or the control system of the autonomous vehicle when the presence of a child is detected, and means for stopping or slowing down the vehicle based on the notification, thereby enabling the presence of a child to be detected in real time and appropriate driving operations to be performed.

[1407] An "area monitoring device" is a device that monitors a specific area or location and collects video data in real time.

[1408] "Video data" refers to data of moving images or still images acquired using equipment such as cameras or sensors.

[1409] A "machine learning algorithm" is an algorithm that automatically identifies specific patterns and features by having a computer learn from data.

[1410] "Means for detecting the presence of a child" refers to a system or method for analyzing video data to identify the child's appearance and confirm their location information.

[1411] A "driver's communication device" is an electronic device such as a smartphone or tablet carried by the driver that receives notifications and alerts.

[1412] An "autonomous vehicle control system" is a system for automatically operating and controlling a vehicle.

[1413] "Means for sending notification" refers to a system or method for sending detected information to the driver's communication terminal or the control system of the autonomous vehicle.

[1414] "Means for stopping or slowing a vehicle" means a system or method for automatically adjusting the speed of a vehicle to stop or slow down based on the notification.

[1415] "Driver behavior data" refers to data such as the speed, acceleration, and frequency of sudden braking when the driver operates the vehicle.

[1416] A "means for providing feedback" is a system or method for analyzing driver behavior data and providing advice or warnings to the driver for safe driving.

[1417] The "means for integrating video data" is a system or method for centrally consolidating video data acquired from multiple monitoring devices.

[1418] A "means for analyzing traffic conditions across an entire region" is a system or method for analyzing the integrated video data and identifying traffic conditions and problem areas in a specific region.

[1419] "Means for optimizing traffic safety measures" refers to a system or method for proposing and implementing appropriate traffic safety measures for each region based on the analysis results.

[1420] "Child presence information detected by the vehicle control system" refers to safety-related information such as the location of a child detected in real time by the vehicle control system.

[1421] The present invention relates to a system for preventing traffic accidents involving children, using a regional monitoring device and a control system for an autonomous vehicle. Specific embodiments for carrying out the invention will be described below.

[1422] Server Roles

[1423] 1. Video data collection

[1424] The server receives video data in real time from local surveillance devices and in-vehicle cameras. For example, video is transmitted from surveillance cameras installed around elementary schools and residential areas, or from cameras mounted on autonomous vehicles.

[1425] 2. Analysis of video data

[1426] The server preprocesses the received video data and uses machine learning algorithms to detect the presence of children. This uses a pre-trained generative AI model using a machine learning framework such as TensorFlow. The model can distinguish between different age groups and accurately distinguish between pedestrians and children.

[1427] 3. Risk Assessment and Notification Generation

[1428] The server performs a risk assessment based on the detected child's location and surrounding traffic information. For example, if a vehicle approaches while a child is crossing the street, it is deemed to be at high risk. In this case, a notification is immediately sent to the driver's communication device and the autonomous vehicle's control system, issuing a warning such as "There is a child ahead. Please slow down."

[1429] 4. Collecting and analyzing operational data

[1430] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and autonomous vehicle control data. This provides real-time feedback to support safe driving. For example, a driver who frequently brakes suddenly may receive an alert saying, "Please try to brake more gently."

[1431] 5. Analysis of traffic conditions across the region

[1432] The server integrates data obtained from multiple monitoring devices and analyzes traffic conditions throughout the region. This data can be used to identify dangerous areas and optimize road safety measures. For example, if traffic accidents frequently occur at a particular intersection, specific recommendations will be made, such as installing traffic lights or relocating pedestrian crossings. Furthermore, information on the presence of children detected by autonomous vehicles can also be reflected in safety measures for the entire region.

[1433] Device Role

[1434] 1. Receiving and viewing notifications

[1435] The device receives notifications sent from the server in real time and displays them to the driver. The notifications are delivered not only as visual displays but also as audio alerts, allowing drivers to immediately pay attention while driving. Furthermore, the autonomous vehicle control system automatically adjusts the vehicle's speed upon receiving the notification.

[1436] 2. Sending driving data

[1437] The device collects driver behavior data and transmits it to a server. This includes GPS data and data from the vehicle's acceleration sensor. In the case of autonomous vehicles, control information is also transmitted. The data is transmitted periodically and is useful for analysis on the server side.

[1438] User (driver) behavior

[1439] 1. Check notifications and drive safely

[1440] When users receive a notification from their device, they are encouraged to follow the instructions to drive safely. For example, if they receive a notification that there is a child ahead, they should immediately slow down and respond more carefully to the surrounding situation.

[1441] 2. Providing Feedback

[1442] Users can provide feedback through the application, such as "The notification was appropriate" or "There was a false alarm." This feedback is collected on the server and used to improve the system.

[1443] Specific examples

[1444] Let's say a camera monitoring an area in front of an elementary school captures footage of a group of children crossing the street at a specific time of day. The server receives the footage and uses a machine learning model to detect the presence of children. Based on this information, the server sends a notification to the driver's device or the autonomous vehicle's control system saying, "Children are at the intersection. Please slow down." The driver or autonomous vehicle then checks the notification, immediately reduces speed, and continues driving while making sure the children cross safely. This process significantly reduces the risk of an accident.

[1445] Example prompt for a generative AI model:

[1446] "Suddenly, children start crossing the street. How should the AI ​​detection system react?"

[1447] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1448] Step 1:

[1449] The server receives video data in real time from area surveillance devices and on-board cameras of autonomous vehicles. This video data is transmitted from each camera through a dedicated communication channel and received by the server. The input is the video data from the surveillance cameras, and the output is the storage of this data on the server. The server temporarily stores the received video data in a buffer.

[1450] Step 2:

[1451] The server preprocesses the received video data. Specifically, it removes noise from the video data and normalizes it for each frame. It also resizes the video data as needed. The input is the video data stored on the server, and the output is the preprocessed video data. This allows the machine learning model to analyze the data efficiently.

[1452] Step 3:

[1453] The server sends the preprocessed video data to a machine learning algorithm to detect the presence of a child. This uses a generative AI model from TensorFlow. The input is the preprocessed video data, and the output is a determination of whether a child is present. The server determines the presence of a child for each frame based on the output of the AI ​​model.

[1454] Step 4:

[1455] The server performs a risk assessment based on the detected child's location and surrounding traffic information. The inputs are the judgment results from the AI ​​model and surrounding traffic information (e.g., the speed and distance of nearby vehicles), and the output is the risk assessment result. The server performs the risk assessment by taking into account multiple factors, and if the risk is high, it determines the necessary measures.

[1456] Step 5:

[1457] Based on the results of the risk assessment, the server sends a notification to the driver's communication device or the autonomous vehicle's control system. The input is the result of the risk assessment, and the output is a notification message. For example, a message saying "There is a child ahead. Please slow down." The notification is sent in real time, allowing the driver or vehicle system to respond immediately.

[1458] Step 6:

[1459] The driver's communication terminal or the autonomous vehicle's control system receives the notification from the server. The input is the notification message from the server, and the output is an alert display for the driver or speed adjustment by the vehicle control system. Specifically, a visual warning display or an audio alert is sounded, and the autonomous vehicle automatically adjusts its speed.

[1460] Step 7:

[1461] The terminal collects behavioral data of the driver or autonomous vehicle and sends it to the server. The input is the behavioral data acquired by the terminal, and the output is the data sent to the server. This behavioral data includes information on speed, acceleration, braking operation, etc. The data is periodically sent to the server and used for further analysis.

[1462] Step 8:

[1463] The server analyzes the received behavioral data and provides feedback to support safe driving. The input is behavioral data sent from the driver or the autonomous vehicle, and the output is a feedback message. For example, advice such as "Please do not brake suddenly too often" is provided to the driver.

[1464] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1465] The present invention combines an AI system using a community monitoring device or user terminal to prevent traffic accidents involving children with an emotion engine. This system recognizes the driver's emotional state and promotes safe driving by adjusting risk assessments and notification content. Specific embodiments for implementing the present invention are described in detail below.

[1466] Server Roles

[1467] 1. Video data collection

[1468] The server receives real-time video data from local surveillance devices and user terminals. For example, surveillance cameras installed around a school transmit images of children going to and from school.

[1469] 2. Preprocessing of video data

[1470] The server performs pre-processing on the received video data, which includes noise removal, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[1471] 3. Child presence detection

[1472] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and applies an age estimation model to identify children among them.

[1473] 4. Risk Assessment and Notification Generation

[1474] The server combines data on the child's presence with surrounding traffic information to assess risk and generates a notification if the risk level is high. For example, if the server detects an approaching car while a child is crossing the street, it sends a warning to the driver's device saying, "There is a child ahead. Please slow down."

[1475] 5. Collecting and analyzing operational data

[1476] The server continuously collects and analyzes driver behavior data (speed, acceleration, sudden braking, etc.) and provides feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive an alert saying, "Please try to brake more gently."

[1477] 6. Operation of the Emotion Engine

[1478] The server analyzes the driver's facial expressions and tone of voice and recognizes their emotional state using an emotion engine, an algorithm for performing emotion analysis based on camera footage and audio data.

[1479] 7. Emotion-Based Risk Assessment

[1480] The server performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards are applied to further promote safe driving.

[1481] 8. Long-term emotional data analysis

[1482] The server monitors the driver's emotional state over the long term and analyzes the data to provide optimal feedback to each individual driver, for example, providing stress management advice to drivers who frequently feel stressed.

[1483] Device Role

[1484] 1. Receiving and viewing notifications

[1485] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[1486] 2. Collecting Emotional Data

[1487] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and sends this data to the emotion engine, which uses it to monitor the driver's emotional state in real time while driving.

[1488] 3. Sending driving data

[1489] The device continuously collects driver behavior data and periodically transmits it to a server, including GPS data and data from the vehicle's acceleration sensor.

[1490] User (driver) behavior

[1491] 1. Check notifications

[1492] The user checks the notification from the device, and the system not only displays a visual notification but also an audio alert to attract the driver's attention.

[1493] 2. Safe driving

[1494] The user takes appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[1495] 3. Providing Feedback

[1496] Users have the ability to provide feedback through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded and sent to the server.

[1497] Specific examples

[1498] For example, on a weekday morning, while drivers are commuting to work and children are on their way to school, a community monitoring device may detect the presence of a child crossing a street. After the information is sent to the server, the server immediately confirms the presence of the child and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the server determines that the driver is stressed, it may also provide additional feedback such as, "Please drive calmly." In this way, the system of the present invention can detect the presence of a child in real time, grasp the driver's emotional state, and provide appropriate notifications, thereby preventing traffic accidents involving children and providing a safe driving environment.

[1499] The processing flow will be explained below.

[1500] Server Processing

[1501] Step 1: Receiving video data

[1502] The server receives real-time video data from local surveillance devices and user terminals, such as cameras monitoring children on their way to and from school.

[1503] Step 2: Preprocessing the video data

[1504] The server performs pre-processing on the received video data, which includes noise reduction, resolution adjustment, background removal, etc. The pre-processed data is then passed to the analysis algorithm.

[1505] Step 3: Detect child presence

[1506] The server feeds the pre-processed video data into a machine learning algorithm to detect the presence of children, which then identifies people in the video and applies an age estimation model to identify children among them.

[1507] Step 4: Risk assessment

[1508] The server combines the child's presence data with surrounding traffic information to assess risk. For example, when a child is about to cross the road, it calculates the risk level based on the speed and location of nearby vehicles.

[1509] Step 5: Generate notifications

[1510] If the risk level exceeds a certain threshold, the server generates a notification to be sent to the driver's device, for example, a specific warning message such as "There is a child ahead, please slow down."

[1511] Step 6: Sending notifications

[1512] The server then sends the generated notification to the driver's communication device using a push notification service.

[1513] Step 7: Collect driving data

[1514] The server collects driver behavior data, including speed, acceleration, sudden braking, etc. The data is periodically transmitted from the device.

[1515] Step 8: Data analysis and feedback generation

[1516] The server analyzes the collected driving data and provides feedback to support safe driving, such as generating a message like, "You braked too often. Please brake more gently."

[1517] Step 9: Collect emotion data

[1518] The server receives the driver's facial expressions and voice data and analyzes them with an emotion engine, thereby recognizing the driver's emotional state in real time.

[1519] Step 10: Emotion-Based Risk Assessment

[1520] The server performs a risk assessment based on the perceived emotional state: for example, if the driver is angry, it sets a higher risk level and applies stricter notification criteria.

[1521] Step 11: Analyze Long-Term Sentiment Data

[1522] The server monitors the driver's emotional state over a long period of time, accumulating and analyzing the data, thereby identifying the driver's individual stress patterns and fatigue levels and providing optimal feedback.

[1523] Terminal handling

[1524] Step 1: Receive notifications

[1525] The device receives notifications sent from the server in real time and warns the driver using a combination of visual and audio alerts.

[1526] Step 2: View notifications

[1527] The device analyzes the received notification and displays it to the driver. For example, a notification such as "There is a child ahead. Please slow down" may be displayed on the screen and read aloud.

[1528] Step 3: Collecting emotion data

[1529] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and passes this data to the emotion engine, which uses it to monitor the driver's emotional state in real time.

[1530] Step 4: Collect driving data

[1531] The device collects driving data using GPS and acceleration sensors, which are temporarily stored in a buffer and periodically sent to a server.

[1532] Step 5: Sending data

[1533] The device transmits the collected driving and emotion data to a server using a secure protocol (e.g., HTTPS).

[1534] User (driver) behavior

[1535] Step 1: Check notifications

[1536] The user checks the notification from the device, receives it both visually and audibly, and understands its content.

[1537] Step 2: Practice safe driving

[1538] The user can take appropriate driving actions based on the notification content. For example, if the user receives a notification that there is a child ahead, the user can slow down and check the surrounding area for safety.

[1539] Step 3: Emotional Management

[1540] Users are expected to pay attention to their emotional state and take appropriate driving behaviors, and the system provides feedback to help them manage stress and fatigue.

[1541] Step 4: Provide feedback

[1542] Users provide feedback to the system through the application, such as "the notification was appropriate" or "there was a false alarm," which is recorded within the app and sent to the server.

[1543] By implementing the above processing steps, the system of the present invention can prevent traffic accidents involving children and provide a safe driving environment. By taking the driver's emotional state into consideration, even more effective safe driving support can be achieved.

[1544] Example 2

[1545] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1546] Conventional area monitoring systems and safe driving support systems do not take into account the driver's psychological state, and therefore do not provide appropriate feedback or notifications according to the situation, resulting in limited effectiveness in preventing traffic accidents. Furthermore, there is no system that not only notifies drivers of the presence of children but also comprehensively evaluates the driver's emotional and behavioral data, which creates problems with risk management during parking. This does not promote safe driving behavior among drivers, and the risk of traffic accidents remains.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1548] In this invention, the server includes means for receiving video data acquired from the local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication device when the presence of a child is detected, and means for adjusting the content of the notification based on the driver's emotional state using an emotion engine that analyzes the driver's emotional state, thereby enabling real-time risk assessment and promotion of safe driving that takes the driver's emotional state into consideration.

[1549] "Area monitoring device" refers to a hardware device that is deployed in a specific area to collect video and other data in real time.

[1550] "Video data" refers to video information acquired from a local monitoring device or a user terminal, and is digital data to be analyzed.

[1551] A "machine learning algorithm" is a computational method for analyzing large amounts of data and learning patterns from that data, and is used, for example, for person detection and age estimation.

[1552] The "means for detecting the presence of a child" is a system element for analyzing video data and identifying a child from within the video data.

[1553] The "driver's communication terminal" refers to a communication device such as a mobile phone, smartphone, or in-car device carried by the driver.

[1554] "Means for sending notification" refers to a communication method or device for immediately conveying detected information and risk assessment results to the driver.

[1555] The "emotion engine" is a system component that analyzes the driver's facial expressions and voice data to recognize their emotional state.

[1556] "Driver behavior data" refers to data that indicates the driving behavior of the driver, including speed, acceleration, use of sudden braking, and the like.

[1557] The "means for providing feedback" is a method or device that provides information or advice to the driver to support safe driving based on the analysis results.

[1558] The "integration means" is a system function that centrally analyzes video data acquired from multiple monitoring devices and grasps the overall situation.

[1559] "Means for optimizing road safety measures" are system elements for planning and implementing local measures to promote safe driving based on integrated data and driver emotion data.

[1560] This invention combines an emotion engine with an AI system that aims to prevent traffic accidents involving children using community monitoring devices and user terminals. The system recognizes the driver's emotional state and adjusts risk assessment and notification content to promote safe driving.

[1561] Hardware Configuration

[1562] The main hardware components of the system include area monitoring devices, servers, and user terminals. Area monitoring devices can be CCTV cameras or other video capture devices. The servers use high-performance computers and cloud services to perform data analysis and processing. User terminals are mainly smartphones or communication devices in cars.

[1563] Software Configuration

[1564] The main software used in the system is as follows:

[1565] OpenCV library: Used for preprocessing video data.

[1566] YOLO, SSD: Person detection as a machine learning algorithm.

[1567] Google Cloud Speech-to-Text API: Used to analyze audio data.

[1568] dlib library: Used for facial expression recognition.

[1569] Data processing flow

[1570] The server receives video data from the area surveillance devices in real time and preprocesses the received data to remove noise and adjust the resolution. The preprocessed video data is then fed into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. If a child is detected, the server assesses the risk and, if necessary, sends a notification to the driver's communication device. The notification is provided in audio and visual formats.

[1571] The server also collects and analyzes driver behavior data from GPS and vehicle sensors to provide feedback to support safe driving. For example, a driver who frequently brakes suddenly will receive advice such as "Try to brake more gently."

[1572] Emotion Engine Operation

[1573] The server collects camera footage and audio data from the user's device and uses an emotion engine to recognize the driver's emotional state. Based on this emotional state, the server adjusts risk assessment and notification content. For example, if the driver is angry, the notification standards will be tightened and a warning such as "Please drive calmly" will be sent.

[1574] Specific examples

[1575] For example, on a weekday morning, while a driver is commuting to work and children are on their way to school, a community monitoring device detects the presence of a child crossing a crosswalk. This information is sent to a server, which immediately verifies it and sends a notification to the driver's device saying, "There is a child ahead. Please reduce your speed." If the system determines that the driver is stressed, it may provide additional feedback such as, "Please drive calmly." In this way, by detecting the presence of a child in real time and providing appropriate notifications while understanding the driver's emotional state, the system can prevent traffic accidents and provide a safe driving environment.

[1576] Prompt Sentence Examples

[1577] "Please explain the role of an AI system aimed at preventing traffic accidents involving children. The system incorporates an emotion engine to recognize the driver's emotional state and perform risk assessments. It also adjusts notifications to promote safe driving."

[1578] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1579] Step 1: Collect video data

[1580] The server receives video data from the area surveillance devices in real time. The input is the video data sent from the area surveillance devices, and the output is the raw video data processed within the server. Specifically, it acquires video streams from the surveillance cameras and transfers the data securely using the TCP / IP protocol.

[1581] Step 2: Preprocessing the video data

[1582] The server performs preprocessing on the received video data, such as noise removal, resolution adjustment, and background removal. The input is raw video data, and the output is preprocessed video data. Specifically, it uses the OpenCV library to apply background subtraction to remove noise and maintain a constant video resolution.

[1583] Step 3: Detect child presence

[1584] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is information on whether the presence of a child was detected. As a concrete example, a video frame is input into the YOLO model, the detection results are saved in JSON format, and the coordinate information where the child was identified is extracted.

[1585] Step 4: Risk assessment and notification generation

[1586] The server assesses risk by combining data on the child's presence with information on surrounding traffic. The inputs are data on the child's presence and traffic information, and the output is a notification message. For example, it obtains traffic information from a real-time traffic API and calculates risk by comparing the timing of a child crossing the street with the speed of the car. If the risk level is high, it sends a notification to the driver's device saying, "There is a child ahead. Please slow down."

[1587] Step 5: Collect and analyze driving data

[1588] The device continuously collects driver behavior data and periodically transmits it to the server. The input is GPS data and vehicle sensor data, and the output is driving data sent to the server. Specifically, the device collects vehicle speed and acceleration data via the smartphone's GPS sensor and OBD-II interface.

[1589] Step 6: Emotion Engine in Action

[1590] The device uses a camera and microphone to collect the driver's facial expressions and tone of voice, and sends them to the server. The input is camera footage and audio data, and the output is the recognition result of the driver's emotional state. The server analyzes this data, performs facial expression recognition using the dlib library, and converts the audio data into text using the Google Cloud Speech-to-Text API to perform emotion analysis.

[1591] Step 7: Emotion-Based Risk Assessment

[1592] The server performs risk assessment based on the emotion recognition results. The input is the emotion recognition results, and the output is a risk level and notification content based on the emotion. For example, if it determines that the driver is angry, it applies stricter notification standards than usual and generates a warning message saying, "Please drive calmly."

[1593] Step 8: Analyze Long-Term Sentiment Data

[1594] The server monitors the driver's emotional state over the long term and analyzes the data. The input is past emotional data, and the output is long-term trend analysis and feedback. The emotional data is aggregated monthly and stress management advice is provided based on the analysis results. For example, relaxation suggestions are made to drivers who frequently feel stressed.

[1595] In this way, the system of the present invention is designed to prevent traffic accidents by detecting the presence of children in real time through multiple processing steps and sending appropriate notifications while taking into account the driver's emotional state.

[1596] (Application example 2)

[1597] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1598] The objective of this invention is to provide a system for preventing traffic accidents involving children in local areas and promoting safe driving. In particular, we aim to build a system that provides appropriate feedback on the driver's emotional state and performs real-time risk assessment to increase the driver's attention and support safe driving.

[1599] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving video data acquired from a local monitoring device, means for analyzing the video data using a machine learning algorithm to detect the presence of a child, means for sending a notification to the driver's communication terminal when the presence of a child is detected, means for analyzing image and audio data to recognize the driver's emotional state, and means for performing a risk assessment based on the driver's emotional state and adjusting the content of the notification. This makes it possible to detect the presence of a child in real time and send an appropriate notification according to the driver's emotional state.

[1600] A "regional monitoring device" is a device that is installed in a specific region and is used to collect video data.

[1601] "Video data" refers to image information acquired by an imaging device such as a camera.

[1602] A "machine learning algorithm" is a mathematical model or calculation method for automatically analyzing data and recognizing patterns.

[1603] "Means for detecting the presence of children" refers to a system that analyzes video data to determine whether a child is in the image.

[1604] "Driver" means a person operating and driving a vehicle.

[1605] A "communication terminal" is an electronic device capable of sending and receiving information. Specifically, it refers to a smartphone or similar device.

[1606] The "means for sending notification" is a mechanism for sending specific information to the driver's communication terminal.

[1607] "Emotional state" refers to the driver's mental state and emotional movements.

[1608] "Image and audio data" refers to visual and audio information acquired through a camera or microphone.

[1609] A "means for risk assessment" is a system for determining the degree of danger based on collected data.

[1610] The "means for adjusting notification content" is a mechanism for optimizing the content of notifications sent based on the driver's emotional state and detected risks.

[1611] The present invention provides a system for preventing traffic accidents involving children, using video data from local surveillance devices and the driver's communication terminal. The system has the function of recognizing the driver's emotional state and adjusting risk assessment and notification content based on that state.

[1612] Server Roles

[1613] 1. Video data collection

[1614] The server receives real-time video data from local surveillance devices, such as cameras installed around a school, capturing images of children walking to and from school. The received video data is analyzed according to the processing steps described below.

[1615] 2. Preprocessing of video data

[1616] The server performs preprocessing on the received video data, such as removing noise, adjusting resolution, and cutting out the background, and the preprocessed data is used as input data for machine learning algorithms.

[1617] 3. Child presence detection

[1618] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of children. The algorithm identifies people and then identifies children using an age estimation model.

[1619] 4. Risk Assessment and Notification Generation

[1620] The server combines the child presence data with traffic information to assess the risk, and if the risk level is high, generates and sends a notification to the driver's communication device, such as "There is a child ahead. Please slow down."

[1621] 5. Recognizing the driver's emotional state

[1622] The server analyzes the image and audio data collected from the communication device to recognize the driver's emotional state: facial expression recognition algorithms are applied to the image data, and voice analysis algorithms are applied to the audio data.

[1623] 6. Emotion-Based Risk Assessment

[1624] The server then performs risk assessment based on the emotion recognition results. For example, if the driver is angry, stricter notification standards will be applied.

[1625] 7. Long-term emotional data analysis

[1626] The server monitors the driver's emotional state over the long term and analyzes the data, providing stress management advice to drivers who frequently experience stress.

[1627] Device Role

[1628] 1. Receiving and viewing notifications

[1629] The device receives notifications sent from the server in real time and displays them to the driver, both visually and as audio alerts.

[1630] 2. Collecting Emotional Data

[1631] The device uses a camera and microphone to capture the driver's facial expressions and tone of voice, and transmits the data to a server, which uses it to monitor the driver's emotional state in real time.

[1632] 3. Sending driving data

[1633] The device continuously collects driver behavior data (e.g., speed, acceleration, sudden braking, etc.) and periodically transmits it to the server, which provides feedback for safe driving.

[1634] User (driver) behavior

[1635] 1. Check notifications

[1636] Users check notifications from their devices, and audio alerts as well as visual notifications are used to draw attention.

[1637] 2. Safe driving

[1638] The user is then advised to take appropriate driving actions based on the notification. For example, if the user receives a notification that there is a child ahead, the user should slow down and carefully check their surroundings.

[1639] 3. Providing Feedback

[1640] Users have the ability to provide feedback through the application. For example, feedback such as "the notification was appropriate" or "there was a false alarm" can be recorded and sent to the server for future improvements.

[1641] Specific examples

[1642] For example, during the morning commute, a driver gets into their vehicle and launches the smartphone app. The app automatically receives video data from local surveillance devices and detects the presence of children. The driver's face is captured by a camera and their emotional state is analyzed using a facial recognition algorithm. If the app determines that the driver is stressed, a notification will appear saying, "There is a child ahead. Please drive calmly." Upon seeing this notification, the driver will slow down and take action to check their surroundings.

[1643] Example prompts to input to a generative AI model:

[1644] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[1645] As a result, the present invention provides a safe driving support system that takes into account the emotional state of the driver, thereby contributing to the prevention of traffic accidents involving children.

[1646] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1647] Step 1:

[1648] The server receives video data in real time from the local surveillance equipment. The input is raw video data from the local surveillance equipment, and the output is video data for preprocessing. Specifically, the video of children going to and from school is transmitted from the surveillance cameras installed around the school.

[1649] Step 2:

[1650] The server performs preprocessing on the received video data. The input is raw video data, and the output is preprocessed video data that has undergone noise removal, resolution adjustment, background removal, etc. This prepares the data so that the analysis algorithm can operate accurately.

[1651] Step 3:

[1652] The server inputs the preprocessed video data into a machine learning algorithm (e.g., YOLO, SSD) to detect the presence of a child. The input is the preprocessed video data, and the output is data indicating the child's location. The algorithm identifies people in the video and uses an age estimation model to identify the child.

[1653] Step 4:

[1654] The server combines the presence data of the child and traffic information to perform risk assessment. The input is the presence data of the child and surrounding traffic information, and the output is assessment data indicating the risk level. If the risk level is high, a notification is generated.

[1655] Step 5:

[1656] The server sends a notification based on the risk assessment results to the driver's communication device. The input is risk level assessment data, and the output is notification information such as "There is a child ahead. Please slow down." This allows the driver to recognize danger early.

[1657] Step 6:

[1658] The server recognizes the driver's emotional state using image and voice data acquired from the driver's communication device. The input is the driver's facial image and voice tone data, and the output is the emotion recognition result. Facial expression recognition algorithms and voice analysis algorithms are used.

[1659] Step 7:

[1660] The server re-performs risk assessment based on the emotion recognition results. The inputs are the emotion recognition results and existing risk assessment data, and the output is adjusted risk assessment data based on the emotional state. For example, if the driver is angry, a more severe notification is generated.

[1661] Step 8:

[1662] The terminal receives notifications sent from the server in real time and displays them to the driver. The input is notification data, and the output is visual and audio notification displays. This allows the driver to immediately prepare to take appropriate driving actions.

[1663] Step 9:

[1664] The device continuously collects the driver's emotional data using a camera and microphone and sends it to the server. The input is the driver's facial expression and tone of voice data, and the output is sent to the server as emotional data. This data is used for subsequent emotional analysis.

[1665] Step 10:

[1666] The server analyzes the driver's behavioral data (speed, acceleration, sudden braking, etc.) and provides long-term feedback. The input is behavioral data, and the output is advice such as "Try to brake more gently."

[1667] Step 11:

[1668] The user checks the notification from the device and performs safe driving behavior. The input is the notification information from the device, and the output is the action taken by the driver. For example, upon receiving a notification that "there is a child ahead," the user slows down and checks their surroundings.

[1669] Example prompt sentence:

[1670] Create an application for autonomous vehicles that recognizes the emotional state of the driver and performs risk assessment to prevent child traffic accidents. Add functionality to send a notification to the driver if a child is detected.

[1671] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1672] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1673] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1674] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1675] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which em...

Claims

1. means for receiving video data acquired from the area monitoring device; means for analyzing the video data using a machine learning algorithm to detect the presence of a child; means for transmitting a notification to a driver's communication terminal when the presence of a child is detected; A system including:

2. a means for collecting driver behavior data; means for analyzing the behavioral data and providing feedback to assist safe driving; The system of claim 1 further comprising:

3. A means for integrating video data acquired from multiple monitoring devices and analyzing traffic conditions across an entire region; A means for optimizing traffic safety measures for each region based on the integrated data; The system of claim 1 further comprising:

4. A means for notifying the driver's communication terminal of the self-diagnosis results of driving behavior and recommended behavior is provided. The system of claim 2 further comprising:

5. A risk assessment is performed based on the detected child's location information and surrounding traffic information. The system of claim 1 further comprising means for generating warning information if the risk is high.

Images