Safety management system for sleeping rest areas based on artificial intelligence and method thereof
The AI-based safety management system for rest areas uses multiple sensors to monitor and alert authorities to dangerous situations, addressing the challenges of delayed response and safety in rest areas, ensuring rapid incident resolution and enhanced safety.
Patent Information
- Application Number
- KR1020250126228
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-09-05
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-09-05
AI Technical Summary
Existing rest areas on highways face challenges in actively addressing accidents, crimes, and health emergencies due to their remote locations and lack of rapid response mechanisms, particularly for drowsy drivers, leading to delayed incident processing and increased risk.
An AI-based safety management system using a PTZ AI vision camera, thermal imaging camera, acoustic sensor, and sound wave sensor to monitor rest areas, analyze vehicle and passenger behavior, and immediately alert relevant agencies upon detecting dangerous situations.
Enables rapid detection and response to accidents, crimes, and health emergencies by providing immediate alerts to emergency services, securing evidence, and guiding vehicles to avoid danger, thus enhancing safety and reducing casualties.
Smart Images

Figure 112025102131649-PAT00002_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to an artificial intelligence-based safety management system for rest areas and a method thereof. More specifically, the invention relates to an artificial intelligence-based safety management system for rest areas and a method thereof that monitors vehicles and passengers using a rest area based on an artificial intelligence model for detecting safety in rest areas using data acquired by equipping a high-magnification pan-tilt-zoom (PTZ) artificial intelligence vision camera, a thermal imaging camera, an acoustic sensor, and a sound wave sensor, and immediately transmits an alert to the relevant agency (hospital, fire station, police station, highway patrol) upon detecting a dangerous situation occurring in the rest area so that the rescue of the passenger or the resolution of the dangerous situation is processed quickly. Background Technology
[0002] The functions of roads are broadly classified into mobility and accessibility. Among these, expressways are designed primarily to rapidly transport a large volume of vehicles between regions, and thus generally have high driving speeds. Since high driving speeds act as a significant collision energy for drivers and passengers in the event of a traffic accident, the fatality rate (number of deaths / number of accidents) on expressways is higher than on other roads.
[0003] The causes of traffic accidents are classified into speeding, drowsy driving, failure to pay attention to the road ahead, vehicle defects, and other reasons. Among these, drowsy driving is known to slow down a driver's cognition and reaction, which consequently leads to a decline in driving performance and increases the likelihood of accidents. In particular, on highways, traffic accidents can cause fatal damage due to high driving speeds.
[0004] When a driver falls asleep while driving, their reaction speed to perceive danger slows down, and as a result, they are unable to take any action regarding the danger, leading to an accident; therefore, traffic accidents caused by drowsy driving result in major casualties, including death.
[0005] As a measure to prevent accidents caused by drowsy driving, small rest areas designed to allow drowsy drivers to take a brief break to relieve drowsiness before resuming driving are being widely installed on one side of roads.
[0006] To prevent drowsy driving, it is crucial to provide safe spaces where drivers can rest when they experience driving fatigue or drowsiness. Accordingly, to expand insufficient rest areas on highways and thereby prevent drowsy driving, rest stops have been installed and operated in sections where existing rest areas are spaced far apart.
[0007] Meanwhile, rest areas are spaces where drivers can safely take a short break. In addition to rest areas, they are equipped with facilities such as portable toilets, pergolas, and landscaping, as well as various exercise equipment to help drivers relax their muscles. Furthermore, numerous studies have proven that the installation and operation of rest areas lead to a decrease in the rate of traffic accidents in the same sections. As such, rest areas are highly effective in reducing traffic accidents on highways, particularly those caused by drowsiness.
[0008] However, despite the aforementioned advantages, it is true that it is difficult to actively address the various accidents that occur at rest areas. For instance, accidents frequently occur while vehicles are parked at these rest areas; due to the nature of highways, processing these incidents takes a long time, and in some cases, the vehicle that caused the accident simply flees the scene.
[0009] Furthermore, because rest areas for drowsy drivers have fewer users than regular service areas, they are more frequently exposed to general crimes. Even when exposed to crime, victims typically report it using their mobile phones; consequently, in the absence of witnesses, the roles of perpetrator and victim may be reversed, and the perpetrator may not be apprehended.
[0010] Furthermore, during the peak of summer, there are cases where drivers suffer health problems or attempt suicide while sleeping in their parked cars at rest areas, but due to a lack of rapid response, many incidents and accidents are occurring.
[0011] Therefore, it is necessary to provide services for the safety management of users and vehicles using rest areas.
[0012] Korean registered patent [10-1819420] discloses a system for notifying entry and exit of a drowsy driver rest area.
[0013] Korean registered patent [10-2405942] discloses a safety management system and method for rest areas using intelligent infrastructure.
[0014] Korean published patent [10-2017-0007875] discloses a safety assistant device and its system device utilizing video and abnormal sound source recognition of a CCTV. Prior art literature
[0015] Korean Registered Patent [10-1819420] (Registration Date: 2018. 01. 10) Korean Registered Patent [10-2405942] (Registration Date: 2022. 05. 31) Korean Published Patent [10-2017-0007875] (Publication Date: 2017. 01. 23) The problem to be solved
[0016] Accordingly, the present invention has been devised to solve the problems described above. The objective of the present invention is to provide an AI-based rest area safety management system and method that monitors vehicles and passengers using a rest area based on an AI model for safety detection of rest areas using data acquired by equipping a high-magnification pan-tilt-zoom (PTZ) AI vision camera, a thermal imaging camera, an acoustic sensor, and a sound wave sensor, and immediately transmits an alert to the relevant agency (hospital, fire station, police station, highway patrol) upon detecting a dangerous situation occurring at the rest area, thereby enabling the rescue of the passenger or the resolution of the dangerous situation to be processed quickly.
[0017] The purposes of the embodiments of the present invention are not limited to those mentioned above, and other unmentioned purposes will be clearly understood by those skilled in the art from the description below. means of solving the problem
[0018] An artificial intelligence-based safety management system for a rest area for drowsy drivers according to an embodiment of the present invention for achieving the above-mentioned purpose comprises: at least one pan-tilt-zoom (PTZ) artificial intelligence (AI) vision camera (100) provided to photograph vehicles and users within the rest area for drowsy drivers; at least one acoustic sensor (microphone) (200) provided to collect sound data within the rest area for drowsy drivers; at least one thermal imaging camera (300) provided to photograph vehicles and users within the rest area for drowsy drivers; at least one sound wave sensor (400) provided to detect the movement of the vehicles; and a safety management server (500) for analyzing high-resolution color images acquired through the PTZ AI vision camera, sound data acquired through the acoustic sensor, thermal images acquired through the thermal imaging camera, and movement data of each vehicle acquired through the sound wave sensor using artificial intelligence to estimate dangerous situations within the rest area for drowsy drivers and the internal state of the vehicles, and for transmitting notifications to relevant agencies according to the type of dangerous situation within the rest area for drowsy drivers.
[0019] The above safety management server (500) comprises: a transceiver (301) for transmitting and receiving data with the PTZ artificial intelligence vision camera, the acoustic sensor, the thermal imaging camera, and the sound wave sensor; a recognition unit (302) for extracting a user recognition area and a vehicle recognition area based on a pre-set recognition model (303) from the high-resolution color image and the thermal imaging image; an analysis unit (304) for recognizing the current situation by comprehensively analyzing the user's behavior and the vehicle's movement from the user recognition area and vehicle recognition area recognized from a plurality of images, the sound data, and the detected vehicle movement data; and a storage unit (305) for storing data received from the PTZ artificial intelligence vision camera, the acoustic sensor, the thermal imaging camera, and the sound wave sensor, and data used for safety management of the drowsy driver rest area. The system is characterized by including: a risk situation determination unit (308) for determining a risk situation of the rest area by estimating the risk possibility from the recognized current situation based on a rest area risk estimation model (306) and estimating the risk possibility of the vehicle interior state based on a vehicle interior state estimation model (307); an alert unit (309) for transmitting an alert to a relevant agency according to the type of risk situation when a risk situation occurs at the rest area; and a control unit (310) for controlling the data flow and operation between the transmission / reception unit, the recognition unit, the analysis unit, the storage unit, the risk situation determination unit, and the alert unit.
[0020] The above-mentioned pre-configured recognition model is characterized by including a user recognition model, a vehicle recognition model, and a behavior recognition model, wherein the user recognition model recognizes a person in the high-resolution color image and the thermal image, the vehicle recognition model recognizes a vehicle in the high-resolution color image and the thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle.
[0021] The analysis unit (304) is characterized by tracking the plurality of thermal images and recognizing that a situation resulting from a temperature change occurs when the distribution change of pixel values within the detection area for each object changes by more than a preset value, or when the temperature of each detected object changes by more than a preset temperature.
[0022] The method is characterized by further including an estimation model learning unit (311) for cumulatively learning the above-mentioned drowsy rest area risk estimation model and the above-mentioned vehicle interior state estimation model; and a feedback verification unit (312) for displaying a warning content including a user confirmation request or outputting a warning signal when no movement of a user inside the vehicle is detected for exceeding a preset time, and for confirming the user's feedback in response to the user confirmation request.
[0023] The above artificial intelligence-based drowsiness rest area safety management system is characterized by further including: a display device (700) provided within the drowsiness rest area for displaying warning content; and a speaker (800) provided within the drowsiness rest area for outputting a warning signal.
[0024] In addition, an artificial intelligence-based safety management method for a drowsy driver rest area according to an embodiment of the present invention for achieving the above-mentioned purpose comprises: a model setting step (S610) in which a recognition model related to a user, a vehicle, and user behavior, and a drowsy driver rest area risk estimation model and a vehicle interior state estimation model based on measurement data are set; an image acquisition step (S620) in which images within the drowsy driver rest area are acquired through a PTZ artificial intelligence vision camera and a thermal imaging camera; an object recognition step (S630) in which a user recognition area and a vehicle recognition area are detected based on a preset recognition model; a movement and sound detection step (S640) in which movement inside the vehicle is detected through a sound wave sensor and sound is detected through an acoustic sensor; a situation recognition step (S650) in which the current situation is recognized by comprehensively analyzing the user behavior and the vehicle movement from the user recognition area and vehicle recognition area recognized from a plurality of images, the sound data, and the detected vehicle movement data; and a risk occurrence estimation step (S660) in which the occurrence of risk in the drowsy driver rest area is estimated according to the recognized current situation based on the drowsy driver rest area risk estimation model. It includes a vehicle internal state estimation step (S670) for estimating each vehicle internal state according to the recognized current situation based on the vehicle internal state estimation model; and a notification signal transmission step (S690) for transmitting a notification signal to a relevant agency according to the type of dangerous situation within the drowsy driver rest area.
[0025] The above artificial intelligence-based drowsy driver rest area safety management method is characterized by further comprising: a model learning step of cumulatively learning and synchronizing the drowsy driver rest area risk estimation model and the vehicle interior state estimation model; and a feedback verification step (S680) of displaying a warning content including a user verification request or outputting a warning signal and verifying the user's feedback in response to the user verification request when no movement of a user inside the vehicle is detected for more than a preset time.
[0026] The above-mentioned pre-configured recognition model is characterized by including a user recognition model, a vehicle recognition model, and a behavior recognition model, wherein the user recognition model recognizes a person in the high-resolution color image and the thermal image, the vehicle recognition model recognizes a vehicle in the high-resolution color image and the thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle.
[0027] In the above situation recognition step (S650), the method is characterized by tracking the plurality of thermal images and recognizing that a situation resulting from a temperature change has occurred when the distribution change of pixel values within the detection area for each object changes by more than a preset value, or when the temperature of each detected object changes by more than a preset temperature.
[0028] In addition, according to one embodiment of the present invention, a computer-readable recording medium is provided that stores a program for implementing the artificial intelligence-based drowsy rest area safety management method.
[0029] In addition, according to one embodiment of the present invention, a program stored on a computer-readable recording medium is provided to implement the artificial intelligence-based drowsy rest area safety management method. Effects of the invention
[0030] According to an artificial intelligence-based drowsy driver rest area safety management system and method according to an embodiment of the present invention, by utilizing data acquired by equipping a high-magnification pan-tilt-zoom (PTZ) artificial intelligence vision camera, a thermal imaging camera, an acoustic sensor, and a sound wave sensor, vehicles and passengers using the drowsy driver rest area are monitored based on an artificial intelligence model for detecting safety in the drowsy driver rest area, and upon detecting a dangerous situation occurring in the drowsy driver rest area, an alert is immediately transmitted to the relevant agency (hospital, fire station, police station, highway patrol), thereby enabling the rescue of the passenger or the resolution of the dangerous situation to be processed quickly.
[0031] In addition, according to an artificial intelligence-based drowsy driver rest area safety management system and method according to one embodiment of the present invention, video data and audio data regarding the location of occurrence when a traffic accident or crime occurs within the drowsy driver rest area can be secured and thus used as evidence.
[0032] In addition, according to an artificial intelligence-based rest area safety management system and method according to an embodiment of the present invention, although it is difficult to confirm the movement of users or vehicles within the rest area using only the artificial intelligence vision camera at night or when visibility is poor, there is an effect of enabling rapid detection of the movement of users or vehicles and fires within the rest area by analyzing thermal images acquired through a thermal imaging camera.
[0033] In addition, according to an artificial intelligence-based rest area safety management system and method according to one embodiment of the present invention, when a dangerous situation occurs at a rest area, the vehicle attempting to enter the rest area can be guided to detour, thereby enabling rapid evacuation from the dangerous situation.
[0034] In addition, according to an artificial intelligence-based drowsy driver rest area safety management system and method according to an embodiment of the present invention, by analyzing vehicle movement using a thermal image, high-resolution video, sound (voice), parking period (time), weather (temperature), vehicle temperature, and sound wave sensor based on a vehicle interior state estimation model, the state of passengers (including the driver) inside each vehicle is estimated, and if there is no movement of passengers inside the vehicle for a preset time, a warning sound is generated, and if there is no feedback from the passengers inside the vehicle after a feedback request, it is determined to be a dangerous situation, thereby providing the effect of accurately determining a dangerous situation. Brief explanation of the drawing
[0035] FIG. 1 is a drawing illustrating a general form of a rest area to which the present invention is applied. FIG. 2 is a configuration diagram of an artificial intelligence-based rest area safety management system according to an embodiment of the present invention. Figure 3 is a detailed configuration diagram of the safety management server of Figure 2. FIG. 4 is a drawing for explaining an example of the installation method of the sound wave sensor of FIG. 2 within a rest area. FIG. 5a is a diagram illustrating a high-resolution image acquired by the high-resolution PTZ AI vision camera of FIG. 2 inside a rest area. FIG. 5b is a diagram illustrating a thermal image acquired by the thermal imaging camera of FIG. 2 inside a rest area. FIG. 6 is a flowchart of an embodiment of an artificial intelligence-based drowsy rest area safety management method according to the present invention. Specific details for implementing the invention
[0036] The present invention is capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention.
[0037] When it is stated that one component is "connected" or "joined" to another component, it should be understood that while it may be directly connected or joined to that other component, there may also be other components in between.
[0038] On the other hand, when it is stated that one component is "directly connected" or "directly coupled" to another component, it should be understood that there are no other components in between.
[0039] The terms used in this specification are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, processes, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, processes, operations, components, parts, or combinations thereof.
[0040] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this application.
[0041] The present invention will be described in more detail below with reference to the attached drawings. Prior to this, terms and words used in this specification and claims should not be interpreted as being limited to their ordinary or dictionary meanings. Based on the principle that the inventor can appropriately define the concepts of terms to best describe their invention, they should be interpreted in a meaning and concept consistent with the technical spirit of the present invention. Furthermore, unless otherwise defined, technical and scientific terms used shall have the meaning commonly understood by those skilled in the art to which this invention pertains. Descriptions of known functions and configurations that could unnecessarily obscure the essence of the present invention in the following description and attached drawings are omitted. The drawings presented below are provided as examples to ensure that the spirit of the present invention is sufficiently conveyed to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Additionally, throughout the specification, the same reference numerals indicate the same components. It should be noted that the same components in the drawings are represented by the same reference numerals wherever possible.
[0042] FIG. 1 is a drawing illustrating a general form of a rest area to which the present invention is applied.
[0043] FIG. 1 is a drawing illustrating a sleep rest area entry / exit notification system according to an embodiment of the present invention.
[0044] Referring to FIG. 1, the rest area (10) is provided with a number of parking areas between the entrance (11) and the exit (12), through which a vehicle enters and exits.
[0045] Here, the rest area (10) forms a site that allows a drowsy driver traveling on the road to exit the road, park their vehicle for a while, and rest.
[0046] The artificial intelligence-based drowsy rest area safety management system (20) according to the present invention is provided within the drowsy rest area (10) and monitors the safety of users and vehicles using the drowsy rest area.
[0047] A number of components included in the above safety management system (20) may be distributed and provided in appropriate places within the above drowsy rest area (10) according to purpose and need.
[0048] As shown in FIG. 1, it is possible to install or move the safety management system (20) provided on the side of a zone equipped with multiple parking areas, the safety management system (20-1) provided in the blocked space between the rest area and the highway, the safety management system (20-2) provided next to the entrance road (11), and the safety management system (20-3) provided before going to the exit road (12).
[0049] For example, a safety management system (20) provided on the side of a zone equipped with multiple parking areas may be equipped with a rail section (30) so that the safety management system can inspect and monitor parked vehicles while moving.
[0050] In particular, a display device such as a large screen is provided in the exit section of the exit (12), and if a dangerous situation is detected within the rest area (10), the corresponding content can be displayed on the display device.
[0051] In addition, a display device such as a large screen is provided before the entrance (11) so that when a dangerous situation occurs at the rest area (10), it can be guided to detour by notifying that a dangerous situation has occurred at the rest area.
[0052] FIG. 2 is a configuration diagram of an artificial intelligence-based rest area safety management system according to one embodiment of the present invention.
[0053] As illustrated in FIG. 2, an artificial intelligence-based rest area safety management system according to one embodiment of the present invention includes a high-resolution pan-tilt-zoom (PTZ) artificial intelligence (AI) vision camera (100), an acoustic sensor (microphone) (200), a thermal imaging camera (300), a sound wave sensor (400), a safety management server (500), a display device (700), and a speaker (800).
[0054] The above PTZ artificial intelligence (AI) vision camera (100) is provided with at least one to photograph vehicles and users within the rest area (10).
[0055] In the above PTZ AI vision camera (100), video data is processed in real time, and a YOLOv5 or YOLOv8-based object recognition model is built-in to monitor dangerous situations within the rest area. Meanwhile, face recognition can be processed through algorithms based on FaceNet and Multi-task Cascaded Convolutional Networks (MTCNN).
[0056] The above PTZ artificial intelligence vision camera (100) is a high-magnification camera with a pan-tilt-zoom function, which can move in various directions and enlarge and reduce through the zoom function.
[0057] The above acoustic sensor (microphone) (200) is provided at least one to collect sound data within the rest area (10).
[0058] Since the above PTZ AI camera (100) may have an acoustic sensor (microphone) and may not have one, they have been distinguished separately, but they may also be provided as a single device.
[0059] The above thermal imaging camera (300) is provided with at least one to photograph the vehicle and the user within the rest area.
[0060] The above sound wave sensor (400) is provided with at least one to detect the movement of each vehicle while the vehicle is parked.
[0061] In the above rest area (10), the above PTZ artificial intelligence (AI) vision camera (100), the above acoustic sensor (microphone) (200), the above thermal imaging camera (300), and the above sound wave sensor (400) may be provided and installed in multiple locations.
[0062] The above safety management server (500) analyzes, through artificial intelligence, high-resolution color images acquired through the PTZ AI vision camera (100), sound data acquired through the acoustic sensor (200), thermal images acquired through the thermal imaging camera (300), and movement data of each vehicle acquired through the sound wave sensor (400) to estimate dangerous situations within the rest area and the internal condition of the vehicle, and transmits notifications to relevant agencies according to the type of estimated dangerous situation within the rest area.
[0063] The above display device (700) is provided in at least one location within the drowsy rest area and displays guidance broadcasts and warning content.
[0064] The above speaker (800) is provided in at least one location within the drowsy rest area and outputs a warning signal.
[0065] An artificial intelligence-based rest area safety management system according to one embodiment of the present invention may have the safety management server (500) itself handle all artificial intelligence tasks, or it may additionally include an AI server (600) for performing tasks in accordance with an artificial intelligence processing request from the safety management server (500).
[0066] In addition, an artificial intelligence-based rest area safety management system according to one embodiment of the present invention may further include a highway control center (900) that transmits and receives information regarding the safety management status and dangerous situations of the rest area related to the safety management of the rest area (10).
[0067] In addition, an artificial intelligence-based rest area safety management system according to one embodiment of the present invention may further include an edge computing unit (not shown) for performing additional preprocessing work on an image acquired from the PTZ artificial intelligence vision camera (100) and transmitting it to the safety management server (500).
[0068] That is, the edge computing unit can perform preprocessing before sending the acquired image to the safety management server (500), and can also perform processing such as object detection and setting of a region of interest.
[0069] Meanwhile, the artificial intelligence-based drowsy rest area safety management system according to one embodiment of the present invention may have a storage unit inside the safety management server (500) to store data, and may also include a separate database (not shown) connected to the safety management server (500).
[0070] In the present invention, the term "database" is a concept that includes not only databases in the narrow sense but also databases in a broad sense that include data records based on computer file systems, and it should be understood that even a simple set of operation processing logs can be included in the database referred to in the present invention if it is possible to extract specific data by searching therefrom.
[0071] Figure 3 is a detailed configuration diagram of the safety management server of Figure 2.
[0072] As illustrated in FIG. 3, the safety management server (500) includes a transmitting / receiving unit (301), a recognition unit (302), an analysis unit (304), a storage unit (305), a danger situation determination unit (308), a notification unit (309), and a control unit (310).
[0073] The above-mentioned transmitting and receiving unit (301) transmits and receives data with the above-mentioned PTZ artificial intelligence vision camera (100), the above-mentioned acoustic sensor (200), the above-mentioned thermal imaging camera (300), and the above-mentioned sound wave sensor (400).
[0074] The above PTZ artificial intelligence vision camera (100) can acquire high-resolution color images, the above acoustic sensor (200) can acquire sound data, the above thermal imaging camera (300) can acquire thermal images, and the above sound wave sensor (400) can acquire vehicle movement data for each vehicle.
[0075] The recognition unit (302) extracts a user recognition area and a vehicle recognition area based on a preset recognition model (303) from the high-resolution color image and the thermal image.
[0076] The analysis unit (304) recognizes the current situation by comprehensively analyzing the user's behavior and the vehicle's movement from the user recognition area and vehicle recognition area recognized from a plurality of images, the sound data, and the detected vehicle movement data.
[0077] The storage unit (305) stores data received from the PTZ artificial intelligence vision camera, the acoustic sensor, the thermal imaging camera, and the sound wave sensor, as well as data used for safety management of the drowsy rest area.
[0078] The above-mentioned danger situation determination unit (308) estimates the possibility of danger from the recognized current situation based on the drowsy rest area danger estimation model (306), and estimates the possibility of danger of the vehicle interior state based on the vehicle interior state estimation model (307) to determine the danger situation of the drowsy rest area.
[0079] The above-mentioned drowsiness rest area risk estimation model synchronizes and analyzes the thermal image, high-resolution video, and sound (voice, scream, collision sound) to determine whether a dangerous situation has occurred in the drowsiness rest area space.
[0080] The above vehicle interior state estimation model determines the state of the vehicle interior by analyzing the thermal image, high-resolution video, sound (voice, scream, collision sound), parking period (time), weather (temperature), vehicle temperature, and vehicle movement using the sound wave sensor (400).
[0081] The above notification unit (309) transmits a notification to a relevant agency according to the type of dangerous situation when a dangerous situation occurs at the above rest area.
[0082] The control unit (310) controls the data flow and operation between the transmission / reception unit (301), the recognition unit (302), the analysis unit (304), the storage unit (305), the danger situation determination unit (308), and the notification unit (309).
[0083] The above-mentioned pre-configured recognition model includes a user recognition model, a vehicle recognition model, and a behavior recognition model.
[0084] The user recognition model recognizes a person in the high-resolution color image and the thermal image, the vehicle recognition model recognizes a vehicle in the high-resolution color image and the thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle.
[0085] Meanwhile, the analysis unit (304) tracks the plurality of thermal images and recognizes that a situation resulting from a temperature change occurs when the distribution of pixel values within the detection area for each object changes beyond a preset value, or when the temperature of each detected object changes beyond a preset temperature.
[0086] For example, in the event of a fire occurring in a rest area, if the thermal image obtained through the thermal imaging camera (300) is analyzed, detection can be made more quickly.
[0087] In addition, when visibility is poor due to night or cloudy weather, it is difficult to track the movement of users or vehicles within the rest area using only the PTZ AI vision camera (100). Therefore, to analyze the movement of users and vehicles, high-resolution images obtained through the PTZ AI vision camera (100) and thermal images obtained through the thermal camera (300) are used together.
[0088] Meanwhile, the above safety management server (300) further includes an estimation model learning unit (311) and a feedback verification unit (312).
[0089] The above estimation model learning unit (311) cumulatively learns the above drowsy rest area risk estimation model (306) and the above vehicle interior state estimation model (307).
[0090] Meanwhile, if the feedback verification unit (312) does not detect movement of a user inside the vehicle for more than a preset time, it displays a warning content including a user verification request on the display device (700) or outputs a warning signal through the speaker (800), and verifies the user's feedback in response to the user verification request.
[0091] For example, if no movement by the driver (user) is detected for more than a preset time while the driver has not exited the vehicle that has entered a rest area (i.e., has exited and re-entered), this can be determined as a dangerous situation.
[0092] Then, the safety management system (20) according to the present invention outputs the warning content to the display device (700) or broadcasts a warning signal or warning through the speaker (800).
[0093] A user verification request is made, such as opening the door and getting out or opening the window and waving one's hand along with the vehicle number, and the feedback verification unit (312) can verify whether the user's feedback is properly performed.
[0094] The above safety management server (500) includes a transmission / reception unit (301), a recognition unit (302), an analysis unit (304), a storage unit (305), a risk situation determination unit (308), an alert unit (309), a control unit (310), an estimation model learning unit (311), and a feedback verification unit (312). The transmission / reception unit (301), the recognition unit (302), the analysis unit (304), the storage unit (305), the risk situation determination unit (308), the alert unit (309), the control unit (310), the estimation model learning unit (311), and the feedback verification unit (312) may be program modules, at least some of which communicate with the safety management server (500). These program modules may be included in the safety management server (500) in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known storage devices. Additionally, these program modules may be stored in a remote storage device capable of communicating with the safety management server (500). Meanwhile, these program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc., that perform specific tasks or execute specific abstract data types as described below according to the present invention.
[0095] Here, the communication network can be configured regardless of the mode of communication, such as wired or wireless, and can be composed of various communication networks, such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), or a Wide Area Network (WAN). Preferably, the communication network referred to in the present invention may be the known World Wide Web (WWW).
[0096] Figure 4 is a diagram illustrating an example of the installation method of the sound wave sensor of Figure 2 within a rest area.
[0097] The sound wave sensor (400) for determining the movement of each vehicle can detect the movement (user movement, sound) of each vehicle while moving within the rest area (10).
[0098] In addition, as shown in FIG. 4, a plurality of sound wave sensors (400-1 to 400-n) may be provided in each parking area provided within the rest area (10).
[0099] FIG. 5a is a diagram illustrating a high-resolution image acquired by the PTZ AI vision camera of FIG. 2 inside a rest area, and FIG. 5b is a diagram illustrating a thermal image acquired by the thermal imaging camera of FIG. 2 inside a rest area.
[0100] Safety within the rest area (10) is managed using a PTZ artificial intelligence vision camera (100), an acoustic center (200), a thermal imaging camera (300), and a sound wave sensor (400) included in the safety management system (20).
[0101] At this time, it is desirable for the PTZ artificial intelligence vision camera (100) and the thermal imaging camera (300) to be provided together to simultaneously photograph the same space and process them in synchronization.
[0102] FIG. 5a is an example of a high-resolution image acquired through a PTZ artificial intelligence vision camera (100), and at the same time, FIG. 5b is a thermal image acquired through a thermal imaging camera (300).
[0103] FIG. 6 is a flowchart of an embodiment of an artificial intelligence-based drowsy rest area safety management method according to the present invention.
[0104] First, a perception model related to the user, vehicle, and user behavior, and a drowsy rest area risk estimation model and a vehicle interior state estimation model based on measurement data are established (S610).
[0105] Afterwards, images of the rest area are obtained through the PTZ artificial intelligence vision camera (100) and the thermal imaging camera (300) (S620).
[0106] Afterwards, user recognition area and vehicle recognition area are detected based on a preset recognition model (S630).
[0107] The above-mentioned pre-configured recognition model is characterized by including a user recognition model, a vehicle recognition model, and a behavior recognition model, wherein the user recognition model recognizes a person in the high-resolution color image and the thermal image, the vehicle recognition model recognizes a vehicle in the high-resolution color image and the thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle.
[0108] Afterwards, movement inside the vehicle is detected through the sound wave sensor (400), and sound is detected through the acoustic sensor (200) (S640).
[0109] Afterwards, the current situation is recognized by comprehensively analyzing the user's behavior and the vehicle's movement from the user recognition area and vehicle recognition area recognized from multiple images, the sound data, and the detected vehicle movement data (S650).
[0110] In the above situation recognition step (S650), by tracking the plurality of thermal images, if the change in the distribution of pixel values within the detection area for each object changes by more than a preset value, or if the temperature of each detected object changes by more than a preset temperature, it can be recognized that a situation due to a temperature change has occurred.
[0111] Afterwards, based on the above-mentioned drowsy driver rest area risk estimation model, the occurrence of a risk at the drowsy driver rest area is estimated according to the recognized current situation (S660).
[0112] The above-mentioned drowsiness rest area risk estimation model synchronizes and analyzes the thermal image, high-resolution video, and sound (voice, scream, collision sound) to determine whether a dangerous situation has occurred in the drowsiness rest area space.
[0113] Afterwards, based on the vehicle internal state estimation model, each vehicle internal state is estimated according to the recognized current situation (S670).
[0114] The above vehicle interior state estimation model determines the state of the vehicle interior by analyzing the thermal image, high-resolution video, sound (voice, scream, collision sound), parking period (time), weather (temperature), vehicle temperature, and vehicle movement using the sound wave sensor (400).
[0115] Afterwards, depending on the type of dangerous situation within the drowsy driver rest area, a notification signal is transmitted to the relevant agency (S690).
[0116] In addition, the artificial intelligence-based drowsy rest area safety management method according to the present invention further includes a feedback verification step (S680) which, when no movement of a user inside a vehicle is detected for more than a preset time, displays a warning content including a user verification request or outputs a warning signal, and verifies the user's feedback in response to the user verification request.
[0117] In addition, the artificial intelligence-based drowsiness rest area safety management method according to the present invention further includes a model learning step of cumulatively learning and synchronizing the drowsiness rest area risk estimation model and the vehicle interior state estimation model.
[0118] Although an artificial intelligence-based safety management method for a drowsy driver rest area according to one embodiment of the present invention has been described above, it is obvious that a computer-readable recording medium storing a program for implementing the artificial intelligence-based safety management method for a drowsy driver rest area and a program stored on the computer-readable recording medium for implementing the artificial intelligence-based safety management method for a drowsy driver rest area can also be implemented.
[0119] In other words, those skilled in the art will readily understand that the aforementioned artificial intelligence-based safety management method for rest areas may be provided by being tangibly implemented as a program of commands for implementing it, and thus included in a computer-readable recording medium. That is to say, it may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, etc., either individually or in combination. The program commands recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program commands, such as ROM, RAM, flash memory, and USB memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.
[0120] The present invention is not limited to the embodiments described above, and its scope of application is diverse. Furthermore, it is understood that various modifications are possible without departing from the essence of the invention as claimed in the claims. Explanation of the symbols
[0121] 10: Rest Area 20: Safety Management System 100: PTZ AI Vision Camera 200: Acoustic Sensor (Microphone) 300: Thermal imaging camera 400: Acoustic sensor 500: Safety Management Server 600: AI Server 700: Display device (display, electronic display) 800: Speaker 900: Highway Control Center 301: Transmitter / Receiver 302: Receiver 303: Recognition Model 304: Analysis Unit 305: Storage Section 306: Rest Area Risk Estimation Model 307; Vehicle Interior State Estimation Model 308: Risk Situation Judgment Unit 309: Notification Unit 310: Control Unit 311: Estimation Model Training Unit 312: Feedback Verification Unit S610: Estimation model setup step S620: Image acquisition stage S630: Object recognition stage S640: Motion and sound detection stage S650: Situation Awareness Stage S660: Risk occurrence estimation step S670: Vehicle internal state estimation stage S680: Feedback verification stage S690: Notification signal transmission step
Claims
Claim 1 An artificial intelligence-based safety management system for rest areas for drowsy drivers comprises: at least one pan-tilt-zoom (PTZ) artificial intelligence (AI) vision camera (100) provided to photograph vehicles and users within the rest area for drowsy drivers; at least one acoustic sensor (microphone) (200) provided to collect sound data within the rest area for drowsy drivers; at least one thermal imaging camera (300) provided to photograph the vehicles and users within the rest area for drowsy drivers; at least one acoustic wave sensor (400) provided to detect the movement of the vehicles; and a safety management server (500) for analyzing high-resolution color images acquired through the PTZ AI vision camera, sound data acquired through the acoustic sensor, thermal images acquired through the thermal imaging camera, and movement data of each vehicle acquired through the acoustic wave sensor using artificial intelligence to estimate dangerous situations within the rest area and the internal state of the vehicles, and to transmit notifications to relevant agencies according to the type of estimated dangerous situation within the rest area for drowsy drivers. The safety management server (500) comprises the PTZ AI vision camera, the acoustic sensor, and the A thermal imaging camera and a transceiver (301) for transmitting and receiving data with the sound wave sensor; a recognition unit (302) for extracting a user recognition area and a vehicle recognition area based on a preset recognition model (303) from the high-resolution color image and the thermal imaging image;An analysis unit (304) that recognizes the current situation by comprehensively analyzing the user's behavior and the vehicle's movement from the user recognition area and vehicle recognition area recognized from multiple images, the sound data, and the detected vehicle movement data, and the analysis includes analyzing high-resolution color images acquired through a PTZ AI vision camera (100), sound data acquired through an acoustic sensor (200), thermal images acquired through a thermal imaging camera (300), and movement data of each vehicle acquired through an acoustic wave sensor (400) using artificial intelligence to estimate dangerous situations within the rest area and the internal state of the vehicle; and a storage unit (305) that stores data received from the PTZ AI vision camera, the acoustic sensor, the thermal imaging camera, and the acoustic wave sensor, as well as data used for safety management of the rest area. A risk situation determination unit (308) for determining a dangerous situation in the rest area by estimating the risk of a dangerous situation in the rest area based on a drowsy rest area risk estimation model (306) based on the recognized current situation, and the drowsy rest area risk estimation model analyzes the thermal image, high-resolution video, and sound (voice, scream, collision sound) in synchronization to determine whether a dangerous situation has occurred in the rest area space, and estimates the risk of a dangerous situation in the vehicle interior state based on a vehicle interior state estimation model (307), and the vehicle interior state estimation model determines the state in the vehicle interior by analyzing the thermal image, high-resolution video, sound (voice, scream, collision sound), parking period (time), weather (temperature), vehicle temperature, and vehicle movement using a sound wave sensor (400); and an notification unit (309) for transmitting a notification to a relevant agency according to the type of dangerous situation when a dangerous situation occurs in the rest area.The AI-based rest area safety management system is characterized by including a control unit (310) for controlling data flow and operation between the transmission / reception unit, the recognition unit, the analysis unit, the storage unit, the risk situation determination unit, and the notification unit; wherein the analysis unit (304) tracks the plurality of thermal images and recognizes that a situation due to temperature change has occurred when the distribution change of pixel values within the detection area for each object changes above a preset value, or when the temperature of each detected object changes above a preset temperature, and further includes an estimation model learning unit (311) for cumulatively learning the rest area risk estimation model and the vehicle interior state estimation model; and further includes a feedback verification unit (312) for displaying a warning content including a user verification request or outputting a warning signal when no movement of a user inside the vehicle is detected for exceeding a preset time, and for verifying the user's feedback in response to the user verification request. Claim 2 delete Claim 3 An AI-based drowsy rest area safety management system according to claim 1, wherein the pre-configured recognition model includes a user recognition model, a vehicle recognition model, and a behavior recognition model, wherein the user recognition model recognizes a person in the high-resolution color image and the thermal image, the vehicle recognition model recognizes a vehicle in the high-resolution color image and the thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle. Claim 4 delete Claim 5 delete Claim 6 An artificial intelligence-based rest area safety management system according to claim 1, further comprising: a display device (700) provided within the rest area for displaying warning content; and a speaker (800) provided within the rest area for outputting a warning signal. Claim 7 In an artificial intelligence-based safety management method for rest areas for drowsy drivers, comprising a program executed by a computational processing means including a computer, the method comprises: a model setting step (S610) in which a recognition model related to a user, a vehicle, and user behavior, and a rest area risk estimation model and a vehicle interior state estimation model based on measurement data are set; an image acquisition step (S620) in which images within the rest area for drowsy drivers are acquired through a PTZ artificial intelligence vision camera and a thermal imaging camera; an object recognition step (S630) in which a user recognition area and a vehicle recognition area are detected based on a preset recognition model; a movement and sound detection step (S640) in which movement inside the vehicle is detected through a sound wave sensor and sound is detected through an acoustic sensor; a situation recognition step (S650) in which the current situation is recognized by comprehensively analyzing the user behavior and the vehicle movement from the user recognition area and vehicle recognition area recognized from a plurality of images, the sound data, and the detected vehicle movement data; and, based on the rest area risk estimation model, whether a risk occurs in the rest area for drowsy drivers according to the recognized current situation, wherein the rest area risk estimation model is... A risk occurrence estimation step (S660) for synchronizing and analyzing a thermal image, high-resolution video, and sound (voice, scream, collision sound) to determine whether a dangerous situation has occurred in a rest area; and a vehicle interior state estimation step (S670) for estimating each vehicle interior state according to the recognized current situation based on the vehicle interior state estimation model, wherein the vehicle interior state estimation model determines the vehicle interior state by analyzing the thermal image, high-resolution video, sound (voice, scream, collision sound), parking period (time), weather (temperature), vehicle temperature, and vehicle movement using a sound wave sensor (400).It includes a notification signal transmission step (S690) for transmitting a notification signal to a relevant agency according to the type of dangerous situation within the drowsy driver rest area, and a model learning step for accumulating and synchronizing the drowsy driver rest area risk estimation model and the vehicle interior state estimation model; The AI-based drowsy driver rest area safety management method further includes a feedback verification step (S680) for displaying a warning content including a user verification request or outputting a warning signal when no movement of a user inside the vehicle is detected for exceeding a preset time, and verifying the user's feedback in accordance with the user verification request; wherein the preset recognition model includes a user recognition model, a vehicle recognition model, and a behavior recognition model; wherein the user recognition model recognizes a person in a high-resolution color image and a thermal image, the vehicle recognition model recognizes a vehicle in a high-resolution color image and a thermal image, and the behavior recognition model tracks a plurality of high-resolution color images and a plurality of thermal images to recognize the movement of the user and the movement of the vehicle according to the movement direction, movement speed, and behavior data of the person and the vehicle; and wherein, in the situation recognition step (S650), the AI recognizes that a situation due to a temperature change occurs when the distribution change of pixel values within the detection area for each object changes by more than a preset value, or when the temperature of each detected object changes by more than a preset temperature. Claim 8 delete Claim 9 delete Claim 10 delete
Citation Information
Patent Citations
.
KR101848479B1
Smart in-cabin monitoring system
KR1020240135160A
Mechanical parking lot management system and method using camera and sensors
KR1020250126297A
System and method for drowsiness shelter safety management using intelligent infrastructure
KR102405942B1
