Method and device for detecting accident risk zones based on heat maps

Abstract

A heatmap-based accident risk zone detection method performed by a processor is disclosed. The heatmap-based accident risk zone detection method includes the steps of generating a heatmap using a video image or a point cloud, and analyzing the heatmap to predict a risk zone.

Description

Technology Field

[0001] The present invention relates to a heatmap-based accident risk zone detection method and apparatus, and more specifically, to a heatmap-based accident risk zone detection method and apparatus for predicting accidents by reflecting various information and responding in advance. Background Technology

[0002] Conventional accident risk zone detection methods utilized cameras, requiring users to manually pre-set risk zones in a polygonal shape within the field of view. Such conventional technology struggled to respond in real-time to changes in traffic patterns or new risk factors, and was limited to simple congestion assessment, object identification, or traffic light-linked monitoring. Furthermore, conventional technology merely counted the number of times pedestrians passed within the pre-set risk zone. Prior art literature

[0003] Korean Published Patent Application No. 10-2024-0065482 (May 14, 2024) The problem to be solved

[0004] The technical problem that the present invention aims to solve is to provide a heatmap-based accident risk zone detection method and apparatus to overcome conventional user-dependent zone setting and simple traffic volume-based analysis. means of solving the problem

[0005] A heatmap-based accident risk zone detection method performed by a processor according to an embodiment of the present invention includes the steps of generating a heatmap using a video image or a point cloud, and predicting a risk zone by analyzing the heatmap.

[0006] The step of analyzing the heatmap to predict a risk zone includes finding grids in the heatmap divided into multiple grids where the grid value is greater than or equal to a first threshold, clustering the found grids to create a zone, and setting the created zone as the risk zone.

[0007] The step of generating a heatmap using the above-mentioned video image or point cloud includes the step of generating a first heatmap based on collision risk, the step of generating a second heatmap based on sudden braking patterns, and the step of generating a third heatmap based on dwell time.

[0008] The step of generating a heatmap using the above-mentioned image or point cloud includes normalizing the range of the first heatmap, the range of the second heatmap, and the range of the third heatmap, applying different weights to the first heatmap, the second heatmap, and the third heatmap respectively according to time period, and generating the heatmap as a weighted sum.

[0009] The step of generating a first heatmap based on the collision risk comprises: a step of calculating a Time To Collision (TTC) based on the position information and velocity information of a first object for each grid corresponding to a first object in a scene divided into multiple grids; a step of updating the collision risk level by accumulating the risk weight of the grid corresponding to the first object if it is determined that the collision risk level is less than or equal to a second threshold value; and a step of generating the first heatmap according to the updated collision risk level.

[0010] The step of generating a second heatmap based on the above sudden braking pattern includes: a step of determining whether the second object is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object in a scene divided into multiple grids; a step of calculating the acceleration of the second object if it is determined that the second object is in a decelerating state; and a step of generating the second heatmap according to the counted number of brakes if it is determined that the absolute value of the acceleration of the second object is greater than or equal to a third threshold value.

[0011] The step of generating a third heatmap based on the dwell time includes: a step of calculating and summing the dwell time of each of the plurality of objects based on the location information of each of the plurality of objects for each of the plurality of grids in a scene divided into a plurality of grids; a step of counting the number of the plurality of objects; a step of calculating the average dwell time of the plurality of objects by dividing the summed dwell time by the number of the plurality of objects; and a step of generating the third heatmap according to the average dwell time of the plurality of objects.

[0012] A computing device according to an embodiment of the present invention includes a processor that executes heatmap-based accident risk zone detection commands, and a memory that stores said commands.

[0013] The above commands are implemented to generate a heatmap using video images or point clouds, and to predict risk zones by analyzing the heatmap.

[0014] Commands for analyzing the heatmap to predict a risk zone are implemented to find grids in the heatmap divided into multiple grids where the grid value is greater than or equal to a first threshold, cluster the found grids to create a zone, and set the created zone as the risk zone.

[0015] Commands for generating a heatmap using the above video image or point cloud are implemented to generate a first heatmap based on collision risk, generate a second heatmap based on sudden braking patterns, and generate a third heatmap based on dwell time.

[0016] Commands for generating a heatmap using the above video image or point cloud are implemented to normalize the range of the first heatmap, the range of the second heatmap, and the range of the third heatmap, apply different weights to the first heatmap, the second heatmap, and the third heatmap respectively according to time period, and generate the heatmap as a weighted sum.

[0017] The commands for generating a first heatmap based on the collision risk are implemented to calculate the Time To Collision (TTC) based on the position and velocity information of the first object for each grid corresponding to the first object in a scene divided into multiple grids, and when the Time To Collision is determined to be less than or equal to a second threshold, to update the collision risk level by accumulating the risk weight of the grid corresponding to the first object, and to generate the first heatmap according to the updated collision risk level.

[0018] Commands for generating a second heatmap based on the above sudden braking pattern are implemented such that, in a scene divided into multiple grids, for each grid corresponding to the second object, the second object is determined to be in a decelerating state based on the speed information of the second object; if it is determined that the second object is in a decelerating state, the acceleration of the second object is calculated; and when it is determined that the absolute value of the acceleration of the second object is greater than or equal to a third threshold, the number of times the second object brakes is counted and the second heatmap is generated according to the counted number of times the brakes are turned.

[0019] The commands for generating a third heatmap based on the dwell time are implemented to calculate and sum the dwell time of each of the plurality of objects based on the location information of each of the plurality of objects for each of the plurality of grids in a scene divided into a plurality of grids, count the number of the plurality of objects, calculate the average dwell time of the plurality of objects by dividing the summed dwell time by the number of the plurality of objects, and generate the third heatmap according to the average dwell time of the plurality of objects. Effects of the invention

[0020] The heatmap-based accident risk zone detection method and apparatus according to an embodiment of the present invention applies video images or point clouds to a neural network to detect objects in a scene, and generates a heatmap using various information of the detected objects, thereby overcoming user-dependent zone setting and simple traffic volume-based analysis and having the effect of automatically detecting data-based intelligent risk zones. Brief explanation of the drawing

[0021] Detailed descriptions of each drawing are provided to help to more fully understand the drawings cited in the detailed description of the present invention. FIG. 1 shows a block diagram of a heatmap-based accident risk zone detection system according to an embodiment of the present invention. FIG. 2 shows a diagram for explaining a heatmap generation operation according to an embodiment of the present invention. Figure 3 shows a heatmap representing a risk zone that changes over time according to an embodiment of the present invention. FIG. 4 shows a flowchart illustrating a heatmap-based accident risk zone detection method according to an embodiment of the present invention. FIG. 5 shows a flowchart for explaining a first heatmap generation operation according to an embodiment of the present invention. FIG. 6 shows a flowchart for explaining a first heatmap generation operation according to an embodiment of the present invention. FIG. 7 shows a flowchart for explaining the first heatmap generation operation according to an embodiment of the present invention. Specific details for implementing the invention

[0022] Specific structural or functional descriptions regarding embodiments according to the concept of the present invention disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described herein.

[0023] Embodiments according to the concept of the present invention may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutions that fall within the spirit and scope of the present invention.

[0024] Terms such as "first" or "second" may be used to describe various components, but said components should not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0025] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, such as "between" and "exactly between," or "adjacent to" and "directly adjacent to," should be interpreted in the same way.

[0026] The terms used herein are used merely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to indicate the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0028] Hereinafter, the present invention will be described in detail by explaining preferred embodiments of the present invention with reference to the attached drawings.

[0029] FIG. 1 shows a block diagram of a heatmap-based accident risk zone detection system according to an embodiment of the present invention.

[0030] Referring to FIG. 1, the heatmap-based accident risk zone detection system (100) refers to a system capable of monitoring accident risk zones based on a heatmap. The heatmap-based accident risk zone detection system (100) includes a computing device (10) and a camera (20). According to an embodiment, the heatmap-based accident risk zone detection system (100) may include a computing device (10) and a LiDAR sensor (not shown). According to another embodiment, the heatmap-based accident risk zone detection system (100) may include a computing device (10), a camera (20), and a LiDAR sensor (not shown). The computing device (10), the camera (20), or the LiDAR sensor can communicate with each other via a network.

[0031] The computing device (10) includes a processor (11) that executes heatmap-based accident risk zone detection commands and a memory (13) that stores said commands. The computing device (10) refers to an electronic device such as a smartphone, tablet PC, laptop, or desktop.

[0032] A camera (20) or a lidar sensor can capture or scan a scene (30). The camera (20) can capture the scene (30) to generate a video image. The lidar sensor can scan the scene (30) to generate a point cloud. The video image generated by the camera (20) or the point cloud generated by the lidar sensor is transmitted to a computing device (10) via a network.

[0033] The computing device (10) can detect objects (e.g., pedestrians, vehicles, etc.) in a scene (30) by applying a video image or a point cloud to a neural network. At this time, the neural network may be a convolutional neural network (CNN) based neural network algorithm.

[0034] The processor (11) generates a heatmap using a video image or a point cloud. Specifically, the processor (11) detects objects in the scene (30) by applying the video image or the point cloud to a neural network. The processor (11) generates the heatmap using various information about the detected objects (e.g., risk of collision with a vehicle, sudden braking patterns of the vehicle, or dwell time of pedestrians, etc.). The heatmap generation operation will be described in detail later.

[0035] The processor (11) analyzes the heatmap to predict the danger zone.

[0036] FIG. 2 shows a diagram for explaining a heatmap generation operation according to an embodiment of the present invention.

[0037] Referring to FIGS. 1 and FIGS. 2, the processor (11) generates a first heatmap (40) based on collision risk.

[0038] The processor (11) divides the scene (30) into multiple grids. The scene (30) refers to an image or point cloud generated from the Field Of View (FOV) of the camera (20) or the LiDAR sensor.

[0039] The processor (11) calculates the Time To Collision (TTC) based on the position information and speed information of the first object for each grid corresponding to the first object (pedestrian, or vehicle) in a scene (30) divided into multiple grids. The Time To Collision may refer to the estimated time to collision between vehicles, or the estimated time to collision between a vehicle and a pedestrian. The position information or speed information of the first object may be predicted using a neural network. The grids corresponding to the first object refer to the grids corresponding to the position of the first object.

[0040] The processor (11) determines whether the estimated collision time is less than or equal to a second threshold (e.g., 3 seconds).

[0041] When it is determined that the above collision prediction time is less than or equal to a second threshold (e.g., 3 seconds), the processor (11) increases the number of collision risk occurrences (Grid[x,y].collision_risk_count) of the grid corresponding to the first object.

[0042] The processor (11) can set the second threshold value differently depending on the time period.

[0043] For example, the processor (11) may set the second threshold to 2.5 seconds during the morning commute (e.g., between 7:00 AM and 9:00 AM) or the evening commute (between 5:00 PM and 7:00 PM), set the second threshold to 3.0 seconds during the afternoon (between 9:00 AM and 5:00 PM), and set it to 3.5 seconds during the evening (between 7:00 PM and 10:00 PM) or the late-night hours (between 10:00 PM and 6:00 AM). By setting the second threshold differently for each time period, the effect of minimizing false positives is achieved.

[0044] The processor (11) generates a first heatmap (40) based on the number of collision risks (Grid[x,y].collision_risk_count) of the grid cells. The more the number of collision risks (Grid[x,y].collision_risk_count) of the grid cells, the more it is represented in red in the first heatmap (40), and the less the number of collision risks (Grid[x,y].collision_risk_count) of the grid cells, the more it is represented in blue in the first heatmap (40). That is, the more the number of collision risks (Grid[x,y].collision_risk_count) of the grid cells, the higher the grid value in the first heatmap (40), and the less the number of collision risks (Grid[x,y].collision_risk_count) of the grid cells, the lower the grid value in the first heatmap (40).

[0045] According to an embodiment, when it is determined that the predicted collision time is less than or equal to a second threshold (e.g., 3 seconds), the processor (11) can accumulate the risk weights of the grid corresponding to the first object to update the collision risk level (Grid[x,y].collision_severity) and generate a first heatmap (40) according to the updated collision risk level (Grid[x,y].collision_severity). The risk weights refer to the grid values. The collision risk level refers to the sum of the risk weights.

[0046] The higher the updated collision risk (Grid[x,y].collision_severity), the more it is represented in red on the first heatmap (40), and the lower the updated collision risk (Grid[x,y].collision_severity), the more it is represented in blue on the first heatmap (40).

[0047] The first heatmap (40) can be implemented as follows. The above commands are stored in memory (13).

[0048] if TTC(pedestrian, vehicle) < second threshold;

[0049] Grid[x,y].collision_risk_count += 1

[0050] Grid[x,y].collision_severity += risk_weight

[0051] The above TTC (pedestrian, vehicle) refers to the time remaining until a collision between a pedestrian and a vehicle. When the above TTC is below a second threshold, the processor (11) determines that there is a risk of collision in the corresponding grid. Specifically, when the above TTC is below the second threshold, the processor (11) may increase the number of collision risk occurrences (Grid[x,y].collision_risk_count) of the grid cell corresponding to the first object, and update the collision risk level (Grid[x,y].collision_severity) by accumulating the risk weight (risk_weight) calculated according to the severity of the situation. The above Grid[x, y] may refer to a grid corresponding to a pedestrian or a vehicle.

[0052] The processor (11) performs a determination on each frame whether the predicted collision time is less than or equal to a second threshold (e.g., 3 seconds). At this time, a frame may refer to a video image or a point cloud. The table below shows examples of the number of collision risks and risk weights in a specific grid per frame.

[0053] Time (frame) TTC Collision risk Number of collision risks Risk weighting t1 2s is 1 +3 t2 4s doesn't exist 1 +3 t3 1.5s is 2 +5 t4 08.s is 3 +5 Cumulative sum 3 16

[0054] The processor (11) can generate a first heatmap (40) based on the cumulative sum of the number of collision risk occurrences (3 times) or the risk weight (+16).

[0055] The risk weight is the collision estimated time weight (W TTC ), velocity weight of the first object (W obj_v ), and the type weight (W) of the first object type It can be calculated as the sum of ).

[0056] Collision Estimation Time Weight (W TTC ) is set larger as the estimated collision time becomes shorter. Since the probability of a collision is higher the shorter the estimated collision time, the estimated collision time weight (W TTC ) is set larger.

[0057] The velocity weight of the first object (W obj_v ) is set larger as the speed of the first object (e.g., a vehicle) increases. Since faster vehicle speeds are more dangerous, the speed weight (W) of the first object obj_v ) is set larger.

[0058] Type weight of the first object (W type ) is set high when the type of the first object is a truck or a bus. Trucks or buses are larger compared to passenger cars, so they are more dangerous. Therefore, when the type of the first object is a truck or a bus, the type weight of the first object (W type ) is set larger.

[0059] For example, when the estimated collision time is within 1 second, the estimated collision time weight (W TTC ) is set to +2, and when the estimated collision time is greater than 1s, the estimated collision time weight (W TTC ) can be set to +1. When the speed of the first object (e.g., a vehicle) is 50 km / h or higher, the speed weight (W) of the first object obj_v ) is set to +2, and when the speed of the first object (e.g., a vehicle) is less than 50 km / h, the speed weight (W) of the first object obj_v ) can be set to +1. When the type of the first object is a truck or a bus, the type weight (W of the first object) type ) is set to +2, and when the type of the first object is a light car or a motorcycle, the type weight (W of the first object) type ) can be set to +1.

[0060] The processor (11) generates a second heatmap (50) based on a sudden stop pattern.

[0061] The processor (11) determines whether the second object (e.g., a vehicle) is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object in a scene (30) divided into multiple grids. The state in which the second object is decelerating can be determined by calculating the speed difference of the second object at different times and determining when the speed difference value is negative. The second object may be the same object as the first object or a different object.

[0062] When it is determined that the second object is in a decelerating state, the processor (11) calculates the acceleration of the second object. The processor (11) determines that the absolute value of the acceleration of the second object is a third threshold value (e.g., 3 m / s²). 2 Determines whether )

[0063] The absolute value of the acceleration of the second object is the third threshold (e.g., 3 m / s² 2 When it is determined that the absolute value of the acceleration of the second object is greater than ), the processor (11) counts the number of brakes of the second object and generates a second heatmap (50) according to the counted number of brakes. 2 When it is determined that the vehicle has applied the brakes, it is determined that the vehicle has applied the brakes.

[0064] The processor (11) generates a second heatmap (50) based on the number of counted breaks. The more the number of counted breaks, the more it is represented in red in the second heatmap (50), and the less the number of counted breaks, the more it is represented in blue in the second heatmap (50).

[0065] According to an embodiment, the processor (11) determines that the absolute value of the acceleration of the second object is a third threshold value (e.g., 3 m / s 2When it is determined that the absolute value of the acceleration of the second object is greater than ), the processor (11) can generate a second heatmap (50) based on the absolute value of the acceleration of the second object. The processor (11) can set the grid value in the second heatmap (50) to have a larger value as the absolute value of the acceleration of the second object increases. That is, the greater the absolute value of the acceleration of the second object, the more red it can be displayed in the second heatmap (50). For example, if the absolute value of the acceleration of the second object is 5 m / s 2 When that happens, the corresponding grid in the second heatmap (50) has an absolute value of acceleration of the second object of 4 m / s 2 When it has a darker red color than the corresponding grid in the second heatmap (50).

[0066] The second heatmap (50) can be implemented as follows. The above commands are stored in memory (13).

[0067] if abs(acceleration of the second object) > third threshold;

[0068] Grid[x,y].emergency_brake_count += 1

[0069] Grid[x,y].brake_intensity += abs(acceleration of the second object)

[0070] The above Grid[x,y].emergency_brake_count represents the number of brakes in the grid corresponding to the second object, and the above Grid[x,y].brake_intensity represents the value corresponding to the acceleration of the second object in the grid corresponding to the second object.

[0071] The processor (11) generates a third heatmap (60) based on the dwell time.

[0072] The processor (11) calculates and sums the dwell time of each of the plurality of objects (e.g., pedestrians) based on the location information of each of the plurality of objects for each of the plurality of grids in a scene (30) divided into a plurality of grids. The dwell time refers to the time difference between when a person appears in the scene (30) and when a person disappears from the scene (30).

[0073] The processor (11) counts the number of the plurality of objects.

[0074] The processor (11) calculates the average stay time of the plurality of objects by dividing the summed stay time by the number of the plurality of objects.

[0075] The processor (11) generates a third heatmap (60) based on the average dwell time of the objects. The longer the average dwell time, the more it is represented in red in the third heatmap (60), and the shorter the average dwell time, the more it is represented in blue in the third heatmap (60). Colors are displayed in the third heatmap (60) according to the current grid where the plurality of objects are located. That is, the longer the average dwell time, the higher the grid value in the third heatmap (60), and the shorter the average dwell time, the lower the grid value in the third heatmap (60).

[0076] The third heatmap (60) can be implemented as follows. The above commands are stored in memory (13).

[0077] for each tracked_object:

[0078] residence_time = tracking_end_time - tracking_start_time

[0079] Grid[x,y].dwell_time_sum += residence_time

[0080] Grid[x,y].dwell_count += 1

[0081] Grid[x,y].dwell_time_sum += dwell_time_sum / dwell_count

[0082] The residence_time above represents the average dwell time of an object, the tracking_end_time above represents the time when an object (e.g., a person) disappears from the scene (30), the tracking_start_time above represents the time when a person appears in the scene (30), the Grid[x,y].dwell_time_sum above represents the sum of the dwell times of each of the multiple objects, the Grid[x,y].dwell_time_count above represents the number of multiple objects, and the Grid[x,y].dwell_time_sum above represents the average dwell time of the multiple objects.

[0083] The processor (11) generates a heatmap (70) using the above image or point cloud. The processor (11) generates a heatmap (70) by fusing the first heatmap (40), the second heatmap (50), and the third heatmap (60). The heatmap (70) may be referred to as the final heatmap (70).

[0084] The present invention can automatically detect data-based intelligent risk zones by generating a heatmap (70) using various information of the detected object (e.g., a first heatmap (40) based on collision risk, a second heatmap (50) based on sudden braking patterns, and a third heatmap (60) based on dwell time), thereby overcoming user-dependent zone setting and simple traffic volume-based analysis.

[0085] The range of grid values ​​of the first heatmap (40), the range of grid values ​​of the second heatmap (50), or the range of grid values ​​of the third heatmap (60) may differ from each other. For example, the range of grid values ​​of the first heatmap (40) may be between 0 and 50, the range of grid values ​​of the second heatmap (50) may be between 0 and 100, and the range of grid values ​​of the third heatmap (60) may be between 0 and 200.

[0086] The processor (11) normalizes the range of grid values ​​of the first heatmap (40), the range of grid values ​​of the second heatmap (50), and the range of grid values ​​of the third heatmap (60) to all have values ​​between 0 and 1.

[0087] The processor (11) applies different weights to the first heatmap (40), the second heatmap (50), and the third heatmap (60) respectively according to the time period, and generates a heatmap (70) as a weighted sum.

[0088] For example, the processor (11) can divide the time zones into a first time zone meaning commuting time (between 7:00 AM and 9:00 AM, or between 5:00 PM and 7:00 PM), a second time zone meaning afternoon time (between 9:00 AM and 5:00 PM), a third time zone meaning evening time (between 7:00 PM and 10:00 PM), and a fourth time zone meaning late night time (between 10:00 PM and 6:00 AM).

[0089] In the first time period, which refers to commuting time (between 7:00 AM and 9:00 AM, or between 5:00 PM and 7:00 PM), the weight of the first heatmap (40) may be 0.5, the weight of the second heatmap (50) may be 0.8, and the weight of the third heatmap (60) may be 0.2.

[0090] In the second time zone, which means afternoon time (between 9 AM and 5 PM), the weight of the first heatmap (40) may be 0.2, the weight of the second heatmap (50) may be 0.2, and the weight of the third heatmap (60) may be 0.6.

[0091] In the third time zone, which means evening time (between 7 PM and 10 PM), the weight of the first heatmap (40) may be 0.3, the weight of the second heatmap (50) may be 0.3, and the weight of the third heatmap (60) may be 0.4.

[0092] In the fourth time zone, which means late-night hours (between 10 PM and 6 AM), the weight of the first heatmap (40) may be 0.5, the weight of the second heatmap (50) may be 0.4, and the weight of the third heatmap (60) may be 0.1.

[0093] The processor (11) finds grids in a heatmap (70) divided into multiple grids where the grid value is greater than or equal to a first threshold (e.g., 0.7). In the heatmap (70), the grid value has a range of values ​​between 0 and 1.

[0094] The processor (11) clusters the found grids to create zones in the heatmap (70). In FIG. 2, the heatmap with created zones is illustrated by the reference numeral '80'. In FIG. 2, the reference numerals 'AGDZ #1, AGDZ #2, AGDZ #3, AGDZ #4, or AGDZ #5' represent zones.

[0095] The processor (11) sets the generated zone as a danger zone. The processor (11) notifies a user (not shown) about the set danger zone through a display (not shown). The user can monitor the danger zone through the display.

[0096] In a heatmap (70) divided into multiple grids, when a grid value has an outlier value and the outlier value is greater than or equal to a first threshold value (e.g., 0.7), a problem may occur in which the outlier value is reflected in the zone. The above outlier value refers to a value generated by an error. Therefore, to solve this problem, instead of comparing each grid value with the first threshold value, the processor (11) sets multiple windows of different sizes (e.g., 3 x 3, 5 x 5, 11 x 11). Each of the multiple windows includes multiple grids. For example, a 3 x 3 window includes three horizontal grids and three vertical grids. The processor (11) averages the grid values ​​included in each window and compares the average grid value with the first threshold value.

[0097] The processor (11) recognizes the overlapping area of ​​windows having an average grid value greater than the first threshold and sets the overlapping area as a danger zone. By recognizing the overlapping area using multiple windows, the danger zone can be set more precisely and accurately.

[0098] Figure 3 shows a heatmap representing a risk zone that changes over time according to an embodiment of the present invention.

[0099] Referring to FIGS. 1 to 3, the danger zone in the heatmap (80) may change over time. For example, even at the same time (10:00 AM), the danger zones on Monday and Tuesday are different. In FIG. 3, the hatching indicates the danger zone.

[0100] The processor (11) identifies dangerous sections in the heatmap (80) on an hourly basis and recognizes the dangerous section pattern. The processor (11) can notify the user (not shown) in advance of the recognized dangerous sections through a display (not shown). Therefore, the present invention has the effect of preventing accidents in advance.

[0101] FIG. 4 shows a flowchart illustrating a heatmap-based accident risk zone detection method according to an embodiment of the present invention.

[0102] Referring to FIGS. 1 to 4, the processor (11) generates a heatmap (80) using a video image or a point cloud (S100).

[0103] The processor (11) generates a first heatmap (40) based on collision risk (S110).

[0104] The processor (11) generates a second heatmap (50) based on a sudden stop pattern (S120).

[0105] The processor (11) generates a third heatmap (60) based on the dwell time (S130).

[0106] The processor (11) applies different weights to the first heatmap (40), the second heatmap (50), and the third heatmap (60) respectively according to the time period, and generates a heatmap (70) as a weighted sum (S140).

[0107] The processor (11) analyzes the heatmap (80) to predict the danger zones (AGDZ#1~AGDZ#5) (S200).

[0108] FIG. 5 shows a flowchart for explaining a first heatmap generation operation according to an embodiment of the present invention.

[0109] Referring to FIGS. 1 to 5, the processor (11) calculates the expected collision time based on the position information and speed information of the first object for each grid corresponding to the first object (e.g., vehicle) in a scene (30) divided into a plurality of grids (S111). This refers to the expected collision time between the first object and another object (e.g., vehicle, or pedestrian).

[0110] The processor (11) determines whether the predicted collision time is less than or equal to the second threshold (S113).

[0111] When it is determined that the above collision prediction time is less than or equal to the second threshold, the processor (11) updates the collision risk level by accumulating the risk weight of the grid corresponding to the first object (S115).

[0112] When it is determined that the above collision prediction time is greater than the second threshold, the processor (11) calculates the collision prediction time based on the position information and velocity information of the first object for each grid corresponding to another object. Alternatively, the processor (11) calculates the collision prediction time based on the position information and velocity information of the first object for each grid corresponding to the first object at a different time in a scene (30) divided into multiple grids.

[0113] The processor (11) generates a first heatmap (40) according to the updated collision risk level (S117).

[0114] FIG. 6 shows a flowchart for explaining a first heatmap generation operation according to an embodiment of the present invention.

[0115] Referring to FIGS. 1 to 6, the processor (11) determines whether the second object is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object in a scene (30) divided into a plurality of grids (S121). The second object may be a different object from the first object.

[0116] When it is determined that the second object is in a decelerating state, the processor (11) calculates the acceleration of the second object (S123).

[0117] When it is determined that the second object is not in a decelerating state, the processor (11) determines whether the other object is in a decelerating state based on the speed information of the other object for each grid corresponding to the other object. Alternatively, at another time, it determines whether the second object is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object.

[0118] The processor (11) determines whether the absolute value of the acceleration of the second object is greater than or equal to the third threshold value (S125).

[0119] When it is determined that the absolute value of the acceleration of the second object is greater than or equal to the third threshold, the processor (11) counts the number of brakes of the second object and generates a second heatmap (50) according to the counted number of brakes (S127).

[0120] When it is determined that the absolute value of the acceleration of the second object is less than the third threshold, the processor (11) determines whether the other object is in a decelerating state based on the velocity information of the other object for each grid corresponding to the other object. Alternatively, at another time, it determines whether the second object is in a decelerating state based on the velocity information of the second object for each grid corresponding to the second object.

[0121] FIG. 7 shows a flowchart for explaining the first heatmap generation operation according to an embodiment of the present invention.

[0122] Referring to FIGS. 1 to 7, the processor (11) calculates and sums the dwell time of each of the plurality of objects based on the location information of each of the plurality of objects for each of the plurality of grids in a scene (30) divided into a plurality of grids (S131).

[0123] The processor (11) counts the number of the plurality of objects (S133).

[0124] The processor (11) calculates the average stay time of the plurality of objects by dividing the summed stay time by the number of the plurality of objects (S135).

[0125] The processor (11) generates the third heatmap (60) based on the average dwell time of the plurality of objects (S137).

[0126] The present invention has been described with reference to an exemplary embodiment illustrated in the drawings, but this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the present invention should be determined by the technical spirit of the appended claims. Explanation of the symbols

[0127] 100: System; 10: Computing device; 11: Processor; 13: Memory; 20: Camera;

Claims

Claim 1 A heatmap-based accident risk zone detection method performed by a processor, comprising: a step of generating a final heatmap using a video image or a point cloud; and a step of analyzing the final heatmap to predict a risk zone, wherein the step of generating the final heatmap includes: a step of detecting an object from the video image or the point cloud; a step of generating a plurality of heatmaps each reflecting different characteristics related to accident risk based on information obtained from the detected object; a step of normalizing the range of grid values ​​for each of the generated plurality of heatmaps; and a step of generating a final heatmap by a weighted sum with different weights applied according to time period for each of the normalized plurality of heatmaps, and the step of analyzing the final heatmap to predict a risk zone includes: a step of finding grids in the final heatmap divided into a plurality of grids in which the grid value is greater than or equal to a first threshold value; a step of clustering the found grids to generate a zone; and a step of setting the generated zone as the risk zone. Claim 2 In claim 1, the step of finding grids in the heatmap divided into a plurality of grids in which the grid value is greater than or equal to a first threshold includes: setting a plurality of windows having different sizes; calculating an average value of grid values ​​included in each of the plurality of windows; and finding grids included in the windows in which the average value is greater than or equal to the first threshold, and the step of clustering the found grids to create a zone includes clustering grids included in the overlapping area of ​​the windows having an average value greater than or equal to the first threshold to create a zone, a heatmap-based accident risk zone detection method. Claim 3 A heatmap-based accident risk zone detection method according to claim 1, wherein the step of generating a plurality of heatmaps each reflecting different characteristics related to accident risk based on information obtained from the detected object comprises: generating a first heatmap based on collision risk; generating a second heatmap based on a sudden braking pattern; and generating a third heatmap based on dwell time. Claim 4 delete Claim 5 In claim 3, the step of generating a first heatmap based on the collision risk comprises: a step of calculating a Time To Collision (TTC) based on the position information and velocity information of a first object for each grid corresponding to a first object in a scene divided into multiple grids; a step of updating the collision risk level by accumulating the risk weight of the grid corresponding to the first object when it is determined that the collision risk level is less than or equal to a second threshold; and a step of generating the first heatmap according to the updated collision risk level. Claim 6 In claim 3, the step of generating a second heatmap based on the sudden braking pattern comprises: a step of determining whether the second object is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object in a scene divided into multiple grids; a step of calculating the acceleration of the second object if it is determined that the second object is in a decelerating state; and a step of generating the second heatmap according to the counted number of brakes if it is determined that the absolute value of the acceleration of the second object is greater than or equal to a third threshold value. Claim 7 In claim 3, the step of generating a third heatmap based on the dwell time comprises: a step of calculating and summing the dwell time of each of the plurality of objects based on the location information of each of the plurality of objects for each of the plurality of grids in a scene divided into a plurality of grids; a step of counting the number of the plurality of objects; a step of calculating the average dwell time of the plurality of objects by dividing the summed dwell time by the number of the plurality of objects; and a step of generating the third heatmap according to the average dwell time of the plurality of objects. Claim 8 A computing device comprising: a processor for executing heatmap-based accident risk zone detection commands; and a memory for storing said commands, wherein said commands are implemented to generate a final heatmap using a video image or a point cloud and to predict a risk zone by analyzing said final heatmap, said commands for generating the final heatmap are implemented to detect an object from said video image or the point cloud and to generate a plurality of heatmaps each reflecting different characteristics related to accident risk based on information obtained from said detected object, normalize the range of grid values ​​for each of said generated plurality of heatmaps and to generate a final heatmap by a weighted sum with different weights applied according to time period for each of said normalized plurality of heatmaps, and said commands for analyzing said final heatmap to predict a risk zone are implemented to find grids in said final heatmap divided into a plurality of grids where the grid value is greater than or equal to a first threshold, cluster said found grids to generate a zone, and set said generated zone as the risk zone. Claim 9 A computing device according to claim 8, wherein commands for finding grids in the heatmap divided into a plurality of grids in which the grid value is greater than or equal to a first threshold are implemented by setting a plurality of windows having different sizes, calculating an average value of grid values ​​included in each of the plurality of windows, and finding grids included in the windows in which the average value is greater than or equal to the first threshold, and commands for clustering the found grids to create a zone are implemented by clustering grids included in the overlapping area of ​​the windows having an average value greater than or equal to the first threshold to create a zone. Claim 10 A computing device according to claim 8, wherein commands for generating a plurality of heatmaps each reflecting different characteristics related to accident risk based on information obtained from the detected object are implemented to generate a first heatmap based on collision risk, generate a second heatmap based on sudden braking patterns, and generate a third heatmap based on dwell time. Claim 11 delete Claim 12 A computing device implemented such that, in claim 10, commands for generating a first heatmap based on the collision risk are, in a scene divided into multiple grids, a Time To Collision (TTC) is calculated based on the position information and velocity information of the first object for each grid corresponding to the first object, and when the collision risk is determined to be less than or equal to a second threshold, the collision risk level is updated by accumulating the risk weight of the grid corresponding to the first object, and the first heatmap is generated according to the updated collision risk level. Claim 13 A computing device implemented in accordance with claim 10, wherein commands for generating a second heatmap based on the above-mentioned sudden braking pattern determine whether the second object is in a decelerating state based on the speed information of the second object for each grid corresponding to the second object in a scene divided into multiple grids, and if it is determined that the second object is in a decelerating state, calculate the acceleration of the second object, and when it is determined that the absolute value of the acceleration of the second object is greater than or equal to a third threshold value, count the number of times the second object brakes and generate the second heatmap according to the counted number of times the brakes are generated. Claim 14 A computing device implemented in accordance with claim 10, wherein commands for generating a third heatmap based on the dwell time are, in a scene divided into multiple grids, based on the location information of each of the multiple objects for each of the multiple grids, to calculate and sum the dwell time of each of the multiple objects, to count the number of the multiple objects, to calculate the average dwell time of the multiple objects by dividing the summed dwell time by the number of the multiple objects, and to generate the third heatmap according to the average dwell time of the multiple objects.

Images