Traffic accident monitoring method using ai model

Abstract

The present invention relates to a method for detecting accident risk situations by analyzing CCTV footage using an artificial intelligence model, and for facilitating accident prevention and response by separately storing and managing monitoring footage regarding the risk situations. A traffic accident monitoring method according to one embodiment of the present invention is characterized by the steps of collecting CCTV footage, identifying a road boundary line and a moving object in the CCTV footage, respectively, and generating a monitoring image by processing the CCTV footage according to the distance between the moving object and the boundary line.

Description

Technology Field

[0001] The present invention relates to a method for detecting accident risk situations by analyzing CCTV footage with an artificial intelligence model, and for facilitating accident prevention and response by separately storing and managing monitoring footage of the risk situations. Background Technology

[0003] According to the 2021 TAAS (Traffic Accident Analysis System) data from the Korea Road Traffic Authority, out of 210,000 traffic accidents in 2020, there were 3,081 fatalities and approximately 300,000 injuries, and among them, the number of fatalities caused by jaywalking accidents was 336, accounting for 10.9% of the total fatalities.

[0004] As such, jaywalking accidents are the type of accident that results in the highest number of fatalities. Even if drivers pay close attention, jaywalking accidents often occur inevitably when pedestrians ignore traffic signals or cross the road without using crosswalks.

[0005] Nevertheless, the current domestic infrastructure for preventing jaywalking traffic accidents is severely lacking, and since the country's representative Intelligent Transport System (ITS) is based on Vehicle-to-Infrastructure (V2I), there is a problem in that pedestrian-centered intelligent risk detection is not being achieved.

[0006] Accordingly, there is a demand for technology that monitors the movement of pedestrians around roads to prevent traffic accidents, and even if an accident occurs, identifies the cause of the accident by identifying the movement of pedestrians before and after the accident and enables quick action. Prior art literature

[0008] Korean Patent Publication No. 10-2322831 (2021.11.01.) The problem to be solved

[0009] The present invention aims to analyze the movement of pedestrians and vehicles by analyzing CCTV footage using an artificial intelligence model and to separately manage CCTV footage of high-risk situations as monitoring footage.

[0010] The objects of the present invention are not limited to those mentioned above, and other unmentioned objects and advantages of the present invention may be understood from the following description and will be more clearly understood by the embodiments of the present invention. Furthermore, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims. means of solving the problem

[0012] A traffic accident monitoring method using an artificial intelligence model according to an embodiment of the present invention for achieving the aforementioned purpose is characterized by comprising the steps of: collecting CCTV footage; identifying a road boundary line and a moving object in the CCTV footage, respectively; and processing the CCTV footage according to the distance between the moving object and the boundary line to generate a monitoring image.

[0013] In one embodiment, the step of identifying the boundary line of the road is characterized by including the step of identifying the boundary line by detecting an edge within the CCTV image.

[0014] In one embodiment, the step of identifying the boundary line of the road comprises: a step of detecting a plurality of edges within the CCTV image by applying an edge detection algorithm to the CCTV image; and a step of identifying the boundary line by applying a Hough transform to the detected plurality of edges.

[0015] In one embodiment, the step of identifying the boundary line comprises: a step of identifying a plurality of straight lines by applying a Hough transform to the detected plurality of edges; and a step of identifying at least one straight line among the plurality of straight lines that corresponds to the direction of the road as the boundary line.

[0016] In one embodiment, the step of identifying the boundary line is characterized by including the step of identifying as the boundary line a straight line in which one component corresponding to the direction of the road among the vertical and horizontal components of the plurality of straight lines is greater than or equal to a reference value than other components.

[0017] In one embodiment, the step of identifying the boundary line comprises identifying the larger component among the vertical and horizontal components of the entire plurality of straight lines, and identifying as the boundary line any straight line among the plurality of straight lines that has one identified component greater than or equal to a reference value than other components.

[0018] In one embodiment, the step of identifying the boundary line is characterized by including the step of identifying as the boundary line a straight line having an angle among the plurality of straight lines that has a similarity to an angle corresponding to the direction of the road greater than or equal to a reference value.

[0019] In one embodiment, the step of identifying the boundary line is characterized by including the step of identifying the longest straight line among the plurality of straight lines and another straight line having an angle that is greater than or equal to a reference value as the boundary line.

[0020] In one embodiment, the step of identifying the moving object is characterized by including the step of inputting the CCTV image into an object detection neural network model to identify the moving object.

[0021] In one embodiment, the step of identifying the moving object is characterized by including the step of inputting the CCTV image into a neural network model to identify a plurality of objects, and the step of identifying an object among the identified plurality of objects whose movement distance according to frame change is greater than or equal to a reference value as the moving object.

[0022] In one embodiment, the step of identifying the moving object comprises: inputting the CCTV image into a neural network model to identify a plurality of objects; identifying at least one object located outside the boundary line among the identified plurality of objects; and identifying an object among the identified at least one object whose movement distance according to frame change is greater than or equal to a reference value as the moving object.

[0023] In one embodiment, the step of identifying the moving object is characterized by including the step of inputting the CCTV image into a neural network model to identify a plurality of objects, and the step of identifying at least one object among the identified plurality of objects whose direction of movement is toward the boundary line as the moving object.

[0024] In one embodiment, the step of identifying the moving object is characterized by including the step of identifying as the moving object at least one object among the identified plurality of objects whose direction of movement vector is similar to the direction perpendicular to the boundary line by a reference value or more.

[0025] In one embodiment, the step of identifying the moving object is characterized by including the step of identifying as the moving object any object in which the smaller of the vertical and horizontal components of the movement vector among the identified plurality of objects is greater than the other component.

[0026] In one embodiment, the step of generating the monitoring image is characterized by including the step of generating the monitoring image by dividing a CCTV image that includes a point in time when the sign of the distance between the moving object and the boundary line changes.

[0027] In one embodiment, the step of generating the monitoring image is characterized by including the step of generating the monitoring image by dividing a CCTV image that includes a point in time when the distance between the moving object and the boundary line becomes closer than or equal to a reference distance within a reference frame.

[0028] In one embodiment, the method further includes the step of storing the monitoring video separately from the CCTV video.

[0029] In one embodiment, the method is characterized by further including the step of transmitting the monitoring video to a control server. Effects of the invention

[0031] The present invention can automatically detect various dangerous situations where traffic accidents may occur without the need for separate monitoring personnel by analyzing the movement of pedestrians and vehicles within CCTV footage using an artificial intelligence model.

[0032] In addition, the present invention has the advantage of facilitating traffic accident prevention and accident response by separately managing CCTV footage of high-risk situations as monitoring footage.

[0033] In addition to the effects described above, the specific effects of the present invention are described together with the specific details for implementing the invention below. Brief explanation of the drawing

[0035] FIG. 1 is a drawing illustrating a traffic accident monitoring system according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a traffic accident monitoring method using an artificial intelligence model according to an embodiment of the present invention. FIG. 3 is a drawing illustrating an example of CCTV footage. FIG. 4 is a diagram showing the edges detected in the CCTV image shown in FIG. 3. FIG. 5 is a drawing showing a straight line detected at the edge shown in FIG. 4. FIG. 6 is a drawing showing the boundary lines identified among the straight lines shown in FIG. 5. FIG. 7 is a drawing illustrating another example of CCTV footage. FIG. 8 is a drawing showing the boundary lines identified through the CCTV footage shown in FIG. 7. FIG. 9 is a drawing showing the boundary lines identified in the CCTV image shown in FIG. 7. FIG. 10 is a diagram illustrating an object detection algorithm applied to the present invention. FIG. 11 is a drawing showing an object detected in the CCTV image shown in FIG. 3. FIGS. 12 to 15 are drawings for explaining the process of identifying a moving object in CCTV footage. Specific details for implementing the invention

[0036] The aforementioned objectives, features, and advantages are described in detail below with reference to the attached drawings, thereby enabling those skilled in the art to easily implement the technical concept of the present invention. In describing the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such descriptions would unnecessarily obscure the essence of the invention. Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. In the drawings, the same reference numerals are used to indicate the same or similar components.

[0037] In this specification, terms such as "first," "second," etc. are used to describe various components, but these components are not limited by these terms. These terms are used merely to distinguish one component from another, and unless specifically stated otherwise, the first component may be the second component.

[0038] Additionally, in this specification, the statement that any configuration is disposed on the "upper (or lower)" or "upper (or lower)" of a component may mean not only that any configuration is disposed in contact with the upper (or lower) surface of said component, but also that another configuration may be interposed between said component and any configuration disposed on (or below) said component.

[0039] Furthermore, where it is stated in this specification that one component is "connected," "coupled," or "connected" to another component, it should be understood that while the components may be directly connected or connected to each other, another component may be "interposed" between each component, or each component may be "connected," "coupled," or "connected" through another component.

[0040] Additionally, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the specification, and should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be included.

[0041] Additionally, in this specification, "A and / or B" means A, B, or A and B unless specifically stated otherwise, and "C to D" means C or more and D or less, unless specifically stated otherwise.

[0042] The present invention relates to a method for detecting jaywalking by pedestrians by analyzing CCTV footage with an artificial intelligence model, and for facilitating accident response by separately storing and managing monitoring footage when an accident occurs. Hereinafter, with reference to FIGS. 1 to 15, a method for monitoring traffic accidents using an artificial intelligence model according to an embodiment of the present invention will be described in detail.

[0043] FIG. 1 is a diagram illustrating a traffic accident monitoring system according to one embodiment of the present invention, and FIG. 2 is a flowchart illustrating a traffic accident monitoring method using an artificial intelligence model according to one embodiment of the present invention.

[0044] FIG. 3 is a diagram illustrating an example of a CCTV image, FIG. 4 is a diagram illustrating an edge detected in the CCTV image illustrated in FIG. 3, FIG. 5 is a diagram illustrating a straight line detected from the edge illustrated in FIG. 4, and FIG. 6 is a diagram illustrating a boundary line identified among the straight lines illustrated in FIG. 5.

[0045] FIG. 7 is a drawing illustrating another example of CCTV footage, FIG. 8 is a drawing illustrating a boundary line identified through the CCTV footage illustrated in FIG. 7, and FIG. 9 is a drawing illustrating a boundary line identified in the CCTV footage illustrated in FIG. 7.

[0046] FIG. 10 is a diagram illustrating an object detection algorithm applied to the present invention.

[0047] FIG. 11 is a drawing showing an object detected in a CCTV image shown in FIG. 3, and FIG. 12 to 15 are drawings for explaining the process of identifying a moving object in a CCTV image.

[0048] Referring to FIG. 1, a traffic accident monitoring system (1) according to one embodiment of the present invention may include a processor (10), a CCTV (20), and a control server (30). However, the traffic accident monitoring system (1) illustrated in FIG. 1 is according to one embodiment, and the components of the invention are not limited to the embodiment illustrated in FIG. 1, and some components may be added, changed, or deleted as needed.

[0049] The CCTV (20) of the present invention is installed around a road to collect traffic information and can capture images of the road and its surroundings. Meanwhile, the control server (30) is a server that controls various events that may occur on the road, such as traffic, safety, and firefighting, and may be, for example, a server of an administrative agency such as the Ministry of Land, Infrastructure and Transport, the Fire Department, or the National Police Agency, or an external server linked thereto.

[0050] The processor (10) is the entity performing the traffic accident monitoring method of the present invention, and can identify the movement of a pedestrian using an artificial intelligence model and generate a monitoring video for traffic accident prevention and follow-up measures based thereon.

[0051] For this operation, the processor (10) may be implemented as a CPU (central processing unit), GPU (graphics processing unit), etc., and may include at least one physical element among ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, and microcontrollers.

[0052] Referring to FIG. 2, a traffic accident monitoring method using an artificial intelligence model according to one embodiment of the present invention (hereinafter, traffic accident monitoring method) may include the steps of collecting CCTV images (S10), identifying a road boundary line and a moving object in the CCTV images (S20), generating a monitoring image based on the distance between the moving object and the boundary line (S30), storing the monitoring image separately (S40), and transmitting the monitoring image to a control server (30) (S50).

[0053] However, the traffic accident monitoring method illustrated in FIG. 2 is according to one embodiment, and the steps constituting the invention are not limited to the embodiment illustrated in FIG. 2, and some steps may be added, changed, or deleted as needed.

[0054] Below, each step illustrated in FIG. 2 will be explained in detail.

[0055] The processor (10) can collect CCTV images (100) from the CCTV (20) (S10). The CCTV images (100) refer to images captured by the CCTV (20) and may be images of the road and the surrounding area of ​​the road.

[0056] Referring to FIG. 3, in one example, the CCTV image (100) may capture a road, a moving vehicle, a sidewalk, a tree, a building, a pedestrian, etc.

[0057] The processor (10) can identify the boundary line (B) of the road and the moving object in the CCTV image (100) respectively (S20), and first, the operation of identifying the boundary line (B) will be explained in detail. In the present invention, the boundary line (B) is a line that divides the road within the CCTV image (100), and may be, for example, the boundary line (B) between the road and the sidewalk.

[0058] The processor (10) can identify a boundary line (B) by detecting edges within a CCTV image (100) using computer vision technology. To do this, the processor (10) can use various edge detection algorithms used in the relevant technical field, and, for example, can detect multiple edges within the CCTV image (100) by applying a Canny edge detection algorithm.

[0059] Referring to FIG. 4, when a Canny edge detection algorithm is applied to a CCTV image (100), the outlines of all edges (E) within the CCTV image (100), such as roads, moving vehicles, sidewalks, trees, buildings, and pedestrians, can be detected.

[0060] Next, the processor (10) may additionally apply a Hough transform to detect only the boundary line (B) among the detected multiple edges (E). Specifically, the processor (10) can identify pixels that have a straight-line relationship, that is, pixels located on a straight line, by applying a Hough line transform to the previously detected multiple edges (E).

[0061] In the Hough transform, the detection threshold can be appropriately set so that straight lines constituting small objects (e.g., vehicles, pedestrians, etc.) within the CCTV image (100) are not detected, and only the boundary line (B) that divides the road is detected.

[0062] Meanwhile, referring to FIG. 5, even if a Hough transform is applied to the previously detected edge (E), multiple other straight lines (L) can be detected in addition to the road boundary line (B). To identify which of these straight lines (L) is the road boundary line (B), the processor (10) can perform the following operations.

[0063] The processor (10) can identify at least one straight line (L) corresponding to the direction of the road as a boundary line (B) among a plurality of straight lines (L) identified by applying a Hough transform. Information regarding the direction of the road may be determined by the processor (10) by referring to the memory, as the information is stored in memory in advance, or the processor (10) may estimate it through CCTV footage (100).

[0064] In one embodiment, referring to FIG. 3 and FIG. 5 together, the processor (10) can determine that the direction of the road is vertical by referring to memory. The processor (10) can identify at least one straight line (L) corresponding to the direction of the road, i.e., the vertical direction, among the multiple straight lines (L) identified through the Hough transform as a boundary line (B). Accordingly, among the multiple straight lines (L) identified in the CCTV image (100), only the straight line (L) corresponding to the direction of the road can be identified as a boundary line (B).

[0065] Meanwhile, reflecting the characteristics of the CCTV image (100) captured according to perspective, the processor (10) can identify a straight line (L) as a boundary line (B) in which one component corresponding to the direction of the road among the vertical and horizontal components of multiple straight lines (L) identified through Hough transform is greater than or equal to a reference value than other components.

[0066] Referring again to FIGS. 3 and FIGS. 5, each line (L) identified through the Hough transform may have vertical and horizontal components. Specifically, if the position vector of a specific line (L) is (x, y), the horizontal component of the line (L) may be x and the vertical component may be y.

[0067] The processor (10) can identify vertical and horizontal components for each of the multiple straight lines (L), and when the direction of the road is vertical as in FIG. 3, the processor can identify the straight line (L) among the multiple straight lines (L) in which the vertical component is greater than the horizontal component by a reference value as a boundary line (B). On the other hand, when the direction of the road is horizontal as in FIG. 3, the processor (10) can identify the straight line (L) among the multiple straight lines (L) in which the horizontal component is greater than the vertical component by a reference value as a boundary line (B). Accordingly, among the multiple straight lines (L) identified in the CCTV image (100), only the straight line (L) corresponding to the direction of the road can be identified as a boundary line (B).

[0068] Since roads are generally longer than other objects captured in the CCTV video (100), even when considering perspective, the boundary line (B) of the road may have a significant tendency according to the direction of the road. Accordingly, in the present invention, the processor (10) can determine that a straight line (L) in which the vertical component is slightly larger than the horizontal component (larger than the reference value) when the direction of the road is vertical is not a boundary line (B) by setting a margin of reference value when comparing vertical and horizontal components.

[0069] In another embodiment, with reference to FIG. 3 and FIG. 5 together, the processor (10) can estimate the direction of the road through the CCTV image (100) and identify a boundary line (B) based thereon. Specifically, the processor (10) can identify the larger of the vertical and horizontal components of a plurality of straight lines (L), and identify a straight line (L) among the plurality of straight lines (L) in which the previously identified component is larger than the other components by a reference value as a boundary line (B).

[0070] The processor (10) can identify the vertical and horizontal components of each straight line (L) shown in FIG. 5. For example, the processor (10) can identify the position vector ((x) of each straight line (L). n , yn ), where n specifies each line (L) to define the perpendicular component (y n ) and horizontal component(x n ) can be identified. Subsequently, the processor (10) adds all the vertical components of each line (L) to obtain the total vertical component (y1+y2+…+y n ) can be calculated, and by adding all the horizontal components of each line (L), the total horizontal component (x1+x2+…+x) n ) can be produced.

[0071] As explained above, since the road is longer than other objects captured in the CCTV video (100), the vertical direction of the road is the total vertical component (y1+y2+…+y n ) is the total horizontal component(x1+x2+…+x n It can be greater than ), and in the case of the horizontal direction of the road, the total horizontal component(x1+x2+…+x n ) is the total vertical component(y1+y2+…+y n It can be larger than ). Accordingly, the processor (10) has a total vertical component (y1+y2+…+y n ) and total horizontal component(x1+x2+…+x n By comparing ), the larger component can be estimated as the direction of the road.

[0072] Next, the processor (10) can identify a line (L) among a plurality of straight lines (L) in which one previously identified component is greater than or equal to a reference value than another component as a boundary line (B), and accordingly, among the plurality of straight lines (L) identified in the CCTV image (100), only the straight line (L) corresponding to the direction of the road can be identified as a boundary line (B). Meanwhile, since the method of identifying a boundary line (B) through the comparison of vertical and horizontal components has been described above, a detailed explanation will be omitted here.

[0073] Referring to FIG. 6, two boundary lines (B) distinguishing a road and a sidewalk can be identified according to the aforementioned boundary line (B) identification method, and the processor (10) can store information about the location of the boundary line (B) in memory to calculate the distance between the boundary line (B) and the moving body described later.

[0074] Meanwhile, since the CCTV images (100) shown in FIGS. 3 to 6 are captured with the road in a vertical direction, accurate identification of the boundary line (B) may be possible using only the method described above. However, in the case of CCTV images (100) where the road extends along a diagonal direction, additional methods may be applied.

[0075] In one example, the processor (10) can identify a straight line (L) having an angle that is greater than or equal to a reference value in terms of similarity to an angle corresponding to the direction of the road among a plurality of straight lines (L) identified according to the method described above as a boundary line (B). The angle information corresponding to the direction of the road may be stored in memory in advance, or the processor (10) may estimate it directly through the CCTV image (100).

[0076] In one embodiment, referring to FIG. 7, the processor (10) refers to memory to find an angle corresponding to the direction of the road (hereinafter, road angle, θ). r ) can be determined. The processor (10) can determine the angle of each straight line (L) by referring to the position vector of the previously identified multiple straight lines (L), and the angle (θ) of the road r A straight line (L) having an angle with a similarity to () greater than or equal to a reference value can be identified as a boundary line (B). Accordingly, among the multiple straight lines (L) identified in the CCTV image (100), only the straight line (L) corresponding to the direction of the road can be identified as a boundary line (B).

[0077] Since roads are generally longer than other objects captured in the CCTV video (100), the boundary line (B) of the road is the road angle (θ rIt can have a significant tendency according to ). Accordingly, in the present invention, the processor (10) has an angle (θ) of each straight line (L) and a road angle. r When comparing ), by setting a margin of reference value, only straight lines (L) that are highly similar to the road angle can be determined as boundary lines (B).

[0078] Additionally, to remove noise, the processor uses the road angle (θ r Among the straight lines (L) that have high similarity to ), only the straight lines (L) whose length is greater than or equal to a reference value can be determined as boundary lines (B). Accordingly, straight lines (L) whose length is short and which are expected not to be boundary lines (B) of the road can be excluded.

[0079] In another embodiment, the processor (10) can estimate the road angle through the CCTV image (100) and identify the boundary line (B) based thereon. Specifically, the processor (10) can identify the longest straight line (L) among a plurality of straight lines (L) and identify another straight line (L) having an angle that is greater than or equal to a reference value with respect to the angle of the identified straight line (L) as the boundary line (B).

[0080] Referring to FIG. 8, the processor (10) can identify the longest straight line (hereinafter the longest straight line, LL) among a plurality of straight lines based, for example, on the magnitude of a position vector, and the angle (θ) of the longest straight line (LL) based on the position vector of the longest straight line (LL). l ) can be calculated. As previously explained, since the road is longer than other objects captured in the CCTV video (100), the longest straight line (LL) identified in the CCTV video (100) may be the boundary line (B) of the road, and the processor (10) can calculate the angle (θ) of the longest straight line (LL). l ) can be estimated as the angle of the road.

[0081] Next, the processor (10) determines the angle (θ) of the longest straight line (LL). lA straight line (L) having an angle with a similarity to a reference value greater than or equal to a reference value can be identified as a boundary line (B), and accordingly, among the multiple straight lines (L) identified in the CCTV image (100), only the straight line (L) corresponding to the direction of the road can be identified as a boundary line (B).

[0082] Meanwhile, even at this time, to remove noise, the processor uses the angle (θ) of the longest straight line (LL). l Among the straight lines (L) that have high similarity to ), only the straight lines (L) whose length is greater than or equal to the reference value can be identified as boundary lines (B).

[0083] Referring to FIG. 9, two boundary lines (B) distinguishing the road and the outside of the road can be identified according to the aforementioned boundary line (B) identification method, and the processor (10) can store information about the location of the boundary line (B) in memory to calculate the distance between the moving body and the boundary line (B) described later.

[0084] Hereinafter, the operation of identifying a moving object of the processor (10) will be described in detail. In the present invention, a moving object refers to an object that moves according to frame changes within the CCTV image (100), and may include, for example, a vehicle, a pedestrian, a bicycle, a motorcycle, etc.

[0085] The processor (10) can identify moving objects within the CCTV video (100) using a neural network model (200). Specifically, the processor (10) can identify moving objects by inputting the CCTV video (100) into a Convolutional Neural Network (CNN)-based object detection neural network model (200).

[0086] To this end, the processor (10) may use various object detection neural network models (200) used in the relevant technical field, but since the processor (10) must perform a moving object identification operation for CCTV images (100) captured in real time, it is preferable that the neural network model (200) in the present invention be a YOLO (You Only Look Once) based neural network model (200) having a simple neural network architecture and fast processing time.

[0087] Specifically, referring to FIG. 10, the neural network model (200) may be a YOLOv5 neural network model (200), and among them, may be a model that detects medium-sized objects. When a CCTV image (100) is input to the YOLOv5 neural network model (200), the backbone part extracts a feature map of the image, and the head part determines the location of the object based on the previously extracted feature map and outputs three different result values ​​of different image sizes.

[0088] Referring to FIG. 11, when a CCTV image (100) is input into a neural network model (200), all objects within the CCTV image (100), such as a car, a truck, a pedestrian, etc., can be detected, and it is obvious that the neural network model (200) can be supervised learning in advance for such object classification and detection.

[0089] Next, the processor (10) can detect only moving objects among the detected objects, and to do this, the processor (10) can identify as moving objects among multiple objects that the moving distance according to the frame change is greater than or equal to a reference value.

[0090] Specifically, the processor (10) can calculate the distance traveled by each object by comparing the positions of each object captured in adjacent frames of the CCTV video (100). To do this, the processor (10) can use an object tracking algorithm known in the art, for example, the Sort (Simple Online and Realtime Tracking) algorithm. The processor (10) can identify an object whose calculated distance traveled is greater than or equal to a reference value as a moving object.

[0091] Meanwhile, jaywalking may occur due to negligence of pedestrians or bicycles, and to selectively monitor them, the processor (10) can identify only moving objects located outside the road.

[0092] Specifically, the processor (10) can identify at least one object located outside the previously identified boundary line (B) among multiple objects identified through the neural network model (200), and can identify an object among the at least one object whose movement distance according to frame change is greater than or equal to a reference value as a moving object.

[0093] Referring to FIG. 12, the processor (10) can identify objects m1, m2, m3, which are people, and c1, c2, which are vehicles, through a neural network model (200). The processor (10) can identify objects located outside the boundary line (B) as m1, m2, and m3 by comparing the location of each object with the location of the boundary line (B). Specifically, the processor (10) can determine that objects (c1, c2) between the two previously identified boundary lines (B) are located inside the boundary line (B), and determine that other objects (m1, m2, m3) are located outside the boundary line (B). Subsequently, the processor (10) can identify only objects among the objects outside the boundary line (B) whose movement distance according to frame change is greater than or equal to a reference value as moving objects.

[0094] Meanwhile, in FIGS. 13 to 15, only the boundary line (B) and the object are schematically depicted for convenience of explanation, and the object is represented by a point and the movement of the object is represented by an arrow.

[0095] Referring to FIG. 13, the processor (10) can calculate the distance traveled (d1, d2, d3) for objects m1, m2, and m3 located outside the boundary line (B) through a frame-by-frame position change. At this time, only m1 and m2, whose distance traveled is greater than or equal to a reference value, can be identified as moving objects.

[0096] Since pedestrians attempting to jaywalk generally move quickly instead of at a normal walking speed, a margin of the threshold value can be applied when identifying the moving objects subject to monitoring, thereby enabling the identification of only pedestrians with inherent risk as moving objects.

[0097] Meanwhile, in the case of a traffic accident, it may occur when a pedestrian enters the road or a vehicle enters the pedestrian walkway, and to monitor this, the processor (10) can identify only moving objects moving toward the boundary line (B).

[0098] Specifically, the processor (10) can identify at least one object among multiple objects identified through the neural network model (200) whose direction of movement is toward the boundary line (B) of the road as a moving object. At this time, the location information of the boundary line (B) may be previously identified by the processor (10) and stored in memory, or the processor (10) may directly estimate it through the direction of the road and the movement of the object.

[0099] In one embodiment, referring to FIG. 14, the processor (10) can determine the location of the boundary line (B) by referring to memory. The processor (10) can identify the direction of movement of each object (m1, m2, m3, c1) based on the movement of the object according to the change in the frame. Subsequently, the processor (10) can identify only the two objects (m1, m2) among the four objects (m1, m2, m3, c1) whose direction of movement is toward the boundary line (B) as moving objects.

[0100] Specifically, the processor (10) calculates the movement vector of each object and can identify at least one object as a moving object if the direction of the movement vector is similar to the direction (z) perpendicular to the boundary line (B) by more than a reference value. Here, the movement vector may be a vector connecting the starting point to the ending point of the object's position that moved according to the frame change.

[0101] Referring again to FIG. 14, the processor (10) can calculate movement vectors (v1, v2, v3, v4) based on the position change of objects (m1, m2, m3, c1) according to frame change. The processor (10) can calculate the similarity between the direction of each movement vector (v1, v2, v3, v4) and the direction (z) perpendicular to the boundary line (B), and can identify objects (m1, m2) whose calculated similarity is greater than or equal to a reference value as moving objects.

[0102] In another embodiment, the processor (10) can estimate the direction of the road or boundary line (B) through the movement vector of each object and, based on this, identify the object facing the boundary line (B) as a moving object. Specifically, the processor (10) can identify an object as a moving object in which either the vertical or horizontal component of the movement vector is smaller than the other component among a plurality of objects.

[0103] Referring to FIG. 15, the processor (10) can calculate movement vectors (v1, v2, v3, v4, v5) based on the position change of objects (m1, m2, m3, m4, c1) according to frame changes. Each movement vector (v n =(v nx , v ny ), where n is each object) is the vertical component (v ny ) and horizontal component (v nx It can have ), and the processor (10) adds all the vertical components of each movement vector to get the total vertical component (v 1y +v 2y +… +v ny ) can be calculated, and the total horizontal component (v) is obtained by adding all the horizontal components of each movement vector. 1x +v 2x +… +v nx ) can be produced.

[0104] Since it is common for all objects around a road to move along the direction of the road, that is, the extension direction of the boundary line (B), when the road is in a vertical direction, the total vertical component (v) 1y +v 2y +… +v ny ) is the total horizontal component (v 1x +v 2x +… +v nx It can be greater than ), and when the road is in the horizontal direction, the total horizontal component (v 1x +v 2x +… +v nx ) is the total vertical component (v 1y +v 2y +… +v ny It can be larger than )

[0105] Accordingly, the processor (10) has a total vertical component (v 1y +v 2y +… +v ny ) and total horizontal component (v 1x +v 2x +… +v nx By comparing ), the smaller component can be estimated as the component perpendicular to the direction of the road, that is, the component facing the boundary line (B).

[0106] Referring again to FIG. 15, the total vertical component (v) for the movement vectors (v1, v2, v3, v4, v5) of each object (m1, m2, m3, m4, c1) 1y +v 2y +… +v 5y ) is the total horizontal component (v 1x +v 2x +… +v 5x Since it is greater than ), the processor (10) can estimate that the road is in a vertical direction, and accordingly, the horizontal component (v nx ) is the vertical component (v ny Only objects (m1) that are larger than ) and preferably larger than a reference value can be identified as moving objects.

[0107] When the boundary line (B) and the moving body are identified according to the method described above, the processor (10) can process the CCTV image (100) based on the distance between the moving body and the boundary line (B) to generate a monitoring image (S30). In other words, the processor (10) can determine the risk level based on the distance between the boundary line (B) and the moving body, and can generate a monitoring image only when the risk level is high.

[0108] In one embodiment, the processor (10) can generate a monitoring image by dividing a CCTV image (100) that includes a point in time when the sign of the distance between the moving object and the boundary line (B) changes.

[0109] For example, as illustrated in FIG. 14, the processor (10) can calculate the distance between the boundary line (B) and the moving body in the vertical direction (z) with respect to the boundary line (B) frame by frame. When a moving body outside the boundary line (B) enters the boundary line (B) or when a moving body inside the boundary line (B) enters the boundary line (B), the sign of each distance value calculated in two adjacent frames may be different.

[0110] The processor (10) can generate a monitoring video by cutting out a CCTV video (100) of a preset time (e.g., 5 minutes) that includes a frame with a changed code at a certain point in time, i.e., a frame with a changed code. Accordingly, the processor (10) can obtain a monitoring video of a CCTV video (100) of a certain time that includes a point in time when a pedestrian entered the road or a vehicle entered the pedestrian walkway.

[0111] In one embodiment, the processor (10) can generate a monitoring image by dividing a CCTV image (100) that includes a point in time when the distance between the moving object and the boundary line (B) becomes closer than the reference distance within the reference frame.

[0112] The processor (10) can calculate the distance between the boundary line (B) and the moving body in a vertical direction (z) with respect to the boundary line (B) on a frame-by-frame basis. If a moving body inside or outside the boundary line (B) moves rapidly toward the boundary line (B), the distance between the boundary line (B) and the moving body may become closer than the reference distance within the reference frame.

[0113] Accordingly, with reference to FIG. 14, even when m1 and m2 are identified as moving bodies according to the method described above, the processor (10) can determine that m1 is closer to the boundary line than the reference distance within the reference frame, and m2 is closer to the boundary line less than the reference distance within the reference frame.

[0114] Accordingly, the processor can generate only monitoring video for m1, and specifically, can generate monitoring video by cutting out CCTV video (100) of a preset time that includes a point in time when the distance between the boundary line (B) and m1 is calculated to be greater than or equal to a reference distance within a reference frame. Accordingly, the processor (10) can obtain monitoring video of CCTV video (100) of a certain time that includes a point in time when a pedestrian attempts to quickly enter the road or a vehicle attempts to quickly enter the pedestrian walkway.

[0115] The processor (10) can store the monitoring video generated according to the above method separately from the CCTV video (100) (S40). A general traffic information collection CCTV (20) can only store video from the last 30 days at most due to the limitation of its internal capacity. Accordingly, when analyzing past traffic accidents, even if a CCTV (20) is installed at the accident scene, if a certain amount of time passes, the video at the time of the accident is deleted, making it difficult to determine the cause of the accident.

[0116] To overcome these limitations, the processor (10) of the present invention can generate a CCTV video (100) of a high-risk situation as a monitoring video and store it separately in a separate database. Here, the database may include any storage device, a cloud server, etc.

[0117] Additionally, the processor (10) can transmit the monitoring video to the control server (30) (S50). As described with reference to FIG. 1, the control server (30) may be a server that controls events that may occur on the road, such as traffic, safety, and firefighting.

[0118] Accordingly, the control server (30) can check monitoring footage of high-risk situations in real time, and can immediately identify and take action quickly even if a traffic accident occurs.

[0119] As described above, the present invention has the advantage of automatically detecting various dangerous situations where traffic accidents may occur without the need for separate monitoring personnel by analyzing the movement of pedestrians and vehicles within CCTV footage using an artificial intelligence model, and facilitating traffic accident prevention and accident response by separately managing CCTV footage of high-risk situations as monitoring footage.

[0120] Although the present invention has been described above with reference to the illustrated drawings, the present invention is not limited by the embodiments and drawings disclosed in this specification, and it is obvious that various modifications can be made by a person skilled in the art within the scope of the technical concept of the present invention. Furthermore, even if the effects of the configuration of the present invention were not explicitly described while explaining the embodiments of the present invention above, it is natural to acknowledge that the effects predictable by said configuration should also be recognized.

Claims

Claim 1 A method for monitoring traffic accidents comprising: a step of collecting CCTV footage; a step of detecting multiple edges within the CCTV footage by applying an edge detection algorithm to the CCTV footage; a step of identifying multiple straight lines by applying a Hough transform to the detected multiple edges; a step of estimating the direction of the road by comparing the sum of the vertical components and the sum of the horizontal components of all the multiple straight lines and determining the larger component; a step of identifying a straight line among the multiple straight lines where one component corresponding to the direction of the road is greater than or equal to a reference value than other components as the boundary line of the road; a step of identifying multiple objects by inputting the CCTV footage into a neural network model; a step of identifying at least one moving object among the identified multiple objects whose direction of movement is toward the boundary line; and a step of generating a monitoring image by processing the CCTV footage according to the distance between the moving object and the boundary line. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 A traffic accident monitoring method according to claim 1, wherein the step of identifying the moving object includes the step of inputting the CCTV video into an object detection neural network model to identify the moving object. Claim 10 A traffic accident monitoring method according to claim 1, wherein the step of identifying the moving object comprises the step of inputting the CCTV video into a neural network model to identify multiple objects, and the step of identifying among the identified multiple objects an object whose movement distance according to frame change is greater than or equal to a reference value. Claim 11 A traffic accident monitoring method according to claim 1, wherein the step of identifying the moving object comprises: identifying at least one object located outside the boundary line among the identified plurality of objects; and identifying as the moving object among the identified at least one object whose movement distance according to frame change is greater than or equal to a reference value. Claim 12 delete Claim 13 A traffic accident monitoring method according to claim 1, wherein the step of identifying the moving object includes the step of identifying at least one object among the identified plurality of objects as the moving object, wherein the direction of the movement vector is similar to the direction perpendicular to the boundary line by a reference value or more. Claim 14 A traffic accident monitoring method according to claim 1, wherein the step of identifying the moving object includes the step of identifying as the moving object any object in which the smaller of the vertical and horizontal components of the movement vector among the identified plurality of objects is greater than the other component. Claim 15 A traffic accident monitoring method according to claim 1, wherein the step of generating the monitoring image includes the step of generating the monitoring image by dividing a CCTV image including a point in time when the sign of the distance between the moving body and the boundary line changes. Claim 16 A traffic accident monitoring method according to claim 1, wherein the step of generating the monitoring image comprises dividing a CCTV image including a point in time when the distance between the moving body and the boundary line becomes closer than or equal to a reference distance within a reference frame to generate the monitoring image. Claim 17 A traffic accident monitoring method according to claim 1, further comprising the step of storing the monitoring video separately from the CCTV video. Claim 18 A traffic accident monitoring method according to claim 1, further comprising the step of transmitting the monitoring video to a control server.

Images