Traffic Accident Detection Method and Traffic Accident Detection System

Abstract

A traffic accident detection method according to a disclosed embodiment comprises: a step of generating bounding boxes for objects included in a plurality of frames captured by a CCTV (Closed-Circuit TeleVision) based on an object recognition algorithm; a step of extracting the trajectory of the object included in the bounding box based on a SORT (Simple Online and Realtime Tracking) algorithm; a step of generating an influence map representing the relationship between the trajectories of the object by adding a notation including at least one of a symbol, a label, text, or a color to the extracted trajectory of the object; and a step of inputting the influence map as an input value to a CNN (Convolutional Neural Network) based algorithm and determining whether the CCTV frame is an accident video based on the output value of the CNN based algorithm.

Description

Technology Field

[0001] The present invention relates to a traffic accident detection method and a traffic accident detection system that detect whether an accident has occurred by using trajectory tracking and influence maps from a plurality of frames captured by a CCTV (Closed-Circuit Television). Background Technology

[0002] Artificial intelligence technologies such as machine learning, object detection, and trajectory tracking are being actively applied in the fields of autonomous driving and traffic accident detection. For example, autonomous vehicles can detect the driving environment in real time through object detection or trajectory tracking while driving on the road, and can detect traffic accidents in real time through the analysis of various images collected by RGB cameras and LiDAR sensors.

[0003] However, conventional artificial intelligence research has largely focused on analyzing traffic accidents or driving environments based on images captured by RGB cameras or LiDAR sensors, and research utilizing images from CCTVs for AI analysis is not actively being conducted. This is because the video or images captured by CCTVs extend beyond just traffic situations to include a wide variety of everyday scenarios, and the coverage area of ​​fixed-installation CCTVs is very limited. Consequently, generally researched AI for image analysis has faced limitations in detecting traffic accidents from images captured by CCTVs.

[0004] (Patent Document 1) KR 10-2022-0098677 A The problem to be solved

[0005] The disclosed embodiment relates to a traffic accident detection method and a traffic accident detection system in which an artificial intelligence determines whether an accident has occurred in a CCTV frame by generating an influence map based on the distance ratio and the size of the bounding box based on the trajectory of an object recognized in a frame captured by a CCTV. means of solving the problem

[0006] A traffic accident detection method according to one embodiment comprises: a step of generating bounding boxes for objects included in a plurality of frames captured by a CCTV (Closed-Circuit TeleVision) based on an object recognition algorithm; a step of extracting the trajectory of an object included in the bounding box based on a SORT (Simple Online and Realtime Tracking) algorithm; and a step of generating an influence map representing the relationship between the trajectories of the object by adding a notation including at least one of a symbol, a label, text, or a color to the extracted trajectory of the object. The method may include the step of inputting the above impact map as an input value to a CNN (Convolutional Neural Network) based algorithm, and determining whether the CCTV frame is an accident video based on the output value of the CNN based algorithm. The step of generating the above impact map may include the step of generating a plurality of first circles, the size and brightness of which are determined based on the distance ratio between the orbit of a first object and the orbit of a second object. The step of generating the plurality of first circles may further include the step of determining the positions of the plurality of first circles based on the position of the object. The step of generating the above impact map may further include the step of generating a second circle, the size and brightness of which are determined based on the position occupied by the orbit of the object. The step of generating the above impact map may include the step of generating a line representing the orbit of the object by connecting the positions of the plurality of first circles. The object recognition algorithm may include a YOLO (You Only Look Once) algorithm, and the SORT algorithm may include a Deep SORT algorithm.If the distance ratio between the first object and the second object decreases, the radius of the first circle generated may increase, and the color of the first circle may become darker. The step of generating the influence map may indicate that if the size of the object increases, the size of the second circle increases, and an increase in the size of the second circle may indicate an increased probability of an accident. A traffic accident detection system according to one embodiment includes a processor for pre-training artificial intelligence, a memory for storing data, and a display for outputting a CCTV frame. The processor includes: a bounding box generator that receives a CCTV frame input from a user and generates a bounding box for an object included in the CCTV frame based on an object recognition algorithm; a trajectory extractor that extracts the trajectory of an object included in the bounding box based on a SORT algorithm; and an influence map generator that generates an influence map indicating the relationship between the trajectories of the object by adding a notation including at least one of a symbol, label, text, or color to the trajectory of the extracted object. and may include an accident determination unit that inputs the above impact map as an input value to a CNN-based algorithm and determines whether the CCTV frame is an accident video based on the output value of the CNN-based algorithm. The impact map generator may generate a plurality of first circles whose size and color brightness are determined based on the distance ratio between the orbit of a first object and the orbit of a second object. The impact map generator may generate a value that distinguishes an accident based on the position occupied by the orbit of the object, and may generate a second circle indicating the point of accident occurrence based on the generated value. The impact map generator may generate a line representing the orbit of each object using the coordinates of the orbit corresponding to the first circle.The above impact map generator may increase the radius of the first circle generated and the color of the first circle as the distance ratio between the first object and the second object decreases. The above impact map generator may indicate that as the size of the object increases, the size of the second circle increases, and that an increase in the size of the second circle indicates an increased probability of an accident. Effects of the invention

[0007] A traffic accident detection method and a traffic accident detection system according to a disclosed embodiment generate an influence map based on the distance ratio and the size of the bounding box based on the trajectory of an object recognized in a video frame, thereby enabling artificial intelligence to determine whether an accident has occurred in a video frame.

[0008] In addition, the traffic accident detection method and traffic accident detection system according to the disclosed embodiment can distinguish even accidents that are difficult for conventional artificial intelligence to distinguish in CCTV frames with high accuracy and a small amount of training. Brief explanation of the drawing

[0009] FIG. 1 is a diagram illustrating the hardware configuration of a disclosed traffic accident detection system. Figure 2 is a control block diagram for each component of a traffic accident detection system. FIG. 3 is a flowchart illustrating a disclosed traffic accident detection method. Figure 4 is a diagram for explaining the flowchart of Figure 3. FIG. 5 is a flowchart for specifically explaining the operation of an influence map generator according to a disclosed embodiment. Figure 6 is an image of an example of a generated influence map. Figure 7 is a table explaining the embodiment of Figure 6 by comparing it with an actual CCTV frame. Figure 8 is a table for describing parameters for training an algorithm included in the disclosed traffic accident detection system. Figure 9 is a table for explaining the specifications of the data set of the disclosed traffic accident detection system. FIGS. 10 and 11 are tables to show the results of a performance evaluation of a disclosed traffic accident detection system. Figures 12 and 13 are graphs for comparing the performance of a conventional general algorithm and a disclosed traffic accident detection system. Figure 14 is a graph comparing methods for determining whether an accident occurred through approaches other than the disclosed impact map. Specific details for implementing the invention

[0010] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. However, the technical concept of the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed content is thorough and complete and to ensure that the concept of the present invention is sufficiently conveyed to those skilled in the art.

[0011] In this specification, when a component is described as being on another component, it means that it may be formed directly on the other component or that a third component may be interposed between them. Additionally, in the drawings, shapes and sizes are exaggerated for the effective illustration of the technical content.

[0012] Additionally, although terms such as first, second, third, etc., have been used to describe various components in the various embodiments of this specification, these components should not be limited by such terms. These terms are used merely to distinguish one component from another. Accordingly, what is referred to as the first component in one embodiment may be referred to as the second component in another embodiment. Each embodiment described and illustrated herein also includes its complementary embodiment. Furthermore, in this specification, "and / or" is used to mean including at least one of the components listed before and after it.

[0013] In the specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, terms such as "include" or "have" are intended to specify the existence of the features, numbers, steps, components, or combinations thereof described in the specification, and should not be understood as excluding the existence or addition of one or more other features, numbers, steps, components, or combinations thereof. Additionally, in this specification, "connection" is used to include both indirectly connecting multiple components and directly connecting them.

[0014] Furthermore, in describing the present invention below, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the invention, such detailed description will be omitted.

[0015] FIG. 1 is a diagram illustrating the hardware configuration of a disclosed traffic accident detection system, and FIG. 2 is a control block diagram for each component of the traffic accident detection system. To avoid redundant descriptions, they are described together below.

[0016] Referring first to FIG. 1, the traffic accident detection system (1) can be configured as an example of a user terminal. The traffic accident detection system (1) can acquire various data necessary for artificial intelligence learning from the outside and learn various artificial intelligence necessary for traffic accident detection through the learning data. In particular, the traffic accident detection system (1) can output to the user the result of detecting whether an accident has occurred in a video captured by a CCTV, and among the multiple frames that make up the video.

[0017] The user terminal that hardware-configures the traffic accident detection system (1) can be implemented as a computer or portable terminal that can be accessed externally through a network. Here, the computer includes, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser, and the portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs). If the user terminal is used as a portable terminal, a large capacity Data storage or AI training can be performed separately through an external server.

[0018] The traffic accident detection system (1) includes a processor (10) that implements an algorithm for performing a traffic accident detection method described below, a memory (11) that stores the algorithm processed by the processor (10) and data required for algorithm processing, and a display (12) that displays CCTV images through CCTV frames and indicates whether an accident has been detected based on the CCTV frames.

[0019] Specifically, the processor (10) implements not only an algorithm for controlling the operation of components within the traffic accident detection system (1), but also an object recognition algorithm for recognizing objects included in CCTV frames, a SORT (Simple Online and Realtime Tracking) algorithm for extracting the trajectory of bounding boxes recognized as objects, an image preprocessing algorithm for generating an impact map from images (Object Trajectories) containing trajectories, and a CNN (Convolutional Neural Network) based algorithm for distinguishing whether an accident occurred in the generated impact map.

[0020] The memory (11) stores not only CCTV frames received from the outside but also large amounts of data required for learning the aforementioned algorithm. The memory (11) stores various data required for the operation of the process (10) and can collect various data required by the traffic accident detection system (1) below.

[0021] The memory (11) may be implemented as at least one of a non-volatile memory device such as a cache, ROM (Read Only Memory), PROM (Programmable ROM), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and flash memory, a volatile memory device such as RAM (Random Access Memory), or a storage medium such as a hard disk drive (HDD) and CD-ROM, but is not limited thereto. The memory (11) may be implemented as a separate chip from the aforementioned processor (10), but may also be implemented as a single chip with the processor (10), unlike as shown in FIG. 1.

[0022] The display (12) is configured to display CCTV frames while displaying the results of the process (10) detecting whether an accident has occurred as a user interface. The display (12) may be provided with a Digital Light Processing (DLP) panel, a Plasma Display Panel, a Liquid Crystal Display (LCD) panel, an Electro Luminescence (EL) panel, an Electrophoretic Display (EPD) panel, an Electrochromic Display (ECD) panel, a Light Emitting Diode (LED) panel, a Cathode Ray Tube (CRT), or an Organic Light Emitting Diode (OLED) panel, but is not limited thereto.

[0023] Meanwhile, the traffic accident detection system (1) may include various additional configurations not shown in FIG. 1.

[0024] For example, the traffic accident detection system (1) may be linked with a cloud server, etc., to receive CCTV frames or various artificial intelligence algorithms from the outside.

[0025] As another example, the traffic accident detection system (1) may further include an input unit that receives user input. Here, the input unit may include a device that is a GUI (Graphical User Interface), i.e., software, such as a touch pad, for user input, and the touch pad may be implemented as a touch screen panel (TSP) to form a layered structure with the display (12).

[0026] Referring to FIG. 2, the disclosed traffic accident detection system (1) includes a bounding box generator (21) that generates a bounding box after recognizing an object in a CCTV frame, a trajectory extractor (22) that extracts the trajectory of each object in a CCTV frame including the generated bounding box, an influence map generator (31) that generates an influence map based on the object trajectories (2D Object Trajectories) generated by the trajectory extractor (22), and a detection executor (32) that distinguishes an accident within an image based on the generated influence map.

[0027] The traffic accident detection method executed by each control block can be divided into a Tracking Phase executed by a bounding box generator (21) and a track extractor (22), and an Execution Phase executed by an impact map generator (31) and an accident judge (32).

[0028] In the tracking phase, a CCTV frame is input, and an image displaying the trajectory of the object included in the frame—namely, a 2D object trajectory—is output. In the execution phase, when the image displaying the trajectory is input, a result determining whether an accident occurred in that image is output.

[0029] Specifically, the bounding box generator (21) generates a bounding box for each object included in the CCTV frame based on an object recognition algorithm. The bounding box generator (21) according to the disclosed embodiment may use the YOLO (You Only Look Once) algorithm and may detect multiple objects simultaneously in a single frame. The CCTV frame containing the bounding box generated by the bounding box generator (21) is input to the orbit extractor (22).

[0030] The bounding box generator (21) recognizes dynamic objects on the road in the CCTV frame. For example, the bounding box generator (21) can distinguish not only vehicles but also pedestrians, and the bounding box generator (21) can use the CADP (Car Accident Detection and Prediction) dataset as training data. A detailed explanation of the data learned by the bounding box generator (21) will be provided later through FIG. 8, etc.

[0031] The orbit extractor (22) extracts the orbit of an object contained in a bounding box based on the SORT (Simple Online and Realtime Tracking) algorithm.

[0032] A trajectory extractor (22) according to a disclosed embodiment can track the trajectory of an object using a Deep SORT (Deep Learning-based SORT) algorithm. Here, the Deep SORT algorithm is a tracking technology that combines deep learning and a SORT algorithm. After extracting the location and features of an object using deep learning technology, the extracted features are stored together with the serial number of the object. When an object is detected again, the trajectory is extracted after confirming whether the object is a previously tracked object by comparing it with the stored features.

[0033] The orbit extractor (22) extracts the trajectories of each of the multiple objects recognized by the bounding box generator (21), and the object trajectories (2D Object Trajectories) extracted by the orbit extractor (22) are input into the influence map generator (31).

[0034] The influence map generator (31) generates an influence map that defines the relationship between the orbits of each object by adding a notation including at least one of a symbol, label, text, or color to the orbit of the object extracted by the orbit extractor (22).

[0035] Specifically, the impact map generator (31) encodes object trajectories (2D Object Trajectories) by adding spatiotemporal information as a notation. The impact map containing spatiotemporal information is input to the accident judgment unit (32), and the accident judgment unit determines (infers) whether there was an accident in the CCTV frame using the impact map as an input value.

[0036] Here, the influence map is a technique primarily used in game development to determine the movement and behavior patterns of NPCs (Non-Player Characters), and in the influence map, a circle represents a type of region. That is, in the influence map, a circle indicates the degree of influence of an object centered on that point, and the larger the size of the circle, the wider the range of influence of the object. The disclosed influence map can generate circles along a trajectory by considering the speed of the object and the level of danger at a specific point. The specific method by which the influence map generator (31) generates the influence map will be described later through other drawings below.

[0037] The accident judgment unit (32) determines whether a CCTV frame is an accident video using a CNN-based algorithm with an impact map as input. The CNN-based algorithm used by the accident judgment unit (32) distinguishes the impact map and biases the impact map while assigning important weights. The accident judgment unit (32) according to the disclosed embodiment may be a CNN composed of 5 convolution layers, 5 max pooling layers, and 3 fully connected layers.

[0038] According to the disclosed example, the impact map input to the accident judge (32) may be a map containing three RGB channels with a size of 224x224 pixels. When the impact map is input, the convolution layer may use a 3 x 3 kernel, and the second and fifth convolution layers may have 64 and 512 channels, respectively. The max polling layer may use a 2 x 2 kernel to maximize the retention of spatiotemporal features extracted from the impact map and minimize the dimensionality of the feature map of the convolution layer. Additionally, in this embodiment, batch normalization and a ReLU (Rectified Linear Unit) activation function are added to the convolution layer, and batch normalization may be performed to prevent instability of the ReLU due to the size of the impact map. The size of the Fully Connected Layer can be 4096, 1024, and 2, and the CNN-based algorithm distinguishes whether a CCTV frame contains (1) or does not contain (0) an accident.

[0039] The value distinguished through the accident judgment device (32) can be output through the display (12), etc.

[0040] Meanwhile, each control block described in Fig. 2 is separated for the convenience of explaining the traffic accident detection method, and in terms of hardware, it can be executed on a single processor.

[0041] Each artificial intelligence algorithm described in FIG. 2 is merely an example, and various variations may exist. For example, the bounding box generator (21) does not necessarily have to use only YOLO, and may use other algorithms capable of object recognition. The orbit extractor (22) also does not necessarily have to extract orbits limited to the Deep SORT algorithm, and likewise, the CNN used by the accident judge (32) is not necessarily limited to the specifications described in FIG. 2, and may include various forms of artificial intelligence capable of judging accidents in an impact map.

[0042] FIG. 3 is a flowchart for explaining a disclosed traffic accident detection method, and FIG. 4 is a diagram for explaining the flowchart of FIG. 3. To avoid redundant explanations, they are described together below.

[0043] The traffic accident detection system (1) generates a bounding box for the object through an object recognition algorithm (40).

[0044] In one disclosed embodiment, when a plurality of CCTV frames are input, the object recognition algorithm that generates bounding boxes may be YOLO V5. As illustrated in FIG. 4, the object recognition algorithm, specifically the bounding box generator (21), can output a plurality of images (2D Object Bounding Boxes) containing bounding boxes for each object.

[0045] The traffic accident detection system (1) extracts the trajectory of an object based on the SORT algorithm (41).

[0046] In one disclosed embodiment, the trajectory extractor (22) can output object trajectories (2D Object Trajectories) through a Deep SORT algorithm. Here, the image with the trajectory displayed is an image generated by extracting a trajectory for each object among several objects recognized in a CCTV frame, and as shown in FIG. 4, an image with multiple trajectories displayed for each object can be generated.

[0047] The traffic accident detection system (1) performs image preprocessing to generate an impact map (42).

[0048] As illustrated in FIG. 4, an impact map according to a disclosed embodiment may be generated in the form of circles using RGB channels representing spatiotemporal information of an object at each trajectory. The impact map generates a plurality of circles (hereinafter referred to as "first circles") whose size and color brightness are determined based on the ratio of distances between the trajectories of each object, calculates the probability of an accident occurring based on the position of the bounding box of each object, generates another circle (hereinafter referred to as "second circle") based on the calculated probability value, and superimposes the trajectories of the objects onto each circle to generate a final impact map. A detailed description of the impact map is further explained through other drawings below.

[0049] The impact map generated by the impact map generator (31) contains information about the spatiotemporal dimensions of the object, and the impact map corresponding to the CCTV frame is input to the accident judgment device (32).

[0050] The accident judgment device (32) determines whether there is an accident based on a CNN-based algorithm (43).

[0051] The accident determiner (32) according to the disclosed embodiment basically determines whether the input impact map contains a traffic accident or not using 0 and 1. Since the accident determiner (32) matches the CCTV frame corresponding to the impact map, if the input impact map contains an accident, it can also output to the user whether the frame corresponding to the impact map contains an accident. As shown in the example illustrated in FIG. 4, the accident determiner (32) can determine that a video of two objects, such as a passenger car and a truck, colliding with each other is a frame containing an accident, and determine that a frame before the two objects collide with each other is a frame not containing an accident.

[0052] Meanwhile, in addition to the result of determining whether an accident occurred in a dichotomous manner as 0 or 1, the traffic accident detection system (1) may also provide the user with various information such as the location of the accident or the speed of an object within the CCTV frame by performing various user interface processing in addition to information including whether an accident occurred through a display.

[0053] FIG. 5 is a flowchart for specifically explaining the operation of an impact map generator according to a disclosed embodiment. FIG. 6 is an image of an example of a generated impact map.

[0054] The impact map is generated based on an image displaying a trajectory generated by a trajectory extractor (22) and object trajectories (2D Object Trajectories). In the past, if an image containing a general trajectory was used as is in a CNN-based Traffic Accident Severity Prediction (TASP), there was a problem in that it was unclear whether a detection was made due to deviation from the trajectory caused by a collision. Therefore, the disclosed traffic accident detection system (1) generates an impact map containing spatiotemporal information and can perform accident differentiation with high accuracy through this.

[0055] Referring to FIG. 5, the influence map generator (31) generates a plurality of circles (hereinafter referred to as first circles) with different sizes and brightness of color based on the ratio of distances between object orbits. Here, the first circles indicated by the notation of the influence map are defined by Equation 1.

[0056]

[0057] Here, r c is the radius of the first circle, and r min is the width (w) of the image with the trajectory displayed, and the object trajectory (2D Object Trajectories). i ) or height(h i It is the larger value among ).

[0058] The distance ratio of the first circle can be adjusted based on the size of the object trajectories (2D Object Trajectories) to calculate the overlapping area between objects. A large distance ratio indicates a large overlapping area between objects, and the color of the first circle becomes darker. Conversely, a low distance ratio means a small overlapping area between objects, and the color of the first circle becomes lighter. The distance ratio and the color of the first circle can be determined through the following mathematical formulas 2 and 3.

[0059]

[0060]

[0061] Here, x is the distance ratio, and d min is the distance when the distance between objects is closest, c is the degree of color change, and T max is the maximum time at which the orbit coordinate exists, and is the time at which the current orbit coordinate is located. The time can be calculated as the time at which the frame is located in the image.

[0062] As can be seen from the aforementioned mathematical formula, the size and color intensity of the first circle are determined based on the distance ratio between the trajectories of objects displayed in the 2D object trajectories. Additionally, the position of the first circle is determined based on the location of the object recognized by the object recognition algorithm. As the distance ratio between each object decreases, the radius of the generated first circle increases, and the color becomes darker.

[0063] When the first circle is generated, the impact map generator (31) generates a circle (hereinafter the second circle) that represents a notation considering the probability of an accident occurring. The second circle can be generated through Equation 4.

[0064]

[0065] Here, y is the area of ​​the bounding box used for object recognition, and w c wah h c is the size of the bounding box. That is, the second circle is determined based on the area and size of the bounding box, and the area of ​​the second circle can represent the probability of an accident occurring.

[0066] As can be seen from the aforementioned mathematical formula, the size and color intensity of the second circle are determined based on the position (coordinates) that the object's trajectory occupies in the object tracking diagram. In other words, as the size of the object increases, the size of the second circle increases. Furthermore, since the position occupied by the object's trajectory serves as a value that distinguishes accidents, an increase in the size of the second circle, which indicates the accident location, signifies a higher probability of an accident.

[0067] When the impact map generator (31) generates up to the second circle, it generates the trajectory of each object that can be identified from each frame on the impact map. That is, as shown in FIG. 5, the impact map generator (31) generates a line representing the object's trajectory by connecting the first circle positions (connecting the coordinates of the trajectory corresponding to the first circle) to generate a final impact map. Referring to FIG. 6, the impact map generated according to one example can be generated using different RGB notations. In FIG. 6, the red line marked in a Y shape represents the trajectory of the first vehicle and the second vehicle, and as the distance between the objects becomes closer, the blue first circle becomes darker and larger in size. The second circle, which indicates the possibility of an accident, increases as the green area in FIG. 6 becomes darker, and this is determined based on the size of the bounding box.

[0068] Figure 7 is a table explaining the embodiment of Figure 6 by comparing it with an actual CCTV frame.

[0069] Referring to FIG. 7, the impact map of FIG. 6 can be generated by matching each CCTV frame. Specifically, 50 frames can be input to the traffic accident detection system (1), and these are input in chronological order.

[0070] The traffic accident detection system (1) can generate an image with a trajectory displayed in the tracking stage, an object trajectory (2D Object Trajectories), corresponding to each frame as shown in FIG. 7. The traffic accident detection system (1) can generate an impact map corresponding to the CCTV frame as shown in FIG. 7 in the impact map generator (31), and the impact map of FIG. 6 is an impact map in which each circle and corrected trajectory are displayed based on the distance ratio between objects and the size of the bounding box of each object. That is, since the impact map according to the disclosed embodiment includes information containing the possibility of an accident based on the speed information and distance information of each object, the CNN-based algorithm can determine whether the CCTV frame is an accident video or not based on much higher accuracy, and as can be seen in FIG. 7, based on the 25th frame, the traffic accident detection system (1) distinguishes subsequent frames as frames containing a traffic accident.

[0071] Figure 8 is a table for describing parameters for training an algorithm included in the disclosed traffic accident detection system.

[0072] The disclosed traffic accident detection system (1) showed excellent performance after training the algorithm using various training parameters (hyperparameters) as shown in the table of FIG. 8.

[0073] Specifically, the parameters of the CCTV frame used in the performance test are the image size (Weight, Height) and the dimension (Dim) is 2. The object trajectories (2D Object Trajectories) are intermediate results generated by the trajectory extractor (22) of the tracking phase, and are in the same image format as the CCTV frame. Therefore, the object trajectories also have the image size (Weight, Height) adjusted as a parameter, and the dimension (Dim) is 2.

[0074] The influence map is the result of the influence map generator (31), which is the result of image preprocessing of the object tracking map. The influence map is set to be uniformly cropped to 224x224 pixels, and the dimension is 3.

[0075] The batch size of the artificial intelligence used for performance evaluation of the traffic accident detection system (1) according to the disclosed embodiment is 64, and the learning rate is 1x10 -5 The epoch of the entire dataset (Set) that the artificial intelligence learns in the traffic accident detection system (1) according to the disclosed embodiment was set to 15, and the Steps Per Epoch was set to 94 to conduct a performance test.

[0076] In the performance evaluation of the traffic accident detection system (1), Stochastic Gradient Descent (SGD) was used as the optimization algorithm (Optimizer), and Softmax was used as the objective function.

[0077] Figure 9 is a table for explaining the specifications of the data set of the disclosed traffic accident detection system.

[0078] For the performance evaluation mentioned in FIG. 8, the traffic accident detection system (1) according to the disclosed embodiment used a CADP (Car Accident Detection and Prediction) dataset. The CADP dataset used in this embodiment consisted of 1,416 CCTV segments, of which 150 CCTV segments were preprocessed, and each CCTV segment consisted of 50 frames. That is, the total number of CCTV frames input to the performance evaluation of the disclosed traffic accident detection system (1) was 7,500, and 7,500 impact maps were generated. Of the 7,500 generated impact maps, 6,750 (90%) were used as training data for the accident judgment unit (32), and 750 (10%) of the impact maps were used as valid data (Validation data).

[0079] The CADP dataset described in the table of Figure 9 contains CCTV frames for various traffic situations, including CCTV frames taken during the day and at night, CCTV frames for various situations involving snow or rain, CCTV frames with traffic congestion, and CCTV frames with various resolutions.

[0080] FIGS. 10 and 11 are tables to show the results of a performance evaluation of a disclosed traffic accident detection system.

[0081] Referring first to FIG. 10, the graph of FIG. 10 shows the training loss and validation loss of the disclosed traffic accident detection system (1), with the X-axis representing the epoch and the Y-axis representing the loss.

[0082] Specifically, the training loss of the disclosed traffic accident detection system (1) showed an initial value of 2.88, and after 4 epochs it transitioned to decreasing, finally recording a training loss of 0.06. The effective result loss also showed an initial value of 3.41, then decreased after 4 epochs and finally recorded 0.12.

[0083] Referring to FIG. 11, the graph of FIG. 11 shows the training accuracy and validation accuracy of the disclosed traffic accident detection system (1), with the X-axis representing the epoch and the Y-axis representing the accuracy.

[0084] Specifically, the training accuracy of the disclosed traffic accident detection system (1) showed an initial value of 55.63, then transitioned to increasing after 2 epochs, and finally recorded 97.13. The valid result accuracy started with an initial value of 77.87 and increased to 95.87 through continuous training.

[0085] It can be confirmed that the traffic accident detection method disclosed in FIGS. 10 and FIGS. 11 effectively extracts features of an impact map and smoothly performs the detection of whether a traffic accident has occurred.

[0086] Figures 12 and 13 are graphs for comparing the performance of a conventional general algorithm and a disclosed traffic accident detection system.

[0087] Specifically, FIGS. 12 and 13 are the results of a comparative experiment determining whether a traffic accident is detected in a CCTV frame using a conventional general CNN-based algorithm (hereinafter referred to as the conventional algorithm), using the same indicators as FIGS. 10 and 11.

[0088] As can be seen in Figure 12, the conventional algorithm shows that the training loss drops sharply from 6.43 to 0.86 before the 5th epoch, and then slowly decreases to 0.37. In addition, the valid result loss is similar to the training loss, showing 6.56 at the beginning of training, and then transitioning to 0.45 after 50 epochs.

[0089] As can be seen in Figure 13, the training accuracy of the conventional algorithm also increases rapidly from 4.67 to 55.4 four epochs before, and then increases slowly to 95.2. The valid result accuracy also increases rapidly from 0.75 to 94.38 after 50 epochs.

[0090] As can be seen in FIGS. 12 and 13, the disclosed traffic accident detection system can be seen to have lower training loss and faster conversion to valid result data compared to a conventional CNN-based algorithm, and the disclosed traffic accident detection system (1) trained for 15 epochs has an accuracy of about 1.93 higher and a conversion rate of 0.31 lower than a conventional algorithm trained for 50 epochs.

[0091] The disclosed traffic accident detection system has superior training efficiency and accuracy in determining traffic accidents from CCTV frames compared to conventional general CNN-based algorithms.

[0092] Figure 14 is a graph comparing methods for determining whether an accident occurred through approaches other than the disclosed impact map.

[0093] An experiment was conducted to compare the object bounding box, object trajectory data type, optical flow, and attention map in addition to the impact map generated by the disclosed traffic accident detection system (1).

[0094] As can be seen in FIG. 14, the disclosed traffic accident detection system (1) showed training accuracy and valid result accuracy of 97.3 and 95.9, respectively, while using only object bounding boxes, it showed accuracy of 96.4 and 91.7, and using object tracking data, it showed accuracy of 93.8 and 93.0. When determining whether there was a traffic accident using an optical flowchart, it recorded low accuracy of 67.0 and 65.9, and when using an attention map, it recorded accuracy of 86.6 and 84.3.

[0095] That is, the disclosed traffic accident detection system (1) can determine whether a traffic accident has occurred in a video frame with higher accuracy and training speed than other algorithms and other types of image preprocessing by generating an influence map based on the distance ratio and the size of the bounding box based on the trajectory of an object recognized in a video frame. In addition, the traffic accident detection system (1) overcomes the low accuracy and training speed shown by conventional general artificial intelligence and can distinguish even accidents that could not be distinguished in CCTV frames. Explanation of the symbols

[0096] 1: Traffic accident detection system 10: Process 11: Memory 12: Display 21: Bounding Box Generator 22: Orbit Extractor 31: Impact Map Generator 32: Incident Judge

Claims

Claim 1 A method for detecting a traffic accident using a traffic accident detection system comprises: a step of generating bounding boxes for objects included in a plurality of frames captured by a CCTV (Closed-Circuit Television) based on an object recognition algorithm; a step of extracting the trajectory of an object included in the bounding box based on a SORT (Simple Online and Realtime Tracking) algorithm; a step of generating an influence map representing the relationship between the trajectories of the object by adding a notation including at least one of a symbol, a label, text, or a color to the extracted trajectory of the object; and a step of inputting the influence map as an input value to a CNN (Convolutional Neural Network) based algorithm and determining whether the CCTV frame is an accident video based on the output value of the CNN based algorithm; wherein the step of generating the influence map comprises a step of generating a plurality of first circles whose size and color intensity are determined based on the distance ratio between the trajectory of a first object and the trajectory of a second object; and the step of generating the plurality of first circles further comprises a step of determining the position of the plurality of first circles based on the position of the object. A method for detecting traffic accidents, comprising: a step of generating an impact map, wherein the step of generating an object tracking map displays a line indicating the trajectory of the object by connecting the positions of the plurality of first circles; a step of adjusting the distance ratio between the trajectory of the first object and the trajectory of the second object based on the size of the object tracking map; and a step of generating a second circle indicating the possibility of an accident based on the position occupied by the trajectory of the object in the object tracking map and the size of the object. Claim 2 delete Claim 3 delete Claim 4 A traffic accident detection method according to claim 1, wherein the step of generating the influence map further comprises the step of generating the second circle, the size and brightness of which are determined based on the position occupied by the orbit of the object. Claim 5 delete Claim 6 A traffic accident detection method according to claim 1, wherein the object recognition algorithm includes a YOLO (You Only Look Once) algorithm and the SORT algorithm includes a Deep SORT algorithm. Claim 7 A traffic accident detection method according to claim 1, characterized in that when the distance ratio between the first object and the second object decreases, the radius of the first circle generated increases and the color of the first circle becomes darker. Claim 8 In claim 4, the step of generating the influence map is a traffic accident detection method in which, as the size of the object increases, the size of the second circle increases, and the increase in the size of the second circle indicates that the probability of the accident increases. Claim 9 The apparatus includes a processor for pre-training artificial intelligence, a memory for storing data and said trained artificial intelligence, and a display for outputting a CCTV frame, wherein the processor comprises: a bounding box generator that receives a CCTV frame input from a user and generates a bounding box for an object included in said CCTV frame based on an object recognition algorithm; an orbit extractor that extracts the orbit of an object included in said bounding box based on a SORT algorithm; and an influence map generator that generates an influence map indicating the relationship between the orbits of said objects by adding a notation including at least one of a symbol, a label, text, or a color to the orbit of said extracted object. The traffic accident detection system includes: an accident determination unit that inputs the impact map as an input value to a CNN-based algorithm and determines whether the CCTV frame is an accident video based on the output value of the CNN-based algorithm; wherein the impact map generator generates a plurality of first circles whose size and color brightness are determined based on the distance ratio between the orbit of a first object and the orbit of a second object, determines the positions of the plurality of first circles based on the position of the object, generates an object tracking map that displays a line representing the orbit of the object by connecting the positions of the plurality of first circles, adjusts the distance ratio between the orbit of the first object and the orbit of the second object based on the size of the object tracking map, and generates a second circle that indicates the possibility of an accident based on the position occupied by the orbit of the object in the object tracking map and the size of the object. Claim 10 delete Claim 11 In claim 9, the above-described impact map generator generates a value distinguishing an accident based on the position occupied by the orbit of the object, and generates the second circle indicating the point of occurrence of the accident based on the generated value, thereby creating a traffic accident detection system. Claim 12 delete Claim 13 In claim 9, the traffic accident detection system is characterized in that, when the distance ratio between the first object and the second object decreases, the radius of the first circle generated increases and the color of the first circle becomes darker. Claim 14 In claim 11, the traffic accident detection system is characterized in that the influence map generator increases the size of the second circle when the size of the object increases, and the increase in the size of the second circle increases the probability of the accident.

Images