Image storage device for vehicle and operating method thereof

Abstract

Disclosed is an image storage device that is mountable on a vehicle and captures and stores images around the vehicle. The image storage device according to the present disclosure comprises: an imaging unit for imaging and acquiring a real-time driving image of a vehicle; an accident prediction unit for predicting the possibility of an accident of the vehicle by analyzing the real-time driving image by using a trained accident detection artificial intelligence model; a storage unit for, when it is determined that there is the possibility of an accident, storing the driving image for a prescribed time including before and after an accident detection time point based on the accident detection time point; and a communication unit for transmitting the driving image for the prescribed time to a fault ratio determination server, wherein the accident detection artificial intelligence model is trained in a supervised learning method by using training data labeled with an accident image and a general driving image.

Description

Vehicle video storage device and its operating method

[0001] The present disclosure relates to a black box for a vehicle, and more particularly, to a vehicle video storage device having an on-device artificial intelligence model and using the artificial intelligence model to predict the occurrence of an accident from real-time driving video and store the accident video.

[0002] In addition, the present disclosure relates to a fault ratio judgment system that transmits accident-related video captured by a vehicle black box to a server and determines the fault ratio from the accident video on the server.

[0003] A vehicle black box is a device attached to a vehicle that records and records accidents or situations that occur while driving. It is used to analyze the cause of an accident and determine responsibility. A vehicle black box is also known as a dashboard camera or dashcam.

[0004] Vehicle black boxes are used to analyze the cause of accidents and determine responsibility. However, they can store unnecessary footage, rather than footage directly related to the accident, which can lead to memory or battery shortages and prevent the acquisition of necessary footage in the event of an accident. For example, conventional black boxes record footage based on motion or impact detection, but the motion or impact detected by the black box is often unrelated to the actual vehicle accident. Furthermore, if the black box's built-in battery fails, the stored footage may be reset or not saved, making it difficult to capture relevant footage in the event of an accident. Furthermore, depending on the black box's storage capacity, previously saved footage may be overwritten when storage capacity is insufficient, preventing the acquisition of accident-related footage.

[0005] In addition, due to the complexity of the process of extracting and transmitting accident videos stored in memory, drivers often cannot submit the original videos when submitting video data to insurance companies, but instead film the black box screen with their personal smartphones and submit it. Videos taken through such indirect filming hinder the accuracy of fault judgment during the insurance processing.

[0006] The present disclosure provides a vehicle video storage device that stores related driving videos when predicting the occurrence of a traffic accident using an accident prediction artificial intelligence model.

[0007] In addition, the present disclosure provides a system that transmits accident videos captured by a vehicle video storage device to an external server so that users or interested parties can share the accident videos.

[0008] In addition, the present disclosure provides a method and device capable of judging the fault ratio between related objects from an accident video using an artificial intelligence model for judging the fault ratio.

[0009] A vehicle video storage device according to one embodiment of the present disclosure includes a shooting unit that captures and acquires real-time driving video of a vehicle, an accident prediction unit that analyzes the real-time driving video using a pre-trained accident detection artificial intelligence model to predict the possibility of an accident occurring in the vehicle, a storage unit that stores driving video for a predetermined period of time including before and after an accident detection time centered on an accident detection time when the possibility of an accident occurring is determined to exist, and a communication unit that transmits the driving video for the predetermined period of time to a fault ratio determination server, wherein the accident detection artificial intelligence model is a model trained by a supervised learning method using training data labeled with accident videos and general driving videos.

[0010] According to embodiments of the present disclosure, the artificial intelligence accident detection model stores images only when a traffic accident occurs, so that indiscriminate storage does not occur, and there is no need to overwrite existing images. Furthermore, the images can be safely stored without affecting the limited capacity and lifespan of the storage space and without the risk of image loss.

[0011] In addition, according to embodiments of the present disclosure, personal information included in accident videos can be masked by transmitting the stored videos through a connection server, the fault ratio between accident vehicles can be determined, and data can be provided in the form of a web or app, including insurance filing, through the platform upon user request.

[0012] FIG. 1 is a diagram showing an entire system for a vehicle video storage device and a vehicle black box that shares video captured by the vehicle video storage device and analyzes accident video to determine the percentage of fault according to one embodiment of the present disclosure.

[0013] FIG. 2 is a block diagram showing the configuration of a vehicle image storage device according to one embodiment of the present disclosure.

[0014] FIG. 3 is a diagram illustrating operations between a vehicle image storage device (310), a server (320), and a user terminal (330) according to one embodiment of the present disclosure.

[0015] FIG. 4 is a diagram illustrating operations between a vehicle image storage device (410), a server (420), and a user terminal (430) according to another embodiment of the present disclosure.

[0016] FIG. 5 is a flowchart illustrating an operation method of a vehicle image storage device according to one embodiment of the present disclosure.

[0017] FIG. 6 is a flowchart illustrating an operation method of a vehicle image storage device according to another embodiment of the present disclosure.

[0018] A vehicle video storage device according to one embodiment of the present disclosure includes a shooting unit that captures and acquires real-time driving video of a vehicle, an accident prediction unit that analyzes the real-time driving video using a pre-trained accident detection artificial intelligence model to predict the possibility of an accident occurring in the vehicle, a storage unit that stores driving video for a predetermined period of time including before and after an accident detection time centered on an accident detection time when the possibility of an accident occurring is determined to exist, and a communication unit that transmits the driving video for the predetermined period of time to a fault ratio determination server, wherein the accident detection artificial intelligence model is a model trained by a supervised learning method using training data labeled with accident videos and general driving videos.

[0019] In one embodiment, the accident detection artificial intelligence model can extract and analyze features of a driving video for each frame of the real-time driving video or for a predetermined number of frame groups to predict the possibility of an accident occurring.

[0020] In one embodiment, the accident detection artificial intelligence model may be a model learned through transfer learning that fine-tunes the weights of a pre-learned model using the accident video and general driving video.

[0021] In one embodiment, the vehicle video storage device further includes a sensor unit including at least one sensor among a gyro sensor, an acceleration sensor, an impact sensor, a GPS, and a lidar sensor, and the accident prediction unit can determine that there is a possibility of an accident occurring for the vehicle when a combined index obtained by combining sensor information acquired through at least one sensor included in the sensor unit and accident occurrence possibility information determined by the accident detection artificial intelligence model is greater than a set threshold value.

[0022] In one embodiment, when the accident occurrence probability information determined by the accident detection artificial intelligence model is p (where p is a real number between 0 and 1), the distance between the vehicle and surrounding objects measured through the lidar sensor is D, α, β are predetermined weights, and ε is a predetermined constant value, the coupling index (C) is expressed by the following mathematical formula: Accordingly, the probability of an accident occurring is determined to be proportional to the probability of an accident occurring as determined by the accident detection artificial intelligence model and inversely proportional to the distance between the vehicle and the surrounding object, and the accident prediction unit can determine that there is a probability of an accident occurring between the vehicle and the surrounding object when the combination index (C) is greater than or equal to the set threshold value.

[0023] In one embodiment, the fault ratio judgment server can identify a key object for predicting the fault ratio from the driving video using a pre-learned object recognition artificial intelligence model, and can judge the fault ratio between the vehicle and the key object from a driving video for a predetermined period of time transmitted from the video storage device using a pre-learned fault ratio judgment artificial intelligence model.

[0024] In one embodiment, the object recognition artificial intelligence model extracts spatial-temporal features from the driving video for the predetermined period of time on a frame-by-frame basis and tracks and analyzes the motion of the object and the change of the scene on a frame-by-frame basis, and the fault ratio judgment artificial intelligence model may be a model trained by supervised learning or unsupervised learning using training data labeled based on the basis of the basic fault ratio classified by the fault ratio information portal using the driving video preprocessed by the accident detection artificial intelligence model.

[0025] An operating method of an image storage device according to one embodiment of the present disclosure includes the steps of capturing and acquiring a real-time driving image of the vehicle, analyzing the real-time driving image using a pre-trained accident detection artificial intelligence model to predict the possibility of an accident occurring of the vehicle, storing the driving image for a predetermined period of time including before and after the accident detection time centered on the accident detection time, and transmitting the driving image for the predetermined period of time to a fault ratio determination server, wherein the accident detection artificial intelligence model may be a model trained by a supervised learning method using training data labeling the accident image and the general driving image.

[0026] According to another embodiment of the present disclosure, an image storage device that can be mounted on a first vehicle and captures and stores an image of the vehicle's surroundings includes a capturing unit that captures and acquires a real-time driving image of the first vehicle, an accident prediction unit that analyzes the real-time driving image using a pre-learned accident detection artificial intelligence model to predict the possibility of an accident occurring in a second vehicle surrounding the first vehicle, a storage unit that stores a driving image for a predetermined period of time including before and after an accident detection time centered on an accident detection time when the possibility of an accident occurring is determined to be greater than a predetermined threshold value, and a communication unit that transmits the driving image for the predetermined period of time to a fault ratio judgment server and / or a terminal of a user of the second vehicle, wherein the accident detection artificial intelligence model may be a model trained by a supervised learning method using training data labeled with an accident image and a general driving image.

[0027] According to another embodiment of the present disclosure, a method for operating an image storage device that can be mounted on a first vehicle and captures and stores an image of the vehicle's surroundings includes the steps of capturing and acquiring a real-time driving image of the first vehicle, analyzing the real-time driving image using a pre-trained accident detection artificial intelligence model to predict the possibility of an accident occurring in a second vehicle surrounding the first vehicle, storing a driving image for a predetermined period of time including before and after an accident detection time centered on an accident detection time point, and transmitting the driving image for the predetermined period of time to a fault ratio determination server and / or a terminal of a user of the second vehicle, wherein the accident detection artificial intelligence model may be a model trained by a supervised learning method using training data labeled with an accident image and a general driving image.

[0028]

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the attached drawings. However, the technical idea of ​​the present disclosure is not limited to the following embodiments and may be implemented in various different forms. The following embodiments are provided only to complete the technical idea of ​​the present disclosure and to fully inform those skilled in the art of the present disclosure of the scope of the present disclosure, and the technical idea of ​​the present disclosure is defined only by the scope of the claims.

[0030] When assigning reference numerals to components in each drawing, it should be noted that identical components are assigned the same numerals whenever possible, even if they appear on different drawings. Furthermore, when describing the present disclosure, if a detailed description of a related known configuration or function is deemed likely to obscure the gist of the present disclosure, such detailed description will be omitted.

[0031] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the same sense as commonly understood by those of ordinary skill in the art to which this disclosure pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. The terminology used herein is for the purpose of describing embodiments and is not intended to limit the disclosure. In this specification, singular forms also include plural forms, unless specifically stated otherwise.

[0032] Additionally, terms such as first, second, A, B, (a), (b), etc. may be used to describe components of the present disclosure. These terms are only intended to distinguish the components from other components, and the nature, order, or sequence of the components are not limited by the terms. When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected or connected to the other component, but another component may also be "connected," "coupled," or "connected" between each component.

[0033] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0034] FIG. 1 is a diagram showing an entire system for a vehicle video storage device and a vehicle black box that shares video captured by the vehicle video storage device and analyzes accident video to determine the percentage of fault according to one embodiment of the present disclosure.

[0035] A system according to one embodiment of the present disclosure includes a vehicle image storage device (110), a user terminal (120), and an artificial intelligence server (130). The vehicle image storage device (110) analyzes real-time driving images using an accident detection artificial intelligence model installed on-device to predict the possibility of an accident. In addition to information on the possibility of an accident, which is an output value of the accident detection artificial intelligence model, the vehicle image storage device (110) obtains distance information with respect to surrounding objects using a lidar sensor, etc., and can predict the possibility of an accident between a vehicle in which the vehicle image storage device (110) is installed and surrounding objects based on the information on the possibility of an accident output by the accident detection artificial intelligence model through analysis of the driving images and the distance information with respect to surrounding objects.

[0036] When an accident is predicted, the vehicle image storage device (110) can transmit an accident image including images before and after the accident to a user terminal (120) or an external server. The external server may be an artificial intelligence server (130). The user can receive the accident image directly from the vehicle image storage device (110) or can receive the accident image from an external server through an application of the user terminal that is linked to the external server. In addition, the vehicle image storage device (110) transmits an alarm to the user terminal (120) when an accident occurs, and the user who receives the alarm signal can access the vehicle image storage device (110) through the user terminal (120) to receive and check the accident image. This accident image can be checked through an application installed on the user terminal (120), etc. Alternatively, the user may access the application from the user terminal (120), access the vehicle video storage device (110) or the external server where the accident video is stored, request transmission of the accident video from the vehicle video storage device (110) or the external server, receive the accident video, and check the accident video. The user terminal (120) may be a digital device equipped with a processor, memory, and computing power, such as a laptop computer, a notebook computer, a desktop computer, a web pad, or a mobile phone.

[0037] A user can communicate with a vehicle video storage device (110) through a platform provided by an application installed on a user terminal (120). The user can access the vehicle video storage device (110) through the user terminal (120) and configure the storage space, deletion, transmission, etc. of the vehicle video storage device (110). If the user is curious about the extent of his / her own fault, he / she can transmit a signal requesting a fault ratio determination to an artificial intelligence server (130) through the user terminal (120) and receive the fault ratio determination result from the artificial intelligence server (130).

[0038] The vehicle video storage device (110) can directly transmit the accident video to an artificial intelligence server (130) for fault determination. Alternatively, the user can check the accident video through a user terminal (120) and then request a fault ratio determination while transmitting the accident video to the artificial intelligence server (130) through the user terminal (120).

[0039] The artificial intelligence server (130) can analyze an accident video transmitted from a vehicle video storage device (110) or a user terminal (120), determine the fault ratio for the accident, and transmit the fault ratio determination result information to a user terminal (120) or a vehicle video storage device (110) linked to the platform. The artificial intelligence server (130) is a model that stores and analyzes the received accident video in a database to predict fault ratio information, and can determine the fault ratio between vehicles by applying traffic accident types, previous precedents, etc. to traffic accidents that occurred in traffic accident video data.

[0040] The artificial intelligence server (130) can identify key objects for predicting fault ratio from a driving video using a pre-trained object recognition artificial intelligence model, and can determine the fault ratio between a vehicle and key objects from a driving video transmitted from a video storage device for a predetermined period of time using a pre-trained fault ratio judgment artificial intelligence model. The artificial intelligence server (130) can determine and output the fault ratio between the current vehicle and surrounding objects involved in the accident based on the fault ratio judgment criteria defined in the "Fault Ratio Information Portal" during learning for fault ratio prediction.

[0041] The artificial intelligence server (130) transmits the received traffic accident video to the linked insurance company, and can support the driver to file a traffic accident insurance claim without a separate wired connection process.

[0042] FIG. 2 is a block diagram illustrating the configuration of a vehicle video storage device according to one embodiment of the present disclosure. The vehicle video storage device (200) may be referred to as a vehicle black box device.

[0043] A vehicle video storage device (200) according to one embodiment of the present disclosure can predict the possibility of an accident using an on-device artificial intelligence model from real-time driving video captured by a vehicle black box device, and if an accident is predicted, can store videos including before and after the accident, and can transmit accident-related videos to an external server. The external server can be an insurance company's server preset by a user of the vehicle video storage device (200). If an accident is predicted as a result of analyzing the real-time driving video, the vehicle video storage device (200) can transmit accident-related videos to the insurance company's server, and can also transmit a signal requesting the insurance company's server to accept traffic accident processing.

[0044] Referring to FIG. 2, a vehicle video storage device (200) may include a camera unit (210), a microphone (220), a sensor unit (230), an input unit (240), a control unit (250), an accident prediction unit (260), an output unit (270), a communication unit (280), and a storage unit (290).

[0045] The photographing unit (210) is at least one camera provided on at least one side of the vehicle image storage device (200), and can capture images of the front, rear, or side of the vehicle depending on the installation direction of the camera and transmit the captured images to the control unit (250). The photographing unit (210) may further include an image sensor for acquiring images in a low-light environment. The image sensor is a device that detects the intensity and color of an optical image and converts it into digital image data, and may include a CCD (Charge Coupled Device) and a CMOS image sensor.

[0046] The image captured by the camera unit (210) may be stored after undergoing a preprocessing process in the control unit (250), or may be stored as raw data in the storage unit (290). The preprocessing process may include a process of compressing the captured image, or a process of converting the RGB 3-channel image input in frame units into a Lab color space and then adjusting the contrast ratio through histogram stretching or histogram equalization to obtain a balanced image. In addition, the preprocessing process may include a process of making the captured image brighter through gamma correction. The quality of the image captured in a low-light environment can be improved through the preprocessing process.

[0047] The microphone (220) can record voices or sounds from the outside or inside of the vehicle and transmit them to the control unit (250). The voices or sounds acquired from the microphone (220) can be combined with images captured by the camera unit (210) after going through a compression process and stored in the storage unit (290), or can be combined with images without a compression process and stored directly in the storage unit (290). The voices or sounds recorded through the microphone (220) can be used to analyze the situation at the time of an accident.

[0048] The sensor unit (230) may include at least one sensor among a gyro sensor, an acceleration sensor, a shock sensor, a GPS, and a lidar sensor. The gyro sensor and the acceleration sensor are used to measure the angular velocity and acceleration of the vehicle in which the vehicle image storage device (200) is installed, respectively. The shock sensor detects an impact applied to the vehicle, and instead of providing a separate impact sensor, if the amount of change in the angular velocity and acceleration detected by the aforementioned gyro sensor and acceleration sensor is greater than a predetermined threshold value, it may be determined that an impact is applied to the vehicle. The GPS (Global Position System) continuously calculates the current location of the vehicle in real time and can use this to calculate vehicle speed information.

[0049] The input unit (240) is a unit that receives an input signal for the user to control the operation of the vehicle image storage device (200), and may be configured with keys or buttons for controlling the functions of the vehicle image storage device (200). The input unit (240) may be formed in the form of a touch screen provided in the display unit of the output unit (270).

[0050] The control unit (250) controls the overall operation of the vehicle storage device (200). If an accident is predicted based on the probability of an accident determined by the accident prediction unit (260), the control unit (250) can control the storage of accident-related images captured by the shooting unit (210) in the storage unit (290) or the transmission of accident-related images to an external server via the communication unit (280).

[0051] The accident prediction unit (260) includes an on-device accident detection artificial intelligence model (261) and can predict the possibility of an accident occurring in a vehicle equipped with a vehicle image storage device (200) by analyzing real-time driving images acquired through the shooting unit (210). In addition, the accident prediction unit (260) can predict the possibility of an accident occurring between surrounding objects other than the vehicle equipped with the vehicle image storage device (200) by analyzing real-time driving images acquired through the shooting unit (210).

[0052] The accident detection artificial intelligence model (261) may be a model trained using a supervised learning method using training data labeled with accident videos and general driving videos that occur in various environments. The accident detection artificial intelligence model (261) may be trained using a transfer learning method, and may apply a learning method that fine-tunes the model by additionally learning the feature information of accident videos and general videos generated while learning the training data labeled with a pre-trained model and the weights of the pre-trained model.

[0053] The accident prediction unit (260) can predict the possibility of an accident from real-time driving video using the pre-learned accident detection artificial intelligence model (261). The real-time driving video can be input to the input node of the accident detection artificial intelligence model (261), and the possibility of an accident can be output to the output node. For example, the accident detection artificial intelligence model (261) can output the possibility of an accident as a binary value such as "accident occurred" or "accident did not occur," or as a probability value indicating the possibility of an accident occurring between 0 and 1.

[0054] The accident detection artificial intelligence model (261) may include multiple layers, and weights may be set for edges connecting nodes of each layer. The weights of the nodes and edges arranged between the input node and the output node may be updated during the learning process. Specifically, the accident detection artificial intelligence model (261) may learn the possibility of an accident in a supervised manner based on real-time driving video data by utilizing deep learning technology, which is a type of machine learning. The accident detection artificial intelligence model (261) may utilize various known deep neural network networks (DNNs). The deep neural network networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), graph neural networks (GNNs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), Q networks, U networks, Siamese networks, etc. The description of the deep neural network described above is only an example and the present disclosure is not limited thereto.

[0055] The learning of the accident detection artificial intelligence model (261) can be performed by adjusting the weights between nodes so that the desired output is generated for the input. Methods such as back propagation and gradient descent can be used for learning the artificial neural network. The weights of the accident detection artificial intelligence model (261) are adjusted in the direction of minimizing the loss function, and the weights can be optimized using a loss function such as focal loss when the distribution of each class of data is unbalanced, and binary cross entropy loss when the distribution of the classes is balanced. The accident detection artificial intelligence model (261) adjusts the weights in the direction of minimizing the loss function through back propagation and gradient descent to classify "accident occurred" and "accident did not occur", and for example, the weights of the accident detection artificial intelligence model (261) can be optimized for accident detection through a gradient descent method such as the following mathematical expression 1.

[0056] [Mathematical Formula 1]

[0057]

[0058] : Previous weight

[0059] : Updated weights

[0060] η: Learning Rate, which controls the size of weight updates

[0061] : Slope of the weights with respect to the loss function

[0062] In this way, by adjusting the parameters of the deep neural network while performing supervised learning using accident videos and general driving videos as training data, a model optimized for accident detection that can predict information related to accident detection can be obtained.

[0063] As input data of an accident detection artificial intelligence model (261) according to one embodiment of the present disclosure, information on surrounding objects identified from a driving video, for example, a feature vector including the direction of movement of surrounding objects identified through analysis of each frame of the driving video, the speed of the identified surrounding objects, the distance between the identified surrounding objects and the current vehicle, etc. may be input. The accident detection artificial intelligence model (261) may receive a feature vector extracted from a driving video as input and output a result value of the possibility of an accident occurring as a binary value or a probability value between 0 and 1.

[0064] The accident detection artificial intelligence model (261) can analyze driving video frame by frame to determine which frame has the highest probability of an accident. The corresponding time of the frame with the highest probability of an accident can be estimated as the time of the accident. When predicting the occurrence of an accident, the accident detection artificial intelligence model (261) can group the input driving video into a predetermined number of frame groups and select a frame group with a high probability of an accident among the frame groups. For example, the accident detection artificial intelligence model (261) can set 10 sequentially input frames as one frame group and determine the frame group with the highest probability of an accident on a frame group basis. For example, when 30 frames are captured per second and 3 frame groups are set per second, the time of accident occurrence can be estimated in units of 1 / 3 (second). The probability of accident occurrence for each frame group can be calculated using the average value of the probability value of the accident occurrence for each of the multiple frames included in each frame group, or the probability value of the accident occurrence can be calculated on a frame group basis.

[0065] When the accident prediction unit (260) detects the risk of a traffic accident, it can store the video being recorded by the recording unit (210) in the storage unit (290). The accident-related video can be set as protected data that cannot be deleted, and can be managed so that it is deleted only through a predetermined deletion process after confirmation by the user, and the accident video can be protected from being deleted arbitrarily.

[0066] Meanwhile, the accident prediction unit (260) according to another embodiment of the present disclosure can determine the possibility of an accident by considering sensor information acquired through the sensor unit (230) in addition to the accident possibility information acquired through the accident detection artificial intelligence model (261) from the driving image acquired from the shooting unit (210). Specifically, the accident prediction unit (260) determines a combined index (C) by combining information on the distance between the current vehicle and surrounding objects acquired through the lidar sensor included in the sensor unit (230) and the accident possibility information acquired through the accident detection artificial intelligence model (261), and can determine that the possibility of an accident exists when the combined index (C) is equal to or greater than a set threshold value. For example, when the probability of an accident occurring determined by the accident detection artificial intelligence model (261) is p (where p is a real number between 0 and 1), the distance between the current vehicle and surrounding objects through the sensor unit (230) is D, α, β are predetermined weights, and ε is a predetermined constant value, the combination index (C) can be determined according to the following mathematical expression 2.

[0067] [Equation 2]

[0068]

[0069] The combination index (C) is a value determined to be proportional to the probability of an accident determined by the accident detection artificial intelligence model (261) and inversely proportional to the distance between the vehicle and surrounding objects. That is, the closer the accident occurrence prediction value of the accident detection artificial intelligence model (261) and the physical distance between the actual surrounding objects become, the greater the value of the combination index (C), which indicates the probability of an accident occurrence. The accident prediction unit (260) can compare the combination index (C) with a predetermined threshold value and determine an accident if the combination index (C) is greater than the predetermined threshold value. A threshold value that can cover most accidents among all accident videos can be determined through learning data of general driving videos and accident videos, and the accuracy of the accident occurrence prediction of the accident prediction unit (260) can be improved depending on the threshold value setting.

[0070] The output unit (270) may include a display unit and a speaker that display UX / UI information for controlling the vehicle image storage device (200) or output an image captured by the capture unit (210). Through the display, the user can check the image captured by the capture unit (210), and by supporting the user's touch input, the user can easily select and set a desired function. As described above, the user input may be input through the display unit when the display unit is provided in the form of a touch screen, or may be received through a separate input unit (240). The speaker may output an audio signal related to a function performed by the vehicle image storage device (200) or output a sound picked up through a microphone (220).

[0071] The communication unit (280) can transmit accident-related images captured by the camera unit (210) or sensor data acquired through the sensor unit (230) to an external server or user terminal when an accident is predicted based on the probability of an accident determined by the accident prediction unit (260). In addition, the communication unit (280) can perform communication to transmit and receive data with the user's terminal, surrounding vehicles, or RSU (Road Side Unit). The communication unit (280) can support wireless networks such as WIFI, Bluetooth, 3G, 4G (LTE), 5G, 6G, WIMAX, WIGI, or wired networks.

[0072] If the communication unit (280) is equipped with Wi-Fi and Bluetooth, the user can access and operate the vehicle video storage device (200) via wireless communication using an application installed on the user terminal. However, since these Wi-Fi and Bluetooth communication technologies only operate at close range, operation is only possible near the vehicle video storage device (200). The user can manage the black box via Wi-Fi and Bluetooth, and can quickly check and download videos without an Internet connection via Wi-Fi in the vicinity of the black box.

[0073] If the communication unit (280) is equipped with wireless communication technology capable of connecting to the Internet, the user can monitor and manage the video captured by the vehicle video storage device (200) in real time even from a distance. Through LPWAN technology such as NB-IoT, the user can access the data of the vehicle video storage device (200) anytime and anywhere regardless of the user's location, download and store important videos immediately, and receive immediate notifications in the event of an accident. As described above, the user can communicate with the vehicle video storage device (200) within the platform using an application installed on the user terminal.

[0074] The storage unit (290) can store the video captured by the shooting unit (210) and the audio data received through the microphone (220) in a single file or in separate file formats. In particular, the storage unit (290) can temporarily protect the video related to the accident from deletion when an accident is predicted based on the probability of an accident determined by the accident prediction unit (260), and can prevent the video related to the accident from being deleted due to a user's misoperation or mistake.

[0075] The storage unit (290) can save storage capacity by temporarily storing the video captured by the shooting unit (210) in a buffer or the like, and only storing the relevant accident video when an actual accident is predicted to occur.

[0076] The storage unit (290) may be configured within the vehicle image storage device (200) or may be configured to be removable through a port of a predetermined format provided in the vehicle image storage device (200). The storage unit (290) may be formed in the form of a hard disk drive, flash memory, or in a removable form such as an SD card, Micro SD card, or USB memory.

[0077] Meanwhile, the vehicle image storage device (200) according to one embodiment of the present disclosure can identify objects from the image when storing a driving image or an accident image, determine a human face or a car license plate among the identified objects for personal information protection, and apply masking to a portion of the object for which personal information is needed. Masking is a process of selecting and protecting or hiding a specific area in an image or video, and can be mainly used for personal information protection. The vehicle image storage device (200) can detect objects and apply masking using a CNN-based deep neural network when identifying objects in the image. Masking can be performed by blurring (Gaussian blurring and mosaicking) the portion of the identified object that needs to be masked. This masking process can be performed through the control unit of the vehicle image storage device (200) or a separately installed image processing module.

[0078] Meanwhile, the accident prediction unit (260) according to another embodiment of the present disclosure can predict the possibility of an accident occurring between surrounding objects identified in an input driving video even if an accident does not occur for the current vehicle equipped with the current vehicle video storage device (200). If an accident occurring between surrounding objects identified in the driving video is predicted, the accident prediction unit (260) can transmit an alarm to the surrounding objects through the communication unit (280) or transmit an accident video involving the photographed surrounding objects to the surrounding objects. In addition, the accident prediction unit (260) can transmit an accident video occurring between surrounding objects to an external server, transmit an alarm signal from the external server to the surrounding objects that are parties to the accident, or transmit the accident video to the surrounding objects from the external server.

[0079] FIG. 3 is a diagram illustrating operations between a vehicle image storage device (310), a server (320), and a user terminal (330) according to one embodiment of the present disclosure.

[0080] Referring to FIG. 3, the vehicle video storage device (310) analyzes real-time driving video using an accident detection artificial intelligence model to predict the possibility of an accident and detect an accident (S301). When an accident is detected, the vehicle video storage device (310) stores the accident video in the storage unit, and at the same time, notifies the fault ratio determination server (320) or the user terminal (330) of the occurrence of the accident and transmits the accident video (S302). When an accident is detected, the vehicle video storage device (310) may immediately transmit the accident video to the fault ratio determination server (320), or transmit the accident notification and the accident video to the user terminal (330) linked through the platform, and after the user checks the accident video stored in the user terminal (330) (S303), the accident video may be transmitted from the user terminal (330) to the fault ratio determination server (320) according to the user's selection. The user can check the accident video via the user terminal (330) and request the fault ratio determination server (320) to determine the fault ratio (S304). As described above, without the user having to go through the process of checking the accident video, the accident video is directly transmitted from the vehicle video storage device (310) to the fault ratio determination server (320), and the fault ratio determination server (320) can store the accident video and then determine the fault ratio.

[0081] When the fault ratio determination server (320) receives an accident video through a vehicle video storage device (310) or a user terminal (330), it stores the accident video (S305) and analyzes the transmitted accident video to determine the fault ratio for the accident (S306). In addition, the fault ratio determination server (320) can transmit the fault ratio determination result information to the user terminal (330) or the vehicle video storage device (310) linked to the platform (S307). The fault ratio determination artificial intelligence model included in the fault ratio determination server (320) is a model that stores and analyzes the received accident video in a database to predict fault ratio information, and can determine the fault ratio between vehicles by using a model learned for each accident type by applying the traffic accident type, previous precedents, etc. to traffic accidents that occurred in traffic accident video data.

[0082] The fault ratio judgment server (320) can identify key objects, i.e., objects related to an accident, for predicting the fault ratio from a driving video using a pre-trained object recognition artificial intelligence model, and can judge the fault ratio between a vehicle and key objects from a driving video transmitted from a video storage device using a pre-trained fault ratio judgment artificial intelligence model. The object recognition artificial intelligence model extracts spatial-temporal features from a driving video for a predetermined period of time in units of frames to track and analyze the motion of objects and changes in scenes in units of frames, and the fault ratio judgment artificial intelligence model can be a model trained in a supervised or unsupervised learning manner using labeled training data based on the basis of the basic fault ratio classified by the fault ratio information portal, traffic accident types, previous precedents, etc. from the driving video preprocessed through the accident detection artificial intelligence model.

[0083] The artificial intelligence model for determining the percentage of fault can utilize various known deep neural networks (DNNs). Deep neural networks may include convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), graph neural networks (GNNs), autoencoders, generative adversarial networks (GANs), restricted Boltzmann machines (RBMs), Q networks, U networks, Siamese networks, and the like. The description of the aforementioned deep neural networks is merely exemplary, and the present disclosure is not limited thereto.

[0084] Labeling is performed by classifying accident situations based on the "Fault Ratio Information Portal" and annotating key features (traffic lights, road type, lane information, collision information, etc.) that occur when an accident happens, so that the fault ratio judgment AI model of the fault ratio judgment server (320) can recognize the 'key features' that occurred in a specific accident situation.

[0085] The fault ratio determination server (320) can compare the main features of each learned accident situation with the main features obtained from the driving video and output the result of determining the fault ratio between accident-related objects based on the fault ratio according to the most similar accident situation.

[0086] The fault rate determination artificial intelligence model of the fault rate determination server (320) can be trained using a transfer learning method, and can apply a learning method that fine-tunes the model by additionally learning the feature information of accident images and general images generated while learning the training data labeled with the weights of the pre-learned model and the pre-learned model. In this process, the pre-learned weights can be adjusted to fit new data while maintaining the initial general features. The adjustment process can be calculated through a loss function to determine how accurate the prediction is by comparing the input of the training data with the actual output (label). The weights are adjusted in the direction of minimizing the loss function, and when the distribution of each class of the data is unbalanced, the weights can be optimized using a loss function such as the focal loss, and when the class distribution is balanced, the cross entropy loss, etc. The fault rate prediction model can be adjusted through the following two main steps.

[0087] The fault ratio determination artificial intelligence model of the fault ratio determination server (320) can extract bounding box information of an object according to the main features of fault ratio prediction in the object detection step and adjust weights according to how the object information extracted in the image classification step contributes to image classification. The fault ratio determination artificial intelligence model of the fault ratio determination server (320) detects important objects (traffic lights, road types, lane information, collision information, etc.) in an accident video and adjusts weights so that the accident situation is classified based on these, so that the fault ratios corresponding to each accident situation can correspond one-to-one.

[0088] Accordingly, the fault ratio judgment artificial intelligence model of the fault ratio judgment server (320) can be implemented as a model optimized for accident detection by performing supervised and unsupervised learning on the weights of a pre-learning model designed as a model capable of predicting fault ratio-related information using the parameters of a deep neural network using the training data described above.

[0089] The fault ratio determination server (320) can use a learned artificial intelligence model to determine and output the fault ratio between the current vehicle and surrounding objects involved in the accident from a driving video. The fault ratio determination server (320) can transmit the received traffic accident video to the associated insurance company, thereby enabling drivers to file traffic accident insurance claims without a separate wired connection process.

[0090] FIG. 4 is a diagram illustrating operations between a vehicle image storage device (410), a server (420), and a user terminal (430) according to another embodiment of the present disclosure.

[0091] According to another embodiment of the present disclosure, a vehicle video storage device (410) can determine the possibility of an accident occurring among surrounding objects acquired through a camera of a photographing unit, and share the accident video with surrounding objects when an accident occurs. Specifically, referring to FIG. 4, the vehicle video storage device (410) analyzes real-time driving video using an accident detection artificial intelligence model to predict the possibility of an accident occurring among surrounding objects identified in the driving video, thereby detecting an accident (S401). The surrounding objects may be objects such as vehicles, people, bicycles, motorcycles, traffic lights, and road facilities surrounding the current vehicle. When an accident is detected, the vehicle video storage device (410) stores the accident video in the storage unit, and notifies the fault ratio determination server (420) or the user terminal (430) of the surrounding object that is a party to the accident of the accident and transmits the accident video (S402). The user terminal (430) may be a black box, a mobile phone, a PC, a smartwatch, a communication-capable smart device, etc. of the surrounding object related to the accident, and is not limited to the above-described examples.

[0092] If the vehicle image storage device (410) and the user terminal (430) of the surrounding object are capable of wireless communication, the vehicle image storage device (410) can directly transmit the accident video to the user terminal (430). Alternatively, the vehicle image storage device (410) can first transmit the accident video to the video sharing service platform through a predetermined video sharing service platform, and then the user terminal (430) of the surrounding object can receive the accident video from the video sharing service platform. The user of the surrounding object related to the accident or the party related to the accident can check the accident video through the user terminal (430) and request the fault ratio determination server (420) to determine the fault ratio (S404).

[0093] When an accident video is received, the fault ratio determination server (420) stores the accident video (S405) and analyzes the transmitted accident video to determine the fault ratio for the accident (S406). In addition, the fault ratio determination server (420) can transmit the fault ratio determination result information to the user terminal (430) of the surrounding object linked to the platform (S407).

[0094] FIG. 5 is a flowchart illustrating an operation method of a vehicle image storage device according to one embodiment of the present disclosure.

[0095] Referring to FIG. 5, the vehicle image storage device (310) captures and acquires real-time driving images of the vehicle through a camera in the shooting unit (S510). The accident prediction unit of the vehicle image storage device (310) analyzes the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the possibility of an accident occurring in the current vehicle in which the vehicle image storage device (310) is installed (S520).

[0096] As described above, the accident prediction unit of the image storage device (310) can predict the possibility of an accident from a real-time driving video using a pre-learned accident detection artificial intelligence model. The accident detection artificial intelligence model can output the possibility of an accident as a binary value such as “accident occurred” or “accident did not occur”, or as a probability value indicating the possibility of an accident between 0 and 1. The accident prediction unit of the image storage device (310) can determine the possibility of an accident by considering sensor information acquired through the sensor unit in addition to the accident possibility information acquired through the accident detection artificial intelligence model from the driving video. The accident prediction unit of the image storage device (310) determines a combined index (C) by combining information on the distance between the current vehicle and surrounding objects acquired through the lidar sensor and the accident possibility information acquired through the accident detection artificial intelligence model (261), as in the above-described mathematical expression 2, and can determine that the possibility of an accident exists when the combined index (C) is greater than or equal to a predetermined threshold value.

[0097] If it is determined that there is a possibility of an accident (S530), the vehicle video storage device (310) stores driving video for a predetermined period of time, including before and after the accident detection time, centered on the accident detection time (S540). In addition, the vehicle video storage device (310) can transmit the accident video while notifying the fault ratio determination server or the user terminal of the current vehicle of the fact of the accident (S550).

[0098] FIG. 6 is a flowchart illustrating an operation method of a vehicle image storage device according to another embodiment of the present disclosure.

[0099] Referring to FIG. 6, the vehicle image storage device (410) captures and acquires real-time driving images of the vehicle through a camera of the shooting unit (S610). The accident prediction unit of the vehicle image storage device (410) analyzes the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the possibility of an accident occurring between surrounding objects around the current vehicle on which the vehicle image storage device (410) is installed (S620).

[0100] If it is determined that there is a possibility of an accident occurring between surrounding objects around the current vehicle (S630), the vehicle video storage device (410) stores driving videos for a predetermined period of time including before and after the accident detection time, centered on the accident detection time (S640). In addition, the vehicle video storage device (410) can notify the fault ratio determination server or the user terminal of the surrounding object of the fact of the accident and transmit the accident video (S650). As described above, if the vehicle video storage device (410) and the user terminal (430) of the surrounding object are capable of wireless communication, the vehicle video storage device (410) can directly transmit the accident video to the user terminal (430), or the vehicle video storage device (410) can first transmit the accident video to the video sharing service platform through a predetermined video sharing service platform, and then the user terminal (430) of the surrounding object can receive the accident video from the video sharing service platform.

[0101] The method for operating a vehicle image storage device according to the present disclosure described above can be created as a program to be executed on a computer and stored in a computer-readable recording medium. Examples of the computer-readable recording medium include ROM, RAM, CDROM, magnetic tape, floppy disk, optical data storage device, etc., and the computer-readable recording medium can be distributed to computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner. In addition, functional programs, program codes, and code segments for implementing the method can be easily inferred by programmers in the technical field to which the present invention pertains.

[0102] In addition, although the preferred embodiments of the present disclosure have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A video storage device that can be mounted on a vehicle and records and stores video of the vehicle's surroundings. A shooting unit that captures and obtains real-time driving video of the above vehicle; An accident prediction unit that analyzes the real-time driving video using a pre-learned accident detection artificial intelligence model to predict the possibility of an accident occurring in the vehicle; When it is determined that there is a possibility of the above accident occurring, a storage unit that stores driving images for a predetermined period of time including before and after the accident detection time, centered on the accident detection time; and Includes a communication unit that transmits driving video for the above-mentioned predetermined period of time to a fault ratio judgment server, A video storage device characterized in that the above accident detection artificial intelligence model is a model trained using a supervised learning method using training data labeled with accident videos and general driving videos.

2. In paragraph 1, The above accident detection artificial intelligence model is a video storage device that extracts and analyzes the features of the driving video for each frame of the real-time driving video or a predetermined number of frame groups to predict the possibility of an accident.

3. In paragraph 1, An image storage device characterized in that the above accident detection artificial intelligence model is a model learned through transfer learning that fine-tunes the weights of a pre-learned model using the above accident video and general driving video.

4. In paragraph 1, Further comprising a sensor unit including at least one sensor among a gyro sensor, an acceleration sensor, a shock sensor, a GPS, and a lidar sensor, An image storage device in which the accident prediction unit determines that there is a possibility of an accident occurring to the vehicle when the combined indicator obtained by combining sensor information obtained through at least one sensor included in the sensor unit and accident occurrence probability information determined by the accident detection artificial intelligence model is greater than or equal to a set threshold.

5. In paragraph 4, When the information on the probability of an accident determined by the above accident detection AI model is p (where p is a real number between 0 and 1), the distance between the vehicle and surrounding objects measured through the above LiDAR sensor is D, α and β are predetermined weights, and ε is a predetermined constant value, the combination index (C) is given by the following mathematical formula: Accordingly, it is determined to be proportional to the probability of an accident occurring as determined by the above accident detection artificial intelligence model and inversely proportional to the distance between the vehicle and surrounding objects. An image storage device in which the accident prediction unit determines that there is a possibility of an accident occurring between the vehicle and the surrounding object when the combination index (C) is greater than or equal to the set threshold.

6. In paragraph 1, The above-mentioned fault ratio determination server is, Using a pre-trained object recognition AI model, key objects for predicting the fault ratio from the driving video are identified, and An image storage device that determines the fault ratio between the vehicle and the main object from a driving video for a predetermined period of time transmitted from the image storage device using a pre-trained fault ratio determination artificial intelligence model.

7. In paragraph 6, The above object recognition AI model extracts spatial-temporal features from the driving video over the above predetermined time period on a frame-by-frame basis to track and analyze the movement of objects and changes in the scene on a frame-by-frame basis, and An image storage device characterized in that the above-mentioned fault ratio determination AI model is a model trained using a supervised or unsupervised learning method on training data labeled based on the basis of basic fault ratios classified by the fault ratio information portal, using driving video preprocessed through the above-mentioned accident detection AI model.

8. A method of operation of an image storage device that can be mounted on a vehicle and captures and stores images of the vehicle's surroundings, A step of capturing and acquiring real-time driving video of the above vehicle; A step of predicting the likelihood of an accident occurring in the vehicle by analyzing the real-time driving video using a pre-trained accident detection artificial intelligence model; If it is determined that there is a possibility of the above-mentioned accident occurring, a step of storing driving footage for a predetermined period including before and after the accident detection time, centered on the accident detection time; and It includes the step of transmitting driving footage for the above-mentioned predetermined time period to a fault ratio determination server, The above accident detection artificial intelligence model is characterized by being a model trained using a supervised learning method with training data labeled with accident videos and normal driving videos. Method of operation of a vehicle video storage device.

9. An image storage device capable of being mounted on a first vehicle and capturing and storing images of the vehicle's surroundings, A shooting unit that captures and acquires real-time driving video of the first vehicle; An accident prediction unit that analyzes the real-time driving video using a pre-learned accident detection artificial intelligence model to predict the possibility of an accident occurring in a second vehicle around the first vehicle; When the possibility of the above accident occurrence is determined to be greater than a predetermined threshold, a storage unit that stores driving images for a predetermined period of time including before and after the accident detection time, centered on the accident detection time; and It includes a communication unit that transmits driving footage during the above-mentioned predetermined time period to a fault ratio determination server and / or a terminal of the user of the second vehicle, and A video storage device characterized in that the above accident detection artificial intelligence model is a model trained using a supervised learning method using training data labeled with accident videos and general driving videos.

10. A method of operation for an image storage device that can be mounted on a first vehicle and captures and stores images of the vehicle's surroundings, A step of capturing and acquiring real-time driving video of the first vehicle; A step of predicting the possibility of an accident occurring in a second vehicle around the first vehicle by analyzing the real-time driving video using a pre-learned accident detection artificial intelligence model; If the possibility of the above accident occurrence is determined to be greater than a predetermined threshold, a step of storing driving images for a predetermined period of time including before and after the accident detection time, centered on the accident detection time; and The method includes the step of transmitting the driving video during the above-mentioned predetermined time period to a fault ratio determination server and / or to the terminal of the user of the second vehicle, and An operating method of an image storage device, characterized in that the above accident detection artificial intelligence model is a model trained using a supervised learning method using training data labeled with accident images and general driving images.

Images