Image storage device for vehicle and operation method thereof

Abstract

Disclosed is an image storage device which can be mounted on a vehicle and which captures and stores images around the vehicle. The image storage device according to the present disclosure comprises: a photographing unit that photographs and acquires a real-time driving image of a vehicle; an accident prediction unit that analyzes the real-time traveling image using a pre-learned accident detection artificial intelligence model and predicts the possibility of an accident occurring in the vehicle; a storage unit that, if it is determined that there is a possibility of occurrence of the accident, stores a traveling image including a predetermined period of time before and after an accident detection time based on the accident detection time; the communication part is used for sending the driving images in the preset time period to a loss ratio judgment server, and the accident detection artificial intelligence model is a model which is trained by utilizing training data marked with accident images and normal driving images and through a supervised learning mode.

Description

Technical Field

[0001] This disclosure relates to vehicle black boxes, particularly to vehicle image storage devices with on-device artificial intelligence models that use these models to predict accidents from real-time driving footage and store accident images.

[0002] Furthermore, this disclosure relates to a fault ratio determination system that sends accident-related images captured by a vehicle's black box to a server, and the server determines the fault ratio in the accident images. Background Technology

[0003] A vehicle black box is a device attached to a vehicle to record accidents or situations that occur during driving, aiming to analyze the causes of accidents and determine liability. Vehicle black boxes are also known as dashboard cameras or dashcams.

[0004] Vehicle black boxes are used to analyze the causes of accidents and determine liability, but they often store unnecessary footage instead of direct accident footage. This can lead to insufficient memory or battery, making it impossible to retain the necessary footage from the accident. For example, older dashcams stored footage through motion recognition or impact detection, but the motions or impacts detected by the black box are often irrelevant to the actual accident. Furthermore, if the black box's built-in battery malfunctions, the stored footage date may be reset or not stored at all, resulting in the inability to retain relevant footage from the accident. Also, depending on the black box's storage capacity, insufficient storage may cause footage to be overwritten with previously stored data, potentially leaving accident-related footage unrecoverable.

[0005] Furthermore, due to the complexity of retrieving and transmitting accident images stored in the memory, drivers often cannot submit the original images when submitting image data to insurance companies. Instead, they use their personal smartphones to take pictures of the black box and submit them. Such indirectly captured images may hinder the accuracy of fault determination during the insurance process. Summary of the Invention

[0006] The purpose of this disclosure is to provide a vehicle image storage device that uses an accident prediction artificial intelligence model to store relevant driving images when a traffic accident is predicted to occur.

[0007] Furthermore, the purpose of this disclosure is to provide a system for sending accident images captured by a vehicle image storage device to an external server, allowing users or stakeholders to share the accident images.

[0008] Furthermore, the purpose of this disclosure is to provide a method and apparatus for determining the fault ratio between relevant parties from accident images using an artificial intelligence model for fault ratio assessment.

[0009] A vehicle image storage device according to an embodiment of the present disclosure includes: a capturing unit for capturing and acquiring real-time driving images of a vehicle; an accident prediction unit for analyzing the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in the vehicle; a storage unit for storing driving images for a predetermined time period, including the period before and after the accident detection time, based on the accident detection time, when it is determined that there is a probability of an accident occurring; and a communication unit for sending the driving images for the predetermined time period to a fault ratio judgment server, wherein the accident detection artificial intelligence model is a model trained using training data labeled with accident images and normal driving images through supervised learning.

[0010] According to the embodiments disclosed herein, the artificial intelligence accident detection model stores images only when a traffic accident occurs, thereby avoiding blind storage, eliminating the need to overwrite existing images, ensuring safe storage without affecting the limited capacity and lifespan of storage space, and eliminating the risk of image loss.

[0011] Furthermore, according to the embodiments disclosed herein, by sending the stored images through a connection to a server, it is possible to mask personal information contained in the accident images and determine the fault ratio between the accident vehicles. When requested by a user, the platform can provide data in the form of web pages and applications, including insurance processing. Attached Figure Description

[0012] Figure 1 This diagram illustrates an embodiment of the present disclosure of a vehicle image storage device and a vehicle black box system that uses images captured by a shared vehicle image storage device to analyze accident images and determine the fault ratio.

[0013] Figure 2 This is a block diagram illustrating the structure of a vehicle image storage device according to an embodiment of the present disclosure.

[0014] Figure 3 This diagram illustrates the operation between a vehicle image storage device 310, a server 320, and a user terminal 330 according to an embodiment of this disclosure.

[0015] Figure 4 This diagram illustrates the operation between a vehicle image storage device 410, a server 420, and a user terminal 430 according to another embodiment of this disclosure.

[0016] Figure 5 This is a flowchart illustrating an operation method of a vehicle image storage device according to an embodiment of the present disclosure.

[0017] Figure 6 This is a flowchart illustrating an operation method of a vehicle image storage device according to another embodiment of the present disclosure. Detailed Implementation

[0018] A vehicle image storage device according to an embodiment of the present disclosure includes: a capturing unit for capturing and acquiring real-time driving images of the vehicle; an accident prediction unit for analyzing the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in the vehicle; a storage unit for storing driving images for a predetermined time period, including the period before and after the accident detection time, based on the accident detection time, when it is determined that there is a probability of an accident occurring; and a communication unit for sending the driving images for the predetermined time period to a fault ratio judgment server. The accident detection artificial intelligence model is a model trained using training data labeled with accident images and normal driving images through supervised learning.

[0019] In one embodiment, the accident detection AI model can extract and analyze features of the driving video according to each frame or a predetermined number of frame groups of the real-time driving video to predict the probability of an accident.

[0020] In one embodiment, the accident detection AI model can be a model that learns by fine-tuning the weight values ​​of a pre-learned model using the accident images and normal driving images.

[0021] In one embodiment, the vehicle image storage device may further include a sensor unit, which includes at least one sensor selected from gyroscope, accelerometer, impact sensor, GPS, and lidar sensor. When the combined index obtained by combining the detection information obtained by the at least one sensor included in the sensor unit with the accident probability information determined by the accident detection artificial intelligence model reaches or exceeds a preset threshold, the accident prediction unit may determine that there is a possibility of an accident occurring in the vehicle.

[0022] In one embodiment, when the probability of an accident determined by the accident detection AI model is p (p is a real number between 0 and 1), the distance between the vehicle and surrounding objects measured by the lidar sensor is D, α and β are predetermined weight values, and ε is a predetermined constant value, the combined index C can be based on the following mathematical formula. The probability of an accident is determined by the accident detection AI model in a way that is directly proportional to the probability of the accident and inversely proportional to the distance between the vehicle and the surrounding objects. When the combined index C reaches or exceeds the preset threshold, the accident prediction unit can determine that there is a possibility of an accident between the vehicle and the surrounding objects.

[0023] In one embodiment, the fault ratio determination server can use a pre-learned object recognition artificial intelligence model to identify the main object for predicting the fault ratio from the driving images, and use the pre-learned fault ratio determination artificial intelligence model to determine the fault ratio between the vehicle and the main object from the driving images sent by the image storage device over a predetermined time period.

[0024] In one embodiment, the object recognition AI model can extract the spatial-temporal features of the driving images for the predetermined time period frame by frame, track and analyze the actions of objects and scene changes on a frame-by-frame basis, and the fault ratio judgment AI model can be a model trained by supervised or unsupervised learning using labeled training data based on the driving images preprocessed by the accident detection AI model, the basic fault ratio criteria classified by the fault ratio information portal, traffic accident types and previous cases, etc.

[0025] An operation method of an image storage device according to an embodiment of the present invention includes: capturing and acquiring real-time driving images of the vehicle; analyzing the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in the vehicle; storing driving images for a predetermined time period including before and after the accident detection time when it is determined that there is a probability of an accident occurring; and sending the driving images for the predetermined time period to a fault ratio judgment server. The accident detection artificial intelligence model may be a model trained using training data labeled with accident images and normal driving images and trained through supervised learning.

[0026] According to another embodiment of the present invention, an image storage device that can be installed on a first vehicle to capture and store images of the area surrounding the vehicle may include: a capturing unit for capturing and acquiring real-time driving images of the first vehicle; an accident prediction unit for analyzing the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in a second vehicle surrounding the first vehicle; a storage unit for storing driving images for a predetermined time period, including the period before and after the accident detection time, based on the accident detection time, when it is determined that the probability of the accident occurring reaches or exceeds a predetermined threshold; and a communication unit for sending the driving images for the predetermined time period to a fault ratio judgment server and / or the terminal of the user of the second vehicle. The accident detection artificial intelligence model may be a model trained using training data labeled with accident images and normal driving images through supervised learning.

[0027] An operation method of an image storage device that can be installed on a first vehicle to capture and store images of the vehicle's surroundings, according to another embodiment of the present invention, may include: capturing and acquiring real-time driving images of the first vehicle; analyzing the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in a second vehicle surrounding the first vehicle; storing driving images for a predetermined time period, including the period before and after the accident detection time, based on the accident detection time; and sending the driving images for the predetermined time period to a fault ratio judgment server and / or the terminal of a user of the second vehicle. The accident detection artificial intelligence model may be a model trained using training data labeled with accident images and normal driving images and through supervised learning.

[0028] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. The advantages and features of this disclosure, as well as the methods for achieving these advantages and features, will become more apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. However, the technical concept of this disclosure is not limited to the following embodiments and can be implemented in many different forms. The following embodiments are provided only to fully illustrate the technical concept of this disclosure and to fully inform those skilled in the art of the subject of this disclosure of the scope of the disclosure. The technical concept of this disclosure is defined only by the scope of the claims.

[0029] When using reference numerals on structural elements in various figures, the same structural elements should have the same reference numerals as much as possible, even if they are shown in different figures. Furthermore, in describing this disclosure, detailed descriptions of relevant well-known structures or functions are omitted if it is believed that such descriptions might obscure the main points of this disclosure.

[0030] Unless otherwise defined, all terms (including technical and scientific terms) used in this specification should be used in the sense that can be commonly understood by one of ordinary skill in the art to which this disclosure pertains. Furthermore, terms already defined in a general dictionary should not be over-interpreted or idealized unless explicitly defined otherwise. The terminology used in this specification is intended to describe embodiments and not to limit the invention. In this specification, the singular form should include the plural form unless specifically mentioned in the sentence.

[0031] Furthermore, when describing the structural elements of this disclosure, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the structural element from other structural elements and do not limit the nature, order, or sequence of the related structural elements. If a structural element is described as being "connected," "combined," or "linked" to other structural elements, it should be understood that the structural element can be directly linked or connected to those other structural elements, and other structural elements can also be "connected," "combined," or "linked" among the structural elements.

[0032] The embodiments disclosed herein will now be described in detail with reference to the accompanying drawings.

[0033] Figure 1 This diagram illustrates an embodiment of the present disclosure of a vehicle image storage device and a vehicle black box system that uses images captured by a shared vehicle image storage device to analyze accident images and determine the fault ratio.

[0034] A system according to one embodiment of the present invention includes a vehicle image storage device 110, a user terminal 120, and an artificial intelligence server 130. The vehicle image storage device 110 analyzes real-time driving images using an on-device accident detection artificial intelligence model to predict the probability of an accident. Based on the accident probability information, which is the output value of the accident detection artificial intelligence, the vehicle image storage device 110 uses a lidar sensor or similar device to acquire distance information to surrounding objects. The accident detection artificial intelligence model, based on the accident probability information output from the driving image analysis and the distance information to surrounding objects, can predict the probability of an accident between the vehicle equipped with the vehicle image storage device 110 and surrounding objects.

[0035] When an accident is predicted, the vehicle image storage device 110 can send accident images, including images before and after the accident, to the user terminal 120 or an external server. The external server can be an artificial intelligence server 130. Users can receive accident images directly from the vehicle image storage device 110 or receive accident images from the external server through an application on the user terminal linked to the external server. Furthermore, the vehicle image storage device 110 sends a notification to the user terminal 120 when an accident occurs. Upon receiving the notification, the user can access the vehicle image storage device 110 through the user terminal 120 to receive and confirm the accident images. These accident images can be confirmed through applications installed on the user terminal 120. Alternatively, users can access the vehicle image storage device 110 or the external server storing the accident images through the application on the user terminal 120, requesting the transmission of accident images from the vehicle image storage device 110 or the external server, thereby receiving and confirming the accident images. The user terminal 120 is a digital device, such as a laptop, desktop computer, tablet computer, or mobile phone, equipped with a processor and memory.

[0036] Users can communicate with the vehicle image storage device 110 through the platform provided by the application installed on the user terminal 120. Users can access the vehicle image storage device 110 through the user terminal 120 and perform environmental settings such as storage space management, deletion, and transmission. If a user is curious about the extent of their fault, they can request a signal to the artificial intelligence server 130 to determine the fault ratio through the user terminal 120 and receive the fault ratio determination result from the artificial intelligence server 130.

[0037] The vehicle image storage device 110 can directly send the accident images to the artificial intelligence server 130 for determining fault. Alternatively, the user can confirm the accident images through the user terminal 120 and then send the accident images to the artificial intelligence server 130 through the user terminal 120 to request a determination of the fault ratio.

[0038] The artificial intelligence server 130 can analyze accident images sent from the vehicle image storage device 110 or the user terminal 120, determine the fault ratio of the accident, and send the fault ratio determination result information to the user terminal 120 or the vehicle image storage device 110 linked to the platform. As a model, the artificial intelligence server 130 stores the received accident images in a database for analysis to predict fault ratio information. It can determine the fault ratio between vehicles based on the type of traffic accident, previous precedents, etc., for traffic accidents occurring based on traffic accident image data.

[0039] The artificial intelligence server 130 can use a pre-learned object recognition artificial intelligence model to identify the main object for predicting the fault ratio from driving images, and use a pre-learned fault ratio judgment artificial intelligence model to determine the fault ratio between the vehicle and the main object from driving images of a predetermined time period sent by the image storage device. When learning for predicting the fault ratio, the artificial intelligence server 130 can determine and output the fault ratio between the current vehicle and surrounding objects involved in the accident after the accident, based on the fault ratio judgment criteria defined in the "Fault Ratio Information Portal".

[0040] The AI ​​server 130 sends the received traffic accident images to the relevant insurance company, enabling drivers to process traffic accident insurance claims without an additional wired connection.

[0041] Figure 2 This is a block diagram illustrating the structure of a vehicle image storage device according to one embodiment of the present disclosure. The vehicle image storage device 200 may be referred to as a vehicle black box device.

[0042] According to one embodiment of the present invention, the vehicle image storage device 200 serves as a vehicle black box. It can utilize an on-device artificial intelligence model to predict the probability of an accident from real-time driving images captured by the vehicle black box device. When an accident is predicted, it can store images including those before and after the accident and simultaneously send accident-related images to an external server. The external server can be a server of an insurance company pre-configured by the user of the vehicle image storage device 200. When the vehicle image storage device 200 analyzes the real-time driving images and predicts an accident, it can send accident-related images to the insurance company's server and simultaneously send a signal requesting the insurance company's server to accept traffic accident processing.

[0043] refer to Figure 2 The vehicle image storage device 200 may include a shooting unit 210, a microphone 220, a sensor unit 230, an input unit 240, a control unit 250, an accident prediction unit 260, an output unit 270, a communication unit 280, and a storage unit 290.

[0044] The imaging unit 210 comprises one or more cameras mounted on at least one side of the vehicle image storage device 200, capable of capturing images of the front, rear, or sides of the vehicle depending on the camera's mounting orientation and transmitting them to the control unit 250. The imaging unit 210 may also include an image sensor for acquiring images in low-light environments. The image sensor is a device that senses the intensity and color of optical images and converts them into digital image data; it may include charge-coupled device (CCD) and complementary metal-oxide-semiconductor (CMOS) image sensors.

[0045] The images captured by the imaging unit 210 can be stored after preprocessing by the control unit 250, or the raw data can be directly stored in the storage unit 290. The preprocessing process may include compressing the captured images, or converting the frame-by-frame input RGB three-channel images to the Lab color space, and then adjusting the brightness and darkness through histogram stretching or histogram equalization to obtain a balanced image. Furthermore, the preprocessing process may also include a process to brighten the captured images through gamma correction. Through preprocessing, the quality of images captured in low-light environments can be improved.

[0046] Microphone 220 can record speech or sound from outside or inside the vehicle and transmit it to control unit 250. The speech or sound acquired from microphone 220 can be compressed and then combined with images captured by camera 210 and stored in storage unit 290, or it can be directly combined with images and stored in storage unit 290 without compression. The speech or sound recorded by microphone 220 can be used to analyze the situation at the time of the accident.

[0047] The sensor unit 230 may include at least one of a gyroscope sensor, an accelerometer, an impact sensor, a GPS sensor, and a lidar sensor. The gyroscope sensor and the accelerometer are used to measure the angular velocity and acceleration of the vehicle on which the vehicle image storage device 200 is installed, respectively. The impact sensor is used to detect impacts applied to the vehicle. In addition to having a separate impact sensor, the vehicle can be determined to have been impacted when the changes in angular velocity and acceleration detected by the aforementioned gyroscope sensor and accelerometer reach or exceed a predetermined threshold. The Global Positioning System (GPS) can continuously calculate the vehicle's current position in real time and use it to calculate the vehicle's speed information.

[0048] The input unit 240 receives input signals for controlling user operation of the vehicle image storage device 200, and can be composed of keys or buttons for controlling the functions of the vehicle image storage device 200. The input unit 240 can be formed in the form of a touch screen provided in the display unit of the output unit 270.

[0049] The control unit 250 controls the overall operation of the vehicle image storage device 200. Based on the probability of an accident determined by the accident prediction unit 260, the control unit 250 can control the device by storing the accident-related images captured by the imaging unit 210 in the storage unit 290 or sending the accident-related images to an external server via the communication unit 280 when an accident is predicted.

[0050] The accident prediction unit 260 includes an on-device accident detection artificial intelligence model 261, which can analyze real-time driving images acquired by the camera unit 210 to predict the likelihood of an accident involving the vehicle equipped with the vehicle image storage device 200. Furthermore, the accident prediction unit 260 can analyze the real-time driving images acquired by the camera unit 210 to predict the likelihood of an accident involving surrounding objects other than the vehicle equipped with the vehicle image storage device 200.

[0051] The accident detection AI model 261 can be a model trained through supervised learning using training data labeled with accident images and normal driving images occurring in various environments. The accident detection AI model 261 can learn through transfer learning, specifically by combining a pre-learned model with training data that labels the weights of the pre-learned model to further learn the feature information of the generated accident images and general images, thus fine-tuning the model.

[0052] The accident prediction unit 260 uses this pre-learned accident detection artificial intelligence model 261 to predict the probability of an accident occurring from real-time driving images. The accident detection artificial intelligence model 261 takes real-time driving images as input nodes and outputs the probability of an accident occurring as output nodes. For example, the accident detection artificial intelligence model 261 can output a binary value of "accident occurred" or "accident did not occur" as the probability of an accident, or output a probability value between 0 and 1, representing the probability of an accident occurring.

[0053] The accident detection AI model 261 can include multiple layers, and the edges connecting nodes between layers can be assigned weights. During the learning process, the weights of nodes and edges configured between input and output nodes can be updated. Specifically, utilizing deep learning, a type of machine learning, the accident detection AI model 261 can perform supervised learning based on real-time driving video data to learn the probability of accidents occurring. The accident detection AI model 261 can utilize various known deep neural networks (DNNs). Deep neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), graph neural networks (GNNs), autoencoders, generative adversarial networks (GANs), restricted Boltzmann machines (RBMs), Q-networks, U-networks, Siamese networks, etc. The aforementioned description of deep neural networks is merely an example, and this disclosure is not limited thereto.

[0054] The accident detection AI model 261 learns by adjusting the weights between nodes to produce the desired output from the input. Artificial neural networks can learn using methods such as back propagation and gradient descent. The weights of the accident detection AI model 261 are adjusted in the direction of minimizing the loss function. When the data distribution across categories is unbalanced, focal loss can be used; when the category distribution is balanced, binary cross-entropy loss can be used to optimize the weights. The accident detection AI model 261 adjusts the weights by minimizing the loss function through back propagation and gradient descent to classify "accident occurred" and "no accident occurred." As an example, the gradient descent method shown in Equation 1 can be used to optimize the weights of the accident detection AI model 261 to align it with accident detection.

[0055] [Mathematical Expression 1]

[0056]

[0057] w new Previous weight values

[0058] w old Updated weight values

[0059] η: Learning rate, used to adjust the size of weight updates.

[0060] gradient with respect to the weights of the loss function

[0061] As described above, by using accident images and normal driving images as training data, supervising the learning of the weight values ​​of the pre-learned model, and adjusting the parameters of the deep neural network, a model optimized for accident detection that can predict accident detection-related information can be obtained.

[0062] According to one embodiment of this disclosure, the input data for an accident detection artificial intelligence model 261 can include information about surrounding objects identified from driving images, such as feature vectors obtained by analyzing the movement direction, speed, and distance between the identified surrounding objects and the current vehicle in each frame of the driving image. The accident detection artificial intelligence model 261, by inputting the feature vectors extracted from the driving images, can output a binary value or a probability value between 0 and 1 indicating the probability of an accident.

[0063] The accident detection AI model 261 can analyze driving images frame by frame to determine which frame has the highest probability of an accident occurring. The relevant time of the frame with the highest probability of an accident can be estimated as the time of the accident. When predicting an accident, the accident detection AI model 261 groups the input driving images into a predetermined number of frame groups and can select the frame groups with a higher probability of an accident. For example, the accident detection AI model 261 can set 10 sequentially input frames into a frame group and determine the frame group with the highest probability of an accident based on the unit of the frame group. For example, if 30 frames are captured per second, and 3 frame groups are set per second, the time of the accident can be estimated in 1 / 3-second time units. The probability of an accident occurring in each frame group can be calculated using the average of the probability values ​​of multiple frames included in each frame group, or the probability value of the accident occurrence can be calculated on a frame group basis.

[0064] When the accident prediction unit 260 detects a risk of a traffic accident, it can store the images being captured by the imaging unit 210 in the storage unit 290. Accident-related images can be set as non-deletable protected data, and can only be deleted by the user through a predetermined deletion process after confirmation, thereby protecting the accident images from being deleted arbitrarily.

[0065] On the other hand, according to another embodiment of the present invention, the accident prediction unit 260, in addition to the accident probability information obtained from the driving image obtained by the accident detection artificial intelligence model 261 from the imaging unit 210, can also consider the detection information obtained by the sensor unit 230 to determine the probability of an accident. Specifically, the accident prediction unit 260 determines a combination index C, which is a combination of the distance information between the current vehicle and surrounding objects obtained by the lidar sensor included in the sensor unit 230 and the accident probability information obtained by the accident detection artificial intelligence model 261. When the combination index C reaches or exceeds a preset predetermined threshold, it can be determined that there is a possibility of an accident occurring. For example, when the accident probability information determined by the accident detection artificial intelligence model 261 is p (p is a real number between 0 and 1), the distance between the current vehicle and surrounding objects obtained by the sensor unit 230 is D, α and β are predetermined weight values, and ε is a predetermined constant value, the combination index C can be determined according to the following mathematical formula 2.

[0066] [Mathematical Expression 2]

[0067]

[0068] The correlation index C is a value determined to be directly proportional to the probability of an accident as determined by the accident detection AI model 261, and inversely proportional to the distance between the vehicle and surrounding objects. In other words, for the correlation index C, as the accident occurrence prediction value of the accident detection AI model 261 approaches the actual physical distance to surrounding objects, the correlation index C of the probability of an accident increases. The accident prediction unit 260 can compare the correlation index C with a predetermined threshold, and when the correlation index C reaches or exceeds the predetermined threshold, it can be judged as an accident. The threshold that can cover most accidents in the overall accident imagery can be determined through learning data from normal driving images and accident images, and by setting the threshold, the accuracy of accident occurrence prediction by the accident prediction unit 260 can be improved.

[0069] The output unit 270 may include a display unit and a speaker for displaying user experience / user interface (UX / UI) information for controlling the vehicle image storage device 200 or for outputting images captured by the capturing unit 210. Through the display, the user can view the images captured by the capturing unit 210 and supports touch input, allowing the user to easily select and set desired functions. As mentioned above, the user can input through the display unit with a touchscreen or receive input through a separate input unit 240. The speaker can output audio signals related to the functions performed by the vehicle image storage device 200, or output sound recorded by the microphone 220.

[0070] When an accident is predicted based on the probability of an accident determined by the accident prediction unit 260, the communication unit 280 can send accident-related images captured by the imaging unit 210 or detection data acquired by the sensor unit 230 to an external server or user terminal. Furthermore, the communication unit 280 can communicate with the user terminal, surrounding vehicles, or roadside units (RSUs) to transmit and receive data. The communication unit 280 supports wireless or wired networks such as Wi-Fi, Bluetooth, 3G, 4G (LTE), 5G, 6G, WiMAX, and WiGI.

[0071] If the communication unit 280 is equipped with Wi-Fi and Bluetooth, the user can access and operate the vehicle image storage device 200 via wireless communication through an application or similar device installed on the user terminal. However, since this Wi-Fi or Bluetooth communication technology only operates at close range, operation is limited to the vicinity of the vehicle image storage device 200. The user can manage the black box via Wi-Fi and Bluetooth, and quickly review and download images around the black box via Wi-Fi without an internet connection.

[0072] If the communication unit 280 is equipped with internet-connected wireless communication technology, users can remotely monitor and manage images captured by the vehicle image storage device 200 in real time. Through LPWAN technologies such as NB-IoT, users can access the data of the vehicle image storage device 200 from anywhere, anytime, anywhere, immediately download important images, and receive instant notifications in the event of an accident. As mentioned earlier, users can communicate with the vehicle image storage device 200 within the platform using an application installed on their user terminal.

[0073] The storage unit 290 can store the images captured by the camera unit 210 and the audio data collected by the microphone 220 in a single file or in separate file formats. In particular, the storage unit 290 can take protective measures to temporarily prevent the deletion of accident-related images when an accident is predicted based on the probability of an accident determined by the accident prediction unit 260, thus preventing the accident-related images from being deleted due to user misoperation or error.

[0074] The storage unit 290 can temporarily store the images captured by the imaging unit 210 in a buffer or the like, and only store the relevant accident images when an accident is actually predicted to occur, thereby saving storage space.

[0075] The storage unit 290 can be built inside the vehicle image storage device 200, or it can be detachably built through a predetermined format port installed on the vehicle image storage device 200. The storage unit 290 can be formed in the form of a hard disk drive, flash memory, or in a detachable form such as an SD card, Micro SD card, or USB memory.

[0076] On the other hand, according to an embodiment of the present invention, when storing driving images or accident images, the vehicle image storage device 200 can identify objects from the images. In order to protect the personal information of the identified objects, it can determine the face or license plate number and mask a portion of the object whose personal information needs to be protected. Masking is the process of selecting a specific area in an image or video for protection or hiding, mainly used for personal information protection. When identifying objects in an image, the vehicle image storage device 200 can use a CNN-based deep neural network for object detection and apply masking. Masking can be achieved by blurring the part of the identified object that needs to be masked (such as Gaussian blur and mosaic). This masking process can be performed by the control unit of the vehicle image storage device 200 or a separately installed image processing module.

[0077] On the other hand, according to another embodiment of this disclosure, the accident prediction unit 260 can predict the probability of an accident between surrounding objects identified in the input driving image, even if no accident has occurred in the current vehicle on which the vehicle image storage device 200 is currently installed. If an accident is predicted to occur between surrounding objects identified in the driving image, the accident prediction unit 260 can send a notification to the surrounding objects via the communication unit 280, or send accident images involving the captured surrounding objects to the surrounding objects. Furthermore, the accident prediction unit 260 can also send accident images between surrounding objects to an external server, and the external server can send notification signals to the surrounding objects related to the accident, or send accident images from the external server to the surrounding objects.

[0078] Figure 3 This diagram illustrates the operation between a vehicle image storage device 310, a server 320, and a user terminal 330 according to an embodiment of this disclosure.

[0079] refer to Figure 3The vehicle image storage device 310 analyzes real-time driving images using an accident detection artificial intelligence model to predict the likelihood of an accident and detect it (S301). Upon detecting an accident, the vehicle image storage device 310 saves the accident image to its storage unit and simultaneously notifies the fault ratio assessment server 320 or the user terminal 330 of the fact that the accident has occurred, and may also send the accident image (S302). The vehicle image storage device 310 may immediately send the accident image to the fault ratio assessment server 320 upon accident detection, or send an accident notification and accident image through the user terminal 330 linked to the platform. After the user confirms the accident image stored on the user terminal 330 (S303), the user may choose to send the accident image from the user terminal 330 to the fault ratio assessment server 320. The user may confirm the accident image through the user terminal 330 and request the fault ratio assessment server 320 to assess the fault ratio (S304). As mentioned above, without the need for the user to confirm the accident images, the vehicle image storage device 310 can directly send the accident images to the fault ratio judgment server 320, which can then store the accident images and determine the fault ratio.

[0080] When the fault ratio determination server 320 receives accident images through the vehicle image storage device 310 or the user terminal 330, it stores the accident images (S305) and analyzes the transmitted accident images to determine the fault ratio of the accident (S306). Furthermore, the fault ratio determination server 320 can send the fault ratio determination result information to the user terminal 330 or the vehicle image storage device 310 linked to the platform (S307). The fault ratio determination artificial intelligence model included in the fault ratio determination server 320, as a model, stores the received accident images in a database for analysis to predict fault ratio information. It can determine the fault ratio between vehicles based on the type of traffic accident, previous precedents, etc., for traffic accidents occurring based on traffic accident image data.

[0081] The fault ratio assessment server 320 can utilize a pre-learned object recognition AI model to identify the main objects used to predict the fault ratio from driving images, i.e., objects related to the accident. It then uses the pre-learned fault ratio assessment AI model to determine the fault ratio between the vehicle and the main objects from driving images sent from the image storage device. The object recognition AI model can extract the spatial-temporal features of driving images frame by frame within a predetermined time period, tracking and analyzing the actions of objects and scene changes frame by frame. The fault ratio assessment AI model can be a model trained using labeled training data through supervised or unsupervised learning, based on driving images pre-processed by an accident detection AI model, basic fault ratio criteria classified by a fault ratio information portal, traffic accident types, and previous cases.

[0082] Error ratio assessment AI models can utilize various well-known deep neural networks (DNNs). Deep neural networks can include convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks (DBNs), graph neural networks (GNNs), autoencoders, generative adversarial networks (GANs), restricted Boltzmann machines (RBMs), Q-networks, U-networks, Siamese networks, etc. The aforementioned descriptions of deep neural networks are merely illustrative, and this disclosure is not limited to them.

[0083] The annotation is based on the classification of accident situations in the "Fault Ratio Information Portal". The annotation adds the main features (traffic lights, road type, lane information, collision information, etc.) at the time of the accident, so that the fault ratio judgment server 320's fault ratio judgment artificial intelligence model can identify the "main features" generated in specific accident situations.

[0084] The fault ratio judgment server 320 can compare the main features of different accident situations learned with the main features obtained from driving images, and output the fault ratio judgment results between accident-related objects based on the fault ratio of the most similar accident situation.

[0085] The error ratio prediction AI model of the error ratio prediction server 320 can be learned through transfer learning, specifically by combining a pre-learned model with training data labeled with the weights of the pre-learned model. This allows for further learning of the feature information of the generated accident and general images, resulting in fine-tuning of the model. During this process, the pre-learned weights, while maintaining their initial general characteristics, can be adjusted to align with the new data. The adjustment process involves comparing the input of the training data with the actual output (labels) and calculating the prediction accuracy using a loss function. The weights are adjusted towards minimizing the loss function. When the distribution of each category in the data is unbalanced, focal loss can be used; when the category distribution is balanced, cross-entropy loss or other loss functions can be used to optimize the weights. The error ratio prediction model can be adjusted through the following two main steps.

[0086] In the object detection step, the fault ratio assessment AI model of the fault ratio assessment server 320 extracts the bounding box information of objects based on the main features of fault ratio prediction. In the image classification step, it adjusts the weight values ​​based on the contribution of the extracted object information to image classification. The fault ratio assessment AI model of the fault ratio assessment server 320 can detect important objects in accident images (traffic lights, road types, lane information, collision information, etc.) and classify accident situations based on these, adjusting the weight values ​​to ensure that they correspond one-to-one with the fault ratio of each type of accident.

[0087] Therefore, the fault ratio judgment artificial intelligence model of the fault ratio judgment server 320 can perform supervised and unsupervised learning on the weight values ​​of the pre-learning model using the above training data to achieve an optimized model for accident detection. The pre-learning model is designed to use the parameters of a deep neural network to predict fault ratio related information.

[0088] The fault ratio assessment server 320 uses a learned artificial intelligence model to determine and output the fault ratio between the vehicle currently involved in the accident and surrounding objects involved in the accident in the driving video. The fault ratio assessment server 320 sends the received traffic accident video to the relevant insurance company, enabling drivers to process traffic accident insurance claims without additional wired connection processes.

[0089] Figure 4 This diagram illustrates the operation between a vehicle image storage device 410, a server 420, and a user terminal 430 according to another embodiment of this disclosure.

[0090] According to another embodiment of the present invention, a vehicle image storage device 410 determines the probability of an accident occurring involving surrounding objects acquired by the imaging unit, and can share the accident image with surrounding objects when an accident occurs. Specifically, refer to Figure 4 The vehicle image storage device 410 analyzes real-time driving images using an accident detection artificial intelligence model to predict the probability of an accident between surrounding objects identified from the driving images, thereby detecting an accident (S401). Surrounding objects can be vehicles, pedestrians, bicycles, motorcycles, traffic lights, roadside equipment, etc., around the current vehicle. After detecting an accident, the vehicle image storage device 410 stores the accident image in the storage unit and simultaneously notifies the fault ratio judgment server 420 or the user terminal 430 of the surrounding objects related to the accident of the fact of the accident, and can send the accident image (S402). The user terminal 430 can be a black box of the surrounding objects related to the accident, a mobile phone, a PC, a smartwatch, a communicating smart device, etc., and is not limited to the aforementioned examples.

[0091] If wireless communication is possible between the vehicle image storage device 410 and the user terminals 430 of surrounding objects, the vehicle image storage device 410 can directly send the accident images to the user terminals 430. Alternatively, the vehicle image storage device 410 can first send the accident images to a predetermined image sharing service platform, from which the user terminals 430 of surrounding objects can receive the accident images. Users of surrounding objects or parties involved in the accident can confirm the accident images through the user terminals 430 and request the fault ratio assessment server 420 to determine the fault ratio (S404).

[0092] When the fault ratio determination server 420 receives an accident image, it stores the accident image (S405) and analyzes the sent accident image to determine the fault ratio of the accident (S406). Furthermore, the fault ratio determination server 420 can send the fault ratio determination result information to the user terminal 430 of the surrounding objects linked to the platform (S407).

[0093] Figure 5 This is a flowchart illustrating an operation method of a vehicle image storage device according to an embodiment of the present disclosure.

[0094] refer to Figure 5 The vehicle image storage device 310 captures and acquires real-time driving images of the vehicle using a camera in the imaging unit (S510). The accident prediction unit of the vehicle image storage device 310 can analyze the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring in the current vehicle on which the vehicle image storage device 310 is installed (S520).

[0095] As previously described, the accident prediction unit of the image storage device 310 can predict the probability of an accident from real-time driving images using a pre-learned accident detection artificial intelligence model. The accident detection artificial intelligence model can output a binary value indicating the probability of an accident as "an accident has occurred" or "no accident has occurred," or a probability value between 0 and 1, representing the likelihood of an accident. In addition to the accident probability information obtained from the driving images through the accident detection artificial intelligence model, the accident prediction unit of the image storage device 310 can also consider detection information obtained by the sensor unit to determine the probability of an accident. The accident prediction unit of the image storage device 310 can determine a combination index C based on the aforementioned mathematical formula 2, which is a combination of the distance information between the current vehicle and surrounding objects obtained by the lidar sensor and the accident probability information obtained by the accident detection artificial intelligence model 261. When the combination index C reaches or exceeds a preset predetermined threshold, it can be determined that there is a possibility of an accident occurring.

[0096] When it is determined that there is a possibility of an accident occurring (S530), the vehicle image storage device 310 stores driving images within a predetermined time period before and after the time of the accident (S540). Furthermore, the vehicle image storage device 310 can notify the fault ratio determination server or the terminal of the user of the current vehicle of the fact of the accident and send the accident images (S550).

[0097] Figure 6 This is a flowchart illustrating an operation method of a vehicle image storage device according to another embodiment of the present disclosure.

[0098] refer to Figure 6 The vehicle image storage device 410 captures and acquires real-time driving images of the vehicle using a camera in the imaging unit (S610). The accident prediction unit of the vehicle image storage device 410 can analyze the real-time driving images using a pre-learned accident detection artificial intelligence model to predict the probability of an accident occurring between objects around the vehicle where the vehicle image storage device 410 is installed (S620).

[0099] When it is determined that there is a possibility of an accident occurring among the surrounding objects of the current vehicle (S630), the vehicle image storage device 410 stores driving images within a predetermined time period, including the period before and after the time of the accident, based on the time of the accident (S640). Furthermore, the vehicle image storage device 410 can notify the fault ratio assessment server or the user terminal of the surrounding object of the fact of the accident and send the accident image (S650). As mentioned above, when wireless communication is possible between the vehicle image storage device 410 and the user terminal 430 of the surrounding object, the vehicle image storage device 410 can directly send the accident image to the user terminal 430, or the vehicle image storage device 410 can first send the accident image to a predetermined image sharing service platform, from which the user terminal 430 of the surrounding object can receive the accident image.

[0100] The operation method of the vehicle image storage device according to the present invention can be created by a program bootable on a computer and stored on a computer-readable recording medium. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. The computer-readable recording medium can be distributed across a network-connected computer system to store and boot the computer-readable code in a distributed manner. Furthermore, the functional programs, program code, and code segments for implementing the method can be readily deduced by a programmer skilled in the art to which this invention pertains.

[0101] Furthermore, although the preferred embodiments of this disclosure have been shown and described above, the present invention is not limited to the specific embodiments described. Any changes and implementations that can be made by those skilled in the art without departing from the spirit of the invention as claimed in the claims should not be understood separately from the technical concept or prospect of the present invention.

Claims

1. An image storage device capable of being installed on a vehicle to capture and store images of the area surrounding the vehicle, characterized in that, include: The camera unit captures and acquires real-time driving images of the vehicle. The accident prediction unit uses a pre-learned accident detection artificial intelligence model to analyze the real-time driving images and predict the probability of an accident occurring to the vehicle. The storage unit, when it is determined that there is a possibility of the accident occurring, stores driving images for a predetermined time period before and after the accident detection time, based on the accident detection time. as well as The communications department sends the driving images for the predetermined time period to the fault ratio determination server. The accident detection AI model is trained using labeled accident images and normal driving images through supervised learning.

2. The image storage device according to claim 1, characterized in that, The accident detection artificial intelligence model extracts and analyzes features of the driving images according to each frame or a predetermined number of frame groups of the real-time driving images to predict the probability of an accident.

3. The image storage device according to claim 1, characterized in that, The accident detection AI model is a model that learns by using transfer learning to fine-tune the weight values ​​of a pre-learned model using accident images and normal driving images.

4. The image storage device according to claim 1, characterized in that, It also includes a sensor unit, which comprises at least one sensor selected from a gyroscope sensor, an accelerometer sensor, an impact sensor, a global positioning system (GPS) sensor, and a lidar sensor. When the combined index obtained by combining the detection information acquired by at least one sensor included in the sensor unit and the accident probability information determined by the accident detection artificial intelligence model reaches or exceeds a preset threshold, the accident prediction unit determines that there is a possibility of an accident occurring in the vehicle.

5. The image storage device according to claim 4, characterized in that, When the probability of an accident determined by the accident detection AI model is p (p is a real number between 0 and 1), the distance between the vehicle and surrounding objects measured by the lidar sensor is D, α and β are predetermined weight values, and ε is a predetermined constant value, the combined index (C) is based on the following mathematical formula. The probability of an accident is determined in a manner that is directly proportional to the probability of the accident as determined by the accident detection AI model and inversely proportional to the distance between the vehicle and surrounding objects. When the combined index (C) reaches or exceeds the preset threshold, the accident prediction unit determines that there is a possibility of an accident between the vehicle and the surrounding objects.

6. The image storage device according to claim 1, characterized in that, The fault ratio determination server uses a pre-learned object recognition artificial intelligence model to identify the main object used to predict the fault ratio from the driving images, and uses the pre-learned fault ratio determination artificial intelligence model to determine the fault ratio between the vehicle and the main object from the driving images sent by the image storage device over a predetermined time period.

7. The image storage device according to claim 6, characterized in that, The object recognition AI model extracts the spatial-temporal features of the driving images within the predetermined time period frame by frame, and tracks and analyzes the actions of objects and scene changes on a frame-by-frame basis. The fault ratio judgment artificial intelligence model is a model trained on training data using supervised or unsupervised learning methods. The training data is labeled based on the basic fault ratio of driving images preprocessed by the accident detection artificial intelligence model in the fault ratio information portal.

8. A method for operating a vehicle image storage device, wherein the vehicle image storage device can be installed on a vehicle to capture and store images of the area surrounding the vehicle, characterized in that... Includes the following steps: Capture and acquire real-time driving images of the vehicle; The real-time driving images are analyzed using a pre-learned accident detection artificial intelligence model to predict the likelihood of an accident involving the vehicle. If it is determined that there is a possibility of the accident occurring, the driving images are stored for a predetermined time period before and after the accident detection time, based on the accident detection time. The driving images for the predetermined time period are sent to the fault ratio determination server. The accident detection AI model is trained using supervised learning with labeled accident images and normal driving images as training data.

9. An image storage device capable of being installed on a first vehicle and capturing and storing images of the area surrounding the vehicle, characterized in that, include: The camera unit captures and acquires real-time driving images of the first vehicle. The accident prediction unit uses a pre-learned accident detection artificial intelligence model to analyze the real-time driving images and predict the probability of an accident involving a second vehicle around the first vehicle. The storage unit, when it determines that the probability of the accident occurring reaches or exceeds a predetermined threshold, stores driving images for a predetermined time period before and after the accident detection time, based on the accident detection time. as well as The communications unit sends the driving images for the predetermined time period to the fault ratio determination server and / or the terminal of the user of the second vehicle. The accident detection AI model is trained using supervised learning with labeled accident images and normal driving images as training data.

10. A method of operating an image storage device, the image storage device being capable of being installed on a first vehicle and capturing and storing images of the area surrounding the vehicle, characterized in that... Includes the following steps: Capture and acquire real-time driving images of the first vehicle; The real-time driving images are analyzed using a pre-learned accident detection artificial intelligence model to predict the probability of an accident involving a second vehicle around the first vehicle. If the probability of an accident occurring is determined to be above a predetermined threshold, driving images for a predetermined time period, including the period before and after the accident detection time, are stored; and The driving images during the predetermined time period are sent to the fault ratio determination server and / or the terminal of the user of the second vehicle. The accident detection AI model is trained using supervised learning with labeled accident images and normal driving images as training data.

Images