Method and system for identifying vehicle using lidar and providing accident information

Abstract

One embodiment of the present disclosure provides a method for identifying a vehicle and providing accident information, which is performed by a system communicatively connected to a camera installed in a vehicle so as to have a first field of view and a LiDAR installed in the vehicle so as to have a second field of view wider than the first field of view. The method comprises the steps of: detecting an object in a LiDAR image received from a LiDAR and assigning a unique key to the object; generating a combined image by combining a camera image received from the camera and the LiDAR image and synchronizing the object in the combined image; and transmitting at least one among the camera image, the lidar image, and the combined image corresponding to an accident point in time to a communication-connected external terminal when an accident involving the object is detected or an accident information request signal for the object is received from the external terminal.

Description

Method and system for vehicle identification and accident information provision using lidar

[0001] The present invention relates to a method and system for vehicle identification and accident information provision using lidar, and more specifically, to a device and method for identifying a vehicle through a camera and lidar installed in the vehicle, and providing information on the accident vehicle and an accident video based on videos captured when an accident occurs in an autonomous vehicle.

[0002] When a traffic accident occurs, liability and the percentage of fault are determined during the investigation process. However, there are cases where determining the cause of the accident based solely on black box camera footage is difficult.

[0003] For example, collecting additional evidence from the area surrounding the accident may be necessary, and in interpersonal disputes, complex procedures and privacy issues may arise, particularly when obtaining video information from others.

[0004] As research on autonomous vehicles progresses, the question of who will be held responsible for accidents that occur after the introduction of autonomous vehicles is being raised.

[0005] Accordingly, lidar is being used in autonomous vehicles to clarify safe driving and liability disputes. Because lidar detects the surrounding physical environment in three dimensions, it can provide accurate information on the causes of traffic accidents that occurred directly, indirectly, or even unrelated to autonomous vehicles.

[0006] However, when using lidar information as evidence after an accident, there is the inconvenience of having to prove that the lidar information is intact and that the vehicle whose movement is being described is the vehicle involved in the accident.

[0007] Accordingly, there is a need for a method to analyze accidents occurring around autonomous vehicles and use lidar data to more clearly resolve traffic accident liability disputes.

[0008] The present disclosure is intended to solve the problems of the prior art described above, and provides a device and method for identifying a vehicle through a camera and lidar installed in the vehicle, and providing information on the accident vehicle and an accident video based on videos captured when an accident of an autonomous vehicle occurs.

[0009] The technical problems to be solved by the present invention are not limited to the technical problems described above, and other technical problems of the present invention can be derived from the following description.

[0010] As a technical means for solving the above-described technical problem, an embodiment according to a first aspect of the present disclosure provides a vehicle identification and accident information providing method performed by a system that is communicatively connected to a camera installed in a vehicle to have a first field of view and a lidar installed in the vehicle to have a second field of view wider than the first field of view. The method includes the steps of detecting an object in a lidar image received from the lidar and assigning a unique key to the object, combining the camera image received from the camera and the lidar image to generate a combined image, and synchronizing the object in the combined image, and when an accident for the object is detected or an accident information request signal for the object is received from a communicatively connected external terminal, the step of transmitting at least one image of the camera image, the lidar image, and the combined image corresponding to the time of the accident to the external terminal.

[0011] In addition, an embodiment according to a second aspect of the present disclosure provides a vehicle identification and accident information providing system. The system includes a communication module for transmitting and receiving information with at least one of a camera installed in a vehicle to have a first field of view and a lidar installed in the vehicle to have a second field of view wider than the first field of view, at least one processor, and a memory electrically connected to the processor and storing at least one code executed by the processor, wherein the memory stores a code that, when executed through the processor, causes the processor to detect an object in a lidar image received from the lidar, assign a unique key to the object, combine the camera image received from the camera and the lidar image to generate a combined image, synchronize the object in the combined image, and, when an accident involving the object is detected or an accident information request signal regarding the object is received from a communication-connected external terminal, transmit at least one image among the camera image, the lidar image, and the combined image corresponding to the time of the accident to the external terminal.

[0012] According to the present invention, by using cameras and even lidar to confirm physical information of traffic accidents, the responsibility and cause analysis of traffic accidents can be made clear.

[0013] Additionally, according to the present invention, it is possible to charge information costs, including personal information masking costs, to the insurance company or the requester regarding traffic accidents in the vicinity.

[0014] In addition, according to the present invention, a data section that can completely explain the location of a vehicle number corresponding to personal information and the causality of a traffic accident can be provided during the problem-solving process.

[0015] The effects of the present invention are not limited to the effects described above, and include all effects understood from the following description.

[0016] FIG. 1 is a drawing illustrating an accident information providing device according to one embodiment of the present invention and a lidar, a camera, and an external terminal connected to the device in communication therewith.

[0017] Figure 2 is a drawing showing a detailed configuration of the accident information provision device illustrated in Figure 1.

[0018] Figure 3 is a diagram illustrating an example of a lidar image and a combined image.

[0019] Figure 4 is a diagram illustrating an example of detecting an accident based on lidar.

[0020] Figure 5 is a flowchart illustrating the sequence of an accident information providing method according to another embodiment of the present invention.

[0021] Figures 6 to 8 are drawings illustrating detailed steps for some steps of the accident information provision method illustrated in Figure 5.

[0022] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, the attached drawings are only intended to facilitate understanding of the embodiments disclosed in the present specification, and the technical concepts disclosed in the present specification are not limited by the attached drawings. All terms, including technical and scientific terms, used herein should be interpreted as having meanings generally understood by a person of ordinary skill in the technical field to which the present disclosure pertains. Terms defined in the dictionary should be interpreted as having additional meanings consistent with the relevant technical literature and the contents of the present disclosure, and shall not be interpreted in an extremely ideal or restrictive sense unless otherwise defined.

[0023] In order to clearly explain the present disclosure in the drawings, parts irrelevant to the description have been omitted, and the size, shape, and appearance of each component shown in the drawings may be modified in various ways. Identical / similar parts throughout the specification are given identical / similar drawing reference numerals.

[0024] The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if a detailed description of a related known technology is judged to obscure the gist of the embodiments disclosed in this specification, the detailed description thereof has been omitted.

[0025] Throughout the specification, when a part is said to be "connected (connected, in contact with, or coupled)" to another part, this includes not only cases where it is "directly connected (connected, in contact with, or coupled)" but also cases where it is "indirectly connected (connected, in contact with, or coupled)" with another member in between. Furthermore, when a part is said to "include (have or provide)" a certain component, this does not mean that it excludes other components, but rather that it may "include (have or provide)" other components, unless otherwise specifically stated.

[0026] As used herein, ordinal terms such as "first," "second," etc., are used solely to distinguish one component from another and do not limit the order or relationship of the components. For example, the first component of the present disclosure may be referred to as the "second component," and similarly, the second component may also be referred to as the "first component." As used herein, singular forms should be construed to include plural forms, unless explicitly stated otherwise.

[0027] FIG. 1 is a drawing illustrating an accident information providing device according to one embodiment of the present invention and a lidar, a camera, and an external terminal connected to the device in communication therewith.

[0028] Referring to FIG. 1, the accident information provision system may include an accident information provision device (100), a lidar (200), a camera (300), and an external terminal (400).

[0029] The accident information provision device (100) can communicate with a lidar (200), a camera (300), and an external terminal (400) through a preset communication network to transmit and receive information.

[0030] The accident information providing device (100) detects an object in a lidar image received from the lidar (200) and assigns a unique key to the object.

[0031] Here, the lidar image may be captured from a wider range than the camera image. The lidar image may include a top-view lidar image, which is an image of the road on which the object is located, to track the object's surroundings and trajectory, and a camera-view lidar image, which is used to verify the lidar's recognition capabilities.

[0032] The accident information provision device (100) combines camera images and lidar images received from a camera (300) to generate a combined image and synchronizes objects within the combined image. Here, the combined image may be an image generated by combining a camera image and a camera view lidar image to combine images of the same object from the camera and lidar.

[0033] When an accident information provision device (100) detects an accident regarding an object or receives a request signal for object information from an external terminal, it transmits at least one image among a camera image, a lidar image, and a combined image corresponding to the time of the accident to the external terminal.

[0034] The accident information providing device (100) can display bounding boxes for each object recognized in each of the camera images and lidar images, and display unique keys for each object recognized in the bounding boxes.

[0035] The accident information provision device (100) may be implemented in the form of a server, a computing device, or various smart devices, and may operate in a cloud computing service model such as SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service). In addition, the web real-time communication-based email phone service provision system (100) may be constructed in the form of a private cloud, a public cloud, or a hybrid cloud system, but the scope of the present invention is not limited thereto.

[0036] LiDAR (Light Detection And Ranging) (200) may be a sensor that uses lasers to recognize surrounding spatial information. LiDAR (200) is more precise and can obtain more information than radar, making it frequently used in autonomous vehicles. The information detected by LiDAR (200) is stored as a 3D point cloud, and the stored data can be further visualized to facilitate understanding.

[0037] The lidar (200) can be installed on a vehicle to have a second field of view wider than the first field of view. The lidar (200) can capture images over a wider range than the camera (300), and the lidar (200) can be installed to capture images of the front, rear, and sides.

[0038] Lidar (200) can transmit Lidar images to an accident information provision device (100).

[0039] The camera (300) may be a device for capturing images. The camera (300) may be installed on a vehicle to have a first field of view.

[0040] The camera (300) can transmit camera images to the accident information providing device (100).

[0041] An external terminal (400) can transmit an accident information request signal and receive a response to the accident information request signal from an accident information provision device (100). Here, the response to the accident information request signal may be at least one of a camera image, a lidar image, and a combined image corresponding to the time at which the accident occurred.

[0042] An external terminal (400) may mean a notebook, desktop, laptop equipped with a web browser, a wireless communication device that guarantees portability and mobility, or any type of handheld-based wireless communication device such as a smartphone or tablet PC.

[0043] Although not illustrated in the drawing, the accident information provision system may include a database. The database may be stored within the accident information provision device (100) or may be connected to the accident information provision device (100).

[0044] The database may store at least one of camera object information, an object unique key table, lidar object information, and accident recording video. Here, the object unique key table may be created in the form of a data table by recognizing the vehicle number of a synchronized object and matching the recognized vehicle number with the unique key assigned to the object.

[0045] Additionally, the accident information provision system may further include a sensor fusion module.

[0046] A sensor fusion module may be a module that integrates information from multiple sensors installed in an autonomous vehicle. Each sensor has its own strengths and weaknesses, so autonomous vehicles utilize multiple sensors for perception, rather than just one. By integrating information from a camera (300) and a lidar (200), it is possible to determine whether objects detected by each sensor are the same object. Furthermore, the movement of an object missed by one sensor (e.g., a camera) can be tracked by another sensor (e.g., a lidar).

[0047] However, the sensor fusion module may be stored in the accident information providing device (100) in the form of an algorithm. For convenience of explanation, the sensor fusion algorithm will be described below in the form in which it is stored in the accident information providing device (100).

[0048] Figure 2 is a drawing showing a detailed configuration of the accident information provision device illustrated in Figure 1.

[0049] Referring to FIG. 2, the accident information providing device (100) may include a communication module (110), a processor (120), and a memory (130).

[0050] The communication module (110) may include a device including hardware and software necessary to transmit and receive signals such as control signals or data signals through wired or wireless connections with other network devices.

[0051] The communication module (110) can receive images from each of the camera and the lidar, receive an accident information request signal from an external terminal, and transmit a response to the accident information request signal to the external terminal.

[0052] The processor (120) may include various types of devices that control and process data. The processor (120) may refer to a data processing device built into hardware that has a physically structured circuit to perform functions expressed in code or instructions included in a program.

[0053] In one example, the processor (120) may be implemented in the form of a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the present invention is not limited thereto.

[0054] The processor (120) performs operations according to the code stored in the memory (130).

[0055] The memory (130) can store at least one of information and data input to the communication module (110), information and data required for functions performed by the processor (120), and data generated according to the execution of the processor (120).

[0056] Memory (130) should be interpreted as a general term for a non-volatile storage device that maintains stored information even when no power is supplied, and a volatile storage device that requires power to maintain stored information. Memory (130) may include magnetic storage media or flash storage media in addition to a volatile storage device that requires power to maintain stored information, but the scope of the present invention is not limited thereto.

[0057] The memory (130) is electrically connected to the processor (120) and stores at least one code to be executed by the processor (120). The memory (130) stores code that, when executed by the processor (120), causes the processor (120) to perform the following functions and procedures.

[0058] The memory (130) stores a code that detects an object in a lidar image received from the lidar and assigns a unique key to the object. Here, the object may be a vehicle with a unique license plate number.

[0059] More specifically, the memory (130) may store code that causes the lidar to detect an object contained within the lidar lifetime.

[0060] The memory (130) may store code that causes the system to determine whether an object is a first-detected object. For example, the memory (130) may store code that causes the system to look up the vehicle number of a detected object based on a pre-stored table.

[0061] The memory (130) may store a code that causes a unique key to be assigned to an object when the object is first detected. For example, the unique key may be generated using English, numbers, Korean, or a combination of English, numbers, and Korean.

[0062] The memory (130) stores a code that combines the camera image and the lidar image received from the camera to create a combined image and causes the object to be synchronized within the combined image.

[0063] More specifically, the memory (130) may store code that causes the camera to detect an object within the image.

[0064] To verify that the vehicle requested via license plate number and the specific vehicle shown in the LIDAR image are the same, the time the license plate number appeared on the camera must be verified. Mapping the vehicle detected by the camera with the vehicle in the LIDAR image through sensor fusion confirms that the vehicle in the LIDAR image is the same vehicle with the requested license plate number.

[0065] Accordingly, the memory (130) may store a code that causes a combined image to be generated by combining an object in a camera image and an object in a lidar image through a sensor fusion algorithm for fusing information from multiple sensors.

[0066] For example, the memory (130) may store a code that causes a camera-lidar calibration task to be performed. Since there is a linear transformation that corresponds three-dimensional spatial coordinates (x, y, z) to two-dimensional camera coordinates (u, v), finding this can easily enable sensor fusion of the camera and the lidar. Here, a point of the lidar corresponds to a point of the camera, and a point of the camera can correspond to a straight line in three dimensions. For example, a u=x, v=y transformation that does not use a z value can correspond to a top view transformation viewed from the air.

[0067] As long as the camera and lidar are not touched, the transformation remains valid, and the transformation matrix can be found by sampling four or more points, which can enable sensor synchronization.

[0068] The memory (130) may store a code that causes the camera and lidar images to be combined and checked for errors at the end of each drive. The memory (130) may store a code that causes the data to be judged as being intact if the error value is less than a preset value based on the checked error value.

[0069] A code that causes the combined image to be analyzed to determine whether the vehicle number of the object is recognized may be stored in the memory (130).

[0070] The memory (130) may store a code that causes the object's unique key, the object's vehicle number detection time, and the object's license plate coordinates to be stored when the object's license number is recognized. For example, the memory (130) may store a code that causes the object's vehicle number to be recognized, the recognized vehicle number to be matched with the object's unique key, and the object's license plate coordinates to be stored in the form of a data table.

[0071] The license plate detection time of the object may include at least one of the time when the camera starts identifying the license plate and the time when the camera ends identifying the license plate.

[0072] In the memory (130), when an accident regarding an object is detected or an accident information request signal regarding an object is received from an external terminal connected to communication, a code is stored that causes at least one image among a camera image, a lidar image, and a combined image corresponding to the time of the accident to be transmitted to the external terminal.

[0073] More specifically, the memory (130) may store code that causes the system to detect an accident on an object through at least one of a camera and a lidar or to receive an accident information request signal from an external terminal.

[0074] A code that causes a determination to be made as to whether or not to store the vehicle number of the accident vehicle based on a unique key of the accident vehicle and a unique key table in which the vehicle number and unique key are matched can be stored in the memory (130).

[0075] In the memory (130), if the license plate number of the accident vehicle is stored, a code that causes at least one of the camera images and lidar images to be provided from the time the license plate number of the accident vehicle is detected to the time the accident occurred may be stored.

[0076] If the license plate number of the vehicle involved in the accident is not stored in the memory (130), a code may be stored that causes at least one of the camera images and the lidar images to be provided to an external terminal from a preset time before the time of the accident to the time of the accident. For example, the memory (130) may be stored with a code that causes at least one of the camera images and the lidar images to be provided to an external terminal from 20 seconds before the time of the accident to the time of the accident.

[0077] The memory (130) may store code that causes a bounding box to be displayed for each object recognized in each of the camera images and the lidar images, and a unique key to be displayed for each object recognized in the bounding box.

[0078] The memory (130) may store code that causes object information to be stored between time and camera frames. For example, the object information between time and camera frames may include at least one of time, a unique key of an object recognized by the camera, whether a vehicle's license plate is identified, and the position of the object in the photo.

[0079] The memory (130) may store code that causes object information to be stored between time and LIDAR frames. For example, the object information between time and LIDAR frames may include at least one of time, a unique key of an object recognized by LIDAR, object information including the velocity of the object, and the coordinates of the object in LIDAR.

[0080] The memory (130) may store code that causes objects to be synchronized based on object information between time and camera frames and object information between time and lidar frames.

[0081] FIG. 3 is a diagram illustrating examples of lidar images and combined images. More specifically, FIG. 3(a) is a top-view lidar image, FIG. 3(b) is a camera-view lidar image, and FIG. 3(c) is a diagram illustrating an example of a combined image combining a camera image and a camera-view lidar image.

[0082] Referring to FIGS. 3(a) to 3(c), the accident information providing device can display bounding boxes for each object recognized in each of the camera images and the lidar images, and display unique keys for each object recognized in the bounding boxes.

[0083] For example, a bounding box can be displayed for each recognized object, and unique keys 1, 2, 3, and 4 can be provided and displayed for each recognized object in the bounding box.

[0084] The accident information providing device can provide an image displaying a bounding box and a unique key to an external terminal.

[0085] LiDAR images may include top-view LiDAR images, which are images of the road on which an object is located, to track the object's surroundings and trajectory, and camera-view LiDAR images, which are images of the road on which the object is located, to verify the LiDAR's recognition capabilities.

[0086] A combined image may be an image created by combining a camera image and a camera-view lidar image to combine the camera and lidar images of the same object.

[0087] Figure 4 is a diagram illustrating an example of detecting an accident based on lidar.

[0088] Referring to Figure 4, assuming that an accident occurred between Vehicle A and Vehicle B at 1:13:23 PM, Vehicle A may request evidence of the accident. Alternatively, Vehicle B may claim that Vehicle D, which was driving recklessly, was the vehicle that caused the accident.

[0089] While the camera did not capture the moment of the accident, the lidar can detect the impact. Therefore, lidar can assist in recording the crash and determining the cause of the accident.

[0090] While lidar records can be provided to prove an accident, their credibility may be limited. Therefore, to enhance the credibility of lidar records, camera footage recorded at the same time should be included.

[0091] However, since camera footage provided must not contain personal information of others, Vehicle A may be required to pay a masking fee if it provides camera footage containing personal information of others. For example, the personal information may be a vehicle license plate number, and the masking fee may be 600,000 won.

[0092] Accordingly, the accident information provision device can provide accident images by analyzing images from each camera and lidar installed in the vehicle to reduce masking costs.

[0093] For example, if the vehicle number of vehicle A is 12gya1234 and the identification time list is 2000-01-23 13:11:21, 2000-01-23 13:12:34, vehicle A can provide its vehicle number 12gya1234 and request data on the last time it and the accident vehicle were identified in order to reduce masking costs.

[0094] The accident information provision device can provide camera images and lidar images corresponding to 13:05:25 to 13:13:23 based on the vehicle number.

[0095] Accordingly, accidents can be identified through continuous video from cameras and lidar visualizations.

[0096] Additionally, when an autonomous vehicle detects an accident, it can save data space by storing only the video corresponding to the accident, and at the same time provide the video corresponding to the accident when requested later.

[0097] Figure 5 is a flowchart illustrating the sequence of an accident information providing method according to another embodiment of the present invention.

[0098] The accident information provision method described below can be performed by the accident information provision method described above with reference to FIGS. 1 to 4. Accordingly, the content of the embodiments of the present disclosure described above with reference to FIGS. 1 to 4 can be equally applied to the embodiments described below, and any content overlapping with the above description will be omitted. The steps described below do not necessarily have to be performed in order, and the order of the steps can be set in various ways, and the steps can be performed almost simultaneously.

[0099] Referring to FIG. 5, the accident information provision method includes an object unique key assignment step (S100), a combined image creation and object synchronization step (S200), and an image provision step corresponding to the accident time (S300).

[0100] The object unique key assignment step (S100) is a step of detecting an object in a lidar image received from the lidar and assigning a unique key to the object. Here, the object may be a vehicle with a unique license plate number.

[0101] The combined image generation and object synchronization step (S200) is a step of generating a combined image by combining camera images and lidar images received from a camera, and synchronizing objects within the combined image. Here, the lidar image may be an image captured over a wider range than the camera image.

[0102] The step of providing an image corresponding to the time of an accident (S300) is a step of transmitting at least one image among a camera image, a lidar image, and a combined image corresponding to the time of the accident to the terminal when an accident regarding an object is detected or an accident information request signal regarding an object is received from an external terminal connected to communication.

[0103] Figures 6 to 8 are drawings illustrating detailed steps for some steps of the accident information provision method illustrated in Figure 5.

[0104] Referring to FIG. 6, the object unique key assignment step (S100) may include a lidar object detection step (S110), a step of determining whether the object is detected for the first time (S120), and a step of assigning a unique key to the object (S130).

[0105] The object detection step (S110) of the lidar may be a step of detecting an object included in the lidar image received from the lidar.

[0106] The step (S120) of determining whether the object is detected for the first time may be a step of determining whether the object is detected for the first time. If the detected object is detected for the first time, a step of assigning a unique key to the object (S130) may be performed, and if the detected object is not detected for the first time, a step of detecting an object using the lidar (S110) may be performed.

[0107] The step of assigning a unique key to an object (S130) may be a step of assigning a unique key to an object when the object is a first-detected object.

[0108] Referring to FIG. 7, the combined image generation and object synchronization step (S200) may include a camera object detection step (S210), a combined image generation step (S220), a step of determining whether recognition of a vehicle number is possible (S230), and a step of recording the vehicle number detection time and license plate coordinates (S240).

[0109] The object detection step (S210) of the camera may be a step of detecting an object within the camera image.

[0110] The combined image generation step (S220) may be a step of generating a combined image by combining an object in a camera image and an object in a lidar image through a sensor fusion algorithm for fusing information from multiple sensors.

[0111] The step (S230) of determining whether the license plate number can be recognized may be a step of analyzing the combined image to determine whether the license plate number of the object is recognized. If the license plate number of the object is recognized by analyzing the combined image, the step (S240) of recording the license plate number detection time and license plate coordinates may be performed, and if the license plate number of the object is not recognized, the step (S210) of detecting the object by the camera may be performed.

[0112] The vehicle number detection time and license plate coordinate recording step (S240) may be a step of storing the vehicle number detection time of the object and the license plate coordinates of the object when the vehicle number of the object is recognized.

[0113] The combined image generation and object synchronization step (S200) may further include a step of recognizing a vehicle number for a synchronized object, matching the recognized vehicle number with a unique key assigned to the object, and storing the result in the form of a data table.

[0114] Referring to FIG. 8, the video provision step (S300) corresponding to the time of an accident may include an accident detection step (S310), a step of determining whether a vehicle number is stored in a unique key table (S320), a video provision step (S330) corresponding to the time of vehicle number detection to the time of accident occurrence, and a video provision step (S340) from a preset time before the time of accident occurrence to the time of accident occurrence.

[0115] The accident detection step (S310) may be a step for detecting an accident involving an object through at least one of a camera and a lidar, or for receiving an accident information request signal from an external terminal. For example, the accident information request signal may be a video request signal for a certain period of time or a video request signal for the time at which an impact was detected.

[0116] The step (S320) of determining whether the license plate number is stored in the unique key table may be a step of determining whether the license plate number of the accident vehicle is stored in the unique key table based on the unique key of the accident vehicle and the unique key table in which the license plate number and the unique key are matched. If the license plate number of the accident vehicle is stored in the unique key table, a step (S330) of providing an image corresponding to the time from the time of license plate detection to the time of accident may be performed, and if the license plate number of the accident vehicle is not stored in the unique key table, a step (S340) of providing an image from a preset time before the time of accident to the time of accident may be performed.

[0117] The step (S330) of providing images corresponding to the time from the time of vehicle number detection to the time of accident occurrence may be a step of providing at least one image among the camera images and lidar images from the time the vehicle number of the accident vehicle is detected to the time of accident occurrence, if the vehicle number of the accident vehicle is stored.

[0118] The step (S340) of providing images from a preset time before the time of the accident to the time of the accident may be a step of providing at least one image from among the camera images and lidar images from a preset time before the time of the accident to the time of the accident to an external terminal if the license plate number of the accident vehicle is not stored.

[0119] Those skilled in the art will appreciate that the present disclosure can be easily modified into other specific forms based on the above description without changing the technical spirit or essential characteristics of the present disclosure. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The scope of the present disclosure is indicated by the following claims, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present disclosure. The scope of the present application is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present application.

[0120] The form for carrying out the invention is substantially the same as the best form for carrying out the invention.

[0121] The present invention can be applied to autonomous vehicle technology and thus has industrial applicability.

Claims

1. A method performed by a system connected to a camera installed in a vehicle to have a first field of view and a lidar installed in the vehicle to have a second field of view wider than the first field of view, a) A step of detecting an object in a lidar image received from the lidar and assigning a unique key to the object; b) a step of combining the camera image received from the camera and the lidar image to create a combined image, and synchronizing the object within the combined image; and c) A method for providing vehicle identification and accident information, comprising a step of transmitting at least one image among the camera image, the lidar image, and the combined image corresponding to the time of the accident to the external terminal when detecting an accident regarding the object or receiving an accident information request signal regarding the object from an external terminal connected to communication.

2. In paragraph 1, The above object is a vehicle with a unique vehicle number, A method for providing vehicle identification and accident information, wherein the step b) above recognizes the vehicle number for the synchronized object, matches the recognized vehicle number with a unique key assigned to the object, and stores the result in the form of a data table.

3. In paragraph 1, Step a) above, a-1) A step of detecting the object included in the lidar image received from the lidar; a-2) a step of determining whether the above object is a first detected object; and a-3) A method for providing vehicle identification and accident information, comprising a step of assigning a unique key to the object when the object is a first detected object.

4. In paragraph 1, Step b) above, b-1) A step of detecting the object in the camera image; b-2) A step of generating the combined image by combining the object in the camera image and the object in the lidar image through a sensor fusion algorithm for fusing information from multiple sensors; b-3) A step of analyzing the combined image to determine whether the vehicle number of the object is recognized; and b-4) A method for providing vehicle identification and accident information, comprising the step of storing the vehicle number detection time of the object and the license plate coordinates of the object when the vehicle number of the object is recognized.

5. In paragraph 1, Step c) above, c-1) A step of detecting an accident regarding the object through at least one of the camera and the lidar or receiving an accident information request signal from the external terminal; c-2) A step of determining whether the vehicle number of the accident vehicle is stored in the unique key table based on the unique key of the accident vehicle and the unique key table in which the vehicle number and the unique key are matched; and c-3) A method for providing vehicle identification and accident information, comprising the step of providing at least one of the camera image and the lidar image from the time the license plate number of the accident vehicle is detected to the time the accident occurred, if the license plate number of the accident vehicle is stored, and providing at least one of the camera image and the lidar image from a preset time before the time the accident occurred to the time the accident occurred to the external terminal, if the license plate number of the accident vehicle is not stored.

6. In paragraph 1, A method for providing vehicle identification and accident information, wherein the above lidar image is an image captured over a wider range than the above camera image.

7. In paragraph 1, The above lidar image is, Including a top-view lidar image, which is an image looking from above at the road where the object is located, to track the surroundings and trajectory of the object, and a camera-view lidar image to check the recognition capability of the lidar, The above combined image is, A method for providing vehicle identification and accident information, wherein the image is generated by combining the camera image and the camera view lidar image to combine the image of the same object by the camera and the lidar.

8. In paragraph 1, A method for providing vehicle identification and accident information, comprising: displaying a bounding box for each object recognized in each of the camera image and the lidar image, and displaying a unique key for each of the recognized objects in the bounding box.

9. A communication module that transmits and receives information with at least one of a camera installed in a vehicle to have a first field of view and a lidar installed in the vehicle to have a second field of view wider than the first field of view; at least one processor; and A memory electrically connected to the processor and storing at least one code to be executed by the processor, The above memory, when executed through the processor, causes the processor to: A vehicle identification and accident information providing system, which detects an object in a lidar image received from the lidar, assigns a unique key to the object, combines the camera image received from the camera and the lidar image to generate a combined image, synchronizes the object in the combined image, and, when an accident with respect to the object is detected or an accident information request signal for the object is received from an external terminal connected to communication, stores a code that causes at least one of the camera image, the lidar image, and the combined image corresponding to the time of the accident to be transmitted to the external terminal.

10. In paragraph 9, The above object is a vehicle with a unique vehicle number, The above memory causes the processor to: A vehicle identification and accident information provision system that stores a code that causes the vehicle number for the synchronized object to be recognized, and the recognized vehicle number to be matched with a unique key assigned to the object and stored in the form of a data table.

11. In paragraph 9, The above memory causes the processor to: A vehicle identification and accident information providing system that detects the object included in the lidar image from the lidar, determines whether the object is a first-detected object, and stores a code that causes the object to be assigned a unique key if the object is a first-detected object.

12. In paragraph 9, The above memory causes the processor to: A vehicle identification and accident information providing system, which detects the object in the camera image, combines the object in the camera image and the object in the lidar image through a sensor fusion algorithm for fusing information from a plurality of sensors to generate the combined image, analyzes the combined image to determine whether the license plate number of the object is recognized, and, if the license plate number of the object is recognized, stores a code that causes the time of detection of the license plate number of the object and the coordinates of the license plate of the object to be stored.

13. In paragraph 9, The above memory causes the processor to: A vehicle identification and accident information providing system, which detects an accident for the object through at least one of the camera and the lidar or receives an accident information request signal from the external terminal, determines whether the license plate number of the accident vehicle is stored in the unique key table based on the unique key of the accident vehicle and the unique key table in which the license plate number and the unique key are matched, and if the license plate number of the accident vehicle is stored, provides at least one image from the time of detecting the license plate number of the accident vehicle to the time of occurrence of the accident, and if the license plate number of the accident vehicle is not stored, stores a code that causes to provide at least one image from the camera image and the lidar image from a preset time before the time of occurrence of the accident to the time of occurrence of the accident to the external terminal.

14. In paragraph 9, A vehicle identification and accident information provision system, wherein the above lidar image is an image captured over a wider range than the above camera image.

15. In paragraph 9, The above lidar image is, Including a top-view lidar image, which is an image looking from above at the road where the object is located, to track the surroundings and trajectory of the object, and a camera-view lidar image to check the recognition capability of the lidar, The above combined image is, A vehicle identification and accident information providing system, wherein the image is generated by combining the camera image and the camera view lidar image to combine the image of the same object by the camera and the lidar.

16. In paragraph 9, The above memory causes the processor to: A vehicle identification and accident information providing system that displays bounding boxes for each of the objects recognized in the camera image and the lidar image, and stores code that causes a unique key for each of the recognized objects to be displayed in the bounding boxes.

Images