Vehicle safety diagnosis system using ultra-wide-angle multi-camera for vehicle

Abstract

Embodiments provide a vehicle safety diagnosis system using an ultra-wide-angle multi-camera for a vehicle. A vehicle safety diagnosis system according to an embodiment of the present invention may comprise: a first camera installed to acquire first image data for an image capturing area including a front area of a vehicle, a left-side area and a right-side area of the vehicle, and an interior area of the vehicle by using an ultra-wide-angle lens; an electronic device for receiving the first image data from the first camera, and correcting the first image data to generate a first corrected image; and a display for outputting the first corrected image, wherein the left side of the vehicle refers to an area where a left side mirror appears through a driver-side window of the vehicle, the right side of the vehicle refers to an area where a right side mirror appears through a passenger-side window of the vehicle, the interior area of the vehicle refers to an area where a dashboard of the vehicle appears, and the electronic device further comprises a memory for storing the first corrected image.

Description

Vehicle safety diagnosis system using ultra-wide-angle multi-camera for vehicles

[0001] Embodiments of the present invention relate to a vehicle safety diagnosis system using an ultra-wide-angle multi-camera for a vehicle.

[0002] With the recent proliferation of video-based security and safety in everyday life, including through CCTV and black boxes, DVR (Digital Video Recorder) products are being developed to achieve this goal. In particular, active research is underway into vehicle black box products that record the pedal box, providing crucial evidence to prove driver negligence in the event of a sudden acceleration accident.

[0003] In particular, recorded footage captured by vehicle DVR products often serves as important evidence proving whether there was a defect in the vehicle in the event of a vehicle accident, such as a rear-end collision or sudden acceleration. However, conventional vehicle DVR products often lack the evidentiary power to prove that there was no driver negligence at the time of the vehicle accident because they record footage by only recording the front area (R_F) or the rear of the vehicle (10) or only recording the pedal box of the vehicle.

[0004] The background technology described above is technology that the inventor possessed or acquired in the process of deriving the disclosure of the present application, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the present application.

[0005] The embodiments provide a vehicle safety diagnosis system using an ultra-wide-angle multi-camera for a vehicle.

[0006] The technical tasks to be achieved in the embodiments are not limited to those mentioned above, and other technical tasks not mentioned can be considered by a person having ordinary skill in the art from the various embodiments described below.

[0007] According to one embodiment of the present invention, a vehicle safety diagnosis system comprises: a first camera installed to acquire first image data for a shooting area including a front area of ​​a vehicle, a left-side area and a right-side area of ​​the vehicle, and an interior area of ​​the vehicle using an ultra-wide-angle lens; an electronic device that receives the first image data from the first camera and corrects the first image data to generate a first corrected image; and a display that outputs the first corrected image; wherein the left-side area of ​​the vehicle refers to an area in which a left-side mirror appears through a driver's side window of the vehicle, the right-side area of ​​the vehicle refers to an area in which a right-side mirror appears through a passenger's side window of the vehicle, and the interior area of ​​the vehicle refers to an area in which a dashboard of the vehicle appears, and the electronic device may further include a memory that stores the first corrected image.

[0008] The vehicle further includes a second camera installed to obtain second image data for a pedal box area, wherein the electronic device receives the second image data, generates a second corrected image based on the second image data, and synchronizes the second corrected image with the first corrected image to output the second corrected image through the display.

[0009] The above electronic device can generate a corrected image by correcting distortion occurring in an edge area of ​​the first image data acquired from the first camera.

[0010] The angle of view of the first camera may be characterized as being 210 to 230 degrees.

[0011] The above first camera may be characterized in that it is installed at the lower part of the rearview mirror of the vehicle.

[0012] According to embodiments, a vehicle ultra-wide-angle multi-camera system simultaneously captures images of the front area of ​​a vehicle, the two side areas, and the interior area of ​​the vehicle (including the interior and the vehicle's instrument panel) with a single camera and stores the images, thereby accurately transmitting information on the driver's steering wheel operation and the vehicle's instrument panel display in the event of a vehicle accident.

[0013] The effects that can be obtained from the examples are not limited to the effects mentioned above, and other effects that are not mentioned can be clearly derived and understood by a person having ordinary skill in the art based on the detailed description below.

[0014] The accompanying drawings, which are included as part of the detailed description to aid understanding of the embodiments, provide various embodiments and, together with the detailed description, describe technical features of the various embodiments.

[0015] FIG. 1 is a schematic drawing of a vehicle (10) to which a vehicle safety diagnosis system (30) using multiple cameras according to one embodiment of the present invention is applied.

[0016] Fig. 2 is a drawing showing the interior of the vehicle (10) of Fig. 1.

[0017] Fig. 3 is a drawing showing the interior of the vehicle (10) of Fig. 1 from a different angle.

[0018] Figure 4 is a drawing for explaining the shooting area of ​​the first camera (100).

[0019] Figure 5 is a block diagram for explaining a safety diagnosis system (30) according to one embodiment of the present invention.

[0020] Figure 5 is a block diagram for explaining the safety diagnosis system (30) shown in Figure 1.

[0021] Fig. 6 is a block diagram illustrating the structure of the electronic device (300) of Fig. 5.

[0022] Figure 7 is a schematic diagram illustrating the multilayer neural network (320) of Figure 6.

[0023] FIG. 8 is a flowchart explaining a distortion correction method of an ultra-wide-angle image captured by a first camera (100) according to one embodiment of the present invention.

[0024] FIG. 9 is a drawing exemplarily illustrating a split screen of a display (400) according to one embodiment of the present invention.

[0025] FIG. 10 is a drawing exemplarily showing a screen captured by a first camera (100) according to one embodiment of the present invention.

[0026] The following embodiments combine components and features of the embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, various embodiments may be formed by combining some components and / or features. The order of operations described in various embodiments may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment.

[0027] In the description of the drawings, procedures or steps that may obscure the gist of various embodiments are not described, and procedures or steps that can be understood by a person with ordinary skill in the art are also not described.

[0028] Throughout the specification, when a part is said to "comprising" (or including) a certain component, this does not mean that other components are excluded, but rather that other components can be included, unless specifically stated otherwise. In addition, terms such as "...part," "...unit," and "module" described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware, software, or a combination of hardware and software. In addition, the words "a" or "an," "one," "the," and similar related words may be used in the singular and plural sense in the context of describing various embodiments (especially in the context of the claims below) unless otherwise indicated herein or clearly contradicted by context.

[0029] Hereinafter, embodiments according to various embodiments will be described in detail with reference to the attached drawings. The detailed description disclosed below, together with the attached drawings, is intended to explain exemplary embodiments of various embodiments and is not intended to represent the only embodiment.

[0030] Additionally, specific terms used in various embodiments are provided to aid understanding of the various embodiments, and the use of such specific terms may be changed in other forms without departing from the technical spirit of the various embodiments.

[0031] FIG. 1 is a schematic diagram illustrating a vehicle (10) to which a vehicle safety diagnosis system (30) using multiple cameras according to one embodiment of the present invention is applied. FIG. 2 is a diagram illustrating the interior of the vehicle (10) of FIG. 1. FIG. 3 is a diagram illustrating the interior of the vehicle (10) of FIG. 1 from a different angle. FIG. 4 is a diagram for explaining a photographing area of ​​a first camera (100). FIG. 5 is a block diagram for explaining a safety diagnosis system (30) according to one embodiment of the present invention.

[0032] Referring to FIGS. 1 to 5, a vehicle safety diagnosis system (30) (hereinafter, safety diagnosis system (30)) using multiple cameras may be mounted on a vehicle (10). The safety diagnosis system (30) may include a first camera (100) installed to obtain first image data for a shooting area including a front area of ​​the vehicle (10), a left-side area and a right-side area of ​​the vehicle (10), and an interior area of ​​the vehicle (10) using an ultra-wide-angle lens, a second camera (200) installed to obtain second image data for a pedal box area of ​​the vehicle (10), an electronic device (300) for correcting the first image data and the second image data to generate a first corrected image and a second corrected image, and a display (400) for outputting the first corrected image and the second corrected image.

[0033] In one embodiment, the first camera (100) may be installed to capture a front area (R_F) and both side areas (R_L, R_R) (R_L, R_R) of the vehicle (10). The second camera (200) may be installed to capture a pedal box (R_P) of the vehicle (10). Each of the plurality of cameras may capture still images and moving images.

[0034] In one embodiment, the first camera (100) may be an ultra-wide-angle camera including an ultra-wide-angle lens so as to simultaneously capture the front area (R_F) of the vehicle (10), both side areas (R_L, R_R), the interior of the vehicle, and the instrument panel (15) of the vehicle. The angle of view of the ultra-wide-angle lens of the first camera (100) may vary. In one embodiment, the angle of view of the first camera (100) may be 160 degrees or more. In another embodiment, the angle of view of the first camera (100) may be 200 degrees or more. In yet another embodiment, the angle of view of the first camera (100) may be 220 degrees or more. The shooting angle, or angle of view, of the first camera (100) may vary, and the embodiments of the present invention are not limited to the above examples. Preferably, the angle of view of the first camera (100) according to the embodiments of the present invention may be 210 to 230 degrees.

[0035] Here, the field of view (FOV) of the first camera (100) refers to the angle formed between the two ends of the shooting area acquired by the ultra-wide-angle lens of the first camera (100) and the ultra-wide-angle lens. Referring to FIG. 4, an exemplary shooting area (R) of the first camera (100) is illustrated. When an image captured by a camera with a wide angle of view is corrected to 2D and output, distortion occurs at the edge of the 2D image. Referring additionally to FIG. 10, an image captured by the first camera (100) when the angle of view of the first camera (100) is 220 degrees is exemplarily illustrated. The ultra-wide-angle lens adopted in one embodiment of the present invention may be a fish-eye lens.

[0036] The electronic device (300) can receive an image captured by the first camera (100) and calculate wide-angle distortion information based on pre-stored reference data. The electronic device (300) can generate a left-side (R_L) captured image, a right-side (R_R) captured image, and a front-side (R_F) captured image from the captured image based on the calculated distortion information. The electronic device (300) can divide the generated left-side (R_L) captured image, right-side (R_R) captured image, front-side (R_F) captured image, and internal-side (R_I) captured image into respective images and provide the images.

[0037] Referring to FIGS. 1 to 5, the interior area (R_I) of the vehicle (10), the front area (R_F) of the vehicle (10), and the two side areas (R_L, R_R) of the vehicle (10) can be simultaneously captured by the first camera (100). The interior area (R_I) of the vehicle (10) captured by the first camera (100) may include the steering wheel (14) of the vehicle (10), the gear box (16) of the vehicle (10), the dashboard (12) of the vehicle (10), and the instrument panel (15) of the vehicle (10). Alternatively, the interior area (R_I) of the vehicle (10) captured by the first camera (100) may further include the pedal (17) of the vehicle (10). The first camera (100) can record video of how the driver operated the steering wheel and gear when a vehicle accident occurred by simultaneously photographing the interior area (R_I) of the vehicle (10), the front area (R_F) of the vehicle (10), and the two side areas (R_L, R_R) of the vehicle (10). In particular, the first camera (100) can simultaneously photograph the front area (R_F) of the vehicle (10) and the instrument panel (15) of the vehicle.

[0038] In one embodiment, the first camera (100) may be installed in the rearview mirror (11) of the vehicle (10) so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed in the rearview mirror (11) of the vehicle (10), at least one lens included in the first camera (100) is positioned to face the front area (R_F) of the vehicle (10), so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10).

[0039] In another embodiment, the first camera (100) may be installed on the dashboard (12) of the vehicle (10) so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the dashboard (12) of the vehicle (10), at least one lens included in the first camera (100) is positioned to face the front area (R_F) of the vehicle (10), so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10).

[0040] In another embodiment, the first camera (100) may be installed on the sun visor (18) of the vehicle (10) so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the sun visor (18) of the vehicle (10), at least one lens included in the first camera (100) is positioned to face the front area (R_F) of the vehicle (10), so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10).

[0041] In another embodiment, the first camera (100) may be installed on the driver's seat (21) or the passenger's seat (22) of the vehicle (10) so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the driver's seat (21) or the passenger's seat (22) of the vehicle (10), at least one lens included in the first camera (100) is positioned to face the front area (R_F) of the vehicle (10), so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the driver's seat (21) of the vehicle (10), it may be installed so as to be fixed to a part of the headrest of the driver's seat (21) by a mount (not shown). When the first camera (100) is installed on the passenger seat (22) of the vehicle (10), it may be installed so as to be fixed to a part of the headrest of the passenger seat (22) by a mount (not shown).

[0042] In another embodiment, the first camera (100) may be installed on the ceiling (13) of the vehicle (10) so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the ceiling (13) of the vehicle (10), at least one lens included in the first camera (100) is positioned to face the front area (R_F) of the vehicle (10), so as to simultaneously capture images of the front area (R_F) and both side areas (R_L, R_R) of the vehicle (10), and the interior area (R_I) of the vehicle (10). When the first camera (100) is installed on the ceiling (13) of the vehicle (10), the first camera (100) may be installed so as to be fixed to a part of the ceiling (13) of the vehicle (10) by a stand (not shown).

[0043] In addition, the position and installation angle at which the first camera (100) is fixed and installed on a part of the vehicle (10) may vary, and the embodiments of the present invention are not limited to the examples described above.

[0044] In one embodiment, the second camera (200) may be installed to capture a pedal box area (R_P), which is an area including the pedal (17).

[0045] As described above, the angle of view and installation position of the first camera (100) of the present invention can be determined in various ways within a range that can capture the front area (R_F) of the vehicle (10), both side areas (R_L, R_R), the interior of the vehicle, and the instrument panel of the vehicle.

[0046] In one embodiment, each of the plurality of cameras (e.g., the first camera (100) and the second camera (200)) may include one or more lenses, image sensors, and image signal processors. Although FIG. 1 illustrates the safety diagnosis system (30) as including two cameras, the present invention is not limited thereto. If necessary, the safety diagnosis system (30) may include three or more cameras.

[0047] Each of the plurality of cameras included in the safety diagnosis system (30) can be installed in the vehicle (10) to capture images and generate image data. The image data may refer to data optically recorded by an image sensor of a specific object or scene. The first camera (100) and the second camera (200) can generate first image data and second image data, respectively. Each image data can include time information corresponding to the time of capturing the image in the form of metadata. The metadata can be stored in a format such as IPTC (International Press Telecommunications Council), XMP (Extensible Metadata Platform from Adobe), EXIF ​​(Exchangeable Image File Format).

[0048] In one embodiment, each of the cameras included in the safety diagnosis system (30) may include a flash that irradiates a light source to a designated location and a sensor that detects the illuminance of the designated location. The flash may be installed to irradiate a light source to, for example, a pedal box (R_P) area. The safety diagnosis system (30) may control the flash to operate when the illuminance measured by the illuminance detection sensor falls below a preset threshold. Through this, even when any one of the multiple cameras included in the safety diagnosis system (e.g., the second camera (200)) captures a dark place, the safety diagnosis system (30) can obtain high-quality image data.

[0049] In one embodiment, a sensing module (not shown) including a plurality of sensors may be installed on the pedal of the vehicle (10). The sensors of the sensing module may detect pressure applied to the pedal of the vehicle (10) and generate an electrical signal. The sensing module may generate sensing data including the electrical signal generated by the sensor and time information corresponding to the time at which pressure was applied to the pedal of the vehicle (10).

[0050]

[0051] Fig. 6 is a block diagram illustrating the structure of the electronic device (300) of Fig. 5. Fig. 7 is a schematic diagram illustrating the multilayer neural network (320) of Fig. 6.

[0052] Referring to FIG. 3, the electronic device (300) may include a data preprocessing unit (310), a multilayer neural network (320), a learning engine (330), and a memory (340).

[0053] Referring further to FIG. 5, the electronic device (300) may include a multilayer neural network (320) and a learning engine (330) to correct distortion of an ultra-wide-angle image captured by the first camera (100). The learning engine (330) may perform pre-supervised learning on the multilayer neural network (320) using a plurality of learning data. A multilayer neural network is a predictive model implemented in software or hardware that imitates the computational ability of a biological system by using a large number of artificial neurons (or nodes).

[0054] The learning engine (330) can supervise the multilayer neural network (320) by using learning data that takes multiple distorted segmented area images as input values ​​and uses the corrected segmented area images as output values ​​so that the multilayer neural network (320) can accurately correct the edge distortion area of ​​the ultra-wide-angle photographed image.

[0055] Here, supervised learning refers to learning that uses data containing input values ​​and corresponding output values ​​as training data to find output values ​​corresponding to given input values. It refers to learning that is performed when the correct answer is known. The set of input and output values ​​provided in supervised learning is called training data.

[0056] Referring to FIG. 4, a multilayer neural network (320) may include an input layer, one or more hidden layers, and an output layer.

[0057] In one embodiment, the multilayer neural network (320) may include an input layer that receives input values ​​and has nodes corresponding to the number of components of the first feature vector, a first hidden layer that multiplies each output value of the input layer by a weight, adds a bias, and outputs the result, a second hidden layer that multiplies each output value of the first hidden layer by a weight, adds a bias, and outputs the result, and an output layer that multiplies each output value of the second hidden layer by a weight and outputs the result using an activation function. Although only two hidden layers are illustrated in FIG. 7, one or more hidden layers may include a greater number of hidden layers in addition to the first hidden layer and the second hidden layer.

[0058] For example, the activation function may be a Softmax function, but embodiments of the present invention are not limited thereto, and the activation function may be various other functions, such as a LeRU function. Weights and biases may be continuously updated through supervised learning.

[0059] Specifically, the output vector can be input to a loss function layer connected to the output layer. The loss function layer can output a loss value using a loss function that compares the output vector with the correct vector for each training data. The parameters of the multilayer neural network (320) can be supervised for learning in a direction that reduces the loss value.

[0060]

[0061] For example, the loss function layer can calculate the loss value according to [Mathematical Formula 1]. In [Mathematical Formula 2], N can represent the number of multiple training data, n can represent a natural number identifying the training data, k can represent a natural number identifying the value of the nth training data, nk can represent the kth value of the nth training data, t can represent the correct data, y can represent the output vector, and E can represent the loss value.

[0062]

[0063] Alternatively, the loss function layer can calculate the loss value according to [Equation 2]. In [Equation 2], n represents the number of training data per class, y and j represent identifiers representing classes, C represents a constant value, M represents the number of classes, x_y represents the probability that training data belongs to class y, x_j represents the probability that training data belongs to class j, and L represents the loss value.

[0064] To this end, the data preprocessing unit (310) of the electronic device (300) may preprocess a plurality of image data to generate image analysis data. The plurality of image data may include first image data and second image data. In one embodiment, the data preprocessing unit (310) may extract a plurality of images from each image data by referring to metadata included in each image analysis data. Here, the metadata may include information about the time at which each image data was recorded (acquired). In addition, the metadata may include information about the time at which each of the plurality of images included in each image data was captured (acquired). The plurality of images may include a first image captured at a first time, a second image captured at a second time subsequent to the first time, a third image captured at a third time subsequent to the second time, and an n-th image captured at an n-th time.

[0065] The data preprocessing unit (310) can set a first specific area on the first image.

[0066] For example, the second camera (200) may be installed to capture a pedal box area of ​​the vehicle (10). In this case, the first specific area may be an image area corresponding to an accelerator pedal of the vehicle (10). The data preprocessing unit (310) may set a first feature point based on the first specific area on the first image. In this case, for example, the first feature point may be a point corresponding to a part of a footrest of an accelerator pedal of the vehicle. Alternatively, the first camera (100) may be installed to capture a left side mirror (23) of the vehicle (10). In this case, the first specific area may be an image area corresponding to a left side mirror (23) of the vehicle (10). In this case, for example, the first feature point may be a point corresponding to a part of a left side mirror of the vehicle.

[0067] The data preprocessing unit (310) may also set a second specific region, which is an image region corresponding to the first specific region, on the second image. That is, the data preprocessing unit (310) may set a second specific region, which is an image region corresponding to the accelerator pedal of the vehicle, on the second image. Similarly, the data preprocessing unit (310) may set a second feature point based on the second specific region on the second image. Here, the second feature point, like the first feature point, may be a point corresponding to a part of the footrest of the accelerator pedal of the vehicle, or may be a point corresponding to a part of the left side mirror of the vehicle.

[0068] The data preprocessing unit (310) can generate one or more feature vectors based on a plurality of feature points. The data preprocessing unit (310) can generate a first feature vector based on a first feature point and a second feature point. In addition, the data preprocessing unit (310) can generate a second feature vector based on a second feature point set based on a second image and a third feature point set based on a third image. Similarly, the data preprocessing unit (310) can generate an n-th feature vector based on an n-th image and an n+1-th image.

[0069] In one embodiment, the data preprocessing unit (310) may store multiple weight masks. The data preprocessing unit (310) may recognize a specific area using the weight masks. The weight masks may include multiple reference points. The multiple reference points included in the weight masks may have different values ​​depending on information about the vehicle and the user's body information. Here, each reference point may correspond to a portion of the vehicle's brake pedal or accelerator pedal. Alternatively, the point may correspond to a portion of the vehicle's left side mirror.

[0070] The data preprocessing unit (310) can generate an object vector based on a plurality of reference points. The object vector can be expressed as a sum of a plurality of vectors that point from one reference point to other reference points. In one embodiment, the object vector can be expressed as a sum of a first vector that points from the main reference point to the first sub-reference point and a second vector that points from the main reference point to the second sub-reference point. Here, the main reference point can be a point corresponding to a part of the brake pedal of the vehicle. The first sub-reference point and the second sub-reference point can be points corresponding to a part of the accelerator pedal of the vehicle. The first sub-reference point and the second sub-reference point can be points corresponding to different parts of the accelerator pedal of the vehicle. Alternatively, the first sub-reference point and the second sub-reference point can be points corresponding to a part of the left side mirror of the vehicle.

[0071] In one embodiment, the data preprocessing unit (310) can recognize a specific area using a weighted mask. The data preprocessing unit (310) can generate an object vector based on a plurality of reference points. In one embodiment, the data preprocessing unit (310) can generate an object vector by calculating a first vector from a main reference point to a first sub-reference point and a second vector from the main reference point to a second sub-reference point.

[0072]

[0073] The electronic device (300) can correct distortion of the first image data acquired from the first camera (100). The electronic device (300) can correct distortion occurring at the edge of the screen of an image captured by a wide-angle lens. As illustrated in FIG. 10, the image captured by the first camera (100) is an image captured by a wide-angle lens, and severe distortion may occur at the edge of the image. The electronic device (300) can correct the image in which distortion has occurred and generate multiple split-screen images.

[0074] In one embodiment, the electronic device (300) may divide the screen of the first image data captured by the first camera (100) and correct the distortion of each of the divided images to generate a plurality of images. For example, the electronic device (300) may correct the distortion of the portion of the first image data in which the front area (R_F) of the vehicle (10) is captured to generate a vehicle front area (R_F) image, the portion of the first image data in which the left-side area (R_L) of the vehicle is captured to generate a vehicle left-side area (R_L) image, the portion of the first image data in which the right-side area (R_R) of the vehicle is captured to generate a vehicle right-side area (R_R) image, and the portion of the first image data in which the interior area (R_I) of the vehicle is captured to generate a vehicle interior image.

[0075] In one embodiment, the display (400) may include four split screens. Each of the four screens may output in real time a vehicle front area (R_F) image, a vehicle left area (R_L) image, a vehicle right area (R_R) image, and a vehicle interior area (R_I) image.

[0076] Additionally, the electronic device (300) can correct the noise of the first image data and the second image data to create a clearer image.

[0077] The electronic device (300) may further include a communication module. The communication module may support the establishment of a wireless communication channel between the safety diagnosis system (30) and an external electronic device (e.g., a user terminal), and the performance of communication through the established communication channel. The communication module may transmit signals or data generated from components of the safety diagnosis system (30) (e.g., the first camera (100) and the second camera (200)) to the external electronic device, or may receive data or signals from the external electronic device and transmit them to each component of the safety diagnosis system (30). The communication module may communicate with external electronic devices via a legacy cellular network, a 5G network, or a next-generation communication network.

[0078] The memory (340) can store various data (e.g., image data) used by at least one component (e.g., the first camera (100) and the second camera (200)) of the safety diagnosis system (30). In one embodiment, the memory (340) can be replaced with an external storage device. For example, the safety diagnosis system (30) has a groove formed so that an external storage device (e.g., an SD (Secure Digital) card) can be inserted, and the external storage device can be accommodated within the groove. Accordingly, image data generated by a plurality of cameras of the safety diagnosis system (30) can be stored in the external storage device.

[0079] The electronic device (300) may further include a synchronization module. The synchronization module may synchronize the second image data acquired from the second camera (200) with the first image data. The synchronization module may synchronize the images based on metadata containing time information corresponding to the shooting times of the first and second image data.

[0080]

[0081] FIG. 8 is a flowchart explaining a distortion correction method of an ultra-wide-angle image captured by a first camera (100) according to one embodiment of the present invention.

[0082] Referring to FIG. 8, first, the electronic device (300) receives first image data captured by the first camera (100) and reference data from the memory (340) (S610).

[0083] Reference data refers to data that includes correction values ​​for each coordinate on the image screen captured by the first camera (100). The correction values ​​of the reference data can be implemented as vector values ​​for each coordinate on the screen. The electronic device (300) can correct the first image data based on the correction values ​​of the reference data.

[0084] The electronic device (300) generates first image data as a left-side (R_L) captured image, a right-side (R_R) captured image, a front-side (R_F) captured image, and an internal-side (R_I) captured image based on the reference data of the first camera (100) (S620).

[0085] The electronic device (300) can generate a segmented image by dividing a specific portion of the first image data based on reference data. The reference data can include region setting values ​​according to specific coordinate values ​​on the image screen of the first image data. The electronic device (300) can generate one or more segmented images based on the region setting values ​​of the reference data.

[0086] The electronic device (300) generates a corrected image by correcting the left-side (R_L) captured image, the right-side (R_R) captured image, the front-side (R_F) captured image, and the internal-side (R_I) captured image based on the reference data of the first camera (100) (S630).

[0087] The electronic device (300) can correct each of the four images (left side (R_L) captured image, right side (R_R) captured image, front area (R_F) captured image, and internal area (R_I) captured image) divided based on the area setting value of the reference data and the correction value according to each coordinate on the image screen.

[0088] The electronic device (300) displays four divided images (left side (R_L) captured image, right side (R_R) captured image, front area (R_F) captured image, and internal area (R_I) captured image) through the display (400) (S640).

[0089] Referring additionally to FIG. 9, among the display split screens, a left-side (R_L) shooting image may be output on a first split screen (D1), a right-side (R_R) shooting image may be output on a second split screen (D2), a front-side (R_F) shooting image may be output on a third split screen (D3), and an interior-side (R_I) shooting image may be output on a fourth split screen (D4). That is, the safety diagnosis system (20) according to one embodiment of the present invention can obtain an image in which all surrounding situations are accurately recorded when a vehicle accident occurs, and through this, the user can simultaneously check the screens of the front, left-side, right-side, and interior directions shot through a single lens.

[0090] The electronic device (300) may be manufactured in the form of at least one hardware chip and mounted on a device. For example, the electronic device (300) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of an existing general-purpose processor (e.g., CPU or application processor) or a graphics-only processor (e.g., GPU) and mounted on the various devices described above.

[0091] In this case, the electronic device (300) may be mounted on one device or may be mounted on separate devices. Meanwhile, the electronic device (300) may be implemented as a software module. When the image processing module is implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable recording medium that can be read by a computer. In addition, in this case, at least one software module may be provided by an operating system (OS) or by a predetermined application. Alternatively, some of at least one software module may be provided by an operating system (OS), and the remaining some may be provided by a predetermined application.

[0092] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0093] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may, independently or collectively, command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0094] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

[0095] Although the embodiments described above have been described with limited drawings, those skilled in the art will appreciate that various technical modifications and variations can be applied based on the above. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components of the described systems, structures, devices, circuits, etc. are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0096] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims described below.

Claims

1. A first camera installed to obtain first image data for a shooting area including a front area of ​​a vehicle, a left area and a right area of ​​the vehicle, and an interior area of ​​the vehicle using an ultra-wide-angle lens; An electronic device that receives the first image data from the first camera and corrects the first image data to generate a first corrected image; and A display for outputting the first correction image; including: The left side of the above vehicle refers to the area where the left side mirror appears through the driver's side window of the above vehicle. The right side of the above vehicle refers to the area where the right side mirror appears through the passenger side window of the above vehicle. The internal area of ​​the above vehicle means the area where the instrument panel of the above vehicle appears, The electronic device further includes a memory for storing the first correction image. Vehicle safety diagnostic system.

2. In paragraph 1, Further comprising a second camera installed to obtain second image data for the pedal box area of ​​the vehicle; The above electronic device, Receive the second image data and generate a second corrected image based on the second image data, Synchronizing the second correction image with the first correction image and outputting it through the display. Vehicle safety diagnostic system.

3. In paragraph 1, The above electronic device, A corrected image is generated by correcting distortion occurring in the edge area of ​​the first image data acquired from the first camera. Vehicle safety diagnostic system.

4. In paragraph 1, The angle of view of the first camera is characterized by being 210 to 230 degrees. Vehicle safety diagnostic system.

5. In paragraph 1, The above first camera, Characterized in that it is installed at the lower part of the room mirror of the above vehicle. Vehicle safety diagnostic system.

Images