Apparatus for recognizing and warning of object in vehicle a-pillar blind spot and method thereof

Abstract

The present invention relates to an apparatus for recognizing and warning of an object in a vehicle A-pillar blind spot and a method thereof. This apparatus for recognizing and warning of an object in a vehicle A-pillar blind spot comprises: a driving determination unit for determining a travel direction of a vehicle in motion by using at least one of a steering wheel angle, a vehicle speed, and a turn signal of the vehicle; a collection unit for collecting image data and electromagnetic wave data of the A-pillar blind spot in the determined travel direction through a camera and a radar sensor installed in the vehicle; a blind spot detection unit for executing a pre-provided blind spot detection algorithm to analyze the collected image data and determine whether an object exists in the A-pillar blind spot; and a warning output unit for outputting a warning in a visual and / or audible manner according to the determination result.

Description

Object recognition and warning device in the blind spot of a vehicle A-pillar and method thereof

[0001] The present invention relates to a device and method for recognizing and warning an object in a blind spot of an A-pillar of a vehicle, and more particularly, to a device and method for recognizing and warning an object in a blind spot of an A-pillar of a vehicle for recognizing and warning an object in the blind spot of an A-pillar of a vehicle.

[0002] A vehicle's A-pillar is a pillar-like structure that supports the windshield on both sides. Its role is to withstand impact in the event of a frontal collision, protecting occupants, and therefore requires a strong design. However, A-pillars can also obstruct the driver's view when turning left or right, potentially contributing to traffic accidents. This risk is particularly high in adverse weather conditions, such as at night or in rain, when visibility is poor.

[0003] According to data from the Road Traffic Authority's Traffic Accident Analysis System (TAAS) for 2018-2019, a total of 8,959 right-turn traffic accidents occurred at intersections during this period, resulting in 218 deaths. The fatality rate of 2.4% is higher than the overall traffic accident average of 1.5%.

[0004] Therefore, although many efforts have been made to solve these problems, it can be said that there have been few cases of them being installed and applied in actual vehicles due to problems such as difficulty in applying the system in terms of implementation and cost, interference with existing devices, and potential danger to drivers in the event of an accident.

[0005] Recent vehicles are increasingly offering optional or mandatory features for safe driving, such as smart cruise control, lane departure warning, front and rear collision warning, rear blind spot warning, and around-view monitoring. These safety features require the addition of technology to detect objects in the vehicle's A-pillar blind spot.

[0006] The technology underlying the invention is disclosed in Republic of Korea Patent Publication No. 10-2023-0155147 (published on November 10, 2023).

[0007] The present invention has been made to solve the above problems, and the technical problem to be achieved by the present invention is to provide a vehicle A-pillar blind spot object recognition and warning device and method thereof, which determines whether a vehicle is turning left or right using a vehicle steering wheel angle, vehicle speed, and a turn signal, recognizes an object existing in the vehicle A-pillar blind spot using a camera and radar sensor installed in the determined direction, and provides visual and auditory warnings.

[0008] In addition, the present invention provides an object recognition and warning device and method for a vehicle A-pillar blind spot that accurately classifies the type of recognized object using an object detection model.

[0009] According to an embodiment of the present invention for achieving such a technical task, a vehicle A-pillar blind spot object recognition and warning device includes: a driving determination unit that determines a driving direction of a vehicle using at least one of a steering wheel angle, a vehicle speed, and a turn signal of a driving vehicle; a collection unit that collects A-pillar blind spot image data and electromagnetic wave data of the determined driving direction through a camera and a radar sensor installed in the vehicle; a blind spot detection unit that analyzes the collected image data by executing a blind spot detection algorithm prepared in advance and determines whether an object exists in the A-pillar blind spot; and a warning output unit that outputs a warning visually and audibly according to the determination result.

[0010] At this time, the blind spot detection unit analyzes the image data and the electromagnetic wave data, and if the presence of an object is detected from either of the two data, it determines that an object exists in the A-pillar blind spot, and if it is determined that the object exists, it inputs the collected image data into a pre-prepared object detection model to recognize the type of the object, and the warning output unit, if the type of object is recognized by the blind spot detection unit, can display the type of the object as an icon or the like and output a warning, and if the type of the object is not recognized, it can display the object as an obstacle and output a warning.

[0011] In addition, the blind spot detection unit executes the blind spot detection algorithm to load the object detection model through a model weight path included in a parameter setting file, inputs an image frame of the video data collected in real time into the object detection model to receive the coordinates of a region of interest (ROI) to be used as a detection area within the frame as a polygon, calculates an overlap between a predefined object box and the region of interest polygon and compares it with a threshold value preset in the setting file, and if the comparison result is greater than the threshold value, labels the object included in the frame together with a bounding box and outputs it to recognize the object within the A-pillar blind spot, and the parameter setting file may include a name of the model, a model weight path, and a region of interest (ROI).

[0012] In addition, the driving judgment unit collects at least one piece of information from a vehicle CAN bus, including a steering wheel angle, a vehicle speed, and a turn signal, of a vehicle in motion, and, when the vehicle speed exceeds 0 km / h, compares the size of the steering wheel angle with a preset threshold angle, and, when the size of the wheel angle is greater than the threshold angle as a result of the comparison, determines that the vehicle is turning left or right.

[0013] In addition, the driving judgment unit can determine whether the vehicle will turn left or right based on the - or + sign of the turn signal and the steering wheel angle when the turn signal information is collected, and can determine whether the vehicle will turn left or right based on the sign of the steering wheel angle when the turn signal information is not collected.

[0014] In addition, a method for recognizing and warning an object in a blind spot of an A-pillar of a vehicle according to another embodiment of the present invention includes the steps of: determining a driving direction of a vehicle using at least one of a steering wheel angle, a vehicle speed, and a turn signal of a driving vehicle; collecting image data and electromagnetic wave data of an A-pillar blind spot in the determined driving direction using a camera and a radar sensor installed in the vehicle; analyzing the collected image data by executing a blind spot detection algorithm prepared in advance and determining whether an object exists in the A-pillar blind spot; and outputting a warning visually and audibly according to the determination result.

[0015] At this time, the step of determining whether the object exists may include analyzing the image data and the electromagnetic wave data, and if the presence of an object is detected from either of the data, determining that an object exists within the A-pillar blind spot, and if it is determined that the object exists, inputting the collected image data into a pre-prepared object detection model to recognize the type of the object, and the step of outputting a warning may output a warning by displaying the type of the object as an icon or the like, if the type of the object is recognized, and outputting a warning by displaying the object as an obstacle, if the type of the object is not recognized.

[0016] In addition, the step of determining whether the object exists is performed by executing the blind spot detection algorithm to load the object detection model through a model weight path included in a parameter setting file, inputting an image frame of the image data collected in real time into the object detection model to receive the coordinates of a region of interest (ROI) to be used as a detection area within the frame as a polygon to determine whether the object exists, calculating an overlap between a predefined object box and the region of interest polygon and comparing it with a threshold value preset in the setting file, and if the comparison result is greater than the threshold value, labeling and outputting the object included in the frame together with a bounding box to recognize the object within the A-pillar blind spot, and the parameter setting file includes a name of the model, a model weight path, and a region of interest (ROI).

[0017] In addition, the step of determining the driving direction of the vehicle may include collecting at least one piece of information from a vehicle CAN bus, including a steering wheel angle, a vehicle speed, and a turn signal of the vehicle while the vehicle is in motion, comparing the size of the steering wheel angle with a preset threshold angle when the vehicle speed exceeds 0 km / h, and determining that the vehicle is turning left or right when the size of the wheel angle is greater than the threshold angle as a result of the comparison.

[0018] In addition, the step of determining the driving direction of the vehicle may determine whether the vehicle turns left or right based on the - or + sign of the turn signal and the steering wheel angle when the turn signal information is collected, and may determine whether the vehicle turns left or right based on the sign of the steering wheel angle when the turn signal information is not collected.

[0019] In this way, according to the present invention, a vehicle's left or right turn is determined using the vehicle's steering wheel angle, vehicle speed, and turn signal, and an object existing in the vehicle's A-pillar blind spot is recognized using a camera and radar sensor installed in the determined direction, and a visual and auditory warning is provided to the driver, thereby having the effect of preventing or reducing left or right turn traffic accidents at intersections or child protection zones, which are among the most frequent traffic accidents.

[0020] In addition, according to the present invention, the type of recognized object can be accurately classified using an object detection model and provided visually and audibly, thereby improving driver satisfaction.

[0021] In addition, according to the present invention, it can be extended to a function that automatically stops the vehicle in an emergency when a collision with an object in a blind spot is expected in conjunction with an autonomous emergency braking system.

[0022] In addition, according to the present invention, by warning of an object in the blind spot of the A-pillar in bad weather conditions such as at night or in rainy weather when visibility is poor, it is possible to prevent traffic accidents or reduce human and material damage and social loss by alerting the driver to mistakes or errors.

[0023] FIG. 1 is a block diagram showing an object recognition and warning device in a vehicle A-pillar blind spot according to an embodiment of the present invention.

[0024] FIG. 2 is an exemplary drawing showing the installation locations of a camera and a radar sensor according to an embodiment of the present invention.

[0025] FIG. 3 is a flowchart illustrating the operation flow of a method for recognizing and warning an object in a blind spot of a vehicle A-pillar according to an embodiment of the present invention.

[0026] Figure 4 is a flowchart illustrating the detailed operation flow of step S10 in Figure 3.

[0027] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. In this process, the thickness of lines and the sizes of components depicted in the drawings may be exaggerated for clarity and convenience of explanation.

[0028] Furthermore, the terms described below are defined based on their functions within the present invention, and may vary depending on the intent or custom of the user or operator. Therefore, the definitions of these terms should be based on the overall content of this specification.

[0029] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.

[0030] First, an object recognition and warning device in a vehicle A-pillar blind spot according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

[0031] FIG. 1 is a block diagram showing an object recognition and warning device in a vehicle A-pillar blind spot according to an embodiment of the present invention.

[0032] As shown in Fig. 1, the vehicle A-pillar blind spot object recognition and warning device (100) according to an embodiment of the present invention includes a driving judgment unit (110), a collection unit (120), a blind spot detection unit (130), and a warning output unit (140).

[0033] The driving judgment unit (110) determines the driving direction of the vehicle by using at least one of the steering wheel angle, vehicle speed, and turn signal of the vehicle in motion.

[0034] That is, in the embodiment of the present invention, when a vehicle driving at a speed of 5 km / h to 40 km / h changes its driving direction by turning left or right, the object existing in the A-pillar blind spot is recognized and a warning is issued. Therefore, it is preferable to first determine the driving direction of the vehicle using at least one of the steering wheel angle, vehicle speed, and turn signal.

[0035] In detail, the driving judgment unit (110) collects at least one piece of information from the vehicle CAN bus, including the steering wheel angle, vehicle speed, and turn signal of the vehicle in motion, and if the vehicle speed exceeds 0 km / h, compares the size of the steering wheel angle with a preset threshold angle, and if the size of the wheel angle is greater than the threshold angle as a result of the comparison, determines that the vehicle is turning left or right.

[0036] In addition, the driving judgment unit (110) may determine whether the vehicle will turn left or right based on the - or + sign of the turn signal and the steering wheel angle when turn signal information is collected, and may determine whether the vehicle will turn left or right based on the sign of the steering wheel angle when turn signal information is not collected.

[0037] That is, the driving judgment unit (110) monitors the steering wheel angle and vehicle speed in real time from the vehicle's canvas, and if the vehicle speed is not 0 km / h, compares the size of the steering wheel angle with a critical angle, and if the size of the steering wheel angle is not greater than the critical angle, continues to monitor, and if the size of the steering wheel angle is greater than the critical angle, it can be determined that the vehicle is turning.

[0038] At this time, it is desirable that the direction of rotation be preset so that if the steering wheel angle is positive (+), the vehicle is judged to be turning right, and if it is negative (-), the vehicle is judged to be turning left.

[0039] And, if more turn signal information is collected, the driving judgment unit (110) may compare the size of the steering wheel angle with a critical angle when the vehicle speed is not 0 km / h, and if the size of the steering wheel angle is not greater than the critical angle, continue monitoring, and if the size of the steering wheel angle is greater than the critical angle, determine that the vehicle is turning.

[0040] At this time, it is desirable to pre-set the direction of rotation so that if the turn signal is +1 and the steering wheel angle is positive (+), the vehicle is judged to be turning right, and if the turn signal is -1 and the steering wheel angle is negative (-), the vehicle is judged to be turning left.

[0041] The collection unit (120) collects A-pillar blind spot image data and electromagnetic wave data in the driving direction determined by the driving judgment unit (110) through a camera and radar sensor installed in the vehicle.

[0042] FIG. 2 is an exemplary drawing showing the installation locations of a camera and a radar sensor according to an embodiment of the present invention.

[0043] As illustrated in FIG. 2, the camera (200) may be installed in front of the vehicle's rearview mirror or near the left and right A-pillars, including the side mirrors, depending on the angle of view and the number of pixels.

[0044] Additionally, the radar sensor (300) may be installed in the direction of the blind spot at the lower part of the front exterior of the vehicle, i.e., inside both corners of the vehicle bumper.

[0045] The radar sensor (300) applied in the embodiment of the present invention may be a short-range radar sensor.

[0046] For example, short-range radar sensors can be Frequency-Modulated Continuous Wave (FMCW) radar and Pulse Doppler radar. In this case, FMCW radar transmits a continuous wave signal by frequency-modulating it through an antenna, receives the signal reflected from an object, and calculates the distance and speed to the object by using the frequency difference between the transmitted and received signals due to the time delay of the radio waves. In addition, Pulse Doppler radar can calculate the distance and relative speed to the object by using the propagation delay time and Doppler frequency change of the electromagnetic waves reflected and received from the object.

[0047] Therefore, it is most desirable to collect in real time the image data captured by the camera (200) installed in the direction determined by the driving judgment unit (110) and the electromagnetic wave data detected by the radar sensor (300).

[0048] The blind spot detection unit (130) executes a pre-prepared blind spot detection algorithm to analyze the image data and electromagnetic wave data collected through the collection unit (120) and determines whether an object exists within the A-pillar blind spot.

[0049] That is, the blind spot detection unit (130) analyzes the image data and electromagnetic wave data collected from the collection unit (120), and if the presence of an object is detected from either of the two data, it can be determined that an object exists within the A-pillar blind spot.

[0050] In detail, the presence of an object may be determined to be present within the A-pillar blind spot if the presence of an object is detected through an object box within a polygon of a region of interest by analyzing image data, the presence of an object is detected from transmitted and received electromagnetic wave signals by analyzing electromagnetic wave data, or the presence of an object is determined to be present within the A-pillar blind spot if the presence of an object is detected in both image data and electromagnetic wave data.

[0051] If the blind spot detection unit (130) determines that an object exists in the A-pillar blind spot, it can input the image data collected through the collection unit (120) into a pre-prepared object detection model to recognize the type of the object.

[0052] Additionally, the blind spot detection algorithm may be applied to the YOLO (v8) algorithm, which can detect and recognize objects in real time, but is not limited thereto.

[0053] In detail, by executing a blind spot detection algorithm, an object detection model is loaded through a model weight path included in a parameter setting file, and an image frame of image data collected in real time through a collection unit (120) is input into the object detection model to receive the coordinates of a region of interest (ROI) to be used as a detection area within the frame as a polygon. Then, the overlap between a predefined object box and the region of interest polygon is calculated and compared with a threshold value preset in the setting file, and if the comparison result is greater than the threshold value, the object included in the frame is labeled together with a bounding box and output, thereby recognizing an object within the A-pillar blind spot.

[0054] At this time, the parameter setting file may include the model name (model_name), the model weight path (model_path), and the file containing the region of interest (ROI) (roi_path).

[0055] That is, when the blind spot detection algorithm is executed, the blind spot detection unit (130) loads a parameter setting file and loads an object detection model using the model weight path specified in the setting file. Then, it reads in image frames collected in real time through the collection unit (120). Then, it inputs the image frames into the object detection model to identify objects. At this time, the required objects are selected through the "useful_class" of the setting file.

[0056] At this time, the output of the object detection model is the bounding box of the desired object. Therefore, the region of interest (ROI) to be used as the detection area for the blind spot caused by the A-pillar within the image frame is loaded via the "roi_path" property in the configuration file. The coordinates of the ROI within the image frame are obtained as a polygon. An algorithm is run to calculate the overlap between the predefined object box and the ROI polygon, and the result is compared with the threshold value "per_th" set in the configuration file. If the comparison result is greater than the threshold, the recognized result (object) within the A-pillar blind spot is labeled and output with a red bounding box.

[0057] Lastly, the warning output unit (140) outputs a warning visually and audibly based on the judgment result of the blind spot detection unit (130).

[0058] In detail, the warning output unit (140) can output a warning by displaying the type of object as an icon or the like when the type of object is recognized by the blind spot detection unit, and can output a warning by displaying the object as an obstacle when the type of object is not recognized.

[0059] At this time, the warning output unit (140) may be provided in the form of an audiovisual display for a visual warning, and may be located near the A-pillar or on the dashboard inside the vehicle so as not to interfere with driving and so as to be easily confirmed when turning left or right. In addition, a separate speaker (not shown) or vehicle audio (not shown) may be used for an audible warning.

[0060] Hereinafter, a method for recognizing and warning an object in a blind spot of a vehicle A-pillar according to an embodiment of the present invention will be described with reference to FIGS. 3 and 4.

[0061] FIG. 3 is a flowchart illustrating the operation flow of a method for recognizing and warning an object in a blind spot of a vehicle A-pillar according to an embodiment of the present invention, and the specific operation of the present invention will be described with reference to this.

[0062] According to an embodiment of the present invention, first, the driving determination unit (110) of the object recognition and warning device (100) determines the driving direction of the vehicle by using at least one of the steering wheel angle, vehicle speed, and turn signal of the vehicle in motion (S10).

[0063] That is, in the embodiment of the present invention, when a vehicle driving at a speed of 5 km / h to 40 km / h changes its driving direction by turning left or right, the object existing in the A-pillar blind spot is recognized and a warning is issued. Therefore, it is preferable to first determine the driving direction of the vehicle using at least one of the steering wheel angle, vehicle speed, and turn signal.

[0064] Figure 4 is a flowchart illustrating the detailed operation flow of step S10 in Figure 3.

[0065] At step S10, the driving judgment unit (110) collects the steering wheel angle and vehicle speed of the driving vehicle from the vehicle CAN bus and determines whether the vehicle speed exceeds 0 km / h (S11).

[0066] As a result of the judgment in step S11, if the vehicle speed exceeds 0 km / h, the size of the steering wheel angle is compared with the preset threshold angle (S12).

[0067] As a result of the comparison at step S12, if the size of the steering wheel angle is greater than the threshold angle, it is determined that the vehicle is turning. At this time, if the steering wheel angle is positive (+), it is determined that the vehicle is turning right (S14), and if it is negative (-), it is determined that the vehicle is turning left (S15).

[0068] In addition, at step S10, the driving judgment unit (110) may determine whether the vehicle will turn left or right based on the - or + sign of the turn signal and the steering wheel angle when turn signal information is collected, and may determine whether the vehicle will turn left or right based on the sign of the steering wheel angle when turn signal information is not collected.

[0069] At this time, it is desirable to pre-set the direction of rotation so that if the turn signal is +1 and the steering wheel angle is positive (+), the vehicle is judged to be turning right, and if the turn signal is -1 and the steering wheel angle is negative (-), the vehicle is judged to be turning left.

[0070] Next, the collection unit (120) collects image data and electromagnetic wave data of the A-pillar blind spot in the driving direction determined in step S10 using a camera (200) and radar sensor (300) installed in the vehicle (S20).

[0071] At this time, the camera (200) may be installed in front of the vehicle's rearview mirror or near the left and right A-pillars including the side mirrors, depending on the angle of view and the number of pixels.

[0072] Additionally, the radar sensor (300) may be installed in the direction of the blind spot on the lower part of the front exterior of the vehicle, i.e., inside both corners of the vehicle bumper. The radar sensor (300) applied in the embodiment of the present invention may be a short-range radar sensor.

[0073] Next, the blind spot detection unit (130) executes a pre-prepared blind spot detection algorithm to analyze the image data and electromagnetic wave data collected in step S20 and determines whether an object exists in the A-pillar blind spot (S30).

[0074] That is, step S30 analyzes the image data and electromagnetic wave data collected in step S20, and if the presence of an object is detected from either of the two data, it can be determined that an object exists within the A-pillar blind spot.

[0075] In detail, the presence of an object may be determined to be present within the A-pillar blind spot if the presence of an object is detected through an object box within a polygon of a region of interest by analyzing image data, the presence of an object is detected from transmitted and received electromagnetic wave signals by analyzing electromagnetic wave data, or the presence of an object is determined to be present within the A-pillar blind spot if the presence of an object is detected in both image data and electromagnetic wave data.

[0076] If it is determined in step S30 that an object exists within the A-pillar blind spot, the image data collected in step S20 is input into a pre-prepared object detection model to recognize the type of the object (S40).

[0077] In detail, by executing the blind spot detection algorithm, the object detection model is loaded through the model weight path included in the parameter setting file, and the image frame of the video data collected in real time in step S20 is input into the object detection model, and the coordinates of the region of interest (ROI) to be used as the detection area within the frame are input as a polygon. Then, the overlap between the predefined object box and the region of interest polygon is calculated and compared with the threshold value preset in the setting file, and if the comparison result is greater than the threshold value, the object included in the frame is labeled together with the bounding box and output, so that the object within the A-pillar blind spot can be recognized.

[0078] At this time, the parameter setting file may include the model name (model_name), the model weight path (model_path), and the file containing the region of interest (ROI) (roi_path).

[0079] If the type of object is recognized at step S40, the warning output unit (140) displays the type of the object and outputs a warning (S50).

[0080] And, if the type of object is not recognized at step S40, the warning output unit (140) displays the object as an obstacle and outputs a warning (S60).

[0081] The method for recognizing and warning of an object in a vehicle A-pillar blind spot according to an embodiment of the present invention has been described above.

[0082] The object recognition and warning method in the blind spot of a vehicle A-pillar according to the embodiment of the present invention as described above may be provided in the form of a computer-readable medium suitable for storing computer program commands and data.

[0083] In particular, the computer program of the present invention can execute a step of determining a driving direction of a vehicle by using at least one of a steering wheel angle, a vehicle speed, and a turn signal of a driving vehicle, a step of collecting image data and electromagnetic wave data of an A-pillar blind spot in the determined driving direction by using a camera and a radar sensor installed in the vehicle, a step of analyzing the collected image data by executing a blind spot detection algorithm prepared in advance and determining whether an object exists in the A-pillar blind spot, and a step of outputting a warning visually and audibly according to the determination result.

[0084] Such computer-readable recording media may contain program commands, data files, data structures, etc. alone or in combination, and include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program commands such as ROMs (Read Only Memory), RAMs (Random Access Memory), and flash memory.

[0085] Additionally, computer-readable recording media can be distributed across network-connected computer systems, allowing computer-readable code to be stored and executed in a distributed manner. Furthermore, functional programs, codes, and code segments for implementing the present invention can be readily inferred by programmers skilled in the art to which the present invention pertains.

[0086] According to the present invention as described above, a vehicle can be turned left or right by using a vehicle steering wheel angle, vehicle speed, and turn signal to determine whether a vehicle is turning left or right, and an object existing in the blind spot of the A-pillar of the vehicle can be recognized by using a camera and radar sensor installed in the determined direction, and a visual and auditory warning can be given to the driver, thereby preventing or reducing left or right turn traffic accidents at intersections or child protection zones, which are among the traffic accidents that frequently occur.

[0087] Additionally, by using an object detection model, the type of recognized object can be accurately classified and provided visually and audibly, thereby improving driver satisfaction.

[0088] Additionally, it can be extended to include a function that automatically stops the vehicle in case of an anticipated collision with an object in the blind spot by linking it with an autonomous emergency braking system.

[0089] In addition, by warning of objects in the A-pillar blind spot in bad weather conditions such as at night or during rain when visibility is poor, it can prevent traffic accidents by alerting drivers to mistakes or errors, and reduce human and material damage and social losses.

[0090] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the following claims.

[0091] [Explanation of symbols]

[0092] 100: Object recognition and warning device 110: Driving judgment unit

[0093] 120: Collection unit 130: Blind spot detection unit

[0094] 140: Warning output 200: Camera

[0095] 300: Radar sensor

Claims

1. A driving judgment unit that determines the driving direction of a vehicle by using at least one of the steering wheel angle, vehicle speed, and turn signal of a driving vehicle; A collection unit that collects image data and electromagnetic wave data of the A-pillar blind spot in the determined driving direction through a camera and radar sensor installed in the vehicle; A blind spot detection unit that analyzes the collected image data and electromagnetic wave data by executing a pre-arranged blind spot detection algorithm and determines whether an object exists within the A-pillar blind spot; and A vehicle A-pillar blind spot object recognition and warning device including a warning output unit that outputs a warning visually and audibly based on the above judgment result.

2. In paragraph 1, The above blind spot detection unit, By analyzing the above image data and electromagnetic wave data, if the presence of an object is detected from either of the two data, it is determined that an object exists within the A-pillar blind spot. If it is determined that the above object exists, the collected image data is input into a pre-prepared object detection model to recognize the type of the object. The above warning output section is, When the type of object is recognized from the above blind spot detection unit, a warning is output by displaying the type of object. An object recognition and warning device in a vehicle A-pillar blind spot that displays the object as an obstacle and outputs a warning when the type of the object is not recognized.

3. In paragraph 2, The above blind spot detection unit, By executing the above blind spot detection algorithm, the object detection model is loaded through the model weight path included in the parameter setting file, and the image frame of the image data collected in real time is input into the object detection model to receive the coordinates of the region of interest (ROI) to be used as the detection area within the frame as a polygon. Compute the overlap of the predefined object box and the polygon of the region of interest and compare it with a threshold value preset in the setting file, and if the comparison result is greater than the threshold value, label the object included in the frame with the bounding box and output it to recognize the object within the A-pillar blind spot. The above parameter setting file is a vehicle A-pillar blind spot object recognition and warning device including the model name, model weight path and region of interest (ROI).

4. In paragraph 1, The above driving judgment unit, A vehicle A-pillar blind spot object recognition and warning device that collects at least one piece of information from a vehicle CAN bus, including a steering wheel angle, a vehicle speed, and a turn signal, and compares the size of the steering wheel angle with a preset threshold angle when the vehicle speed exceeds 0 km / h, and determines that the vehicle is turning left or right when the size of the wheel angle is greater than the threshold angle as a result of the comparison.

5. In paragraph 4, The above driving judgment unit, When the above turn signal information is collected, the vehicle's left or right turn is determined based on the turn signal and the - or + sign of the steering wheel angle. An object recognition and warning device in the blind spot of a vehicle's A-pillar that determines whether the vehicle will turn left or right based on the sign of the steering wheel angle when the above turn signal information is not collected.

6. In a method for recognizing and warning an object in a blind spot of a vehicle A-pillar, performed by an object recognition and warning device in a blind spot of a vehicle A-pillar, A step of determining the driving direction of a vehicle by using at least one of the steering wheel angle, vehicle speed, and turn signal of the vehicle in motion; A step of collecting image data and electromagnetic wave data of the A-pillar blind spot in the determined driving direction using a camera and radar sensor installed in the vehicle; A step of analyzing the collected image data and electromagnetic wave data by executing a pre-arranged blind spot detection algorithm and determining whether an object exists within the A-pillar blind spot; and A method for recognizing and warning an object in a blind spot of a vehicle A-pillar, comprising a step of outputting a warning visually and audibly based on the above judgment result.

7. In paragraph 6, The step of determining whether the above object exists is: By analyzing the above image data and electromagnetic wave data, if the presence of an object is detected from either of the two data, it is determined that an object exists within the A-pillar blind spot. If it is determined that the above object exists, the collected image data is input into a pre-prepared object detection model to recognize the type of the object. The steps to output the above warning are: If the type of the above object is recognized, a warning is displayed indicating the type of the object. A method for recognizing and warning an object in a vehicle A-pillar blind spot, which displays the object as an obstacle and outputs a warning when the type of the object is not recognized.

8. In paragraph 6, The step of determining whether the above object exists is: By executing the above blind spot detection algorithm, the object detection model is loaded through the model weight path included in the parameter setting file, and the image frame of the video data collected in real time is input into the object detection model, and the coordinates of the region of interest (ROI) to be used as the detection area within the frame are input as a polygon to determine whether the object exists. Compute the overlap between the predefined object box and the polygon of the region of interest and compare it with a threshold value preset in the setting file, and if the comparison result is greater than the threshold value, label the object included in the frame with the bounding box and output it to recognize the object within the A-pillar blind spot. The above parameter setting file is a method for recognizing and warning objects in a vehicle A-pillar blind spot, including the model name, model weight path, and region of interest (ROI).

9. In paragraph 6, The step of determining the driving direction of the above vehicle is: A method for recognizing and warning an object in a blind spot of a vehicle A-pillar, which collects at least one piece of information from a vehicle CAN bus, including a steering wheel angle, a vehicle speed, and a turn signal, and compares the size of the steering wheel angle with a preset threshold angle when the vehicle speed exceeds 0 km / h, and determines that the vehicle is turning left or right when the size of the wheel angle is greater than the threshold angle as a result of the comparison.

10. In paragraph 9, The step of determining the driving direction of the above vehicle is: When the above turn signal information is collected, the vehicle's left or right turn is determined based on the turn signal and the - or + sign of the steering wheel angle. A method for recognizing and warning an object in a blind spot of a vehicle A-pillar, which determines whether the vehicle will turn left or right based on the sign of the steering wheel angle, when the above turn signal information is not collected.

Images