Collision Avoidance Systems

Abstract

To provide a collision-avoidance system capable of reliably and efficiently preventing contact between a vehicle and a collision avoidance object by detecting the collision avoidance object regardless of attitudes or directions of various types of collision avoidance objects existing around the vehicle being driven, specifying a type thereof, and preventing excessive detection and erroneous detection.SOLUTION: A collision-avoidance system includes: a machine learning setting part 11 executing machine learning on the basis of a learning image including a collision avoidance object to be detected and constructing a machine learning model; an image acquisition part 12 for acquiring an image obtained by photographing a periphery of a vehicle while the vehicle is driven; a machine learning inference part 14 for discriminating whether a collision avoidance object exists around the vehicle from the image acquired by the image acquisition part 12 and the machine learning model; a collision avoidance setting part 15 for acquiring position information of the collision avoidance object detected by the machine learning inference part 14 and determining a collision possibility of the vehicle and the collision avoidance object; and an alarm output part 16 for outputting an alarm to a driver of the vehicle on the basis of the determination of the collision avoidance setting part 15.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a collision avoidance system for preventing collision accidents with various objects to be avoided, including other vehicles and people, while driving various vehicles including various heavy machinery. [Background technology]

[0002] Traditionally, various vehicles, including heavy machinery, are used at civil engineering and construction sites, and there is a need to prevent vehicle contact accidents to ensure the safety of workers operating the vehicles and workers working around the vehicles. For example, Patent Document 1 discloses a vehicle collision prevention device that detects the brake light illumination of a leading vehicle from an image captured in front of the vehicle, detects the deceleration state of the leading vehicle, and determines the possibility of a collision between the leading vehicle and the vehicle itself. Furthermore, Patent Document 2 discloses a construction vehicle equipped with an obstacle detection means for detecting obstacles present behind the self-propelled vehicle body that can move forward and backward, and a collision prevention auxiliary device for executing collision prevention control based on detection information from the obstacle detection means when the vehicle body is moving backward. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 11-39597 [Patent Document 2] Utility Model Registration No. 3219005 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in Patent Document 1, since it is premised on detecting the illumination of the brake lights of the preceding vehicle, the collision avoidance target is limited to a vehicle having brake lights and traveling in front of the vehicle. Therefore, there is a problem that the collision prevention device does not function when only the front or side of the vehicle to be avoided is visible from the vehicle, or when the vehicle to be avoided is stopped. In addition, there is a problem that it is not possible to avoid collisions with various obstacles other than the vehicle, such as stationary objects such as materials placed at the work site, fixed structures such as fences and pillars installed at the work site, and workers, and there is a problem that it lacks versatility and functionality. In Patent Document 2, an obstacle (collision avoidance target) present behind the vehicle is detected by an obstacle detection means, and collision prevention control is executed based on the detection information of the obstacle detection means when the vehicle is moving backwards, so it does not function when the vehicle is moving forward, resulting in a problem of lack of safety. Also, since the obstacle detection means only detects the presence or absence of an obstacle using a 2D laser sensor or the like and cannot identify the type of obstacle, unnecessary (unnecessary) deceleration or stopping is performed, which may reduce work efficiency and result in a problem of lack of practicality.

[0005] The present invention has been made in consideration of the above circumstances, and aims to provide a collision avoidance system that can detect and identify the type of collision avoidance target regardless of the posture or orientation (arrangement) of various collision avoidance targets present around a vehicle while it is being driven, and that can prevent excessive detection and erroneous detection, thereby reliably and efficiently preventing contact between the vehicle and the collision avoidance target. [Means for solving the problem]

[0006] A collision avoidance system according to the present invention that meets the above-mentioned objective is a collision avoidance system that detects a collision avoidance target using image recognition by machine learning while a vehicle is being driven, and prevents a collision between the vehicle and the collision avoidance target, a machine learning setting unit that performs machine learning based on a learning image including the collision avoidance target to be detected and constructs a machine learning model; ,beforean image acquisition unit that acquires an image of the periphery of the vehicle while the vehicle is being driven; a machine learning inference unit that determines whether the collision avoidance target is present around the vehicle based on the image acquired by the image acquisition unit and the machine learning model; a collision avoidance setting unit that acquires position information of the collision avoidance target detected by the machine learning inference unit and determines a possibility of a collision between the vehicle and the collision avoidance target; and an alarm output unit that outputs an alarm to a driver of the vehicle based on a determination by the collision avoidance setting unit. a display unit that displays a real-time video created based on the image; Equipped with One or more horizontal lines that are displayed parallel to a width direction of the vehicle and indicate a distance from the vehicle are drawn on the real-time video, and when the machine learning inference unit determines that the collision avoidance target exists, a rectangular frame that indicates a range of the collision avoidance target is drawn on the real-time video in addition to the horizontal line, and the collision avoidance setting unit determines a possibility of collision between the vehicle and the collision avoidance target from a positional relationship between a bottom side of the rectangular frame and the horizontal line. do.

[0007]

[0008] In the collision avoidance system of the present invention, it is preferable that the collision avoidance setting unit has a vehicle state recognition unit that detects whether a reverse signal is being output from the vehicle and determines whether the vehicle is reversing or not, and that the collision avoidance setting unit disables the output of an alarm by the alarm output unit or switches the type of alarm output by the alarm output unit based on the determination result of the vehicle state recognition unit.

[0009] In the collision avoidance system of the present invention, there may be provided a vehicle state recognition unit that detects whether a reverse signal is being output from the vehicle and determines whether the vehicle is reversing or not, and the collision avoidance setting unit may be capable of selecting whether to disable the output of an alarm by the alarm output unit or switch the type of alarm output by the alarm output unit based on a determination result of the vehicle state recognition unit, or to forcibly stop operation of the vehicle state recognition unit and have the alarm output by the alarm output unit output regardless of whether the vehicle is reversing or not.

[0010] In the collision avoidance system of the present invention, it is preferable that the system has a display unit on which a real-time video created based on the image is displayed, and the real-time video has one or more horizontal lines drawn parallel to the width direction of the vehicle and indicating the distance from the vehicle, and when the machine learning inference unit determines that a collision avoidance target exists, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to the horizontal lines.

[0011] In the collision avoidance system according to the present invention, The image and The real-time video is preferably stored in the machine learning settings unit.

[0012] In the collision avoidance system according to the present invention, it is further preferable that the images are stored for a predetermined period going back from when the machine learning inference unit determined that the collision avoidance target exists.

[0013]

[0014] In the collision avoidance system of the present invention, it is further preferable that the collision avoidance setting unit adjusts a collision determination area in which it is determined that there is a possibility of collision between the vehicle and the collision avoidance target by expanding or contracting the length of the horizontal line.

[0015] A collision avoidance method for detecting a collision avoidance target using image recognition by machine learning while a vehicle is being driven and preventing a collision between the vehicle and the collision avoidance target includes a machine learning process for performing machine learning based on a learning image including the collision avoidance target to be detected and constructing a machine learning model, an image acquisition process for acquiring images taken of the surroundings of the vehicle while the vehicle is being driven, an object detection process for determining whether the collision avoidance target is present in the surroundings of the vehicle from the images acquired in the image acquisition process and the machine learning model, a collision determination process for acquiring position information of the collision avoidance target detected in the object detection process and determining the possibility of a collision between the vehicle and the collision avoidance target, and an alarm output process for outputting an alarm to a driver of the vehicle when it is determined in the collision determination process that there is a possibility of a collision between the vehicle and the collision avoidance target.

[0016] In the above collision avoidance method, it is preferable that in the collision determination step, it is determined whether or not the vehicle is moving in reverse, and the output of the warning by the warning output step is invalidated or the type of warning to be output by the warning output step is switched.

[0017] In the above-described collision avoidance method, in the collision determination step, it may be possible to select whether or not the vehicle is reversing and to disable output of an alarm by the alarm output step, or to switch the type of alarm output by the alarm output step, or to select whether or not an alarm is output by the alarm output step regardless of whether the vehicle is reversing or not.

[0018] In the above collision avoidance method, it is preferable that the method further comprises a video display process for displaying a real-time video created based on the image, wherein one or more horizontal lines are drawn on the real-time video parallel to the width direction of the vehicle and indicating the distance from the vehicle, and when it is determined in the object detection process that a collision avoidance object exists, a rectangular frame indicating the range of the collision avoidance object is drawn on the real-time video in addition to the horizontal lines.

[0019] In the above collision avoidance method, it is preferable that in the collision determination step, a possibility of a collision between the vehicle and the object for collision avoidance is determined based on a positional relationship between a bottom side of the rectangular frame and the horizontal line.

[0020] In the above collision avoidance method, it is further preferable that in the collision determination step, a collision determination area in which it is determined that there is a possibility of collision between the vehicle and the collision avoidance target is adjusted by expanding or contracting a length of the horizontal line. Effect of the Invention

[0021] The collision avoidance system of the present invention can detect a collision avoidance object using image recognition based on machine learning while the vehicle is being driven, obtain position information of the collision avoidance object, and output an alarm to the driver of the vehicle when it determines that there is a possibility of a collision between the vehicle and the collision avoidance object.This prevents excessive detection and erroneous detection, and can reliably and efficiently prevent contact with the collision avoidance object, thereby providing excellent safety.

[0022] In the collision avoidance system according to the present invention, the image acquisition unit is mounted on the vehicle and has one or more cameras that capture images of at least the rear of the vehicle and the blind spot of the driver. caseThis makes it possible to prevent collision avoidance targets from being overlooked, resulting in excellent reliability in collision avoidance.

[0023] In the collision avoidance system of the present invention, when the output of an alarm is disabled or the type of alarm to be output is switched based on the result of determining whether the vehicle is reversing or not, it is possible to prevent unnecessary (excessive) alarms from being output and effectively avoid a collision.

[0024] In the collision avoidance system of the present invention, if it is possible to select whether to disable the output of an alarm or switch the type of alarm to be output based on the result of determining whether the vehicle is reversing or not, or to output an alarm regardless of whether the vehicle is reversing or not, the conditions for outputting an alarm can be switched (selected) as needed, making the system highly versatile.

[0025] In the collision avoidance system of the present invention, a real-time video is created based on images acquired while the vehicle is being driven, and when it is determined that a collision avoidance object is present around the vehicle, a rectangular frame indicating the range of the collision avoidance object is drawn on the real-time video in addition to a horizontal line indicating the distance from the vehicle. This prevents the driver of the vehicle from overlooking the collision avoidance object, and allows the driver to accurately grasp the distance from the vehicle to the collision avoidance object, enabling them to take prompt action to avoid a collision.

[0026] In the collision avoidance system of the present invention, when the images and real-time videos acquired by the image acquisition unit are stored in the machine learning setting unit, machine learning is performed using those images and real-time videos, and the detection accuracy of collision avoidance targets is continuously improved.

[0027] In the collision avoidance system of the present invention, when images are saved for a period going back a predetermined time from when the machine learning inference unit determined that a collision avoidance target exists, there is a possibility that images will be acquired from a period (time zone) when a collision avoidance target was not detected, even though a collision avoidance target actually existed, and by using these images for machine learning, the detection accuracy of the collision avoidance target is further improved.

[0028] In the collision avoidance system according to the present invention, the possibility of a collision between the vehicle and the object for collision avoidance is determined based on the positional relationship between the bottom side of the rectangular frame and the horizontal line. So Regardless of the posture or orientation of the collision avoidance object, the possibility of a collision between the collision avoidance object and the vehicle is determined with high accuracy according to the distance from the vehicle to the collision avoidance object, and a collision is reliably avoided.

[0029] In the collision avoidance system of the present invention, when the length of the horizontal line is expanded or contracted to adjust the collision judgment area in which it is determined that there is a possibility of a collision between the vehicle and the object for collision avoidance, the possibility of a collision between the vehicle and the object for collision avoidance is effectively determined only in the intended area, excessive warnings are reduced, and the burden on the driver is also lightened. [Brief description of the drawings]

[0030] [Figure 1] 1 is a block diagram showing a configuration of a collision avoidance system according to an embodiment of the present invention; [Diagram 2] This is an image of one frame from a real-time video displayed on the display unit of the collision avoidance system. [Diagram 3] 4 is an explanatory diagram of a collision determination process in the collision avoidance method executed in the collision avoidance system. FIG. [Figure 4] 10 is a flowchart showing a process for saving real-time video in the collision avoidance method executed in the collision avoidance system. [Diagram 5] 4 is a flowchart showing a collision avoidance process in the collision avoidance method executed in the collision avoidance system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Next, with reference to the attached drawings, an embodiment of the present invention will be described for understanding the present invention. Hereinafter, a collision avoidance system 10 and a collision avoidance method according to an embodiment of the present invention will be described with reference to FIGS. The collision avoidance system 10 and the collision avoidance method prevent contact accidents with various collision avoidance targets including other vehicles and people while driving various vehicles including various heavy machinery. Specifically, while driving the vehicle, it is possible to detect a collision avoidance target using image recognition by machine learning and prevent a collision between the vehicle and the collision avoidance target. Here, while driving the vehicle includes not only while the vehicle is moving forward or backward, but also while the vehicle is stopped.

[0032] As shown in FIG. 1, the collision avoidance system 10 includes a machine learning setting unit 11 that performs machine learning based on learning images that include collision avoidance objects to be detected (e.g., people, vehicles including heavy machinery, obstacles that impede passage, etc.) and constructs a machine learning model. The collision avoidance system 10 also includes an image acquisition unit 12 that acquires images of the surroundings of the vehicle while the vehicle is being driven, and a display unit 13 that displays a real-time video created based on the images acquired by the image acquisition unit 12. The collision avoidance system 10 is equipped with a machine learning inference unit 14 that determines whether a collision avoidance target is present around the vehicle based on the image acquired by the image acquisition unit 12 and a machine learning model, and a collision avoidance setting unit 15 that acquires position information of the collision avoidance target detected by the machine learning inference unit 14 and determines the possibility of a collision between the vehicle and the collision avoidance target. The collision avoidance system 10 also includes an alarm output unit 16 that outputs an alarm to the driver of the vehicle based on the determination of the collision avoidance setting unit 15. Furthermore, the collision avoidance system 10 has a vehicle state recognition unit 17 that detects whether or not a reverse signal is being output from the vehicle to determine whether or not the vehicle is moving backward. This collision avoidance system 10 can be easily retrofitted (installed) on existing vehicles, but can also be incorporated during the vehicle manufacturing stage.

[0033] Here, a part of the collision avoidance system 10 can be configured with a computer. That is, a program for executing the collision avoidance method is installed in the computer, and the CPU and GPU of the computer execute the program, thereby making the computer function as the above-mentioned machine learning setting unit 11, image acquisition unit 12, machine learning inference unit 14, collision avoidance setting unit 15, and vehicle state recognition unit 17. As the display unit 13, a display integrated with or connected to the computer is preferably used, but if the vehicle is equipped with a car navigation system or a television, the monitor of the display unit 13 may be used as the display unit 13, or the windshield of the vehicle having a display function may be used as the display unit 13. The machine learning setting unit 11 constructs a machine learning model using a conventionally known algorithm such as a neural network. The image acquisition unit 12 is mounted on the vehicle and has one or more cameras that capture at least the rear of the vehicle and areas that are blind spots for the driver (for example, the rear parts on both the left and right sides of the vehicle). The image acquisition unit 12 may have cameras that capture the front and left and right sides of the vehicle, and the number and arrangement of the cameras are appropriately selected according to the capture range (viewing angle) of each camera, etc. When the image acquisition unit 12 has multiple cameras, the display unit 13 may display multiple real-time videos created by the image acquisition unit 12 based on images captured by each camera side by side, or one real-time video may be selected sequentially from the multiple real-time videos at a predetermined time interval and displayed. Alternatively, a real-time video in a direction determined by the collision avoidance setting unit 15 to be a collision possibility may be preferentially displayed.

[0034] As shown in FIG. 2, a plurality of (three in this example) horizontal lines 20 are drawn in the real-time video, which are displayed parallel to the width direction of the vehicle and indicate the distance from the vehicle. When the machine learning inference unit 14 determines that a collision avoidance target (a person in this example) 21 exists, a rectangular frame 22 indicating the range of the collision avoidance target 21 is drawn in addition to the horizontal lines 20 in the real-time video. The display color of each horizontal line is appropriately selected, but for example, the horizontal lines may be color-coded in order of proximity to the vehicle, such as red, yellow, and green, to indicate danger, or all may be the same color. In addition, the number of horizontal lines and the distance from the vehicle to each horizontal line (the arrangement interval of the horizontal lines) are appropriately selected. An alarm can be issued when the collision avoidance target 21 approaches the vehicle and the collision avoidance target 21 crosses any of the horizontal lines 20. In this embodiment, the distance from the vehicle to each horizontal line 20 is displayed, but this display may be omitted, or the driver may operate to switch between display and non-display. Furthermore, in this embodiment, auxiliary lines 23 are drawn alternately with the horizontal lines 20, allowing the driver to know the distance from the vehicle in detail, but the number, arrangement interval, length, display color, etc. of the auxiliary lines may be selected appropriately and may be omitted. In addition, the display color of the rectangular frame 22 is selected appropriately, but the display color of the rectangular frame 22 may be changed or the rectangular frame 22 may be made to blink when a warning is issued. In FIG. 2, reference numeral 24 denotes a road on which the vehicle is traveling.

[0035] Next, a method for determining the possibility of a collision between the vehicle and a collision avoidance target by the collision avoidance setting unit 15 will be specifically described with reference to FIG. In FIG. 3, an excavator is detected as a collision avoidance target 25, and three horizontal lines 20 and a rectangular frame 22 indicating the range of the collision avoidance target 25 are drawn, similarly to FIG. Here, the coordinates of the left and right ends of the n-th horizontal line 20, counting from the side closest to the vehicle, are respectively (XLn,Yn) and (XRn,Yn). Also, the coordinates of the left and right ends of the bottom side of the rectangular frame 22 at a certain time are respectively (XtL,Yt) and (XtR,Yt). The length of the horizontal line 20 is selected appropriately, but it is preferably equal to the width of the vehicle. At this time, if either or both of XLn≦XtL≦XRn and XLn≦XtR≦XRn are satisfied, and Yt≦Yn is satisfied, the collision avoidance target 25 is located closer to the vehicle than the position of the nth horizontal line 20, and at least a part of the collision avoidance target 25 overlaps with the nth horizontal line 20 (the width of the vehicle). Therefore, the collision avoidance setting unit 15 determines that there is a possibility of a collision with the collision avoidance target 25 if the vehicle continues to move forward, and a specified alarm is output by the alarm output unit 16. Therefore, the collision avoidance setting unit 15 adjusts the collision determination area in which it is determined that there is a possibility of a collision between the vehicle and the collision avoidance object 25 by expanding or contracting the length of the horizontal line 20. In other words, if the length of the horizontal line 20 is long, the collision determination area is set wide, and if the length of the horizontal line 20 is short, the collision determination area is set narrow. It is possible to select which horizontal line 20 must satisfy the above condition to determine that there is a possibility of the vehicle colliding with the collision avoidance object 25.

[0036] The warning output unit 16 is required to be able to output a warning to the driver in the form of sound and / or light when the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and a collision avoidance target 25 (when action is required to avoid a collision between the vehicle and the collision avoidance target 25). The above-mentioned display unit 13, a speaker built into or connected to a computer, or a warning light mounted on a vehicle and connected to a computer are preferably used as the warning output unit 16. For example, the warning can be issued by displaying characters, figures, or symbols (marks, signs) that indicate a danger on the display unit 13, making them emit light or blink, emitting a sound from a speaker, or turning on or blinking a warning light. The sound (warning sound) emitted when warning is issued includes not only a buzzer or chime, but also a voice (word) that notifies the danger of a collision. The type of warning output from the warning output unit 16 can be appropriately selected (changed) according to the type of collision avoidance target and the distance from the vehicle to the collision avoidance target. For example, as the distance from the vehicle to the collision avoidance target decreases, the warning sound can be changed from a quiet (small) sound to a loud (large) sound, or the color of the warning light can be changed (e.g., green → yellow → red). Also, when the distance from the vehicle to the collision avoidance target is far, only the display unit can emit light or blink, or the warning light can be turned on or blinked, and when the distance from the vehicle to the collision avoidance target is close, the warning sound can be used in addition. In either case, the driver's attention can be effectively alerted.

[0037] The reverse signal detected by the vehicle state recognition unit 17 is an electrical signal output from the vehicle when the vehicle gear is in reverse, and this electrical signal is converted into a digital signal and input to the vehicle state recognition unit 17, thereby determining that the vehicle is moving backward. When no reverse signal is output from the vehicle, the vehicle is determined to be moving forward or stopped (including when the gear is in neutral or parking). Then, the collision avoidance setting unit 15 disables the output of an alarm by the alarm output unit 16 or switches the type of alarm output by the alarm output unit based on the determination result of the vehicle state recognition unit 17. For example, as described above, even if the machine learning inference unit 14 determines that there is a collision avoidance target in front of the vehicle and the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and the collision avoidance target based on the position information of the collision avoidance target (the distance from the vehicle to the collision avoidance target), when the vehicle state recognition unit 17 detects a reverse signal and determines that the vehicle is reversing, it is possible to configure the vehicle to not output an alarm (disable the alarm output) or to output a weak alarm (switch the type of alarm), and when the reverse signal is not detected and it is determined that the vehicle is not reversing (moving forward or stopped), it is possible to output an alarm (strong alarm) from the alarm output unit 16 (switch the type of alarm).

[0038] In addition, even if the machine learning inference unit 14 determines that there is a collision avoidance target behind the vehicle and the collision avoidance setting unit 15 determines that there is a possibility of a collision between the vehicle and the collision avoidance target based on the position information of the collision avoidance target, when the vehicle state recognition unit 17 determines that a reverse signal is not detected and the vehicle is not reversing, the alarm output unit 16 can be set to not output an alarm (disable the alarm output) or to output a weak alarm (switch the type of alarm), and when a reverse signal is detected and it is determined that the vehicle is reversing, an alarm (strong alarm) can be output (switch the type of alarm). In other words, by disabling the output of an alarm or switching (selecting) the type (strength) of the alarm to be output depending on a combination of the position of the collision avoidance target (the direction in which the collision avoidance target is located as seen from the vehicle and the distance from the vehicle to the collision avoidance target) and the vehicle's traveling direction (the vehicle is reversing, moving forward, or stopped), the driver can be effectively notified of the risk of collision.

[0039] As described above, the collision avoidance setting unit 15 can select whether to disable the output of the alarm by the alarm output unit 16 or to switch the type of alarm output by the alarm output unit 16 based on the determination result of the vehicle state recognition unit 17, or to forcibly stop the operation of the vehicle state recognition unit 17 so that the alarm output unit 16 outputs an alarm regardless of whether the vehicle is moving backwards or not. By allowing the driver to select either of the two, versatility is improved. For example, if the operation of the vehicle state recognition unit 17 is forcibly stopped so that the alarm output unit 16 outputs an alarm regardless of whether the vehicle is moving backwards or not, an alarm is output without fail when a collision avoidance target existing around the vehicle approaches within a certain distance, and the driver can be alerted. In addition, the images acquired by the image acquisition unit 12 and the real-time video created based on the images are stored in the machine learning setting unit 11. These images and real-time videos are stored in the memory of the computer at the time of acquisition and creation, respectively, but the storage here means that the data is stored in various storage media (not shown) such as a hard disk built into or external to the computer, or a CD-ROM or USB memory connected to the computer, and is in a state where it can be read by the machine learning setting unit 11 when necessary. As a result, the stored images and real-time videos are used for machine learning in the machine learning setting unit 11, and the detection accuracy of the collision avoidance target is improved. Note that the images are stored for a period going back a predetermined time from when the machine learning inference unit 14 determined that a collision avoidance target exists around the vehicle. For example, images for the past 10 frames from the time when the machine learning inference unit 14 detected the collision avoidance target are stored. In addition, the images to be stored are images (raw images) acquired by the image acquisition unit 12 (camera), images drawn with a horizontal line superimposed on the raw image, and images drawn with a horizontal line and a rectangular frame superimposed on the raw image. The real-time video is preferably stored while the collision avoidance system 10 is active, regardless of whether a collision avoidance object is detected or not.

[0040] Next, a collision avoidance method will be described for detecting a collision avoidance target using image recognition based on machine learning while the vehicle is being driven, and preventing a collision between the vehicle and the collision avoidance target. First, in the machine learning process, machine learning is performed based on learning images that contain collision avoidance objects to be detected, and a machine learning model is constructed. This is a process in which various collision avoidance objects (people, various vehicles including various heavy machinery, other obstacles, etc.) that must be avoided while driving the vehicle are learned in advance. This makes it possible to detect whether or not a collision avoidance object exists while the vehicle is actually driving (at the time of recognition), and to identify the type of collision avoidance object. Next, in an image acquisition process, images of the surroundings of the vehicle are acquired while the vehicle is being driven, and in an object detection process, it is determined whether a collision avoidance object is present around the vehicle based on the images acquired in the image acquisition process and a machine learning model. Then, when it is determined in the object detection process that a collision avoidance object exists, in a collision determination process, position information of the collision avoidance object detected in the object detection process is acquired and the possibility of collision between the vehicle and the collision avoidance object is determined.

[0041] The warning output step outputs a warning to a driver of the vehicle based on the determination by the collision avoidance setting unit. That is, the warning is output when the collision determination step determines that there is a possibility of a collision between the vehicle and the collision avoidance target. The driver can avoid the collision by changing the course of the vehicle or stopping the vehicle. In addition, in the collision determination step, it is determined whether the vehicle is moving backwards, and the output of the alarm by the alarm output step is invalidated or the type of alarm to be output by the alarm output step is switched. Therefore, depending on whether the vehicle is moving away from or approaching the collision avoidance target, the output of the alarm can be invalidated or the type of alarm to be output can be appropriately switched, thereby preventing excessive and unnecessary alarms from being output.

[0042] In addition, in the collision determination step, it may be possible to select whether to determine whether the vehicle is reversing or not and to disable the output of the alarm by the alarm output step or to switch the type of alarm to be output by the alarm output step, or to output the alarm by the alarm output step regardless of whether the vehicle is reversing or not. In this case, as described above, in addition to disabling the output of the alarm or switching the type of alarm to be output based on the positional relationship between the vehicle and the collision avoidance target, it is possible to forcibly output an alarm when the collision avoidance target is present within a certain distance from the vehicle, regardless of whether the vehicle is reversing or not (the tendency of the distance between the vehicle and the collision avoidance target to increase or decrease), and it is possible to use both methods as necessary (for example, depending on the type of vehicle or the collision avoidance target, etc.). During the driving of the vehicle, a video display process is executed, and a real-time video is created based on the images acquired in the image acquisition process, and is displayed on a display unit mounted on the vehicle. In the real-time video, one or more horizontal lines are drawn parallel to the width direction of the vehicle and indicate the distance from the vehicle. This allows the driver to check how far various objects around the vehicle are from the vehicle while driving the vehicle. Furthermore, when it is determined in the above-mentioned object detection process that a collision avoidance object exists, a rectangular frame indicating the range of the collision avoidance object is drawn in addition to the horizontal line in the real-time video. Therefore, the driver will not miss the collision avoidance object, and can easily grasp the distance from the vehicle to the collision avoidance object.

[0043] Next, the process of saving real-time video in the collision avoidance method executed in the collision avoidance system according to the embodiment of the present invention will be described with reference to FIG. When the collision avoidance system 10 is started, first, it is confirmed whether there is free space in the video storage folder in which real-time video is stored. Here, the collision avoidance system 10 is preferably started in conjunction with the start of the vehicle engine, but may be started by the driver separately pressing a button, etc. (S1: Step 1). If it is determined in step 1 that there is no free space in the video storage folder, old real-time video files stored in the video storage folder are deleted. Here, the capacity per real-time video file is predetermined, and old real-time video files are repeatedly deleted until it is determined that there is free space in the video storage folder for at least one file. It is preferable that the files are automatically deleted in order of oldest saved date and time, but it is also possible to allow the driver to select the files to delete (S2: step 2).

[0044] If it is determined in step 1 that there is free space in the video storage folder, creation of a new real-time video starts (S3: step 3). First, images of the outside of the vehicle are captured by cameras mounted on the vehicle. The number and arrangement of the cameras are as described above (S4: step 4). Then, based on the image acquired in step 4 and the machine learning model, it is determined whether there is a collision avoidance target around the vehicle (S5: step 5). If it is determined in step 5 that no collision avoidance target exists, a drawn image is created in which a horizontal line 20 (see Figures 2 and 3) is drawn on the image captured by the camera (raw camera image) (S6: step 6). If it is determined in step 5 that a collision avoidance target exists, a drawn image is created in which a horizontal line 20 is drawn on the image captured by the camera (raw camera image) as well as a rectangular frame 22 (see Figures 2 and 3) indicating the range of the collision avoidance target 21 or 25 (S7: step 7).

[0045] The drawn image created in step 6 or step 7 is acquired frame by frame (S8: step 8) and added to the real-time video being created. That is, a computer having a program for executing the collision avoidance method installed therein processes the image (creates the drawn image), and the processed images (drawn images) are connected in sequence frame by frame to create (update) the real-time video (S9: step 9). Then, it is confirmed whether the capacity of the real-time video file after the new drawn image is added in step 9 has reached the upper limit of the capacity per file (S10: step 10). If it is not determined in step 10 that the upper limit of the capacity per file has been reached, the process returns to step 4 to continue creating the real-time video, and the operations of steps 4 to 10 are repeated (S11: step 11). If it is determined in step 10 that the upper limit of the capacity per file has been reached, the real-time video being created is saved in a file, and the process returns to step 1 to start creating a new real-time video, and the operations of steps 1 to 10 are repeated (S12: step 12). While the collision avoidance system 10 is activated, the process of saving real-time video continues.

[0046] Next, the collision avoidance process that is performed in parallel with the real-time video saving process shown in FIG. 4 will be described with reference to FIG. If it is determined in step 5 of FIG. 4 that a collision avoidance target exists, the collision avoidance method shown in FIG. 5 is performed in parallel while the processing from step 7 onward is being performed as the real-time video storage processing. First, it is determined whether the setting (mode) of the collision avoidance system 10 is a digital signal input mode (S13: step 13). If it is determined in step 13 that the digital signal input mode is selected, it is confirmed whether a reverse signal is present (whether a reverse signal is being output from the vehicle) (S14: step 14). If it is determined in step 14 that a reverse signal is present, it is determined that the vehicle is moving backwards, and a first alarm is issued (output) (S15: step 15). If it is determined in step 14 that there is no reverse signal, it is determined that the host vehicle is moving forward or stopped, and a second alarm is issued (output) (S16: step 16). If it is determined in step 13 that the digital signal input mode is not in effect, the digital signal (reverse signal) is not detected, and the third alarm is issued (output) regardless of the vehicle state (whether the vehicle is reversing or not) (S17: step 17).

[0047] When any of the first to third alarms is issued, it is checked whether there is free space in the image storage folder in which the image captured by the camera (raw image) and the processed image (rendered image) are stored (S18: step 18). If it is determined in step 18 that there is no free space in the image storage folder, the files of old images (raw images and drawn images) stored in the image storage folder are deleted. Here, the deletion of old image files is repeated until it is determined that there is free space equal to or greater than the total capacity of the newly acquired images (raw images and drawn images). It is preferable that the files are automatically deleted in order of oldest saved date and time, but it is also possible to allow the driver to select the files to delete (S19: step 19). If it is determined in step 18 that the image folder has free space, the images (raw images and drawn images) at the time when the collision avoidance target was detected in step 5 (see FIG. 4) are saved. Here, images from a predetermined period going back from when the collision avoidance target was detected (e.g., 10 frames) are saved. In addition, text data of the type of collision avoidance target detected in step 5 and its location information (such as the distance from the vehicle to the collision avoidance target) are also saved (S20: step 20). In this embodiment, the contents of the first warning and the third warning are the same, but the contents of the first to third warnings are appropriately selected and may be the same in part or in whole, or may be different. Also, depending on conditions (for example, a combination of the position of a collision avoidance target and the traveling direction of the vehicle), any of the first to third warnings may be omitted (invalidated).

[0048] Although the embodiments of the present invention have been described above, the present invention is not limited to the configurations described in the above embodiments, and also includes other embodiments and variations that can be considered within the scope of the matters described in the claims. [Explanation of symbols]

[0049] 10: collision avoidance system, 11: machine learning setting unit, 12: image acquisition unit, 13: display unit, 14: machine learning inference unit, 15: collision avoidance setting unit, 16: alarm output unit, 17: vehicle state recognition unit, 20: horizontal line, 21: collision avoidance target, 22: rectangular frame, 23: auxiliary line, 24: road, 25: collision avoidance target

Claims

1. A collision avoidance system that detects a collision avoidance target using image recognition by machine learning while a vehicle is being driven, and prevents a collision between the vehicle and the collision avoidance target, a machine learning setting unit that performs machine learning based on learning images including the collision avoidance target to be detected and constructs a machine learning model; an image acquisition unit that acquires images of the periphery of the vehicle while the vehicle is being driven; a machine learning inference unit that determines whether the collision avoidance target is present around the vehicle based on the images acquired by the image acquisition unit and the machine learning model; a collision avoidance setting unit that acquires position information of the collision avoidance target detected by the machine learning inference unit and determines the possibility of a collision between the vehicle and the collision avoidance target; an alarm output unit that outputs an alarm to a driver of the vehicle based on the determination of the collision avoidance setting unit; and a real-time video created based on the images is displayed. a display unit, wherein one or more horizontal lines are drawn on the real-time video parallel to a width direction of the vehicle and indicating a distance from the vehicle, and when the machine learning inference unit determines that a collision avoidance target is present, a rectangular frame indicating the range of the collision avoidance target is drawn on the real-time video in addition to the horizontal line, and the collision avoidance setting unit adjusts a collision determination area in which it is determined that there is a possibility of collision between the vehicle and the collision avoidance target by expanding or contracting a length of the horizontal line, and the possibility of collision between the vehicle and the collision avoidance target is determined from the positional relationship between a bottom side of the rectangular frame and the horizontal line.

2. The collision avoidance system according to claim 1 , wherein the images and the real-time video are stored in the machine learning setting unit.

3. 2. The collision avoidance system according to claim 1, wherein the collision avoidance setting unit switches the type of warning output by the warning output unit depending on the type of the collision avoidance target and the distance from the vehicle to the collision avoidance target.

4. 4. The collision avoidance system according to claim 3, wherein the alarm output unit switches the type of alarm by sound volume and / or lighting, blinking or changing the color of a light.

Images