Collision avoidance system

Abstract

The purpose of the present invention is to provide a collision avoidance system device that can avoid, with high accuracy, a collision with a moving object that appears out of a blind spot area. In order to achieve this purpose, a vehicle control device (100) calculates indirect moving object information including position, moving direction, and moving speed of a moving object (204) on the basis of indirect information of the moving object (204) detected by moving object detection devices (111, 112), sets reliability of the indirect moving object information on the basis of clarity of the indirect information and stability of the moving direction, and controls, on the basis of the indirect moving object information, a vehicle (110) so as to avoid a collision between the vehicle (110) and the moving object (204) if the moving object detection device (111) cannot detect direct information and the reliability is equal to or greater than a prescribed threshold.

Description

Collision Avoidance System

[0001] The present invention relates to a collision avoidance system that controls a vehicle to avoid a collision with a moving object that appears from a blind spot area.

[0002] In recent years, vehicles equipped with driving assistance functions and autonomous driving functions have been put into practical use, primarily for the purpose of reducing traffic accidents and reducing driving burden. For example, a device is known that detects moving objects around the vehicle based on information detected by various sensors, such as an exterior camera or LiDAR (Light Detection and Ranging) installed on the vehicle, and assists the vehicle driver in avoiding a collision between the vehicle and the moving object. However, some traffic accidents, such as sudden emergence from a blind spot, can be difficult to avoid unless the driver takes preparatory action, such as slowing down, in anticipation of an accident. Patent Documents 1 and 2, for example, disclose technologies for solving such problems.

[0003] Patent Document 1 discloses a driving assistance device that assists driving to avoid collision with an object, the driving assistance device comprising one or more processors and one or more memories communicably connected to the one or more processors, wherein the processor detects the shadow of an object present in a blind spot area as seen from the vehicle based on image data captured around the vehicle, estimates the position of a light source that forms the shadow, calculates the position, direction of movement and speed of the object based on the position of the light source and changes in the shadow over time, and determines the possibility of a collision between the vehicle and the object.

[0004] Furthermore, Patent Document 2 discloses a blind spot information acquisition device that has a sound acquisition unit that acquires sounds around the vehicle, a sound generation position estimation unit that estimates the generation position of the sound acquired by the sound acquisition unit, an object acquisition unit that acquires information about objects around the vehicle, and a sound source position identification unit that recognizes that the sound is occurring in a blind spot when the sound source cannot be identified from the information acquired by the object acquisition unit based on the generation position of the sound estimated by the sound generation position estimation unit.

[0005] International Publication No. WO2023 / 053355 Japanese Patent Application Laid-Open No. 2021-125021

[0006] However, the driving assistance device of Patent Document 1 cannot correctly calculate the position, direction of movement, and speed of a moving object in a blind spot, for example, in an environment where no shadows are created or where shadows are obscured due to weather conditions or the positional relationship of light sources. As a result, the device is unable to correctly determine the possibility of a collision, and may not be able to cause the vehicle to take appropriate collision avoidance action, or may unnecessarily slow down or stop the vehicle.

[0007] Furthermore, the blind spot information acquisition device of Patent Document 2 cannot correctly calculate the position, direction of movement, and speed of a moving object in a blind spot, for example, in an environment where there are multiple sound sources in the blind spot, and therefore presents the same problems as the driving assistance device of Patent Document 1.

[0008] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide a collision avoidance system device that can avoid a collision with a moving object that appears from a blind spot area with high accuracy.

[0009] In order to achieve the above-mentioned object, the present invention provides a collision avoidance system comprising a moving body detection device that detects information about moving bodies present around a vehicle, and a vehicle control device that calculates direct moving body information including the position, moving direction, and moving speed of the moving body based on direct information about the moving body detected by the moving body detection device, determines whether or not there is a possibility of a collision between the vehicle and the moving body based on the direct moving body information, and if there is a possibility of a collision, controls the vehicle to avoid a collision with the moving body based on the direct moving body information, wherein the vehicle control device calculates indirect moving body information including the position, moving direction, and moving speed of the moving body based on indirect information about the moving body detected by the moving body detection device, sets a reliability of the indirect moving body information based on the clarity of the indirect information and the stability of the moving direction, and if the moving body detection device cannot detect the direct information and the reliability is equal to or greater than a predetermined threshold, controls the vehicle to avoid a collision between the vehicle and the moving body based on the indirect moving body information.

[0010] According to the present invention, it is possible to avoid a collision with a moving object that appears from a blind spot area with high accuracy.

[0011] 1 is a configuration diagram of a collision avoidance system. FIG. 2 is a diagram showing the processing of a first moving object recognition unit. FIG. 3 is a diagram showing the processing of a first moving object recognition unit. FIG. 4 is a diagram showing the processing of a first moving object recognition unit. FIG. 5 is a diagram showing the processing of a second moving object recognition unit. FIG. 6 is a diagram showing the processing of a third moving object recognition unit. FIG. 7 is a diagram showing an operation example 1 of the collision avoidance system. FIG. 8 is a diagram showing an operation example 2 of the collision avoidance system. FIG. 9 is a diagram showing a method for setting the reliability of each piece of first to third indirect moving object information. FIG. 10 is a diagram showing a method for setting the importance of each piece of first to third indirect moving object information. A flowchart showing the processing of the collision avoidance system. FIG. 11 is a diagram showing an operation example 3 of the collision avoidance system.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same reference numerals are used to designate the same components, and redundant description will be omitted.

[0013] 1 is a configuration diagram of a collision avoidance system 1. The collision avoidance system 1 includes an image sensor 111, a sound sensor 112, and an actuator 113 mounted on a vehicle 110, and a vehicle control device 100 that controls the vehicle 110. The image sensor 111 detects images of the surroundings of the vehicle 110. The sound sensor 112 detects sounds around the vehicle 110. The actuator 113 drives the engine system, brake system, and steering system of the vehicle 110. The image sensor 111 and the sound sensor 112 are examples of a moving object detection device that detects information about moving objects present around the vehicle 110.

[0014] The vehicle control device 100 has a first moving object recognition unit 101, a second moving object recognition unit 102, a third moving object recognition unit 103, a collision determination unit 104, and an actuator control unit 105. The vehicle control device 100 is equipped with a calculation unit such as a CPU, storage devices such as a ROM and RAM, an input / output interface for inputting and outputting signals to and from external devices, and the functions of each unit are realized by executing a program stored in the ROM or the like.

[0015] The first moving object recognition unit 101 calculates the position, moving direction, and moving speed of the moving object (direct moving object information) based on time changes in images (direct information) of the moving object (other vehicles, etc.) detected by the image sensor 111. The first moving object recognition unit 101 also calculates the position, moving direction, and moving speed of the moving object (first indirect moving object information) based on reflected light (indirect information) of light emitted from the moving object (headlights, tail lamps, tail lamps, turn signals, etc.) reflected on the road surface. Note that the first moving object recognition unit 101 may also calculate the position, moving direction, and moving speed of the moving object (indirect moving object information) based on reflected images (indirect information) of the moving object (e.g., other vehicle 204 shown in FIG. 2 ), light emitted from the moving object, or the shadow of the moving object reflected by the glass of a building, a curved mirror, etc.

[0016] The second moving object recognition unit 102 calculates the position, moving direction and moving speed of the moving object (second indirect moving object information) based on the shadow of the moving object (indirect information) detected by the image sensor 111 .

[0017] The third moving object recognition unit 103 calculates the position, moving direction, and moving speed of the moving object (third indirect moving object information) based on the sound of the moving object detected by the sound sensor 112 (for example, the running sound 601 of another vehicle 204 shown in Figure 6).

[0018] When direct moving object information is input from the first moving object recognition unit 101, the collision determination unit 104 determines whether or not there is a possibility of a collision between the vehicle 110 and a moving object based on the direct moving object information. On the other hand, when direct moving object information is not input, the collision determination unit 104 integrates the first to third indirect moving object information input from the first to third moving object recognition units 101 to 103, respectively, and determines whether or not there is a possibility of a collision between the vehicle 110 and a moving object based on the integrated indirect moving object information (integrated indirect moving object information).

[0019] When it is determined that there is a possibility of a collision between the vehicle 110 and a moving body, the actuator control unit 105 controls the actuator 113 based on the direct moving body information or the integrated indirect moving body information to avoid a collision between the vehicle 110 and the moving body.

[0020] Next, a specific example of the processing performed by the first to third mobile object recognition units 101 to 103 will be described.

[0021] As shown in Figure 2, when a vehicle 110 enters a road 202 from an alley 201 with poor visibility surrounded by fences on both sides, if there is another vehicle 204 traveling on the road 202 within the detection range 203 of the image sensor 111, the first moving object recognition unit 101 calculates the position, direction of movement and speed of movement (direct moving object information) of the other vehicle 204 based on changes in the image (direct information) of the other vehicle 204 over time.

[0022] 3, when the vehicle 110 enters the road 202 from the alley 201 at night, even if there is no other vehicle 204 traveling on the road 202 within the detection range 203 of the image sensor 111, if reflected light 301 (indirect information) of the headlights of the other vehicle 204 reflected on the surface of the road 202 is present within the detection range 203, the first moving object recognition unit 101 calculates the position, moving direction, and moving speed of the other vehicle 204 (first indirect moving object information) based on the time change of the reflected light 301. Furthermore, as shown in FIG. 4, if reflected light 401 of the turn signal of the other vehicle 204 reflected on the surface of the road 202 is present within the detection range 203, the first moving object recognition unit 101 calculates the position, moving direction, and moving speed of the other vehicle 204 (first indirect moving object information) based on the time change of the reflected light 401.

[0023] As shown in Figure 5, even if there are no other vehicles 204 traveling on the road 202 within the detection range 203 of the image sensor 111 during the day, if a shadow 501 (indirect information) of the other vehicle 204 is present within the detection range 203, the second moving object recognition unit 102 calculates the position, direction of movement, and speed of movement of the other vehicle 204 (second indirect moving object information) based on the change in the shadow 501 over time.

[0024] As shown in Figure 6, even if there is no other vehicle 204 traveling on the road 202 within the detection range 203 of the image sensor 111, if the sound sensor 112 detects a sound 601 (indirect information) of the other vehicle 204, the third moving object recognition unit 103 calculates the position, moving direction and moving speed of the other vehicle 204 (third indirect moving object information) based on the change in the sound 601 over time.

[0025] Here, the first moving object recognition unit 101 reduces the accuracy of calculating the position, moving direction, and moving speed of the other vehicle 204 (first indirect moving object information) when the clarity of the reflected light 301, 401 or the stability of the moving direction is low. Furthermore, the second moving object recognition unit 102 reduces the accuracy of calculating the position, moving direction, and moving speed of the other vehicle 204 (second indirect moving object information) when the clarity of the shadow 501 or the stability of the moving direction is low. For example, as shown in FIG. 7 , when the weather changes from sunny to cloudy, the shadow of the other vehicle 204 that was present in the detection range 203 becomes unclear, and the accuracy of calculating the position, moving direction, and moving speed of the other vehicle 204 (second indirect moving object information) reduces. Furthermore, as shown in FIG. 8 , when the direction of the shadow 501 of the other vehicle 204 fluctuates depending on the positional relationship between the other vehicle 204 and streetlights 801, 802 adjacent to the road 202, the accuracy of calculating the position, moving direction, and moving speed of the other vehicle 204 (second indirect moving object information) reduces. Therefore, the collision determination unit 104 sets the reliability of each piece of first to third indirect moving object information in accordance with the rules shown in FIG. 9, and sets the importance of each piece of first to third indirect moving object information in accordance with the rules shown in FIG.

[0026] 9 , when the clarity or stability of the movement direction of the reflected light 301, 401 is high, the reliability of the first indirect moving object information is set high. When the clarity or stability of the movement direction of the reflected light 301, 401 is low, the reliability of the first indirect moving object information is set low. Furthermore, when the clarity or stability of the movement direction of the shadow 501 is high, the reliability of the second indirect moving object information is set high. When the clarity or stability of the movement direction of the shadow 501 is low, the reliability of the second indirect moving object information is set low. Furthermore, when the clarity or stability of the movement direction of the sound 601 is high, the reliability of the third indirect moving object information is set high. When the clarity or stability of the movement direction of the sound 601 is low, the reliability of the third indirect moving object information is set low. Each reliability may be defined in a range from 0 to a positive value, or in a positive or negative range.

[0027] 10 , if the time period is nighttime or there are no other light sources, the importance level of the first indirect moving object information is set high, and if the time period is daytime or there are other light sources, the importance level of the first indirect moving object information is set low. Furthermore, if the weather is clear, the time period is daytime, or there are no other light sources, the importance level of the second indirect moving object information is set high, and if the weather is cloudy or rainy, the time period is nighttime, or there are other light sources, the importance level of the second indirect moving object information is set low. Furthermore, if the time period is nighttime or there are no other sound sources, the importance level of the third indirect moving object information is set high, and if the time period is daytime or there are other sound sources, the importance level of the third indirect moving object information is set low. Each importance level may be defined within a range of 0 to a positive value, or within a positive or negative range.

[0028] The collision determination unit 104 calculates integrated joint moving body information by integrating the first to third joint moving body information according to the reliability and importance of each of the first to third joint moving body information. There are various conceivable methods for integrating the first to third joint moving body information. For example, the first to third joint moving body information with the highest sum or multiplication value of the reliability and importance may be selected as the integrated joint moving body information. Alternatively, the integrated joint moving body information may be calculated as a weighted average of the first to third joint moving body information weighted by the sum or multiplication value of the reliability and importance. Alternatively, the integrated joint moving body information may be calculated as an average of the two first to third joint moving body information with the highest sum or multiplication value of the reliability and importance.

[0029] FIG. 11 is a flowchart showing the processing of the collision avoidance system 1 when no moving object is present within the detection range 203 of the image sensor 111.

[0030] First, the first moving object recognition unit 101 determines whether or not the shadow of a moving object exists within the detection range 203 (step S101).

[0031] If the determination result in step S101 is Yes, the first moving object recognition unit 101 calculates the position, moving direction, and moving speed of the moving object (second indirect moving object information) based on the change in the shadow over time.

[0032] Following step S102, the first moving object recognition unit 101 sets the reliability of the second indirect moving object information based on the clarity of the shadow and the stability of the moving direction.

[0033] Following step S103, or if the determination result of step S101 is No, the first moving object recognition unit 101 determines whether reflected light of light emitted from a moving object is present within the detection range 203 (step S104).

[0034] If the determination result in step S104 is Yes, the first moving object recognition unit 101 calculates the position, moving direction, and moving speed of the moving object (first indirect moving object information) based on the change over time of the reflected light (step S105).

[0035] Following step S105, the first moving object recognizing unit 101 sets the reliability of the first indirect moving object information based on the clarity of the reflected light and the stability of the moving direction (step S106).

[0036] Following step S106, or if the determination result of step S104 is No, the second moving object recognition unit 102 determines whether or not the sound sensor 112 has detected the sound of a moving object (step S107).

[0037] If the determination result in step S107 is Yes, the third moving object recognition unit 103 calculates the position, moving direction, and moving speed of the moving object (second indirect moving object information) based on the change in sound over time (step S108).

[0038] Following step S108, the third moving object recognition unit 103 sets the reliability of the third indirect moving object information based on the clarity of the sound and the stability of the moving direction (step S109).

[0039] Following step S109, or if the determination result in step S107 is No, the collision determination unit 104 determines whether the reliability of any of the first to third joint moving object information 101 to 103 is equal to or greater than a predetermined threshold (step S110).

[0040] If the determination result in step S110 is No, the flow ends so that the driving operation by the driver of the vehicle 110 is given priority.

[0041] If the determination result in step S110 is Yes, the collision determination unit 104 sets the importance level of each of the first to third indirect moving object information based on weather conditions, time period, environmental factors, etc. (step S111).

[0042] Following step S111, the collision determination unit 104 calculates integrated joint moving object information by integrating the first to third joint moving object information according to the respective reliability levels and respective importance levels (step S112).

[0043] Following step S112, the collision determination unit 104 determines whether there is a possibility of a collision between the vehicle 110 and the moving object based on the integrated joint moving object information (step S113).

[0044] If the determination result in step S113 is Yes, the actuator control unit 105 executes collision avoidance control to avoid a collision between the vehicle 110 and the moving object (step S114), and ends the flow.

[0045] If the determination result in step S113 is No, the flow ends.

[0046] An example of the operation of the collision avoidance system 1 configured as above will be described.

[0047] <Operation Example 1> In FIG. 7 , when the shadow 501 becomes unclear, the reliability and importance of the second indirect moving body information decrease, and therefore the vehicle 110 is controlled so as to avoid a collision between the vehicle 110 and the other vehicle 204 based on the first indirect moving body information or the third indirect moving body information, which have high reliability and importance.

[0048] <Operation Example 2> In FIG. 8 , when the moving direction of the shadow 501 becomes unstable due to the presence of other light sources (street lights 801, 802), the reliability and importance of the second indirect moving body information decreases, and therefore the vehicle 110 is controlled so as to avoid a collision between the vehicle 110 and the other vehicle 204 based on the first indirect moving body information or the third indirect moving body information, which have high reliability and importance.

[0049] <Operation Example 3> In FIG. 12 , if there is no other vehicle 204 within the detection range 203 and the reliability of each of the first to third indirect moving object information is less than a threshold, the vehicle 110 performs deceleration and stop control in the alley 201 just before the road 202. If the other vehicle 204 immediately enters the detection range 203, the vehicle 110 transitions from a deceleration or stop state to collision avoidance control based on an image (direct information) of the other vehicle 204, which may cause the vehicle 110 to become unstable. Therefore, a smooth transition from deceleration and stop control to collision avoidance control based on direct information is made. For example, a weighted average of the control commands for deceleration and stop control and collision avoidance control is calculated, and each weight is continuously changed according to the elapsed time. However, the interval between the transition from a 100% weight for deceleration and stop control to a 100% weight for collision avoidance control is set shorter as the degree of proximity (urgency) to the other vehicle 204 calculated based on the direct information increases.

[0050] (Summary) In this embodiment, a collision prevention device is provided which includes moving body detection devices 111 and 112 that detect information about moving bodies 204 that exist around a vehicle 110, and a vehicle control device that calculates direct moving body information including the position, moving direction, and moving speed of the moving body 204 based on direct information about the moving body 204 detected by the moving body detection devices 111 and 112, determines whether or not there is a possibility of a collision between the vehicle 110 and the moving body 204 based on the direct moving body information, and, if there is a possibility of a collision, controls the vehicle 110 based on the direct moving body information so as to avoid a collision between the vehicle 110 and the moving body 204. In the collision avoidance system, the vehicle control device 100 calculates indirect moving object information including the position, moving direction, and moving speed of the moving object 204 based on indirect information of the moving object 204 detected by the moving object detection devices 111 and 112, sets the reliability of the indirect moving object information based on the clarity of the indirect information and the stability of the moving direction, and controls the vehicle 110 based on the indirect moving object information so as to avoid a collision between the vehicle 110 and the moving object 204 when the moving object detection device 111 cannot detect the direct information and the reliability is above a predetermined threshold.

[0051] According to the present embodiment configured as described above, even when direct information about the moving body 204 cannot be detected, the vehicle 110 is controlled based on highly reliable indirect moving body information calculated based on indirect information to avoid a collision between the vehicle 110 and the moving body 204. This makes it possible to avoid a collision between the vehicle 110 and the moving body 204 that appears out of a blind spot area with high accuracy.

[0052] In this embodiment, the indirect information includes reflected light 301, 401, which is light emitted from the moving object 204 and reflected by the road surface. This makes it possible to calculate indirect moving object information based on the light emitted from the moving object 204.

[0053] In this embodiment, the indirect information includes the shadow 501 of the moving object 204. This makes it possible to calculate indirect moving object information based on the shadow 501 of the moving object 204.

[0054] In this embodiment, the indirect information includes the sound 601 of the moving object 204. This makes it possible to calculate indirect moving object information based on the sound 601 of the moving object 204.

[0055] In this embodiment, the indirect information includes the reflected image of the moving object 204, the light emitted from the moving object 204, or the shadow of the moving object 204 reflected on the glass of buildings or convex mirrors around the vehicle 110. This makes it possible to calculate indirect moving object information based on the reflected image of the moving object 204, the light emitted from the moving object 204, or the shadow of the moving object 204 reflected on the glass of buildings or convex mirrors.

[0056] Furthermore, in this embodiment, the indirect information includes a plurality of pieces of indirect moving object information (reflected light, shadow, and sound), and the vehicle control device 100 calculates a plurality of pieces of indirect moving object information (first to third indirect moving object information) based on the plurality of pieces of indirect information, sets the reliability of each piece of indirect moving object information based on the clarity of each piece of indirect information and the stability of each movement direction of each piece of indirect moving object information, sets the importance of each piece of indirect moving object information based on the environmental conditions around the vehicle 110, calculates integrated indirect moving object information by integrating the plurality of pieces of indirect moving object information according to the reliability and the importance, and controls the vehicle 110 based on the integrated indirect moving object information to avoid a collision between the vehicle 110 and the moving object 204. This makes it possible to avoid a collision between the vehicle 110 and the moving object 204 that appears in a blind spot area with even greater accuracy.

[0057] It should be noted that the present invention is not limited to the above-described embodiment, and includes various modifications. For example, the above-described embodiment is shown to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the described configurations.

[0058] 1...collision avoidance system, 100...vehicle control device, 101...first moving object recognition unit, 102...second moving object recognition unit, 103...third moving object recognition unit, 104...collision judgment unit, 105...actuator control unit, 110...vehicle, 111...image sensor (moving object detection device), 112...sound sensor (moving object detection device), 113...actuator, 201...alley, 202...road, 203...detection range, 204...other vehicles (moving objects), 301, 401...reflected light, 501...shadow, 601...sound, 801, 802...street lights.

Claims

1. A collision avoidance system comprising: a moving object detection device that detects information about moving objects present around a vehicle; and a vehicle control device that calculates direct moving object information including the position, moving direction, and moving speed of the moving object based on direct information about the moving object detected by the moving object detection device, determines whether or not there is a possibility of a collision between the vehicle and the moving object based on the direct moving object information, and controls the vehicle to avoid collision with the moving object based on the direct moving object information if there is a possibility of a collision, wherein the vehicle control device calculates indirect moving object information including the position, moving direction, and moving speed of the moving object based on indirect information about the moving object detected by the moving object detection device, sets a reliability of the indirect moving object information based on the clarity of the indirect information and the stability of the moving direction, and controls the vehicle to avoid collision between the vehicle and the moving object based on the indirect moving object information if the moving object detection device cannot detect the direct information and the reliability is equal to or greater than a predetermined threshold.

2. A collision avoidance system according to claim 1, wherein the indirect information includes light emitted from the moving object and reflected by the road surface.

3. A collision avoidance system according to claim 1, wherein the indirect information includes a shadow of the moving object.

4. A collision avoidance system according to claim 1, wherein the indirect information includes the sound of the moving object.

5. A collision avoidance system as described in claim 1, wherein the indirect information includes a reflected image of the moving object, light emitted from the moving object, or a shadow of the moving object reflected on the glass or convex mirror of a building around the vehicle.

6. A collision avoidance system as described in claim 1, wherein the indirect information includes a plurality of pieces of indirect information, and the vehicle control device calculates a plurality of pieces of indirect moving object information based on the plurality of pieces of indirect information, sets the reliability of each piece of indirect moving object information based on the clarity of each piece of indirect information and the stability of each moving direction of each piece of indirect information, sets the importance of each piece of indirect moving object information based on the environmental conditions around the vehicle, calculates integrated indirect moving object information by integrating the plurality of pieces of indirect moving object information according to each reliability and each importance, and controls the vehicle based on the integrated indirect moving object information to avoid collision between the vehicle and the moving object.

Images