Vehicle device and vehicle control method

Abstract

The present application relates to a vehicle device and a vehicle control method. The vehicle device includes: a first power supply unit (21); a second power supply unit (22); a first sensor unit (30, 40) that operates with power supplied from the first power supply unit (21), detects a front obstacle of a vehicle, and detects information related to a position in a road width direction of the vehicle, i.e., lateral position information; a SoC (71) that operates with power supplied from the first power supply unit (21), controls the vehicle based on a detection result of the first sensor unit; a second sensor unit (50, 60) that operates with power supplied from the second power supply unit (22), detects the front obstacle, and detects the lateral position information; and a SoC (72) that operates with power supplied from the second power supply unit (22), controls the vehicle based on a detection result of the second sensor unit.

Description

[0001] Cross-references to related applications

[0002] This application is based on Japanese Patent Application No. 2021-114258, filed on July 9, 2021, and the contents of the base application are incorporated herein by reference in their entirety. Technical Field

[0003] This invention relates to a device for vehicles and a vehicle control method. Background Technology

[0004] Various devices for controlling vehicles are known. Patent Document 1 discloses a device that integrates sensor information from multiple sensors and determines abnormalities in the integrated unit. Furthermore, Patent Document 1 also discloses a method for ensuring the reliability of the integrated unit through redundancy.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-45024

[0006] By redundantizing the integrated unit or the sensors integrated within it, the likelihood of continuing vehicle control even if a structural anomaly occurs somewhere can be increased. However, there are concerns that redundancy increases the number of components required for vehicle control while maintaining the same level of accuracy. Summary of the Invention

[0007] This disclosure is made based on this situation, and its purpose is to provide a vehicle device and a vehicle control method that suppresses the increase of constituent elements and enables the vehicle to continue driving even when sensors or the like are malfunctioning.

[0008] The aforementioned objective is achieved through a combination of the features described in the independent claims, and furthermore, the dependent claims provide more advantageous specific examples. The reference numerals enclosed in parentheses in the claims indicate a correspondence with the specific embodiments described below as examples, and do not limit the scope of the disclosed technology.

[0009] One disclosure relates to a vehicle device for achieving the above-mentioned objectives is a vehicle device used in a vehicle, comprising:

[0010] First power supply section;

[0011] Second power supply unit;

[0012] The first sensor unit operates by being powered by the first power supply unit, and detects obstacles in front of the vehicle and detects information related to the vehicle's position in the road width direction, namely, lateral position information.

[0013] The first control device operates by being powered by the first power supply unit and controls the vehicle based on the detection results of the first sensor unit;

[0014] The second sensor unit, powered by the second power supply unit, detects obstacles ahead and detects lateral position information; and

[0015] The second control unit operates by being powered by the second power supply unit and controls the vehicle based on the detection results of the second sensor unit.

[0016] According to the vehicle device, a vehicle control method as shown below can be performed. This vehicle control method is a vehicle control method executed by a processor in a vehicle equipped with a first power supply unit; a second power supply unit; a first sensor unit powered from the first power supply unit that detects obstacles ahead of the vehicle and detects information related to the vehicle's position in the road width direction, i.e., lateral position information; and a second sensor unit powered from the second power supply unit that detects obstacles ahead and detects lateral position information.

[0017] When the first power supply unit, the first sensor unit, the second power supply unit, and the second sensor unit are functioning normally, at least one of longitudinal synthesis processing and lateral synthesis processing is performed. Longitudinal synthesis processing is the process of obtaining detection results related to obstacles in front from the first sensor unit and the second sensor unit respectively and integrating the detection results related to obstacles in front. Lateral synthesis processing is the process of obtaining lateral position information from the first sensor unit and the second sensor unit respectively and integrating the lateral position information.

[0018] If an anomaly is detected in at least one of the second power supply unit and the second sensor unit, the first sensor unit is used instead of the second sensor unit to perform limiting control. This limiting control is a vehicle control that, while limited compared to normal autonomous driving control performed when no anomaly is detected, detects obstacles ahead and lateral position information and enables the vehicle to move.

[0019] If an abnormality is detected in at least one of the first power supply unit and the first sensor unit, the second sensor unit is used instead of the first sensor unit to perform the restriction control.

[0020] According to this vehicle control method, even if any one or more of the first sensor unit, the first control device, and the first power supply unit malfunctions, the second sensor unit, the second control device, and the second power supply unit can still perform limiting control to detect obstacles ahead and lateral position information and continue vehicle movement. Furthermore, even if any one or more of the second sensor unit, the second control device, and the second power supply unit malfunctions, the first sensor unit, the first control device, and the first power supply unit can still perform limiting control.

[0021] Furthermore, if the first sensor unit, the second sensor unit, the first power supply unit, and the second power supply unit are functioning normally, at least one of longitudinal and lateral integrated processing, which combines the detection results from the first and second sensor units, is performed, thus enabling high-precision vehicle control. This vehicle control method also uses either the first or second sensor unit during abnormal vehicle operation. Therefore, it can suppress the increase of constituent elements and continue vehicle operation even during abnormal situations. Attached Figure Description

[0022] Figure 1 This is a diagram showing the structure of the vehicle device 10 according to the first embodiment.

[0023] Figure 2 This diagram illustrates the mounting position of the sensor and the field of view 51L, 51L of the front-side millimeter-wave radar 50.

[0024] Figure 3 This diagram illustrates the shooting range of the surrounding cameras (60).

[0025] Figure 4 This is a diagram representing the automatic driving control during normal operation.

[0026] Figure 5 This is a diagram representing control measures in case of an anomaly.

[0027] Figure 6 This diagram represents a sensor that can be used in either the first sensor section or the second sensor section.

[0028] Figure 7 It means Figure 6 A diagram showing a portion of the sensor.

[0029] Figure 8 This is a diagram showing the combination of sensors that can be realized by the first sensor unit and the second sensor unit. Detailed Implementation

[0030] <First Implementation>

[0031] The embodiments will now be described with reference to the accompanying drawings. Figure 1 This is a diagram showing the structure of the vehicle device 10 according to the first embodiment. The vehicle device 10 is mounted on... Figure 2 The vehicle C shown. The vehicle device 10 allows for the control of one or both of the vehicle C's steering and speed, at least temporarily without the need for driver intervention.

[0032] The vehicle device 10 is capable of performing autonomous driving at level 3. At level 3, the device performs all driving operations under limited conditions. However, in emergencies, the driver takes over driving duties. The vehicle device 10 can also perform autonomous driving levels other than level 3, such as levels 1, 2, and 4. Furthermore, the vehicle C can still drive even when the vehicle device 10 is not functioning, i.e., at level 0.

[0033] The vehicle device 10 includes two power supply units, a first power supply unit 21 and a second power supply unit 22, multiple sensors for detecting the surrounding conditions of the vehicle C, and an ECU 70. The first power supply unit 21 and the second power supply unit 22 are independent power supply units. Both the first power supply unit 21 and the second power supply unit 22 can be charged by electricity generated by a generator installed in the vehicle C. The first power supply unit 21 and the second power supply unit 22 can use batteries made of various materials, such as lead-acid batteries, nickel-metal hydride batteries, and lithium-ion batteries.

[0034] In order to detect the surrounding conditions of vehicle C, specifically, in the vehicle device 10 of the first embodiment, multiple sensors are a front millimeter-wave radar 30, a front camera 40, a front side millimeter-wave radar 50, and a peripheral camera 60.

[0035] The front millimeter-wave radar 30 transmits millimeter waves into its field of view and receives reflected waves generated by objects reflecting these waves. Based on the time difference between transmission and reception and the direction of the transmitted waves, the front millimeter-wave radar 30 detects the position and orientation of objects from vehicle C to the outside. The field of view of the front millimeter-wave radar 30 includes the area in front of vehicle C, and it detects obstacles present in front of vehicle C. These obstacles include stationary and moving objects. An example of a moving object is a vehicle traveling in the same lane as vehicle C. The front millimeter-wave radar 30 is installed, for example, at the front of vehicle C and at the center in the vehicle width direction. An example of the field of view angle of the front millimeter-wave radar 30 is approximately ±60 degrees from the front of vehicle C, with 0 degrees as the reference point.

[0036] The front camera 40 is a monocular camera that captures images of the front of vehicle C. In this embodiment, the front camera 40 is used to detect lane boundaries. Lane boundaries define the boundaries of a lane. An example of a lane boundary is... Figure 2 Lane marking 8 is shown. On roads where lane marking 8 does not exist, the road edge becomes the road boundary. For example... Figure 2As shown, the front camera 40 is positioned, for example, near the front end of the roof inside the passenger compartment of vehicle C. Preferably, the front camera 40 has a wide field of view of 100° or more, so that it can detect lane markings 8 near vehicle C. However, since lane markings 8 also extend forward along vehicle C, the field of view can also be narrower than 100°.

[0037] These front millimeter-wave radars 30 and front cameras 40 are first sensor units that operate by being powered by the first power supply unit 21.

[0038] The front-side millimeter-wave radar 50 includes a left front-side millimeter-wave radar 50L and a right front-side millimeter-wave radar 50R. These left front-side millimeter-wave radars 50L and right front-side millimeter-wave radars 50R are used to detect obstacles present on the left and right sides of vehicle C. The left front-side millimeter-wave radar 50L is located at the left front end of vehicle C, and the right front-side millimeter-wave radar 50R is located at the right front end of vehicle C. Obstacles need to be detected over a relatively wide area on the sides of vehicle C. Therefore, the left front-side millimeter-wave radar 50L and right front-side millimeter-wave radar 50R have a wider field of view compared to the front millimeter-wave radar 30. Figure 2 The diagram illustrates the fields of view 51L and 51R of the left front millimeter-wave radar 50L and the right front millimeter-wave radar 50R, respectively. The left front millimeter-wave radar 50L and the right front millimeter-wave radar 50R each view the oblique front of the vehicle C as their frontal field of view.

[0039] The front side millimeter-wave radar 50, like the front millimeter-wave radar 30, detects the position and orientation of objects from vehicle C to the outside based on the time difference between transmission and reception of millimeter waves and the direction of illumination of the transmitted waves.

[0040] The fields of view 51L and 51R of the left front millimeter-wave radar 50L and the right front millimeter-wave radar 50R are relatively wide. Therefore, the combined field of view of the two fields of view 51L and 51R becomes a field of view that also includes the front of vehicle C. When specifically illustrating the fields of view 51L and 51R, the center is ±60° with respect to the frontal direction of the left front millimeter-wave radar 50L and the right front millimeter-wave radar 50R.

[0041] The peripheral cameras 60 include a front peripheral camera 60F, a left-side peripheral camera 60L, a right-side peripheral camera 60R, and a rear peripheral camera 60B. The front peripheral camera 60F is positioned at the front of vehicle C, centered in the vehicle width direction. The left-side peripheral camera 60L is positioned on the lower surface of the left-side rearview mirror of vehicle C. The right-side peripheral camera 60R is positioned on the lower surface of the right-side rearview mirror of vehicle C. The rear peripheral camera 60B is positioned at the rear of vehicle C, centered in the vehicle width direction.

[0042] The purpose of setting up four peripheral cameras 60 is to perform coordinate transformation on the images captured by these four peripheral cameras 60 to create a bird's-eye view of vehicle C from above. Since these cameras are designed for this purpose, the range captured by the peripheral cameras 60 is the periphery of vehicle C. In other words, the range captured by the peripheral cameras 60 is closer to vehicle C than the range captured by the front camera 40.

[0043] Figure 3 This diagram illustrates the shooting range of the peripheral cameras 60. The front peripheral camera 60F, the left peripheral camera 60L, the right peripheral camera 60R, and the rear peripheral camera 60B all have wide-angle fields of view. Therefore, the shooting range 61F of the front peripheral camera 60F, the shooting range 61L of the left peripheral camera 60L, the shooting range 61R of the right peripheral camera 60R, and the shooting range 61B of the rear peripheral camera 60B all include the vehicle C or its vicinity. Furthermore, shooting ranges 61F, 61L, 61R, and 61B partially overlap with two adjacent shooting ranges. Because images of these shooting ranges 61F, 61L, 61R, and 61B are captured, a bird's-eye view image can be created by performing coordinate transformation on the images captured by the four peripheral cameras 60.

[0044] The front millimeter-wave radar 50 and the peripheral camera 60 are second sensor units that operate by being powered by the second power supply unit 2.

[0045] The ECU 70 has two SoCs (System-on-a-chip) 71 and 72. SoC 71 is the first control device, and SoC 72 is the second control device, each with a processor. For example, SoCs 71 and 72 have a structure including a processor, non-volatile memory, RAM, and buses connecting these structures. The non-volatile memory stores the vehicle control program executed by the processor. By utilizing the temporary storage function of RAM and executing the program stored in the non-volatile memory, SoCs 71 and 72 can respectively execute the abnormal situation control described later. In addition, SoCs 71 and 72 can also execute the normal driving control described later. Executing these controls means executing the vehicle control method corresponding to the program.

[0046] Thus, although SoC71 and SoC72 can perform the same control, they are powered by different power sources. SoC71 operates by receiving power from the first power supply unit 21, while SoC72 operates by receiving power from the second power supply unit 22.

[0047] [Automatic driving control during normal operation]

[0048] Next, the normal-time automated driving control executed by SoC71 and SoC72 will be explained. Normal-time automated driving control is executed by either SoC71 or SoC72, which is pre-defined. Hereinafter, we will assume that SoC71 executes the normal-time automated driving control. Normal-time automated driving control is the control executed when vehicle C is under automated driving control and SoC71 does not detect any abnormalities. Furthermore, the automated driving control is assumed to be at automated driving level 3.

[0049] exist Figure 4 The flowchart illustrates the normal autonomous driving control. Only S1 is executed by SoC71 and SoC72 together. SoC71 and SoC72 determine whether an anomaly is detected. The objects to be determined as an anomaly are the first power supply unit 21, the second power supply unit 22, the front millimeter-wave radar 30, the front camera 40, the front side millimeter-wave radar 50, the peripheral camera 60, SoC71, and SoC72.

[0050] Whether the front millimeter-wave radar 30, the front camera 40, the front side millimeter-wave radar 50, and the peripheral camera 60 are malfunctioning can also be determined by either SoC 71 or 72. Similarly, whether the first power supply unit 21 and the second power supply unit 22 are malfunctioning can also be determined by either SoC 71 or 72. However, it is preferable that SoC 72, which does not receive power from the first power supply unit 21, determines whether the first power supply unit 21 is malfunctioning, and SoC 71, which does not receive power from the second power supply unit 22, determines whether the second power supply unit 22 is malfunctioning. Whether SoC 71 is malfunctioning is determined by SoC 72, and whether SoC 72 is malfunctioning is determined by SoC 71.

[0051] A malfunction in power supply units 21 and 22 indicates that the voltage input is outside the specified range. When a disconnection or a fault occurs in power supply units 21 and 22, the voltage input to ECU 70 becomes below the specified range. Additionally, if a higher voltage exceeding the specified range is input to ECU 70, power supply units 21 and 22 will also malfunction. Whether the front millimeter-wave radar 30, front camera 40, front side millimeter-wave radar 50, and peripheral camera 60 are malfunctioning is determined, for example, based on the level of signals obtained from them. Furthermore, regarding the component in SoC 71 and 72 that is not performing normal autonomous driving control, since control is not being performed, the malfunction determination can be omitted.

[0052] If the result of S1 is "no", then SoC71 proceeds the processing to S2. In S2, the front millimeter-wave radar 30, the front camera 40, the front side millimeter-wave radar 50, and the peripheral camera 60 are controlled to acquire sensor signals from these sensors.

[0053] In the following S3, based on the sensor signals acquired in S2, a detection result related to the obstacle in front is determined for each sensor. Additionally, lateral position information is determined for each sensor. The detection result related to the obstacle in front includes the presence or absence of the obstacle and its position (in other words, the distance to the obstacle). Furthermore, the size and type of the obstacle in front can also be included in the detection result. In this embodiment, the sensors used to detect the obstacle in front are a front millimeter-wave radar 30 and a front lateral millimeter-wave radar 50. The SoC 71 determines the detection result related to the obstacle in front based on the sensor signals acquired from these front millimeter-wave radars 30 and lateral millimeter-wave radars 50.

[0054] Lateral position information is information related to the position of vehicle C in the road width direction. An example of lateral position information is the lane boundary existing to the side of vehicle C. Furthermore, "side" here includes not only the lateral side but also the oblique side. In this embodiment, the sensors used to detect lateral position information are a front camera 40 and a peripheral camera 60. These front cameras 40 and peripheral cameras 60 are capable of capturing images including the lane boundary existing to the side of vehicle C. SoC 71 detects lane markings 8 or road edges from the images captured by the front camera 40 or the peripheral camera 60 through image processing.

[0055] After executing S3, proceed to S4. In S4, detection results and lateral position information related to the obstacle ahead are obtained from the first sensor unit and the second sensor unit, respectively. Therefore, comprehensive processing is performed in S4 and S5. The processing performed in S4 is longitudinal comprehensive processing, and the processing performed in S5 is lateral comprehensive processing. In S4, the detection results related to the obstacle ahead determined by each sensor in S3 are synthesized. For example, the position of the obstacle ahead is determined by performing a simple average or a weighted average of the positions of the obstacle ahead determined by each sensor.

[0056] In S5, the lateral position information determined by each sensor in S3 is integrated. For example, the position of the obstacle ahead is determined by performing a simple average or a weighted average of the lane boundary positions determined by each sensor. The integration process in S4 and S5 is also known as sensor fusion.

[0057] Additionally, SoC71 calculates the distance in the road width direction from vehicle C to the lane boundary. The positions and fields of view of the front camera 40 and the peripheral cameras 60 are fixed, thus corresponding one-to-one with the road surface position at any location on the image. Therefore, if the location of the lane boundary can be determined from the image captured by the front camera 40 or the peripheral cameras 60, the distance in the road width direction from vehicle C to the lane boundary can be calculated.

[0058] In S6, the processing results from S4 and S5 are used to determine the longitudinal and lateral movements of vehicle C. The longitudinal movement of vehicle C determines its speed. The lateral movement of vehicle C determines its steering angle. For example, SoC71 may decide to reduce the speed of vehicle C when the distance to an obstacle ahead decreases. Additionally, SoC71 may decide to change the steering angle to align with the shape of lane markings 8. When performing autonomous driving control, SoC71 can use various information, including detection results related to obstacles ahead and lateral position information, as well as path information to the destination. Furthermore, SoC71 may also determine lane-changing steering operations when the distance to an obstacle ahead decreases or for left or right turns.

[0059] In S7, the control devices for accelerating and decelerating vehicle C and for steering vehicle C are instructed to execute the movements determined in S6. After executing S7, the process returns to S1.

[0060] [Abnormal situation control]

[0061] Next, regarding Figure 4 The exception handling procedure is explained when the S1 condition is "yes". Figure 5 The control mechanism in case of an anomaly is shown in section S11. In section S11, a message requesting driver takeover is output. This message may be one or both visual and auditory. For example, a message requesting driver takeover is displayed on a monitor located in vehicle C at a position that the driver can visually confirm.

[0062] In the following S12, restriction control is performed using normal components. Restriction control is the control that continues the automatic control of vehicle C without using the structure that detected the anomaly in the S1 process. If an anomaly is detected in power supply units 21 and 22, the components that supply power from the detected power supply units 21 and 22 are also not used. For example, if an anomaly is detected in the first power supply unit 21, the front millimeter-wave radar 30 and the front camera 40 are not used in the control. Furthermore, if an anomaly is detected in the first power supply unit 21, restriction control is not performed by SoC 71, which receives power from the first power supply unit 21, but by SoC 72. In restriction control, if only one of the front millimeter-wave radar 30 and the front camera 40, which are the first sensor units, is abnormal, the entire first sensor unit may not be used in the control. Similarly, in restriction control, if only one of the front side millimeter-wave radar 50 and the peripheral camera 60, which are the second sensor units, is abnormal, the entire second sensor unit may not be used in the control. Automatic driving of vehicle C based on restriction control also includes a situation called degraded driving.

[0063] Compared to normal operation, the restriction control can utilize one less structure. Therefore, the restriction control is less reliable than normal autonomous driving control. Consequently, the restriction control performs more limited control than normal autonomous driving control. Specific examples of restriction include limiting functions, such as prohibiting lane changes. When lane changes are prohibited, lane keeping control continues. Additionally, the maximum speed limit for autonomous driving can be limited. Furthermore, the duration of autonomous driving can also be limited.

[0064] Although the control is more restricted than normal autonomous driving control, the restricted control can simultaneously control the longitudinal and lateral movements of vehicle C, preventing contact with obstacles in front, and enabling vehicle control without leaving the driving lane. It includes a front millimeter-wave radar 30 and a front camera 40 as first sensor units receiving power from the first power supply unit 21, and a front side millimeter-wave radar 50 and a peripheral camera 60 as second sensor units receiving power from the first power supply unit 21. Therefore, even if one of these sensors malfunctions, or even if either the first power supply unit 21 or the second power supply unit 22 malfunctions, longitudinal and lateral control can continue.

[0065] In S13, it is determined whether the elapsed time since the request for takeover from the driver in S11 has exceeded the preset takeover time. The takeover time can be a constant time or a time that varies depending on the type of abnormal structure or the type of road. An example of a takeover time is 15 seconds. If the result of S13 is "no", proceed to S14.

[0066] In step S14, it is determined whether the takeover is complete. The method for determining whether the takeover is complete can be set to various methods. For example, detecting that the driver's hands are on the steering wheel, such as through an in-vehicle camera, can be set as a condition for the takeover to be complete. Alternatively, detecting that the driver presses a pre-prepared takeover completion button can also be set as a condition for the takeover to be complete.

[0067] If the result of S14 is "yes", then proceed to S15. In S15, switch to manual control. Therefore, the restriction control ends. If the result of S14 is "no", then return to S12 and continue restriction control.

[0068] If the judgment in S13 is "yes," meaning that the takeover time has elapsed during the restriction control, proceed to S16. In S16, vehicle C is brought to an emergency stop. Furthermore, even if the takeover time has not elapsed, if an anomaly occurs in a portion of the structure, vehicle C can also be brought to an emergency stop.

[0069] [Summary of Implementation Methods]

[0070] In the embodiment described above, the front millimeter-wave radar 30, the front camera 40, and the SoC 71 are connected to the first power supply unit 21, and the front side millimeter-wave radar 50, the peripheral camera 60, and the SoC 72 are connected to the second power supply unit 22.

[0071] Therefore, even if any one or more of the front millimeter-wave radar 30, front camera 40, SoC 71, and first power supply unit 21 malfunctions, the vehicle C can be controlled simultaneously in both longitudinal and lateral directions and continue driving by using the front side millimeter-wave radar 50, peripheral camera 60, SoC 72, and second power supply unit 22 as limiting control (S12). Furthermore, even if any one or more of the front side millimeter-wave radar 50, peripheral camera 60, SoC 72, and second power supply unit 22 malfunctions, the vehicle C can be controlled simultaneously in both longitudinal and lateral directions and continue driving by using the front millimeter-wave radar 30, front camera 40, SoC 71, and first power supply unit 21 as limiting control (S12).

[0072] Furthermore, if the front millimeter-wave radar 30, the front camera 40, the front side millimeter-wave radar 50, the peripheral camera 60, the first power supply unit 21, and the second power supply unit 22 are all functioning normally, then all the sensors can be used to perform high-precision normal-time automatic driving control.

[0073] In other words, when based on normal autonomous driving control using all of the front millimeter-wave radar 30, front camera 40, front side millimeter-wave radar 50, and peripheral cameras 60, the vehicle device 10 of this embodiment also uses the same sensors as in normal autonomous driving control under restricted control. Therefore, it is possible to suppress the increase of constituent elements and simultaneously control longitudinal and lateral movements and continue vehicle travel even in abnormal situations.

[0074] The vehicle device 10 includes a first sensor unit consisting of a front millimeter-wave radar 30 and a front camera 40. Both the front millimeter-wave radar 30 and the front camera 40 are sensors for the front system, and are installed even in vehicles not operating at Level 3 autonomous driving for purposes such as inter-vehicle distance control and lane keeping control. Therefore, based on the structure of using these front millimeter-wave radars 30 and front cameras 40 as the first sensor unit, it is easy to subsequently add a second sensor unit, thus adding components to the vehicle device 10.

[0075] <Other Implementation Methods>

[0076] exist Figure 6 The diagram shows sensors that can be used in either the first or second sensor unit, including the sensors described in the first embodiment. Figure 6 The sensors marked with a checkmark in the longitudinal control section, namely V1~V3 and VH, indicate that they are sensors capable of detecting the information required for longitudinal control of vehicle C. Figure 6 The sensors marked with a checkmark in the lateral control section, namely H1 to H3 and VH, indicate that they are sensors capable of detecting the information required for lateral control of vehicle C.

[0077] The front millimeter-wave radar 30 in the first row and the front lateral millimeter-wave radar 50 in the second row have been described in the first embodiment. They are sensors capable of detecting information necessary for longitudinal control of the vehicle C. A lidar 80 is shown in the third row, serving as a sensor capable of detecting information necessary for longitudinal control of the vehicle C. Figure 7 The image illustrates the mounting location of the LiDAR 80. Figure 7 In the design, the LiDAR 80 is mounted at the front end of vehicle C, centered in the width direction. However, the LiDAR 80 can also be mounted at the location of the front camera 40, and its location is not limited to this. Figure 7 The position is shown. The field of view of the lidar 80 can also be the same as that of the front millimeter-wave radar 30. Alternatively, the field of view of the lidar 80 can be wider than that of the front millimeter-wave radar 30, for example, the same as that of the front camera 40. The lidar 80 detects the position of an object by illuminating its field of view with a laser and receiving the reflected light from that laser.

[0078] Figure 6 The front camera 40 in the fourth row and the peripheral camera 60 in the fifth row have also been described in the first embodiment. They are described as sensors capable of detecting information required for lateral control of the vehicle C.

[0079] The sixth row shows the position detection sensor 90, which is a sensor capable of detecting information necessary for lateral control of vehicle C. The position detection sensor 90 is used in conjunction with map 91. Figure 7 The diagram illustrates a position detection sensor 90 and a map 91. The position detection sensor 90 is a sensor that detects the current position of vehicle C. For example, a GNSS receiver is a specific example of the position detection sensor 90. Alternatively, the position detection sensor 90 can be configured by using an inertial sensor to successively detect the direction and distance of movement of vehicle C. Furthermore, the position detection sensor 90 can also be configured by combining a GNSS receiver and an inertial sensor.

[0080] Map 91, stored in a designated memory, is digital map data depicting the positions of lanes. The memory may contain all the digital map data pre-stored, or it may download and save data for areas determined based on the current position of vehicle C. ECU 70 functions as a position detection processing unit 92. The position detection processing unit 92 obtains signals from the position detection sensor 90 to determine the coordinates representing the position of vehicle C. The coordinates include (x, y) information. Additionally, the coordinates may also include height information.

[0081] Furthermore, the position detection processing unit 92 uses the coordinates and map 91 to successively determine the distance in the road width direction between vehicle C and the nearest road boundary to vehicle C, i.e., the road width direction distance. Since the coordinates contain (x, y) information, they are an example of lateral position information. By using the lane positions contained in map 91 with the coordinates, the road width direction distance can be determined.

[0082] In addition to the fourth line, the front camera 40 is also shown in the seventh line. However, in the seventh line, the... Figure 7 The distance detection processing unit 93 shown is used in conjunction with the front camera 40. Thus, as... Figure 6 As shown, it is capable of both longitudinal and lateral control.

[0083] The processing of the distance detection processing unit 93 will be explained. The distance detection processing unit 93 acquires image data captured by the front camera 40, which is a monocular camera, and calculates the distance to objects reflected in the image based on the image data. For example, the distance to an object can be calculated by utilizing the difference in the shape of the blur before and after the focal point in the captured image. Furthermore, for objects whose size can be determined or estimated, such as vehicles, the distance can also be calculated based on the size shown in the image. Therefore, the distance to obstacles in front can also be calculated, thereby achieving longitudinal control.

[0084] exist Figure 8 The diagram illustrates the sensor combination modes that the first sensor unit and the second sensor unit can achieve. Figure 8 In the "Combinations" column, the alphanumeric characters are recorded and... Figure 6 The alphanumeric symbols shown correspond to each other. The four or three sensors shown in the "combination" each contain two V and two H. The sensors shown in the "combination" are matched with the first sensor section and the second sensor section so that each contains one V and one H.

[0085] exist Figure 8 In the diagram, the line following the first sensor unit is labeled as a first power supply unit 21 and SoC 71, indicating that power is supplied from the first power supply unit 21 to the first sensor unit, and power is also supplied from the first power supply unit 21 to SoC 71. The line following the second sensor unit is labeled as a second power supply unit 22 and SoC 72, indicating that power is supplied from the second power supply unit 22 to the second sensor unit, and power is also supplied from the second power supply unit 22 to SoC 72.

[0086] Through such Figure 8 As shown in the sensor combination, similar to the first embodiment, even if any one or more of the first sensor unit, first power supply unit 21, and SoC 71 malfunctions, longitudinal and lateral control of the vehicle C can continue through the second sensor unit, second power supply unit 22, and SoC 72. Furthermore, even if any one or more of the second sensor unit, second power supply unit 22, and SoC 72 malfunctions, longitudinal and lateral control of the vehicle C can continue through the first sensor unit, first power supply unit 21, and SoC 71. Moreover, if both the first and second sensor units are functioning normally, high-precision normal-time autonomous driving control can be performed simultaneously using both the first and second sensor units.

[0087] In the second row of Mode 1, the first sensor unit includes a front-side millimeter-wave radar 50 and a front camera 40, and the second sensor unit includes a front-side millimeter-wave radar 30 and a peripheral camera 60. When combined, they form the same sensor combination as in the first embodiment. Therefore, the combination in the second row of Mode 1 enables the same normal-time autonomous driving control as in the first embodiment.

[0088] In the first row of Mode 2, the first sensor unit includes a front-side millimeter-wave radar 50 and a front camera 40, while the second sensor unit includes a lidar 80, a position detection sensor 90, and a map 91. The first sensor unit is the same as that in the first embodiment. Therefore, it is easy to subsequently add a second sensor unit, thus adding to the structure of the vehicle device 10. In the first row of Mode 2, since the first sensor unit includes a front millimeter-wave radar 30 and the second sensor unit includes a lidar 80, high-precision longitudinal control using the front millimeter-wave radar 30 and the lidar 80 can be performed under normal autonomous driving control.

[0089] When the first sensor unit and the second sensor unit are combined, the second row of Mode 2 is the same as the first row of Mode 2. Therefore, similar to the first row of Mode 2, high-precision longitudinal control using the front millimeter-wave radar 30 and the lidar 80 is possible during normal autonomous driving control.

[0090] In the first row of Mode 3, the first sensor unit includes a front-side millimeter-wave radar 50 and a front camera 40, and the second sensor unit includes a lidar 80 and a peripheral camera 60. The first sensor unit is the same as the first sensor unit in the first embodiment. Therefore, it is easy to subsequently add a second sensor unit, thereby adding to the structure of the vehicle device 10. In addition, under normal autonomous driving control, high-precision longitudinal control using the front millimeter-wave radar 30 and lidar 80 can be performed.

[0091] When the first sensor unit and the second sensor unit are combined, the second row of Mode 3 is the same as the first row of Mode 3. Therefore, in this mode, high-precision longitudinal control using the front millimeter-wave radar 30 and lidar 80 can also be performed in normal autonomous driving control.

[0092] Mode 4 is a structure in which the first sensor unit has a front camera 40 and a distance detection and processing unit 93, and the second sensor unit has a LiDAR 80, a position detection sensor 90 and a map 91.

[0093] Mode 5 is a structure in which the first sensor unit includes a front camera 40 and a distance detection processing unit 93, and the second sensor unit also includes a front camera 40 and a distance detection processing unit 93. Figure 6 In the symbols shown, both are VH. However, to indicate that they are distinct constituent elements, in Figure 8 The two are referred to as VH1 and VH2. The front camera 40 of the first sensor unit is a first front camera, and the front camera 40 of the second sensor unit is a second front camera.

[0094] According to the structure of Mode 5, in normal autonomous driving control, since two front cameras 40 can be used, these two front cameras 40 can function as stereo cameras.

[0095] Mode 6 is a structure in which the first sensor unit has a front camera 40 and a distance detection and processing unit 93, and the second sensor unit has a front side millimeter-wave radar 50 and a peripheral camera 60.

[0096] In Mode 7, the first row shows a structure where the first sensor unit includes a front-side millimeter-wave radar 50 and a front camera 40, and the second sensor unit includes the front-side millimeter-wave radar 50, a position detection sensor 90, and a map 91. The first sensor unit is the same as the first sensor unit in the first embodiment. Therefore, it is easy to subsequently add the second sensor unit to construct the structure of the vehicle device 10.

[0097] In the second row of Mode 7, the first sensor unit includes a front-side millimeter-wave radar 50 and a front camera 40, while the second sensor unit includes a front-side millimeter-wave radar 30, a position detection sensor 90, and a map 91. When the first and second sensor units are combined, the second row of Mode 7 is the same as the first row of Mode 7.

[0098] The implementation methods have been described above, but the disclosed technology is not limited to the above-described implementation methods. The following variations are also included in the scope of the disclosure, and various changes and implementations can be made without departing from the spirit of the text, in addition to the following.

[0099] <Variation Example 1>

[0100] In this implementation, both SoC71 and SoC72 are capable of performing autonomous driving control under normal conditions. However, it is also possible for only either SoC71 or SoC72 to be capable of autonomous driving control under normal conditions.

[0101] <Variation Example 2>

[0102] In this implementation, if no anomaly is detected (S1: "No"), vertical synthesis (S4) and horizontal synthesis (S5) are performed. However, either S4 or S5 may be omitted, and only either vertical synthesis or horizontal synthesis is performed.

[0103] <Variation Example 3>

[0104] In S1, in addition to determining whether the power supply units 21 and 22, the first sensor unit, and the second sensor unit are abnormal, it also determines whether SoCs 71 and 72 are abnormal. However, for the SoC 71 or 72 that does not perform normal autonomous driving control, it is not necessary to determine whether it is abnormal. This is because it will not hinder normal autonomous driving control.

[0105] <Variation Example 4>

[0106] The SoCs 71 and 72 described in this disclosure are control units as described below. This control unit and its methods can also be implemented using a dedicated computer, which constitutes a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the control unit and its methods described in this disclosure can also be implemented using dedicated hardware logic circuitry. Alternatively, the control unit and its methods described in this disclosure can also be implemented using more than one dedicated computer, which is composed of a processor executing a computer program and a combination of more than one hardware logic circuit. The hardware logic circuitry is, for example, an ASIC or an FPGA.

[0107] Furthermore, the storage medium for storing computer programs is not limited to ROM; instructions executed by the computer can be stored on a non-transferable tangible recording medium that the computer can read. For example, the aforementioned program can also be stored in flash memory.

Claims

1. A vehicle device, used in a vehicle, wherein, have: First power supply section; Second power supply unit; The first sensor unit operates by being powered by the first power supply unit, and detects obstacles in front of the vehicle and detects information related to the vehicle's position in the road width direction, i.e., lateral position information. The first control device operates by supplying power from the first power supply unit and controls the vehicle based on the detection results of the first sensor unit. The second sensor unit operates by being powered by the second power supply unit, and detects the obstacle in front and the lateral position information. as well as The second control device operates by receiving power from the second power supply unit and controls the vehicle based on the detection results of the second sensor unit. When the first control device detects an anomaly in at least one of the second power supply unit, the second sensor unit, and the second control device, it performs degraded driving by using the first sensor unit instead of the second sensor unit. This degraded driving, while more restrictive than normal automatic driving control performed when no anomalies are detected, is still a vehicle control system that detects obstacles ahead and lateral position information and drives the vehicle. If at least one of the first power supply unit, the first sensor unit, and the first control unit malfunctions, the second control device will use the second sensor unit instead of the first sensor unit to perform the degraded driving. The aforementioned first sensor unit includes: a front millimeter-wave radar that uses a field of view encompassing the front of the vehicle to detect obstacles in front via millimeter waves; and a front camera that uses a field of view encompassing the front of the vehicle to its oblique side to detect lane boundaries present on the side of the vehicle as lateral position information. The aforementioned second sensor unit includes: a front-side millimeter-wave radar that covers a field of view extending from the side of the vehicle to the front of the vehicle, and detects obstacles in front of the vehicle using millimeter waves; and a peripheral camera that covers the area to the side of the vehicle and its periphery within its field of view, and detects lane boundaries existing to the side of the vehicle. The shooting range of the aforementioned peripheral cameras is closer to the aforementioned vehicle than the shooting range of the aforementioned front camera.

2. The vehicle device according to claim 1, wherein, For one or both of the aforementioned first control device and the aforementioned second control device, When the first power supply unit, the first sensor unit, the first control device, the second power supply unit, the second sensor unit, and the second control device are functioning normally, at least one of longitudinal integration processing and lateral integration processing is performed. The longitudinal integration processing is the process of obtaining detection results related to the obstacle in front from the first sensor unit and the second sensor unit respectively and integrating the detection results related to the obstacle in front. The lateral integration processing is the process of obtaining the lateral position information from the first sensor unit and the second sensor unit respectively and integrating the lateral position information.

3. The vehicle device according to claim 1 or 2, wherein, The second sensor unit includes: a lidar that uses the area including the front of the vehicle as its field of view to detect obstacles in front using laser light; and a position detection sensor that detects the coordinates of the vehicle as its lateral position information. The second control device obtains the position of the lane of the road on which the vehicle is traveling from the map data, and determines the distance in the road width direction between the vehicle and the lane based on the position of the lane and the coordinates of the vehicle detected by the second sensor unit.

4. The vehicle device according to claim 1 or 2, wherein, The second sensor unit includes: a lidar that uses the area including the front of the vehicle as its field of view to detect obstacles in front of it using laser; and a peripheral camera that includes the area to the side of the vehicle and around the vehicle in its field of view to detect lane boundaries existing to the side of the vehicle.

5. The vehicle device according to claim 1 or 2, wherein, The second sensor unit described above includes a front-side millimeter-wave radar that covers a range extending from the side of the vehicle to the front of the vehicle within its field of view, and detects obstacles in front of the vehicle using millimeter waves. And a position detection sensor to detect the coordinates of the vehicle as the aforementioned lateral position information. The second control device obtains the position of the lane of the road on which the vehicle is traveling from the map data, and determines the distance in the road width direction between the vehicle and the lane based on the position of the lane and the coordinates of the vehicle detected by the second sensor unit.

6. The vehicle device according to claim 1 or 2, wherein, The aforementioned first sensor unit includes: a front-side millimeter-wave radar that covers a field of view extending from the side of the vehicle to the front of the vehicle, and detects obstacles in front using millimeter waves; and a front-side camera that covers the front to the oblique side of the vehicle in its field of view, and detects lane boundaries existing on the side of the vehicle as lateral position information. The aforementioned second sensor unit includes: a front millimeter-wave radar that uses the area encompassing the front of the vehicle as its field of view to detect obstacles in front using millimeter waves; and a peripheral camera that uses the portion to the side of the vehicle and its periphery as its field of view to detect lane boundaries existing to the side of the vehicle.

7. The vehicle device according to claim 1 or 2, wherein, The first sensor unit includes a first front-facing camera, which has a field of view that includes the front of the vehicle to its oblique side, and detects lane boundaries and obstacles in front of the vehicle as lateral position information. The second sensor unit includes a second front camera, which has a field of view that includes the front of the vehicle to the oblique side, and detects the lane boundary and the obstacle in front.

8. The vehicle device according to claim 1 or 2, wherein, The aforementioned first sensor unit includes a front camera, which has a field of view that includes the front of the vehicle to its oblique side, and detects obstacles in front of the vehicle and lane boundaries existing on the side of the vehicle as lateral position information. The aforementioned second sensor unit includes: a front side millimeter-wave radar that covers a range extending from the side of the vehicle to the front of the vehicle in its field of view and detects obstacles in front of the vehicle using millimeter waves; and a peripheral camera that covers the side of the vehicle and the area around the vehicle in its field of view and detects lane boundaries existing on the side of the vehicle.

9. The vehicle device according to claim 1 or 2, wherein, Either the first sensor unit or the second sensor unit described above is equipped with a position detection sensor that detects the coordinates of the vehicle as the lateral position information. The control device in the first control device and the second control device, which supplies power to the power supply unit that supplies power to the position detection sensor, obtains the position of the lane of the road on which the vehicle is traveling from map data, and determines the road width distance between the vehicle and the lane based on the position of the lane and the coordinates of the vehicle detected by the position detection sensor.

10. The vehicle device according to claim 1 or 2, wherein, The second control device does not accept power supply from the first power supply unit and determines whether the first power supply unit is abnormal. The first control device does not accept power supply from the second power supply unit and determines whether the second power supply unit is malfunctioning.

11. A vehicle control method, executed by a processor in a vehicle equipped with a first power supply unit; a second power supply unit; a first sensor unit powered from the first power supply unit that detects obstacles in front of the vehicle and detects information related to the vehicle's position in the road width direction, i.e., lateral position information; and a second sensor unit powered from the second power supply unit that detects the obstacles in front and detects the lateral position information, wherein... When the first power supply unit, the first sensor unit, the second power supply unit, and the second sensor unit are functioning normally, at least one of longitudinal integration processing and lateral integration processing is performed. The longitudinal integration processing is the process of obtaining detection results related to the obstacle in front from the first sensor unit and the second sensor unit respectively and integrating the detection results related to the obstacle in front. The lateral integration processing is the process of obtaining lateral position information from the first sensor unit and the second sensor unit respectively and integrating the lateral position information. If an anomaly is detected in at least one of the second power supply unit and the second sensor unit, degraded driving is performed using the first sensor unit instead of the second sensor unit. This degraded driving, although limited compared to normal automated driving control performed when no anomaly is detected, is a vehicle control system that detects obstacles ahead and lateral position information and drives the vehicle. If an abnormality is detected in at least one of the first power supply unit and the first sensor unit, the degraded driving is performed using the second sensor unit instead of the first sensor unit. The aforementioned first sensor unit includes: a front millimeter-wave radar that uses a field of view encompassing the front of the vehicle to detect obstacles in front via millimeter waves; and a front camera that uses a field of view encompassing the front of the vehicle to its oblique side to detect lane boundaries present on the side of the vehicle as lateral position information. The aforementioned second sensor unit includes: a front-side millimeter-wave radar that covers a field of view extending from the side of the vehicle to the front of the vehicle, and detects obstacles in front of the vehicle using millimeter waves; and a peripheral camera that covers the area to the side of the vehicle and its periphery within its field of view, and detects lane boundaries existing to the side of the vehicle. The shooting range of the aforementioned peripheral cameras is closer to the aforementioned vehicle than the shooting range of the aforementioned front camera.

Images