Steering adjustment device and steering adjustment method

Abstract

A steering adjustment device and method are provided to suppress steering operations by the driver in directions in which the driver is not fully aware of the situation during automatic driving. The steering adjustment device detects the driver's gaze direction from a facial image representing the driver's face. When the driver's operation on the steering operation receiving unit is detected during automatic driving control of the vehicle's steering, the device determines whether the driver's gaze direction matches the steering direction of the operation. The device adjusts the reaction force so that the reaction force corresponding to the operation on the steering operation receiving unit when the gaze direction matches the steering direction is less than the reaction force when the gaze direction does not match the steering direction.

Description

Technical Field

[0001] This disclosure relates to a steering adjustment device and a steering adjustment method for adjusting the steering of a vehicle. Background Technology

[0002] The following steering adjustment device is known: in order to enable the driver to operate the steering wheel appropriately according to the vehicle's condition, it generates a reaction force on the "operation of the steering wheel" according to the vehicle's condition.

[0003] For example, the steering force correction device described in Patent Document 1 uses the speed and distance of other vehicles traveling in adjacent lanes to calculate the degree of danger (risk level) indicating the possibility that the other vehicle will approach its own vehicle in the future. For example, if it is believed that turning right is less dangerous than going straight, the steering force on the right side is lighter than the steering force on the left side.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 9-066853 Summary of the Invention

[0007] The problem that the invention aims to solve

[0008] During automated driving, when the vehicle is controlled by a driving control unit, if the driver operates the steering wheel beyond a predetermined threshold, the driving control unit must prioritize "control quantities based on the driver's operation" over "control quantities determined by automated driving" to control the vehicle's steering direction and steering amount. In this situation, if the driver does not adequately understand the situation in the steering direction corresponding to the steering wheel operation (e.g., the position of other vehicles traveling in adjacent lanes in the steering direction), safety after steering may not be adequately ensured.

[0009] The purpose of this disclosure is to provide a steering adjustment device that can suppress steering operations by the driver in a direction in which the driver is not fully aware of the situation during autopilot.

[0010] Technical solutions for solving the problem

[0011] The main points of this disclosure are as follows.

[0012] (1) A steering adjustment device, comprising:

[0013] The detection unit detects the driver's gaze direction from a facial image representing the driver's face; and

[0014] During automatic driving, when the vehicle's steering is automatically controlled, the reaction force adjustment unit detects an operation by the driver to a steering operation receiving unit that receives an operation specifying the steering direction and degree of steering of the vehicle performed by the driver. It then determines whether the driver's line of sight aligns with the steering direction in the driver's operation, and adjusts the reaction force such that the reaction force corresponding to the operation by the steering operation receiving unit is less when the line of sight does not align with the steering direction.

[0015] (2) According to the steering adjustment device described in (1) above,

[0016] The reaction force adjustment unit adjusts the reaction force in such a way that the reaction force corresponding to the operation of the steering operation receiving unit during the autopilot is of a predetermined magnitude, and adjusts the reaction force in such a way that the reaction force when the line of sight does not conform to the steering direction is greater than the reaction force during the autopilot.

[0017] (3) The steering adjustment device according to (1) or (2) above,

[0018] It also includes a hazard determination unit that, based on the relative positional relationship between objects detected from a surrounding image representing the vehicle's surroundings and the vehicle itself, determines the degree of hazard indicating the likelihood of a dangerous situation occurring while the vehicle is moving in the steering direction.

[0019] The reaction force adjustment unit adjusts the reaction force in such a way that the higher the degree of danger, the heavier the reaction force.

[0020] (4) A steering adjustment method, performed by a steering adjustment device capable of adjusting a reaction force corresponding to an operation on a steering operation receiving unit, wherein the steering operation receiving unit is subject to an operation by the driver of the vehicle specifying a steering direction and steering degree.

[0021] The steering adjustment method includes:

[0022] Detect the driver's gaze direction from a facial image representing the driver's face in the vehicle; and

[0023] If, during automatic driving of the vehicle, the driver's operation on the steering operation receiving unit is detected, it is determined whether the driver's line of sight is consistent with the steering direction of the driver's operation. The reaction force is adjusted in such a way that the reaction force corresponding to the operation on the steering operation receiving unit when the line of sight is consistent with the steering direction is less than the reaction force when the line of sight is inconsistent with the steering direction.

[0024] (5) A non-transitory computer-readable medium storing a steering adjustment computer program that causes a computer mounted on a vehicle to execute:

[0025] Detect the driver's gaze direction from a facial image representing the driver's face in the vehicle; and

[0026] During automatic driving, when the driver operates a steering operation receiving unit that receives an operation specifying the steering direction and degree of steering of the vehicle, the system determines whether the driver's line of sight aligns with the steering direction in the driver's operation. The system then adjusts the reaction force such that the reaction force corresponding to the operation of the steering operation receiving unit is less when the line of sight aligns with the steering direction than when the line of sight does not align with the steering direction.

[0027] According to the steering adjustment device disclosed herein, it is possible to suppress steering operations by the driver in a direction in which the driver is not fully aware of the situation during autopilot. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of a vehicle equipped with a steering adjustment device.

[0029] Figure 2 This is a hardware schematic diagram of the steering adjustment device.

[0030] Figure 3 This is a functional block diagram of the processor in the steering adjustment device.

[0031] Figure 4 This is a schematic diagram illustrating an example of reaction force adjustment.

[0032] Figure 5 This is a flowchart of the steering adjustment process.

[0033] Explanation of reference numerals in the attached figures

[0034] 1: Vehicle;

[0035] 8: Steering adjustment device;

[0036] 831: Driving control unit;

[0037] 832: Testing Department;

[0038] 833: Hazard Assessment Department;

[0039] 834: Reaction force adjustment section. Detailed Implementation

[0040] Hereinafter, with reference to the accompanying drawings, a steering adjustment device that can suppress steering operations by the driver in directions in which the driver is not fully aware of the situation during automatic driving will be described in detail. The steering adjustment device detects the driver's gaze direction from a facial image representing the driver's face. When the steering adjustment device detects the driver's operation on the steering operation receiver during automatic driving, which automatically controls the vehicle's steering, it determines whether the driver's gaze direction matches the steering direction of the driver's operation. The steering operation receiver is a device, such as a steering wheel, that receives operations performed by the driver specifying the steering direction and degree of steering of the vehicle. The steering adjustment device adjusts the reaction force in such a way that the reaction force corresponding to the operation on the steering operation receiver when the gaze direction matches the steering direction is less than the reaction force when the gaze direction does not match the steering direction.

[0041] Figure 1 This is a schematic diagram of a vehicle equipped with a steering adjustment device.

[0042] Vehicle 1 has a peripheral camera 2, a driver monitoring camera 3, a steering wheel 4, a steering controller 5, a GNSS receiver 6, a storage device 7, and a steering adjustment device 8. The peripheral camera 2, the driver monitoring camera 3, the steering controller 5, the GNSS receiver 6, and the storage device 7 are communicatively connected to the steering adjustment device 8 via an in-vehicle network based on the Controller Area Network (CLAN) standard.

[0043] The peripheral camera 2 is an example of a peripheral sensor used to generate peripheral data representing the surrounding conditions of the vehicle 1. The peripheral camera 2 has a 2D detector composed of an array of photoelectric conversion elements such as CCD or C-MOS that are sensitive to visible light, and an optical imaging system that images the area on the 2D detector that is the subject of the photograph. The peripheral camera 2 is mounted, for example, facing forward in the upper front part of the vehicle interior. The peripheral camera 2 takes pictures of the surrounding conditions of the vehicle 1 through the windshield at predetermined shooting cycles (e.g., 1 / 30 to 1 / 10 of a second), and outputs peripheral images as peripheral data representing the surrounding conditions of the vehicle 1. Alternatively, the vehicle 1 may also have, for example, a LiDAR (Light Detection and Range) sensor as a peripheral sensor, which generates a distance image based on the surrounding conditions of the vehicle 1, where each pixel has a value corresponding to the distance to the object represented by that pixel, as peripheral data.

[0044] The driver monitoring camera 3 is an example of an in-vehicle sensor used to generate an output signal representing the driver's condition. The driver monitoring camera 3 has a 2D detector composed of an array of photoelectric conversion elements such as CCDs or C-MOS that are sensitive to infrared light, and an imaging optical system that images the area on the 2D detector that is the subject of the photograph. Additionally, the driver monitoring camera 3 has a light source that emits infrared light. The driver monitoring camera 3 is mounted, for example, at the front of the vehicle interior, facing the face of the driver seated in the driver's seat. The driver monitoring camera 3 illuminates the driver with infrared light at predetermined shooting intervals (e.g., 1 / 30 to 1 / 10 of a second), and outputs a facial image representing the driver's face as an output signal in a time sequence.

[0045] Steering wheel 4 is an example of a steering operation receiving unit that receives and adjusts the steering direction and steering amount of vehicle 1. Steering wheel 4 outputs a signal corresponding to the "operation by the driver requesting the steering mechanism to operate the vehicle 1." The "operation requesting the steering mechanism to operate" is, for example, rotating steering wheel 4 to the right or left. Steering wheel 4 has an actuator that generates a reaction force corresponding to the driver's operation.

[0046] The steering wheel 4 may also have a steering wheel holding sensor. The steering wheel holding sensor is an example of a sensor installed in the vehicle 1. The steering wheel holding sensor outputs a steering wheel holding signal to the steering controller 5 corresponding to whether the driver holds (grips) the steering wheel 4. The steering wheel holding sensor is, for example, a capacitive sensor installed in the steering wheel 4. The steering wheel holding sensor outputs a signal corresponding to the different capacitances when the driver holds (grips) the steering wheel 4 and is not holding (gripping) it, thus not touching the steering wheel 4.

[0047] The steering controller 5 is an ECU (Electronic Control Unit) with a communication interface, memory, and processor. The steering controller 5 is an example of a control device located in the vehicle 1. It receives signals corresponding to the driver's operations and steering wheel holding signals from the steering wheel 4 and sends them to the steering adjustment device 8.

[0048] GNSS receiver 6 receives GNSS signals from GNSS (Global Navigation Satellite System) satellites at predetermined intervals and determines the vehicle 1's own position based on the received GNSS signals. At predetermined intervals, GNSS receiver 6 outputs a positioning signal, representing the determination result of the vehicle 1's own position based on the GNSS signals, to the steering adjustment device 8 via the in-vehicle network.

[0049] Storage device 7 is an example of a storage unit, such as a hard disk drive or a non-volatile semiconductor memory. Storage device 7 stores map data containing information related to land features such as lane markings in a location-dependent manner.

[0050] The steering adjustment device 8 detects the driver's gaze direction from the facial image. Furthermore, during automatic driving, if the steering adjustment device 8 detects whether the driver is operating the steering wheel 4, it determines whether the driver's gaze direction matches the steering direction being operated. If the gaze direction does not match the steering direction, the steering adjustment device 8 adjusts the reaction force in a manner that makes the reaction force corresponding to the operation of the steering wheel 4 less than the reaction force when the gaze direction does not match the steering direction.

[0051] Figure 2 This is a hardware schematic diagram of the steering adjustment device 8. The steering adjustment device 8 includes a communication interface 81, a memory 82, and a processor 83.

[0052] Communication interface 81, as an example of the communication unit, has a communication interface circuit for connecting the steering adjustment device 8 to the in-vehicle network. Communication interface 81 supplies received data to processor 83. Additionally, communication interface 81 outputs data supplied from processor 83 to external devices.

[0053] The memory 82 includes both volatile and non-volatile semiconductor memory. The memory 82 stores various data used in the processing performed by the processor 83, such as the value of the reaction force of the steering wheel 4 corresponding to the driver's line of sight and steering direction, and threshold values ​​for the steering wheel 4's operation amount used to determine "whether to prioritize the control amount based on driver operation to adjust the steering amount of vehicle 1 during automatic driving." Additionally, the memory 82 stores various application programs, such as steering adjustment programs that perform steering adjustment processing.

[0054] Processor 83 is an example of a control unit, having one or more processors and their peripheral circuitry. Processor 83 may also have other arithmetic circuitry such as logic operation units, numerical operation units, or graphics processing units.

[0055] Figure 3 This is a functional block diagram of the processor 83 in the steering adjustment device 8.

[0056] The processor 83 of the steering adjustment device 8 has a driving control unit 831, a detection unit 832, a danger determination unit 833, and a reaction force adjustment unit 834 as functional blocks. Each of these units in the processor 83 is a functional module installed via a program executed on the processor 83. The computer program that implements the functions of each unit in the processor 83 can be provided in the form of a computer-readable removable recording medium such as a semiconductor memory, magnetic recording medium, or optical recording medium. Alternatively, each of these units in the processor 83 can also be installed in the steering adjustment device 8 as a separate integrated circuit, microprocessor, or firmware.

[0057] The driving control unit 831 controls the steering of the vehicle 1. In addition to steering the vehicle 1, the driving control unit 831 can also control the acceleration and deceleration of the vehicle 1.

[0058] The driving control unit 831 receives surrounding images generated by the surrounding camera 2 via the communication interface 81. The driving control unit 831 detects lane markings around the vehicle 1 by inputting the received surrounding images into a recognizer that has been pre-learned for detecting lane markings. Additionally, the driving control unit 831 detects other vehicles around the vehicle 1 by inputting the received surrounding images into a recognizer that has been pre-learned for detecting other vehicles.

[0059] The recognizer can be configured, for example, as a convolutional neural network (CNN) with multiple convolutional layers connected in series from the input side to the output side. By feeding the CNN with images containing lane markings or other vehicles as teacher data and learning from them, the CNN acts as a recognizer that detects lane markings or other vehicles from images.

[0060] For example, the driving control unit 831 determines the acceleration and steering input of vehicle 1 based on the detected lane markings, in a manner that ensures vehicle 1 travels appropriately within the lane and maintains a distance greater than a predetermined distance from other detected vehicles. Then, the driving control unit 831 outputs control signals corresponding to the determined acceleration and steering inputs to the driving mechanism (not shown) of vehicle 1 via the communication interface 81. The driving mechanism includes, for example, a drive source such as an engine or motor that supplies power to vehicle 1, brakes that reduce the speed of vehicle 1, and a steering mechanism for steering vehicle 1.

[0061] The function block corresponding to the driving control unit 831 can be installed in a processor different from the steering adjustment device 8. In this case, the steering adjustment device 8 may not have a driving control unit 831.

[0062] The detection unit 832 detects the driver's gaze direction from a facial image representing the driver's face of vehicle 1.

[0063] The detection unit 832 acquires the facial image generated by the driver monitoring camera 3 via the communication interface 81. For example, the detection unit 832 detects the pupil and the corneal reflection of the light source by performing template matching between each template in the template of the pupil and the template of the corneal reflection of the light source and the facial image, and detects the driver's gaze direction based on their positional relationship.

[0064] The danger determination unit 833 determines the degree of danger of a dangerous situation that may occur when the vehicle 1 is moving in the steering direction, based on the relative positional relationship between objects detected from a surrounding image representing the surrounding conditions of the vehicle 1 and the vehicle 1.

[0065] The danger determination unit 833 detects other vehicles from the series of surrounding images by inputting a series of surrounding images obtained by the surrounding camera 2 within a recent predetermined period into a recognizer that has been pre-learned for detecting objects such as vehicles from the images.

[0066] The hazard assessment unit 833 determines the orientation of other vehicles as observed from vehicle 1 based on the detected positions of other vehicles in the surrounding image. The hazard assessment unit 833 determines the driving direction of the lane located at the determined orientation of other vehicles based on its own position in the lane information obtained from the storage device 7 according to its own position indicated by the positioning signal received from the GNSS receiver 6, and estimates the orientation of other vehicles relative to vehicle 1. The hazard assessment unit 833 estimates the distance to other vehicles based on the ratio of the size of the area representing other vehicles in the surrounding image to the reference size of other vehicles in the image at the estimated orientation, assuming the distance to other vehicles is a reference distance, and the actual spatial dimensions of other vehicles.

[0067] The danger determination unit 833 estimates the relative positions of other vehicles based on vehicle 1, based on the orientation and distance of other vehicles observed from vehicle 1 from a series of peripheral images obtained by the peripheral camera 2 within the most recent predetermined period, and the position of vehicle 1 when the series of peripheral images were generated.

[0068] The hazard determination unit 833 estimates the relative speed of other vehicles corresponding to the time interval of obtaining the pair of peripheral images by dividing the interval of the relative positions of other vehicles in each pair of peripheral images in a series of peripheral images by the time interval of obtaining the pair of peripheral images. The reference distance, the reference size of the detected other vehicles on the image, and the size of the actual space can be pre-stored in the memory 82, for example.

[0069] The danger assessment unit 833 calculates the degree of danger of the possibility of a dangerous situation occurring when vehicle 1 is traveling to the right and left of the current direction of travel, respectively, based on the estimated relative position and relative speed of other vehicles.

[0070] For example, the danger determination unit 833 can apply the distance between other vehicles detected on one side of the vehicle 1 (left or right) and the relative speed of those other vehicles to a calculation formula pre-stored in the memory 82 to calculate the degree of danger if the vehicle is traveling towards that side.

[0071] The formula used to calculate the degree of danger can be set such that the shorter the distance from other vehicles, the higher the degree of danger.

[0072] Furthermore, when the relative speeds of other vehicles ahead of vehicle 1 are positive, the distance between vehicle 1 and other vehicles increases over time; conversely, when the relative speeds are negative, the distance decreases. Therefore, the formula for calculating the degree of danger can be set such that the greater the negative relative speed of other vehicles, the higher the degree of danger.

[0073] When the reaction force adjustment unit 834 detects the driver's operation on the steering wheel 4 during automatic driving, it adjusts the driver's line of sight to match the steering direction of the driver's operation, and generates a reaction force corresponding to the operation on the steering wheel 4.

[0074] Figure 4 This is a schematic diagram illustrating an example of reaction force adjustment.

[0075] The reaction force adjustment unit 834 first determines whether the driver of vehicle 1 has operated the steering wheel 4 during automatic driving.

[0076] The reaction force adjustment unit 834 can determine whether it is in autopilot mode by referring to the operating state of the driving control unit 831. Furthermore, the reaction force adjustment unit 834 can determine whether operation on the steering wheel 4 is detected based on signals received from the steering controller 5 via the communication interface 81. The reaction force adjustment unit 834 can also detect operation on the steering wheel 4 if the amount of operation based on the signal received from the steering controller 5 exceeds an operation amount threshold stored in the memory 82. Figure 4 In the example, the reaction force adjustment unit 834 detected a right-hand steering operation SO R .

[0077] When driver operation on steering wheel 4 is detected during autonomous driving, reaction force adjustment unit 834 determines whether the driver's line of sight obtained from detection unit 832 matches the driver's steering direction detected based on the signal received from steering controller 5. Reaction force adjustment unit 834 can determine that the line of sight matches the steering direction if both the line of sight and the steering direction are in the same direction (right or left) relative to the vehicle 1's direction of travel. Furthermore, reaction force adjustment unit 834 can determine that the line of sight matches the steering direction if the angle between the "line of sight" and the "direction of travel of vehicle 1 when steering is performed according to the steering amount received from steering controller 5" is less than a predetermined angle threshold.

[0078] The reaction force adjustment unit 834 adjusts the reaction force so that the reaction force corresponding to the operation of the steering wheel 4 when the line of sight is in the steering direction is less than the reaction force when the line of sight is not in the steering direction.

[0079] exist Figure 4 In the example, the driver's line of sight LD1, which is for the left direction, does not conform to the steering operation SO, which is for the right direction. R On the other hand, the driver's line of sight LD2, which is for right-hand drive, corresponds to the steering operation SO, which is for right-hand drive. R The reaction force adjustment unit 834 adjusts the force so that the reaction force RF2 when the line of sight direction LD2 is detected is less than the reaction force RF1 when the line of sight direction LD1 is detected.

[0080] For example, when autopilot is activated, the reaction force adjustment unit 834 sends a control signal to the steering controller 5 in such a way that the value of the reaction force corresponding to the operation of the steering wheel 4 becomes a first value (e.g., 100). When the reaction force adjustment unit 834 detects an operation of the steering wheel 4 during autopilot, it sends a control signal to the steering controller 5 in such a way that a second value (e.g., 80) representing the reaction force corresponding to the operation of the steering wheel 4, where the line of sight aligns with the steering direction, becomes less than a third value (e.g., 120) representing the reaction force, where the line of sight does not align with the steering direction.

[0081] Furthermore, the third value can also be equal to or less than the first value (e.g., 80). In this case, the reaction force adjustment unit 834 sends a control signal to the steering controller 5 in such a way that the second value becomes a smaller value than the third value, which is equal to or less than the first value (e.g., 60).

[0082] The reaction force adjustment unit 834 can also adjust the reaction force so that the reaction force when the line of sight is not aligned with the steering direction is greater than the reaction force during autopilot. By operating in this way, the steering adjustment device 8 can suppress steering operations when the pilot is not aligned with the line of sight.

[0083] For example, if the reaction force adjustment unit 834 detects an operation on the steering wheel 4 during autopilot, it sends a control signal to the steering controller 5 in such a way that the value of the reaction force indicating that the line of sight does not conform to the steering direction becomes the third value mentioned above.

[0084] Furthermore, the second value can also be equal to or greater than the first value (e.g., 120). In this case, the reaction force adjustment unit 834 sends a control signal to the steering controller 5 in such a way that the third value becomes a larger value than the second value, which is equal to or greater than the first value (e.g., 140).

[0085] Furthermore, the reaction force adjustment unit 834 can also adjust the reaction force in such a way that the higher the degree of danger, the stronger the reaction force. By operating in this way, the steering adjustment device 8 can suppress steering operations in the direction of high danger.

[0086] For example, the reaction force adjustment unit 834 obtains the degree of danger when the vehicle travels in the steering direction from the danger determination unit 833. Then, the reaction force adjustment unit 834 sends a control signal to the steering controller 5 in such a way that the higher the degree of danger, the greater the reaction force.

[0087] Figure 5This is a flowchart of the steering adjustment process. During the period when the vehicle 1 is in automatic driving mode, the processor 83 of the steering adjustment device 8 repeatedly executes the steering adjustment process described below at predetermined cycles.

[0088] First, the detection unit 832 of the processor 83 of the steering adjustment device 8 detects the direction of the driver's gaze of the vehicle 1 from the facial image generated by the driver monitoring camera 3 (step S1).

[0089] The reaction force adjustment unit 834 of the processor 83 determines whether the driver of the vehicle 1 has operated the steering wheel 4 (step S2). If no operation on the steering wheel 4 is detected (step S2: N), the reaction force adjustment unit 834 ends the steering adjustment process.

[0090] If an operation on the steering wheel 4 is detected (step S2: Y), the reaction force adjustment unit 834 determines whether the driver's line of sight conforms to the steering direction detected by the driver based on the signal received from the steering controller 5 (step S3). If the line of sight does not conform to the steering direction (step S3: N), the reaction force adjustment unit 834 terminates the steering adjustment process.

[0091] When the line of sight is in line with the steering direction (step S3: Y), the reaction force adjustment unit 834 adjusts the reaction force in such a way that the reaction force corresponding to the operation of the steering wheel 4 is less than the reaction force when the line of sight is not in line with the steering direction (step S4), and ends the steering adjustment process.

[0092] By performing steering adjustment processing in this way, it is possible to suppress the driver's steering operations in directions in which the driver is not fully aware of the situation during autopilot.

[0093] It should be understood that those skilled in the art can make various changes, substitutions and modifications to this disclosure without departing from its spirit and scope.

Claims

1. A steering adjustment device, comprising: The detection unit detects the driver's gaze direction from a facial image representing the driver's face; and The reaction force adjustment unit, during automatic driving of the vehicle, detects an operation by the driver on a steering operation receiving unit that receives an operation specifying the steering direction and degree of steering of the vehicle performed by the driver. It then determines whether the driver's line of sight aligns with the steering direction in the driver's operation, and adjusts the reaction force in such a way that the reaction force corresponding to the operation on the steering operation receiving unit when the line of sight aligns with the steering direction is less than the reaction force when the line of sight does not align with the steering direction.

2. The steering adjustment device according to claim 1, The reaction force adjustment unit adjusts the reaction force in such a way that the reaction force corresponding to the operation of the steering operation receiving unit during the autopilot is of a predetermined magnitude, and adjusts the reaction force in such a way that the reaction force when the line of sight does not conform to the steering direction is greater than the reaction force during the autopilot.

3. The steering adjustment device according to claim 1, It also includes a hazard determination unit that, based on the relative positional relationship between objects detected from a surrounding image representing the vehicle's surroundings and the vehicle itself, determines the degree of hazard indicating the likelihood of a dangerous situation occurring while the vehicle is moving in the steering direction. The reaction force adjustment unit adjusts the reaction force in such a way that the higher the degree of danger, the heavier the reaction force.

4. A steering adjustment method, performed by a steering adjustment device capable of adjusting a reaction force corresponding to an operation on a steering operation receiving unit, wherein the steering operation receiving unit is subject to a specified steering direction and steering degree operation performed by the driver of the vehicle. The steering adjustment method includes: Detect the driver's gaze direction from a facial image representing the driver's face in the vehicle; and If, during automatic driving of the vehicle, the driver's operation on the steering operation receiving unit is detected, it is determined whether the driver's line of sight is consistent with the steering direction of the driver's operation. The reaction force is adjusted in such a way that the reaction force corresponding to the operation on the steering operation receiving unit when the line of sight is consistent with the steering direction is less than the reaction force when the line of sight is inconsistent with the steering direction.

5. A non-transitory computer-readable medium storing a steering adjustment computer program for causing a computer mounted in a vehicle to perform the following processes: Detect the driver's gaze direction from a facial image representing the driver's face in the vehicle; and During automatic driving, when the driver's operation of the steering operation receiving unit is detected, indicating that the steering direction and degree of steering of the vehicle are specified by the driver, it is determined whether the driver's line of sight is consistent with the steering direction in the driver's operation. The reaction force is adjusted in such a way that the reaction force corresponding to the operation of the steering operation receiving unit when the line of sight is consistent with the steering direction is less than the reaction force when the line of sight is inconsistent with the steering direction.

Images