Mobile

Abstract

A vehicle (100) performs autonomous driving. The vehicle (100) comprises: an acquisition unit (115) that acquires line-of-sight information, a plurality of images obtained by imaging an area ahead of the driver, a request for switching to semi-autonomous driving or manual driving, and risk potential energy; a detection unit (118) that detects gaze point coordinates on the basis of one of the plurality of images and the line-of-sight information; a calculation unit (114) that calculates a velocity aftereffect energy using a plurality of field-of-view images and the gaze point coordinates and, in cases where velocity aftereffect energy is present, calculates a velocity aftereffect duration from the velocity aftereffect energy; and a determination unit (120) that, using the velocity aftereffect duration, decides whether to switch the driving system from autonomous driving to semi-autonomous driving or manual driving.

Description

[Technical Field]

[0001] The present disclosure relates to mobile objects. [Background technology]

[0002] Autonomous driving of cars is well known, and technologies relating to autonomous driving have been proposed (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7173105 [Non-patent literature]

[0004] [Non-Patent Document 1] Yusuke Yashiro et al., "Development of Obstacle Avoidance Technology for Robotic Products Using Potential Method," Mitsubishi Heavy Industries Technical Review Vol. 51 No. 1, 2014 Summary of the Invention [Problem to be solved by the invention]

[0005] It is known that driving systems can switch from automated to manual driving. If the driving system switches from automated to manual driving without understanding the driver's state, there is a high possibility that the driver will not be able to drive the car correctly.

[0006] The objective of the present disclosure is to gradually switch from automated driving to manual driving while the driving system grasps the driver's situation and assists the driver in driving operations so that the driver can continue to drive manually when the driving system switches from automated driving to manual driving. [Means for solving the problem]

[0007] According to one aspect of the present disclosure, there is provided a mobile body. The mobile body is driven by a driver and is operating autonomously. The mobile body includes: an acquisition unit that acquires gaze information indicating the driver's gaze, video images including multiple pieces of distance information obtained by capturing images of the mobile body's direction of travel, a request to switch between manual driving and semi-autonomous driving, and a risk potential energy (RPE) representing the state of the road on which the mobile body is traveling and obstacles, expressed as energy; a detection unit that detects gaze point coordinates corresponding to the driver's gaze point based on one of the multiple videos and the gaze information; a calculation unit that calculates speed aftereffect energy indicating the energy of a speed aftereffect using the multiple videos and the gaze point coordinates, and calculates a speed aftereffect duration during which the speed aftereffect remains on the driver using a duration for which the speed aftereffect energy continues; and a determination unit that determines the timing to switch the driving system from autonomous driving to manual driving or semi-autonomous driving using the speed aftereffect duration. [Effects of the Invention]

[0008] According to the present disclosure, the driving system can grasp the driver's state and switch while assisting the driver in grasping the situation and performing driving operations to enable manual driving. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of a vehicle according to a first embodiment. [Figure 2] 1 is a diagram illustrating hardware included in a vehicle according to a first embodiment. [Figure 3] 1 is a block diagram showing the functions of a vehicle according to a first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of calculation of risk potential energy according to the first embodiment. [Figure 5] 1 is a flowchart (part 1) illustrating an example of processing executed by the vehicle in the first embodiment. [Figure 6] 10 is a flowchart (part 2) illustrating an example of processing executed by the vehicle in the first embodiment. [Figure 7]10 is a flowchart (part 3) illustrating an example of processing executed by the vehicle in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.

[0011] Embodiment 1 FIG. 1 is a diagram showing an example of a car according to the first embodiment. FIG. 1 shows a car 100. The car 100 is an example of a moving body. A driver is on board the car 100. The car 100 is driving automatically. The car 100 can be driven semi-automatically or manually. That is, the driver can drive the car 100 semi-automatically or manually.

[0012] Next, the hardware of the vehicle 100 will be described. 2 is a diagram showing hardware included in the car of embodiment 1. The car 100 includes a processor 101, a volatile storage device 102, a non-volatile storage device 103, an actuator 104, an outside-vehicle information acquisition device 105, a vehicle information acquisition device 106, a position information acquisition device 107, and a gaze detection sensor 108.

[0013] The processor 101 controls the entire vehicle 100. For example, the processor 101 is a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), an SoC (System On Chip), a GPGPU (General-Purpose computing on Graphics Processing Units), an ASIC (Application Specific Integrated Circuit), etc. The processor 101 may be composed of multiple processors. The vehicle 100 also has a processing circuit.

[0014] The volatile storage device 102 is a main storage device of the vehicle 100. For example, the volatile storage device 102 is a RAM (Random Access Memory). The non-volatile storage device 103 is an auxiliary storage device of the vehicle 100. For example, the non-volatile storage device 103 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0015] The actuator 104 is a driving device, a braking device, a steering device, or the like. The vehicle exterior information acquisition device 105 acquires information about the outside of the vehicle 100. For example, the vehicle exterior information acquisition device 105 is a millimeter wave radar, a LiDAR, an ultrasonic sonar, a camera, or the like. The vehicle information acquisition device 106 acquires information such as the steering angle, speed, attitude, and torque of the vehicle 100 . The position information acquisition device 107 acquires position information of the vehicle 100. For example, the position information acquisition device 107 is a Global Navigation Satellite System (GNSS) receiver. The gaze detection sensor 108 detects the driver's gaze.

[0016] Next, the functions of the vehicle 100 will be described. 3 is a block diagram showing the functions of the vehicle according to embodiment 1. Vehicle 100 includes memory unit 111, information processing unit 112, route generation unit 113, calculation unit 114, acquisition unit 115, control unit 116, analysis unit 117, detection unit 118, determination unit 119, decision unit 120, and provision unit 121.

[0017] The storage unit 111 may be realized as a storage area secured in the volatile storage device 102 or the nonvolatile storage device 103 . A part or all of the information processing unit 112, the path generating unit 113, the calculation unit 114, the acquisition unit 115, the control unit 116, the analysis unit 117, the detection unit 118, the determination unit 119, the decision unit 120, and the provision unit 121 may be realized by a processing circuit. Also, a part or all of the information processing unit 112, the path generating unit 113, the calculation unit 114, the acquisition unit 115, the control unit 116, the analysis unit 117, the detection unit 118, the determination unit 119, the decision unit 120, and the provision unit 121 may be realized as a module of a program executed by the processor 101.

[0018] The storage unit 111 stores various information.

[0019] The information processing unit 112 detects objects (e.g., other vehicles, pedestrians), road surfaces, etc. present around the vehicle 100 based on the information obtained from the vehicle exterior information acquisition device 105. The information processing unit 112 also detects background such as mountains, oceans, rivers, vegetation, and buildings based on the information obtained from the vehicle exterior information acquisition device 105.

[0020] The information processing unit 112 removes noise contained in the information obtained from the vehicle exterior information acquisition device 105. Based on the information obtained by the removal, the information processing unit 112 generates a relative coordinate system vehicle surroundings map with the vehicle 100 as the origin. The relative coordinate system vehicle surroundings map is three-dimensional information. The information processing unit 112 converts the relative coordinate system vehicle surroundings map into an absolute coordinate system vehicle surroundings map using the position information of the vehicle 100, the odometry of the vehicle 100, a six-axis gyro sensor, the position information acquisition device 107, etc. The absolute coordinate system vehicle surroundings map is a map in an absolute coordinate system on Earth.

[0021] The information processing unit 112 receives the n-1 The absolute coordinate system vehicle surrounding map and the current time t n The information processing unit 112 generates an absolute coordinate system vehicle surroundings map and a vehicle surroundings map in the absolute coordinate system. By comparing the two absolute coordinate system vehicle surroundings maps, the information processing unit 112 can distinguish between stationary objects and moving objects. The information processing unit 112 classifies moving objects based on size, speed, etc. The information processing unit 112 also further subdivides moving objects into categories such as vehicles, people, etc.

[0022] The information processing unit 112 detects the time t n+1 For example, if the moving object is a vehicle, the information processing unit 112 predicts the position of the moving object at the current time t n Furthermore, for example, if the moving object is a person, the information processing unit 112 predicts the position of the moving object moving straight ahead from the current time t n The range of movement possible from the position (i.e., the entire range) is predicted as the future position.

[0023] The route generation unit 113 generates a short-term route using information obtained by processing by the information processing unit 112 and the current position of the vehicle 100. Specifically, the route generation unit 113 generates the short-term route by taking into consideration factors such as road width, road shape, energy efficiency, and collision avoidance with obstacles. Note that the long-term route is generated by the driver inputting the destination into the navigation system before departure. The route generation unit 113 may generate the short-term route using a risk potential method. Specifically, the route generation unit 113 can generate the short-term route by creating a risk potential map based on risk potential energy and searching for a route that passes through flat portions in the risk potential map. Note that the risk potential method is described, for example, in Non-Patent Document 1. Note that the route generation unit 113 cannot generate a route if it cannot search for a route that passes through flat portions.

[0024] Here, risk potential energy will be explained. The calculation unit 114 uses information obtained by processing by the information processing unit 112 (for example, an absolute coordinate system vehicle surroundings map, stationary objects, moving objects, etc.) to calculate the collision probability of a collision due to the shape of the road on which the vehicle 100 is traveling as risk potential energy expressed in terms of energy. The risk potential energy may be expressed as potential energy expressed in terms of energy for the condition of the road on which the mobile body is traveling and obstacles. The calculation process will be explained using a specific example.

[0025] FIG. 4 is a diagram showing an example of calculation of risk potential energy in the first embodiment. FIG. 4 shows car 200 parked on the left side of a one-way street relative to car 100. FIG. 4 shows car 100 in a lane on the right side of the road. The right direction in FIG. 4 is the x-axis direction. The upward direction in FIG. 4 is the y-axis direction. In this situation, the road boundary repulsive potential function U w (x, y) is expressed using equation (1). wc is the y coordinate of the center of gravity of car 100. w is the weighting coefficient. w denotes the variance.

[0026]

number

[0027] Obstacle repulsion potential function U o (x, y) is expressed using equation (2). The x coordinate of the rear of the car 200 is x or The x coordinate of the front of car 200 is x of y o is the y coordinate of the center of gravity of car 200. o is the weighting coefficient. o denotes the variance.

[0028]

number

[0029] The calculation unit 114 calculates the road boundary repulsive potential function U w (x,y) and the obstacle repulsion potential function U o Using (x, y), the risk potential energy U a Specifically, the calculation unit 114 calculates the risk potential energy U a Calculate (x,y).

[0030]

number

[0031] The calculation unit 114 calculates the risk potential energy U a The risk potential energy of all (x, y) coordinates is calculated from (x, y). In this way, the risk potential energy is calculated. The calculation unit 114 generates a risk potential map based on the risk potential energy. The acquisition unit 115 acquires the risk potential map.

[0032] The control unit 116 determines the control content, which indicates the control amount of the actuator 104 and the control timing of the actuator 104, for the car 100 to travel by autonomous driving along the route generated by the route generation unit 113. For example, the control unit 116 determines the steering angle of the steering wheel at time t. Also, for example, the control unit 116 determines the travel speed at time t. In this way, the car 100 performs various processes while it is driving autonomously.

[0033] Furthermore, the control unit 116 determines that control is not possible if the control content will result in a permanent stop or a collision, or if the vehicle's attitude becomes uncontrollable, etc. If control is not possible, the control unit 116 sends a request to switch from autonomous driving to semi-autonomous driving or manual driving to the acquisition unit 115.

[0034] Next, the details regarding switching from automatic driving to semi-automatic driving or manual driving will be described. The acquisition unit 115 acquires a request to switch to manual driving or semi-automated driving. For example, if the route generation unit 113 cannot generate an automated driving route (i.e., a short-term route) based on the risk potential map, the determination unit 119 transmits the switch request to the acquisition unit 115. The acquisition unit 115 then acquires the switch request. The acquisition unit 115 may also acquire the switch request from a device not shown.

[0035] The acquisition unit 115 acquires gaze information indicating the driver's gaze. For example, the acquisition unit 115 acquires the gaze information from the gaze detection sensor 108. Alternatively, the gaze information may be acquired as follows: The acquisition unit 115 acquires an image including the driver's face from a visible light camera, an infrared camera, or the like. The analysis unit 117 uses the image to extract the driver's facial direction, eye position, pupil position, and the like. The analysis unit 117 estimates the gaze using the extracted information. The acquisition unit 115 acquires the estimated gaze information.

[0036] Furthermore, the gaze information also includes information acquired as follows: Using the analyzed gaze information and an outside-vehicle image captured by a camera in the vehicle's traveling direction, the analysis unit 117 analyzes and extracts a gaze point on the outside-vehicle image at which the driver's gaze is directed and a field-of-view image centered on the gaze point.

[0037] The calculation unit 114 calculates speed aftereffect energy, which indicates the energy of the speed aftereffect, using the acquired line-of-sight information. A method for calculating the speed aftereffect energy will be specifically described below.

[0038] First, the image intensity of the retinal image represented by coordinates (p, q) on the retina seen by the eye at time t is defined as I(p, q, t). p, q, and t are variables and represent a still image. It can be expressed that multiple still images are obtained by changing t. Multiple still images are also called moving images. Image intensity is the stimulation intensity on the retina, and may be represented by a luminance image. The origin of the retinal image represented by (p, q) on the retina is the gaze point coordinate of the gaze information. The retinal image and the image in the physical real world are almost similar to each other. Next, the calculation unit 114 calculates the even function f of the spatial impulse response using equations (4) and (5). odd (p,q), odd function f even Calculate (p, q). Note that σ indicates the standard deviation, λ indicates the wavelength, and φ indicates the phase angle.

[0039]

number

[0040]

number

[0041] Next, the calculation unit 114 calculates the time impulse response h using equations (6) and (7). slow (t),f fast (t) is calculated. Note that k indicates the response time. slow denotes the inferred slow time filter. fast denotes the inferred fast temporal filter, and β denotes the biphasic beta parameter following the beta distribution.

[0042]

number

[0043]

number

[0044] The calculation unit 114 calculates the two-phase response Res using equations (8) to (11). se (p,q,t),Res fe (p,q,t),Res so (p,q,t),Res fo Calculate (p,q,t).

[0045]

number

[0046]

number

[0047]

number

[0048]

number

[0049] Calculation unit 114 calculates the directivity using equations (12) to (15). Note that L indicates the left direction, and R indicates the right direction. Furthermore, odd spatial responses indicate negative directions, and even spatial responses indicate positive directions.

[0050]

number

[0051]

number

[0052]

number

[0053]

number

[0054] The calculation unit 114 calculates the directional spatiotemporal response Res using equations (16) to (19). L_1 ,Res L_2 ,Res R_1 ,Res R_2 Calculate.

[0055]

number

[0056]

number

[0057]

number

[0058]

number

[0059] The calculation unit 114 calculates the left directional energy E L and the directed energy E on the right R Specifically, the calculation unit 114 calculates the directional energy E L and directed energy E R and normalize it. e is the sum of the four directional spatiotemporal responses for normalization.

[0060]

number

[0061]

number

[0062] The calculation unit 114 calculates the directional energy E L and directed energy E R Specifically, the calculation unit 114 calculates ME using equation (22).

[0063]

number

[0064] If ME is not zero, it means that the velocity aftereffect is occurring. Note that equation (22) is an equation for calculating the velocity aftereffect energy in a certain direction. In this way, the calculation unit 114 calculates the speed residual energy using a plurality of equations.

[0065] In addition, when speed residual effect energy exists, the calculation unit 114 calculates the speed residual effect duration until the value of the speed residual effect energy becomes zero using equation (22), and stores the speed residual effect energy and the speed residual effect duration in the memory unit 111.

[0066] When the determination unit 119 determines that switching from autonomous driving to semi-autonomous driving or manual driving is necessary, the decision unit 120 receives a switching request via the memory unit 111. The decision unit 120 acquires the speed aftereffect energy ME and the speed aftereffect duration from the memory unit 111. When the speed aftereffect energy ME is not zero, the decision unit 120 determines whether the speed aftereffect duration is zero. When the speed aftereffect duration is not zero, the decision unit 120 determines to switch to semi-autonomous driving. When the speed aftereffect duration is zero, the decision unit 120 determines to switch to manual driving and notifies the route generation unit 113 and the control unit 116 that switching to manual driving has been decided. In the case of semi-autonomous driving, as soon as control of the actuator 104 at time t is completed, the decision unit 120 again determines whether the speed aftereffect duration is not zero. The decision and control are then repeated until the speed aftereffect duration becomes zero.

[0067] The determination unit 119 uses the speed residual effect duration and risk potential energy to determine whether to switch the driving system from automated driving to manual driving or semi-automated driving. Semi-automated driving is a control mode that is based on manual driving but also includes some automated driving functions, such as automatic acceleration / deceleration control to mitigate collisions, automatic steering control to avoid collisions, and automatic vehicle attitude control to prevent the vehicle body from sliding left and right. The determination process will be explained in detail.

[0068] The decision unit 120 receives the speed aftereffect energy value and speed aftereffect duration from the calculation unit 114. If the speed aftereffect energy value is not zero, the decision unit 120 determines that there is an aftereffect influence. If the speed aftereffect duration is not zero, the decision unit 120 switches to semi-automated driving and starts a control mode in which the driver performs partial automatic driving and driving assistance instead of manual driving. When the semi-automated driving control at time t is completed, the processing time is subtracted from the speed aftereffect duration, and a loop process for performing semi-automated driving control again is performed, and this is repeated until the speed aftereffect duration becomes zero or less. When the speed aftereffect duration becomes zero or less, it is determined that the influence of the driver's speed aftereffect has disappeared, and a decision is made to switch to manual driving.

[0069] When it is determined that the driving system will switch to semi-automatic driving or manual driving, the control unit 116 performs control to perform semi-automatic driving or manual driving.

[0070] After it is determined that the driving system will switch to semi-autonomous driving or manual driving, the providing unit 121 provides the driver with information indicating that the driving system will switch to semi-autonomous driving or manual driving. The information may be provided as an image or video. For example, the providing unit 121 provides an image or video to a display of the vehicle 100. The information may also be provided as audio information. For example, the providing unit 121 provides audio information to a speaker of the vehicle 100. By receiving the information, the driver can recognize that the driving system will switch to semi-autonomous driving or manual driving. The information may also include information about obstacles present around the vehicle 100. For example, the providing unit 121 provides a surroundings map on a display of the vehicle 100, which provides an overview of the obstacle's location and type (e.g., car, person, bicycle, wall, etc.) displayed graphically, as well as the predicted arrival time at the current speed and direction of the vehicle 100, with the vehicle as the origin. The information may also be provided as audio information. For example, the providing unit 121 provides audio information to a speaker of the car 100, with the car 100 as the origin, including the direction and distance of an obstacle, its type (e.g., automobile, person, bicycle, wall, etc.), the current speed of the car 100, and the predicted arrival time in that direction.

[0071] Furthermore, when the driving system is switched to semi-automated driving or manual driving, the control unit 116 controls the speed or acceleration so that it does not exceed a predetermined value even when the driver steps on the accelerator. For example, when the vehicle's posture bulges outward due to centrifugal force generated by the steering angle, driving speed, and acceleration, causing the vehicle's behavior to become uncontrollable, such as spinning or tipping over, the control unit 116 controls the speed or acceleration to be below the predetermined value. Here, when speed aftereffect occurs, the driver perceives the movement speed of the surrounding scenery as slow. Therefore, if the driver steps on the accelerator too hard, the car 100 moves at an abnormally high speed. To prevent such a situation, the control unit 116 performs control. Note that when the speed aftereffect duration reaches zero, the control unit 116 sends a request to switch from semi-automated driving to manual driving to the decision unit 120.

[0072] When the decision unit 120 decides that the driving system will be switched to semi-automated driving, the control unit 116 performs control to perform semi-automated driving.

[0073] After the decision unit 120 decides that the driving system will switch to semi-automated driving, the providing unit 121 provides the driver with information indicating that the driving system will switch to semi-automated driving.

[0074] Furthermore, when the driving system is switched to semi-automated driving, the control unit 116 performs control so that the speed does not exceed a predetermined speed even if the driver steps on the accelerator.

[0075] Next, the process executed by the vehicle 100 will be described with reference to a flowchart. 5 is a flowchart (part 1) showing an example of processing executed by the vehicle according to embodiment 1. It is assumed that the vehicle 100 is performing autonomous driving. (Step S11) The information processing unit 112 acquires from the outside-vehicle information acquisition device 105 position information of objects such as road surfaces, obstacles, and buildings around the vehicle, as well as other background information. (Step S12) The calculation unit 114 calculates the risk potential energy for each object from the position information of the object around the host vehicle from the information processing unit 112. (Step S13) The calculation unit 114 generates a risk potential map from the risk potential energy of each object. (Step S14) The route generation unit 113 acquires the risk potential map generated by the calculation unit 114 and generates a route toward the destination along the flattest part of the map. (Step S15) The determination unit 119 determines whether or not the route generated by the route generation unit 113 has been determined. If the route has been determined, the process proceeds to step S16, and if the route has not been determined, the process proceeds to step S31. (Step S16) The acquisition unit 115 acquires vehicle information such as the vehicle speed, acceleration in six axial directions, and attitude of the vehicle from the vehicle information acquisition device 106. (Step S17) The calculation unit 114 uses the route information from the determination unit 119 and the vehicle information from the acquisition unit 115 to calculate the control amount and control timing of each actuator such as drive, braking, steering, etc. (Step S18) The control unit 116 acquires the control amount and control timing of each actuator from the calculation unit 114, and determines whether these values ​​are valid (whether they have been calculated). If they are valid, the process proceeds to step S19, and if they are invalid, the process proceeds to step S31. (Step S19) The control unit 116 acquires the control amount and control timing of each actuator from the calculation unit 114, and determines whether the difference between the control amount and timing at the time t-1 determined immediately before the current time t is equal to or less than a predetermined threshold. If the difference is equal to or less than the threshold, the process proceeds to step S110, and if the difference is equal to or greater than the threshold, the process proceeds to step S31. (Step S110) The control unit 116 sets the control amount for each actuator when the time t reaches the control timing.

[0076] FIG. 6 is a flowchart (part 2) illustrating an example of processing executed by the vehicle according to the first embodiment. (Step S21) The acquisition unit 115 acquires images of the vehicle traveling direction from one or more cameras related to the vehicle traveling direction detected by the detection unit 118 from the outside vehicle information acquisition device 105. (Step S22) The acquisition unit 115 acquires the driver's line of sight information detected by the detection unit 118 from the line of sight detection sensor . (Step S23) The calculation unit 114 receives the vehicle traveling direction image and the line of sight information from the acquisition unit 115, and calculates the gaze point and field of view image on the vehicle traveling direction image. (Step S24) The calculation unit 114 receives the gaze point and the visual field image from the analysis unit 117, and analyzes and calculates the velocity aftereffect energy and the velocity aftereffect duration. (Step S25) The acquisition unit 115 acquires the speed aftereffect energy and the speed aftereffect duration, and stores them in the storage unit 111.

[0077] FIG. 7 is a flowchart (part 3) illustrating an example of processing executed by the vehicle according to the first embodiment. (Step S31) The determination unit 120 extracts the speed residual energy calculated by the calculation unit 114 from the storage unit 111. (Step S32) The determination unit 120 determines whether the speed residual energy is zero or not. If it is not zero, the process proceeds to step S33, and if it is zero, the process proceeds to step S36. (Step S33) The decision unit 120 decides to transition to the semi-automated driving mode, and sends an instruction to the control unit 116, the route generation unit 113, and the information processing unit 112 to transition. (Step S34) The determination unit 120 subtracts the speed aftereffect duration from the current time. (Step S35) The determination unit 120 determines whether the velocity aftereffect duration, which is the analysis result of the calculation unit 114, has become zero. If it has become zero, the process proceeds to step S36, and if it has not become zero, the process proceeds to step S34. (Step S36) The decision unit 120 decides to transition to the manual driving mode, and sends an instruction to the control unit 116, the route generation unit 113, and the information processing unit 112 to transition.

[0078] It is known that the driving system can switch from automatic to manual driving. If the driving system switches from automatic to manual driving without understanding the driver's state, there is a high possibility that the driver will not be able to drive the car correctly.

[0079] According to the first embodiment, the car 100 determines whether the switching level to be used when switching is semi-automated driving or manual driving, taking into account the speed aftereffect. In other words, the car 100 determines the switching level when switching, taking into account the driver's state. Therefore, the car 100 can grasp the driver's state and provide driving assistance that matches the state.

[0080] Embodiment 2 Next, a description will be given of embodiment 2. In embodiment 2, differences from embodiment 1 will be mainly described. Furthermore, in embodiment 2, description of matters common to embodiment 1 will be omitted.

[0081] In the first embodiment, a case where the speed residual energy is calculated using an equation has been described. In the second embodiment, a case where the speed residual energy is calculated using a different method will be described.

[0082] The acquisition unit 115 acquires the trained model from the storage unit 111. The calculation unit 114 calculates the speed residual energy using the multiple images, the gaze point coordinates, and the trained model. In detail, when the calculation unit 114 inputs into the trained model a field of view image centered on the gaze point obtained based on the multiple images in the vehicle traveling direction and the gaze point coordinates, the speed of the vehicle 100, the six-axis acceleration of the vehicle 100, and the duration during which the images were taken to obtain the multiple images, the trained model outputs the speed residual energy. In this way, the vehicle 100 may use the trained model to calculate the speed residual energy.

[0083] Furthermore, the calculation unit 114 may calculate the speed residual energy using some of the equations shown in embodiment 1 and the trained model. For example, the calculation unit 114 executes equations (4) to (11), and then calculates the speed residual energy using the calculation results and the trained model.

[0084] The features of each of the above-described embodiments can be combined with each other as appropriate. In each embodiment, the case where the vehicle 100 is executed has been described. Each of the embodiments may be realized by a moving body other than a vehicle. For example, the moving body other than a vehicle is a railroad vehicle. [Explanation of symbols]

[0085] 100 Vehicle, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 104 Actuator, 105 Outside vehicle information acquisition device, 106 Vehicle information acquisition device, 107 Position information acquisition device, 108 Gaze detection sensor, 111 Memory unit, 112 Information processing unit, 113 Route generation unit, 114 Calculation unit, 115 Acquisition unit, 116 Control unit, 117 Analysis unit, 118 Detection unit, 119 Judgment unit, 120 Decision unit, 121 Provision unit, 200 Vehicle.

Claims

1. A vehicle with a driver on board and operating automatically, an acquisition unit that acquires line-of-sight information indicating the line of sight of the driver, a plurality of images obtained by photographing an area ahead of the driver, a request to switch driving modes, and the status of a road on which the moving object is traveling and obstacles as risk potential energy expressed in energy; a detection unit that detects gaze point coordinates that are coordinates corresponding to the gaze point of the driver based on one image of the plurality of images and the line of sight information; a calculation unit that calculates a velocity aftereffect energy and a velocity aftereffect duration, which indicate the energy of the velocity aftereffect, using the plurality of images and the gaze point coordinates; a decision unit that, when the speed residual effect energy exists, determines whether to have the driver immediately perform manual driving using the risk potential energy and the speed residual effect energy, or to have the driver perform manual driving in a semi-automated driving mode for the speed residual effect duration, and then to have the driver perform manual driving; A mobile object having the above configuration.

2. Further comprising a control unit corresponding to the manual operation, semi-automatic operation, and automatic operation, The moving body according to claim 1 .

3. When the driving system is switched to semi-automated driving or manual driving, the control unit further comprises: a control unit that suppresses the amount of driving, steering, or braking operation by the driver to a level that does not cause the tires of the vehicle body to slip and become uncontrollable; 3. A moving body according to claim 1 or 2.

4. The driving system has a route determination unit and a vehicle control unit during autonomous driving, and requests driving mode switching when the route, vehicle control amount, or vehicle control timing cannot be determined.

3. A moving body according to claim 1 or 2.

5. Further comprising a storage unit, The acquisition unit acquires a trained model from the storage unit, The calculation unit calculates the speed residual energy using the plurality of images, the gaze point coordinates, and the trained model.

3. A moving body according to claim 1 or 2.

6. the calculation unit calculates an even function of a spatial impulse response, an odd function of a spatial impulse response, and a time impulse response using the plurality of images and the gaze point coordinates, calculates a biphasic response based on the even function of the spatial impulse response, the odd function of the spatial impulse response, and the time impulse response, and calculates the speed residual energy using the biphasic response and a trained model. The moving body according to claim 4.

Images