Control device and control method

Abstract

To reduce a processing load and a processing time accompanying determination processing when determining safety of an occupant.SOLUTION: The control device of the vehicle 100 for transferring the occupant includes a position movement detection part 332 for detecting the position movement amount of the occupant in the vehicle, a skeleton movement detection part 333 for detecting the movement amount of the skeleton of each occupant, a state detection part 334 for detecting the state of each occupant, and a start propriety determination part 336 for determining the propriety of the start of the stopped vehicle. When it is determined that all of a first condition that a position movement amount of the occupant is less than a first threshold value, a second condition that a movement amount of a skeleton of the occupant is less than a second threshold value, and a third condition that a state of the occupant is a state accompanied by a stable posture are satisfied, the departure permission determination unit permits departure of the vehicle, determines whether or not the conditions are satisfied in order of the first condition, the second condition, and the third condition, omits remaining determination processing at a time point at which an occupant who does not satisfy any of the conditions is detected, and prohibits departure of the vehicle.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a control device or a control method. [Background technology]

[0002] Conventionally, there has been known a vehicle safety device that suspends the start of a stopped vehicle in order to ensure the safety of the vehicle occupants when it is determined that the vehicle occupants are not riding in a stable posture (Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-36795 Summary of the Invention [Problem to be solved by the invention]

[0004] When a vehicle has many occupants, if it is attempted to determine whether all occupants are always riding in a stable posture, the processing load for determining the posture of the occupants increases and the processing time becomes long.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to reduce the processing load and processing time involved in the determination process for determining the safety of an occupant. [Means for solving the problem]

[0006] The gist of the present disclosure is as follows.

[0007] (1) A control device for a vehicle that transports an occupant, a position movement detection unit that detects a position movement amount of an occupant in the vehicle; a skeleton movement detection unit that detects the amount of movement of the skeleton of each occupant; a state detection unit that detects the state of each occupant; a start possibility determination unit that determines whether or not the stopped vehicle can start, The departure possibility determination unit permitting the vehicle to start when it is determined that all of the following conditions are satisfied: a first condition that the amount of positional movement of the occupant is less than a first threshold value, a second condition that the amount of movement of the occupant's skeleton is less than a second threshold value, and a third condition that the occupant's state is a state accompanied by a stable posture; A control device that determines whether the first condition, the second condition, and the third condition are met in that order, and when an occupant who does not meet any of the conditions is detected, skips the remaining determination processes and prohibits the vehicle from starting. (2) The control device described in (1) above, in which the departure possibility determination unit, when all occupants satisfy the first condition, determines whether there are any occupants who do not satisfy the second condition, in order of occupants with the largest amount of positional movement. (3) The control device described in (1) or (2) above, wherein when all occupants satisfy the second condition, the departure possibility determination unit determines whether there are any occupants who do not satisfy the third condition, starting with the occupant with the greatest amount of skeletal movement. (4) A grip determination unit is further provided to determine whether the occupant detected as being in a standing state by the state detection unit is gripping a body support facility around the occupant, The departure possibility determination unit permitting the vehicle to start when it is determined that a fourth condition, in which the occupant detected to be in a standing state is holding a body support device, is satisfied in addition to the first condition to the third condition; A control device described in any one of (1) to (3) above, which determines whether the conditions are met in the order of the first condition, the second condition, the third condition, and the fourth condition, and when an occupant who does not meet any of the conditions is detected, skips the remaining determination processes and prohibits the vehicle from starting. (5) A method for controlling a vehicle for transporting an occupant, comprising: Detecting a positional movement amount of an occupant in the vehicle; Detecting the amount of movement of each occupant's skeleton; Detecting the state of each occupant; permitting the vehicle to start from a stop when it is determined that all of the following conditions are satisfied: a first condition that the amount of positional movement of the occupant is less than a first threshold, a second condition that the amount of movement of the occupant's skeleton is less than a second threshold, and a third condition that the state of the occupant is a state accompanied by a stable posture; determining whether the first condition, the second condition, and the third condition are satisfied in that order, and, when an occupant who does not satisfy any of the conditions is detected, skipping the remaining determination processes and prohibiting the vehicle from starting from a stopped state. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to reduce the processing load and processing time involved in the determination process when determining the safety of an occupant. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating a vehicle control system. [Figure 2] FIG. 2 is a functional block diagram of the processor of the ECU. [Figure 3] FIG. 3 is a diagram showing an image captured by an in-vehicle camera. [Figure 4] FIG. 4 is a diagram showing the movement of the bounding box. [Figure 5] FIG. 5 shows an image of one occupant. [Figure 6] FIG. 6 is a flowchart showing the flow of the vehicle control process. [Figure 7] FIG. 7 is a flowchart showing the flow of the vehicle control process. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the following description, like components are designated by like reference numerals.

[0011] <Vehicle control system configuration> A vehicle control system 1 in which a vehicle control device is implemented will be described with reference to Fig. 1. Fig. 1 is a schematic configuration diagram of the vehicle control system 1 in which a vehicle control device according to one embodiment is implemented. The vehicle control system 1 is mounted on a vehicle 100, and determines whether or not the vehicle 100, which is stopped, can start moving, depending on the status of the occupants of the vehicle 100.

[0012] Furthermore, vehicle 100 is a vehicle that can accommodate passengers other than a driver and transports such passengers. In particular, in this embodiment, vehicle 100 is a vehicle that can travel without any passengers seated. Vehicle 100 is, for example, a bus. However, vehicle 100 may also be a vehicle that basically travels with passengers seated, such as a passenger car.

[0013] In this embodiment, as shown in FIG. 1, the vehicle control system 1 has an in-vehicle camera 11, a grip sensor 12, a driver operation sensor 13, a human-machine interface (hereinafter referred to as "HMI") 14, a vehicle actuator 20, and an electronic control unit (hereinafter referred to as "ECU") 30.

[0014] The in-vehicle camera 11, the grip sensor 12, the driver operation sensor 13, the HMI 14, and the ECU 30 are communicatively connected via an in-vehicle network 5. The in-vehicle network 5 is a network that complies with standards such as the Controller Area Network (CAN). The ECU 30 is also connected to the vehicle actuator 20 via a signal line. The ECU 30 may be connected to the vehicle actuator 20 via the in-vehicle network 5, or may be connected to the in-vehicle camera 11, etc., via a signal line.

[0015] The in-vehicle camera 11 is a sensor that captures images of the interior of the vehicle 100. In particular, in this embodiment, the in-vehicle camera 11 is installed so as to be able to capture images of all occupants in the vehicle 100. In this embodiment, the vehicle 100 is provided with a plurality of in-vehicle cameras 11. For example, two in-vehicle cameras 11 are provided at the front, center, and rear of the vehicle 100. The in-vehicle cameras 11 are, for example, CMOS cameras or CCD cameras that are sensitive to visible light. The in-vehicle cameras 11 capture images of the interior of the vehicle 100 at predetermined imaging intervals and generate image data showing the interior of the vehicle 100. The in-vehicle cameras 11 output the generated image data to the ECU 30 via the in-vehicle network 5 at predetermined intervals.

[0016] The grip sensor 12 is a sensor that generates data indicating whether or not an occupant is gripping a given location within the vehicle 100. In this embodiment, the grip sensor 12 generates data indicating whether or not an occupant is gripping a body support facility that a standing occupant grips to support their body. The body support facility includes, for example, a strap or a handrail (handrail). Specifically, the grip sensor 12 is a pressure sensor provided at a given location, and outputs pressure when the occupant grips a given location and a predetermined pressure or more is applied to that location. Note that any sensor other than a pressure sensor may be used as the grip sensor 12 as long as it can generate data indicating whether or not an occupant is gripping a given location within the vehicle 100.

[0017] The driver operation sensor 13 is a sensor that detects the operation status of the vehicle 100's operating devices (e.g., brake pedal, accelerator pedal, and steering wheel) by the driver and generates operation data that represents the operation status of the operating devices. The driver operation sensor 13 includes, for example, a brake sensor that detects the amount of depression of the brake pedal, an accelerator sensor that detects the amount of depression of the accelerator pedal, and a steering sensor that detects the operation of the steering wheel. The driver operation sensor 13 outputs the operation data to the ECU 30 via the in-vehicle network 5 at predetermined intervals.

[0018] The HMI 14 is a user interface for exchanging information between the ECU 30 of the vehicle 100 and an occupant of the vehicle 100. The HMI 14 has an input device 15 that receives input from the occupant of the vehicle 100 and an output device 16 that notifies the occupant of the vehicle 100. The input device 15 is a device that receives inputs such as physical operations or audio operations by the occupant, and includes at least one of a touch panel, a switch, a button, a microphone, etc. On the other hand, the output device 16 is a device that notifies the occupant through the occupant's five senses (e.g., sight, hearing, touch, etc.), and includes at least one of a display device (e.g., a liquid crystal display, a head-up display, a warning light, etc.), a speaker, a vibration unit, etc.

[0019] The HMI 14 transmits input data received from the occupant via the input device 15 to the ECU 30 via the in-vehicle network 5. The HMI 14 also notifies the occupant via the output device 16 of information corresponding to a signal received from the ECU 30 via the in-vehicle network 5.

[0020] The vehicle actuators 20 are actuators used to control the operation of the vehicle 100. Specifically, the vehicle actuators 20 include, for example, a drive actuator that controls a prime mover (an internal combustion engine or an electric motor) for driving the vehicle 100, a braking actuator that controls a brake that brakes the vehicle 100, and a steering actuator that controls the steering of the vehicle 100. The vehicle actuators 20 control the acceleration, braking, and steering of the vehicle 100 in accordance with control signals transmitted from the ECU 30 via signal lines.

[0021] <Vehicle control device overview> The ECU 30 functions as a vehicle control device that controls the vehicle 100. In this embodiment, the ECU 30 determines whether or not the stopped vehicle 100 can start, depending on the status of the occupants of the vehicle 100, and notifies the driver based on the determination result. In the example shown in FIG. 1, the vehicle control system 1 includes one ECU 30, but may also include multiple ECUs 30 that are separated by function. The ECU 30 includes a communication interface 31, a storage unit 32, and a processor 33. The communication interface 31, the storage unit 32, and the processor 33 may be separate circuits, or may be configured as a single integrated circuit.

[0022] The communication interface 31 is a circuit for connecting the ECU 30 to the in-vehicle network 5 .

[0023] The storage unit 32 is a non-transitory storage medium that stores data. The storage unit 32 includes, for example, at least one of a volatile semiconductor memory, a non-volatile semiconductor memory, a hard disk drive (HDD), and a solid state drive (SSD). The storage unit 32 stores computer programs executed by the processor 33 of the ECU 30. The storage unit 32 also stores data used in the computer programs executed by the processor 33, such as data transmitted from sensors such as the in-vehicle camera 11, the grip sensor 12, and the driver operation sensor 13.

[0024] The processor 33 has one or more central processing units (CPUs) and their peripheral circuits. The processor 33 may further have other arithmetic circuits such as a logic operation unit or a numerical operation unit. The processor 33 executes a computer program stored in the storage unit 32.

[0025] 2 is a functional block diagram of the processor 33 of the ECU 30. As shown in FIG. 2, the processor 33 includes an occupant detection unit 331, a position movement detection unit 332, a skeleton movement detection unit 333, a state detection unit 334, a grip determination unit 335, a start possibility determination unit 336, and a notification / control unit 337. Each of these units included in the processor 33 is a functional module realized by, for example, a computer program running on the processor 33. Alternatively, each unit included in the processor 33 may be implemented in the ECU 30 as an independent integrated circuit, microprocessor, or firmware.

[0026] The occupant detection unit 331 detects occupants of the vehicle 100 based on image data transmitted from the in-vehicle camera 11. In this embodiment, the occupant detection unit 331 outputs an area in which the occupant is located within the image represented by the image data transmitted from the in-vehicle camera 11. For example, in this embodiment, the occupant detection unit 331 outputs the position of a bounding box that surrounds the area in which the occupant is represented within the image represented by the image data. Such a bounding box will be described with reference to FIG. 3.

[0027] Fig. 3 is a diagram showing an image I captured by one in-vehicle camera 11. In the example shown in Fig. 3, four occupants P1 to P4 are captured in the image captured by this in-vehicle camera 11. In the example shown in Fig. 3, bounding boxes B1 to B4 surrounding each of the occupants P1 to P4 are also shown. Each of the bounding boxes B1 to B4 is displayed so as to surround the entire area in which the corresponding occupant P1 to P4 is displayed.

[0028] The occupant detection unit 331, for example, sequentially inputs data of images captured by the in-vehicle camera 11 to a classifier, and outputs the area in each image in which an occupant is represented. For example, when image data is input, the classifier outputs the coordinates of the center of gravity of a bounding box surrounding each occupant in the image represented by the image data and its vertical size. Alternatively, the classifier may output the coordinates of the upper left corner and the lower right corner of the bounding box. The classifier is, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side.

[0029] The position movement detection unit 332 detects the amount of movement per unit time of the position of the occupant in the vehicle 100. For example, the position movement detection unit 332 detects the amount of movement of the position of the occupant detected by the occupant detection unit 331.

[0030] As described above, in this embodiment, the occupant detection unit 331 outputs the position of a bounding box that surrounds the area in the image in which the occupant is depicted. The position of this bounding box changes when the position of the occupant changes. Therefore, the movement of the occupant can be detected based on the change in the position of the bounding box. Therefore, in this embodiment, the position movement detection unit 332 calculates the amount of movement of the position of the bounding box corresponding to each occupant as the amount of movement of that occupant.

[0031] In particular, in this embodiment, the position movement detection unit 332 calculates the amount of movement of the center of gravity of the bounding box corresponding to each occupant as the amount of movement of that occupant. FIG. 4 is a diagram showing the state of movement of the bounding box. In the example shown in FIG. 4, at time t-1, the coordinates of the center of gravity of the bounding box are (x t-1 , y t-1 ), and at time t, the coordinates of the center of gravity of the bounding box are (x t , y t Therefore, the position movement detection unit 332 calculates the amount of movement of the occupant surrounded by this bounding box as ((x t -xt-1 ) 2 +(y t -y t-1 ) 2 ) 1 / 2 It is calculated as follows.

[0032] In this embodiment, the position movement detection unit 332 calculates the amount of movement of the center of gravity of the bounding box corresponding to each occupant as the amount of movement of that occupant. However, the amount of movement of that occupant may also be calculated based on the amount of movement of other parameters, such as the amount of movement of the corner coordinates of the bounding box corresponding to that occupant.

[0033] The skeletal movement detection unit 333 detects the amount of movement of the skeleton of each occupant per unit time. The skeletal movement detection unit 333 detects the positions of multiple skeleton points (or joint points) of each occupant from the image of that occupant shown in the image captured by the in-vehicle camera 11. The skeletal movement detection unit 333 then detects the amount of movement of the skeleton of that occupant based on the amount of movement of the position of each skeleton point of that occupant detected in this way. The detection of the amount of movement of the skeleton of each occupant by the skeletal movement detection unit 333 imposes a higher calculation load on the processor 33 than the detection of movement of the position of the occupant by the position movement detection unit 332.

[0034] Fig. 5 is a diagram showing an image of one occupant. Fig. 5 shows, for example, an area surrounded by a bounding box B corresponding to one occupant in an image captured by the in-vehicle camera 11. When an image such as that shown in Fig. 5 is input, the skeleton movement detection unit 333 outputs the positions of skeleton points J of the occupant shown in this image. In the example shown in Fig. 5, 15 skeleton points J are output.

[0035] The skeleton movement detection unit 333, for example, sequentially inputs image data of the area surrounded by each bounding box B to a classifier, thereby outputting the positions of each of the multiple skeleton points J of the occupant represented by the image. When image data is input, for example, the classifier outputs the coordinates of each skeleton point J of the occupant represented by the image data. The classifier is, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side.

[0036] In addition, the skeleton movement detection unit 333 detects the movement of the position of the skeleton point J of each occupant detected in this manner as the amount of movement of the skeleton of that occupant. Specifically, the skeleton movement detection unit 333 detects the average or integrated value of the amount of movement of each skeleton point J of each occupant as the amount of movement of the skeleton of that occupant. Alternatively, the skeleton movement detection unit 333 may multiply each skeleton point J by a different weight, calculate the average or integrated value, and detect this value as the amount of movement of the skeleton of that occupant. In this case, for example, since even a slight movement of the occupant's arm will result in a large calculated amount of movement of the skeleton point J, the weight for the skeleton point of the arm may be small.

[0037] The state detection unit 334 detects the state of each occupant of the vehicle 100. The state detection unit 334 detects the state of each occupant from an image of the occupant shown in an image captured by the in-vehicle camera 11. The state detection unit 334 detects, for example, which of a plurality of preset states (e.g., seated state, standing state, walking state, fallen state, etc.) the state of each occupant corresponds to. The detection of the state of each occupant by the state detection unit 334 imposes a higher computational load on the processor 33 than the detection of the amount of movement of the skeleton of each occupant by the skeleton movement detection unit 333.

[0038] The state detection unit 334, for example, sequentially inputs image data of the area surrounded by each bounding box B to a classifier, and outputs the state of the occupant represented by the image. For example, when image data is input, the classifier calculates the reliability of each state of the occupant represented by the image data and outputs the state with the highest reliability as the current state of the occupant. For example, when image data is input to the classifier, if the reliability of the seated state is 50, the reliability of the standing state is 30, the reliability of the walking state is 15, and the reliability of the fallen state is 5, the classifier outputs the state of the occupant as the seated state.

[0039] The grip determination unit 335 determines whether an occupant detected by the state detection unit 334 to be in a standing state is gripping the surrounding body support equipment. In this embodiment, the determination is made based on the output of the grip sensor 12 to determine whether the occupant is gripping the surrounding body support equipment. For example, when an occupant is detected to be in a standing state by the state detection unit 334, the grip determination unit 335 makes a determination based on the output of the grip sensor 12 provided on the body support equipment located around the occupant. Note that the grip determination unit 335 may make a determination based on the output of any sensor as long as it can determine whether an occupant detected to be in a standing state is gripping the surrounding body support equipment. Therefore, the grip determination unit 335 may make a determination based on, for example, an image captured by the in-vehicle camera 11, using a classifier that outputs whether the occupant is gripping the surrounding body support equipment when an image of the occupant is input.

[0040] The start possibility determination unit 336 determines whether or not the stopped vehicle 100 is allowed to start. The start possibility determination unit 336 basically permits the vehicle 100 to start when the occupants of the vehicle 100 are riding in a stable state, and prohibits the vehicle 100 from starting when the occupants of the vehicle 100 are riding in an unstable state. In particular, in this embodiment, the start possibility determination unit 336 permits the vehicle 100 to start when all the occupants of the vehicle 100 are riding in a stable state, and prohibits the vehicle 100 from starting when some of the occupants of the vehicle 100 are riding in an unstable state.

[0041] In this embodiment, the start possibility determination unit 336 permits the start of the vehicle 100 when all of the following conditions are satisfied: a first condition related to the amount of movement of the occupant's position, a second condition related to the amount of movement of the occupant's skeleton, a third condition related to the occupant's state, and a fourth condition related to the occupant's grip on the body support equipment. On the other hand, the start possibility determination unit 336 prohibits the start of the vehicle 100 when any of the first to fourth conditions is not satisfied.

[0042] The first condition is satisfied when the amount of movement of the occupant's position detected by the position movement detection unit 332 is less than a predetermined first threshold, i.e., when the amount of movement of the occupant's position is small. In this embodiment, specifically, the first condition is satisfied when the amount of movement of the center of gravity of the bounding box that encloses the area in which each occupant is represented is equal to or greater than a predetermined first threshold.

[0043] The second condition is satisfied when the amount of movement of the occupant's skeleton detected by the skeleton movement detection unit 333 is less than a predetermined second threshold, i.e., when the occupant's movement is small. In this embodiment, specifically, the second condition is satisfied when the average or integrated value of the amount of movement of each skeleton point J of the occupant is equal to or greater than the predetermined second threshold.

[0044] The third condition is satisfied when the state of the occupant detected by the state detection unit 334 is a state accompanied by a stable posture. The third condition is satisfied, for example, when the state of the occupant is a seated state or a standing state. On the other hand, the third condition is not satisfied, for example, when the state of the occupant is a walking state or a fallen state.

[0045] The fourth condition is satisfied when the grip determination unit 335 determines that the occupant in a standing state is gripping the body support equipment. In this embodiment, the fourth condition is satisfied, for example, when the grip sensors 12 provided on the body support equipment around the occupant detected by the state detection unit 334 as being in a standing state detect that the occupant is gripping the body support equipment.

[0046] Furthermore, the start possibility determination unit 336 determines whether the conditions are satisfied in the order of the first condition, the second condition, the third condition, and the fourth condition. Therefore, the start possibility determination unit 336 first determines whether all occupants of the vehicle 100 satisfy the first condition. Then, if the start possibility determination unit 336 detects an occupant who does not satisfy the first condition, it prohibits the start of the vehicle 100.

[0047] If the start possibility determination unit 336 determines that all occupants of the vehicle 100 satisfy the first condition, it then determines whether all occupants of the vehicle 100 satisfy the second condition. At this time, the start possibility determination unit 336 determines whether the second condition is satisfied for occupants in descending order of the amount of movement of the occupant's position detected by the position movement detection unit 332. This makes it possible to determine whether the second condition is satisfied for occupants in descending order of the likelihood that the second condition is satisfied. If an occupant who does not satisfy the second condition is detected, the start possibility determination unit 336 prohibits the start of the vehicle 100.

[0048] If the start possibility determination unit 336 determines that all occupants of the vehicle 100 satisfy the second condition, it then determines whether all occupants of the vehicle 100 satisfy the third condition. At this time, the start possibility determination unit 336 determines whether the third condition is satisfied, starting with the occupant whose skeletal movement amount detected by the skeletal movement detection unit 333 is greatest. This makes it possible to determine whether the third condition is satisfied, starting with the occupant who is least likely to satisfy the third condition. If an occupant who does not satisfy the third condition is detected, the start possibility determination unit 336 prohibits the vehicle 100 from starting.

[0049] When the start possibility determination unit 336 determines that all occupants of the vehicle 100 satisfy the third condition, it determines whether or not the occupants detected by the state detection unit 334 to be standing satisfy the fourth condition. When the start possibility determination unit 336 detects an occupant who does not satisfy the fourth condition, it prohibits the start of the vehicle 100. On the other hand, when the start possibility determination unit 336 determines that all occupants detected by the state detection unit 334 to be standing satisfy the fourth condition, it permits the start of the vehicle 100.

[0050] As described above, the start permission determination unit 336 determines whether the conditions are satisfied in the order of the first condition, the second condition, the third condition, and the fourth condition. When an occupant who does not satisfy any of the conditions is detected, the remaining determination processes are skipped and the vehicle is prohibited from starting. Here, as described above, the calculation load on the processor 33 increases in the order of detection of occupant position movement by the position movement detection unit 332, detection of the amount of skeletal movement of each occupant by the skeletal movement detection unit 333, and detection of each occupant's state by the state detection unit 334. In other words, the calculation load on the processor 33 associated with determining whether the conditions are satisfied increases in the order of the first condition, the second condition, and the third condition. Therefore, according to this embodiment, when some occupants of the vehicle 100 are riding in an unstable state, such occupants can be detected early with a low calculation load.

[0051] In the above embodiment, the start feasibility determination unit 336 determines whether or not the occupant detected to be standing satisfies the fourth condition. However, the start feasibility determination unit 336 may determine whether or not the vehicle 100 is able to start without determining whether or not the fourth condition is satisfied. In this case, the start feasibility determination unit 336 determines whether or not the conditions are satisfied in the order of the first condition, the second condition, and the third condition, and when an occupant who does not satisfy any of the conditions is detected, the remaining determination processes are omitted and the vehicle is prohibited from starting.

[0052] The notification control unit 337 notifies the driver whether or not the vehicle 100 is permitted to start. Specifically, when the vehicle 100 is permitted to start, the notification control unit 337 notifies the driver of that fact via the output device 16. For example, the notification control unit 337 causes a display device to display that the vehicle 100 is permitted to start, and also causes a speaker to output a sound to that effect. On the other hand, when the vehicle 100 is prohibited from starting, the notification control unit 337 notifies the driver of that fact via the output device 16. For example, the notification control unit 337 causes a display device to display that the vehicle 100 is prohibited from starting, and also causes a speaker to output a sound to that effect. Furthermore, when the vehicle 100 is prohibited from starting, the notification control unit 337 may not operate the vehicle actuator 20 even if the driver operates the operating device.

[0053] <Vehicle control processing> Next, the vehicle control process will be described with reference to Figures 6 and 7. Figures 6 and 7 are flowcharts showing the flow of the vehicle control process. The illustrated vehicle control process is executed by the processor 33 of the ECU 30.

[0054] 6 and 7, when the vehicle control process is started, first, the position movement detection unit 332 detects the amount of movement of the center of gravity position of the bounding box corresponding to any occupant (step S11). Next, the start feasibility determination unit 336 determines whether the amount of movement of the center of gravity position detected in step S11 is less than a predetermined first threshold value (whether the first condition is satisfied) (step S12). If it is determined in step S12 that the amount of movement of the center of gravity position is equal to or greater than the first threshold value, the start feasibility determination unit 336 prohibits the start of the vehicle 100, and the notification / control unit 337 notifies the prohibition of the start (step S26).

[0055] If it is determined in step S12 that the amount of center-of-gravity position movement is less than the first threshold, the departure possibility determination unit 336 determines whether or not the processing of steps S11 and S12 has been completed for all occupants of the vehicle 100 (step S13). If it is determined in step S13 that the processing has not been completed for all occupants, steps S11 and S12 are repeated until the processing has been completed for all occupants.

[0056] On the other hand, if it is determined in step S13 that the processing has been completed for all occupants, the start feasibility determination unit 336 determines the order of occupants for which the amount of skeletal movement is to be detected (step S14). As described above, the start feasibility determination unit 336 determines the order so that the amount of skeletal movement is detected in descending order of the amount of occupant position movement detected by the position movement detection unit 332. Next, the skeletal movement detection unit 333 detects the amount of skeletal movement of the occupants in accordance with the order determined in step S14 (step S15). Next, the start feasibility determination unit 336 determines whether the amount of skeletal movement detected in step S15 is less than a predetermined second threshold value (whether the second condition is satisfied) (step S16). If it is determined in step S16 that the amount of skeletal movement is equal to or greater than the second threshold value, the start feasibility determination unit 336 prohibits the start of the vehicle 100, and the notification / control unit 337 notifies the prohibition of the start (step S26).

[0057] If it is determined in step S16 that the amount of movement of the skeleton is less than the second threshold, the departure possibility determination unit 336 determines (step S17) whether the processing of steps S15 and S16 has been completed for all occupants of the vehicle 100. If it is determined in step S17 that the processing has not been completed for all occupants, steps S15 and S16 are repeated until the processing has been completed for all occupants.

[0058] On the other hand, if it is determined in step S17 that the processing has been completed for all occupants, the start feasibility determination unit 336 determines the order in which the occupants' states are to be detected (step S18). As described above, the start feasibility determination unit 336 determines the order so that the occupant's state is detected in descending order of the amount of skeleton movement detected by the skeleton movement detection unit 333. Next, the state detection unit 334 detects the occupant's state in accordance with the order determined in step S18 (step S19). Next, the start feasibility determination unit 336 determines whether the occupant's state detected in step S19 is a state accompanied by a stable posture (whether the third condition is satisfied) (step S20). If it is determined in step S20 that the occupant's state is not a state accompanied by a stable posture, the start feasibility determination unit 336 prohibits the start of the vehicle 100, and the notification / control unit 337 notifies the prohibition of the start (step S26).

[0059] If it is determined in step S20 that the state of the occupant is a state involving a stable posture, the departure possibility determination unit 336 determines whether or not the processing of steps S19 and S20 has been completed for all occupants of the vehicle 100 (step S21). If it is determined in step S21 that the processing is not completed for all occupants, steps S19 and S20 are repeated until the processing is completed for all occupants.

[0060] On the other hand, if it is determined in step S21 that the processing has been completed for all occupants, the start permission determination unit 336 determines whether or not there are any occupants who were determined to be standing in step S19 (step S22).If it is determined in step S22 that there are no occupants who are standing, the start permission determination unit 336 permits the vehicle 100 to start, and the notification / control unit 337 notifies the vehicle 100 that the start is permitted (step S25).

[0061] On the other hand, if it is determined in step S22 that there is a standing occupant, it is determined whether or not any occupant determined to be standing is holding a body support facility (whether or not the fourth condition is satisfied) (step S23).If it is determined in step S23 that the occupant is not holding a body support facility, the departure permission determination unit 336 prohibits the departure of the vehicle 100, and the notification / control unit 337 notifies the prohibition of departure (step S26).

[0062] If it is determined in step S23 that the occupant is gripping the body support equipment, the start permission determination unit 336 determines whether the processing of step S23 has been completed for all occupants who are standing (step S24). If it is determined in step S24 that the processing has not been completed for any occupant, step S23 is repeated until the processing has been completed for all occupants who are standing. On the other hand, if it is determined in step S21 that the processing has been completed for all occupants who are standing, the start permission determination unit 336 permits the vehicle 100 to start, and the notification / control unit 337 notifies the vehicle 100 that the start is permitted (step S25).

[0063] Although preferred embodiments according to the present disclosure have been described above, the present disclosure is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. [Explanation of symbols]

[0064] 1 Vehicle control system, 11 In-vehicle camera, 12 Grasp sensor, 13 Driver operation sensor, 14 HMI, 20 Vehicle actuator, 30 ECU, 33 Processor, 100 Vehicle

Claims

1. A control device for a vehicle that transports occupants, a position movement detection unit that detects a position movement amount of an occupant in the vehicle; a skeleton movement detection unit that detects the amount of movement of the skeleton of each occupant; a state detection unit that detects the state of each occupant; a start possibility determination unit that determines whether or not the stopped vehicle can start, The departure possibility determination unit permitting the vehicle to start when it is determined that all of the following conditions are satisfied: a first condition that the amount of positional movement of the occupant is less than a first threshold value, a second condition that the amount of movement of the occupant's skeleton is less than a second threshold value, and a third condition that the occupant is in a state involving a stable posture; A control device that determines whether the conditions are met in the order of the first condition, the second condition, and the third condition, and when an occupant who does not meet any of the conditions is detected, skips the remaining determination processes and prohibits the vehicle from starting.

2. The control device according to claim 1, wherein when all occupants satisfy the first condition, the departure possibility determination unit determines whether there are any occupants who do not satisfy the second condition, in order of occupants with the largest amount of positional movement.

3. The control device according to claim 1 or 2, wherein when all occupants satisfy the second condition, the departure possibility determination unit determines whether there are any occupants who do not satisfy the third condition, in order of occupants with the greatest amount of skeletal movement.

4. a grip determination unit that determines whether the occupant detected as being in a standing state by the state detection unit is gripping a body support facility around the occupant, The departure possibility determination unit permitting the vehicle to start when it is determined that a fourth condition, in which the occupant detected to be in a standing state is holding a body support device, is satisfied in addition to the first to third conditions; 3. The control device according to claim 1, wherein the control device determines whether the conditions are met in the order of the first condition, the second condition, the third condition, and the fourth condition, and when an occupant who does not meet any of the conditions is detected, the control device skips the remaining determination processes and prohibits the vehicle from starting.

5. A method for controlling a vehicle for transporting an occupant, comprising: Detecting a positional movement amount of an occupant in the vehicle; Detecting the amount of movement of each occupant's skeleton; Detecting the state of each occupant; permitting the vehicle to start from a stop when it is determined that all of the following conditions are satisfied: a first condition that the amount of positional movement of the occupant is less than a first threshold value, a second condition that the amount of movement of the occupant's skeleton is less than a second threshold value, and a third condition that the state of the occupant is a state accompanied by a stable posture; determining whether the first condition, the second condition, and the third condition are satisfied in that order, and, when an occupant who does not satisfy any of the conditions is detected, skipping the remaining determination processes and prohibiting the vehicle from starting from a stopped state.

Images