Vehicle control device

Abstract

The present disclosure relates to a vehicle control device, which improves techniques related to posture determination based on likelihood of a person's posture. A vehicle control device (20) is provided with a control unit (200) that derives, from an image of an object at a first time point, a reference orientation likelihood indicating a likelihood that the orientation of the object is a reference orientation at the first time point, and a nearest orientation likelihood indicating a likelihood that the orientation of the object is a nearest orientation. Calculating a likelihood ratio, which is the ratio of the reference posture likelihood and the nearest posture likelihood at the first time, the nearest posture being a posture that may be taken before and after the object has transitioned to the reference posture; when the reference posture likelihood at the first time is greater than or equal to a threshold value and the likelihood ratio at the first time is greater than or equal to a first predetermined value, a mark indicating that the posture of the object is the reference posture is ON. When the reference posture likelihood at the first time is lower than a threshold value or the likelihood ratio at the first time is lower than a first predetermined value, the marker is set to OFF.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a vehicle control device. BACKGROUND

[0002] In the past, a technique has been known in which a posture of a person is determined based on a posture likelihood indicating a likelihood that the posture of the person is a specific posture. For example, in Patent Literature 1, a technique is disclosed in which a likelihood of a posture of a person is output, and in a case where the likelihood is equal to or higher than a threshold value, it is determined that the posture of the person is a specific posture (for example, a lying posture).

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2024-046924 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In a case where the posture determination is used to determine whether it is permissible to start a vehicle, it is necessary to determine that the posture of a person in the vehicle is a safe posture, such as a seated posture. In a case where the seated determination is performed by image recognition, it is possible to determine the seated posture based on whether a seated likelihood, which indicates a likelihood that the posture of the subject is a specific posture, exceeds a threshold value. However, depending on factors such as a sitting posture of the person, a position of the seat, an internal configuration of the vehicle, brightness at the time of imaging, and a deviation of an input image signal caused by a performance of the camera, the seated likelihood can increase or decrease. Therefore, in order to accurately determine the seated posture, it is necessary to set the threshold value to be high. If the threshold value is high, a time required until the seated likelihood exceeds the threshold value increases, and it is possible that stress is given to the passenger. In addition, the seated determination is performed on all the persons in the vehicle. Therefore, waiting until the seated likelihood of all the persons exceeds the threshold value is possible to give further stress to the passenger. On the other hand, if the threshold value is low, it is difficult to accurately determine the seated posture.

[0008] An object of the present disclosure accomplished in light of the above-described circumstances is to improve a technique related to posture determination based on a likelihood of a posture of a person.

[0009] TECHNICAL SOLUTION TO THE PROBLEM

[0010] A vehicle control device of one embodiment of the present disclosure includes a control unit that

[0011] The control section derives, from the image of the object at the first time, a reference posture likelihood that represents a likelihood that the posture of the object is a reference posture and a recent posture likelihood that represents a likelihood that the posture of the object is a recent posture at the first time, and calculates a ratio of the reference posture likelihood to the recent posture likelihood, that is, a likelihood ratio at the first time, the recent posture being a posture that the object is likely to take before and after the reference posture is assumed,

[0012] In a case where the reference posture likelihood at the first time is equal to or higher than a threshold value and the likelihood ratio at the first time is equal to or higher than a first predetermined value, the control section sets a mark that represents that the posture of the object is the reference posture to ON.

[0013] In a case where the reference posture likelihood at the first time is lower than the threshold value or the likelihood ratio at the first time is lower than the first predetermined value, the control section sets the mark to OFF.

[0014] Effects of Invention

[0015] According to an embodiment of the present disclosure, a technology related to posture determination based on a likelihood of a posture of a person is improved. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a block diagram showing an outline structure of a vehicle control system of the present embodiment.

[0017] Figure 2 is an outline graph showing an example of a time change of a posture and a posture likelihood before and after an object assumes a seated posture.

[0018] Figure 3 is a flowchart showing an action of a vehicle control device of the present embodiment.

[0019] (Symbol Explanation)

[0020] 1: system; 10: imaging device; 20: vehicle control device; 200: control section; 201: communication section; 202: storage section; 30: network. DETAILED DESCRIPTION

[0021] Hereinafter, an embodiment of the present disclosure will be described with reference to the accompanying drawings.

[0022] (Outline of the Present Embodiment)

[0023] Reference Signs Figure 1, and the outline of the vehicle control system 1 of the present embodiment is explained. The vehicle control system 1 has the imaging device 10 and the vehicle control device 20. The imaging device 10 and the vehicle control device 20 are connected in a manner that enables them to communicate with each other via the network 30 including, for example, the Internet and a mobile communication network.

[0024] The imaging device 10 is at least one vehicle-mounted camera. The imaging device 10 captures an object in a seat and in the vicinity of the seat in a vehicle during a period in which the vehicle is temporarily parked.

[0025] The vehicle is any means of transport, for example, a car, a bus, or an inter-zone bus, that can carry one or more passengers. The entrance and the seat can also be arranged at other locations. In the present embodiment, the vehicle is an automated driving vehicle that can perform automated driving at a level of 1 to 5 defined in SAE (Society of Automotive Engineers). The vehicle can also be a manual driving vehicle at a level of 0. The vehicle can also be remotely monitored by a monitor outside the vehicle. The vehicle can also be a MaaS-dedicated vehicle. "MaaS" is an abbreviation for Mobility as a Service.

[0026] The vehicle control device 20 is an electronic device, for example, a computer, mounted on the vehicle. The vehicle control device 20 detects an object from an image captured by the imaging device 10 using any object detection technique. The image can be a still image or a moving image. The object is a person in the vehicle. The vehicle control device 20 derives a posture likelihood degree indicating a likelihood that a posture of the object is a specific posture from the image using any posture estimation technique, for example, machine learning. The vehicle control device 20 estimates the posture of the object using a plurality of posture likelihood degrees and decides whether to allow the vehicle to start based on the estimation result.

[0027] First, the outline of the present embodiment is explained, and the details are described later. The vehicle control device of the present embodiment has a control section. The control section derives a reference posture likelihood degree indicating a likelihood that a posture of the object is a reference posture and a recent posture likelihood degree indicating a likelihood that the posture of the object is a recent posture from an image of the object at a first time, and calculates a ratio of the reference posture likelihood degree to the recent posture likelihood degree, that is, a likelihood ratio at the first time. The recent posture is a posture that the object is likely to assume before and after changing into the reference posture. The control section sets a flag indicating that the posture of the object is the reference posture to ON in a case where the reference posture likelihood degree at the first time is equal to or higher than a threshold value and the likelihood ratio at the first time is equal to or higher than a first predetermined value. The control section sets the flag to OFF in a case where the reference posture likelihood degree at the first time is lower than the threshold value or the likelihood ratio at the first time is lower than the first predetermined value.

[0028] Due to differences in sitting posture, determining the most recent posture is sometimes easier than determining the baseline posture (sitting posture) for different individuals. In this embodiment, even when it is difficult to determine the sitting state using only the sitting likelihood, i.e., when the sitting likelihood is below a predetermined threshold, the sitting posture can be accurately determined by comparing the sitting likelihood with the likelihood of the most recent posture. As a result, the time required for sitting determination is reduced, and passenger stress is decreased.

[0029] The reduction in seating determination time is useful for driverless autonomous vehicles. Such vehicles are, for example, remotely monitored by a monitor at a base station. If the vehicle does not start immediately, passengers may perceive a malfunction and communicate with the monitor to request a solution. Increased communication frequency between passengers and the monitor can increase the monitor's workload. This workload can increase further when one monitor oversees multiple autonomous vehicles. The vehicle control device of this embodiment reduces the seating determination time, thereby reducing the workload of such monitors.

[0030] If multiple cameras or other sensors are used to improve the accuracy of seating determination, the cost of seating determination increases. The vehicle control device of this embodiment reduces the number of cameras or other sensors required by utilizing the likelihood of a reference pose and the most recent pose derived from a single image, thereby reducing the cost of seating determination.

[0031] Therefore, this embodiment improves the technology related to posture determination based on the likelihood of human posture.

[0032] Next, the structure of the vehicle control system 1 will be explained.

[0033] (Structure of camera device 10)

[0034] The camera device 10 is any camera module installed inside the vehicle and capable of capturing images of all seats and objects within the vehicle. The camera module includes one or more cameras. In this embodiment, the camera device 10 is a single camera installed on the ceiling near the vehicle's entrance. The camera device 10 could also be two cameras installed in other locations, such as the center and rear of the vehicle's ceiling. The camera device 10 could also be a 180-degree camera or a 360-degree camera. The camera device 10 transmits the captured images to the vehicle control device 20 via network 30.

[0035] (Vehicle control device 20)

[0036] The vehicle control device 20 includes a control unit 200, a communication unit 201, and a storage unit 202.

[0037] The control unit 200 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or combinations thereof. The control unit 200 controls the operation of the vehicle control device 20.

[0038] The communication unit 201 includes at least one communication interface connected to the network 30. This communication interface may correspond to mobile communication standards such as 4G or 5G, V2X communication standards such as DSRC or cellular V2X, or wireless LAN communication standards such as IEEE 802.11. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. "DSRC" is an abbreviation for dedicated short-range communications. "V2X" is an abbreviation for vehicle-to-everything. "IEEE" is an abbreviation for the Institute of Electrical and Electronics Engineers.

[0039] Storage unit 202 includes one or more memories. Each memory included in storage unit 202 may function as a main storage device, an auxiliary storage device, or a cache memory. Storage unit 202 stores any information used in the operation of vehicle control device 20. For example, storage unit 202 stores system programs, application programs, embedded software, and any data used in object detection and posture estimation. Storage unit 202 may also pre-store information on the position and shape of each seat, or models of postures used for seating estimation, such as reference postures and recent postures. These models may also be generated through machine learning. The information stored in storage unit 202 can also be updated by control unit 200. In this embodiment, a marker indicating that the object's posture is a seating posture is stored in storage unit 202 and updated by control unit 200.

[0040] (Postural likelihood)

[0041] Next, we will explain posture likelihood. Posture likelihood represents the probability that an object's posture is a specific posture. The control unit 200 of the vehicle control device 20 derives a reference posture likelihood, which represents the probability that the object's posture is a reference posture, and a recent posture likelihood, which represents the probability that the object's posture is a recent posture. The recent posture is the posture that the object may take before or after transitioning to the reference posture. As the recent posture in the seating determination, examples include a hunched posture, a standing posture, a walking posture, and a squatting posture. In this embodiment, the reference posture is the seating posture, and the recent posture is the hunched posture. The seating posture is the posture of the object when it is seated. The hunched posture is the posture of the object bending over just before sitting down. The standing posture is the posture while standing. The walking posture is the posture while walking. The squatting posture is the posture of the object squatting down on the floor or a seat.

[0042] For an example of the temporal changes in pose and pose likelihood before and after an object transforms into a seated posture, refer to... Figure 2 Let me explain. Figure 2 The upper side shows the change in the object's pose. Figure 2 In this scenario, the object's posture changes from the most recent posture to a seated posture, and then returns to the most recent posture. The changes in the object's posture are categorized into states A through E. In state A, the object's posture is the most recent posture, changing from a walking posture through a standing posture to a bowed posture. In state B, the object's posture gradually changes from the most recent posture (bowing posture) to a seated posture. In state C, the object's posture is a seated posture. In state D, the object's posture gradually changes from a seated posture to the most recent posture. In state E, the object's posture is the most recent posture, changing from a bowed posture through a standing posture to a walking posture.

[0043] Figure 2 The lower part of the diagram shows the changes in the likelihood of sitting (X) and the likelihood of leaning (Y). In state A, the likelihood of sitting (X) is below the second threshold (X2), and the likelihood of leaning (Y) is above the likelihood of sitting (X). In state B, the likelihood of sitting (X) is above the second threshold (X2) and below the first threshold (X1). As the posture changes, the likelihood of sitting (X) increases, while the likelihood of leaning (Y) decreases, and the likelihood of sitting (X) becomes higher than the likelihood of leaning (Y). In state C, the likelihood of sitting (X) is above the first threshold (X1), reaching its highest value, while the likelihood of leaning (Y) reaches its minimum value. In state D, the likelihood of sitting (X) is above the third threshold (X3) and below the first threshold (X1). As the posture changes, the likelihood of sitting (X) decreases, while the likelihood of leaning (Y) increases, and the likelihood of leaning (Y) becomes higher than the likelihood of sitting (X). In state E, the likelihood of sitting down X is lower than the third threshold X3, and the likelihood of leaning forward Y is higher than the likelihood of sitting down X. The first threshold X1 is set to be higher than both the second threshold X2 and the third threshold X3. The second threshold X2 is... Figure 2The value can be set to be higher than the third threshold X3, but it can also be set to be the same as or lower than the third threshold X3.

[0044] exist Figure 2 Although not shown in the figure, the walking posture, standing posture, and bowing posture reach their maximum values ​​in the order of walking posture, standing posture, and bowing posture in state A, and in the reverse order in state E.

[0045] In states B and D, the boundary between the sitting posture and the leaning posture is unclear, so it is difficult to perform accurate sitting determination using only the sitting likelihood. The vehicle control device 20 of this embodiment uses both sitting likelihood and leaning likelihood, so that it can perform accurate sitting determination even in states B and D.

[0046] The pose of an object is sometimes related to Figure 2 The postures change in different ways. For example, when the subject is a child, the subject's posture may sometimes change from a squatting posture to a leaning posture, then to a sitting posture, and then back to a squatting posture in the reverse order. The likelihood of these postures reaches its maximum in the order of squatting posture, leaning posture, sitting posture, leaning posture, and squatting posture.

[0047] (Operational flow of Control Unit 200)

[0048] Reference Figure 3 This section explains the operation of the control unit 200 in the vehicle control device 20 of this embodiment. For example, during a temporary stop of the vehicle at a station, the control unit 200 executes the following steps S101 to S106. In S101 to S106, the control unit 200 determines whether the subject's posture has changed to a seated posture. The flag indicating that the subject's posture is a seated posture is initially set to OFF.

[0049] S101: The control unit 200 detects objects from the image at time 1.

[0050] The first moment is the moment when the camera device 10 captures the seat and objects near the seat during the period when the vehicle is parked. The control unit 200 receives the image captured at the first moment from the camera device 10 via the communication unit 201. The control unit 200 detects objects from the image at the first moment using arbitrary object detection technology.

[0051] S102: The control unit 200 derives the seating likelihood and the leaning likelihood at the first moment from the image of the object at the first moment, and calculates the likelihood ratio at the first moment.

[0052] The likelihood ratio is the ratio of the baseline posture likelihood to the most recent posture likelihood, expressed by the formula R = X / Y. Here, R is the likelihood ratio, X is the baseline posture likelihood, and Y is the most recent posture likelihood. In this embodiment, the baseline posture likelihood is the sitting likelihood, and the most recent posture likelihood is the leaning-over likelihood.

[0053] S103: Control unit 200 determines whether the seating likelihood is greater than or equal to the first threshold X1. If the seating likelihood is greater than or equal to the first threshold X1 (S103 - "Yes"), the process proceeds to S105. If not (S103 - "No"), the process proceeds to S104.

[0054] S104: Control unit 200 determines whether the seating likelihood at time 1 is greater than or equal to the second threshold x2 and whether the likelihood ratio at time 1 is greater than or equal to the first predetermined value R1. If it is determined that the seating likelihood at time 1 is greater than or equal to the second threshold x2 and the likelihood ratio at time 1 is greater than or equal to the first predetermined value R1 (S104 - "Yes"), the process proceeds to S105. If not (S104 - "No"), the process proceeds to S106.

[0055] S103~S104 correspond to Figure 2 The seating determination is performed under states A through C. The control unit 200 may also omit the determination in S103. In S104, the control unit 200 may also omit the determination regarding whether the seating likelihood at the first time point is greater than or equal to the second threshold x2.

[0056] S105: Control unit 200 sets the flag to ON. When the flag is already ON, control unit 200 does nothing.

[0057] S106: Control unit 200 sets the flag to OFF. Afterwards, processing returns to S102.

[0058] The control unit 200 repeatedly performs S102 to S106 until the flag is set to ON.

[0059] The control unit 200 performs processing steps S101 to S106 for each object within the vehicle. When all object flags within the vehicle are set to ON, the control unit 200 allows the vehicle to start. When more than one object flag is set to OFF, the control unit 200 does not allow the vehicle to start.

[0060] When the vehicle starts moving after the indicator is set to ON in S106, and then stops again due to a pedestrian crossing or traffic light, the control unit 200 executes the following steps S107 to S109. In S107 to S109, the control unit 200 determines whether the subject's posture has changed from a seated posture.

[0061] S107: The control unit 200 derives the seating likelihood and leaning likelihood at the second time moment from the image at the second time moment, and calculates the likelihood ratio at the second time moment.

[0062] The second moment is the time after the marker in S105 is set to ON and the vehicle starts moving. In this embodiment, the second moment is the time when the camera device 10 captures the seat and objects near the seat during the period after the vehicle stops again following the marker being set to ON in S105 and the vehicle starting moving. The control unit 200 receives the image captured at the second moment from the camera device 10 via the communication unit 201.

[0063] S108: Control unit 200 determines whether the seating likelihood at the second time point is greater than or equal to the first threshold X1. If it is determined that the seating likelihood at the second time point is greater than or equal to the first threshold X1 (S108 - "Yes"), the process returns to S105. If it is determined that the seating likelihood is less than or equal to the first threshold X1 (S108 - "No"), the process proceeds to S109.

[0064] S109: Control unit 200 determines whether the seating likelihood at the second time moment is greater than or equal to the third threshold x3 and whether the likelihood ratio at the second time moment is greater than or equal to the second predetermined value R2. If it is determined that the seating likelihood at the second time moment is greater than or equal to the third threshold x3 and whether the likelihood ratio at the second time moment is greater than or equal to the second predetermined value R2 (S109 - "Yes"), the process returns to S105. If not (S109 - "No"), the process returns to S106.

[0065] S108~S109 correspond to Figure 2 The seating determination is performed under states C to E. The control unit 200 may also omit the determination in S108. In S109, the control unit 200 may also omit the determination of whether the seating likelihood at the second time point is greater than or equal to the third threshold x3. If the process returns to S106 from S108 or S109, the control unit 200 executes S102 to S106 again.

[0066] Control unit 200 executes S107 to S109 for each object within the vehicle. When all object flags are set to ON, control unit 200 allows the vehicle to start. When more than one object flag is set to OFF, control unit 200 does not allow the vehicle to start. Control unit 200 executes S107 to S109 whenever the vehicle stops.

[0067] In other embodiments, in S102, S104, S107, and S109, the most recent posture may also be a standing posture, a walking posture, or a squatting posture. In other embodiments, in S102 and S107, the control unit 200 may also derive the likelihood of a hunched posture, a standing posture, a walking posture, and a squatting posture, and calculate the likelihood ratios based on the likelihood of each posture. In S104, the control unit 200 may set the flag to ON if the seating likelihood at the first time moment is greater than or equal to the second threshold x2, and at least one of these likelihood ratios at the first time moment is greater than or equal to the first predetermined value R1; otherwise, the flag may be set to OFF. In S108, the control unit 200 may set the flag to ON if the seating likelihood at the second time moment is greater than or equal to the second threshold x2, and at least one of these likelihood ratios at the second time moment is greater than or equal to the second predetermined value R2; otherwise, the flag may be set to OFF. If the object's behavior is rapid, deriving the likelihood of a hunching posture may fail in determining the hunching posture. In other implementations, even if deriving the likelihood of a nearest posture fails, other closest probabilities can be derived to perform the seating determination.

[0068] The second predetermined value R2 can also be different from the first predetermined value R1. When the time for determining seating is limited, such as when the car is stopped near a pedestrian crossing or traffic light, it is necessary to reduce the time for determining seating. Therefore, the second predetermined value R2 can also be lower than the first predetermined value R1.

[0069] The first predetermined value R1 and the second predetermined value R2 can also be set differently for each of the most recent poses. For example... Figure 2 As shown above, when the object is seated, the object's posture changes from walking posture, standing posture, to a bent-over posture to a seated posture, and the object's safety increases in this order. Therefore, in order to improve the object's safety, the first predetermined value R1 can also be set to increase in the order of bent-over likelihood, standing likelihood, and walking likelihood.

[0070] The first predetermined value R1 and the second predetermined value R2 can also be set differently depending on the orientation of the object (person) relative to the camera. It is more difficult to grasp the feature points of a person's posture from the front than from the side. For example, a person is upright in a standing posture, whereas in a walking posture, the feet move forward relative to the torso. The difference between standing and walking postures is more difficult to grasp when observing a person from the front than when observing them from the side. Therefore, the first predetermined value R1 and the second predetermined value R2 when the object is facing the front relative to the camera can also be set below the first predetermined value R1 and the second predetermined value R2 when the object is facing laterally relative to the camera. "Frontal orientation" refers to the direction from which the object faces the camera and the direction of inclination within a range greater than 0 degrees and less than 30 degrees relative to that direction. "Lateral orientation" refers to the direction of inclination within a range greater than 30 degrees and less than 150 degrees relative to the direction from which the object faces the camera.

[0071] Table 1 shows examples of the first predetermined value R1 and the second predetermined value R2 when the object is facing forward relative to the camera. Table 2 shows examples of the first predetermined value R1 and the second predetermined value R2 when the object is facing laterally relative to the camera.

[0072] Table 1

[0073] Table 2

[0074] To expedite the seating determination in S104 and S109, the likelihood ratio can also be defined as R = (X + α) / Y. Here, R is the likelihood ratio, X is the baseline pose likelihood, Y is the most recent pose likelihood, and α is any constant greater than 0. The greater the increase in α, the less time is required until the likelihood ratio exceeds a first predetermined value R1 or a second predetermined value R2. Table 3 shows examples of α when the object is facing forward or laterally relative to the camera.

[0075] Table 3

[0076] To improve the safety of the object, as shown in Table 3, α can also be set to increase in the order of leaning likelihood, standing likelihood, and walking likelihood.

[0077] As described above, the vehicle control device of this embodiment includes a control unit. The control unit derives from the image of the object at a first moment a base pose likelihood (representing the likelihood that the object's pose is a base pose) and a recent pose likelihood (representing the likelihood that the object's pose is a recent pose) at the first moment, and calculates the ratio of the base pose likelihood to the recent pose likelihood, i.e., the likelihood ratio. The recent pose is the pose the object might take before or after transitioning to the base pose. If the base pose likelihood at the first moment is above a threshold and the likelihood ratio at the first moment is above a first predetermined value R1, the control unit sets the flag indicating that the object's pose is a base pose to ON. If the base pose likelihood at the first moment is below a threshold or the likelihood ratio at the first moment is below the first predetermined value R1, the control unit sets the flag to OFF.

[0078] Due to differences in sitting posture, determining the most recent posture is sometimes easier than determining the baseline posture (sitting posture) for different individuals. Based on the above structure, even when the sitting likelihood is insufficient to determine the sitting state using only the sitting likelihood (i.e., when the sitting likelihood is below a predetermined threshold), the sitting posture can be accurately determined by comparing the sitting likelihood with the likelihood of the most recent posture. This reduces the time required for seating determination and decreases passenger stress.

[0079] The reduction in seating determination time is useful for driverless autonomous vehicles. Such vehicles are, for example, remotely monitored by a monitor at a base station. If the vehicle does not start immediately, passengers may perceive a malfunction and communicate with the monitor to request a solution. Increased communication frequency between passengers and the monitor can increase the monitor's workload. This workload can increase further when one monitor oversees multiple autonomous vehicles. The vehicle control device of this embodiment reduces this monitor's workload by decreasing the seating determination time.

[0080] If multiple cameras or other sensors are used to improve the accuracy of seating determination, the cost of seating determination increases. The vehicle control device of this embodiment reduces the number of cameras or other sensors required by utilizing the likelihood of a reference pose and the most recent pose derived from a single image, thereby reducing the cost of seating determination.

[0081] This disclosure has been described with reference to the accompanying drawings and embodiments, but those skilled in the art will understand that various modifications and alterations can be made based on this disclosure. Therefore, it should be noted that such modifications and alterations are included within the scope of this disclosure.

[0082] The values ​​and their relationships described in the above embodiments can also be appropriately changed. In this embodiment, in S104 and S109, the seating determination is performed based on the likelihood ratio. In other embodiments, the seating determination can also be performed based on the likelihood difference between the reference posture likelihood and the immediately preceding posture likelihood.

[0083] The functions included in each structural component or step can be reconfigured in a logically consistent manner, and multiple structural components or steps can be combined into one or divided. For example, in the above embodiment, it is also possible to distribute the structure and operation of the vehicle control device 20 among multiple computers that can communicate with each other. In addition, in the above embodiment, it is also possible to house part or all of the camera device 10 and the control device 20 in the same device.

Claims

1. A vehicle control device provided with a control section, the control section derives, from an image of a subject at a first time, a reference posture likelihood that indicates a likelihood that a posture of the subject at the first time is a reference posture, and a recent posture likelihood that indicates a likelihood that the posture of the subject at the first time is a recent posture, which is a posture that the subject is likely to take before and after the posture changes to the reference posture, and calculates a ratio of the reference posture likelihood to the recent posture likelihood at the first time, which is a likelihood ratio, in a case where the reference posture likelihood at the first time is equal to or higher than a threshold value and the likelihood ratio at the first time is equal to or higher than a first predetermined value, the control section sets a mark that indicates that the posture of the subject is the reference posture to ON, in a case where the reference posture likelihood at the first time is lower than the threshold value or the likelihood ratio at the first time is lower than the first predetermined value, the control section sets the mark to OFF.

2. The vehicle control device according to claim 1, wherein the control section derives, from an image of the subject at a second time after the mark is set to ON, the reference posture likelihood and the recent posture likelihood at the second time, and calculates the likelihood ratio at the second time, in a case where the reference posture likelihood at the second time is lower than a threshold value or the likelihood ratio at the second time is lower than a second predetermined value, the control section sets the mark to OFF.

3. The vehicle control device according to claim 2, wherein the second predetermined value is lower than the first predetermined value.

4. The vehicle control device according to claim 1, wherein the reference posture is a seated posture, and the recent posture is a forward-leaning posture, a standing posture, a walking posture, or a crouching posture.

5. The vehicle control device according to claim 1, wherein the control section permits a vehicle to start in a case where the mark is ON, and does not permit the vehicle to start in a case where the mark is OFF.

6. The vehicle control device according to claims 1 to 5, wherein the vehicle is an automated driving vehicle.

Images