Illumination control device and illumination control method

Abstract

An illumination control device for controlling a plurality of illuminations provided in a vehicle interior, the illumination control device being provided with: an image acquisition unit (11) for acquiring a captured image from a camera (1) for capturing an image of the vehicle interior; a recognition unit (12) that recognizes, on the basis of the captured image, the behavior of at least one occupant among the plurality of occupants, and the attribute of another occupant among the plurality of occupants, which is different from the one occupant, and / or the state of the other occupant; a state information acquisition unit (13) that acquires vehicle state information indicating a vehicle state of the vehicle; and an illumination control unit (14) that selects a target illumination to be an illumination target on the basis of the recognition result of the recognition unit (12) and the vehicle state, and that illuminates the selected target illumination.

Description

Technical Field

[0001] The invention relates to a lighting control device and a lighting control method. Background Art

[0002] Conventionally, there is known an interior light control device for controlling the lighting of interior lights installed in a vehicle (e.g., Patent Document 1). The interior light control device described in Patent Document 1 receives input of a camera image containing at least a portion of the interior of the vehicle, detects occupants in the vehicle from the received camera image, determines whether the occupants are performing a specific action, and illuminates the interior lights if it is determined that the occupants are performing the specific action.

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2016-107864 Summary of the Invention

[0006] Problems to be solved by the invention

[0007] The interior light control device described in Patent Document 1 has the following problem: when it is determined that a certain occupant is performing a specific action, the interior lights are turned on regardless of the behavior and attributes of other occupants except the occupant who performed the specific action, thereby damaging the comfort of other occupants due to the turning on of the interior lights.

[0008] The problem to be solved by the present invention is to provide a lighting control device and a lighting control method for controlling lighting while maintaining the comfort of passengers in a vehicle cabin.

[0009] Solutions for solving problems

[0010] The present invention solves the above-mentioned problem by acquiring a camera image from a camera, identifying the behavior of at least one occupant among a plurality of occupants, as well as the attributes and / or status of other occupants based on the camera image, acquiring vehicle status information, selecting object lighting to be the lighting object based on the recognition result and the vehicle status, and lighting the selected object lighting.

[0011] Effects of the Invention

[0012] According to the present invention, it is possible to control lighting while maintaining the comfort of passengers in the vehicle cabin. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing an example of an interior component control system according to the present embodiment.

[0014] Figure 2AThis is a table showing the correlation between the target occupant, other occupants (first column, second column), vehicle status, and target lighting.

[0015] Figure 2B This is a table showing the correlation between the target occupant, other occupants (first column, second column), vehicle status, and target lighting.

[0016] Figure 3 It shows Figure 1 Flowchart of the controller control process.

[0017] Figure 4 It shows Figure 3 Flowchart of the sub-routine of step S40 is shown.

[0018] Figure 5 It shows Figure 3 Flowchart of the subroutine of step S90 is shown. DETAILED DESCRIPTION

[0019] First Implementation Method

[0020] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Figure 1 This is a block diagram illustrating an example of a lighting control system according to this embodiment. The lighting control system according to this embodiment is installed in a vehicle and identifies the behavior of at least one occupant among multiple occupants, as well as attributes of other occupants different from the occupant whose behavior was identified, and / or the status of other occupants, based on images captured by an onboard camera. Based on the identification results and the vehicle status, the lighting control system selects target lighting to be illuminated and illuminates the selected target lighting.

[0021] Below, refer to Figure 1 2 , description will be given of the components included in the interior component control system of the present embodiment. Figure 1 This is a block diagram of the components of the lighting control system. The lighting control system includes a camera 1, an onboard device 2, footlights 4, a reading light 5, a cup holder light 6, a map light 7, a room light 8, and a lighting control device 9.

[0022] Camera 1 is installed in the vehicle and is used to capture images of the interior. It activates in conjunction with the vehicle's power switch or the power supply for an accessory, continuously capturing images of the interior of the vehicle and outputting them to lighting control device 9. Here, the accessory power supply refers to a power source that operates when the power switch is off. In-vehicle device 2 includes equipment such as a shift position, power switch, or accessory power switch that can be operated by the occupants to control the vehicle's status, as well as detection devices such as a speed sensor and torque sensor that indicate the vehicle's current status.

[0023] The footlights 4, reading lights 5, cup holder lighting 6, map lights 7, and interior lights 8 are lighting installed in the vehicle cabin. The footlights 4, reading lights 5, and cup holder lighting 6 are respectively located in the driver's seat area, the front passenger seat area, the left rear seat area, the center rear seat area, and the right rear seat area. The footlights 4 are lighting that illuminates the feet of the passengers and are located under the seats. The reading lights are lighting that illuminates the hand when the passengers are holding items such as books, portable terminals (such as smartphones, tablets, and other electronic devices). The reading lights are located on the headrests of the seats. The cup holder lighting 6 is lighting that illuminates the cup holders installed in the vehicle cabin. The cup holder has a hollow portion with an opening to hold a cylindrical container, and the cup holder lighting 6 is composed of a ring-shaped LED along the edge of the opening. The map light 7 is located in the center of the ceiling portion of the front seat, and the interior light 8 is located in the center of the ceiling portion between the front seat and the rear seat.

[0024] The lighting control device 9 recognizes the behavior of the occupants from the camera's captured image, and based on the recognition result and the vehicle status, outputs a control instruction for switching the lighting such as the footlights 4 on and off. The lighting control device 9 has a controller 10. The controller 10 has at least the function of acquiring a captured image, the function of recognizing the behavior of the occupants, the function of acquiring information on the vehicle status, and the function of controlling the lighting, and has an image acquisition unit 11, an identification unit 12, a status information acquisition unit 13, and a lighting control unit 14 as functional blocks for each function. The controller 10 has a processor and a memory storing a program. Moreover, the processor executes each function by processing the program. In addition, the controller 10 may also have functions other than the image acquisition unit 11.

[0025] The image acquisition unit 11 acquires a camera image obtained by shooting the interior of the vehicle from the camera 1. The recognition unit 12 recognizes the behavior of at least one of the multiple passengers based on the camera image. The behavior of the passenger recognized by the recognition unit 12 is recognized by the position of the body parts such as the hands, eyes, and mouth of the passenger, the movement of the parts, the facial expression, the skeleton of the passenger, etc. The recognition unit 12 performs image processing on the camera image to extract the position of the body parts and the hand parts and / or the movement of the parts. An example of the behavior recognized by the recognition unit 12 is given. For example, it is the behavior of the passenger looking for something at the feet (under the seat), the behavior of holding and using an item such as a book or a portable terminal (such as a smart phone, tablet computer, etc.), the behavior of drinking a beverage (the behavior of moving a beverage container to a cup holder, the behavior of taking a beverage container out of the cup holder, the behavior of drinking a beverage in a beverage container), etc.

[0026] Furthermore, the recognition unit 12 can also identify items (objects) used by the passenger from the captured image. For example, if the passenger is drinking from a beverage container, the recognition unit 12 will identify the beverage container as the passenger's item. For example, if the passenger is reading a book or looking at a portable device, the recognition unit 12 will identify the book or portable device.

[0027] The recognition unit 12 recognizes the attributes of other passengers and / or the states of other passengers that are different from the passenger whose behavior is recognized. For example, when the passenger in the front passenger seat performs an action of searching for something at his feet, the recognition unit 12 recognizes the behavior of the passenger in the front passenger seat and recognizes the attributes and states of the passengers in the driver's seat and the back seat, who are other passengers other than the passenger in the front passenger seat. The attributes of the passengers include the difference between adults and children, age, gender, etc. The states of the passengers include the state in which the passengers are sleeping, the state in which the passengers are talking, etc. In addition, the recognition unit 12 only needs to recognize at least one of the attributes and states of the other passengers. In the case of multiple other passengers, the attributes and states can also be recognized for each passenger.

[0028] The state information acquisition unit 13 acquires vehicle state information indicating the vehicle's state. Vehicle states include parked, parked, driving, and started states. The started state indicates when the vehicle has switched from parked or parked to driving. The state information acquisition unit 13 may acquire vehicle state information from the onboard device 2, such as a gear position sensor, a vehicle speed sensor, a power switch, or an accessory power switch.

[0029] The lighting control unit 14 selects target lighting to be illuminated based on the recognition results of the recognition unit 12 and the vehicle status acquired by the status information acquisition unit 13, and illuminates the selected target lighting. The lighting control unit 14 selects the target lighting based on the seat position of the occupant whose behavior was recognized by the recognition unit 12 and the recognized behavior. For example, if the recognition unit 12 recognizes the driver's seat occupant's behavior as "searching for something at their feet," the lighting control unit 14 selects the driver's seat footlights 4 and the center zone map lights 7 as the target lighting from the lighting in the driver's seat area and the multiple lighting in the center zone. In the case of "searching for something at their feet," the lighting control unit 14 selects at least the footlights 4 that focus on illuminating the area below the driver's seat as the target lighting. Alternatively, the lighting control unit 14 may select the footlights 4 in the area near the occupant, where part of the occupant's body is out of the camera 1's field of view, as the target lighting.

[0030] In addition, in the case of the behavior of "looking for something at the feet", the lighting control unit 14 may also set the map light 7 as the lighting target to make the lower part of the driver's seat brighter. The map light 7 not only illuminates the driver's seat but also the passenger seat. For example, when the passenger in the passenger seat is sleeping, the lighting of the map light 7 may cause the passenger to wake up. In addition, as another example, in the case of the behavior of "looking for something at the feet", the interior light 8 may also be set as the lighting target to make the lower part of the driver's seat brighter. The interior light 8 not only illuminates the driver's seat but also the rear seat. For example, when a child (especially a baby) is sitting in the rear seat, the lighting of the interior light 8 may scare the child and cause the child to cry or make noise. The lighting control unit 14 selects the target lighting based on the attributes of other passengers different from the passenger whose behavior is identified and / or the status of other passengers.

[0031] For example, if the recognition unit 12 identifies the driver's seat occupant's behavior as "searching for something at their feet," and identifies the passenger's attribute as "adult" and their status as "sleeping," the lighting control unit 14 excludes the map light 7 and interior light 8 from the target lighting, or does not select them as target lighting. On the other hand, if the passenger's attribute is identified as "adult" and their status is "not sleeping," the lighting control unit 14 selects the map light 7 and interior light 8 as target lighting. As another example, if the recognition unit 12 identifies the driver's seat occupant's behavior as "searching for something at their feet," and identifies the rear seat occupant as "child," the lighting control unit 14 excludes the map light 7 and interior light 8 from the target lighting, or does not select them as target lighting. Furthermore, if the recognition unit 12 identifies that the driver's seat occupant is "searching for something at their feet" and there are no other occupants besides the driver's seat occupant, the lighting control unit 14 selects the map light 7 and the interior light 8 as target lighting. Thus, for example, even if the passenger seat occupant is "sleeping" or the rear seat occupant is a "child," the footlights 4 can be illuminated to facilitate the driver's search for items at their feet while maintaining the comfort of the passenger seat occupants and rear seat occupants.

[0032] The lighting control unit 14 selects the target lighting according to the vehicle state. The lighting control unit 14 determines the vehicle state represented by the vehicle state information. For example, when the vehicle is traveling, the lighting control unit 14 does not select the map light 7 and the interior light 8 as the target lighting, regardless of the behavior of the identified occupant. For example, when the vehicle is traveling and the passenger seat occupant is "looking for something at their feet" and there are no passengers other than the driver seat occupant and the passenger seat occupant, the lighting control unit 14 selects the footlights 4 in the passenger seat area as the target lighting, and does not select the map light 7 and the interior light 8 as the target lighting. On the other hand, when the vehicle is in a stopped state or a parked state, and the passenger seat occupant is "looking for something at their feet" and there are no passengers other than the driver seat occupant and the passenger seat occupant, the lighting control unit 14 selects the map light 7 and the interior light 8 as the target lighting in addition to the footlights 4 in the passenger seat area. That is, since it is not appropriate to light the map lamp 7 and the interior lamp 8 during driving, the lighting control unit 14 excludes them from the target lighting.

[0033] Furthermore, while the above description describes how the lighting control unit 14 selects target lighting based on individual factors, including the occupant's behavior, the attributes / status of other occupants, and the vehicle's status, the lighting control unit 14 may also select target lighting based on a combination of all of these factors. Furthermore, the lighting control unit 14 may select target lighting based on the behavior of another occupant, different from the identified occupant's behavior. For example, if the driver's seat occupant is searching for something at their feet while the vehicle is parked or stopped, and the passenger seat occupant is also searching for something at their feet, the occupants are likely searching not just under a specific seat but across a wide area of ​​the vehicle floor. In this case, the lighting control unit 14 selects at least the driver's and passenger seat footlights 4 as target lighting, and also the map lights 7 and interior lights 8. Furthermore, the lighting control unit 14 selects the rear passenger seat footlights 4 as target lighting.

[0034] The lighting control unit 14 refers to, for example, Figure 2A and Figure 2B The table shown selects object lighting based on the occupant's behavior, the attributes / statuses of other occupants, and the vehicle state. Figure 2A and Figure 2B is a table showing the correlation between the target occupant, other occupants (first column and second column), and vehicle status and the target lighting. For example, when the recognition unit 12 recognizes that the passenger's behavior is "reading", the driver's seat occupant's state is "sleeping", and the vehicle state is "parked", the lighting control unit 14 sets the target lighting from Figure 2AThe action number "19" is extracted from the table shown, and the target lighting "passenger seat reading light 5 (passenger seat reading light)" corresponding to the recognition result and the vehicle state is determined, thereby selecting the target lighting. Figure 2A and Figure 2B In the figure, “target occupant” indicates an occupant whose behavior is recognized by the recognition unit 12 , and “unconditional” indicates that there is no restriction on the attributes or status of the occupant.

[0035] After selecting the target lighting, the lighting control unit 14 sets the brightness of the target lighting based on the brightness inside the vehicle cabin. The lighting control unit 14 determines the brightness based on the image captured by the camera 1. Alternatively, the lighting control unit 14 may determine the brightness inside the vehicle cabin based on the detection value of an illuminance sensor installed inside the vehicle. The lighting control unit 14 then sets the brightness of the target lighting so that the illuminance of the target lighting increases as the brightness inside the vehicle cabin darkens. While the vehicle is in motion, the lighting control unit 14 sets the illuminance of the target lighting to a level that does not affect the driver's driving. The lighting control unit 14 then outputs a control command to the target lighting so that the target lighting illuminates at the set illuminance.

[0036] After the target lighting is turned on, the lighting control unit 14 determines whether a deactivation condition is satisfied based on the recognition result of the recognition unit 12 and / or the vehicle status. The deactivation condition is a condition for deactivating the target lighting. The deactivation condition includes at least one of the following: the recognized action has ended, the vehicle has been started, and a predetermined time has passed since the lighting was turned on.

[0037] The recognition unit 12 recognizes the behavior of the occupant while the target lighting is on. When the behavior recognized when the target lighting is selected has ended, the lighting control unit 14 determines that the extinguishing condition is satisfied and extinguishes the target lighting. In addition, when the lighting control unit 14 determines that the vehicle is in the startup state based on the vehicle state while the target lighting is on, it determines that the extinguishing condition is satisfied and extinguishes the target lighting. For example, when the gear position is switched from parking to drive while the driver's seat footlights 4 are on, or when the vehicle state is switched from parking to driving based on the vehicle speed, the lighting control unit 14 extinguishes the driver's seat footlights 4 to avoid adversely affecting the driver's driving.

[0038] For example, assume the following scenario: the driver's seat occupant behaves by "looking at the portable terminal" while the vehicle is parked and no other occupants are in the vehicle. In such a scenario, the lighting control unit 14 lights up the reading lamp 5 in the driver's seat area and the map lamp 7 in the center area based on the vehicle state and the behavior of the driver's seat occupant (see Figure 2A"Action number (11)"). It is then assumed that the driver's seat occupant has finished the behavior of "looking at the portable terminal". When the vehicle interior is dark or in an environment that is not suitable for shooting by the camera 1, the recognition unit 12 may not be able to recognize from the camera image that the behavior of "looking at the portable terminal" has ended, or it may take time to recognize that the behavior of "looking at the portable terminal" has ended. That is, when the lighting control unit 14 performs a process such as turning off the map light 7 because it determines that the turning-off condition "the recognized behavior has ended" is satisfied, it takes time to turn off the map light 7. In this embodiment, by including the vehicle start-up in the turning-off condition, even in a situation where the recognition unit 12 needs time to recognize the occupant's behavior, the target lighting such as the map light 7 can be turned off due to the vehicle start-up. In addition, in the case where the target lighting is turned off due to the vehicle start-up, the lighting control unit 14 can turn off the lighting in the target lighting that affects the driver's driving.

[0039] The lighting control unit 14 turns on the target lighting and measures the time elapsed from the time point, and turns off the target lighting when the elapsed time is longer than a predetermined time. The predetermined time is a time that defines a so-called timeout and is set in advance.

[0040] Next, use Figures 3 to 5 The control flow of the controller 10 will be described. Figure 3 : is a flowchart showing the control process of the controller 10 . Figure 4 yes Figure 3 The sub-flowchart of step S40 is shown. Figure 5 yes Figure 3 The sub-flowchart of step S90 is shown. In addition, Figures 3 to 5 The control flow shown is executed while the vehicle is stopped or parked.

[0041] In step S10, the controller 10 determines whether filming is permitted. For example, if filming by the camera 1 is not permitted for privacy protection, the controller 10 terminates the control flow ("No" in step S10). On the other hand, if filming is permitted, in step S20, the image acquisition unit 11 acquires a captured image from the camera 1. In step S30, the state information acquisition unit 13 acquires vehicle state information indicating the vehicle state.

[0042] In step S40, the recognition unit 12 performs image processing on the camera image and executes behavior recognition processing. The control flow of the behavior recognition processing is as follows: Figure 4The process is shown in the sub-routine of FIG. In step S41, the recognition unit 12 recognizes an occupant. The identification of the occupant includes, for example, the occupant's seating position. In step S42, the recognition unit 12 recognizes the occupant's behavior. In step S43, the recognition unit 12 identifies the attributes of other occupants and / or the states of other occupants different from the occupant whose behavior was identified. After completing the sub-routine processing of steps S41 to S43, the controller 10 executes the control process of step S50.

[0043] In step S50, the lighting control unit 14 determines whether the recognized behavior is consistent with the Figure 2A and Figure 2B The specific behavior is consistent with the specific behavior specified in the table shown. The specific behavior corresponds to the lighting that becomes the target of lighting control. Figure 2A and Figure 2B The behavior shown in the "Target Occupant - Behavior" column of the ECU corresponds to the specific behavior. If the recognized behavior is inconsistent with the specific behavior, the controller 10 returns the control flow to step S20 and executes the control flow after step S20. If the recognized behavior is consistent with the specific behavior, in step S60, the lighting control unit 14 selects the target lighting to be lit based on the recognition result of the recognition unit 12 and the vehicle state. Specifically, the lighting control unit 14 refers to Figure 2A and Figure 2B The table shown determines object lighting corresponding to a specific behavior consistent with the identified behavior, attributes of other occupants, and / or other occupants and vehicle information, thereby selecting the object lighting.

[0044] In step S70, the lighting control unit 14 sets the brightness of the target lighting according to the brightness inside the vehicle cabin. In step S80, the lighting control unit 14 illuminates the selected target lighting at the set brightness. In step S90, while the target lighting is illuminated, the lighting control unit 14 determines whether the lighting extinguishing condition is satisfied based on the recognition result of the recognition unit 12 and / or the vehicle status.

[0045] The control flow of whether the extinguishing condition is met is Figure 5 . In step S91, the image acquisition unit 11 acquires a camera image, and the recognition unit 12 recognizes the occupant behavior based on the camera image. In step S92, the state information acquisition unit 13 acquires vehicle state information. In step S93, the lighting control unit 14 determines whether the specific behavior has ended based on the behavior recognized in the control process of step S91. The specific behavior is the specific behavior in the determination process of step S50, which is equivalent to the behavior recognized when the object lighting is selected. When it is determined that the specific behavior has ended, the lighting control unit 14 determines that the extinguishing condition is met (step S97).

[0046] If it is determined that the specific behavior has not ended, in step S94, the lighting control unit 14 determines whether the vehicle is started based on the vehicle state information obtained in the control process of step S92. If the vehicle is started, the lighting control unit 14 determines that the lighting off condition is satisfied (step S97).

[0047] If the vehicle is determined not to be started, in step S95, the lighting control unit 14 turns on the target lighting and determines whether the time elapsed from the time point is greater than or equal to a predetermined time. If the elapsed time is determined to be less than the predetermined time ("No" in step S95), the lighting control unit 14 determines that the lighting control condition is not satisfied (step S96). If the elapsed time is determined to be greater than or equal to the predetermined time, the lighting control unit 14 determines that the lighting control condition is satisfied (step S97).

[0048] like Figure 3 As shown, if the control flow of steps S90 and S91 to S97 determines that the extinguishing condition is not met, the controller 10 repeats the control flow of step S90. If the extinguishing condition is met, the lighting control unit 14 extinguishes the target lighting in step S100. The controller 10 then ends the control flow.

[0049] As described above, the lighting control device and lighting control method according to this embodiment capture images from camera 1, identify the behavior of at least one of a plurality of occupants, as well as the attributes and / or status of other occupants among the plurality of occupants different from the one occupant, based on the captured images, obtain vehicle state information indicating the vehicle state, select target lighting to be illuminated based on the recognition result and the vehicle state, and illuminate the selected target lighting. This allows lighting control while maintaining the comfort of not only the occupant who has performed a specific behavior but also the comfort of other occupants.

[0050] In addition, in the present embodiment, the lighting control unit 14 sets the brightness of the target lighting according to the brightness in the vehicle cabin. This allows the brightness of the target lighting to be adjusted to an appropriate illuminance according to the brightness in the vehicle cabin.

[0051] In this embodiment, the lighting control unit 14 determines whether the lighting condition is satisfied based on the recognition result of the recognition unit 12 and / or the vehicle state, and turns off the target lighting when the lighting condition is satisfied.

[0052] In addition, in the present embodiment, when the vehicle starts while the target lighting is on, the lighting control unit 14 turns off the target lighting. This can prevent the driver's driving from being adversely affected by the lighting of the target lighting.

[0053] In addition, in the present embodiment, when photography of the interior of the vehicle is permitted, the image acquisition unit 11 acquires the captured image, thereby enabling privacy protection.

[0054] In addition, in the present embodiment, if the elapsed time from the time when the target lighting was turned on is longer than a predetermined time, the lighting control unit 14 turns off the target lighting.

[0055] Furthermore, as a variation of this embodiment, while the vehicle is being driven, the lighting control unit 14 of the controller 10 illuminates the target lighting when the vehicle is being controlled by autonomous driving control, and prohibits illumination of the target lighting when the vehicle is being controlled by manual driving control. In particular, if the lighting of the driver's seat area is included in the target lighting, the lighting control unit 14 can also control the illumination and extinguishing of the target lighting as in the variation. This improves the comfort of the driver's seat occupant during autonomous driving and prevents adverse effects on the driver's driving caused by the illumination of the target lighting during manual driving.

[0056] In addition, Figures 3 to 5 The control flow shown in the figure shows an example of execution during the vehicle is in a stopped state or a parked state. However, during the vehicle is running, the controller 10 executes the control flow by omitting step S94. Figures 3 to 5 The control flow shown is sufficient.

[0057] Furthermore, in this embodiment, Figures 3 to 5 The control flow shown does not require all processes to be executed, and some processes may be omitted or the order of the processes may be changed.

[0058] Second Implementation Method

[0059] Next, the interior component control system according to the second embodiment will be described. Compared to the interior component control system according to the first embodiment, the second embodiment adds some control functions to the recognition unit 12 and the lighting control unit 13. Aside from the additional functions of the recognition unit 12 and the lighting control unit 14, the structure is identical to that of the first embodiment. Therefore, the description already described applies to the same structures as the first embodiment.

[0060] Camera 1 is positioned so that the seat is within its field of view. Therefore, when an occupant is seated, camera 1 is positioned so that its field of view encompasses a portion of the occupant's body. For example, when an occupant is seated, camera 1's field of view encompasses at least the occupant's upper body, but excludes the occupant's feet. Furthermore, to capture rear-seat occupants, camera 1 can be mounted on the vehicle roof (the interior ceiling). Alternatively, camera 1 can be configured as a wide-angle camera to capture the entire rear seat surface.

[0061] The recognition unit 12 recognizes the trajectory of the occupant's body when a part of the occupant's body is out of the field of view of the camera 1. For example, a scene in which a occupant bends down from a seated position to pick up an object at his feet (hereinafter also referred to as a "scene of picking up an object") is cited as an example. In this scene, when the occupant bends down from a seated position, the occupant's upper body moves downward relative to the seat. The recognition unit 12 acquires camera images in a time series and extracts the movement of the occupant's part (upper body) from a plurality of camera images, thereby recognizing the trajectory of the occupant's upper body. For example, in a case where the center of the field of view of the camera 1 is the center of the occupant's upper body, if it is a scene of picking up an object, the trajectory of the upper body is a line extending downward from the center point of the field of view. In this way, the recognition unit 12 recognizes the occupant's behavior when a part of the occupant's body is out of the field of view of the camera 1.

[0062] The lighting control unit 14 outputs control commands to the footlights 4 based on the body trajectory identified by the recognition unit 12. First, the lighting control unit 14 determines the direction in which the body trajectory deviates from the field of view (hereinafter referred to as the "field of view deviation direction"). Then, the lighting control unit 14 outputs control commands to turn on the footlights 4 located in the field of view deviation direction. The notification device 32, to which the control commands are sent, is located outside the field of view of the camera 1 and in an area in the field of view deviation direction. If the scene involves picking up an object, the lighting control unit 14 determines the area below the seat as the field of view deviation direction. The lighting control unit 14 outputs control commands to turn on the footlights located at the feet of the occupant who is bending over. In other words, based on the body trajectory, the lighting control unit 14 determines the footlights 4 located outside the field of view of the camera 1 and in the field of view deviation direction as the devices to which the control commands are sent. As a result, when the occupant bends over to pick up an object, the footlights are illuminated.

[0063] As an example other than the scenario of picking up an object, assume that while the vehicle is parked, the driver reaches for an object in the roof pocket. The driver's hand moves from the steering wheel toward the roof pocket. The recognition unit 12 recognizes the trajectory of the hand by extracting the movement of the hand. The lighting control unit 14 determines the direction from the steering wheel toward the roof pocket as the viewing angle deviation direction. The viewing angle deviation direction is the direction in which the hand trajectory deviates from the viewing angle. The lighting control unit 14 outputs a control instruction for turning on the lighting (interior light 8) set in the viewing angle direction. Thus, when the driver wants to take out the object in the roof pocket, the lighting set near the roof pocket is lit before the hand touches the roof pocket.

[0064] Furthermore, the recognition unit 12 recognizes that a portion of the occupant's body has returned to the camera 1's field of view after it has been removed from the camera's field of view. For example, after picking up an object, the occupant returns their upper body to its original position, repositioning themselves as if sitting in their seat. In this case, the upper body temporarily leaves the field of view to pick up the object, then returns to the field of view. The recognition unit 12 recognizes that the upper body has returned to the field of view. In other words, the recognition unit 12 recognizes that the upper body has returned to the field of view after having been removed from the field of view based on a series of movements of the upper body.

[0065] If the lighting control unit 14 recognizes that a portion of the occupant's body has returned to the viewing angle, it outputs an off command to the footlights 4 to turn off the lighting. If the occupant picks up an object at their feet and then returns to their original sitting position, the lighting control unit 14 outputs an off command to turn off the lighting of the footlights 4 if it recognizes that the occupant's upper body has returned to the viewing angle. Alternatively, the lighting control unit 14 may output an off command to the footlights 4 after a predetermined time has elapsed since the occupant's upper body was recognized to have returned to the viewing angle. This allows the lighting of the footlights 4 to be switched on and off according to the occupant's behavior.

[0066] Furthermore, the lighting control unit 14 may output a shutdown command to the notification device when a predetermined time has passed since a portion of the occupant's body was recognized to have left the field of view of the camera 1. Thus, even if a portion of the occupant's body cannot be recognized from the camera image as returning to the field of view, the illumination of the footlights 4 can be turned off.

[0067] As described above, the interior component control device or method according to this embodiment identifies the trajectory of an occupant's body when a portion of the occupant's body is out of the field of view of camera 1 and outputs control commands to the footlights 4 and / or interior lights 8 based on the identified body trajectory. This allows identification of the occupant's behavior as specific to the interior component 3, even when the interior component 3 is outside the field of view of camera 1. Furthermore, in the above embodiment, the lighting control unit 14 outputs control commands to the footlights 4 and interior lights 8. However, these commands are not limited to the footlights 4 and interior lights 8 and may also output control commands for switching the reading lights 5, cup holder lighting 6, and map lights 7 on and off. Specifically, the lighting control unit 14 may output lighting control commands to at least one of the reading lights 5, cup holder lighting 6, and map lights 7 based on the body trajectory identified by the recognition unit 12.

[0068] Description of Reference Numerals

[0069] 1: Camera; 2: On-board device; 4: Footlight; 5: Reading light; 6: Cup holder lighting; 7: Map light; 8: Interior light; 9: Lighting control device; 10: Controller; 11: Image acquisition unit; 12: Recognition unit; 13: Status information acquisition unit; 14: Lighting control unit.

Claims

1. A lighting control device for controlling lighting provided in a vehicle compartment, the lighting control device comprising: an image acquisition unit for acquiring a camera image from a camera for capturing images of the interior of the vehicle compartment; a recognition unit that recognizes, based on the captured image, a behavior of at least one occupant among a plurality of occupants, and attributes of other occupants among the plurality of occupants different from the one occupant and / or a state of the other occupants; a state information acquisition unit that acquires vehicle state information indicating a vehicle state of the vehicle; as well as A lighting control unit selects a target lighting to be turned on for the lighting based on the recognition result of the recognition unit and the vehicle state, and turns on the selected target lighting.

2. The lighting control device according to claim 1, wherein: The lighting control unit sets the brightness of the target lighting according to the brightness in the vehicle cabin.

3. The lighting control device according to claim 1 or 2, wherein: The lighting control unit determines whether a lighting condition is satisfied based on the recognition result and / or the vehicle state. The lighting control unit turns off the target lighting when the turn-off condition is satisfied.

4. The lighting control device according to any one of claims 1 to 3, wherein: The lighting control unit turns off the target lighting when the vehicle starts while the target lighting is on.

5. The lighting control device according to any one of claims 1 to 4, wherein: The image acquisition unit acquires the captured image when imaging of the interior of the vehicle compartment is permitted.

6. The lighting control device according to any one of claims 1 to 5, wherein: The lighting control unit turns off the target lighting when the elapsed time from the time point when the target lighting was turned on is equal to or longer than a predetermined time.

7. The lighting control device according to any one of claims 1 to 6, wherein: When the vehicle is controlled by autonomous driving control, the lighting control unit turns on the target lighting. The lighting control unit prohibits lighting of the target lighting when the vehicle is controlled by manual driving control.

8. A lighting control method, executed by a controller, for controlling lighting provided in a vehicle compartment, wherein: The controller performs the following processing: Acquire a camera image from a camera used to capture images inside the vehicle compartment; recognizing, based on the captured image, a behavior of at least one occupant among a plurality of occupants, and attributes of other occupants among the plurality of occupants different from the one occupant and / or a status of the other occupants; acquiring vehicle status information indicating a vehicle status of the vehicle; as well as A target lighting to be turned on of the lighting is selected based on the recognition result and the vehicle state, and the selected target lighting is turned on.

Images