Occupant monitoring device and occupant monitoring procedure

Abstract

An occupant monitoring device (20) comprising: a vehicle interior imaging unit (21) for recording an area in which at least the face of an occupant is located in a space of a vehicle, and a lighting unit (22) for emitting light into the area in which at least the face of the occupant is located in the space, wherein the vehicle interior imaging unit (21) and the lighting unit (22) are attached to a display unit (12) which is provided on an A-pillar (104) arranged in the space.

Description

Technical area

[0001] The present disclosure relates to an occupant monitoring device and an occupant monitoring method. Background technology

[0002] An electronic mirror device may be installed in a vehicle. Such an electronic mirror device is used to image the surroundings of a vehicle using an exterior imaging unit and to display the imaged images (including videos) on a display unit provided in the passenger compartment. A vehicle occupant can therefore determine the conditions in the surroundings of the vehicle through the image images displayed on the display unit.

[0003] A conventional electronic mirror device is disclosed, for example, in Patent Document 1. The display unit of the electronic mirror device disclosed in Patent Document 1 is provided on an A-pillar in the passenger compartment. Prior art documentsPatent documents

[0004] Patent Document 1: Patent Publication No. JP 2019-84988 A Overview of the inventionProblem to be solved by the inventionInformation

[0005] Recently, an occupant monitoring device that monitors the condition of occupants in a vehicle has become increasingly popular. This occupant monitoring device includes an in-vehicle imaging unit in the passenger compartment for monitoring occupants. The in-vehicle imaging unit of the occupant monitoring device is preferably arranged in a location where an occupant can be imaged with high precision and is therefore often installed in a dashboard forming the front portion of the passenger compartment.

[0006] If the occupant monitoring device is installed in a commercial vehicle such as a truck or bus, the dashboard is positioned at a lower level than a seated occupant, compared to an installation in a passenger car. The vehicle interior imaging unit therefore images an occupant from below, which poses the risk of not achieving precise imaging. If the occupant monitoring device is installed in a commercial vehicle, it is therefore conceivable to install the vehicle interior imaging unit on an A-pillar, which is positioned higher than the dashboard.

[0007] However, if the above electronic mirror device is used in a commercial vehicle to which the occupant monitoring device is to be installed, the display unit of the electronic mirror device and the vehicle interior imaging unit of the occupant monitoring device must be installed on a single A-pillar. Therefore, it is necessary to provide an installation space for the display unit and an installation space for the vehicle interior imaging unit, respectively.

[0008] The present disclosure has been made to solve the above problem and intends to provide an occupant monitoring device in which the vehicle interior imaging unit can be provided at an A-pillar while securing the installation space for the display unit at a position where precise imaging is possible. Means of solving the task

[0009] The occupant monitoring device according to the present disclosure includes a vehicle interior imaging unit for capturing an area where at least the face of an occupant is located in a compartment of a vehicle, and an illumination unit for irradiating light into the area where at least the face of the occupant is located in the compartment, wherein the vehicle interior imaging unit and the illumination unit are mounted on a display unit provided on an A-pillar arranged in the compartment. Effects of the invention

[0010] According to the present disclosure, a vehicle interior imaging unit can be provided at an A-pillar while securing the installation space for the display unit at a position where precise imaging is possible. Brief explanation of the drawings [ Fig. 1] Fig. 1 is a schematic perspective view showing the state in the passenger compartment of a vehicle in which an occupant monitoring apparatus according to a first embodiment is used. [ Fig. 2] Fig. 2 is a block diagram showing the structure of the occupant monitoring apparatus according to the first embodiment. [ Fig. 3] Fig. 3 is a view showing examples of attaching a vehicle interior imaging unit and lighting units to a display unit. Fig. 3A is a view when attaching the vehicle interior imaging unit and lighting units to an upper peripheral edge of a frame portion. Fig. 3B is a view when attaching the vehicle interior imaging unit and the lighting units to a lower peripheral edge of the frame portion. Fig. 3C is a view when attaching the vehicle interior imaging unit and the lighting units to the upper edge and the lower edge of the frame portion, respectively. Fig. 3D is a view of attaching a mounting plate, which integrates the vehicle interior imaging unit and lighting units, to the frame section. [ Fig. 4] Fig. 4 is a flowchart showing an occupant monitoring method according to the first embodiment. [ Fig. 5] Fig. 5A and Fig. 5B are views showing examples of the hardware structure of a monitoring control apparatus according to the first embodiment. Forms for carrying out the invention

[0011] In order to explain the present disclosure in more detail, a form for carrying out the present disclosure will be explained below with reference to the accompanying drawings. First embodiment

[0012] Based on Fig. 1 to Fig. 5, an occupant monitoring device 20 according to the first embodiment is explained.

[0013] First, based on Fig. 1 and Fig. 2 the structure of the occupant monitoring device 20 is explained. Fig. 1 is a schematic perspective view showing the state in the passenger compartment of a vehicle 100 in which the occupant monitoring device 20 according to the first embodiment is used. Fig. 2 is a block diagram showing the structure of the occupant monitoring device 20 according to the first embodiment.

[0014] As in Fig. 1, on the front of the passenger compartment of the vehicle 100, for example, a driver's seat 101, a windshield 102, a side door 103, an A-pillar 104 and a dashboard 105 are provided.

[0015] The vehicle 100 is assumed to be a commercial vehicle such as a truck or a bus. The side door 103 is a right side door corresponding to the driver's seat 101. The side door 103 has a door glass 103a. The A-pillar 104 is arranged between the windshield 102 and the side door 103. The A-pillar 104 is formed to extend in the vertical direction of the vehicle 100. The instrument panel 105 is arranged in front of the driver's seat 101 and below the windshield 102. The instrument panel 105 is a structural body extending in the vehicle width direction.

[0016] The vehicle 100 is further equipped with an electronic mirror device 10 and the occupant monitoring device 20.

[0017] As in Fig. 2, the electronic mirror device 10 has a vehicle exterior imaging unit 11 and a display unit 12.

[0018] The vehicle exterior imaging unit 11 is provided on the outside of the vehicle 100. The vehicle exterior imaging unit 11 images an area including at least an area behind the vehicle 100. For example, the vehicle exterior imaging unit 11 captures a rearward range from the left side of the vehicle 100 when the vehicle exterior imaging unit 11 is provided on the left side of the vehicle 100. Furthermore, the vehicle exterior imaging unit 11 transmits a rearward range from the right side of the vehicle 100 when the vehicle exterior imaging unit 11 is provided on the right side of the vehicle 100. The vehicle exterior imaging unit 11 further outputs captured images (including videos) to the display unit 12. The vehicle exterior imaging unit 11 is, for example, a camera.

[0019] The display unit 12 acquires image images output from the vehicle exterior imaging unit 11. Furthermore, the display unit 12 displays the acquired image images. The display unit 12 is provided on the A-pillar 104. Therefore, the display unit 12 is installed in a location where it can be easily visually perceived by the passenger (driver) sitting on the driver's seat 101. Thus, the display unit 12 is used instead of a conventional side mirror or rearview mirror and is useful when pulling over and parking the vehicle 100.

[0020] The display unit 12 further includes a display surface 12a and a frame portion 12b. The display surface 12a is used to display a captured image. The frame portion 12b is formed in a rectangular bezel shape and arranged to surround the entire outer peripheral end of the display surface 12a. In addition to image images displayed by the vehicle exterior imaging unit 11, the display content displayed on the display surface 12a may also be vehicle information related to the vehicle 100.

[0021] As in Fig. 2, the occupant monitoring device 20 includes a vehicle interior imaging unit 21, lighting units 22, and a monitoring control device 20A.

[0022] As in Fig. As shown in Figure 1, a vehicle interior imaging unit 21 and two lighting units 22 are integrally mounted with respect to the display unit 12 provided on the A-pillar. Quantitatively, there may be one or a plurality of lighting units 22.

[0023] The vehicle interior imaging unit 21 is arranged from the display unit 12 toward the driver's seat 101. The vehicle exterior imaging unit 21 images an area in which at least the face of an occupant sitting in the driver's seat 101 is located. The vehicle interior imaging unit 21 further outputs imaged captured images (including videos) to a gaze direction detection unit 24.

[0024] The lighting units 22 are arranged from the display unit 12 toward the driver's seat 101. The lighting units 22 emit light into the area in which at least the face of the occupant sitting on the driver's seat 101 is located.

[0025] That is, in the occupant monitoring device 20, light is emitted by the illumination units 22 toward the occupant sitting in the driver's seat 101, and an image is generated by the vehicle interior imaging unit 21. The vehicle interior imaging unit 21 is, for example, a camera. The illumination units 22 are, for example, LEDs (light-emitting diodes).

[0026] As in Fig. 2, the vehicle interior imaging unit 21 and the lighting units 22 are electrically connected to the display unit 12. The operation of the vehicle interior imaging unit 21 and the lighting units 22 has already begun when an image is displayed on the display surface 12a of the display unit 12.

[0027] The monitoring control device 20A includes an acquisition unit 23, the gaze direction detection unit 24, a gaze direction determination unit 25, and a warning control unit 26.

[0028] The acquisition unit 23 acquires the image images output by the vehicle interior imaging unit 21. Furthermore, the acquisition unit 23 outputs an acquired image image to the gaze direction detection unit 24.

[0029] The gaze direction detection unit 24 acquires the image output from the acquisition unit 23. Based on the acquired image, the gaze direction detection unit 24 detects the direction of the gaze of the occupant sitting in the driver's seat 101. Furthermore, the gaze direction detection unit 24 outputs the direction of the detected gaze of the occupant to the gaze direction determination unit 25.

[0030] The gaze direction determination unit 25 acquires the direction of the occupant's gaze output from the gaze direction detection unit 24. Furthermore, the gaze direction determination unit 25 determines whether the acquired direction of the occupant's gaze is directed toward the display surface 12a of the display unit 12. If the occupant's gaze meets predetermined conditions, the gaze direction determination unit 25 determines that the occupant's gaze is directed toward the display surface 12a. Predetermined conditions include, for example, whether the time for the occupant to visually perceive the display surface 12a is greater than or equal to a certain time, or whether the frequency with which the occupant to visually perceive the display surface 12a is greater than or equal to a certain frequency.

[0031] The detection of the time at which the occupant visually observes the display surface 12a and the frequency with which the occupant visually observes the display surface 12a are performed based on image images acquired by the gaze direction detection unit 24. The gaze direction detection unit 24 outputs these detections to the gaze direction determination unit 25.

[0032] For example, if the line of sight detection unit 25 determines that the direction of the occupant's line of sight is toward the display surface 12a, an operation signal for operating the warning control unit 26 is also output to the warning control unit 26. On the other hand, if the line of sight detection unit 25 determines that the direction of the occupant's line of sight is not toward the display surface 12a, a stop signal for stopping the warning control unit 26 is output to the warning control unit 26.

[0033] The warning control unit 26 receives an operation signal or stop signal output from the viewing direction detection unit 25. Upon receiving the operation signal, the warning control unit 26 issues a warning to the occupant indicating that the occupant is not paying attention to the images displayed on the display surface 12a. Specifically, the warning control unit 26 activates at least one of the lighting units 22 and a sound output unit 31. The warning control unit 26 causes the lighting units 22 to flash at predetermined intervals. Furthermore, the warning control unit 26 causes the sound output unit 31 to generate sounds directed at the occupant. This causes the occupant to view image information displayed on the display surface 12a.

[0034] On the other hand, if it has received the stop signal, the warning control unit 26 does not issue a warning to the occupant. That is, the warning control unit 26 interrupts or stops the operation of the lighting units 22 and the sound output unit 31. The sound output unit 31 is, for example, a loudspeaker provided in the passenger compartment.

[0035] Next, examples of attaching the vehicle interior imaging unit 21 and the lighting units 22 to the display unit 12 will be explained with reference to Fig. 3 explained. Fig. 3 are views showing examples of attaching the vehicle interior imaging unit 21 and the lighting units 22 to the display unit 12.

[0036] As in Fig. As shown in Figure 3A, the vehicle interior imaging unit 21 and the illumination units 22 are provided in an upper peripheral edge of the frame portion 12b of the display unit 12. The vehicle interior imaging unit 21 is arranged between two illumination units 22.

[0037] As in Fig. As shown in Figure 3B, the vehicle interior imaging unit 21 and the illumination units 22 are provided in a lower peripheral edge of the frame portion 12b of the display unit 12. The vehicle interior imaging unit 21 is arranged between two illumination units 22.

[0038] As in Fig. As shown in Figure 3C, the vehicle interior imaging unit 21 is provided in the upper peripheral edge of the frame portion 12b of the display unit 12. The illumination units 22 are provided in the lower peripheral edge of the frame portion 12b.

[0039] As in Fig. 3A to 3B, when the vehicle interior imaging unit 21 and the illumination units 22 are provided in the frame portion 12b of the display unit 12, the vehicle interior imaging unit 21 and the illumination units 22 may be provided at a peripheral edge of either the upper peripheral edge, lower peripheral edge, left peripheral edge, or right peripheral edge of the frame portion 12b.

[0040] As in Fig. 3D, the vehicle interior imaging unit 21 and the illumination units 22 are mounted on a mounting plate 27. The length of the mounting plate 27 is substantially equal to the length of the upper peripheral edge of the frame portion 12b of the display unit 12. The vehicle interior imaging unit 21 is arranged between two illumination units 22 in the length direction of the mounting plate 27. Therefore, the mounting plate 27, in which the vehicle interior imaging unit 21 and the illumination units 22 are integrally formed, can be mounted to extend toward the display unit 12 along the upper peripheral edge of the frame portion 12b. The mounting plate 27, in which the vehicle interior imaging unit 21 and the illumination units 22 are integrally formed, can also be mounted to extend toward the display unit 12 along the lower peripheral edge, the left peripheral edge, or the right peripheral edge of the frame portion 12b.

[0041] Next, Fig. 4 explains an occupant monitoring method according to a first embodiment. Fig. 4 is a flowchart showing the occupant monitoring method according to the first embodiment.

[0042] In step ST11, the lighting unit 22 radiates light into an area in which at least the face of an occupant sitting on the driver's seat 101 is located.

[0043] In step ST12, the vehicle interior imaging unit 21 captures an area in which at least the face of the occupant sitting on the driver's seat 101 is located.

[0044] In step ST13, the acquisition section 23 acquires the image image imaged by the vehicle interior imaging unit 21.

[0045] In step ST14, the line of sight detecting unit 24 detects the direction of the line of sight of the occupant based on the image image displayed by the vehicle interior imaging unit 21.

[0046] In step ST15, the gaze direction determination unit 25 determines whether the direction of the occupant's gaze detected by the gaze direction detection unit 24 is directed toward the display unit 12. If the gaze direction determination unit 25 determines that the occupant's gaze direction is directed toward the display unit 12, the occupant monitoring process returns to step ST14. If the gaze direction determination unit 25 further determines that the occupant's gaze direction is not directed toward the display unit 12, the occupant monitoring process proceeds to step ST16.

[0047] In step ST16, the warning control unit 26 controls at least one of the lighting units 22 and the sound output unit 31 based on the determination result of the line of sight determination unit 25. Then, the occupant monitoring process is ended.

[0048] Next, examples of the hardware structure of the monitoring control device 20A will be explained with reference to Fig. 5 explained. Fig. 5 are views showing examples of the hardware structure of the monitoring control device 20A according to the first embodiment.

[0049] As in Fig. As shown in Figure 5A, the monitoring control device 20A is comprised of a computer, including a processor 51 and a memory 52. ​​Memory 52 stores programs that enable the computer to function as an acquisition unit 23, a gaze direction detection unit 24, a gaze direction determination unit 25, and a warning control unit 26. By reading and executing the programs stored in memory 52, the processor 51 realizes the functions of the acquisition unit 23, the gaze direction detection unit 24, the gaze direction determination unit 25, and the warning control unit 26.

[0050] As in Fig. 5B, the monitoring control device 20A may also include a processing circuit 53. In this case, it may also be implemented by the acquisition unit 23, the gaze direction detection unit 24, the gaze direction determination unit 25, and the warning control unit 26.

[0051] The monitoring control device 20A may further include the processor 51, the memory 52, and the processing circuit 53 (not shown). In this case, some of the functions of the acquisition unit 23, the gaze direction detection unit 24, the gaze direction determination unit 25, and the warning control unit 26 may be realized by the processor gate 51 and the memory 52, and the remaining functions may be realized by the processing circuit 53.

[0052] For example, the processor 51 uses at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcomputer, and a DSP (Digital Signal Processor).

[0053] The memory 52 uses at least one of a semiconductor memory and a magnetic disk. More specifically, the memory 52 uses at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), an SSD (Solid State Drive), or an HDD (Hard Disk Drive).

[0054] The processing circuit 53 uses, for example, at least one of ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field Programmable Gate Array), SoC (System-on-a-Chip), or System-LSI (Large Scale Integration).

[0055] The above-mentioned passenger monitoring device 20 according to the first embodiment includes the vehicle interior imaging unit 21 that captures an area where at least the face of an occupant is located in the passenger compartment of the vehicle 100, and illumination units 22 that irradiate light into the area where at least the face of the occupant is located in the passenger compartment. The vehicle interior imaging unit 21 and the illumination units 22 are mounted on a display unit 12 provided on an A-pillar 104 located in the passenger compartment. Therefore, in the passenger monitoring device 20, the vehicle interior imaging unit 21 can be provided at an A-pillar, while securing the installation space for the display unit 12 at a location where precise imaging is possible.

[0056] In the passenger monitoring device 20, the vehicle interior imaging unit 21 and the lighting units 22 are mounted on the frame portion 12b of the display unit 12. Therefore, space saving for the vehicle interior imaging unit 21 and the lighting units 22 can be achieved.

[0057] The occupant monitoring device 20 includes the mounting plate 27 on which the vehicle interior imaging unit 21 and the lighting units 22 are provided. The mounting plate 27 is attached to the frame portion 12b in a state in which the vehicle interior imaging unit 21 and the lighting units 22 are integrally formed. The occupant monitoring device 20 can improve the operability of mounting the vehicle interior imaging unit 21 and the lighting units 22.

[0058] The occupant monitoring device 20 includes the acquisition unit 23 that acquires image images imaged by the vehicle interior imaging unit 21, the gaze direction detection unit 24 that detects the direction of the occupant's gaze based on the image images acquired by the acquisition unit 23, the gaze direction determination unit 25 that determines whether the direction of the occupant's gaze detected by the gaze direction detection unit 24 is toward the display unit 12, and the warning control unit 26 that controls at least one of the lighting units 22 and the sound output unit 31 based on the determination result of the gaze direction determination unit 25. Thus, the occupant monitoring device 20 can warn the occupant that the occupant is not paying attention to the image images displayed on the display surface 12a.

[0059] In the occupant monitoring device 20, the gaze direction determination unit 25 determines that the direction of the occupant's gaze is directed toward the display unit 12 when the occupant's gaze direction detected by the gaze direction detection unit 24 meets predetermined conditions. Therefore, in the occupant monitoring device 20, an appropriate determination can be performed by the gaze direction determination unit 25.

[0060] If the gaze direction detection unit 25 of the occupant monitoring device 20 determines that the occupant's gaze direction is directed toward the display unit 12, the warning control unit 26 causes the lighting units 22 to flash. This allows the occupant monitoring device 20 to reliably warn the occupant.

[0061] If the gaze direction detection unit 25 of the occupant monitoring device 20 determines that the occupant's gaze direction is directed toward the display unit 12, the warning control unit 26 causes the sound output unit 31 to generate sounds. This allows the occupant monitoring device 20 to reliably warn the occupant.

[0062] In the present disclosure, any change to the components of the embodiment or any omission of any components of the embodiment is possible within the scope of this disclosure.

[0063] In the following, various aspects of the present disclosure are summarized as additions. (Addition 1)

[0064] Occupant monitoring device, characterized by a vehicle interior imaging unit that captures an area in which at least the face of an occupant is located in the passenger compartment of a vehicle, and a lighting unit that emits light into the area in which at least the face of the occupant is located in the passenger compartment, wherein the vehicle interior imaging unit and the lighting unit are mounted on a display unit provided on an A-pillar arranged in the passenger compartment. (Addition 2)

[0065] The occupant monitoring device of claim 1, characterized in that the vehicle interior imaging unit and the illumination unit are mounted on a frame portion of the display unit. (Addition 3)

[0066] Occupant monitoring device of Addition 1, characterized by a mounting plate on which the vehicle interior imaging unit and the lighting unit are provided, wherein the mounting plate is attached to the frame portion of the display unit in a state in which the vehicle interior imaging unit and the lighting unit are integrally formed. (Addition 4)

[0067] Occupant monitoring device of any one of the additions 1 to 3, characterized by an acquisition unit that acquires image images imaged by the vehicle interior imaging unit, a gaze direction detection unit that detects the direction of the occupant's gaze based on the images acquired by the acquisition unit, a gaze direction detection unit that determines whether the direction of the occupant's gaze detected by the gaze direction detection unit is directed toward the display unit, and a warning control unit that controls at least one of the lighting unit and the sound output unit based on the determination result of the line of sight determination unit. (Addition 5)

[0068] Occupant monitoring device of Addition 4, characterized in that the gaze direction detection unit determines that the direction of the occupant's line of sight is directed toward the display surface if the occupant's line of sight detected by the line of sight detection unit satisfies specified conditions. (Addition 6)

[0069] Occupant monitoring device of Addition 4 or Addition 5, characterized in that the warning control unit ensures that the lighting unit flashes when the gaze direction detection unit has detected that the direction of the occupant's gaze is directed towards the display unit. (Addition 7)

[0070] Occupant monitoring device of one of the additions 4 to 6, characterized in that the warning control unit ensures that sounds are generated by the sound output unit when the gaze direction detection unit has detected that the direction of the occupant's gaze is directed towards the display unit. (Addition 8)

[0071] Occupant monitoring method, characterized in that in an occupant monitoring device comprising a vehicle interior imaging unit that captures an area in which at least the face of an occupant is located in the passenger compartment of a vehicle, and a lighting unit that emits light into the area in which at least the face of the occupant is located in the passenger compartment, wherein the vehicle interior imaging unit and the lighting unit are mounted on a display unit provided on an A-pillar arranged in the passenger compartment, an acquisition unit acquires image images displayed by the vehicle interior imaging unit, a gaze direction detection unit uses the images acquired by the acquisition unit to detect the direction of the occupant's gaze, a gaze direction detection unit determines whether the direction of the occupant's gaze detected by the gaze direction detection unit is directed towards the display unit, and a warning control unit controls at least one of the lighting unit and the sound output unit based on the determination result of the line of sight determination unit. Industrial application

[0072] In the occupant monitoring device of the present disclosure, by attaching a vehicle interior imaging unit and a lighting unit to a display unit provided at an A-pillar, the vehicle interior imaging unit can be provided at a location where precise imaging is possible while securing the installation space for the display unit, thereby being suitable for use in an occupant monitoring device. Explanation of reference symbols 10 Electronic mirror device 11 Vehicle exterior imaging unit 12 Display unit 12a Display area 12b frame section 20 Occupant monitoring device 20A monitoring control device 21 Vehicle interior imaging unit 22 lighting unit 23 Acquisition Unit 24 Gaze direction detection unit 25 Gaze direction detection unit 26 Warning control unit 27 Mounting plate 31 Sound output unit 51 processor 52 storage 53 Processing circuit 100 vehicles 101 Driver's seat 102 Windshield 103 Front door 103a door pane 104 A-pillar 105 Dashboard QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2019-84988 A

[0004]

Claims

[1] An occupant monitoring device comprising: a vehicle interior imaging unit for capturing an area in which at least the face of an occupant is located in a space of a vehicle, and a lighting unit for radiating light into the area where at least the face of the occupant is located in the room, wherein the vehicle interior imaging unit and the lighting unit are mounted on a display unit provided on an A-pillar arranged in the space. [2] The occupant monitoring device according to claim 1, wherein the vehicle interior imaging unit and the illumination unit are mounted on a frame portion of the display unit. [3] The occupant monitoring device according to claim 1, comprising a mounting plate on which the vehicle interior imaging unit and the lighting unit are provided, wherein the mounting plate is attached to the frame portion of the display unit in a state in which the vehicle interior imaging unit and the lighting unit are integrally formed. [4] The occupant monitoring device according to any one of claims 1 to 3, comprising an acquisition unit for acquiring the image images imaged by the vehicle interior imaging unit, a gaze direction detection unit for detecting, based on the image images acquired by the acquisition unit, a gaze direction detection unit for determining whether the gaze direction of the occupant detected by the gaze direction detection unit is directed towards the display unit, and a warning control unit for controlling, based on the determination result of the line of sight determination unit, at least one of the lighting unit and a sound output unit. [5] The occupant monitoring device according to claim 4, wherein the gaze direction detection unit determines that the direction of the occupant's line of sight is directed toward the display unit when the occupant's line of sight detected by the line of sight detection unit satisfies a predetermined condition. [6] The occupant monitoring device according to claim 4, wherein the warning control unit causes the lighting unit to flash when it is determined by the line of sight determination unit that the line of sight of the occupant is directed toward the display unit. [7] The occupant monitoring device according to claim 4, wherein the warning control unit causes the sound output unit to generate sounds when it is determined by the line of sight determination unit that the line of sight of the occupant is directed toward the display unit. [8] An occupant monitoring method for an occupant monitoring device comprising a vehicle interior imaging unit for capturing an area in which at least the face of an occupant is located in a space of a vehicle, and a lighting unit for radiating light into the area where at least the face of the occupant is located in the room, wherein the vehicle interior imaging unit and the lighting unit are mounted on a display unit provided on an A-pillar arranged in the space, and the method comprising: Capturing, by an acquisition unit, image images captured by the vehicle interior imaging unit, Proving, by a gaze direction detection unit, based on the images captured by the acquisition unit, a gaze direction of the occupant, Determining, by a gaze direction detection unit, whether the gaze direction of the occupant detected by the gaze direction detection unit is directed towards the display unit, and Controlling, by a warning control unit, based on a detection result of the line of sight detection unit, at least one of the lighting unit and a sound output unit.

Images