Passenger monitoring device and passenger monitoring method

Abstract

An occupant monitoring device (20) comprises: a vehicle-interior imaging unit (21) that images a range in which there exists at least the face of an occupant in the interior of a vehicle (100); and an illumination unit (22) that projects light into the range in which there exists at least the face of the occupant in the vehicle interior. The vehicle-interior imaging unit (21) and the illumination unit (22) are attached to a display unit (12) provided to an A pillar (104) disposed in the vehicle interior.

Description

[Technical field]

[0001] The present disclosure relates to an occupant monitoring device and an occupant monitoring method. [Background technology]

[0002] Some vehicles are equipped with an electronic mirror device. This electronic mirror device captures the surroundings of the vehicle using an exterior imaging unit, and displays the captured image (including video) on a display unit provided inside the vehicle. This allows the vehicle occupants to check the surroundings of the vehicle through the captured image displayed on the display unit.

[0003] A conventional digital mirror device is disclosed, for example, in Patent Document 1. The display unit of the digital mirror device disclosed in Patent Document 1 is provided on an A-pillar inside the vehicle cabin. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-84988 A Summary of the Invention [Problem to be solved by the invention]

[0005] information In recent years, passenger monitoring devices that monitor the status of passengers in a vehicle have become widespread. These passenger monitoring devices are equipped with an in-vehicle image capturing unit for monitoring passengers inside the vehicle cabin. The in-vehicle image capturing unit of the passenger monitoring device is preferably disposed in a position where it can capture an image of the passenger with high accuracy, and is therefore often disposed in the instrument panel that constitutes the front part of the vehicle cabin.

[0006] When the passenger monitoring device is installed in a commercial vehicle such as a truck or a bus, the instrument panel is disposed at a lower position than the seated passengers, as compared to when the device is installed in a passenger vehicle. Therefore, the in-vehicle image capturing unit captures images of the passengers from below, which may result in poor image capture. Therefore, when the passenger monitoring device is installed in a commercial vehicle, it is considered to install the in-vehicle image capturing unit in the A-pillar, which is disposed at a higher position than the instrument panel.

[0007] However, when the digital mirror device is applied to a commercial vehicle in which a passenger monitoring device is to be installed, it is necessary to provide a display unit of the digital mirror device and an in-vehicle image capturing unit of the passenger monitoring device on one A-pillar, which requires securing an installation space for both the display unit and the in-vehicle image capturing unit.

[0008] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an occupant monitoring device that can provide an in-vehicle imaging unit in a position on the A-pillar that allows for accurate imaging while ensuring installation space for a display unit. [Means for solving the problem]

[0009] The passenger monitoring device according to the present disclosure is configured to detect an area in the passenger compartment of a vehicle where at least the face of the passenger is present. Imaging and a lighting unit that irradiates light onto an area in the vehicle interior where at least the face of the occupant is present. and a mounting plate on which the in-vehicle imaging unit and the lighting unit are provided, the mounting plate being configured to: A display unit installed on the A-pillar inside the vehicle One edge of the frame in Attached to The length of the mounting plate and the length of the edge are the same. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide an interior imaging unit on an A-pillar at a position that allows imaging with high accuracy while ensuring an installation space for a display unit. [Brief description of the drawings]

[0011] [Figure 1] 1 is a schematic perspective view showing the interior of a vehicle cabin to which an occupant monitoring device according to a first embodiment is applied; [Diagram 2] 1 is a block diagram showing a configuration of an occupant monitoring device according to a first embodiment. [Diagram 3] Fig. 3A is a diagram showing an example of mounting an interior image capturing unit and an illumination unit to a display unit. Fig. 3A is a diagram showing an interior image capturing unit and an illumination unit mounted to an upper edge of a picture frame. Fig. 3B is a diagram showing an interior image capturing unit and an illumination unit mounted to a lower edge of a picture frame. Fig. 3C is a diagram showing an interior image capturing unit and an illumination unit mounted to the upper and lower edges of a picture frame. Fig. 3D is a diagram showing an attachment plate integrated with an interior image capturing unit and an illumination unit mounted to a picture frame. [Figure 4] 4 is a flowchart showing an occupant monitoring method according to the first embodiment. [Diagram 5] 5A and 5B are diagrams illustrating an example of a hardware configuration of the monitoring and controlling device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0013] Embodiment 1 An occupant monitoring device 20 according to the first embodiment will be described with reference to FIGS. 1 to 5. FIG.

[0014] First, the configuration of the occupant monitoring device 20 will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a schematic perspective view showing the interior of a vehicle cabin of a vehicle 100 to which the occupant monitoring device 20 according to the first embodiment is applied. Fig. 2 is a block diagram showing the configuration of the occupant monitoring device 20 according to the first embodiment.

[0015] As shown in FIG. 1, at the front side of the interior of a vehicle 100, a driver's seat 101, a windshield 102, a side door 103, an A-pillar 104, and an instrument panel 105 are provided, for example.

[0016] The vehicle 100 is assumed to be a commercial vehicle such as a truck or a bus. The side door 103 is a right side door corresponding to the driver's seat 101. The side door 103 has a door glass 103a. The A-pillar 104 is disposed between the windshield 102 and the side door 103. The A-pillar 104 is formed so as to extend in the up-down direction of the vehicle 100. The instrument panel 105 is disposed in front of the driver's seat 101 and below the windshield 102. The instrument panel 105 is a structure extending in the vehicle width direction.

[0017] The vehicle 100 is also equipped with an electronic mirror device 10 and an occupant monitoring device 20.

[0018] As shown in FIG. 2, the digital mirror device 10 includes an outside-of-vehicle imaging unit 11 and a display unit 12.

[0019] The outside-vehicle imaging unit 11 is provided outside the vehicle 100. The outside-vehicle imaging unit 11 captures an image of at least an area including the rear of the vehicle 100. For example, when the outside-vehicle imaging unit 11 is provided on the left side of the vehicle 100, the outside-vehicle imaging unit 11 captures an image of the area from the left side of the vehicle 100 to the rear. Imaging In addition, when the vehicle exterior image capturing unit 11 is provided on the right side of the vehicle 100, the vehicle exterior image capturing unit 11 captures the range from the right side of the vehicle 100 to the rear. Imaging In addition, the vehicle exterior image capturing unit 11 captures the image. Imaging The image (including video) is output to the display unit 12. The vehicle outside image capturing unit 11 is, for example, a camera.

[0020] The display unit 12 acquires the captured image output from the vehicle exterior imaging unit 11. The display unit 12 also displays the acquired captured image. The display unit 12 is provided on the A-pillar 104. Therefore, the display unit 12 is provided at a position where it can be easily viewed by a passenger (driver) seated in the driver's seat 101. As a result, the display unit 12 is used in place of a conventional side mirror or rear-view mirror, and is effective when the vehicle 100 is pulled over or backed into parking space.

[0021] The display unit 12 has a display surface 12a and a frame portion 12b. The display surface 12a is Imaging The frame portion 12b is formed in a rectangular frame shape and is arranged so as to surround the entire outer periphery of the display surface 12a. The display content displayed on the display surface 12a is not limited to the captured image captured by the outside-of-vehicle image capturing unit 11, and may be any vehicle information related to the vehicle 100.

[0022] As shown in FIG. 2, the occupant monitoring device 20 includes an in-vehicle imaging unit 21, a lighting unit 22, and a monitoring control device 20A.

[0023] 1, one in-vehicle imaging unit 21 and two lighting units 22 are integrally attached to a display unit 12 provided on an A-pillar 104. The number of lighting units 22 may be one or more.

[0024] The interior image capturing unit 21 is disposed so as to face the driver's seat 101 from the display unit 12. The interior image capturing unit 21 captures at least an image of an area including the face of an occupant seated in the driver's seat 101. The interior image capturing unit 21 also captures the captured image. Imaging The image (including video) is output to the line of sight detection unit 24.

[0025] The lighting unit 22 is disposed so as to face the driver's seat 101 from the display unit 12. The lighting unit 22 irradiates light at least to an area in which the face of an occupant seated in the driver's seat 101 is present.

[0026] That is, the occupant monitoring device 20 irradiates light to an occupant seated in the driver's seat 101 using the lighting unit 22, and captures an image through the interior imaging unit 21. The interior imaging unit 21 is, for example, a camera. The lighting unit 22 is, for example, an LED (Light Emitting Diode).

[0027] 2, the interior imaging unit 21 and the illumination unit 22 are electrically connected to the display unit 12. When an image is displayed on the display surface 12a of the display unit 12, the interior imaging unit 21 and the illumination unit 22 have already started their operations.

[0028] The monitoring and controlling device 20A includes an acquisition unit 23, a line-of-sight detection unit 24, a line-of-sight determination unit 25, and a warning control unit .

[0029] The acquisition unit 23 acquires the captured image output from the interior imaging unit 21. The acquisition unit 23 also outputs the acquired captured image to the line of sight detection unit 24.

[0030] The gaze detection unit 24 acquires the captured image output from the acquisition unit 23. Moreover, the gaze detection unit 24 detects the gaze direction of the occupant sitting in the driver's seat 101 from the acquired captured image. Furthermore, the gaze detection unit 24 outputs the detected gaze direction of the occupant to the gaze determination unit 25.

[0031] The line-of-sight determination unit 25 acquires the direction of the occupant's line of sight output from the line-of-sight detection unit 24. The line-of-sight determination unit 25 also determines whether or not the acquired direction of the occupant's line of sight is directed toward the display surface 12a of the display unit 12. At this time, when the occupant's line of sight satisfies a preset condition, the line-of-sight determination unit 25 determines that the direction of the occupant's line of sight is directed toward the display surface 12a. The preset condition is, for example, when the occupant views the display surface 12a for a predetermined period of time or more, and when the occupant views the display surface 12a a predetermined number of times or more.

[0032] The time during which the occupant views the display surface 12a and the frequency with which the occupant views the display surface 12a are detected by the above-mentioned line-of-sight detection unit 24 based on the captured image. The line-of-sight detection unit 24 outputs the detected information to the line-of-sight determination unit 25.

[0033] Furthermore, for example, the line-of-sight determination unit 25 determines whether the direction of the passenger's line of sight is aligned with the display surface 12a. Not suitableOn the other hand, when the line of sight determination unit 25 determines that the direction of the line of sight of the occupant is aligned with the display surface 12a, the line of sight determination unit 25 outputs an operation signal for operating the warning control unit 26 to the warning control unit 26. Suitable If so, the control unit 26 outputs a stop signal to the warning control unit 26 to stop the warning control unit 26 .

[0034] The warning control unit 26 receives the operation signal or the stop signal output from the line of sight determination unit 25. Furthermore, when the warning control unit 26 receives the operation signal, it issues a warning to the occupant indicating that the occupant is not gazing at the captured image displayed on the display surface 12a. Specifically, the warning control unit 26 operates at least one of the illumination unit 22 and the audio output unit 31. The warning control unit 26 blinks the illumination unit 22 at a predetermined interval that is set in advance. Furthermore, the warning control unit 26 generates audio from the audio output unit 31 toward the occupant. As a result, the occupant begins to look at the captured image information displayed on the display surface 12a.

[0035] On the other hand, when the warning control unit 26 receives the stop signal, it does not issue a warning to the occupants. That is, the warning control unit 26 stops or halts the operation of the lighting unit 22 and the audio output unit 31. The audio output unit 31 is, for example, a speaker provided in the vehicle cabin.

[0036] Next, an example of how the interior image capturing unit 21 and the illumination unit 22 are attached to the display unit 12 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of how the interior image capturing unit 21 and the illumination unit 22 are attached to the display unit 12.

[0037] 3A, the interior imaging unit 21 and the illumination unit 22 are provided on the upper edge of the frame portion 12b of the display unit 12. The interior imaging unit 21 is disposed between the two illumination units 22.

[0038] 3B, the interior imaging unit 21 and the illumination unit 22 are provided on the lower edge of the frame portion 12b of the display unit 12. The interior imaging unit 21 is disposed between the two illumination units 22.

[0039] 3C, the interior imaging unit 21 is provided on the upper edge of the frame portion 12b of the display unit 12. The illumination unit 22 is provided on the lower edge of the frame portion 12b.

[0040] In addition, as shown in Figures 3A and 3B, when the interior imaging unit 21 and the lighting unit 22 are provided in the frame portion 12b of the display unit 12, the interior imaging unit 21 and the lighting unit 22 may be provided on any one of the upper edge, lower edge, left edge, and right edge of the frame portion 12b.

[0041] As shown in FIG. 3D, the interior imaging unit 21 and the illumination unit 22 are attached to a mounting plate 27. The length of the mounting plate 27 is substantially the same as the length of the upper edge of the frame portion 12b of the display unit 12. The interior imaging unit 21 is disposed between the two illumination units 22 in the length direction of the mounting plate 27. In this manner, the mounting plate 27 integrating the interior imaging unit 21 and the illumination unit 22 is attached to the display unit 12 so as to be aligned along the upper edge of the frame portion 12b. Note that the mounting plate 27 integrating the interior imaging unit 21 and the illumination unit 22 may be provided to be aligned along the lower edge, left edge, or right edge of the frame portion 12b of the display unit 12.

[0042] Next, the occupant monitoring method according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the occupant monitoring method according to the first embodiment.

[0043] In step ST11, the illumination unit 22 irradiates light onto an area in which at least the face of the occupant seated in the driver's seat 101 is present.

[0044] In step ST12, the vehicle interior image capturing unit 21 captures an area in which at least the face of the occupant seated in the driver's seat 101 is present. Imaging do.

[0045] In step ST13, the acquisition unit 23 acquires the captured image captured by the interior imaging unit 21.

[0046] In step ST14, the gaze detection unit 24 detects the direction of the gaze of the occupant from the captured image captured by the interior image capture unit 21.

[0047] In step ST15, the line-of-sight determination unit 25 determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit 24 is directed toward the display unit 12. If the line-of-sight determination unit 25 determines that the direction of the line of sight of the occupant is directed toward the display unit 12, the occupant monitoring method returns to step ST14. If the line-of-sight determination unit 25 determines that the direction of the line of sight of the occupant is not directed toward the display unit 12, the occupant monitoring method proceeds to step ST16.

[0048] In step ST16, the warning control unit 26 controls at least one of the illumination unit 22 and the audio output unit 31 based on the determination result of the line-of-sight determination unit 25. Then, the occupant monitoring method ends.

[0049] Next, an example of a hardware configuration of the monitoring control device 20A will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of a hardware configuration of the monitoring control device 20A according to the first embodiment.

[0050] 5A, the monitoring control device 20A is configured by a computer, and the computer has a processor 51 and a memory 52. ​​The memory 52 stores programs for causing the computer to function as the acquisition unit 23, the gaze detection unit 24, the gaze determination unit 25, and the warning control unit 26. The processor 51 reads out and executes the programs stored in the memory 52, thereby realizing the functions of the acquisition unit 23, the gaze detection unit 24, the gaze determination unit 25, and the warning control unit 26.

[0051] 5B, the monitoring control device 20A may include a processing circuit 53. In this case, the monitoring control device 20A may be realized by the acquisition unit 23, the gaze detection unit 24, the gaze determination unit 25, and the warning control unit 26.

[0052] Alternatively, the monitoring control device 20A may have a processor 51, a memory 52, and a processing circuit 53 (not shown). In this case, some of the functions of the acquisition unit 23, the gaze detection unit 24, the gaze determination unit 25, and the warning control unit 26 may be realized by the processor 51 and the memory 52, and the remaining functions may be realized by the processing circuit 53.

[0053] The processor 51 is, for example, one that uses at least one of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, a microcontroller, or a digital signal processor (DSP).

[0054] The memory 52 uses, for example, at least one of a semiconductor memory or a magnetic disk. More specifically, the memory 52 uses at least one of a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a solid state drive (SSD), or a hard disk drive (HDD).

[0055] The processing circuit 53 uses, for example, at least one of an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration).

[0056] As described above, the passenger monitoring device 20 according to the first embodiment is configured to detect an area in the passenger compartment of the vehicle 100 where at least the face of the passenger is present. Imaging The occupant monitoring device 20 includes an interior imaging unit 21 that captures an image of an occupant's face and an illumination unit 22 that irradiates light onto an area in the vehicle cabin where at least the face of the occupant is present, and the interior imaging unit 21 and the illumination unit 22 are attached to a display unit 12 that is provided on an A-pillar 104 disposed inside the vehicle cabin. Therefore, the occupant monitoring device 20 can provide the interior imaging unit 21 at a position in the A-pillar 104 where it can capture an image with high accuracy while ensuring an installation space for the display unit 12.

[0057] In the occupant monitoring device 20, the interior imaging unit 21 and the lighting unit 22 are attached to the frame portion 12b of the display unit 12. Therefore, the occupant monitoring device 20 can reduce the space required for the interior imaging unit 21 and the lighting unit 22.

[0058] The occupant monitoring device 20 includes a mounting plate 27 on which the interior imaging unit 21 and the lighting unit 22 are provided, and the mounting plate 27 is attached to the frame portion 12b with the interior imaging unit 21 and the lighting unit 22 integrated together. The occupant monitoring device 20 can improve the ease of installation of the interior imaging unit 21 and the lighting unit 22.

[0059] The occupant monitoring device 20 includes an acquisition unit 23 that acquires an image captured by the in-vehicle imaging unit 21, a gaze detection unit 24 that detects the direction of the gaze of the occupant from the captured image acquired by the acquisition unit 23, a gaze determination unit 25 that determines whether the direction of the gaze of the occupant detected by the gaze detection unit 24 is directed toward the display unit 12, and a warning control unit 26 that controls at least one of the illumination unit 22 and the audio output unit 31 based on the determination result of the gaze determination unit 25. Therefore, the occupant monitoring device 20 can issue a warning to the occupant indicating that the occupant is not gazing at the captured image displayed on the display surface 12a.

[0060] In the occupant monitoring device 20, when the line of sight of an occupant detected by the line of sight detection unit 24 satisfies a preset condition, the line of sight determination unit 25 determines that the line of sight of the occupant is directed toward the display unit 12. Therefore, the occupant monitoring device 20 can appropriately perform the determination of the line of sight determination unit 25.

[0061] In the passenger monitoring device 20, the line of sight determination unit 25 detects whether the direction of the passenger's line of sight is on the display unit 12. Not suitable If it is determined that the warning control unit 26 is in a standby state, the warning control unit 26 blinks the illumination unit 22. Therefore, the occupant monitoring device 20 can reliably issue a warning to the occupant.

[0062] In the passenger monitoring device 20, the line of sight determination unit 25 detects whether the direction of the passenger's line of sight is on the display unit 12. Not suitable If it is determined that the vehicle occupant is in a dangerous state, the warning control unit 26 generates a sound from the sound output unit 31. Therefore, the occupant monitoring device 20 can reliably warn the occupant.

[0063] It should be noted that, within the scope of the present disclosure, any of the components of the embodiments may be modified or any of the components of the embodiments may be omitted.

[0064] Various aspects of the present disclosure are summarized below as appendices.

[0065] (Appendix 1) The range in the vehicle interior where at least the face of the occupant is present is determined. Imaging An in-vehicle imaging unit for capturing an image of a vehicle; a lighting unit that irradiates light onto an area in the vehicle interior where at least the face of an occupant is present, The in-vehicle imaging unit and the lighting unit are attached to a display unit provided on an A-pillar arranged in the vehicle interior. An occupant monitoring device comprising: (Appendix 2) The in-vehicle imaging unit and the lighting unit are attached to a frame portion of the display unit. 2. An occupant monitoring device according to claim 1. (Appendix 3) A mounting plate on which the in-vehicle imaging unit and the lighting unit are provided, The mounting plate is attached to the frame portion of the display unit with the in-vehicle imaging unit and the lighting unit integrated together. 2. An occupant monitoring device according to claim 1. (Appendix 4) an acquisition unit that acquires an image captured by the in-vehicle image capturing unit; a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; a warning control unit that controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit. 4. An occupant monitoring device according to any one of claims 1 to 3. (Appendix 5) The line of sight determination unit is When the line of sight of the occupant detected by the line of sight detection unit satisfies a preset condition, it is determined that the line of sight of the occupant is directed toward the display unit. 5. An occupant monitoring device according to claim 4. (Appendix 6) The line of sight determination unit detects the direction of the line of sight of the occupant on the display unit. Not suitable When it is determined that the warning control unit blinks the illumination unit, 6. An occupant monitoring device according to claim 4 or 5. (Appendix 7) The line of sight determination unit detects the direction of the line of sight of the occupant on the display unit. Not suitable When it is determined that the warning control unit generates a sound from the sound output unit, 7. An occupant monitoring device according to any one of claims 4 to 6, (Appendix 8) The range in the vehicle interior where at least the face of the occupant is present is determined. Imaging An in-vehicle imaging unit for capturing an image of a vehicle; a lighting unit that irradiates light onto an area in the vehicle interior where at least the face of an occupant is present, In the occupant monitoring device, the in-vehicle imaging unit and the lighting unit are attached to a display unit provided on an A-pillar arranged in the vehicle cabin, An acquisition unit acquires an image captured by the in-vehicle imaging unit, a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; A warning control unit controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit. A passenger monitoring method comprising: [Industrial Applicability]

[0066] The occupant monitoring device of the present disclosure has an in-vehicle imaging unit and lighting unit attached to a display unit provided on the A-pillar, so that the in-vehicle imaging unit can be provided in a position on the A-pillar where it can capture images with high accuracy while ensuring installation space for the display unit, making it suitable for use in occupant monitoring devices, etc. [Explanation of symbols]

[0067] 10 electronic mirror device, 11 vehicle exterior imaging unit, 12 display unit, 12a display surface, 12b frame unit, 20 occupant monitoring device, 20A monitoring control device, 21 vehicle interior imaging unit, 22 lighting unit, 23 acquisition unit, 24 gaze detection unit, 25 gaze determination unit, 26 warning control unit, 27 mounting plate, 31 audio output unit, 51 processor, 52 memory, 53 processing circuit, 100 vehicle, 101 driver's seat, 102 windshield, 103 front door, 103a door glass, 104 A-pillar, 105 instrument panel.

Claims

1. an interior image capturing unit configured to capture an image of at least an area in a passenger compartment of the vehicle where the face of an occupant is present; an illumination unit that irradiates light onto an area within the vehicle interior where at least the face of an occupant is present; a mounting plate on which the in-vehicle imaging unit and the lighting unit are provided, the mounting plate is attached to one edge of a frame portion of a display unit provided on an A-pillar disposed in the vehicle cabin with the interior imaging unit and the lighting unit integrated together, The length of the mounting plate and the length of the edge are the same. An occupant monitoring device comprising:

2. an interior image capturing unit configured to capture an image of at least an area in a passenger compartment of the vehicle where the face of an occupant is present; a lighting unit that irradiates light onto an area in the vehicle interior where at least the face of an occupant is present, the interior imaging unit and the lighting unit are attached to a display unit provided on an A-pillar disposed in the vehicle interior, an acquisition unit that acquires an image captured by the in-vehicle image capturing unit; a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; a warning control unit that controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit, The line of sight determination unit is When the frequency with which the occupant visually checks the display unit is equal to or greater than a predetermined number of times, it is determined that the occupant's line of sight is directed toward the display unit. An occupant monitoring device comprising:

3. an acquisition unit that acquires an image captured by the in-vehicle image capturing unit; a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; a warning control unit that controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit, When the line of sight determination unit determines that the direction of the line of sight of the occupant is not directed toward the display unit, the warning control unit blinks the illumination unit.

2. The passenger monitoring device according to claim 1.

4. an acquisition unit that acquires an image captured by the in-vehicle image capturing unit; a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; a warning control unit that controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit, When the line of sight determination unit determines that the direction of the line of sight of the occupant is not directed toward the display unit, the warning control unit generates a sound from the sound output unit.

2. The passenger monitoring device according to claim 1.

5. When the line of sight determination unit determines that the direction of the line of sight of the occupant is not directed toward the display unit, the warning control unit blinks the illumination unit.

3. The passenger monitoring device according to claim 2.

6. When the line of sight determination unit determines that the direction of the line of sight of the occupant is not directed toward the display unit, the warning control unit generates a sound from the sound output unit.

3. The passenger monitoring device according to claim 2.

7. an interior image capturing unit configured to capture an image of at least an area in a passenger compartment of the vehicle where the face of an occupant is present; a lighting unit that irradiates light onto an area in the vehicle interior where at least the face of an occupant is present, In the occupant monitoring device, the in-vehicle imaging unit and the lighting unit are attached to a display unit provided on an A-pillar disposed in the vehicle cabin, An acquisition unit acquires an image captured by the in-vehicle imaging unit, a gaze detection unit that detects a gaze direction of an occupant from the captured image acquired by the acquisition unit; a line-of-sight determination unit that determines whether the direction of the line of sight of the occupant detected by the line-of-sight detection unit is directed toward the display unit; a warning control unit that controls at least one of the illumination unit and the audio output unit based on a determination result of the line-of-sight determination unit; The line of sight determination unit is When the frequency with which the occupant visually checks the display unit is equal to or greater than a predetermined number of times, it is determined that the occupant's line of sight is directed toward the display unit. A passenger monitoring method comprising:

Images