Imaging control device and imaging control method

Abstract

An imaging control device (100) comprises: an imaging unit (10) that images, in the same field of view, a first location and a second location different from the first location; a first emission unit (21) that emits light toward the first location; a second emission unit (22) that emits light toward the second location; a brightness information acquisition unit (532) that acquires information pertaining to the brightness of the first location and information pertaining to the brightness of the second location; and an emission time control unit (53) that, on the basis of the information acquired by the brightness information acquisition unit, controls the light emission times of the first emission unit and the second emission unit in an exposure time, during which the imaging unit performs imaging, such that the time during which light is emitted toward the one location, from among the first location and the second location, that has the lower brightness is longer than the time during which light is emitted toward the other location.

Description

Imaging control device and imaging control method

[0001] The present disclosure relates to an imaging control device and an imaging control method.

[0002] BACKGROUND ART A vehicle has been disclosed that is equipped with a camera that photographs the interior of the vehicle, and that acquires information about multiple occupants in the vehicle by photographing the multiple occupants with a single camera (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2022-190504

[0004] In general, when a camera or the like captures images of multiple objects located at different positions within the same field of view, the brightness of the imaged objects may differ depending on the positions of the objects. However, the vehicle described in Patent Document 1 has a problem in that, when the brightness differs depending on the positions of the occupants, it may be difficult to obtain information accurately enough about the occupants located in dark positions.

[0005] The present disclosure is intended to solve the above-described problems, and has an object to provide an imaging control device and an imaging control method that can improve the accuracy of information acquisition.

[0006] The imaging control device according to the present disclosure is characterized by including an imaging unit that images a first position and a second position different from the first position within the same field of view; a first irradiation unit that irradiates light toward the first position; a second irradiation unit that irradiates light toward the second position; a brightness information acquisition unit that acquires information related to the brightness of the first position and information related to the brightness of the second position; and an irradiation time control unit that controls the irradiation time of light by the first irradiation unit and the second irradiation unit during an exposure time during which imaging is performed by the imaging unit, based on the information acquired by the brightness information acquisition unit, so that the time during which light is irradiated toward one of the first and second positions, which is less bright, is longer than the time during which light is irradiated toward the other position.

[0007] According to the present disclosure, the irradiation time of light toward the position to be imaged is controlled based on information regarding the brightness of the position to be imaged, thereby making it possible to suppress differences in brightness between multiple positions and improving the accuracy of information acquisition.

[0008] 1 is a schematic plan view showing an occupant monitoring system according to embodiment 1. FIG. 2 is a block diagram showing the configuration of the occupant monitoring system according to embodiment 1. FIG. 3 is a block diagram showing an example of the hardware configuration of an imaging control device according to embodiment 1. FIG. 4 is a block diagram showing an example of the hardware configuration of an imaging control device according to embodiment 1. FIG. 5 is a flowchart showing processing performed by the imaging control device according to embodiment 1. FIG. 6 is a timing chart showing the exposure time of the imaging unit and the irradiation time of the irradiation unit controlled by the imaging control device according to embodiment 1. FIG. 7 is a schematic plan view showing an occupant monitoring system according to embodiment 2. FIG. 8 is a block diagram showing the configuration of the occupant monitoring system according to embodiment 2. FIG. 9 is a flowchart showing processing performed by the imaging control device according to embodiment 2. FIG. 10 is a timing chart showing the exposure time of the imaging unit and the irradiation time of the irradiation unit controlled by the imaging control device according to embodiment 2. FIG. 11 is a timing chart showing the exposure time of the imaging unit and the irradiation time of the irradiation unit controlled by the imaging control device according to embodiment 1.

[0009] Embodiments of the present disclosure will be described in detail below with reference to the drawings. Embodiment 1. First, a schematic configuration of a vehicle according to embodiment 1 will be described with reference to FIGS. 1 and 2. FIG. 1 is a schematic plan view illustrating an occupant monitoring system 100 according to embodiment 1, and FIG. 2 is a block diagram illustrating the configuration of the occupant monitoring system 100 according to embodiment 1. For example, the occupant monitoring system 100 is a system for detecting an occupant of the vehicle V1 by capturing an image of the interior S1 of the vehicle V1. Furthermore, for example, by analyzing the captured image information of the occupant detected by the occupant monitoring system 100 using an image analysis unit (not shown), it is possible to obtain information about the occupant, such as occupant identification information, occupant position information, occupant posture information, etc.

[0010] 1 and 2 , an occupant monitoring system 100 according to the first embodiment includes an imaging unit 10, an irradiation device 20, and an imaging control device 50. The imaging control device 50 is electrically connected to the imaging unit 10 and the irradiation device 20 so as to enable transmission and reception of information, and controls the imaging unit 10 and the irradiation device 20 based on information from the imaging unit 10. Note that the imaging control device 50 according to the first embodiment may be provided in an external device to which some of the functions described below are connected via a network. Note that in the first embodiment, the front direction of a driver seated in a driver's seat P1 of a vehicle V1 in a driving position is defined as the forward direction, and the front-rear, up-down, left-right, and right directions are defined based on this.

[0011] The illumination device 20 emits light to irradiate the interior S1 of the vehicle V1. For example, the illumination device 20 is configured with multiple LEDs and receives a current to irradiate the interior S1 of the vehicle V1 with infrared light. Furthermore, for example, the illumination device 20 has multiple illumination units that are arranged at different positions and emit light. Specifically, the illumination device 20 has a first illumination unit 21, a second illumination unit 22, a third illumination unit 23, and a fourth illumination unit, and the first to fourth illumination units 21 to 24 irradiate light toward each part of the interior S1 of the vehicle V1. For example, the first illumination unit 21 irradiates an illumination range 21a toward the driver's seat P1 of the vehicle V1, which is a first position. The second illumination unit 22 irradiates an illumination range 22a toward one of the rear seats P2 of the vehicle V1, which is a second position. The third irradiating unit 23 irradiates light onto an irradiation range 23a toward a passenger seat P3 of the vehicle V1. The fourth irradiating unit 24 irradiates light onto an irradiation range 24a toward the other rear seat P4 of the vehicle V1. Note that the first irradiating unit may be arranged to irradiate light onto a seat other than the driver's seat, and the second irradiating unit may be configured to irradiate light onto a seat other than one of the rear seats.

[0012] The imaging unit 10 captures an image within a specific field of view 10a within the vehicle and outputs imaging information obtained by the imaging. In other words, the imaging unit 10 captures an image within the field of view 10a, which is a specific imaging range within the vehicle, and outputs a signal corresponding to the imaging information obtained by the imaging. For example, the imaging unit 10 captures images of the driver's seat P1, one rear seat P2, a passenger seat P3, and the other rear seat P4 of the vehicle V1 within the same field of view. Furthermore, for example, the imaging unit 10 is configured by an infrared camera having an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0013] The imaging control device 50 includes an exposure time control unit 51, an occupant detection unit 52, and an irradiation time control unit 53. The exposure time control unit 51 controls the exposure time during which imaging is performed by the imaging unit 10. For example, the exposure time calculation unit 511 calculates the length of the exposure time during which imaging is performed by the imaging unit 10 based on information from the imaging unit 10 and information from the irradiation time control unit 53. The exposure time control unit 51 includes an exposure time calculation unit 511 and an exposure time setting unit 512. The exposure time setting unit 512 sets the exposure time calculated by the exposure time calculation unit 511. The exposure time setting unit 512 also outputs a control signal to the imaging unit 10 in accordance with the exposure time set by the exposure time setting unit 512 and a preset exposure frequency. In other words, the exposure time setting unit 512 outputs a control signal to the imaging unit 10 in accordance with the exposure time set by the exposure time setting unit 512 and a preset exposure start timing.

[0014] The occupant detection unit 52 detects an occupant in the interior S1 of the vehicle V1 as a specific target. In other words, the occupant detection unit 52 detects an occupant located within the field of view of the image capture unit 10. For example, the occupant detection unit 52 detects an occupant located within the field of view of the image capture unit 10 based on image capture information obtained by the image capture unit 10, and outputs a signal according to the detection result. Specifically, the occupant detection unit 52 detects the presence or absence of an occupant for each seat of the vehicle V1 based on the image capture information obtained by the image capture unit 10. Note that the occupant detection unit 52 may be configured to detect the presence or absence of an occupant using a sensor (not shown) that detects whether a seat belt is fastened in each seat, a weight sensor that detects the weight on each seat, or other sensors (not shown).

[0015] The irradiation time control unit 53 controls the irradiation time of light by the irradiation device 20 based on the imaging information obtained by the imaging unit 10 and the occupant detection result by the occupant detection unit 52. The irradiation time control unit 53 has a brightness target value setting unit 531, a brightness information acquisition unit 532, a brightness determination unit 533, an irradiation time calculation unit 534, and an irradiation time setting unit 535.

[0016] The brightness target value setting unit 531, which serves as a target value acquisition unit, acquires target values ​​related to brightness acquired from each seat by the image capture unit 10 and sets the acquired target values. In other words, it acquires and sets target values ​​for brightness for each seat in the image capture information, and then sets the acquired target values. For example, the brightness target value setting unit 531 sets target values ​​for brightness for each seat in the image capture information so that when the image capture unit 10 captures an image of the interior S1 of the vehicle V1, the image capture unit 10 can acquire a sufficient amount of light from each seat. In other words, it sets target values ​​so that the image capture information acquired by the image capture unit 10 is bright enough to detect occupants. For example, the brightness target value set by the target value setting unit 531 is a digital signal value (DN; Digital Number) of the image sensor. The digital signal value refers to a value (without physical quantity) obtained by converting the amount of photons received by the photodiode of the image sensor into a voltage and then AD-converting the converted voltage. Specifically, if the digital signal value is 8 bits, the target brightness value is 0x80, and the digital signal value DN corresponding to the actual brightness of the seat (occupant) is 0x40, the actual brightness can be said to be half of the target value. In such a case, the irradiation time control unit 53 can adjust the digital signal value DN corresponding to the actual brightness of the seat so that it reaches the target value by doubling the light emission time of the light source to double the radiant flux (W).

[0017] The brightness target value setting unit 531 may be configured to acquire the target value from another component of the imaging control device 50 (not shown), or may be configured to acquire the target value from a device external to the imaging control device 50 (not shown), or may be configured to acquire the target value based on a signal from an input device (not shown) operated by an operator, or may be configured to acquire the target value by the brightness target value setting unit 531 setting the target value based on various conditions. Furthermore, the brightness target value setting unit 531 may be configured to set the target value as a single value, or may be configured to set a lower limit target value and an upper limit target value that indicate a target range, or may be configured to set different target values ​​for each seat.

[0018] The brightness information acquisition unit 532 acquires information about brightness in the interior S1 of the vehicle V1. For example, the brightness information acquisition unit 532 acquires information about brightness of each seat in the vehicle V1. Specifically, the brightness information acquisition unit 532 acquires information about the amount of light acquired from each of the driver's seat P1, one rear seat P2, the passenger seat P3, and the other rear seat P4 of the vehicle V1 based on imaging information acquired by the imaging unit 10. For example, the brightness information acquisition unit 532 acquires information indicating the amount of light acquired from each seat in the vehicle V1 based on image analysis of the imaging information acquired by the imaging unit 10.

[0019] The brightness determination unit 533 determines whether the brightness in the interior S1 of the vehicle V1 reaches the target value set by the brightness target value setting unit 531, based on the information about the brightness of the interior S1 of the vehicle V1 acquired by the brightness information acquisition unit 532. For example, the brightness determination unit 533 determines whether the amount of light acquired from each seat reaches the target value by comparing the brightness of each seat in the vehicle V1 in the imaging information with the target value for brightness set by the brightness target value setting unit 531. In other words, the brightness determination unit 533 determines whether the imaging information obtained by imaging by the imaging unit 10 has sufficient brightness corresponding to the target value.

[0020] The irradiation time calculation unit 534 calculates the irradiation time of light by the irradiation device 20 based on the information about the brightness of the interior S1 of the vehicle V1 acquired by the brightness information acquisition unit 532. Specifically, the irradiation time calculation unit 534 calculates the irradiation time of light by each of the first irradiation unit 21 to the fourth irradiation unit 24 so as to reduce the difference in brightness between the seats based on the information about the brightness of the interior S1 of the vehicle V1 acquired by the brightness information acquisition unit 532. For example, the irradiation time calculation unit 534 calculates the length of continuous irradiation time of each of the first irradiation unit 21 to the fourth irradiation unit 24 so as to reduce the difference in brightness between the seats based on the information about the brightness of the interior S1 of the vehicle V1 acquired by the brightness information acquisition unit 532.

[0021] The irradiation time setting unit 535 sets the time for irradiating light by the irradiation device 20 calculated by the irradiation time calculation unit 534. For example, the irradiation time setting unit 535 outputs a current for causing the irradiation device 20 to emit light according to the set irradiation time to each of the first to fourth irradiation units 21 to 24. Furthermore, for example, the irradiation time setting unit 535 outputs a control signal for causing the irradiation device 20 to emit light based on the set irradiation time to each of the first to fourth irradiation units 21 to 24.

[0022] Next, the hardware configuration of the imaging control device 50 will be described with reference to FIGS. 3 and 4. FIG. 3 is a block diagram showing an example of the hardware configuration of the imaging control device 50 according to the first embodiment, and FIG. 4 is a block diagram showing an example of a hardware configuration of the imaging control device 50 according to the first embodiment, which is different from that shown in FIG. 3. For example, as shown in FIG. 3, the imaging control device 50 includes a processor 50a, a memory 50b, and an I / O port 50c, and is configured so that the processor 50a reads and executes a program stored in the memory 50b. The memory 50b may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM. The memory 50b may also be a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, a DVD, or the like. The memory 50b may also be an HDD or an SSD.

[0023] 4, the imaging control device 50 includes a processing circuit 50d and an I / O port 50c, which are dedicated hardware. The processing circuit 50d is configured, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a system LSI (Large-Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the imaging control device 50 is realized by the processor 50a or the processing circuit 50d, which is dedicated hardware, executing a program that is software, firmware, or a combination of software and firmware.

[0024] Next, details of the processing performed by the imaging control device 50 according to the first embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the processing performed by the imaging control device 50 according to the first embodiment. As shown in Fig. 5, when the processing starts, the imaging control device 50 sets a target value X for the brightness of each seat of the vehicle V1 (step ST1). In this processing, the brightness target value setting unit 531 sets a target value for the brightness of each seat in the imaging information obtained by imaging by the imaging unit 10.

[0025] After performing the processing of step ST1, the imaging control device 50 causes the imaging unit 10 to capture an image of the interior of the vehicle V1 (step ST2). In this processing, the exposure time control unit 51 causes the imaging unit 10 to capture an image of the interior of the vehicle V1 based on the length of the exposure time and the preset imaging frequency, which are preset by the exposure time setting unit 512. Also, in this processing, the imaging control device 50 causes the imaging unit 10 to capture an image of the interior of the vehicle V1, thereby acquiring imaging information of the image of the interior of the vehicle V1. Also, in this processing, the irradiation device 20 irradiates light from each of the first to fourth irradiation units 21 to 24 during the exposure time based on the preset irradiation time.

[0026] After the process of step ST2, the imaging control device 50 acquires information about the brightness of each seat of the vehicle V1 (step ST3). In this process, the brightness information acquisition unit 532 acquires the information about the brightness of each seat based on the imaging information acquired in the process of step ST2.

[0027] After the process of step ST3, the imaging control device 50 detects the presence or absence of an occupant in each seat of the vehicle V1 (step ST4). For example, in this process, the occupant detection unit 52 detects the presence or absence of an occupant in each seat based on the imaging information acquired in the process of step ST2.

[0028] After performing the processing of step ST4, the imaging control device 50 determines whether the brightness of each seat of the vehicle V1 is equal to or greater than the target value X set by the brightness target value setting unit 531 (step ST5). In other words, the imaging control device 50 determines whether the brightness of each seat of the vehicle V1 has reached the target value X set by the brightness target value setting unit 531. For example, the imaging control device 50 determines, using the brightness determination unit 533, whether the brightness of all seats of the vehicle V1 has reached the target value X set by the brightness target value setting unit 531. Note that in this processing, the brightness determination unit 533 may be configured to exclude seats that are already known to have no occupant from the targets for determining whether the brightness has reached the target value X.

[0029] In the process of step ST5, if the brightness of all seats is equal to or greater than the target value X (YES in step ST5), the imaging control device 50 determines whether or not an occupant has been detected in each seat of the vehicle V1 in the process of step ST4 (step ST6). For example, the imaging control device 50 determines whether or not an occupant has been detected in each seat of the vehicle V1. In this process, the occupant detection unit 52 determines whether or not an occupant has been detected in each seat of the vehicle V1 based on the imaging information obtained by imaging by the imaging unit 10. Note that in this process, the occupant detection unit 52 may be configured to exclude seats that are already known to be empty from the targets for determining whether or not an occupant is present.

[0030] If the brightness of all seats is not equal to or greater than the target value X in the process of step ST5 (NO in step ST5), or if no occupants are detected in any seats in the process of step ST6 (NO in step ST6), the imaging control device 50 calculates the exposure time for imaging the interior of the vehicle V1 using the imaging unit 10 (step ST7). In this process, the imaging control device 50 calculates the exposure time appropriate for imaging by the imaging unit 10 based on whether the brightness of any seat does not reach the target value X or whether an occupant was not detected in any seat even though the brightness of all seats reaches the target value X. For example, in this process, the exposure time calculation unit 511 calculates, for the seat with the lowest brightness among the seats, the exposure time at which the brightness of the seat reaches the target value X when the exposure time matches the irradiation time of light from any of the first to fourth irradiation units 21 to 24 that irradiate light toward the seat.

[0031] After performing the process of step ST7, the imaging control device 50 calculates, for each seat other than the seat with the lowest brightness, the light irradiation time of each of the first to fourth irradiation units 21 to 24 so that the brightness of each seat becomes the target value X (step ST8). In this process, the irradiation time calculation unit 534 calculates the light irradiation time of the first to fourth irradiation units 21 to 24 so that the difference in brightness between each seat, including the seat with the lowest brightness, becomes small, using the target value X for brightness as a reference.

[0032] After performing the process of step ST8, the imaging control device 50 determines whether or not there was a seat whose brightness reached the target value X in the process of step ST3 (step ST9). In this process, the brightness determination unit 533 determines whether or not there was a seat for which information indicating that the brightness was sufficient was acquired in the process of step ST3.

[0033] In the process of step ST9, if there is a seat whose brightness has reached the target value X (YES in step ST9), the imaging control device 50 determines whether or not an occupant has been detected for each seat whose brightness has reached the target value X (step ST10). For example, in this process, the occupant detection unit 52 determines whether or not an occupant has been detected for all seats whose brightness has reached the target value X. In this process, the occupant detection unit 52 determines whether or not there was a seat in which an occupant was not detected despite the occupant having sufficient brightness. Note that the occupant detection unit 52 may be configured to exclude seats that are already known to have no occupant before performing this process from the targets for determining whether or not an occupant has been detected.

[0034] In the process of step ST10, if an occupant has not been detected in any of the seats whose brightness has reached the target value X (NO in step ST10), in other words, if there is a seat in which no occupant has been detected even though the brightness has reached the target value X in the process of step ST10, the imaging control device 50 shortens the light-emitting time of the irradiation unit that irradiates the seat in which no occupant has been detected (step ST11). For example, in the process of step ST10, if there is a seat in which no occupant has been detected even though the brightness has reached the target value X, the imaging control device 50 sets the light-emitting time of the irradiation unit that irradiates the seat in which no occupant has been detected to zero. In other words, in the process of step ST10, if there is a seat in which no occupant has been detected even though the brightness has reached the target value X, the imaging control device 50 sets the light-emitting time of the irradiation unit that irradiates the seat in which no occupant has been detected to zero. In this process, the irradiation time setting unit 535 stops the light emission of the irradiation unit corresponding to the seat in which no occupant has been detected even though the brightness was sufficient, thereby reducing the amount of current consumed by the irradiation device 20.

[0035] In the process of step ST9, if there is no seat whose brightness has reached the target value X (NO in step ST9), and if the process of step ST11 has been performed, the imaging control device 50 sets the light emission time of each of the first to fourth irradiation units 21 to 24 (step ST12). In this process, the irradiation time setting unit 535 sets the irradiation time of light by each of the first to fourth irradiation units 21 to 24 calculated by the irradiation time calculation unit 534 in step ST8.

[0036] In the process of step ST6, if occupants are detected in all seats (YES in step ST6), and if the process of step ST12 is performed, the imaging control device 50 ends the process. By performing the processes of steps ST1 to ST12 at a preset frequency, the imaging control device 50 repeatedly captures images of the interior of the vehicle V1, making it possible to obtain information about the occupants inside the vehicle V1.

[0037] 6 is a timing chart showing the exposure time of the imaging unit 10 and the irradiation time of each irradiation unit, which are controlled by the imaging control device 50 according to the first embodiment. As shown in FIG. 6 , the imaging control device 50 sets the length of the exposure time and the exposure start timing, and the imaging unit 10 repeatedly images the interior of the vehicle V1 at a period Tf. The imaging control device 50 also sets the irradiation time, and controls the first to fourth irradiation units 21 to 24 to irradiate light during the exposure time. In other words, the imaging control device 50 controls the first to fourth irradiation units 21 to 24 so that a portion of the exposure time overlaps with the light irradiation time. In other words, the imaging control device 50 controls the first to fourth irradiators 21 to 24 so that the light irradiation time is included in the exposure time.

[0038] For example, the imaging control device 50 controls the first to fourth irradiators 21 to 24 to start irradiating light in accordance with the start timing of the exposure time and to irradiate light according to the irradiation time set for each irradiator. For example, the imaging control device 50 controls the first to fourth irradiators 21 to 24 to irradiate light continuously from the start of the exposure time until the irradiation time set for each irradiator has elapsed. The first to fourth irradiators 21 to 24 irradiate light for exposure time T1 and then irradiate light again for exposure time T2, which is a period Tf after the start of exposure time T1. In this way, the imaging control device 50 according to the first embodiment controls the irradiation time of the first to fourth irradiators 21 to 24 without adjusting the light intensity of the first to fourth irradiators 21 to 24, thereby enabling imaging information of sufficient brightness to be obtained by imaging by the imaging unit 10. The irradiation time setting unit 535 only needs to be configured to set the irradiation time during which each irradiation unit irradiates light during the exposure time. In addition to setting the irradiation time so that light is irradiated continuously, the irradiation time setting unit 535 may set the irradiation time so that a duty ratio of irradiation during the exposure time is set and PWM (Pulse Width Modulation) control is performed.

[0039] Furthermore, the irradiation time setting unit 535 is configured to control the irradiation time of each irradiation unit so that at least one of the irradiation start time and irradiation end time of each irradiation unit is synchronized. This reduces the processing load of the device and enables the device to be made smaller than when the irradiation start time and irradiation end time of each irradiation unit are controlled individually.

[0040] As described above, the occupant monitoring system 100 according to the first embodiment includes an imaging unit 10 that images the driver's seat P1 and one of the rear seats different from the driver's seat P1 within the same field of view 10a, a first irradiation unit 21 that irradiates light toward the driver's seat P1, a second irradiation unit 22 that irradiates light toward one of the rear seats P2, a brightness information acquisition unit 532 that acquires information regarding the brightness of the driver's seat P1 and information regarding the brightness of one of the rear seats P2, and an irradiation time control unit 53 that controls the irradiation time of light by the first irradiation unit 21 and the second irradiation unit 22 during the exposure time T1 during which imaging is performed by the imaging unit 10, based on the information acquired by the brightness information acquisition unit 532, so as to reduce the difference in brightness between the driver's seat P1 and one of the rear seats P2.

[0041] Generally, when capturing images of the interior of a vehicle using an imaging unit such as a camera, if the brightness differs depending on the position of the occupants, it is difficult to capture images of multiple occupants within the same field of view and detect the occupants based on the image information obtained by the image capture. For example, if a light source and a camera are located closer to the driver's seat than the rear seats, the obtained image information is likely to be dark in the rear seats. When capturing images under such conditions, it is difficult to improve the accuracy of information acquisition when detecting occupants from the image information due to the difference in brightness between the driver's seat and the rear seats.

[0042] Since the occupant monitoring system 100 according to the first embodiment is configured as described above, the illumination time of the light from the plurality of illumination units directed toward the imaged position is controlled based on information about the brightness of the imaged position, thereby making it possible to reduce the difference in brightness between the plurality of positions and improve the accuracy of information acquisition. Furthermore, since the difference in brightness between the plurality of positions in the imaged information can be reduced without controlling the light intensity of the illumination units, it is possible to reduce the difference in brightness between the plurality of positions in the imaged information with higher accuracy than when the light intensity is controlled, and it is also possible to simplify the processing and reduce the size of the device.

[0043] The imaging control device 50 according to the first embodiment is configured to detect occupants inside the vehicle V1 based on imaging information obtained by imaging the interior of the vehicle V1, but is not limited to this. The imaging control device may be configured to control the irradiation times of the multiple irradiation units that irradiate light toward each position based on the brightness of each position within at least the same field of view. For example, the imaging control device may not perform any detection based on the imaging information, and the detection target may not be an occupant or may be something other than a person. Furthermore, the range imaged by the imaging unit does not have to be the interior of a vehicle, but may be the interior of a ship, an aircraft, a railroad car, the interior of a building, or outdoors.

[0044] Embodiment 2 Next, an occupant monitoring system 200 according to embodiment 2 will be described. The occupant monitoring system 200 according to embodiment 2 differs from the occupant monitoring system 100 according to embodiment 1 in the configuration of the lighting device and part of the configuration of the imaging control device, but the other configuration is the same as the occupant monitoring system 100 according to embodiment 1. Therefore, the same names or the same reference numerals will be used to designate the same configuration as the occupant monitoring system 100 according to embodiment 1, and description thereof will be omitted.

[0045] Fig. 7 is a schematic plan view of an occupant monitoring system 200 according to embodiment 2, and Fig. 8 is a block diagram showing the configuration of the occupant monitoring system 200 according to embodiment 2. As shown in Figs. 7 and 8, the occupant monitoring system 200 according to embodiment 2 includes an imaging unit 10, an irradiation device 30, and an imaging control device 60.

[0046] The illumination device 30 emits light to irradiate the interior of the vehicle V2. For example, the illumination device 30 is configured with multiple LEDs and receives a current to irradiate the interior of the vehicle V2 with infrared light. Furthermore, for example, the illumination device 30 has multiple illumination units that are arranged at different positions and emit light. Specifically, the illumination device 30 has a first illumination unit 21, a second illumination unit 32, and a third illumination unit 23, and the first to third illumination units 21 to 23 irradiate light toward each part of the interior of the vehicle V2. For example, the first illumination unit 21 irradiates an illumination range 21a toward the driver's seat P1 of the vehicle V2, which is a first position. The second illumination unit 32 irradiates an illumination range 32a toward one rear seat P2 and the other rear seat P4 of the vehicle V2, which are second positions. The third illumination unit 23 irradiates an illumination range 23a toward the passenger seat P3 of the vehicle V2. Furthermore, in the region A1, the light irradiation area 32a by the second irradiating section 32 partially overlaps with the light irradiation area 21a by the first irradiating section 21 and the light irradiation area 23a by the third irradiating section 23. In other words, the direct light from the first irradiating section 21, the direct light from the second irradiating section 32, and the direct light from the third irradiating section 23 overlap with each other in the region A1.

[0047] In this way, when there are areas where the direct light from multiple irradiation units overlaps with one another and areas where it does not overlap, the areas where the direct light overlaps are brighter than the areas where it does not overlap, resulting in uneven brightness across the entire field of view 10a. When brightness becomes uneven within the field of view 10a, the accuracy of information acquisition through imaging information may decrease.

[0048] For this reason, the imaging control device 60 of embodiment 2 aims to improve the accuracy of information acquisition from imaging information by reducing the amount of light acquired by the imaging unit 10 from areas where direct light from multiple irradiation units overlaps with each other.

[0049] The imaging control device 60 includes an exposure time control unit 51, an occupant detection unit 52, an irradiation time control unit 53, and an imaging information correction unit 64. The imaging information correction unit 64 performs a process of reducing brightness in an area where direct light from a plurality of preset irradiation units overlaps with each other in imaging information obtained by imaging with the imaging unit 10. For example, the imaging information correction unit 64 corrects the imaging information obtained by imaging with the imaging unit 10 so as to reduce brightness in an area where direct light from a plurality of preset irradiation units overlaps with each other. The occupant detection unit 52 detects an occupant based on the imaging information whose brightness has been corrected by the imaging information correction unit 64. Furthermore, for example, the imaging information correction unit 64 corrects the imaging information so as to reduce brightness in an area where direct light from a plurality of preset irradiation units overlaps with each other in accordance with the duration of overlap of the direct light.

[0050] The hardware configuration of the imaging control device 60 according to the second embodiment is similar to the hardware configuration of the imaging control device 50 according to the first embodiment, and therefore a description thereof will be omitted.

[0051] 9 is a flowchart showing the processing performed by the imaging control device 60 according to embodiment 2. The processing performed by the imaging control device 60 according to embodiment 2 differs from the processing performed by the imaging control device 50 according to embodiment 1 in that step ST13 is performed between steps ST3 and ST4, but the other steps are the same as those of the imaging control device 50 according to embodiment 1. Therefore, steps similar to those of the processing performed by the imaging control device 50 according to embodiment 1 are denoted by the same reference numerals and descriptions thereof will be omitted.

[0052] After performing the process of step ST3, the imaging control device 60 performs an imaging information correction process to correct the imaging information obtained by the imaging unit 10 so as to reduce the brightness of an area where direct light from a plurality of preset irradiation units overlaps with one another. For example, in this process, the imaging information correction unit 64 as a brightness reduction unit performs an imaging information correction process as a brightness reduction process to correct the imaging information obtained by the imaging of step ST2 so as to reduce the brightness of an area A1 (see FIG. 7 ) where direct light from the first irradiation unit 21, the second irradiation unit 32, and the third irradiation unit 23 overlap with one another. Note that the imaging information correction unit may be configured to reduce the brightness of an area where direct light from a plurality of irradiation units overlaps with one another in the imaging information obtained by the imaging unit 10. For example, the imaging information correction unit may be configured to control the imaging unit 10 so as to reduce the amount of light acquired by the imaging unit 10 from the area A1 when the imaging unit 10 performs imaging, or may be configured to thin out part of the information corresponding to the area A1 when acquiring imaging information from the imaging unit 10.

[0053] After performing the process of step ST13, the imaging control device 60 performs the process of step ST4.

[0054] 10 is a timing chart showing the exposure time of the imaging unit 10 and the irradiation time of the irradiation unit controlled by the imaging control device 60 according to embodiment 2. Similar to the imaging control device 50 according to embodiment 1, the imaging control device 60 according to embodiment 2 controls the imaging unit 10 and each irradiation unit, and causes the imaging unit 10 to repeatedly capture images of the interior of the vehicle V1 at a period Tf.

[0055] Embodiment 3 Next, an occupant monitoring system 300 according to embodiment 3 will be described. The occupant monitoring system 300 according to embodiment 3 differs from the occupant monitoring system 200 according to embodiment 2 in the processing related to the imaging information correction unit, but other configurations are similar to the occupant monitoring system 200 according to embodiment 2. Therefore, configurations similar to those of the occupant monitoring system 200 according to embodiment 2 are given the same names or the same reference numerals, and description thereof will be omitted.

[0056] 8 , an occupant monitoring system 300 according to the third embodiment includes an imaging unit 10, an irradiation device 30, and an imaging control device 70. The imaging control device 70 includes an exposure time control unit 51, an occupant detection unit 52, an irradiation time control unit 53, and an imaging information correction unit 74.

[0057] 7 , when light G1 as disturbance light is incident from the outside of the vehicle V2 toward the interior S1 through a window of the vehicle V2, depending on the intensity of the light G1 and the length of time the light G1 is incident, the amount of light acquired by the imaging unit 10 during the exposure time may become excessive, and the imaging information obtained by imaging may become too bright, resulting in insufficient accuracy in acquiring information from the imaging information. For this reason, the exposure time calculation unit 511 according to the third embodiment calculates the length of the exposure time by the imaging unit 10 based on the information acquired by the brightness information acquisition unit 532, so that when the brightness of any seat exceeds a preset brightness, the exposure time is shorter than when the brightness does not exceed the preset brightness.

[0058] 11 is a timing chart showing the exposure time of the imaging unit 10 and the irradiation time of the irradiation unit controlled by the imaging control device 70 according to embodiment 3. For example, if the amount of light acquired by the imaging unit 10 from any seat exceeds a preset value during the exposure time, the imaging control device 70 controls the imaging unit 10 so that the exposure time for subsequent imaging is set to exposure time T4, which is shorter than exposure times T1 and T2 shown in FIG. 10 , which are exposure times when the amount of light acquired from any seat does not exceed the preset value. This makes it possible to prevent the brightness of the imaging information from becoming excessive (bloated highlights) and improve the accuracy of acquiring information from the imaging information.

[0059] Furthermore, for seats where the brightness of the image information has decreased due to the shortened exposure time, the accuracy of acquiring information from the image information can be improved by partially overlapping the image information from multiple times, for example, two times. For example, the image information correcting unit 74 according to the third embodiment is configured to partially overlap the first image information acquired during the first exposure time T1 and the second image information acquired during the second exposure time T2 at a position of each seat where the brightness of the image information is lower than at other positions, and the occupant detection unit 52 detects an occupant within the field of view based on the overlapped information.

[0060] The hardware configuration of the imaging control device 70 according to the third embodiment is similar to the hardware configuration of the imaging control device 50 according to the first embodiment, and therefore a description thereof will be omitted.

[0061] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0062] An imaging device according to the present disclosure can be used, for example, in an occupant monitoring system that acquires information about an occupant in a vehicle by capturing an image of the occupant.

[0063] Various aspects of the present disclosure are summarized below as appendices.

[0064] (Supplementary Note 1) An imaging control device comprising: an imaging unit that images a first position and a second position different from the first position within the same field of view; a first illumination unit that irradiates light toward the first position; a second illumination unit that irradiates light toward the second position; a brightness information acquisition unit that acquires information on brightness of the first position and information on brightness of the second position; and an irradiation time control unit that controls, based on the information acquired by the brightness information acquisition unit, irradiation times of light by the first illumination unit and the second illumination unit during an exposure time during which imaging is performed by the imaging unit, so that a time during which light is irradiated toward one of the first position and the second position, which is less bright, is longer than a time during which light is irradiated toward the other position. (Supplementary Note 2) The imaging control device according to Supplementary Note 1, wherein the brightness information acquisition unit acquires information on brightness of the first position and the second position based on imaging information obtained by imaging by the imaging unit. (Supplementary Note 3) The imaging control device according to Supplementary Note 1 or 2, further comprising: a detection unit that detects a specific object located within the field of view. (Supplementary Note 4) The imaging control device according to any one of Supplements 1 to 3, characterized in that the detection unit detects a specific object within the field of view based on imaging information obtained by imaging by the imaging unit. (Supplementary Note 5) The imaging unit images the first position and the second position within the same field of view during a first exposure time and a second exposure time different from the first exposure time, and the detection unit detects the specific object within the field of view based on information obtained by superimposing first imaging information acquired by the imaging unit during the first exposure time and second imaging information acquired by the imaging unit during the second exposure time. (Supplementary Note 6) The imaging control device according to any one of Supplements 1 to 5, characterized in that the detection unit detects the specific object within the field of view based on information obtained by superimposing the first imaging information and the second imaging information of a position of lower brightness between the first position and the second position, based on information acquired by the brightness information acquisition unit.(Supplementary Note 7) The imaging control device according to any one of Supplements 1 to 6, wherein, when the detection unit does not detect a specific object located at one of the first position and the second position, the irradiation time control unit controls the irradiation time of light from one of the first position and the second position so that the irradiation time of light is shorter than when a specific object is detected. (Supplementary Note 8) The imaging control device according to any one of Supplements 1 to 7, wherein the irradiation time control unit controls the irradiation time of light by the first irradiating unit and the second irradiating unit so that at least one of the irradiation start times and the irradiation end times of the first irradiating unit and the second irradiating unit are aligned. (Supplementary Note 9) The imaging control device according to any one of Supplements 1 to 8, further comprising: a target value acquisition unit that acquires a target value related to the amount of light acquired by the imaging unit from the first position and the second position, and the irradiation time control unit controls the irradiation time of light by the first irradiating unit and the second irradiating unit based on information acquired by the brightness information acquisition unit so that the amount of light acquired by the imaging unit from the first position and the second position approaches the target value. (Supplementary Note 10) The imaging control device according to any one of Supplementary Notes 1 to 9, further comprising an exposure time control unit that controls the length of the exposure time based on information acquired by the brightness information acquisition unit. (Supplementary Note 11) The imaging control device according to any one of Supplementary Notes 1 to 10, further comprising: an exposure time control unit that controls the length of the exposure time based on information acquired by the brightness information acquisition unit, when the brightness of at least one of the first position and the second position exceeds a preset brightness, so that the exposure time is shorter than when the brightness does not exceed the preset brightness. (Supplementary Note 12) The imaging control device according to any one of Supplementary Notes 1 to 11, further comprising: a brightness reduction unit that performs brightness reduction processing to reduce brightness of an area where direct light from the first irradiation unit and direct light from the second irradiation unit overlap, in imaging information obtained by imaging by the imaging unit.(Supplementary Note 13) The imaging control device according to any one of Supplementary Notes 1 to 12, wherein the brightness reduction unit performs the brightness reduction process based on irradiation times of the first irradiation unit and the second irradiation unit. (Supplementary Note 14) An imaging control method performed by a device including an imaging unit, a first irradiating unit, a second irradiating unit, a brightness information acquisition unit, and an irradiation time control unit, comprising: a step by the imaging unit capturing an image of a first position and a second position different from the first position within the same field of view; a step by the first irradiating unit irradiating light toward the first position; a step by the second irradiating unit irradiating light toward the second position; a step by the brightness information acquisition unit acquiring information on the brightness of the first position and information on the brightness of the second position; and a step by the irradiation time control unit controlling, based on the information acquired by the brightness information acquisition unit, irradiation times of light by the first irradiating unit and the second irradiating unit during an exposure time during which imaging is performed by the imaging unit, so that a time for which light is irradiated toward one of the first position and the second position, which is less bright, is longer than a time for which light is irradiated toward the other position.

[0065] 10 imaging unit, 10a field of view, 20 irradiation device, 21 first irradiation unit, 21a irradiation range, 22 second irradiation unit, 22a irradiation range, 23 third irradiation unit, 23a irradiation range, 24 fourth irradiation unit, 24a irradiation range, 30 irradiation device, 32 second irradiation unit, 32a irradiation range, 50 imaging control device, 51 exposure time control unit, 52 occupant detection unit (detection unit), 53 irradiation time control unit, 60 imaging control device, 64 imaging information correction unit (brightness reduction unit), 70 imaging control device, 74 imaging information correction unit, 100 occupant monitoring system (imaging control device), 200 occupant monitoring system (imaging control device), 300 occupant monitoring system (imaging control device), 511 exposure time calculation unit, 512 exposure time setting unit, 531 brightness target value setting unit (target value acquisition unit), 532 brightness information acquisition unit, 533 Brightness determination unit, 534 irradiation time calculation unit, 535 irradiation time setting unit, A1 area, G1 light (disturbance light), P1 driver's seat, P2 rear seat, P3 passenger seat, P4 rear seat, S1 interior, T1 exposure time (first exposure time), T2 exposure time (second exposure time), T4 exposure time, Tf cycle, V1 vehicle, V2 vehicle, X target value.

Claims

1. an imaging unit that images a first position and a second position different from the first position within the same field of view; a first irradiating unit that irradiates light toward the first position; a second irradiating unit that irradiates light toward the second position; a brightness information acquisition unit that acquires information about brightness at the first position and information about brightness at the second position; an irradiation time control unit that controls, based on the information acquired by the brightness information acquisition unit, irradiation times of light by the first irradiation unit and the second irradiation unit during an exposure time during which image capture is performed by the image capture unit, so that a time during which light is irradiated toward one of the first position and the second position, which has lower brightness, is longer than a time during which light is irradiated toward the other position; a detection unit that detects a specific object located within the field of view, When the detection unit does not detect a specific target located at one of the first position and the second position, the irradiation time control unit controls the irradiation time of light from one of the first position and the second position so that the irradiation time of light from one of the first position and the second position is shorter than when the detection unit detects a specific target. An imaging control device characterized by:

2. An imaging unit that images a first position and a second position different from the first position within the same field of view; a first irradiating unit that irradiates light toward the first position; a second irradiating unit that irradiates light toward the second position; a brightness information acquisition unit that acquires information about brightness at the first position and information about brightness at the second position; an irradiation time control unit that controls, based on the information acquired by the brightness information acquisition unit, irradiation times of light by the first irradiation unit and the second irradiation unit during an exposure time during which image capture is performed by the image capture unit, so that a time during which light is irradiated toward one of the first position and the second position, which has lower brightness, is longer than a time during which light is irradiated toward the other position; a brightness reduction unit that performs a brightness reduction process to reduce brightness in an area where direct light from the first irradiation unit and direct light from the second irradiation unit overlap in imaging information obtained by imaging by the imaging unit. An imaging control device characterized by:

3. The brightness information acquisition unit acquires information about brightness at the first position and the second position based on imaging information obtained by imaging by the imaging unit.

3. The imaging control device according to claim 1 or 2.

4. A detection unit for detecting a specific object located within the field of view is provided.

3. The imaging control device according to claim 2.

5. The detection unit detects a specific object within the field of view based on image information obtained by the image capture unit.

5. The imaging control device according to claim 1 or 4.

6. the imaging unit images the first position and the second position within the same field of view during a first exposure time and a second exposure time different from the first exposure time; The detection unit detects a specific object within the field of view based on information obtained by superimposing first imaging information acquired by the imaging unit during the first exposure time and second imaging information acquired by the imaging unit during the second exposure time.

6. The imaging control device according to claim 5.

7. The detection unit detects a specific object within the field of view based on information acquired by the brightness information acquisition unit, and based on information obtained by superimposing the first imaging information and the second imaging information at a position where the brightness of the imaging information is lower between the first position and the second position.

7. The imaging control device according to claim 6.

8. When the detection unit does not detect a specific target located at one of the first position and the second position, the irradiation time control unit controls the irradiation time of light from one of the first position and the second position so that the irradiation time of light from one of the first position and the second position is shorter than when the detection unit detects a specific target.

5. The imaging control device according to claim 4.

9. The irradiation time control unit controls the irradiation times of the light by the first irradiation unit and the second irradiation unit so that at least one of an irradiation start time and an irradiation end time of the first irradiation unit and the second irradiation unit is aligned.

3. The imaging control device according to claim 1 or 2.

10. a target value acquisition unit that acquires a target value related to the amount of light acquired by the imaging unit from the first position and the second position; The irradiation time control unit controls irradiation times of light by the first irradiation unit and the second irradiation unit based on the information acquired by the brightness information acquisition unit so that the amount of light acquired by the imaging unit from the first position and the second position approaches the target value.

3. The imaging control device according to claim 1 or 2.

11. an exposure time control unit that controls the length of the exposure time based on the information acquired by the brightness information acquisition unit; 3. The imaging control device according to claim 1 or 2.

12. The exposure time control unit controls the length of the exposure time when the brightness of at least one of the first position and the second position exceeds a preset brightness based on the information acquired by the brightness information acquisition unit, so that the exposure time is shorter than when the brightness does not exceed the preset brightness.

12. The imaging control device according to claim 11.

13. a brightness reduction unit that performs a brightness reduction process to reduce brightness in an area where direct light from the first irradiation unit and direct light from the second irradiation unit overlap in the imaging information obtained by imaging with the imaging unit; 2. The imaging control device according to claim 1.

14. The brightness reducing unit performs the brightness reducing process based on irradiation times of the first irradiation unit and the second irradiation unit.

14. The imaging control device according to claim 2 or 13.

15. An imaging control method performed by a device including an imaging unit, a first irradiation unit, a second irradiation unit, a brightness information acquisition unit, an irradiation time control unit, and a detection unit, the imaging unit capturing an image of a first position and a second position different from the first position within the same field of view; a step in which the first irradiating unit irradiates light toward the first position; a step in which the second irradiating unit irradiates light toward the second position; the brightness information acquisition unit acquiring information about brightness at the first position and information about brightness at the second position; the illumination time control unit controls, based on the information acquired by the brightness information acquisition unit, illumination times of the first illumination unit and the second illumination unit during an exposure time during which the imaging unit captures an image, so that a time during which light is illuminated toward one of the first position and the second position, which has lower brightness, is longer than a time during which light is illuminated toward the other position; The detection unit detects a specific object located within the field of view, When the detection unit does not detect a specific target located at one of the first position and the second position, the irradiation time control unit controls the irradiation time of light from one of the first position and the second position so that the irradiation time of light from one of the first position and the second position is shorter than when the detection unit detects a specific target. An imaging control method comprising: