Cabin imaging apparatus and cabin imaging method

Abstract

To provide a cabin imaging apparatus capable of making a crew member grasp being imaged.SOLUTION: A cabin imaging apparatus comprises: a camera arranged within a cabin of a vehicle to image a crew member within the cabin; a plurality of light-emitting units provided around a lens of the camera; and a control unit for controlling imaging of the crew member by the camera. The control unit differentiates the light-emitting mode of a first light-emitting unit corresponding to the crew member to be imaged by the camera from the light-emitting modes of light-emitting units other than the first light-emitting unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an in-vehicle imaging device and an in-vehicle imaging method. [Background technology]

[0002] A technology has been proposed for indicating the shooting status of a camera when taking pictures with the camera. Patent Document 1 describes that the state of at least one of the focal length, depth of field, and brightness of an optical element is indicated by the light emission state of a light-emitting section provided around the lens. Patent Document 2 describes that the current shooting direction is indicated by a red light and the next shooting direction by a green light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-189300 [Patent Document 2] Japanese Patent Application Publication No. 2023-125839 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, driver monitoring systems (DMS) have been used, which monitor the status of occupants by installing cameras inside the vehicle and capturing images of the occupants. The cameras used in DMS are often placed inconspicuously so as not to detract from the interior design of the vehicle. However, from the perspective of protecting privacy, it is preferable for occupants to be aware of the images captured by the DMS cameras.

[0005] An object of one aspect of the disclosed technology is to provide an in-vehicle imaging device and an in-vehicle imaging method that allow occupants to be aware that they are being photographed. [Means for solving the problem]

[0006] One aspect of the disclosed technology is exemplified by the following in-vehicle image capturing device. The in-vehicle image capturing device includes a camera disposed in the vehicle interior for capturing images of occupants in the vehicle interior, a plurality of light-emitting units provided around the lens of the camera, and a control unit that controls the camera to capture images of the occupants. The control unit sets a light emission mode of a first light-emitting unit corresponding to the occupant being captured by the camera to a light emission mode different from that of the light-emitting units other than the first light-emitting unit. [Effects of the Invention]

[0007] According to the disclosed technology, it is possible to make the occupants aware that they are being photographed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a vehicle according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the appearance of a DMS according to an embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a hardware configuration of a DMS according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a processing block of a DMS according to the embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of a processing flow of the DMS according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of the appearance of a DMS according to the first modified example. [Figure 7] FIG. 7 is a diagram illustrating an example of a processing block of a DMS according to the first modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] <Embodiment> Hereinafter, an embodiment will be described with reference to the drawings. Fig. 1 is a diagram showing an example of a vehicle 100 according to an embodiment. Fig. 1 is a diagram showing the vehicle 100 as viewed from above. In Fig. 1, the ceiling is omitted in order to explain the interior of the vehicle 100. The vehicle 100 is, for example, a passenger car.

[0010] Vehicle 100 has passenger seats, including a driver's seat 1, a passenger seat 2, a right rear seat 3, a center rear seat 4, and a left rear seat 5. Driver's seat 1 and passenger seat 2 are arranged in a line along the width direction of vehicle 100. In addition, right rear seat 3, center rear seat 4, and left rear seat 5 are arranged in a line along the width direction of vehicle 100 behind driver's seat 1 and passenger seat 2. A dashboard 10 is arranged in front of driver's seat 1 and passenger seat 2. A steering wheel 20 is arranged in front of driver's seat 1. DMS 30 is arranged on dashboard 10. DMS 30 is arranged with its display unit 33 (see FIG. 2 ) facing the passenger seats (driver's seat 1, passenger seat 2, right rear seat 3, center rear seat 4, and left rear seat 5). DMS 30 is an example of an "in-vehicle imaging device."

[0011] The vehicle 100 is equipped with a front right tire 61, a front left tire 62, a rear right tire 64, and a rear left tire 65. When the front right tire 61 and the front left tire 62 are not distinguished from each other, they are also referred to as front wheels 63. When the rear right tire 64 and the rear left tire 65 are not distinguished from each other, they are also referred to as rear wheels 66.

[0012] Fig. 2 is a diagram showing an example of the appearance of the DMS 30 according to the embodiment. Fig. 2 shows an example of the appearance of the DMS 30 as seen from the passenger seat side. The DMS 30 includes a housing 31, a camera 32, a display unit 33, a lens 34, a first light-emitting unit 35, and a second light-emitting unit 36.

[0013] The housing 31 is formed, for example, in a plate shape, and stores various electronic components that constitute the DMS 30 inside. A display unit 33 is disposed on the front surface (passenger seat side) of the housing 31. The display unit 33 is, for example, a liquid crystal display (LCD), a plasma display panel (PDP), an inorganic electroluminescence (EL) panel, or an organic EL panel. For example, a touch panel that detects touch operations by the user's finger or the like may be provided superimposed on the display unit 33. By providing a touch panel superimposed on the display unit 33, the DMS 30 can provide the user with an intuitive operating environment.

[0014] A protrusion 37 is provided above the housing 31 (toward the ceiling) so as to protrude upward from the housing 31. A lens 34 is provided on the surface of the protrusion 37 on the passenger seat side. The lens 34 is used to photograph passengers inside the cabin of the vehicle 100. The lens 34 is disposed, for example, in the center of the cabin of the vehicle 100 in the width direction. The lens 34 is also disposed forward of the driver's seat 1 and the passenger seat 2. The camera 32 provided in the DMS 30 includes the protrusion 37 and the lens 34. The camera 32 converts light input to the imaging element through the lens 34 into an electrical signal to capture an image. Examples of the imaging element include a charge coupled device (CCD) sensor and a complementary metal oxide semiconductor (CMOS) sensor.

[0015] A first light-emitting unit 35 and a second light-emitting unit 36 ​​are provided around the lens 34. The first light-emitting unit 35 is provided on the right side of the lens 34 (the driver's seat 1 side). The second light-emitting unit 36 ​​is provided on the left side of the lens 34 (the passenger's seat 2 side). In the example of FIG. 2, the first light-emitting unit 35 and the lens 34 are arranged so that the first light-emitting unit 35 and the second light-emitting unit 36 ​​form a ring surrounding the lens 34, but the arrangement of the first light-emitting unit 35 and the lens 34 is not limited to an arrangement in which a ring is formed around the lens 34. The first light-emitting unit 35 and the lens 34 are provided on the driver's seat 1. The first light-emitting unit 35 may be arranged at the corresponding position, and the lens 34 may be arranged at the position corresponding to the passenger seat 2. Here, the corresponding position only needs to be understood in advance by the vehicle occupants of the vehicle 100 with respect to the correspondence between the lens 34 and the first light-emitting unit 35 and the driver's seat 1 and the passenger seat 2. If the correspondence is understood, for example, the first light-emitting unit 35 may be arranged above the lens 34, and the second light-emitting unit 36 may be arranged below the lens 34.

[0016] The first light-emitting unit 35 and the second light-emitting unit 36 can change their light-emitting modes in various ways. Examples of the light-emitting mode include color, light-emitting interval, and light-emitting intensity. The first light-emitting unit 35 and the second light-emitting unit 36 are, for example, Light Emitting Diode (LED).

[0017] <Hardware Configuration of DMS30> FIG. 3 is a diagram showing an example of the hardware configuration of DMS30 according to an embodiment. DMS30 includes a Central Processing Unit (CPU) 301, a main memory unit 302, an auxiliary storage unit 303, a drive unit 304, a display unit 33, a first light-emitting unit 35, a second light-emitting unit 36, and a connection bus B1. The CPU 301, the main memory unit 302, the auxiliary storage unit 303, the drive unit 304, the display unit 33, the first light-emitting unit 35, and the second light-emitting unit 36 are interconnected by the connection bus B1. Further, the Central Processing Unit (CPU) 301, the main memory unit 302, the auxiliary storage unit 303, the drive unit 304, the display unit 33, the first light-emitting unit 35, the second light-emitting unit 36, and the connection bus B1 are housed in the housing 31.

[0018] The CPU 301 is also referred to as a microprocessor unit (MPU) or processor. The CPU 301 is not limited to a single processor and may have a multi-processor configuration. Furthermore, a single CPU 301 connected via a single socket may have a multi-core configuration. At least a portion of the processing performed by the CPU 301 may be performed by a processor other than the CPU 301, such as a dedicated processor, such as a digital signal processor (DSP), a graphics processing unit (GPU), a numerical calculation processor, a vector processor, or an image processing processor. At least a portion of the processing performed by the CPU 301 may be performed by an integrated circuit (IC) or other digital circuit. At least a portion of the CPU 301 may include an analog circuit. Examples of integrated circuits include large-scale integrated circuits (LSIs), application-specific integrated circuits (ASICs), and programmable logic devices (PLDs). Examples of PLDs include field-programmable gate arrays (FPGAs). The CPU 301 may be a combination of a processor and an integrated circuit. This combination is called, for example, a microcontroller unit (MCU), a system-on-a-chip (SoC), a system LSI, or a chipset. In the DMS 30, the CPU 301 loads a program stored in the auxiliary memory 303 into a work area in the main memory 302 and controls peripheral devices through the execution of the program. This enables the DMS 30 to execute processing that meets a predetermined purpose. The main memory 302 and the auxiliary memory 303 are recording media that the CPU 301 can read.

[0019] The main memory unit 302 is a memory unit that is directly accessed by the CPU 301. The main memory unit 302 includes a random access memory (RAM) and a read only memory (ROM).

[0020] The auxiliary storage unit 303 reads and writes various programs and various data to and from a recording medium. The auxiliary storage unit 303 is also called an external storage device. The auxiliary storage unit 303 stores an operating system (OS), various programs, various tables, and the like.

[0021] The auxiliary storage unit 303 may be a storage unit fixed within the housing 31. Examples of the storage unit fixed within the housing 31 include an Erasable Programmable ROM (EPROM), a Solid State Drive (SSD), a Hard Disk Drive (HDD), and the like. Further, the auxiliary storage unit 303 may be a portable storage unit. Examples of the portable storage unit include a Compact Disc (CD) drive device, a Digital Versatile Disc (DVD) drive device, a Blu-ray (registered trademark) Disc (BD) drive device, and the like.

[0022] The drive unit 304 changes the direction of the optical axis of the lens 34 by driving the lens 34 in accordance with an instruction from the CPU 301, thereby changing the shooting direction of the camera 32. As a method of driving the lens 34 by the drive unit 304, various known methods can be adopted.

[0023] The display unit 33 displays an image using a plurality of pixels arranged vertically and horizontally. The first light emitting unit 35 and the second light emitting unit 36 change their light emission modes in accordance with an instruction from the CPU 301.

[0024] <Processing blocks of the DMS 30> FIG. 4 is a diagram showing an example of the processing blocks of the DMS 30 according to the embodiment. The DMS 30 includes a direction control unit 311, a light emission control unit 312, an analysis unit 313, and an output unit 314. The CPU 301 executes a computer program developed executable in the main storage unit 302, thereby executing the processing of the DMS 30 as each of the direction control unit 311, the light emission control unit 312, the analysis unit 313, the output unit 314, and the like.

[0025] The direction control unit 311 drives the lens 34 using the drive unit 304 to control the shooting direction of the camera 32. The direction control unit 311 may control the shooting direction of the camera 32 in a direction according to an instruction from the occupant, for example. The direction control unit 311 may also switch the shooting direction of the camera 32 to either the driver's seat 1 or the passenger's seat 2 at a timing according to a predetermined program.

[0026] The light-emission control unit 312 controls the light-emission modes of the first light-emitting unit 35 and the second light-emitting unit 36 ​​so as to allow the occupants of the vehicle 100 to understand the shooting direction of the camera 32. For example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312 may cause the first light-emitting unit 35 to emit light and the second light-emitting unit 36 ​​to turn off. For example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312 may cause the first light-emitting unit 35 to emit light in a color indicating that shooting is in progress (for example, red) and the second light-emitting unit 36 ​​to emit light in a color indicating that shooting is not in progress (for example, green). Furthermore, for example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312 may cause the first light-emitting unit 35 to blink and the second light-emitting unit 36 ​​to turn on. In other words, the light emission control unit 312 simply causes the light emission unit of the first light emission unit 35 and the second light emission unit 36 ​​that corresponds to the shooting direction and the light emission unit of the first light emission unit 35 and the second light emission unit 36 ​​that corresponds to a direction different from the shooting direction to emit light in different light emission modes.

[0027] The analysis unit 313 analyzes the state of the occupant based on the video of the occupant captured by the camera 32. Examples of the state of the occupant that is analyzed include the degree of fatigue of the occupant and whether the occupant is dozing.

[0028] The output unit 314 outputs the state of the occupant analyzed by the analysis unit 313 to the display unit 33 .

[0029] <Processing flow> FIG. 5 is a diagram showing an example of a processing flow of the DMS 30 according to the embodiment. An example of the processing flow of the DMS 30 will be described with reference to Fig. 5. The processing flow illustrated in Fig. 5 is an example of a "vehicle interior imaging method."

[0030] In step S1, the direction control unit 311 controls the drive unit 304 to control the shooting direction of the camera 32 using the lens 34. Here, it is assumed that the shooting direction is directed toward the driver's seat 1, for example.

[0031] In step S2, the light emission control unit 312 causes the first light emission unit 35 corresponding to the driver's seat 1, to which the shooting direction was directed in step S1, to emit light, and turns off the second light emission unit 36 ​​corresponding to a direction different from the driver's seat 1 (here, the passenger seat 2).

[0032] In step S3, the analysis unit 313 uses the camera 32 to capture images in the shooting direction pointed in step S1 (here, the direction in which the driver's seat 1 is photographed), and analyzes the condition of the occupant seated in the driver's seat 1 based on the captured images.

[0033] In step S4, the output unit 314 outputs to the display unit 33 the state of the occupant analyzed in step S3.

[0034] <Effects of the embodiment> According to this embodiment, the shooting direction of the camera 32 can be notified to the occupants of the vehicle 100 by the light emitting states of the first light emitting unit 35 and the second light emitting unit 36. That is, according to this embodiment, the occupants who are the subject of the shooting can be made aware that the shooting direction of the camera 32 is directed towards them. Therefore, according to this embodiment, the occupants can be made aware that they are being photographed. Consequently, according to this embodiment, the occupants' conditions can be analyzed after they are made aware that they are being photographed.

[0035] In this embodiment, the first light-emitting portion 35 and the second light-emitting portion 36 are arranged so as to surround the periphery of the lens 34. Therefore, according to this embodiment, it is possible to make the occupant easily understand that the light emission modes of the first light-emitting portion 35 and the second light-emitting portion 36 correspond to the shooting direction using the lens 34.

[0036] In this embodiment, the lens 34 is disposed, for example, at the center in the width direction of the interior of the vehicle 100. Therefore, the shooting direction can be easily changed for either the driver's seat 1 or the passenger seat 2.

[0037] <First Modification> In the embodiment described above, the lens 34 is disposed outside the display unit 33, and the first light-emitting unit 35 and the second light-emitting unit 36 ​​are disposed around the lens 34. In the first modified example, a configuration in which the lens 34 is disposed inside the display unit 33 will be described. Components common to the embodiment will be given the same reference numerals, and descriptions thereof will be omitted. Below, the first modified example will be described with reference to the drawings.

[0038] Fig. 6 is a diagram showing an example of the appearance of a DMS 30A according to a first modified example. Fig. 6 shows an example of the appearance of the DMS 30A as seen from the passenger seat side. The DMS 30A is provided with a notch 37A that protrudes from the housing 31 into the display unit 33. No pixels that make up the display unit 33 are arranged in the notch 37A, and instead a lens 34 is arranged therein. The camera 32A provided in the DMS 30A includes the notch 37A and the lens 34.

[0039] In the first modification, the first light-emitting section 35A and the second light-emitting section 36A are formed by using a plurality of pixels of the display section 33 that are arranged around the notch 37A (i.e., around the lens 34). The first light-emitting portion 35A is formed using a plurality of pixels arranged on the right side of the lens 34 (the driver's seat 1 side). The second light-emitting portion 36A is formed using a plurality of pixels arranged on the left side of the lens 34 (the passenger seat 2 side).

[0040] 7 is a diagram showing an example of a processing block of a DMS 30A according to Modification 1. The DMS 30A differs from the DMS 30 according to the embodiment in that it includes a light emission control unit 312A instead of the light emission control unit 312.

[0041] The light-emission control unit 312A controls the light emission modes of the first light-emitting unit 35A and the second light-emitting unit 36A so as to allow the occupants of the vehicle 100 to understand the shooting direction of the camera 32A. For example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312A may cause the first light-emitting unit 35A to emit light and the second light-emitting unit 36A to turn off. For example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312A may cause the first light-emitting unit 35A to emit light in a color indicating that shooting is in progress (for example, red) and the second light-emitting unit 36A to emit light in a color indicating that shooting is not in progress (for example, green). Furthermore, for example, when the shooting direction is facing the driver's seat 1, the light-emission control unit 312A may cause the first light-emitting unit 35A to blink and the second light-emitting unit 36A to turn on. In other words, the light emission control unit 312A simply causes the first light emitting unit 35A or the second light emitting unit 36A corresponding to the shooting direction and the first light emitting unit 35A or the second light emitting unit 36A corresponding to a direction different from the shooting direction to emit light in different light emission modes.

[0042] The light emission mode can include not only simple on / off and light emission color but also patterns displayed by pixels constituting the light-emitting unit. That is, the light emission control unit 312A can control the pixels surrounding the lens 34 to display patterns such as figures and symbols. For example, if the shooting direction is to the right, an arrow mark can be displayed on the right side of the lens 34, or a circle (or oval) can be displayed surrounding the lens 34, and the position of the circle can be adjusted so that the positional relationship between the circle and the camera 32 is such that the camera is at the right end of the circle, as if the camera, corresponding to the pupil, is looking to the right in a circle simulating a human eye, thereby enabling the occupant to understand the shooting direction.

[0043] According to the first modification, the appearance of the DMS 30A can be made neater because there is no need to provide the protrusion 37 that protrudes upward from the housing 31. Furthermore, in the first modification, the first light-emitting unit 35A and the second light-emitting unit 36A are realized using the pixels of the display unit 33, so there is no need to add a light source such as an LED to the DMS 30A.

[0044] In the first modified example, notch 37A is formed so as to protrude into display unit 33, but notch 37A may be omitted. That is, lens 34 may be arranged in display unit 33 with notch 37A omitted. In addition, in the first modified example, lens 34 is arranged so that the entire lens 34 is located within display unit 33, but lens 34 may be arranged so that at least a portion of lens 34 is located within display unit 33. For example, lens 34 may be arranged so that the lower half of lens 34 is located within display unit 33. Conversely, lens 34 may be arranged so that it is surrounded by display unit 33 (i.e., in an island shape).

[0045] <Other variations> In the embodiment and first modified example described above, two light emitting units are arranged around the lens 34, but the number of light emitting units may be three or more.

[0046] The embodiments and modifications disclosed above can be combined with each other. [Explanation of symbols]

[0047] 1. Driver's seat 2...Passenger seat 3. Right rear seat 4. Center rear seat 5. Left rear seat 10. Dashboard 20··Steering wheel 30··DMS 30A··DMS 31··Case 32 Camera 32A··Camera 33...Display section 34 Lens 35 First light-emitting part 35A··First light-emitting part 36 Second light-emitting part 36A Second light-emitting part 37...Protrusion 37A Notch 61··Front right tire 62··Front left tire 63 Front wheel 64··Rear right tire 65··Rear left tire 66··Rear wheel 100 vehicles 301 CPU 302...Main memory 303...Auxiliary storage unit 304··Drive unit 311··Directional Control Unit 312 Light emission control unit 313...Analysis Department 314··Output section B1 Connecting bus

Claims

1. a camera disposed in a vehicle cabin to capture images of occupants in the vehicle cabin; a plurality of light emitting units provided around the lens of the camera; a control unit that controls the camera to capture an image of the occupant, the control unit sets a light emission mode of a first light-emitting unit corresponding to the occupant who is the subject of image capture by the camera to a light emission mode different from a light emission mode of the light-emitting units other than the first light-emitting unit. In-car camera.

2. The plurality of light emitting units are arranged to surround the periphery of the lens. The vehicle interior imaging device according to claim 1 .

3. The control unit turning on the first light-emitting unit and turning off the light-emitting units other than the first light-emitting unit; The vehicle interior imaging device according to claim 1 .

4. The control unit causing the first light-emitting unit and the light-emitting units other than the first light-emitting unit to emit light in different colors; The vehicle interior imaging device according to claim 1 .

5. the camera is disposed at a center in a width direction of the vehicle cabin and forward of a driver's seat and a passenger seat disposed in the vehicle cabin, the plurality of light-emitting units are disposed at positions corresponding to an imaging direction in which the camera images an occupant seated in the driver's seat and at positions corresponding to an imaging direction in which the camera images an occupant seated in the passenger seat, respectively; The vehicle interior imaging device according to claim 1 .

6. the plurality of light-emitting units include a first light-emitting unit disposed on the driver's seat side of the lens and a second light-emitting unit disposed on the passenger's seat side of the lens, The periphery of the lens is surrounded by the first light-emitting unit and the second light-emitting unit in a ring shape. The vehicle interior imaging device according to claim 5.

7. a display unit disposed in the vehicle interior and displaying an image using a plurality of pixels; the lens is disposed so that at least a portion of the lens is located within the display unit; each of the plurality of light-emitting sections is formed by at least some of the plurality of pixels; the control unit controls light emission of the at least some of the pixels to make the light emission mode of the first light-emitting unit different from the light-emitting units other than the first light-emitting unit. The vehicle interior imaging device according to claim 1 .

8. The control unit Analyzing the state of the occupant who is the subject of the image capture based on the image captured by the camera. The vehicle interior imaging device according to any one of claims 1 to 7.

9. A vehicle interior photographing method using a vehicle interior photographing device including a camera disposed in a vehicle interior to photograph occupants in the vehicle interior and a plurality of light emitting units provided around a lens of the camera, The vehicle interior imaging device a light emitting state of a first light emitting unit corresponding to the occupant who is the subject of the image capture by the camera being different from a light emitting state of the light emitting units other than the first light emitting unit; How to shoot inside a car.

Images