Shielding detection system and crew monitoring device

Abstract

To provide a shielding section detection system configured to properly image an occupant while protecting privacy of the occupant, and an occupant monitoring apparatus.SOLUTION: A shielding section detection system includes an imaging unit 10 having an imaging surface 10a and a shielding section 30, and a control unit 20. The shielding section 30 is configured to be displaced between a shielding position and an open position, and has periodic shielding section detection patterns PA1, PA2, PA3, PA4 on an inside surface that faces the imaging surface when located in the shielding position. The control unit 20 has a shielding section detection pattern determination unit 23. The shielding section detection pattern determination unit compares first data corresponding to a periodic pattern detected from an image captured by the imaging unit with second data corresponding to the shielding section detection patterns, to perform shielding section detection processing to detect a position of the shielding section.SELECTED DRAWING: Figure 7

Description

Technical Field

[0006] , , , ,

[0001] The present invention relates to a shielding portion detection system and an occupant monitoring device.

Background Art

[0002] Patent Document 1 discloses a technique related to an image monitoring system configured to include a concealment detection device that detects the presence or absence of concealment of a monitoring camera. This image monitoring system includes a reception unit that receives image data captured by the monitoring camera, and a determination unit that calculates two values, namely, the standard deviation value of the luminance value of the image data received by the reception unit and the amount of change in the luminance value, and determines the presence or absence of concealment of the monitoring camera based on the two calculated values.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in an occupant monitoring device that monitors an occupant in a vehicle interior, there is no privacy protection mechanism that interrupts imaging by the imaging unit of the occupant monitoring device. For this reason, in the occupant monitoring device, there is a possibility that the imaging unit may capture an image even when the occupant does not want to be captured.

[0005] An object of the present invention is to provide a shielding portion detection system and an occupant monitoring device that can appropriately capture an occupant while protecting the privacy of the occupant in view of the above circumstances.

Means for Solving the Problems

[0006] The shielding detection system of the present invention comprises an imaging unit mounted on a vehicle and positioned facing the occupants of the vehicle, an imaging surface and a shielding portion capable of shielding the imaging surface, and a control unit that controls the imaging unit, wherein the shielding portion is configured to be displaceable between a shielding position that shields the imaging surface facing it and an open position that is retracted from the shielding position, and has a periodic shielding detection pattern on its inner surface facing the imaging surface when positioned in the shielding position, and the control unit has a shielding detection pattern determination unit and the shielding detection pattern The pattern determination unit is configured to perform an occlusion detection process to detect the position of the occlusion by comparing first data corresponding to a periodic pattern detected from an image captured by the imaging unit with second data corresponding to the pattern for detecting the occlusion. In the occlusion detection process, the pattern determination unit for detecting the occlusion determines that the occlusion is in the occlusion position if the first data and the second data correspond, and determines that the occlusion is not in the occlusion position if the first data and the second data do not correspond.

[0007] The occupant monitoring device of the present invention is mounted on a vehicle and comprises an imaging unit having an imaging surface positioned facing the occupants of the vehicle and a shielding portion capable of shielding the imaging surface, and a control unit that controls the imaging unit, the control unit having an occupant monitoring unit and a pattern determination unit for detecting the shielding portion, the occupant monitoring unit being configured to perform occupant monitoring processing to detect the occupants from images captured by the imaging unit, the shielding portion being configured to be displaceable between a shielding position that shields the imaging surface facing it and an open position that is retracted from the shielding position, and the inner surface facing the imaging surface when positioned in the shielding position The device has a periodic pattern for detecting occluded areas, and the pattern determination unit for detecting occluded areas is configured to perform an occluded area detection process to detect the position of the occluded area by comparing first data corresponding to the periodic pattern detected from the image captured by the imaging unit with second data corresponding to the pattern for detecting occluded areas, wherein the pattern determination unit for detecting occluded areas determines that the occluded area is in the occluded position when the first data and the second data correspond, and determines that the occluded area is not in the occluded position when the first data and the second data do not correspond. [Effects of the Invention]

[0008] The shielding detection system and occupant monitoring device according to the present invention have the effect of being able to properly image occupants while protecting their privacy. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing a crew monitoring device according to an embodiment. [Figure 2] Figure 2 is an exploded perspective view showing the imaging unit of the embodiment. [Figure 3] Figure 3 is a perspective view showing the imaging unit of the embodiment. [Figure 4] Figure 4 shows the pattern for detecting the shielding portion in the embodiment. [Figure 5] Figure 5 shows an image of the shielding detection pattern of the embodiment. [Figure 6] Figure 6 shows the relationship between brightness and position in an image of the shielding detection pattern of the embodiment. [Figure 7] Figure 7 is a flowchart showing the shielding detection process of the embodiment. [Figure 8] Figure 8 shows the shielding detection pattern for the first modified example. [Figure 9] Figure 9 shows the relationship between brightness and time in an image of the shielding detection pattern of the first modified example. [Figure 10] Figure 10 shows a pattern for detecting the shielding portion of a second modified example. [Figure 11] Figure 11 shows a pattern for detecting the shielding portion of a third modified example. [Figure 12] Figure 12 shows the relationship between the position of the shielding portion and time in the third modified example. [Modes for carrying out the invention]

[0010] The shielding detection system according to an embodiment of the present invention will be described in detail below with reference to the drawings. However, this embodiment does not limit the present invention. Furthermore, the components in the following embodiment include those that are easily conceivable by those skilled in the art or that are substantially identical.

[0011] [Embodiment] An embodiment will be described with reference to Figures 1 to 7. This embodiment relates to an occluding detection system and a crew monitoring device. Figure 1 is a schematic configuration diagram showing the crew monitoring device according to the embodiment. Figure 2 is an exploded perspective view showing the imaging unit of the embodiment. Figure 3 is a perspective view showing the imaging unit of the embodiment. Figure 4 is a diagram showing the occluding detection pattern of the embodiment. Figure 5 is a diagram showing an image captured of the occluding detection pattern of the embodiment. Figure 6 is a diagram showing the relationship between brightness and position in the image captured of the occluding detection pattern of the embodiment. Figure 7 is a flowchart showing the occluding detection process of the embodiment.

[0012] The shielding part detection system 1 according to the embodiment shown in FIG. 1 is included in the occupant monitoring device 100 mounted on a vehicle, and is a system for specifying the position of a shielding part 30 provided in an imaging part 10 described later. The occupant monitoring device 100 monitors a monitoring target person (occupant) in the vehicle interior and performs various processes when the state of the monitoring target person is abnormal. The occupant monitoring device is, for example, a driver monitoring system (DMS: Driver Monitoring System).

[0013] As shown in FIG. 1, the occupant monitoring device 100 according to the embodiment includes an imaging part 10 and a control part 20. The imaging part 10 captures a still image or a moving image (hereinafter simply referred to as an "image"). In the embodiment, the imaging part 10 is mounted on a vehicle and captures an image including a driver who is a monitoring target person in the vehicle interior. For example, the imaging part 10 is mounted on the roof part of the vehicle. Also, the imaging part 10 may be built in a pillar of the vehicle, an in-vehicle meter, or an in-vehicle display. The imaging part 10 is activated when the ACC (accessory) power supply or the IG (ignition) power supply of the vehicle is turned on, and captures an image of the monitoring target person of the vehicle until these power supplies are turned off.

[0014] The control part 20 is a member that controls the imaging part 10. The control part 20 of the embodiment includes an imaging control part 21, an occupant monitoring part 22, and a shielding part detection pattern determination part 23. In the embodiment, among the components of the occupant monitoring device 100, the imaging part 10, the imaging control part 21, and the shielding part detection pattern determination part 23 constitute the shielding part detection system 1.

[0015] As shown in FIG. 2, the imaging part 10 includes a camera substrate 11, a front cover 12a, a back cover 12b, an optical filter 16, and a shielding part 30.

[0016] The camera board 11 is a so-called printed circuit board (Printed Circuit Board) on which various electronic components and optical components are mounted and which constitutes an electronic circuit for electrically connecting the electronic components. The base material of the camera board 11 is, for example, an insulating layer made of an insulating material such as epoxy resin, glass epoxy resin, paper epoxy resin or ceramic, on which a wiring pattern (printed pattern) is formed (printed) by a conductive member such as a copper foil. In the embodiment, the camera board 11 is formed in a rectangular plate shape.

[0017] In the imaging unit 10 of the embodiment, a camera module 11a including an optical component and a light source (infrared LED) 11b are mounted on one main surface of the camera board 11, and are electrically connected to other electronic components mounted on the camera board 11. The camera module 11a is, for example, a near-infrared camera, and is mounted substantially at the center of one main surface of the camera board 11. The light source 11b is mounted in the vicinity of the camera module 11a on one main surface of the camera board 11. For example, the camera module 11a is connected to the control unit 20 via a connector (not shown) provided in the camera module 11a. The imaging unit 10 receives the reflected light of the near-infrared light irradiated from the light source 11b toward the monitoring target person, and images the monitoring target person.

[0018] The front cover 12a and the back cover 12b are members that constitute the housing (camera module case) of the imaging unit 10. The front cover 12a covers one main surface of the camera board 11 on which the camera module 11a and the light source 11b etc. are mounted, and the back cover 12b is arranged to cover the other main surface of the camera board 11. Then, the housing of the imaging unit 10 is constituted by assembling the front cover 12a and the back cover 12b to each other. An opening is formed in the front cover 12a at a position corresponding to the camera module 11a and the light source 11b, and the camera module 11a and the light source 11b are exposed to the outside of the housing.

[0019] The optical filter 16 is an optical component that selectively transmits light of a specific wavelength. The optical filter 16 is provided in front of the camera module 11a and the light source 11b, and selectively transmits light of a specific wavelength incident on the camera module 11a. The optical filter 16 in this embodiment is a visible light cut filter that transmits near-infrared light and absorbs light of wavelengths in the visible light region. The optical filter 16 is a rectangular plate-shaped filter, and the front cover 12a has a recess into which the optical filter 16 can be fitted. In this recess, there is an opening in the front cover 12a that exposes the camera module 11a and the light source 11b to the outside of the housing of the imaging unit 10. The optical filter 16 is attached to the recess in the front cover 12a and fixed to the front cover 12a so as to cover the camera module 11a and the light source 11b.

[0020] The imaging unit 10 has an imaging surface 10a and is mounted on the vehicle so that the imaging surface 10a faces the person being monitored (occupant) inside the vehicle, in order to capture images of the person being monitored (occupant) inside the vehicle using the camera module 11a. In this embodiment, the imaging surface 10a is the surface of the optical filter 16 (see Figure 3).

[0021] The shielding portion 30 is a plate-shaped member that shields the imaging surface 10a. The shielding portion 30 is located on the front surface of the front cover 12a. As shown in Figure 3, the shielding portion 30 is configured to be displaceable between a shielding position that faces the imaging surface 10a and shields the imaging surface 10a, and an open position that is retracted from the shielding position.

[0022] For example, the shielding section 30 is a sliding shutter and is mounted on the front cover 12a so as to be displaceable between a shielded position and an open position by guide rails 15a and 15b provided on the front cover 12a. Guide rail 15a is a rail-shaped member provided on the upper part of the main surface of the front cover 12a and extending in the width direction of the imaging section 10. Guide rail 15b is a rail-shaped member provided on the lower part of the main surface of the front cover 12a and extending in the width direction of the imaging section 10. The shielding section 30 is mounted between guide rails 15a and 15b.

[0023] The shielding section 30 is provided to protect the privacy of the person being monitored when the imaging unit 10 is in operation. The person being monitored can, at their own discretion, move the position of the shielding section 30 between the open position and the shielded position. In addition, the shielding section 30 can protect the imaging surface 10a by being positioned in the shielded position when the imaging unit 10 is not in use.

[0024] As shown in Figure 3, in this embodiment, the shielding portion 30 in the open position is positioned at one end in the width direction of the imaging portion 10 so as not to overlap with the imaging surface 10a.

[0025] In this embodiment, the movement of the shielding portion 30 from the open position to the shielded position is performed manually by the person being monitored in the vehicle. The shielding portion 30 is provided with a pinch portion 30a so that the driver DR can manually move the shielding portion 30. The pinch portion 30a is formed at the upper end of the shielding portion 30 by causing a part of the shielding portion 30 to protrude upward. The driver DR can move the shielding portion 30 from the open position to the shielded position by pinching the pinch portion 30a with their fingertips and pulling the shielding portion 30 in the direction of arrow Y0 shown in Figure 3. When the end of the shielding portion 30 reaches the shielded position, the entire imaging surface 10a is covered by the shielding portion 30. The person being monitored can also move the shielding portion 30 from the imaging surface 10a to the open position by moving the shielding portion 30 from the shielded position in the opposite direction to arrow Y1 shown in Figure 3. With this configuration, the shielding portion 30 of the embodiment is displaceable between a shielded position and an open position.

[0026] The shielding portion 30 has a periodic shielding portion detection pattern PA1 on its inner surface facing the imaging surface 10a when the shielding portion 30 is positioned in the shielding position. As shown in Figure 4, the shielding portion detection pattern PA1 in this embodiment is a striped pattern in which white lines extending in the vertical direction and black lines extending in the vertical direction are arranged alternately in the width direction. The white lines are set to be relatively thicker in the width direction than the black lines. The white lines and black lines constituting the shielding portion detection pattern PA1 are arranged at equal intervals and form a periodic pattern.

[0027] The control unit 20 in this embodiment is a control board equipped with a program for controlling the occupant monitoring device. The control unit 20 may also be integrated into an in-vehicle meter or display mounted in the vehicle.

[0028] As described above, the control unit 20 of the embodiment is configured to include an image capture control unit 21, a crew monitoring unit 22, and a pattern determination unit 23 for detecting occupant areas. The image capture control unit 21 is configured to control camera settings (camera setting parameters) such as shutter speed, aperture, white balance, and gain of the image capture unit 10. The crew monitoring unit 22 is configured to perform crew monitoring processing to detect crew members from images captured by the image capture unit 10.

[0029] In the occupant monitoring process of this embodiment, the occupant monitoring unit 22 captures an image of the person to be monitored (for example, the driver of the vehicle) with the imaging unit 10 and detects the face of the person to be monitored from the captured image. If the face of the person to be monitored is detected from the image captured by the imaging unit 10 during the occupant monitoring process, the occupant monitoring device 100 determines the state of the person to be monitored and takes appropriate action according to the result of the state determination. For example, if it is determined that the driver, who is the person to be monitored, is in a state where driving is difficult due to illness or other reasons, the device takes action such as notifying the person to stop driving the vehicle.

[0030] The shielding area detection pattern determination unit 23 is configured to perform a shielding area detection process. The shielding area detection process determines the position of the shielding area 30 based on whether or not the shielding area detection pattern PA1 of the shielding area 30 has been detected. As described above, the shielding area 30 of the embodiment has a shielding area detection pattern PA1. In the shielding area detection process, the shielding area detection pattern determination unit 23 performs imaging by the imaging unit 10 using a pre-set shielding area detection camera setting in order to image the shielding area detection pattern PA1.

[0031] The camera settings for detecting the occupant (camera setting parameters for detecting the occupant) are different from the camera settings (camera setting parameters) used when the imaging unit 10 images occupants, etc., when the occupant monitoring device 100 is in the open position. The camera settings when the occupant monitoring device 100 monitors the occupants of the vehicle are set so that the gain, white balance, etc. are automatically adjusted to respond to changes in brightness when the vehicle is in motion. On the other hand, when detecting the occupant detection pattern PA1, it is preferable to always image under constant conditions (constant camera settings) using dedicated camera settings (camera setting parameters) without automatically adjusting the gain, white balance, etc. For this reason, a predetermined constant camera setting is used as the camera setting for detecting the occupant.

[0032] Figure 5 shows an image taken by the imaging unit 10 of the inner surface of the shielding part 30 (shielding part detection pattern PA1) when the shielding part 30 is in the shielding position during the shielding part detection process. Even if the imaging unit 10 used in the crew monitoring device does not have a function to focus on the shielding part detection pattern PA1 which is located close to the imaging surface 10a, when the imaging unit 10 images the shielding part detection pattern PA1, a blurred image IM1 as shown in Figure 5 is captured.

[0033] In the captured image IM1, a difference arises between the brightness of the portion corresponding to the black line and the portion corresponding to the white line of the occlusion detection pattern PA1. For example, as shown in Figure 5, the portion corresponding to the black line is captured with relatively low brightness, while the portion corresponding to the white line is captured with relatively high brightness. As a result, the occlusion detection pattern PA1 is captured as an image containing a periodic pattern in which portions with relatively low brightness and portions with relatively high brightness are arranged alternately along the width direction.

[0034] The pattern determination unit 23 for detecting occlusions detects a periodic pattern from the image IM1 after imaging is performed by the imaging unit 10, and acquires first data corresponding to the detected periodic pattern. In this embodiment, the pattern determination unit 23 detects a periodic pattern by performing spatial frequency analysis on the brightness of the image IM1, and acquires first data corresponding to the detected periodic pattern.

[0035] Figure 6 shows the results of the spatial frequency analysis of image IM1. In Figure 6, "Brightness" corresponds to the brightness of image IM1, and "Position" corresponds to the position of image IM1 in the width direction. By performing spatial frequency analysis on image IM1, a graph of waves (sine waves) like the one shown in Figure 6 can be obtained.

[0036] In Figure 6, multiple peaks P1 in the wave graph correspond to the centers of the white lines of the shielding detection pattern PA1 in the width direction, and multiple bottoms b1 in the wave graph correspond to the centers of the black lines of the shielding detection pattern PA1 in the width direction. In this embodiment, the shielding detection pattern determination unit 23 acquires the number of peaks P1 as the first data. The first data may also be the number of bottoms b1.

[0037] The control unit 20 of this embodiment stores second data corresponding to the shielding detection pattern PA1. The second data is data corresponding to the shielding detection pattern PA1. In this embodiment, the second data is the number of peaks that can be obtained by subjecting the image acquired by imaging the shielding detection pattern PA1 at the shielded position with the imaging unit 10 using the shielding detection camera setting to spatial frequency analysis. If the number of bottoms b1 is used as the first data, the number of bottoms corresponding to the shielding detection pattern PA1 is used as the second data.

[0038] In the shielding area detection process, the shielding area detection pattern determination unit 23 detects the position of the shielding area 30 by comparing the first data and the second data. The shielding area detection pattern determination unit 23 determines that the shielding area 30 is in a shielding position if the first data and the second data correspond, and determines that the shielding area 30 is not in a shielding position if the first data and the second data do not correspond.

[0039] In this embodiment, the shielding portion 30 is determined to be in a shielded position if the number of peaks P1 in the first data and the number of peaks in the second data substantially match. If the number of peaks P1 in the first data is 0, the shielding portion 30 is determined to be in an open position. Furthermore, if the number of peaks P1 in the first data is 1 or more and less than the number of peaks in the second data, the shielding portion 30 is determined to be in a partially shielded position. The shielding portion detection system 1 may then detect the extent to which the imaging surface 10a is shielded by the shielding portion 30 based on the number of peaks P1 at this time. Note that a partially shielded position refers to a position between the shielded position and the open position.

[0040] Next, an example of the occlusion detection process in the occlusion detection system will be explained using Figure 7. As shown in Figure 7, first, the occlusion detection pattern determination unit 23 reads the occlusion detection camera settings for capturing the occlusion detection pattern PA1 (step S1). Then, the occlusion detection pattern determination unit 23 captures an image using the imaging unit 10 and acquires the image (step S2).

[0041] Then, the pattern determination unit 23 for detecting occlusions performs spatial frequency analysis on the acquired image (step S3). The pattern determination unit 23 then detects a periodic pattern related to the brightness of the image through spatial frequency analysis, calculates the number of brightness peaks from the detected periodic pattern, and acquires the calculated number of peaks as first data (step S4).

[0042] Then, the shielding detection pattern determination unit 23 determines whether the number of peaks as the first data acquired in step S4 and the number of peaks as the second data are substantially equal (step S5). Here, the number of peaks as the second data is the number of peaks acquired when the shielding part 30 is in the shielding position (number of peaks at the shielding position).

[0043] In step S5, if the number of peaks as the first data and the number of peaks as the second data are substantially equal, the shielding detection pattern determination unit 23 determines that the shielding portion 30 is in a shielding position (step S6). On the other hand, in step S5, if the number of peaks as the first data and the number of peaks as the second data are different, the shielding detection pattern determination unit 23 determines whether the number of peaks as the first data is 0 or not (step S7).

[0044] In step S7, if the number of peaks as the first data is 0, the shielding detection pattern determination unit 23 determines that the shielding portion 30 is in the open position. On the other hand, in step S7, if the number of peaks as the first data is not 0, the shielding detection pattern determination unit 23 determines that the shielding portion 30 is in the partially shielded position. The shielding detection pattern determination unit 23 determines the position of the shielding portion 30 in the manner described above.

[0045] In this embodiment, the shielding portion 30 may be configured to be automatically displaceable between a shielded position and an open position. In this case, as shown in Figures 2 and 3, the imaging unit 10 includes a switching unit 14 as a shielding portion movement mechanism that displaces the position of the shielding portion 30 between a shielded position and an open position. Furthermore, if a switching unit 14 is provided, the occupant monitoring device may be configured so that when the power of the occupant monitoring device is turned ON, the shielding portion 30 automatically moves to the open position, and when the power of the occupant monitoring device is turned OFF, the shielding portion 30 automatically moves to the shielded position. In this embodiment, a switch or the like may be provided by the switching unit 14 that allows the position of the shielding portion 30 to be displaced between a shielded position and an open position at the discretion of the monitored person DR.

[0046] Furthermore, in the embodiment, if the pattern determination unit 23 for detecting the shielding portion determines that the shielding portion 30 is in a shielding position, and detection by the imaging unit 10 is necessary (for example, when the vehicle is in motion or when monitoring and recording of the driver DR is necessary), the control unit 20 may notify the driver DR that the imaging surface 10a is shielded. Also, if the position of the shielding portion 30 can be automatically shifted between a shielded position and an open position, the control unit 20 may use the switching unit 14 to automatically shift the position of the shielding portion 30 from a shielded state to an open state.

[0047] As described above, the shielding detection system 1 according to the embodiment comprises an imaging unit 10 having an imaging surface 10a mounted on a vehicle and positioned facing the occupants of the vehicle, and a shielding unit 30 capable of shielding the imaging surface 10a, and a control unit 20 that controls the imaging unit 10, wherein the shielding unit 30 is configured to be displaceable between a shielding position that faces and shields the imaging surface 10a and an open position that is retracted from the shielding position, and has a periodic shielding detection pattern PA1 on the inner surface facing the imaging surface 10a when positioned in the shielding position, and the control unit 20 controls the shielding detection pattern The device has a fixed unit 23, and the pattern determination unit 23 for detecting occlusion is configured to perform an occlusion detection process to detect the position of the occlusion 30 by comparing first data corresponding to a periodic pattern detected from an image captured by the imaging unit 10 with second data corresponding to an occlusion detection pattern PA1. In the occlusion detection process, the pattern determination unit 23 for detecting occlusion determines that the occlusion 30 is in an occlusion position if the first data and the second data correspond, and determines that the occlusion 30 is not in an occlusion position if the first data and the second data do not correspond.

[0048] The shielding detection system 1 according to the embodiment can protect the privacy of the occupants by covering the imaging surface 10a with the shielding part 30. Furthermore, the shielding detection system 1 according to the embodiment can detect the shielding part 30 with high accuracy by comparing the first data with the second data using the shielding part detection pattern determination unit 21, even without providing a mechanism such as a switch on the imaging unit 10 to detect the opening and closing of the shielding part 30. With this configuration, the shielding detection system 1 according to the embodiment can properly image the occupants while protecting their privacy with the shielding part 30 provided on the imaging unit 10.

[0049] Furthermore, in the shielding detection system 1 according to the embodiment, the shielding detection pattern determination unit 23 performs imaging by the imaging unit 10 using a pre-set shielding detection camera setting to image the shielding detection pattern PA1 during the shielding detection process. The shielding detection camera setting is a different camera setting from the camera setting used when the imaging unit 10 performs imaging when the shielding unit 30 is in the open position.

[0050] In the embodiment of the shielding detection system 1, imaging can be performed by the imaging unit 10 under certain conditions during the shielding detection process, enabling detection of the shielding unit 30 with stable accuracy.

[0051] Furthermore, the occupant monitoring device 100 according to the embodiment comprises an imaging unit 10 having an imaging surface 10a mounted on a vehicle and positioned facing the occupants of the vehicle, and a shielding part 30 capable of shielding the imaging surface 10a, and a control unit 20 that controls the imaging unit 10, the control unit 20 having an occupant monitoring unit 22 and a shielding part detection pattern determination unit 23, the occupant monitoring unit 22 is configured to perform occupant monitoring processing to detect occupants from images captured by the imaging unit 10, the shielding part 30 is configured to be displaceable between a shielding position that faces the imaging surface 10a and shields the imaging surface 10a and an open position that is retracted from the shielding position, and when positioned in the shielding position the imaging surface The inner surface facing 10a has a periodic occlusion detection pattern PA1, and the occlusion detection pattern determination unit 23 is configured to perform occlusion detection processing to detect the position of the occlusion 30 by comparing first data corresponding to a periodic pattern detected from an image captured by the imaging unit 10 with second data corresponding to the occlusion detection pattern PA1. In the occlusion detection processing, the occlusion detection pattern determination unit 23 determines that the occlusion 30 is in an occlusion position if the first data and the second data correspond, and determines that the occlusion 30 is not in an occlusion position if the first data and the second data do not correspond.

[0052] The occupant monitoring device 100 according to this embodiment can protect the privacy of occupants by covering the imaging surface 10a with the shielding portion 30. Furthermore, the occupant monitoring device 100 according to this embodiment can detect the shielding portion 30 with high accuracy by comparing the first data with the second data using the shielding portion detection pattern determination unit 21, even without providing a mechanism such as a switch on the imaging unit 10 to detect the opening and closing of the shielding portion 30. With this configuration, the occupant monitoring device 100 according to this embodiment can properly image occupants while protecting their privacy with the shielding portion 30 provided on the imaging unit 10.

[0053] [Modified Example of Embodiment 1] A first modified example of the embodiment will now be described. Figure 8 shows the pattern for detecting the occluded portion of the first modified example. Figure 9 shows the relationship between brightness and time in an image of the occluded portion detection pattern of the first modified example. The difference between the occluded portion detection system 1 according to the first modified example and the above embodiment is that in the occluded portion detection process, the image IM1 is acquired using a line scan method. Another difference from the above embodiment is that the occluded portion 30 moves not in the width direction, but in the vertical direction (the direction along the arrow Y1 in Figure 8), causing the position of the occluded portion 30 to be displaced between the occluded position and the open position.

[0054] As shown in Figure 8, the shielding detection pattern PA2 of the first modified example is a striped pattern in which white lines extending in the width direction and black lines extending in the width direction are arranged alternately in the vertical direction. The white lines are set to be relatively thicker than the black lines in the vertical direction. The white lines and black lines that make up the shielding detection pattern PA2 are arranged at equal intervals, forming a periodic pattern.

[0055] In the first modified example, the shielding detection process is performed when the shielding part 30 moves from the open position. For example, the movement of the shielding part 30 from the open position is detected by a mechanism such as a sensor or switch provided in the imaging unit 10. In the shielding detection process, the shielding detection pattern determination unit 23 captures the shielding detection pattern PA2 using a line scan method and detects a periodic pattern from the captured image.

[0056] Specifically, by imaging a portion of the area that the imaging unit 10 can image, specifically the area SN1 that the shielding detection pattern PA2 passes through when the shielding unit 30 moves from the open position to the shielded position, for a predetermined time, an image similar to the image IM1 shown in Figure 5 can be obtained. In the first modified example, spatial frequency analysis is performed on this image to obtain the first data. Figure 9 shows the results of the spatial frequency analysis. In Figure 9, "Brightness" corresponds to the brightness of the image captured using the line scan method, and "Time" corresponds to the time the image was captured using the line scan method. When the shielding unit 30 is in the shielded position, a wave (sine wave) graph like the one shown in Figure 9 is obtained by performing spatial frequency analysis.

[0057] In Figure 9, the multiple peaks P1 included in the wave graph correspond to the center of the white line of the shielding detection pattern PA2 in the vertical direction, and the multiple bottoms b1 of the wave graph correspond to the center of the black line of the shielding detection pattern PA2 in the vertical direction. The other configurations in the first modified example are the same as in the above embodiment. For example, in the first modified example, as in the above embodiment, the shielding detection pattern determination unit 23 acquires the number of peaks P1 as first data. The first data may also be the number of bottoms b1.

[0058] In the first modified example of the shielding part detection system 1, the shielding part detection process is performed when the shielding part 30 moves from the open position, and the shielding part detection pattern determination unit 23 detects a periodic pattern from the image obtained by imaging for a predetermined time a portion of the area that the imaging unit 10 can image, which is the area through which the shielding part detection pattern PA2 passes when the shielding part 30 moves from the open position to the shielded position.

[0059] The first modified example of the shielding detection system 1, even if the imaging unit 10 images only a portion of the area SN1 of the area it can image using a line scan method, can detect the shielding portion 30 with high accuracy by comparing the first data and second data obtained by imaging the shielding portion detection pattern PA2. This configuration allows for proper imaging of occupants while protecting their privacy with the shielding portion 30 provided on the imaging unit 10.

[0060] [Second modified example of the embodiment] A second modified example of the embodiment will now be described. Figure 10 shows a pattern for detecting occlusion in the second modified example. In the occlusion detection system 1 according to the second modified example, the difference from the above embodiment is that, similar to the first modified example, the image IM1 is acquired using a line scan method in the occlusion detection process.

[0061] Furthermore, the shielding portion 30 is formed in a circular shape, and the shielding portion detection pattern PA3 is a pattern in which black circular lines and white circular lines are alternately arranged in the radial direction, which is also different from the above embodiment. In the second modified example, the shielding portion 30 is configured to be automatically displaceable between a shielded position and an open position by the switching portion 14.

[0062] The shielding portion 30 of the second modification is applied, for example, when the imaging surface 10a is circular in shape. In the open position, the shielding portion 30 of the second modification is positioned to surround the opening that exposes the imaging surface 10a. The shielding portion 30 of the second modification is positioned in the shielded position by moving along the arrow Y2 shown in Figure 10, reducing the size of the opening from the outside to the inside in the radial direction of the opening.

[0063] In the second modified example, the imaging unit 10 can acquire an image similar to the image IM1 shown in Figure 5 by imaging a portion of the area that the imaging unit 10 can image, specifically the area SN2 that the shielding detection pattern PA2 passes through when the shielding unit 30 moves from the open position to the shielded position, for a predetermined time. For example, area SN2 is a circumferentially shaped area along the outer circumference of the opening that exposes the imaging surface 10a. The other configurations in the second modified example are the same as those in the first modified example.

[0064] [Third Modification of the Embodiment] A third modified embodiment will now be described. Figure 11 is a diagram showing the pattern for detecting the occluded portion in the third modified embodiment. Figure 12 is a diagram showing the relationship between the position of the occluded portion and time in the third modified embodiment. In the occluded portion detection system 1 according to the third modified embodiment, the difference from the above embodiment is that, similar to the first modified embodiment, in the occluded portion detection process, the region SN1 is imaged using a line scan method to acquire the image IM1.

[0065] Furthermore, the imaging unit 10 includes a shielding unit movement mechanism that displaces the position of the shielding unit 30 between the shielded position and the open position, and the shielding unit 30 is configured to be automatically displaceable between the shielded position and the open position, which is another difference from the above embodiment. In addition, the period of the shielding unit detection pattern PA4 is corrected to a period corresponding to the change in the speed of the shielding unit 30 that occurs when the shielding unit 30 moves from the open position to the shielded position by the shielding unit movement mechanism, which is another difference from the above embodiment. The shielding unit 30 moves from the open position to the shielded position by moving along the direction of arrow Y3 shown in Figure 11. In the third modified example, the movement speed of the shielding unit 30 by the shielding unit movement mechanism is not constant.

[0066] In Figure 12, the horizontal axis represents the "time" it takes for the shielding part 30 to move from the open position to the shielded position by the shielding part movement mechanism, and the vertical axis represents the "position" of the shielding part 30. The data SP in Figure 12 shows the relationship between "time" and "position" when the shielding part 30 moves from the open position to the shielded position. As shown in Figure 12, the movement speed of the shielding part 30 moved by the shielding part movement mechanism increases as time progresses.

[0067] In the third modified example, the period of the shielding detection pattern PA4 is corrected to a period corresponding to the change in the speed of the shielding part 30 that occurs when the shielding part 30 moves from the open position to the shielded position by the shielding part moving mechanism, so that the first data and the second data obtained by imaging the area SN1 shielding detection pattern PA4 using a line scan method correspond to each other.

[0068] Specifically, as shown in Figure 11, the shielding detection pattern PA4 is a striped pattern in which white lines extending vertically and black lines extending vertically are arranged alternately in the width direction. Among the multiple black lines, the black lines closer to the open position are set to be thicker. Similarly, among the multiple white lines, the white lines closer to the open position are set to be thinner. The other configurations of the third modified example are the same as those of the first modified example described above.

[0069] In the third modified example of the shielding part detection system 1, the imaging unit 10 includes a shielding part moving mechanism that displaces the position of the shielding part 30 between a shielded position and an open position based on control by the control unit 20, and the period of the shielding part detection pattern PA4 is corrected to a period corresponding to the change in the speed of the shielding part 30 that occurs when the shielding part 30 moves from the open position to the shielded position by the shielding part moving mechanism, such that the first data and second data obtained by imaging the shielding part detection pattern PA4 correspond to each other.

[0070] The shielding detection system 1 according to the third modified example can automatically position the shielding part 30 in the shielding position using the shielding part moving mechanism, and can perform the shielding part detection process smoothly. Furthermore, by correcting the period of the shielding part detection pattern PA4 to correspond to the change in the speed of the shielding part 30 that occurs when the shielding part 30 moves from the open position to the shielding position by the shielding part moving mechanism, the shielding part 30 can be detected with high accuracy even when the speed of the shielding part 30 is fast or when the speed change is large.

[0071] The embodiments and modifications disclosed above can be combined and implemented as appropriate. [Explanation of Symbols]

[0072] 1: Anomaly detection system; 10: Imaging unit; 10a: Imaging surface 11: Camera circuit board, 12a: Front cover, 12b: Back cover, 14: Switching section, 16: Optical filter 20: Control Unit 21: Imaging Control Unit 22: Crew Monitoring Unit 23: Pattern determination unit for detecting shielding part 30: Shielding part 100: Crew monitoring device PA1, PA2, PA3, PA4: Patterns for detecting shielding areas; IM1: Image

Claims

1. An imaging unit having an imaging surface mounted on a vehicle and positioned facing the occupants of the vehicle, and a shielding portion capable of shielding the imaging surface, A control unit that controls the imaging unit, Equipped with, The shielding portion is configured to be displaceable between a shielding position that shields the imaging surface in opposition to the imaging surface and an open position that is retracted from the shielding position, and has a periodic pattern for detecting the shielding portion on the inner surface facing the imaging surface when positioned in the shielding position. The control unit has a pattern determination unit for detecting the shielding portion, The occlusion detection pattern determination unit is configured to perform an occlusion detection process by comparing first data corresponding to a periodic pattern detected from an image captured by the imaging unit with second data corresponding to the occlusion detection pattern, thereby detecting the position of the occlusion. In the shielding portion detection process, the shielding portion detection pattern determination unit determines that the shielding portion is in the shielding position when the first data and the second data correspond, and determines that the shielding portion is not in the shielding position when the first data and the second data do not correspond. Shielding detection system.

2. The shielding detection process is performed when the shielding moves from the open position. The pattern determination unit for detecting the occluding portion detects the periodic pattern from an image obtained by capturing for a predetermined time a portion of the area that the imaging unit can capture, which is the area through which the pattern for detecting the occluding portion passes when the occluding portion moves from the open position to the occluding position, during the occluding portion detection process. The shielding detection system according to claim 1.

3. The imaging unit includes a shielding unit moving mechanism that, based on control by the control unit, displaces the position of the shielding unit between the shielding position and the open position. The period of the pattern for detecting the shielding portion is corrected to a period corresponding to the change in the speed of the shielding portion that occurs when the shielding portion moves from the open position to the shielded position by the shielding portion moving mechanism, such that the first data and the second data obtained by imaging the pattern for detecting the shielding portion correspond to each other. The shielding detection system according to claim 2.

4. The pattern determination unit for detecting the occluding portion performs imaging by the imaging unit using the camera settings for detecting the occluding portion, which are set in advance in order to capture the pattern for detecting the occluding portion during the occluding portion detection process. The camera setting for detecting the shielding portion is different from the camera setting used when the imaging unit takes an image when the shielding portion is in the open position. A shielding detection system according to any one of claims 1 to 3.

5. An imaging unit having an imaging surface mounted on a vehicle and positioned facing the occupants of the vehicle, and a shielding portion capable of shielding the imaging surface, A control unit that controls the imaging unit, Equipped with, The control unit includes a crew monitoring unit and a pattern determination unit for detecting shielding units. The crew monitoring unit is configured to detect the crew from the image captured by the imaging unit and perform crew monitoring processing. The shielding portion is configured to be displaceable between a shielding position that shields the imaging surface in opposition to the imaging surface and an open position that is retracted from the shielding position, and has a periodic pattern for detecting the shielding portion on the inner surface facing the imaging surface when positioned in the shielding position. The occlusion detection pattern determination unit is configured to perform an occlusion detection process by comparing first data corresponding to a periodic pattern detected from an image captured by the imaging unit with second data corresponding to the occlusion detection pattern, thereby detecting the position of the occlusion. In the shielding portion detection process, the shielding portion detection pattern determination unit determines that the shielding portion is in the shielding position when the first data and the second data correspond, and determines that the shielding portion is not in the shielding position when the first data and the second data do not correspond. Crew monitoring device.

Images