Image processing device, image processing method, and computer-readable storage medium

Abstract

This invention provides an image processing apparatus, an image processing method, and a computer-readable storage medium. The image processing apparatus (100) includes a light irradiation unit (11), an image acquisition unit (31), a motion detection unit (33), and a shooting control unit (34). The light irradiation unit (11) irradiates light onto the face of a subject. The image acquisition unit (31) acquires an image of the face of the subject. The motion detection unit (33) detects the speed of movement of at least a portion of the face from the image acquired by the image acquisition unit (31). The shooting control unit (34) adjusts the shutter opening period in the image acquisition unit (31) and the light emission intensity of the light irradiation unit (11) based on the speed detected by the motion detection unit (33).

Description

Technical Field

[0001] The invention relates to an image processing device, an image processing method and a computer-readable storage medium. Background Art

[0002] For example, in an image processing device such as a driver monitor mounted in a vehicle, an image captured in the vehicle cabin is stored in a storage device. The image processing device stores the conditions in the vehicle cabin as an image, or stores images centered on the driver.

[0003] For example, Patent Document 1 discloses an anti-shake camera that appropriately controls the use of a flash. The anti-shake camera detects the amount of camera hand shake through a shake amount detection unit, and determines the shutter speed when the flash is not used based on the brightness of the subject through a shutter speed determination unit. The anti-shake camera predicts the amount of shake relative to the shutter speed when the flash is not used determined by the shutter speed determination unit through a shake amount prediction unit. The anti-shake camera determines whether to fire the flash through a flash emission determination unit based on the output of the shake amount prediction unit.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent document 1: Japanese Patent Application Laid-Open No. 06-313918. Summary of the invention

[0007] The conventional anti-shake camera described in Patent Document 1 appropriately controls the flash to emit or not emit light, but when the flash emits light, the light intensity is not variable in a manner of increasing or decreasing. For example, when the face of a vehicle driver is photographed by irradiating infrared light or other light with a certain light intensity in a driver monitor, it causes a burden on the driver's eyes. The inventors of the present invention have noticed that when detecting the movement of the eye contour or the like in order to analyze the state of the eye opening and closing of the vehicle driver or the like, the shutter opening period and the light intensity are changed according to the movement speed, thereby reducing the burden on the eyes.

[0008] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing device, an image processing method, and a computer-readable storage medium that can reduce the burden on eyes caused by irradiation with light.

[0009] An image processing device according to one embodiment of the present invention includes: a light irradiation unit for irradiating light onto the face of a subject; an image acquisition unit for acquiring an image obtained by photographing the face of the subject; a motion detection unit for detecting the speed of movement of at least a part of the face from the image acquired by the image acquisition unit; and a shooting control unit for changing the opening period of a shutter in the image acquisition unit and the light emission intensity of the light irradiation unit based on the speed detected by the motion detection unit.

[0010] In addition, another mode of the present embodiment is an image processing method. The image processing method includes: a light irradiation step of irradiating light onto the face of the subject; an image acquisition step of acquiring an image of the face of the subject; a motion detection step of detecting the speed of the motion of at least a part of the face from the image acquired by the image acquisition step; and a shooting control step of changing the opening period of the shutter in the image acquisition step and the light emission intensity in the light irradiation step based on the speed detected by the motion detection step.

[0011] In addition, another mode of the present embodiment is a computer-readable storage medium storing an image processing program. The image processing program causes a computer to execute: a light irradiation step of irradiating light onto the face of a subject; an image acquisition step of acquiring an image of the face of the subject; a motion detection step of detecting the speed of motion of at least a part of the face from the image acquired in the image acquisition step; and a shooting control step of changing the shutter opening period in the image acquisition step and the light emission intensity in the light irradiation step based on the speed detected in the motion detection step.

[0012] According to this embodiment, the burden on eyes caused by irradiation with light can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a block diagram showing the structure of the image processing device according to the first embodiment.

[0014] Figure 2 (a) to (c) are schematic diagrams showing examples of motion detection by the motion detection unit.

[0015] Figure 3 This is a timing chart showing a case where the open period of the shutter is shortened.

[0016] Figure 4 This is a timing chart showing a case where the shutter open period is prolonged.

[0017] Figure 5 This is a flowchart showing the procedure of shutter and light intensity change processing in the image processing device.

[0018] Figure 6 This is a block diagram showing the structure of the image processing device according to the second embodiment. DETAILED DESCRIPTION

[0019] Below, based on the preferred embodiment, refer to Figures 1 to 6 The present invention is described. The same or equivalent components and parts shown in the drawings are marked with the same reference numerals, and repeated descriptions are appropriately omitted. In addition, in each drawing, a part of the components that are not important are omitted for describing the embodiment.

[0020] (Implementation Method 1)

[0021] Figure 1 This is a block diagram showing the structure of the image processing device 100 of Embodiment 1. The image processing device 100 is, for example, a driver monitor, which is mounted on a vehicle and continuously captures images of passengers in the vehicle cabin. The image processing device 100 may record the captured images or may discard them without recording them. The image processing device 100 is, for example, arranged on a dashboard below the windshield of the vehicle. The image processing device 100 captures the driver of the vehicle as a subject, and recognizes the entire face of the captured subject, or parts of the face such as the eyes.

[0022] The image processing device 100 irradiates the face of the subject with infrared light or other light to shoot the subject so that the subject can be identified even in dark scenes such as at night or in tunnels. Although the infrared light used for shooting has the property of being difficult to identify visually by the driver, etc., by further suppressing the luminous intensity, the burden on the eyes can be reduced.

[0023] The image captured by the image processing device 100 is used to determine the outline of the left and right eyes, for example, and is used for analysis related to the eye parts such as the degree of opening and closing of the eyes, the inclination of the left and right eyes, etc. In the present embodiment, the analysis function related to the state of the facial parts such as the eyes is omitted, but the image processing device 100 may also be provided with an analysis function related to the degree of opening and closing of the eyes, the inclination of the left and right eyes, etc. In addition, the analysis related to the facial parts may also be performed in an external device based on the image stored or output by the image processing device 100. In addition, the analysis function of the degree of opening and closing of the eyes, the inclination of the left and right eyes, etc. is used, for example, to determine the sleepiness of the subject, determine the emotional state (such as normal state, nervous state, etc.), etc.

[0024] The image processing device 100 includes an image capturing unit 10, a light irradiation unit 11, a recording unit 20, an image processing unit 30, an external input unit 40, an external output unit 41, an image accuracy detection unit 50, etc. The image capturing unit 10 is a camera having a detector such as a CCD, and captures a person such as a driver riding in a vehicle as a subject. The image capturing unit 10 acquires images continuously in time and sends them to an image acquisition unit 31 described later.

[0025] The light irradiation unit 11 appropriately irradiates infrared light or the like to the subject, especially the face part, so that the subject such as the driver can be photographed even in a dark night. The luminous intensity of the light irradiated from the light irradiation unit 11 is set by the shooting control unit 34 of the image processing unit 30 described later. In addition, it is preferred that the luminous period of the light irradiated by the light irradiation unit 11 coincides with the opening period of the shutter, but there may be a slight time deviation between the opening period of the shutter.

[0026] The recording unit 20 is, for example, a removable medium such as an SD card, a USB memory, a hard disk, etc., and can record and delete the image acquired by the image acquisition unit 31. In the following, a configuration provided with the recording unit 20 is described, but in the case where the image processing device 100 does not have a portion for recording images, the recording unit 20 may not be provided. The recording unit 20 can be detached from the image processing device 100, and can be removed from the image processing device 100 to reproduce the image by another PC or the like.

[0027] The external input unit 40 obtains the speed information and position information of the vehicle from an external device. In addition, the external output unit 41 outputs the image information processed by the image processing unit 30 to the external device. The image processing device 100 can also add the speed information and position information obtained by the external input unit 40 to the image for recording or external output.

[0028] The image processing unit 30 includes an image acquisition unit 31, an image recognition unit 32, a motion detection unit 33, and a shooting control unit 34. The image processing unit 30 is composed of, for example, a CPU, and executes the processing of each of the above units by operating according to a computer program. The storage unit 30a is composed of a data storage device such as a RAM (Random Access Memory), a flash memory, and a hard disk storage device, and stores the computer program executed by the image processing unit 30. In addition, the storage unit 30a stores a recognition dictionary for recognizing the face, eyes, etc. of the subject from the captured image.

[0029] The image acquisition unit 31 acquires the image captured by the imaging unit 10, performs data compression and other processing, and outputs it to the recording unit 20. In addition, the image acquisition unit 31 may also be configured to include the imaging unit 10. The opening period of the shutter for imaging the subject in the image acquisition unit 31 is set by the imaging control unit 34 described later. In addition, the frame rate of the image frame is also set by the imaging control unit 34.

[0030] The image recognition unit 32 recognizes the face and eyes of the subject from the image input from the image acquisition unit 31 based on the recognition dictionary stored in the storage unit 30a. The recognition dictionary stored in the storage unit 30a includes shape data of the face, eyes, etc. The image recognition unit 32 extracts the shape pattern shown in the image and compares it with the shape data included in the recognition dictionary to recognize the face, eyes, etc. The image recognition unit 32 can recognize the face, eyes, etc. from the image using a well-known image recognition processing method developed in various fields.

[0031] The motion detection unit 33 detects the motion of at least a part of the face of the subject, and in particular, detects the motion speed. Figure 2 (a) to (c) are schematic diagrams showing examples of motion detection by the motion detection unit 33. Figure 2 As shown in (a) of FIG. 1 , the motion detection unit 33 detects the blinking speed of the eye portion recognized by the image recognition unit 32 and outputs the speed to the imaging control unit 34. Eye blinks are caused by the movement of the eye contour portion moving up and down moment by moment.

[0032] When the blinking speed is slow, the opening and closing of the eye caused by blinking can be recognized even if the opening period of the shutter in the image acquisition unit 31 is extended. On the contrary, when the blinking speed is fast, if the opening period of the shutter in the image acquisition unit 31 is long, the opening and closing of the eye caused by blinking cannot be recognized, so it is necessary to set the opening period of the shutter to be shorter.

[0033] In addition, if Figure 2 As shown in (b), the motion detection unit 33 detects the speed of the line of sight movement in the eye portion recognized by the image recognition unit 32 and outputs it to the imaging control unit 34. The movement of the line of sight of the eye is caused by the pupil portion moving leftward or rightward or up and down moment by moment.

[0034] When the speed of the line of sight is slow, the line of sight movement can be recognized from the captured image even if the shutter opening period in the image acquisition unit 31 is extended. In addition, when the line of sight movement is fast, the shutter opening period in the image acquisition unit 31 needs to be set shorter.

[0035] In addition, if Figure 2 As shown in (c) of FIG. Figure 2 , the motion detection unit 33 detects the motion speed of the overall face shake for the entire face recognized by the image recognition unit 32 and outputs it to the shooting control unit 34. The motion of the face shake is generated by the motion of the contour of the face or a specific part (e.g., the nose part) moving left and right or up and down at all times.

[0036] When the motion speed of the face shake is slow, even if the opening period of the shutter in the image acquisition unit 31 is extended, the motion of the face shake can be image-recognized from the captured image. In addition, when the motion speed of the face shake is fast, it is necessary to set the opening period of the shutter in the image acquisition unit 31 to be smaller.

[0037] The shooting control unit 34 makes settings to change the opening period of the shutter in the image acquisition unit 31 and the light emission intensity of the light irradiation unit 11 based on the motion speed of at least a part of the face of the subject detected by the motion detection unit 33. Figure 3 is a timing chart showing the case where the opening period of the shutter is shortened, Figure 4 is a timing chart showing the case where the opening period of the shutter is extended.

[0038] When the motion speed of at least a part of the face of the subject is fast, as Figure 3 shown, the opening period of the shutter in the image acquisition unit 31 is set to a smaller value L1, and the light emission intensity of the light irradiation unit 11 is set to a larger value P2. The shooting control unit 34 presets a specified threshold for the motion speed of at least a part of the face of the subject. When the motion speed of at least a part of the face input from the motion detection unit 33 is equal to or higher than the threshold, the motion speed is fast, and when it is less than the threshold, the motion speed is slow.

[0039] When the motion speed of at least a part of the face of the subject is slow, as Figure 4 shown, the opening period of the shutter in the image acquisition unit 31 is set to a larger value L2 (L2 > L1), and the light emission intensity of the light irradiation unit 11 is set to a smaller value P1 (P1 < P2). The shooting control unit 34 presets a specified threshold for the motion speed of at least a part of the face of the subject. When the motion speed of at least a part of the face input from the motion detection unit 33 is equal to or higher than the threshold, the motion speed is fast, and when it is less than the threshold, the motion speed is slow. In addition, L1 and L2 for the opening period of the shutter and P1 and P2 for the light emission intensity can also be determined in advance in consideration of a darker environment such as at night and the motion speed of the face of the subject.

[0040] When the speed of movement of at least a part of the face of the subject is slow, the shooting control unit 34 extends the opening period of the shutter in the image acquisition unit 31 and sets the light emission intensity of the light irradiation unit 11 to be small compared to the case where the speed of movement is fast. Alternatively, the shooting control unit 34 may set the opening period of the shutter and the light emission intensity by continuously changing according to the speed of movement of at least a part of the face of the subject.

[0041] As described above, the movement of at least a part of the subject's face includes blinking, movement of the line of sight, and shaking of the face, and the imaging control unit 34 changes and sets the shutter open period and light emission intensity according to these movements.

[0042] The image accuracy detection unit 50 detects the image recognition accuracy of the subject's face, eyes, nose and other parts of the face, and checks whether sufficient image recognition accuracy is ensured. Image recognition accuracy is, for example, a measure of whether the face parts such as eyes and nose, the contour of the face, etc. can be distinguished from the adjacent parts, and specifically, the degree of image blurring of the boundary part is used as an index.

[0043] In addition, when analyzing the degree of opening and closing of the eyes and the inclination of the eyes, it is also possible to confirm whether the image accuracy is ensured to a degree that allows analysis. When analyzing the degree of opening and closing of the eyes and the inclination of the eyes, for example, whether the brightness values ​​of each part of the eyelid, the white of the eye, and the pupil have a difference that can distinguish each part becomes a problem, and the image accuracy can also be specified based on the difference in the brightness value of each part. That is, when the difference in the brightness value of each part is small, the image accuracy is low, and when the difference in the brightness value is large, the image accuracy is high.

[0044] For example, when changing the shutter opening period from a smaller value to a larger value, if the image recognition accuracy is better than a predetermined threshold value, the shooting control unit 34 changes the shutter opening period to a larger value. After changing the shutter opening period to a larger value, if the image recognition accuracy cannot be sufficiently ensured, the shooting control unit 34 returns the shutter opening period to a smaller value. Similarly, for example, when changing the luminous intensity from a larger value to a smaller value, if the image recognition accuracy is better than a predetermined threshold value, the shooting control unit 34 changes the luminous intensity to a smaller value. After changing the luminous intensity to a smaller value, if the image recognition accuracy cannot be sufficiently ensured, the shooting control unit 34 returns the luminous intensity to a larger value.

[0045] In addition, the shooting control unit 34 may change the shutter opening period from a small value to a large value while confirming that the image recognition accuracy in the image accuracy detection unit 50 is good, when the shutter opening period is changed to a large value. Similarly, the shooting control unit 34 may change the light emission intensity from a large value to a small value while confirming that the image recognition accuracy in the image accuracy detection unit 50 is good, when the light emission intensity is changed to a small value.

[0046] Next, the operation of the image processing device 100 based on the shutter and the light emission intensity change process will be described. Figure 5 1 is a flowchart showing the sequence of the shutter and light intensity change processing in the image processing device 100. The light irradiation unit 11 of the image processing device 100 starts irradiating light to the subject (S1), and the image acquisition unit 31 acquires the image of the subject captured by the imaging unit 10 (S2). The image recognition unit 32 performs image recognition on at least a part of the face of the subject, and the motion detection unit 33 detects the motion speed of the recognized part (S3).

[0047] The shooting control unit 34 determines whether the motion speed detected by the motion detection unit 33 is greater than a predetermined threshold value (S4). When the shooting control unit 34 determines that the motion speed of at least a part of the face of the subject is greater than the threshold value (S4: Yes), the shooting control unit 34 sets the shutter opening period in the image acquisition unit 31 to a smaller value and sets the light emission intensity of the light irradiation unit 11 to a larger value (S5), and ends the processing. However, when the shutter opening period is shorter and the light emission intensity is set to a larger value before the determination in step S4, the shooting control unit 34 maintains the setting state.

[0048] When it is determined in step S4 that the movement speed of at least a part of the face of the subject is less than the threshold value (S4: No), the shooting control unit 34 sets the opening period of the shutter in the image acquisition unit 31 to a larger value and sets the light emission intensity of the light irradiation unit 11 to a smaller value (S6), and ends the processing.

[0049] The shooting control unit 34 may also set the shutter opening period to a smaller value and change the light emission intensity of the light irradiation unit 11 to a larger value when the image recognition accuracy detected by the image accuracy detection unit 50 is lower than that of the image acquired after step S6. In addition, as described above, the shooting control unit 34 may also set the shutter opening period and the light emission intensity to be continuously changed according to the movement speed of at least a part of the face of the subject.

[0050] When the movement speed of at least a part of the face of the subject is slower than a predetermined threshold, the image processing device 100 sets the shutter opening period in the image acquisition unit 31 to a larger value and sets the light emission intensity of the light irradiation unit 11 to a smaller value. The image processing device 100 reduces the burden on the eyes of the subject by setting the light emission intensity to a smaller value, and irradiates the subject with the amount of light required for shooting by setting the shutter opening period to a larger value, thereby being able to acquire a bright image.

[0051] When the speed of movement of at least a part of the face of the subject is faster than a predetermined threshold, the image processing device 100 sets the shutter opening period in the image acquisition unit 31 to a smaller value and sets the light emission intensity of the light irradiation unit 11 to a larger value. The image processing device 100 can suppress the reduction of image recognition accuracy when at least a part of the face of the subject moves quickly by setting the shutter opening period to a smaller value, and can acquire a bright image by setting the light emission intensity to a larger value.

[0052] The image processing device 100 detects eye blinking, line of sight movement, and face shaking as the movement of at least a part of the face of the subject through the motion detection unit 33. By changing the shutter opening period and light emission intensity according to the eye blinking speed of the subject, images of the open and closed eyes can be reliably obtained.

[0053] In addition, by changing the shutter opening period and the light intensity according to the speed of the line of sight, it is possible to reliably obtain an image that captures the movement of the line of sight. Similarly, by changing the shutter opening period and the light intensity according to the speed of the shaking of the face, it is possible to reliably obtain an image that captures which direction the subject is facing, left or right or up or down.

[0054] (Implementation Method 2)

[0055] Figure 6 1 is a block diagram showing the structure of the image processing device 100 according to Embodiment 2. The image processing device 100 according to Embodiment 2 further includes a shake amount detection unit 35 in addition to the structure of Embodiment 1. The shake amount detection unit 35 detects the shake amount in the image of the subject caused by the vibration of the vehicle on which the image processing device 100 is mounted, for example, based on the degree of ghosting or shake of the edge portion of the image acquired by the image acquisition unit 31.

[0056] The shooting control unit 34 changes the shutter opening periods L1 and L2, and the light emission intensities P1 and P2 according to the amount of shake detected by the shake amount detection unit 35. When the amount of shake is large, the shooting control unit 34 sets the shutter opening period to be short and the light emission intensity to be large. When the amount of shake is small, the shooting control unit 34 sets the shutter opening period to be long and the light emission intensity to be small.

[0057] The image processing device 100 changes the shutter open periods L1 and L2 and the light emission intensities P1 and P2 according to the shake amount detected by the shake amount detection unit 35 , thereby suppressing a decrease in image recognition accuracy and reducing the burden on the subject's eyes.

[0058] (Variation Example)

[0059] In the above-mentioned embodiments, the shutter opening period and the light emission intensity are made variable within the range of not changing the frame rate of the image, but the frame rate of the image may be further changed. The shooting control unit 34 may also be set to reduce the frame rate of the image acquired by the image acquisition unit 31 within the range of sufficiently ensuring the image recognition accuracy for analyzing the opening and closing degree of the eyes, the inclination of the left and right eyes, etc., when the movement speed of at least a part of the face of the subject is slow and the amount of shaking is small.

[0060] The image processing device 100 can further reduce the burden on the subject's eyes by lowering the frame rate of the acquired image through the imaging control unit 34, thereby extending the shutter open period and reducing the light emission intensity.

[0061] In addition, in the above-mentioned embodiments, an example is shown in which the motion detection unit 33 detects eye blinking, line of sight movement, and face shaking as the movement of at least a part of the subject's face, but it is also possible to detect the movement of the mouth, nose, etc. In addition, the motion detection unit 33 may detect one or more movements such as eye blinking and line of sight movement as the movement of at least a part of the subject's face. The shooting control unit 34 may also change the shutter opening period and the light emission intensity based on one or more movement speeds.

[0062] In addition, the shooting control unit 34 may also set the luminous intensity of the light irradiation unit 11 to 0 when the external light intensity and light quantity irradiated to the subject are sufficiently ensured. If there is a surplus of light quantity, the shooting control unit 34 may also set the shutter opening period to be shorter in order to improve the image recognition accuracy. In addition, the image processing device 100 may also be provided with an eye opening and closing detection unit (not shown) for detecting the open state and closed state of the eyes from the image. When the eyes are closed, the image recognition accuracy is prioritized, the shutter opening period is shortened, and the luminous intensity is increased.

[0063] In addition, the image processing device 100 can also detect the presence of sunglasses that cut the wavelength band of the light source used in the light irradiation unit 11. When the subject is wearing corresponding sunglasses, image recognition accuracy is prioritized, the shutter opening period is shortened, and the light intensity is increased.

[0064] Next, features of the image processing apparatus 100 , the image processing method, and the image processing program according to each of the above-described embodiments and modifications will be described.

[0065] The image processing device 100 includes a light irradiation unit 11, an image acquisition unit 31, a motion detection unit 33, and a shooting control unit 34. The light irradiation unit 11 irradiates light to the face of the subject. The image acquisition unit 31 acquires an image obtained by photographing the face of the subject. The motion detection unit 33 detects the movement speed of at least a part of the face from the image acquired by the image acquisition unit 31. When the speed detected by the motion detection unit 33 is slower than a predetermined threshold, the shooting control unit 34 extends the opening period of the shutter in the image acquisition unit 31 and reduces the light emission intensity of the light irradiation unit 11. When the speed is faster than the predetermined threshold, the shooting control unit 34 shortens the opening period of the shutter in the image acquisition unit 31 and increases the light emission intensity of the light irradiation unit 11. Thus, the image processing device 100 sets the light emission intensity to a smaller value, and can reduce the burden of light irradiation on the eyes of the subject.

[0066] In addition, the motion detection unit 33 detects the blinking speed of the eye from the image. Thus, the image processing device 100 can change the shutter opening period and the light emission intensity according to the blinking speed of the eye, and reliably acquire images of the open and closed states of the eye.

[0067] Furthermore, the motion detection unit 33 detects the speed of the line of sight from the image. Thus, the image processing device 100 can change the shutter opening period and the light emission intensity according to the speed of the line of sight, and reliably acquire an image that captures the movement of the line of sight.

[0068] Furthermore, the motion detection unit 33 detects the speed of the face shaking from the image. Thus, the image processing device 100 can reliably acquire an image that captures the left-right and up-down direction of the subject by changing the shutter opening period and the light emission intensity according to the speed of the face shaking.

[0069] The image processing device 100 further includes a shake amount detection unit 35 for detecting the shake amount of the subject. The shooting control unit 34 changes the shutter opening period and the light emission intensity according to the shake amount detected by the shake amount detection unit 35. Thus, the image processing device 100 can suppress the reduction of image recognition accuracy and reduce the burden on the eyes of the subject.

[0070] In addition, the image processing device 100 changes the frame rate of the image acquired by the image acquisition unit 31 based on the speed detected by the motion detection unit 33. Thus, the image processing device 100 can further reduce the burden on the subject's eyes by reducing the frame rate of the acquired image, thereby extending the shutter open period and reducing the light intensity.

[0071] The image processing method includes a light irradiation step, an image acquisition step, a motion detection step, and a shooting control step. The light irradiation step irradiates light to the face of the subject. The image acquisition step acquires an image obtained by photographing the face of the subject. The motion detection step detects the movement speed of at least a part of the face from the image acquired by the image acquisition step. The shooting control step extends the opening period of the shutter in the image acquisition step and reduces the luminous intensity in the light irradiation step when the speed detected by the motion detection step is slower than a predetermined threshold value, and shortens the opening period of the shutter in the image acquisition step and increases the luminous intensity in the light irradiation step when the speed is faster than the predetermined threshold value. According to the image processing method, the luminous intensity is set to a smaller value, so that the burden of light irradiation on the eyes of the subject can be reduced.

[0072] The image processing program causes the computer to execute a light irradiation step, an image acquisition step, a motion detection step, and a shooting control step. The light irradiation step irradiates light to the face of the subject. The image acquisition step acquires an image obtained by photographing the face of the subject. The motion detection step detects the movement speed of at least a part of the face from the image acquired by the image acquisition step. The shooting control step extends the opening period of the shutter in the image acquisition step and reduces the luminous intensity in the light irradiation step when the speed detected by the motion detection step is slower than a predetermined threshold value, and shortens the opening period of the shutter in the image acquisition step and increases the luminous intensity in the light irradiation step when the speed is faster than a predetermined threshold value. According to the image processing program, the luminous intensity is set to a smaller value, so that the burden of light irradiation on the eyes of the subject can be reduced.

[0073] The above is an explanation based on the embodiments of the present invention. These embodiments are for illustration only, and those skilled in the art will appreciate that various modifications and changes can be made within the scope of the claims of the present invention, and such modifications and changes are also within the scope of the claims of the present invention. Therefore, the descriptions and drawings in this specification are not restrictive and should be treated as illustrative.

[0074] Industrial Applicability

[0075] The present invention relates to an image processing device such as a driver monitor, an image processing method, and a computer-readable storage medium.

[0076] Explanation of symbols

[0077] 11: light irradiation unit; 31: image acquisition unit; 33: motion detection unit; 34: shooting control unit; 35: shake amount detection unit; 100: image processing device.

Claims

1. An image processing device, characterized in that: include: A light irradiation unit irradiates light toward the face of the subject; An image acquisition unit, which acquires an image obtained by photographing the face of the subject; a motion detection unit for detecting a speed of motion of at least a part of a face from the image acquired by the image acquisition unit; an eye opening and closing detection unit for detecting an opening state and a closing state of an eye from the image acquired by the image acquisition unit; as well as a photographing control unit that changes the opening period of the shutter in the image acquisition unit and the light emission intensity of the light irradiation unit based on the speed detected by the motion detection unit and the opening and closing state of the eyes detected by the eye opening and closing detection unit, When the speed detected by the motion detection unit is slower than a predetermined threshold, the imaging control unit extends the opening period of the shutter in the image acquisition unit and reduces the light emission intensity of the light irradiation unit. When the eyes are closed, the imaging control unit shortens the open period of the shutter and increases the light emission intensity.

2. The image processing device according to claim 1, characterized in that: The motion detection unit detects a blinking speed of an eye portion from the image.

3. The image processing device according to claim 1, characterized in that: The motion detection unit detects a speed of a line of sight from the image.

4. The image processing device according to claim 1, wherein: The motion detection unit detects a motion speed of the face shaking from the image.

5. The image processing device according to claim 1, characterized in that: Also includes: a shake amount detection unit for detecting a shake amount of the subject, The imaging control unit changes an open period of a shutter and light emission intensity according to the shake amount detected by the shake amount detection unit.

6. The image processing device according to any one of claims 1 to 5, characterized in that: The imaging control unit changes a frame rate of the image acquired by the image acquisition unit based on the speed detected by the motion detection unit.

7. The image processing device according to any one of claims 1 to 5, characterized in that: Also includes: an image accuracy detection unit for detecting image recognition accuracy for an image acquired after the shooting control unit performs control to change the shutter opening period and the light emission intensity, The imaging control unit changes settings of a shutter opening period and light emission intensity according to the detected image recognition accuracy.

8. The image processing device according to any one of claims 1 to 5, characterized in that: The photographing control section changes an opening period of a shutter and light emission intensity when the presence of an object that reduces a wavelength band of light irradiated by the light irradiation section is detected.

9. An image processing method, characterized in that: include: a light irradiation step of irradiating light onto the face of the subject; An image acquisition step of acquiring an image obtained by photographing the face of the subject; a motion detection step of detecting a speed of motion of at least a part of a face from the image acquired by the image acquisition step; an eye opening and closing detection step of detecting an opening state and a closing state of the eyes from the image acquired by the image acquisition step; as well as a shooting control step of changing the opening period of the shutter in the image acquisition step and the light emission intensity in the light irradiation step based on the speed detected by the motion detection step and the eye opening and closing state detected by the eye opening and closing detection step, In the shooting control step, when the speed detected by the motion detection step is slower than a predetermined threshold value, the opening period of the shutter in the image acquisition step is extended, and the light emission intensity in the light irradiation step is reduced. In the shooting control step, when the eyes are closed, the open period of the shutter is shortened and the light emission intensity is increased.

10. A computer-readable storage medium storing an image processing program, wherein the storage medium is characterized in that the image processing program enables a computer to execute: a light irradiation step of irradiating light onto the face of the subject; An image acquisition step of acquiring an image obtained by photographing the face of the subject; a motion detection step of detecting a speed of motion of at least a part of a face from the image acquired by the image acquisition step; an eye opening and closing detection step of detecting an open state and a closed state of the eyes from the image acquired by the image acquisition step; and a shooting control step of changing the opening period of the shutter in the image acquisition step and the light emission intensity in the light irradiation step based on the speed detected by the motion detection step and the eye opening and closing state detected by the eye opening and closing detection step, In the shooting control step, when the speed detected by the motion detection step is slower than a predetermined threshold value, the opening period of the shutter in the image acquisition step is extended, and the light emission intensity in the light irradiation step is reduced. In the shooting control step, when the eyes are closed, the open period of the shutter is shortened and the light emission intensity is increased.

Images