Information processing device and information processing method

Abstract

An information processing device according to the present invention comprises: an image acquisition unit (201) that acquires, from an imaging device (1), a captured image in which a person is projected; an image rotation unit (202) that, on the basis of an angle of installation of the imaging device (1) with respect to a reference angle, rotates the captured image acquired by the image acquisition unit (201) toward the reference angle side by the amount of the installation angle; a facial area detection unit (203) that detects the facial area of the person from the captured image on the basis of the captured image after the rotation by the image rotation unit (202); a reference area detection unit (204) that, on the basis of the facial area of the person detected by the facial area detection unit (203), detects, as a reference area from the captured image acquired by the image acquisition unit (201), a rectangular area having no inclination inscribed in the facial area of the person; and an information output unit (205) that outputs, to the imaging device (1) on the basis of the reference area detected by the reference area detection unit (204), information indicating the reference area or the average luminance in the reference area.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] Conventionally, in face detection or face authentication using a captured image acquired by an imaging device, in order to perform optimal exposure control on the captured image, an area inside the face area in the captured image is set as a reference area used for the exposure control. Usually, the reference area is a rectangular area without inclination with respect to the captured image.

[0003] On the other hand, for example, depending on the installation conditions of the imaging device, when the installation angle (roll angle) of the imaging device is inclined, the face area in the captured image may be inclined. When the face area is inclined in this way, the background area enters the reference area. And when exposure control is performed using a reference area including a background area other than the face area, ideal exposure control cannot be performed. For example, at night, since the background area is darker than the face area, the average luminance in the reference area becomes lower than expected, and the captured image may be overexposed by the exposure control. Therefore, even when the face area is inclined, it is required to detect an ideal reference area that does not include the background area.

[0004] On the other hand, for example, in the imaging device disclosed in Patent Document 1, a configuration is shown in which the background is not included in the photometric area even when the face is inclined. That is, in this imaging device, the angle of the face is detected, and the angle of the photometric area is inclined according to the detected angle.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] As described above, the imaging device disclosed in Patent Document 1 can be made to prevent the background from being included in the photometering area even when the face is tilted, by tilting the angle of the photometering area according to the angle of the face. However, the means disclosed in Patent Document 1 cannot be applied to imaging devices that require a rectangular area without tilt relative to the captured image to be used as the reference area (photometric area).

[0007] This disclosure was made to solve the above-mentioned problems and aims to provide an information processing device that can detect a suitable reference area even when detecting a reference area that is not tilted relative to the captured image as a reference area for exposure control. [Means for solving the problem]

[0008] The information processing device according to this disclosure is characterized by comprising: an image acquisition unit that acquires an image of a person from an imaging device; an image rotation unit that rotates the image acquired by the image acquisition unit toward the reference angle by the amount of the installation angle of the imaging device relative to a reference angle; a face region detection unit that detects the region of a person's face from the image based on the image after rotation by the image rotation unit; a reference region detection unit that detects a non-tilted rectangular region inscribed in the region of a person's face from the image acquired by the image acquisition unit as a reference region based on the region of a person's face detected by the face region detection unit; and an information output unit that outputs information indicating the reference region or the average brightness in the reference region to the imaging device based on the reference region detected by the reference region detection unit. [Effects of the Invention]

[0009] According to this disclosure, with the configuration described above, even when detecting a reference region that is not tilted relative to the captured image as a reference region for exposure control, it is possible to detect a suitable reference region. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example configuration of a crew status detection system equipped with an information processing device according to Embodiment 1. [Figure 2] This figure shows an example of the configuration of the information processing device according to Embodiment 1. [Figure 3] This is a flowchart showing an example of the operation of the information processing device according to Embodiment 1. [Figure 4] Figures 4A and 4B show examples of the operation of the image rotation unit in Embodiment 1. Figure 4A shows an example of an image captured by the image acquisition unit (image captured before rotation by the image rotation unit), and Figure 4B shows an example of an image captured after rotation by the image rotation unit. [Figure 5] This figure shows an example of the operation of the face region detection unit in Embodiment 1. [Figure 6] Figures 6A to 6C are diagrams showing examples of the operation of the reference region detection unit in Embodiment 1. Figure 6A shows an example of detection of an external rectangular region by the reference region detection unit, Figure 6B shows an example of detection of two intersection points by the reference region detection unit, and Figure 6C shows an example of detection of an internal rectangular region by the reference region detection unit. [Figure 7] This figure shows an example of the configuration of the information processing device according to Embodiment 2. [Figure 8] This flowchart shows an example of the operation of the information processing device according to Embodiment 2. [Figure 9] Figures 9A and 9B show examples of operation of the information processing device according to Embodiment 2. Figure 9A shows an example of reference area detection when the person is not wearing an accessory, and Figure 9B shows an example of reference area detection when the person is wearing an accessory (mask). [Figure 10] Figures 10A and 10B show examples of the hardware configuration of the information processing device according to Embodiments 1 and 2. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings. Embodiment 1. FIG. 1 is a diagram showing a configuration example of an occupant state detection system including an information processing apparatus 2 according to Embodiment 1. The occupant state detection system according to Embodiment 1 is a system for detecting the state of an occupant riding in a vehicle. Examples of the occupant state detected by the occupant state detection system include information related to face detection or face authentication. Examples of the information related to face detection include information such as the presence or absence of an occupant, whether the occupant is not drowsy, or whether the occupant is not looking sideways. Examples of the information related to face authentication include information such as who the occupant is. As shown in FIG. 1, for example, this occupant state detection system includes an imaging device 1 and an information processing apparatus 2.

[0012] Here, the case where the imaging device 1 and the information processing apparatus 2 are applied to the occupant state detection system is shown. However, the present invention is not limited to this, and the imaging device 1 and the information processing apparatus 2 may be applied to other systems. Also, here, the case where the imaging device 1 and the information processing apparatus 2 are separate is shown. However, the present invention is not limited to this, and the imaging device 1 and the information processing apparatus 2 may be integrated.

[0013] The imaging device 1 acquires a captured image by imaging the interior of the vehicle. A person, here an occupant riding in the vehicle, may be shown in the captured image acquired by this imaging device 1. As this imaging device 1, for example, a near-infrared camera is used. The information indicating the captured image acquired by this imaging device 1 is output to the information processing apparatus 2.

[0014] The information processing apparatus 2 detects a reference region for exposure control in the imaging device 1 based on the captured image acquired by the imaging device 1. In the information processing apparatus 2 shown in FIG. 2, in addition to the above function, it also has a function of detecting the state of a person, here the state of an occupant riding in the vehicle. As shown in, for example, FIG. 2, this information processing apparatus 2 includes an image acquisition unit 201, an image rotation unit 202, a face region detection unit 203, a reference region detection unit 204, an information output unit 205, and an occupant state detection unit (person state detection unit) 206.

[0015] The image acquisition unit 201 acquires a captured image from the imaging device 1. Information indicating the captured image acquired by this image acquisition unit 201 is output to the image rotation unit 202 and the reference region detection unit 204.

[0016] Based on the installation angle (roll angle) of the imaging device 1 with respect to the reference angle, the image rotation unit 202 rotates the captured image acquired by the image acquisition unit 201 by the amount of the installation angle toward the reference angle side. Note that the information processing apparatus 2 has previously acquired information indicating the installation angle of the imaging device 1 with respect to the reference angle. The reference angle is, for example, horizontal. Information indicating the captured image after rotation by this image rotation unit 202 is output to the face region detection unit 203.

[0017] Based on the captured image after rotation by the image rotation unit 202, the face region detection unit 203 detects the region of a person's face from the captured image. At this time, for example, the face region detection unit 203 detects the region of a person's face from the captured image by detecting the feature points of the person's face based on the captured image after rotation by the image rotation unit 202. Note that the region of the person's face detected by the face region detection unit 203 is a rectangular region that is not inclined with respect to the state after rotation by the image rotation unit 202. Information indicating the region of the person's face detected by this face region detection unit 203 is output to the reference region detection unit 204.

[0018] In this process, for example, the face region detection unit 203 detects coordinates indicating the face region of a person based on the detected face region of that person, that is, coordinates indicating each vertex of the rectangular region that constitutes the face region of that person. The face region detection unit 203 then detects coordinates obtained by rotating the detected coordinates indicating the face region of the person in the opposite direction from the reference angle by the amount of the installation angle of the imaging device 1. The face region detection unit 203 then outputs the information indicating the coordinates after the reverse rotation to the reference region detection unit 204 as information indicating the face region of the person. Alternatively, the face region detection unit 203 outputs coordinates indicating the face region of the detected person to the reference region detection unit 204 as information indicating the face region of that person. The reference region detection unit 204 may then detect coordinates obtained from the face region detection unit 203 that are rotated in the opposite direction from the reference angle by the installation angle of the imaging device 1.

[0019] The reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203, as a reference region. In this case, for example, the reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203, which is similar in shape to the flat rectangular region circumscribed outside the face region of a person, and is also a flat rectangular region inscribed within the face region of a person, as a reference region. The information indicating the reference region detected by the reference region detection unit 204 is output to the information output unit 205.

[0020] The information output unit 205 outputs information to the imaging device 1 indicating the reference region or the average brightness within the reference region, based on the reference region detected by the reference region detection unit 204. Subsequently, the imaging device 1 performs brightness control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs brightness control on the captured image so that the average brightness based on the information output by the information output unit 205 approaches the target brightness. The target brightness is a predetermined brightness that enables the detection of a person's face with high accuracy, and is set as appropriate.

[0021] The occupant status detection unit 206 detects the status of an occupant based on the facial region of the person detected by the face region detection unit 203. The occupant status detected by the occupant status detection unit 206 includes information related to face detection or face recognition. Information related to face detection includes information such as whether there is an occupant, whether the occupant is dozing off, or whether the occupant is distracted. Information related to face recognition includes information such as who the occupant is.

[0022] Figure 2 shows a case where the information processing device 2 has a crew state detection unit (person state detection unit) 206 and has the function of detecting the state of the crew. However, the crew state detection unit (person state detection unit) 206 is not an essential component of the information processing device 2, and it is not necessary for the information processing device 2 to have a crew state detection unit (person state detection unit) 206.

[0023] Next, an example of the operation of the information processing device 2 according to Embodiment 1 shown in Figure 2 will be explained with reference to Figure 3. In an example of the operation of the information processing device 2 according to Embodiment 1 shown in Figure 2, for example as shown in Figure 3, first, the image acquisition unit 201 acquires an image from the imaging device 1 (step ST101). Information indicating the captured image acquired by the image acquisition unit 201 is output to the image rotation unit 202 and the reference region detection unit 204.

[0024] Next, the image rotation unit 202 rotates the image acquired by the image acquisition unit 201 toward the reference angle by the amount of the installation angle (roll angle) of the imaging device 1 relative to the reference angle (step ST102). Information indicating the image after rotation by the image rotation unit 202 is output to the face region detection unit 203.

[0025] For example, as shown in Figure 4A, the imaging device 1 is tilted by an angle relative to the reference angle, and as a result, the crew members who appear in the captured image acquired by the image acquisition unit 201 are tilted relative to the image. In this case, for example as shown in Figure 4B, the image rotation unit 202 rotates the captured image to eliminate the tilt of the person in the captured image caused by the installation angle of the imaging device 1.

[0026] Next, the face region detection unit 203 detects the region of a person's face from the captured image based on the image rotated by the image rotation unit 202 (step ST103). In this case, for example, the face region detection unit 203 detects the region of a person's face by detecting characteristic points of the person's face from the captured image based on the image rotated by the image rotation unit 202. The region of a person's face detected by the face region detection unit 203 is a rectangular region that is not tilted relative to the state after rotation by the image rotation unit 202. The information indicating the facial region of a person detected by the face region detection unit 203 is output to the reference region detection unit 204.

[0027] In this process, for example, the face region detection unit 203 detects coordinates indicating the face region of a person based on the detected face region of that person, that is, coordinates indicating each vertex of the rectangular region that constitutes the face region of that person. The face region detection unit 203 then detects coordinates obtained by rotating the detected coordinates indicating the face region of the person in the opposite direction from the reference angle by the amount of the installation angle of the imaging device 1. The face region detection unit 203 then outputs the information indicating the coordinates after the reverse rotation to the reference region detection unit 204 as information indicating the face region of the person. Alternatively, the face region detection unit 203 outputs coordinates indicating the face region of the detected person to the reference region detection unit 204 as information indicating the face region of that person. The reference region detection unit 204 may then detect coordinates obtained from the face region detection unit 203 that are rotated in the opposite direction from the reference angle by the installation angle of the imaging device 1.

[0028] For example, as shown in Figure 5, the face region detection unit 203 detects the region of a person's face from the image captured after rotation by the image rotation unit 202 shown in Figure 4B. In Figure 5, reference numeral 11 indicates the region of a person's face detected by the face region detection unit 203. As shown in Figure 5, the region of a person's face detected by the face region detection unit 203 is a rectangular region with no tilt relative to the state after rotation by the image rotation unit 202. Also, in Figure 5, the vertices of the region of a person's face detected by the face region detection unit 203 are denoted as a, b, c, and d.

[0029] Next, the reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person and set in motion from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203, as a reference region (step ST104). In this case, for example, the reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person and set in motion from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203, as a reference region. The information indicating the reference region detected by the reference region detection unit 204 is output to the information output unit 205.

[0030] For example, as shown in Figure 6A, the reference region detection unit 204 first detects a non-tilted rectangular region (circumscribed rectangular region) that circumscribes the region of the person's face detected by the face region detection unit 203 from the captured image acquired by the image acquisition unit 201. Existing methods can be used to detect a rectangular region that circumscribes a given rectangular region and is not tilted relative to the captured image. In Figure 6, reference numeral 12 indicates the non-tilted rectangular region (circumscribed rectangular region) that circumscribes the region of the face. Next, as shown in Figure 6B, for example, the reference region detection unit 204 detects the diagonal of the circumscribed rectangular region as a line segment ef from the captured image acquired by the image acquisition unit 201. In this case, the line segment ef is defined as the line segment that intersects the longer side of the region of the person's face detected by the face region detection unit 203. The reference region detection unit 204 then detects the intersection point g of line segment ef and line segment ab, and the intersection point h of line segment ef and line segment cd. It should be noted that existing methods can be used to detect the intersection points of two lines. Next, as shown in Figure 6C, for example, the reference region detection unit 204 detects from intersection point g and intersection point h a rectangular region that is similar in shape to the rectangular region without inclination that is circumscribed around the face region, and a rectangular region without inclination that is circumscribed around the face region (inscribed rectangular region). The reference region detection unit 204 then uses the detected inscribed rectangular region as the reference region for exposure control. In Figure 6, reference numeral 13 indicates the reference region (inscribed rectangular region).

[0031] In the above example, the reference region detection unit 204 detects a reference region from the captured image acquired by the image acquisition unit 201, based on the facial region of the person detected by the facial region detection unit 203. This reference region is similar in shape to a non-tilted rectangular region that circumscribes the facial region of the person, and is also a non-tilted rectangular region that circumscribes the facial region of the person. However, the method of detecting the reference region by the reference region detection unit 204 is not limited to this. The reference region detection unit 204 can use any detection method that enables it to detect a flat rectangular region inscribed within the region of a person's face detected by the face region detection unit 203 from the captured image acquired by the image acquisition unit 201 as a reference region. For example, the reference region detection unit 204 may directly detect a flat rectangular region inscribed within the region of a person's face from the region of a person's face, without detecting a flat rectangular region circumscribed outside the region of a person's face.

[0032] Next, the information output unit 205 outputs information indicating the reference region or the average brightness in the reference region to the imaging device 1 based on the reference region detected by the reference region detection unit 204 (step ST105). Subsequently, the imaging device 1 performs brightness control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs brightness control on the captured image so that the average brightness based on the information output by the information output unit 205 approaches the target brightness.

[0033] Furthermore, the occupant status detection unit 206 detects information about the occupant based on the facial region of the person detected by the face region detection unit 203 (step ST106). The occupant status detected by the occupant status detection unit 206 includes information related to face detection or face recognition. Information related to face detection includes information such as whether there is an occupant, whether the occupant is dozing off, or whether the occupant is looking away. Information related to face recognition includes information such as who the occupant is.

[0034] As described above, in the information processing device 2 according to Embodiment 1, the captured image acquired from the imaging device 1 is rotated according to the installation angle of the imaging device 1, the face region is detected from the rotated captured image, and a non-tilted rectangular region inscribed within the face region is detected as a reference region from the captured image acquired from the imaging device 1. As a result, the information processing device 2 according to Embodiment 1 can be used with an imaging device 1 that requires a rectangular area without tilt relative to the captured image to be used as the reference area, and it is also possible to ensure that the background area is not included in the reference area even when the face area is tilted.

[0035] As described above, according to this embodiment 1, the information processing device 2 includes an image acquisition unit 201 that acquires an image of a person from the imaging device 1, an image rotation unit 202 that rotates the image acquired by the image acquisition unit 201 toward the reference angle by the amount of the installation angle of the imaging device 1 with respect to a reference angle, a face region detection unit 203 that detects the region of a person's face from the image based on the image after rotation by the image rotation unit 202, a reference region detection unit 204 that detects a non-tilted rectangular region inscribed in the region of a person's face from the image acquired by the image acquisition unit 201 as a reference region based on the region of a person's face detected by the face region detection unit 203, and an information output unit 205 that outputs information indicating the reference region or the average brightness in the reference region to the imaging device 1 based on the reference region detected by the reference region detection unit 204. Furthermore, according to this embodiment 1, the reference region detection unit 204 detects a reference region from the captured image acquired by the image acquisition unit 201, based on the facial region of the person detected by the facial region detection unit 203. This reference region is similar in shape to a non-tilted rectangular region that circumscribes the facial region of the person and is inscribed within the facial region of the person. As a result, the information processing device 2 according to Embodiment 1 can detect a suitable reference area even when detecting a reference area that is not tilted relative to the captured image as a reference area for exposure control. In other words, the information processing device 2 according to Embodiment 1 can detect the optimal reference area for exposure control regardless of the position of the person's face and the tilt of the imaging device 1. As a result, the information processing device 2 according to Embodiment 1 enables stable exposure control in the imaging device 1.

[0036] Furthermore, according to this embodiment 1, the information processing device 2 includes a person state detection unit 206 that detects the state of a person based on the facial region of the person detected by the face region detection unit 203. As a result, the information processing device 2 according to Embodiment 1 can detect information about a person's state, such as face detection or facial recognition, with greater accuracy than conventional devices.

[0037] Furthermore, according to this embodiment 1, the information processing method includes the steps of: an image acquisition unit 201 acquiring an image of a person from the imaging device 1; an image rotation unit 202 rotating the image acquired by the image acquisition unit 201 toward the reference angle based on the installation angle of the imaging device 1 with respect to a reference angle; a face region detection unit 203 detecting the face region of a person from the image based on the image after rotation by the image rotation unit 202; a reference region detection unit 204 detecting a non-tilted rectangular region inscribed within the face region of a person from the image acquired by the image acquisition unit 201 as a reference region based on the face region of a person detected by the face region detection unit 203; and an information output unit 205 outputting information indicating the reference region or the average brightness in the reference region to the imaging device 1 based on the reference region detected by the reference region detection unit 204. As a result, the information processing method according to Embodiment 1 makes it possible to detect a suitable reference area even when detecting a reference area that is not tilted relative to the captured image as a reference area for exposure control. In other words, the information processing method according to Embodiment 1 makes it possible to detect the optimal reference area for exposure control regardless of the position of the person's face and the tilt of the imaging device 1. As a result, the information processing method according to Embodiment 1 enables stable exposure control in the imaging device 1.

[0038] Embodiment 2. Figure 7 shows an example of the configuration of the information processing device 2 according to Embodiment 2. In the information processing device 2 according to Embodiment 2 shown in Figure 7, an accessory detection unit 207 is added compared to the information processing device 2 according to Embodiment 1 shown in Figure 2. Other configuration examples of the information processing device 2 according to Embodiment 2 shown in Figure 7 are the same as the configuration example of the information processing device 2 according to Embodiment 1 shown in Figure 2, and only the different parts will be described using the same reference numerals.

[0039] The accessory detection unit 207 detects whether a person is wearing an accessory based on the captured image acquired by the image acquisition unit 201. In this case, for example, the accessory detection unit 207 detects whether a person is wearing an accessory by comparing the area of ​​the person's face in the captured image acquired by the image acquisition unit 201 with reference face information. Here, the reference face information is, for example, face information when the person is not wearing an accessory, or face information when the person is wearing an accessory. The accessory detection unit 207 may also reuse the area of ​​the person's face detected by the face area detection unit 203 as the area of ​​the person's face in the captured image acquired by the image acquisition unit 201. The accessories that the accessory detection unit 207 detects include, for example, masks, glasses, or sunglasses. Information indicating the detection result by the attached accessory detection unit 207 is output to the reference area detection unit 204.

[0040] Furthermore, in Embodiment 2, if the attached accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory from the reference area detection target of the captured image acquired by the image acquisition unit 201.

[0041] Next, an example of the operation of the information processing device 2 according to Embodiment 2 shown in Figure 7 will be explained with reference to Figure 8. In the example of operation of the information processing device 2 according to Embodiment 2 shown in Figure 7, for example as shown in Figure 8, first, the image acquisition unit 201 acquires an image from the imaging device 1 (step ST201). Information indicating the captured image acquired by the image acquisition unit 201 is output to the image rotation unit 202 and the reference region detection unit 204.

[0042] Next, the image rotation unit 202 rotates the image acquired by the image acquisition unit 201 toward the reference angle by the amount of the installation angle (roll angle) of the imaging device 1 relative to the reference angle (step ST202). Information indicating the image after rotation by the image rotation unit 202 is output to the face region detection unit 203.

[0043] Next, the face region detection unit 203 detects the region of a person's face from the captured image based on the image rotated by the image rotation unit 202 (step ST203). In this case, for example, the face region detection unit 203 detects the region of a person's face by detecting characteristic points of a person's face from the captured image based on the image acquired by the image acquisition unit 201. The region of a person's face detected by the face region detection unit 203 is a rectangular region that is not tilted relative to the image rotated by the image rotation unit 202. The information indicating the facial region of a person detected by the face region detection unit 203 is output to the reference region detection unit 204.

[0044] In this process, for example, the face region detection unit 203 detects coordinates indicating the face region of a person based on the detected face region of that person, that is, coordinates indicating each vertex of the rectangular region that constitutes the face region of that person. The face region detection unit 203 then detects coordinates obtained by rotating the detected coordinates indicating the face region of the person in the opposite direction from the reference angle by the amount of the installation angle of the imaging device 1. The face region detection unit 203 then outputs the information indicating the coordinates after the reverse rotation to the reference region detection unit 204 as information indicating the face region of the person. Alternatively, the face region detection unit 203 outputs coordinates indicating the face region of the detected person to the reference region detection unit 204 as information indicating the face region of that person. The reference region detection unit 204 may then detect coordinates obtained from the face region detection unit 203 that are rotated in the opposite direction from the reference angle by the installation angle of the imaging device 1.

[0045] Furthermore, the accessory detection unit 207 detects whether or not a person is wearing an accessory based on the captured image acquired by the image acquisition unit 201 (step ST204). In this case, for example, the accessory detection unit 207 detects whether or not a person is wearing an accessory by comparing the area of ​​the person's face in the captured image acquired by the image acquisition unit 201 with reference face information. Information indicating the detection result by the attached accessory detection unit 207 is output to the reference area detection unit 204.

[0046] Next, the reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person as a reference region from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203 (step ST205). In this case, for example, the reference region detection unit 204 detects a flat rectangular region inscribed within the face region of a person as a reference region from the captured image acquired by the image acquisition unit 201, based on the face region of a person detected by the face region detection unit 203, which is similar in shape to the flat rectangular region circumscribed outside the face region of a person and is also inscribed within the face region of a person.

[0047] In the second embodiment, if the accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory from the reference area detection target in the captured image acquired by the image acquisition unit 201.

[0048] For example, as shown in Figure 9A, if the accessory detection unit 207 detects that the person is not wearing an accessory, the reference area detection unit 204 detects the reference area using the procedure shown in Embodiment 1. Figure 9A is the same figure as Figure 6C. On the other hand, as shown in Figure 9B, for example, if the accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory from the reference area detection target of the captured image acquired by the image acquisition unit 201. Figure 9B shows an example where a person is wearing a mask, in which case the reference area detection unit 204 targets the area above the mask as the reference area detection target. In Figure 9B, reference numeral 14 indicates the mask.

[0049] Note that Figure 9B shows the case where the bounding rectangle is set to be outside the accessory's area, but it is not limited to this; it is sufficient if the area of ​​the person's face detected by the face area detection unit 203 is outside the accessory's area.

[0050] The information indicating the reference region detected by the reference region detection unit 204 is output to the information output unit 205.

[0051] Next, the information output unit 205 outputs information indicating the reference region or the average brightness in the reference region to the imaging device 1 based on the reference region detected by the reference region detection unit 204 (step ST206). Subsequently, the imaging device 1 performs brightness control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs brightness control on the captured image so that the average brightness based on the information output by the information output unit 205 approaches the target brightness.

[0052] Furthermore, the occupant status detection unit 206 detects the status of the occupant based on the facial region of the person detected by the face region detection unit 203 (step ST207). The occupant status detected by the occupant status detection unit 206 includes information related to face detection or face recognition. Information related to face detection includes information such as whether there is an occupant, whether the occupant is dozing off, or whether the occupant is distracted. Information related to face recognition includes information such as who the occupant is.

[0053] As described above, according to this embodiment 2, the information processing device 2 includes an accessory detection unit 207 that detects whether or not a person is wearing an accessory based on the captured image acquired by the image acquisition unit 201, and the reference area detection unit 204 excludes the area of ​​the accessory from the reference area detection target of the captured image acquired by the image acquisition unit 201 when the accessory detection unit 207 detects that a person is wearing an accessory. Furthermore, according to this second embodiment, the accessory to be detected by the attached accessory detection unit 207 is a mask. As a result, the information processing device 2 according to Embodiment 2, in addition to the effects of Embodiment 1, can detect a reference area for optimal exposure control regardless of whether or not the person is wearing any clothing.

[0054] Finally, with reference to Figure 10, an example of the hardware configuration of the information processing device 2 according to Embodiments 1 and 2 will be described. Note that the following description will focus on the hardware configuration example of the information processing device 2 according to Embodiment 1, but the same applies to the hardware configuration example of the information processing device 2 according to Embodiment 2. The image acquisition unit 201, image rotation unit 202, face region detection unit 203, reference region detection unit 204, information output unit 205, and person state detection unit 206 functions in the information processing device 2 are realized by the processing circuit 51. The processing circuit 51 may be dedicated hardware as shown in Figure 10A, or it may be a CPU (Central Processing Unit, also known as a processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 that executes a program stored in the memory 53, as shown in Figure 10B.

[0055] If the processing circuit 51 is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each part, the image acquisition unit 201, the image rotation unit 202, the face region detection unit 203, the reference region detection unit 204, the information output unit 205, and the person state detection unit 206, may be implemented individually by the processing circuit 51, or the functions of each part may be implemented together by the processing circuit 51.

[0056] When the processing circuit 51 is the CPU 52, the functions of the image acquisition unit 201, image rotation unit 202, face region detection unit 203, reference region detection unit 204, information output unit 205, and person state detection unit 206 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in memory 53. The processing circuit 51 realizes the functions of each unit by reading and executing the programs stored in memory 53. In other words, the information processing device 2 has memory for storing programs that, when executed by the processing circuit 51, result in the execution of each step shown in Figure 3, for example. These programs can also be said to cause the computer to execute the procedures and methods of the image acquisition unit 201, image rotation unit 202, face region detection unit 203, reference region detection unit 204, information output unit 205, and person state detection unit 206. Here, memory 53 includes, for example, non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidiscs, or DVDs (Digital Versatile Discs).

[0057] Furthermore, the functions of the image acquisition unit 201, image rotation unit 202, face region detection unit 203, reference region detection unit 204, information output unit 205, and person state detection unit 206 may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the image acquisition unit 201 can be implemented by a processing circuit 51 as dedicated hardware, while the image rotation unit 202, face region detection unit 203, reference region detection unit 204, information output unit 205, and person state detection unit 206 can be implemented by the processing circuit 51 reading and executing a program stored in memory 53.

[0058] Thus, the processing circuit 51 can realize each of the above-mentioned functions through hardware, software, firmware, or a combination thereof.

[0059] Furthermore, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component in each embodiment. [Industrial applicability]

[0060] The information processing device according to this disclosure is capable of detecting a more suitable reference area than conventional methods, even when detecting a reference area that is not tilted relative to the captured image as a reference area for exposure control, and is therefore suitable for use in information processing devices and the like. [Explanation of symbols]

[0061] 1 Imaging device, 2 Information processing device, 51 Processing circuit, 52 CPU, 53 Memory, 201 Image acquisition unit, 202 Image rotation unit, 203 Face area detection unit, 204 Reference area detection unit, 205 Information output unit, 206 Crew state detection unit (Person state detection unit), 207 Mounted accessory detection unit.

Claims

1. An image acquisition unit that acquires captured images of a person from an imaging device, An image rotation unit rotates the image acquired by the image acquisition unit toward the reference angle by the amount of the installation angle of the imaging device relative to a reference angle, A face region detection unit detects the region of a person's face from the captured image based on the image captured by the image rotation unit, A reference region detection unit detects a reference region from the captured image acquired by the image acquisition unit, based on the facial region of the person detected by the facial region detection unit, which contains a rectangular region without tilt that is inscribed within the facial region of the person. An information output unit outputs information to the imaging device indicating the reference region or the average brightness in the reference region based on the reference region detected by the reference region detection unit. Equipped with an information processing device.

2. The reference region detection unit, based on the facial region of the person detected by the facial region detection unit, detects a reference region from the captured image acquired by the image acquisition unit that is similar in shape to a non-tilted rectangular region circumscribing the facial region of the person, and is also a non-tilted rectangular region circumscribing the facial region of the person. The information processing apparatus according to claim 1, characterized in that it is a product of the present invention.

3. The system includes an accessory detection unit that detects whether or not a person is wearing an accessory based on the captured image acquired by the image acquisition unit. When the accessory detection unit detects that a person is wearing an accessory, the reference area detection unit excludes the area of ​​the accessory from the reference area detection target in the captured image acquired by the image acquisition unit. The information processing apparatus according to claim 1, characterized in that it is a product of the present invention.

4. The accessory to be detected by the aforementioned accessory detection unit is a mask. The information processing apparatus according to claim 3, characterized in that it is a product of the same method.

5. The system includes a person state detection unit that detects the state of a person based on the facial region of the person detected by the face region detection unit. The information processing apparatus according to claim 1, characterized in that it is a product of the present invention.

6. The image acquisition unit acquires an image of a person from the imaging device, The image rotation unit rotates the image acquired by the image acquisition unit toward the reference angle by the amount of the installation angle of the imaging device relative to the reference angle, The face region detection unit detects the region of a person's face from the captured image based on the image captured after rotation by the image rotation unit, The reference region detection unit detects a non-tilted rectangular region inscribed within the region of the person's face from the captured image acquired by the image acquisition unit, based on the region of the person's face detected by the face region detection unit, as a reference region. The information output unit outputs information to the imaging device indicating the reference region or the average brightness in the reference region, based on the reference region detected by the reference region detection unit. An information processing method having

Images