Information processing device and information processing method

Abstract

An information processing device according to the present invention comprises: an image acquisition unit (201) that acquires, from an imaging device (1), a captured image in which a person is projected; an image rotation unit (202) that, on the basis of an angle of installation of the imaging device (1) with respect to a reference angle, rotates the captured image acquired by the image acquisition unit (201) toward the reference angle side by the amount of the installation angle; a facial area detection unit (203) that detects the facial area of the person from the captured image on the basis of the captured image after the rotation by the image rotation unit (202); a reference area detection unit (204) that, on the basis of the facial area of the person detected by the facial area detection unit (203), detects, as a reference area from the captured image acquired by the image acquisition unit (201), a rectangular area having no inclination inscribed in the facial area of the person; and an information output unit (205) that outputs, to the imaging device (1) on the basis of the reference area detected by the reference area detection unit (204), information indicating the reference area or the average luminance in the reference area.

Description

Information processing device and information processing method

[0001] The present disclosure relates to an information processing device and an information processing method.

[0002] In conventional face detection or face authentication using a captured image acquired by an imaging device, an area inside the face area in the captured image is set as a reference area used for optimal exposure control for the captured image. Note that the reference area is usually a rectangular area that is not tilted relative to the captured image.

[0003] On the other hand, for example, if the installation angle (roll angle) of the imaging device is tilted due to the installation conditions of the imaging device, the facial area in the captured image may be tilted. When the facial area is tilted in this way, the background area enters the reference area. Furthermore, if exposure control is performed using a reference area that includes the background area in addition to the facial area, ideal exposure control cannot be performed. For example, at night, the background area is darker than the facial area, so the average brightness in the reference area may be lower than expected, and the captured image may be blown out due to exposure control. Therefore, even if the facial area is tilted, it is necessary to detect an ideal reference area that does not include the background area.

[0004] In response to this, for example, the image capture device disclosed in Patent Document 1 shows a configuration in which the background is not included in the photometry area even when the face is tilted. That is, this image capture device detects the angle of the face and tilts the angle of the photometry area according to that angle.

[0005] Japanese Patent Application Laid-Open No. 2010-200057

[0006] As described above, the imaging device disclosed in Patent Document 1 tilts the angle of the photometry area according to the angle of the face, so that the background is not included in the photometry area even when the face is tilted. However, the method disclosed in Patent Document 1 cannot be applied to imaging devices that require a rectangular area that is not tilted relative to the captured image as the reference area (photometry area).

[0007] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an information processing device that is capable of detecting an appropriate reference area even when detecting a reference area that is not tilted relative to a captured image as a reference area for exposure control.

[0008] The information processing device according to the present disclosure is characterized by including an image acquisition unit that acquires an image of a person from an imaging device; an image rotation unit that rotates the image acquired by the image acquisition unit toward a reference angle by an amount equal to the installation angle of the imaging device relative to the reference angle, based on the installation angle of the imaging device relative to the reference angle; a face area detection unit that detects an area of ​​the person's face from the image after rotation by the image rotation unit; a reference area detection unit that detects, as a reference area, a non-inclined rectangular area inscribed in the area of ​​the person's face from the image acquired by the image acquisition unit, based on the area of ​​the person's face detected by the face area detection unit; and an information output unit that outputs information indicating the reference area or the average luminance in the reference area to the imaging device, based on the reference area detected by the reference area detection unit.

[0009] According to the present disclosure, with the above-described configuration, even when detecting a reference area that is not tilted with respect to a captured image as a reference area for exposure control, it is possible to detect an appropriate reference area.

[0010] FIG. 6A is a diagram showing an example of the configuration of an occupant state detection system including an information processing device according to Embodiment 1. FIG. 6B is a diagram showing an example of the configuration of an information processing device according to Embodiment 1. FIG. 6C is a diagram showing an example of the configuration of an information processing device according to Embodiment 2. FIG. 6C is a diagram showing an example of the configuration of an information processing device according to Embodiment 2. FIG. 6A is a diagram showing an example of the configuration of an information processing device according to Embodiment 2. FIG. 6B is a diagram showing an example of the configuration of an information processing device according to Embodiment 2. FIG. 6C is a diagram showing an example of the configuration of an information processing device according to Embodiment 2. 9A and 9B are diagrams showing an example of operation of an information processing device according to embodiment 2, in which Fig. 9A is a diagram showing an example of detection of a reference region when a person is not wearing an accessory, and Fig. 9B is a diagram showing an example of detection of a reference region when a person is wearing an accessory (mask). Figs. 10A and 10B are diagrams showing an example of hardware configurations of the information processing devices according to embodiments 1 and 2.

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings. Embodiment 1. FIG. 1 is a diagram showing an example of the configuration of an occupant state detection system including an information processing device 2 according to embodiment 1. The occupant state detection system according to embodiment 1 is a system for detecting the state of an occupant riding in a vehicle. The occupant state detected by the occupant state detection system can be information related to face detection or face authentication. Information related to face detection can be information such as whether an occupant is present, whether the occupant is dozing, or whether the occupant is looking away. Information related to face authentication can be information such as who the occupant is. This occupant state detection system can be equipped with an imaging device 1 and an information processing device 2, as shown in FIG. 1, for example.

[0012] Here, the image capturing device 1 and the information processing device 2 are shown as being applied to an occupant state detection system. However, the image capturing device 1 and the information processing device 2 may be applied to other systems. Also, the image capturing device 1 and the information processing device 2 are shown as being separate entities. However, the image capturing device 1 and the information processing device 2 may be integrated into one body.

[0013] The imaging device 1 captures an image of the interior of a vehicle to obtain the captured image. The captured image obtained by the imaging device 1 may include a person, in this case, a passenger riding in the vehicle. For example, a near-infrared camera is used as the imaging device 1. Information indicating the captured image obtained by the imaging device 1 is output to the information processing device 2.

[0014] The information processing device 2 detects a reference area for exposure control in the imaging device 1 based on the captured image acquired by the imaging device 1. In addition to the above functions, the information processing device 2 shown in Fig. 2 also has a function for detecting the state of a person, in this case, the state of an occupant riding in a vehicle. For example, as shown in Fig. 2, the information processing device 2 includes an image acquisition unit 201, an image rotation unit 202, a face area detection unit 203, a reference area detection unit 204, an information output unit 205, and an occupant state detection unit (person state detection unit) 206.

[0015] The image acquisition unit 201 acquires a captured image from the imaging device 1. Information indicating the captured image acquired by the image acquisition unit 201 is output to the image rotation unit 202 and the reference area detection unit 204.

[0016] Based on the installation angle (roll angle) of the imaging device 1 relative to a reference angle, the image rotation unit 202 rotates the captured image acquired by the image acquisition unit 201 toward the reference angle by the installation angle. Note that the information processing device 2 has previously acquired information indicating the installation angle of the imaging device 1 relative to the reference angle. The reference angle is, for example, horizontal. Information indicating the captured image after rotation by the image rotation unit 202 is output to the face area detection unit 203.

[0017] The face area detection unit 203 detects a person's face area from the captured image based on the captured image after rotation by the image rotation unit 202. At this time, for example, the face area detection unit 203 detects the person's face area by detecting feature points of the person's face from the captured image based on the captured image after rotation by the image rotation unit 202. Note that the person's face area detected by the face area detection unit 203 is a rectangular area that is not inclined with respect to the state after rotation by the image rotation unit 202. Information indicating the person's face area detected by the face area detection unit 203 is output to the reference area detection unit 204.

[0018] At this time, for example, the face area detection unit 203 detects coordinates indicating the face area of ​​the person, i.e., coordinates indicating each vertex of a rectangular area that is the face area of ​​the person, based on the detected face area of ​​the person. The face area detection unit 203 then detects coordinates obtained by rotating the coordinates indicating the detected face area of ​​the person in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1. The face area detection unit 203 then outputs information indicating the coordinates after the reverse rotation to the reference area detection unit 204 as information indicating the face area of ​​the person. Alternatively, the face area detection unit 203 may output the coordinates indicating the detected face area of ​​the person to the reference area detection unit 204 as information indicating the face area of ​​the person. The reference area detection unit 204 may then detect coordinates obtained by rotating the coordinates indicating the face area of ​​the person acquired from the face area detection unit 203 in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1.

[0019] The reference area detection unit 204 detects, as a reference area, a rectangular area with no tilt that is inscribed in the area of ​​the person's face from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203. At this time, for example, the reference area detection unit 204 detects, as a reference area, a rectangular area with no tilt that is inscribed in the area of ​​the person's face and has a similar shape to the rectangular area with no tilt that circumscribes the area of ​​the person's face from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203. Information indicating the reference area detected by the reference area detection unit 204 is output to the information output unit 205.

[0020] Based on the reference area detected by the reference area detection unit 204, the information output unit 205 outputs information indicating the reference area or the average luminance in the reference area to the imaging device 1. Thereafter, the imaging device 1 performs luminance control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs luminance control on the captured image so that the average luminance based on the information output by the information output unit 205 approaches the target luminance. Note that the target luminance is a luminance that is predetermined as a luminance that enables detection of a person's face with high accuracy, and is set appropriately.

[0021] The occupant state detection unit 206 detects the state of the occupant based on the face area of ​​the person detected by the face area detection unit 203. The occupant state detected by the occupant state detection unit 206 includes information related to face detection or face authentication. Information related to face detection includes information on whether or not there is an occupant, whether the occupant is dozing, or whether the occupant is looking away from the vehicle. Information related to face authentication includes information on who the occupant is.

[0022] 2 illustrates a case where the information processing device 2 has an occupant state detection unit (person state detection unit) 206 and has a function of detecting the state of an occupant. However, the occupant state detection unit (person state detection unit) 206 is not an essential component of the information processing device 2, and the occupant state detection unit (person state detection unit) 206 does not necessarily have to be provided in the information processing device 2.

[0023] Next, an example of operation of the information processing device 2 according to the first embodiment shown in Fig. 2 will be described with reference to Fig. 3. In the example of operation of the information processing device 2 according to the first embodiment shown in Fig. 2, first, as shown in Fig. 3, the image acquisition unit 201 acquires a captured image from the imaging device 1 (step ST101). Information indicating the captured image acquired by the image acquisition unit 201 is output to the image rotation unit 202 and the reference area detection unit 204.

[0024] Next, based on the installation angle (roll angle) of the imaging device 1 relative to the reference angle, the image rotation unit 202 rotates the captured image acquired by the image acquisition unit 201 by the installation angle toward the reference angle (step ST102). Information indicating the captured image after rotation by the image rotation unit 202 is output to the face area detection unit 203.

[0025] For example, as shown in Fig. 4A, assume that the image capture device 1 is tilted by an installation angle with respect to a reference angle, and as a result, the occupant, who is a person, captured in the captured image acquired by the image acquisition unit 201, is tilted with respect to the captured image. In this case, as shown in Fig. 4B, for example, the image rotation unit 202 rotates the captured image so as to cancel out the tilt of the person captured in the captured image due to the installation angle of the image capture device 1.

[0026] Next, the face area detection unit 203 detects a person's face area from the captured image based on the captured image after rotation by the image rotation unit 202 (step ST103). At this time, for example, the face area detection unit 203 detects the person's face area by detecting feature points of the person's face from the captured image based on the captured image after rotation by the image rotation unit 202. Note that the person's face area detected by the face area detection unit 203 is a rectangular area that is not inclined with respect to the state after rotation by the image rotation unit 202. Information indicating the person's face area detected by the face area detection unit 203 is output to the reference area detection unit 204.

[0027] At this time, for example, the face area detection unit 203 detects coordinates indicating the face area of ​​the person, i.e., coordinates indicating each vertex of a rectangular area that is the face area of ​​the person, based on the detected face area of ​​the person. The face area detection unit 203 then detects coordinates obtained by rotating the coordinates indicating the detected face area of ​​the person in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1. The face area detection unit 203 then outputs information indicating the coordinates after the reverse rotation to the reference area detection unit 204 as information indicating the face area of ​​the person. Alternatively, the face area detection unit 203 may output the coordinates indicating the detected face area of ​​the person to the reference area detection unit 204 as information indicating the face area of ​​the person. The reference area detection unit 204 may then detect coordinates obtained by rotating the coordinates indicating the face area of ​​the person acquired from the face area detection unit 203 in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1.

[0028] For example, as shown in Fig. 5, the face area detection unit 203 detects a person's face area from the captured image after rotation by the image rotation unit 202 shown in Fig. 4B. In Fig. 5, reference numeral 11 indicates the person's face area detected by the face area detection unit 203. As shown in Fig. 5, the person's face area detected by the face area detection unit 203 is a rectangular area that is not tilted with respect to the state after rotation by the image rotation unit 202. In Fig. 5, the vertices of the person's face area detected by the face area detection unit 203 are designated a, b, c, and d.

[0029] Next, the reference area detection unit 204 detects, as a reference area, a non-tilted rectangular area inscribed in the area of ​​the person's face from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203 (step ST104). At this time, for example, the reference area detection unit 204 detects, as a reference area, a non-tilted rectangular area inscribed in the area of ​​the person's face, which has a similar shape to the non-tilted rectangular area circumscribed in the area of ​​the person's face, from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203. Information indicating the reference area detected by the reference area detection unit 204 is output to the information output unit 205.

[0030] For example, as shown in FIG. 6A , first, the reference area detection unit 204 detects, from the captured image acquired by the image acquisition unit 201, a rectangular area (circumscribed rectangular area) that is not tilted and circumscribes the face area of ​​the person detected by the face area detection unit 203. Note that an existing method can be used to detect, from a rectangular area, a rectangular area that circumscribes the rectangular area and is not tilted with respect to the captured image. In FIG. 6 , reference numeral 12 denotes the rectangular area (circumscribed rectangular area) that is not tilted and circumscribes the face area. Next, as shown in FIG. 6B , for example, the reference area detection unit 204 detects the diagonal of the circumscribed rectangular area as line segment ef from the captured image acquired by the image acquisition unit 201. Here, line segment ef is defined as a line segment that intersects with the long side of the face area of ​​the person detected by the face area detection unit 203. The reference area detection unit 204 then detects an intersection g between line segment ef and line segment ab, and an intersection h between line segment ef and line segment cd. Note that an existing method can be used to detect the intersection of the two straight lines. Next, as shown in Fig. 6C , for example, the reference area detection unit 204 detects, from intersection points g and h, an uninclined rectangular area (inscribed rectangular area) that is similar in shape to the uninclined rectangular area circumscribing the face area and that is inscribed in the face area. The reference area detection unit 204 then uses the detected inscribed rectangular area as a reference area for exposure control. In Fig. 6 , reference numeral 13 denotes the reference area (inscribed rectangular area).

[0031] Note that the above description shows a case in which the reference area detection unit 204 detects, as a reference area, from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203, an uninclined rectangular area that has a similar shape to an uninclined rectangular area circumscribing the area of ​​the person's face and that is inscribed in the area of ​​the person's face. However, the method of detecting a reference area by the reference area detection unit 204 is not limited to this, and any detection method may be used as long as the reference area detection unit 204 can detect, as a reference area, an uninclined rectangular area inscribed in the area of ​​the person's face detected by the face area detection unit 203 from the captured image acquired by the image acquisition unit 201. For example, the reference area detection unit 204 may directly detect, from the area of ​​the person's face, an uninclined rectangular area inscribed in the area of ​​the person's face, without detecting an uninclined rectangular area circumscribing the area of ​​the person's face.

[0032] Next, based on the reference area detected by the reference area detection unit 204, the information output unit 205 outputs information indicating the reference area or the average luminance in the reference area to the imaging device 1 (step ST105). Thereafter, the imaging device 1 performs luminance control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs luminance control on the captured image so that the average luminance based on the information output by the information output unit 205 approaches the target luminance.

[0033] Furthermore, the occupant state detection unit 206 detects information about the occupant based on the face area of ​​the person detected by the face area detection unit 203 (step ST106). The occupant state detected by the occupant state detection unit 206 can be information about face detection or face authentication. The face detection information can be information about whether or not there is an occupant, whether the occupant is dozing, or whether the occupant is looking away. The face authentication information can be information about who the occupant is.

[0034] In this way, the information processing device 2 according to embodiment 1 rotates the captured image acquired from the imaging device 1 in accordance with the installation angle of the imaging device 1, detects a face area from the rotated captured image, and detects, as a reference area, a rectangular area with no tilt that is inscribed in the face area from the captured image acquired from the imaging device 1. This makes it possible for the information processing device 2 according to embodiment 1 to handle imaging devices 1 that require a rectangular area with no tilt relative to the captured image to be used as the reference area, and also makes it possible to prevent the background area from being included in the reference area even when the face area is tilted.

[0035] As described above, according to this embodiment 1, the information processing device 2 includes an image acquisition unit 201 that acquires a captured image of a person from the imaging device 1; an image rotation unit 202 that rotates the captured image acquired by the image acquisition unit 201 toward the reference angle by the amount of the installation angle of the imaging device 1 relative to the reference angle; a face area detection unit 203 that detects the area of ​​the person's face from the captured image after rotation by the image rotation unit 202; a reference area detection unit 204 that detects, as a reference area, a non-inclined rectangular area inscribed in the area of ​​the person's face from the captured image acquired by the image acquisition unit 201 based on the area of ​​the person's face detected by the face area detection unit 203; and an information output unit 205 that outputs, to the imaging device 1, information indicating the reference area or the average brightness in the reference area based on the reference area detected by the reference area detection unit 204. Furthermore, according to this embodiment 1, the reference area detection unit 204 detects, as a reference area, from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203, an untilted rectangular area that is similar in shape to an untilted rectangular area circumscribing the area of ​​the person's face and that is untilted inscribed in the area of ​​the person's face. As a result, the information processing device 2 according to embodiment 1 can detect an appropriate reference area even when detecting a reference area that is untilted relative to the captured image as a reference area for exposure control. In other words, the information processing device 2 according to embodiment 1 can detect a reference area for optimal exposure control regardless of the position of the person's face and the tilt of the imaging device 1. As a result, the information processing device 2 according to embodiment 1 enables stable exposure control in the imaging device 1.

[0036] Furthermore, according to this embodiment 1, the information processing device 2 includes a person state detection unit 206 that detects the state of a person based on the area of ​​the person's face detected by the face area detection unit 203. This makes it possible for the information processing device 2 according to embodiment 1 to detect the state of a person, for example, information related to face detection or face authentication, with higher accuracy than conventional devices.

[0037] Furthermore, according to this embodiment 1, the information processing method includes the steps of: an image acquisition unit 201 acquiring a captured image showing a person from the imaging device 1; an image rotation unit 202 rotating the captured image acquired by the image acquisition unit 201 toward the reference angle by the amount of the installation angle of the imaging device 1 relative to the reference angle, based on the installation angle of the imaging device 1 relative to the reference angle; a face area detection unit 203 detecting a face area of ​​the person from the captured image after rotation by the image rotation unit 202; a reference area detection unit 204 detecting, as a reference area, a non-inclined rectangular area inscribed in the face area of ​​the person from the captured image acquired by the image acquisition unit 201, based on the face area of ​​the person detected by the face area detection unit 203; and an information output unit 205 outputting, to the imaging device 1, information indicating the reference area or the average brightness in the reference area, based on the reference area detected by the reference area detection unit 204. As a result, the information processing method according to the first embodiment makes it possible to detect an appropriate reference area even when detecting a reference area for exposure control that is not tilted with respect to the captured image. That is, the information processing method according to the first embodiment makes it possible to detect a reference area for optimal exposure control regardless of the position of a person's face and the tilt of the imaging device 1. As a result, the information processing method according to the first embodiment makes it possible to perform stable exposure control in the imaging device 1.

[0038] Embodiment 2. Figure 7 is a diagram showing an example of the configuration of an information processing device 2 according to embodiment 2. In the information processing device 2 according to embodiment 2 shown in Figure 7, an attached accessory detection unit 207 is added to the information processing device 2 according to embodiment 1 shown in Figure 2. The other example of the configuration of the information processing device 2 according to embodiment 2 shown in Figure 7 is the same as the example of the configuration of the information processing device 2 according to embodiment 1 shown in Figure 2, and the same reference numerals are used, and only the different parts will be described.

[0039] The worn accessory detection unit 207 detects whether a person is wearing an accessory based on the captured image acquired by the image acquisition unit 201. In this case, for example, the worn accessory detection unit 207 detects whether a person is wearing an accessory by comparing the area of ​​the person's face in the captured image acquired by the image acquisition unit 201 with reference face information. Here, the reference face information is, for example, face information when the person is not wearing an accessory or face information when the person is wearing an accessory. Furthermore, the worn accessory detection unit 207 may use the area of ​​the person's face detected by the face area detection unit 203 as the area of ​​the person's face in the captured image acquired by the image acquisition unit 201. Furthermore, accessories that the worn accessory detection unit 207 detects include, for example, masks, glasses, sunglasses, etc. Information indicating the detection result by the worn accessory detection unit 207 is output to the reference area detection unit 204.

[0040] In addition, in embodiment 2, when the worn accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory from the captured image acquired by the image acquisition unit 201 from the detection targets for the reference area.

[0041] Next, an example of operation of the information processing device 2 according to the second embodiment shown in Fig. 7 will be described with reference to Fig. 8. In the example of operation of the information processing device 2 according to the second embodiment shown in Fig. 7, first, as shown in Fig. 8, for example, the image acquisition unit 201 acquires a captured image from the imaging device 1 (step ST201). Information indicating the captured image acquired by the image acquisition unit 201 is output to the image rotation unit 202 and the reference area detection unit 204.

[0042] Next, based on the installation angle (roll angle) of the imaging device 1 relative to the reference angle, the image rotation unit 202 rotates the captured image acquired by the image acquisition unit 201 by the installation angle toward the reference angle (step ST202). Information indicating the captured image after rotation by the image rotation unit 202 is output to the face area detection unit 203.

[0043] Next, the face area detection unit 203 detects a person's face area from the captured image based on the captured image after rotation by the image rotation unit 202 (step ST203). At this time, for example, the face area detection unit 203 detects the person's face area by detecting feature points of the person's face from the captured image based on the captured image acquired by the image acquisition unit 201. Note that the person's face area detected by the face area detection unit 203 is a rectangular area that is not inclined with respect to the captured image after rotation by the image rotation unit 202. Information indicating the person's face area detected by the face area detection unit 203 is output to the reference area detection unit 204.

[0044] At this time, for example, the face area detection unit 203 detects coordinates indicating the face area of ​​the person, i.e., coordinates indicating each vertex of a rectangular area that is the face area of ​​the person, based on the detected face area of ​​the person. The face area detection unit 203 then detects coordinates obtained by rotating the coordinates indicating the detected face area of ​​the person in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1. The face area detection unit 203 then outputs information indicating the coordinates after the reverse rotation to the reference area detection unit 204 as information indicating the face area of ​​the person. Alternatively, the face area detection unit 203 may output the coordinates indicating the detected face area of ​​the person to the reference area detection unit 204 as information indicating the face area of ​​the person. The reference area detection unit 204 may then detect coordinates obtained by rotating the coordinates indicating the face area of ​​the person acquired from the face area detection unit 203 in a direction opposite to the reference angle by an amount corresponding to the installation angle of the imaging device 1.

[0045] Furthermore, the worn accessory detection unit 207 detects whether or not the person is wearing an accessory based on the captured image acquired by the image acquisition unit 201 (step ST204). At this time, for example, the worn accessory detection unit 207 detects whether or not the person is wearing an accessory by comparing the area of ​​the person's face in the captured image acquired by the image acquisition unit 201 with reference face information. Information indicating the detection result by the worn accessory detection unit 207 is output to the reference area detection unit 204.

[0046] Next, the reference area detection unit 204 detects, as a reference area, a non-tilted rectangular area inscribed in the area of ​​the person's face from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203 (step ST205). At this time, for example, the reference area detection unit 204 detects, as a reference area, from the captured image acquired by the image acquisition unit 201, based on the area of ​​the person's face detected by the face area detection unit 203, a non-tilted rectangular area that has a similar shape to the non-tilted rectangular area circumscribed in the area of ​​the person's face and that is inscribed in the area of ​​the person's face.

[0047] In addition, in embodiment 2, when the worn accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory from the captured image acquired by the image acquisition unit 201 from the detection targets for the reference area.

[0048] For example, as shown in FIG. 9A , when the worn accessory detection unit 207 detects that the person is not wearing an accessory, the reference area detection unit 204 detects the reference area using the procedure described in embodiment 1. FIG. 9A is the same as FIG. 6C . On the other hand, when the worn accessory detection unit 207 detects that the person is wearing an accessory, as shown in FIG. 9B , the reference area detection unit 204 excludes the area of ​​the accessory in the captured image acquired by the image acquisition unit 201 from the detection target for the reference area. Note that FIG. 9B shows an example where the person is wearing a mask, and in this case, the reference area detection unit 204 detects the area above the mask area as the detection target for the reference area. In FIG. 9B , reference numeral 14 denotes a mask.

[0049] Note that Figure 9B shows a case where the circumscribing rectangular area is set outside the area of ​​the accessory, but this is not limited to this, and it is sufficient that the area of ​​the person's face detected by the face area detection unit 203 is outside the area of ​​the accessory.

[0050] Information indicating the reference area detected by the reference area detection unit 204 is output to the information output unit 205 .

[0051] Next, based on the reference area detected by the reference area detection unit 204, the information output unit 205 outputs information indicating the reference area or the average luminance in the reference area to the imaging device 1 (step ST206). Thereafter, the imaging device 1 performs luminance control on the captured image based on the information output by the information output unit 205. That is, for example, the imaging device 1 performs luminance control on the captured image so that the average luminance based on the information output by the information output unit 205 approaches the target luminance.

[0052] Furthermore, the occupant state detection unit 206 detects the state of the occupant based on the area of ​​the person's face detected by the face area detection unit 203 (step ST207). The occupant state detected by the occupant state detection unit 206 can include information related to face detection or face authentication. Information related to face detection can include information on whether or not there is an occupant, whether the occupant is dozing, or whether the occupant is looking away from the vehicle. Information related to face authentication can include information on who the occupant is.

[0053] As described above, according to the second embodiment, the information processing device 2 includes the worn accessory detection unit 207 that detects whether or not a person is wearing an accessory based on the captured image acquired by the image acquisition unit 201, and when the worn accessory detection unit 207 detects that a person is wearing an accessory, the reference area detection unit 204 excludes the area of ​​the accessory in the captured image acquired by the image acquisition unit 201 from the detection target for the reference area. Also, according to the second embodiment, the accessory that is the detection target for the worn accessory detection unit 207 is a mask. As a result, the information processing device 2 according to the second embodiment can detect a reference area for optimal exposure control, in addition to the effects of the first embodiment, regardless of whether or not a person is wearing an accessory.

[0054] Finally, with reference to FIG. 10 , an example of the hardware configuration of the information processing device 2 according to the first and second embodiments will be described. Note that, while an example of the hardware configuration of the information processing device 2 according to the first embodiment will be described below, the same applies to an example of the hardware configuration of the information processing device 2 according to the second embodiment. The functions of the image acquisition unit 201, image rotation unit 202, face area detection unit 203, reference area detection unit 204, information output unit 205, and person state detection unit 206 in the information processing device 2 are realized by a processing circuit 51. The processing circuit 51 may be dedicated hardware as shown in FIG. 10A , or may be a CPU (also referred to as a central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 that executes a program stored in a memory 53 as shown in FIG. 10B .

[0055] When the processing circuitry 51 is dedicated hardware, the processing circuitry 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of each of the image acquisition unit 201, the image rotation unit 202, the face area detection unit 203, the reference area detection unit 204, the information output unit 205, and the person state detection unit 206 may be realized individually by the processing circuitry 51, or the functions of each unit may be realized collectively by the processing circuitry 51.

[0056] When the processing circuit 51 is a CPU 52, the functions of the image acquisition unit 201, image rotation unit 202, face area detection unit 203, reference area detection unit 204, information output unit 205, and person state detection unit 206 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 53. The processing circuit 51 realizes the functions of each unit by reading and executing the programs stored in the memory 53. That is, the information processing device 2 includes a memory for storing programs that, when executed by the processing circuit 51, result in the execution of, for example, each step shown in FIG. 3 . Furthermore, these programs can also be said to cause a computer to execute the procedures and methods of the image acquisition unit 201, image rotation unit 202, face area detection unit 203, reference area detection unit 204, information output unit 205, and person state detection unit 206. Here, examples of the memory 53 include non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs (Digital Versatile Discs).

[0057] It should be noted that the functions of the image acquisition unit 201, image rotation unit 202, face area detection unit 203, reference area detection unit 204, information output unit 205, and person state detection unit 206 may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the function of the image acquisition unit 201 may be implemented by the processing circuit 51 as dedicated hardware, and the functions of the image rotation unit 202, face area detection unit 203, reference area detection unit 204, information output unit 205, and person state detection unit 206 may be implemented by the processing circuit 51 reading out and executing programs stored in the memory 53.

[0058] In this way, the processing circuitry 51 can realize each of the above-described functions by hardware, software, firmware, or a combination of these.

[0059] It should be noted that the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted.

[0060] The information processing device of the present disclosure is capable of detecting a more suitable reference area than conventional devices, even when detecting a reference area that is not tilted relative to a captured image as a reference area for exposure control, and is suitable for use in information processing devices, etc.

[0061] REFERENCE SIGNS LIST 1 Imaging device, 2 Information processing device, 51 Processing circuit, 52 CPU, 53 Memory, 201 Image acquisition unit, 202 Image rotation unit, 203 Face area detection unit, 204 Reference area detection unit, 205 Information output unit, 206 Occupant state detection unit (person state detection unit), 207 Mounted accessory detection unit.

Claims

1. An information processing device comprising: an image acquisition unit that acquires an image of a person from an imaging device; an image rotation unit that rotates the image acquired by the image acquisition unit toward a reference angle by the amount of the installation angle of the imaging device based on the installation angle of the imaging device relative to the reference angle; a face area detection unit that detects a face area of ​​the person from the image after rotation by the image rotation unit; a reference area detection unit that detects a non-inclined rectangular area inscribed in the face area of ​​the person from the image acquired by the image acquisition unit as a reference area based on the face area of ​​the person detected by the face area detection unit; and an information output unit that outputs information indicating the reference area or the average luminance of the reference area to the imaging device based on the reference area detected by the reference area detection unit.

2. The information processing device according to claim 1, characterized in that the reference area detection unit detects, based on the area of ​​a person's face detected by the face area detection unit, a rectangular area from the captured image acquired by the image acquisition unit that is similar in shape to a rectangular area with no tilt that circumscribes the area of ​​the person's face and that is also inscribed in the area of ​​the person's face, as a reference area.

3. An information processing device as described in claim 1, further comprising a worn accessory detection unit that detects whether or not a person is wearing an accessory based on the captured image acquired by the image acquisition unit, and wherein when the worn accessory detection unit detects that the person is wearing an accessory, the reference area detection unit excludes the area of ​​the accessory from the captured image acquired by the image acquisition unit as a detection target for the reference area.

4. The information processing device according to claim 3, wherein the accessory to be detected by the attached accessory detection unit is a mask.

5. An information processing apparatus according to claim 1, further comprising a person state detection unit that detects the state of a person based on the area of ​​the person's face detected by said face area detection unit.

6. An information processing method comprising the steps of: an image acquisition unit acquiring an image of a person from an imaging device; an image rotation unit rotating the image acquired by the image acquisition unit toward a reference angle by the installation angle of the imaging device based on the installation angle of the imaging device relative to the reference angle; a face area detection unit detecting a face area of ​​the person from the image after rotation by the image rotation unit; a reference area detection unit detecting, as a reference area, a rectangular area with no inclination that is inscribed in the face area of ​​the person from the image acquired by the image acquisition unit based on the face area of ​​the person detected by the face area detection unit; and an information output unit outputting, to the imaging device, information indicating the reference area or the average brightness in the reference area based on the reference area detected by the reference area detection unit.

Images