Image processing device, image processing method, and program

Abstract

To improve the accuracy of extracting an image capturing the face of a person from an image.SOLUTION: An image processing apparatus comprises: detection means that detects a face candidate area from an image by using a first face detector; acquisition means that acquires the positions of facial organs from an image of a first area detected as the face candidate area by using a facial organ detector; creation means that creates a converted image obtained by converting the image of the first area based on the positions of the facial organs; and control means that controls whether to output the image of the first area based on a detection result of detecting the face from the converted image by using a second face detector.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, as a method for detecting a human face from an image, there is a method for searching for a human face region on an image and outputting a face candidate region having a likelihood equal to or greater than a certain value as a detection result. Since a face may not exist in the face candidate region detected by this method, it is further determined whether or not a face candidate region having a high likelihood is a face region. In addition, when creating images for training a machine learning model for a face detection method and a face recognition method, and images for use in a face recognition system, it is necessary to eliminate low-quality images that do not include a human face.

[0003] Taking the above problems into consideration, Patent Document 1 proposes a method of detecting facial organs such as the inner corners of the eyes and the center of the mouth from face candidate areas with high likelihood, and determining whether the face candidate areas with high likelihood are face areas or not based on the number of facial organs detected. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4884251 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method of Patent Document 1 determines a face candidate area with high likelihood to be a face area when a human face is not present in the face candidate area with high likelihood but the number of detected face parts is large. On the other hand, the method of Patent Document 1 determines a face candidate area with low likelihood to not be a face area when a human face is included in the face candidate area with low likelihood but the number of detected face parts is small. Thus, the technology of Patent Document 1 has a problem that it creates an image without a face because of low accuracy in determining whether a face candidate area is a face area or not.

[0006] Therefore, an object of the present invention is to improve the accuracy of extracting images in which a person's face appears from an image. [Means for solving the problem]

[0007] In order to achieve the object of the present invention, an image processing device according to an embodiment of the present invention has the following configuration: That is, the image processing device has a detection means for detecting a candidate face region from an image using a face detector, an acquisition means for acquiring the positions of face organs from the image of the candidate face region detected by the detection means using a face organ detector, a generation means for generating a converted image by converting the image of the candidate face region based on the positions of the face organs, and a control means for controlling whether or not to output the image of the candidate face region as a face image based on a result of performing face detection processing on the converted image. The first face detector for detecting candidate face areas is composed of a neural network different from a second face detector used to perform face detection processing on the converted image, the first face detector is a neural network having a smaller neural network scale and a lower correct detection rate than the second face detector, and the second face detector is a neural network having a larger neural network scale and a higher correct detection rate than the first face detector. Effect of the Invention

[0008] According to the present invention, it is possible to improve the accuracy of extracting images in which a person's face appears from an image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing an example of the hardware configuration of an image processing system according to a first embodiment. [Diagram 2] FIG. 1 is a diagram showing an example of the hardware configuration of an image processing apparatus according to a first embodiment. [Diagram 3]FIG. 1 is a block diagram showing an example of the functional configuration of an image processing apparatus according to a first embodiment. [Figure 4] FIG. 2 is a view showing an example of an image to be subjected to image processing according to the first embodiment. [Diagram 5] FIG. 4 is a view showing an example of a detection result of a face candidate region according to the first embodiment. [Figure 6] FIG. 4 is a view showing an example of a detection result of face parts detected from a face candidate region according to the first embodiment. [Figure 7] 5 is a flowchart illustrating a flow of processing for generating a converted image according to the first embodiment. [Figure 8] FIG. 2 is a diagram for explaining the correspondence relationship between positions of face features according to the first embodiment. [Figure 9] FIG. 2 is a diagram showing an overview of generating a converted image from an image according to the first embodiment. [Figure 10] 5 is a flowchart illustrating the flow of image processing according to the first embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of the functional configuration of an image processing apparatus according to a second embodiment. [Figure 12] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 13] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 14] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 15] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 16] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 17] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 18] 10 is a flowchart illustrating the flow of image processing according to the second embodiment. [Figure 19] FIG. 11 is a block diagram showing an example of the functional configuration of an image processing apparatus according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] (First embodiment) This embodiment can be used as an image processing system that combines an imaging device and an image processing device.

[0012] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing system according to the first embodiment.

[0013] The image processing system 10 includes an imaging device 100 , an image processing device 110 , and a network 120 .

[0014] The imaging device 100 is a camera that captures an image of a subject, and may be, for example, a digital camera or a network camera. The imaging device 100 is one camera, but may be two or more cameras.

[0015] The image processing device 110 is a device that detects a person's face candidate region and the positions of facial parts from an image, and is, for example, a desktop computer or a laptop computer, but is not limited to this. The image processing device 110 may be, for example, a smartphone or a tablet terminal.

[0016] The network 120 connects the imaging device 100 and the image processing device 110. The network 120 is, for example, a wired LAN or a wireless LAN.

[0017] FIG. 2 is a block diagram showing an example of a hardware configuration of the image processing device according to the first embodiment.

[0018] The image processing device 110 includes an input unit 201 , a display unit 202 , an I / F 203 , a CPU 204 , a RAM 205 , a ROM 206 , a storage unit 207 , and a data bus 208 .

[0019] The input unit 201 is a device through which a user inputs various data, and includes, for example, a keyboard, a mouse, and a touch panel.

[0020] The display unit 202 is a device that displays various data, and includes, for example, a liquid crystal display (LCD).

[0021] The I / F 203 transmits and receives various types of information between the image processing device 110 and other devices (not shown) via a network 120 such as the Internet.

[0022] The CPU 204 is a processor that comprehensively controls each unit in the image processing device 110. The CPU 204 performs various controls by reading out a control program in the ROM 206, loading it into the RAM 205, and executing the program. The CPU 204 executes the image processing program in the ROM 206 and the storage unit 207, thereby realizing image processing of image data.

[0023] The RAM 205 is a temporary storage area for programs executed by the CPU 204, a work memory, and the like.

[0024] The ROM 206 stores a control program for controlling each unit in the image processing device 110 .

[0025] The storage unit 207 is a device that stores various data, and stores, for example, image data, setting parameters, various programs, etc. The storage unit 207 can also store data from an external device (not shown) via the I / F 203.

[0026] The data bus 208 is a transmission path for transmitting data, and transmits image data and the like received from an external device via the I / F 203 to the CPU 204, the RAM 205, and the ROM 206. The data bus 208 also transmits image data and the like from the image processing device 110 to the external device.

[0027] FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing device according to the first embodiment.

[0028] The image processing device 110 includes a face area detection unit 300, a face organ detection unit 301, a generation unit 302, a face detection unit 303, a determination unit 304, a DNN_A 305, a DNN_B 306, and a DNN_C 307. DNN (Deep Neural Network) is an abbreviation for deep neural network.

[0029] The face area detection unit 300 acquires an image from the storage unit 207 or the like. The image is at least one of an image captured by the imaging device 100 and an image previously stored in the storage unit 207 or the like. The face area detection unit 300 detects a face candidate area of ​​a person in the image using the DNN_A 305, and generates an image including the face candidate area. The face area detection unit 300 transmits the image including the face candidate area to the face organ detection unit 301.

[0030] The face organ detection unit 301 receives an image including a face candidate region from the face region detection unit 300. The face organ detection unit 301 detects face organs from the face candidate region of the image using DNN_B 306. Furthermore, the face organ detection unit 301 transmits information on the face organs detected from the face candidate region to the generation unit 302. The information on the face organs includes, for example, information on the position (two-dimensional coordinates), size, and orientation of the face organs.

[0031] The generating unit 302 receives information on face organs detected from the face candidate region from the face organ detection unit 301. The generating unit 302 transforms the image by a geometric transformation (e.g., affine transformation) based on the information on the face organs to generate a transformed image. Furthermore, the generating unit 302 transmits the transformed image to the face detection unit 303.

[0032] The face detection unit 303 receives the converted image from the generation unit 302. The face detection unit 303 detects a human face from the converted image using the DNN_C 307. Furthermore, the face detection unit 303 transmits the detection result to the determination unit 304.

[0033] The determination unit 304 receives the converted image from the generation unit 302. The determination unit 304 receives the detection result of the face of the person detected in the converted image from the face detection unit 303. Then, based on the detection result of the face of the person in the converted image, the determination unit 304 determines whether or not to output information indicating a detection area detected as a face candidate area from the original image as information indicating a face area. Furthermore, based on the determination result, the determination unit 304 outputs the detection area of ​​the original image determined to be a face area to the storage unit 207, etc.

[0034] In this embodiment, the DNN_A305 of the face area detection unit 300, the DNN_B306 of the face organ detection unit 301, and the DNN_C307 of the face detection unit 303 are different from each other, but are not limited to this. For example, the DNN of the face area detection unit 300 and the DNN of the face detection unit 303 may be the same. In addition, when a DNN capable of simultaneously detecting a face candidate area and face organs is available, the DNNs of the face area detection unit 300, the face organ detection unit 301, and the face detection unit 303 may all be the same. Here, the DNN of the face area detection unit 300 is defined as a "first face detector", the DNN of the face organ detection unit 301 is defined as a "face organ detector", and the DNN of the face detection unit 303 is defined as a "second face detector".

[0035] Generally, the DNN is trained using images captured under various shooting conditions for a person, such as the size of the image to be processed, and the size and orientation of the face of the person in the image. As a result, the face area detection unit 300 and the face detection unit 303 can detect the face candidate area and the face area from the image with high accuracy using the DNN trained using images of people captured under various shooting conditions. In addition, the face organ detection unit 301 can detect the position of the face organ from the image with high accuracy using the DNN trained using the image of the face candidate area (corresponding to the image of the first area) detected by the face area detection unit 300 from the image of people captured under various shooting conditions. Below, examples 1 to 3 will be described, which are effective in improving the detection accuracy of the face candidate area and the face area by applying different DNNs to each of the face area detection unit 300 and the face detection unit 303. Note that the methods of examples 1 to 3 are independent of each other, but a method that combines examples 1 to 3 may be used.

[0036] (Example 1) When the size of the input image to the face area detection unit 300 is arbitrary and the size of the input image to the face detection unit 303 is fixed, the DNN of the face area detection unit 300 and the DNN of the face detection unit 303 may be different from each other. For example, the DNN of the face area detection unit 300 is a general-purpose DNN that can operate regardless of the size of the image. On the other hand, the DNN of the face detection unit 303 is a DNN obtained by additionally training the general-purpose DNN with training data with a fixed image size.

[0037] (Example 2) The size and orientation of the face of a person appearing in an input image of the face area detection unit 300 varies greatly. On the other hand, the size and orientation of the face of a person appearing in an input image of the face detection unit 303, i.e., a converted image generated (converted) by the generation unit 302, varies less than the size and orientation of the face of a person appearing in an input image of the face area detection unit 300. In this case, the DNN of the face area detection unit 300 and the DNN of the face detection unit 303 may be different from each other. For example, the DNN of the face area detection unit 300 is a general-purpose DNN that can operate regardless of the size and orientation of the face. On the other hand, the DNN of the face detection unit 303 is a DNN obtained by additionally learning the general-purpose DNN using learning data (i.e., a converted image) created using a conversion method by the generation unit 302. The additional learning of the general-purpose DNN may be performed in advance using software and a learning device other than the image processing device 110. Alternatively, as in a third embodiment described later, learning of a general-purpose DNN may be performed using a learning unit 1905 connected to the image processing device 110 and an input image of the image processing device 110.

[0038] (Example 3) In face recognition systems that require high-speed operation in real time, and face recognition systems that operate on computers (e.g., cameras, smartphones) with small physical hardware size and low processing performance, face detection processing with a low amount of calculation is required. In this case, a combination of DNN_1, which has a small number of DNN layers and a low correct detection rate, and DNN_2, which has a large number of DNN layers and a high correct detection rate, may be applied to the face recognition system. In general, the larger the image size, the greater the amount of calculation required by the DNN. Therefore, for example, when the input image of the face area detection unit 300 is larger than the input image of the face detection unit 303, it is preferable to apply DNN_1 to the face area detection unit 300 and DNN_2 to the face detection unit 303. Also, when the input image of the face area detection unit 300 is smaller than the input image of the face detection unit 303, it is preferable to apply DNN_2 to the face area detection unit 300 and DNN_1 to the face detection unit 303. On the other hand, for example, when there is a demand to minimize the amount of calculation of the DNN due to design constraints such as the allocation of computational resources of the face authentication system, it is preferable to apply DNN_1 that can operate with the least computational resources to each of the face area detection unit 300 and the face detection unit 303. In addition, when there are many computational resources that can be allocated to the face authentication system, it is preferable to apply DNN_2 that can operate with the most computational resources to each of the face area detection unit 300 and the face detection unit 303 from the viewpoint of maximizing the authentication accuracy of the face authentication system. It is not limited to the above examples of DNN_1 and DNN_2, but it is also possible to provide DNN_3 that can operate with computational resources intermediate between DNN_1 and DNN_2. In other words, it is possible to appropriately select the DNNs to be allocated to each of the face area detection unit 300 and the face detection unit 303 based on the overall computational resources of the face authentication system, DNNs that can operate with various computational resources, and the operating environment.

[0039] 4 is a diagram showing an example of an image to be subjected to image processing according to the first embodiment. Three people are shown in the image 400. The number of people in the image 400 is not limited to three, and may be two or less, or four or more.

[0040] FIG. 5 is a diagram showing an example of a detection result of a face candidate region according to the first embodiment.

[0041] 5 shows an example of five face candidate regions detected by the face region detection section 300 from an image 400. The image 400 includes a region 510, a region 520, a region 530, a region 540, and a region 550.

[0042] Region 510 is a region that includes the person's face. Region 520 is a region that includes a part of the person's right side as a region that is a part of the person. Region 530 is a region that includes the person's face. Region 540 is a region that does not include the person's face. Region 550 is a region that does not include the person's face. A part of region 550 is located outside image 400. In this case, face region detection unit 300 complements a part of region 550 that is located outside image 400 with pixels having a luminance value of 0. Note that face region detection unit 300 may complement a part of region 550 with a luminance value other than 0, for example, or may complement a part of region 550 with a partial region of the original image or a luminance value that is the inverse of the partial region.

[0043] FIG. 6 is a diagram showing an example of a detection result of face parts detected from a face candidate region according to the first embodiment.

[0044] 6 shows an example in which the face organ detection unit 301 detects five face organs from the area 510. The area 510 includes a face organ 610, a face organ 620, a face organ 630, a face organ 640, and a face organ 650.

[0045] In this way, when the region 510 includes all of the face organs 610 to 650, the generation unit 302 generates a partial image (described later) by converting the region 510. On the other hand, when the region 510 includes four or fewer face organs, or when the position of a face organ is outside the region 510, the generation unit 302 ends the process without generating a partial image from the region 510.

[0046] 7 is a flowchart for explaining the flow of processing for generating a converted image according to the first embodiment. The generating unit 302 performs geometric image transformation on an image based on information (e.g., positions) of face parts to generate a converted image. The generating process of the converted image by the generating unit 302 will be explained below.

[0047] In S700, the generation unit 302 generates a region of the converted image. The region of the converted image is a region that is set based on the size and shape of the desired image. For example, the generation unit 302 generates a square region of 112 x 112 pixels as the region of the converted image.

[0048] In S701, the generation unit 302 sets, in the transformed image area, face organ reference positions 810-850 (FIG. 8) corresponding to the five face organs 610-650. The face organ reference positions refer to positions that are predefined as positions of face organs in the transformed image area.

[0049] When generating a converted image, the generating unit 302 converts the image so that the positions of the five face organs 610 to 650 are as close as possible to the reference positions 810 to 850. For example, the reference positions 810 to 850 in Fig. 8 are the average positions of the face organs obtained from face images of many people photographed from the front.

[0050] An example of the reference positions 810 to 850 will be described. First, if the upper left corner of a 112x112 pixel square area is taken as the origin, the right direction of the image is the positive X direction, and the downward direction of the image is the positive Y direction. Next, the positions of the center of the left eye, the center of the right eye, the tip of the nose, the left end point of the mouth, and the right end point of the mouth are expressed by two-dimensional coordinates (X, Y). Thus, the generation unit 302 sets two-dimensional coordinates (40, 60), (73, 60), (55, 80), (42, 100), and (81, 100) corresponding to the reference positions 810 to 850, respectively.

[0051] In S702, the generation unit 302 calculates a transformation matrix for generating a transformed image. For example, the generation unit 302 calculates a transformation matrix M that brings the positions of the five face parts 610-650 as close as possible to the reference positions 810-850 by image transformation processing.

[0052] FIG. 8 is a diagram for explaining the correspondence relationship between the positions of face features according to the first embodiment.

[0053] 8 shows the correspondence between the positions of five face organs 610-650 and reference positions 810-850. Generally, when there are four or more pairs of correspondence relationships between the positions of face organs and the reference positions, there is no transformation matrix M in which all differences between the positions of face organs and the reference positions are 0. Therefore, the generation unit 302 calculates a transformation matrix M that minimizes the difference between the positions of face organs and the reference positions by a convergence calculation of the difference between the positions of face organs and the reference positions.

[0054] The generating unit 302 calculates the sum of the differences between the positions of the face organs 610-650 and the reference positions 810-850 as the magnitude of the difference, but is not limited to this. The generating unit 302 may calculate the sum of the differences with a bias by lowering the distance between certain points. For example, the tip of the nose (face organ 630) is far from the face surface, so there are cases where the positional deviation is large with respect to the change in the face direction. Therefore, the generating unit 302 may calculate the difference by multiplying the difference in the position between the tip of the nose point (face organ 630) and the reference position 830 by 0.5.

[0055] In S703, the generation unit 302 generates a transformed image by transforming the original image 400 using the transformation matrix M calculated in S702.

[0056] Here, FIG. 9 is a diagram showing an overview of generating a converted image from an image according to the first embodiment.

[0057] 9 shows image 400 before conversion processing and converted image 900 after conversion processing is performed on image 400. Image 400 includes region 510. Converted image 900 is an image obtained by rotating image 400 counterclockwise, and includes partial image 910.

[0058] In S704 , the generation unit 302 cuts out a partial image 910 from the converted image 900 , and stores the partial image 910 in the storage unit 207 .

[0059] FIG. 10 is a flowchart illustrating the flow of image processing according to the first embodiment.

[0060] In S1000, the user prepares an image containing a person's face. For example, the user prepares an image of a person photographed with a general digital camera. Below, a case where the image processing device 110 detects a person's face from one image will be described. When the image processing device 110 detects a person's face from two or more images, the image processing device 110 sequentially performs a process of detecting a person's face from one image according to the number of images. This allows the image processing device 110 to detect a person's face regardless of the number of images.

[0061] In S1001, the face area detection unit 300 detects a human face candidate area from an image acquired from the storage unit 207, etc. Here, the face area detection unit 300 detects a rectangular area surrounded by line segments parallel to the vertical and horizontal directions of the image as a face candidate area. The face candidate area is not limited to a rectangular area, and may be, for example, an elliptical area centered on the center of the face.

[0062] In S1002, the face organ detection unit 301 detects the positions of face organs from the face candidate region. The face organs are five organs including the center of the left eye, the center of the right eye, the tip of the nose, the left end point of the mouth, and the right end point of the mouth, but are not limited to these and may be other organs. The face organs may be, for example, the four end points of the eyes or mouth, i.e., top, bottom, left, and right.

[0063] In S1003, the generation unit 302 determines whether or not the five face organs are located in the face candidate region, based on the information on the face organs in the face candidate region received from the face organ detection unit 301. If the generation unit 302 determines that the five face organs are located in the face candidate region (Yes in S1003), the process proceeds to S1004. If the generation unit 302 determines that the five face organs are not located in the face candidate region (No in S1003), the process ends.

[0064] In S1004, the generation unit 302 performs a geometric image transformation (for example, an affine transformation) on the image based on the position information of the face parts, thereby generating a transformed image.

[0065] For example, the generating unit 302 sets in advance coordinates corresponding to five facial organs in the transformed image. The generating unit 302 calculates a transformation matrix M that minimizes the difference between the positions (coordinates) of the five facial organs in the image before transformation and the positions (coordinates) of the five facial organs in the transformed image. The generating unit 302 performs a geometric image transformation on the image using the transformation matrix M to generate the transformed image. Note that the method of transforming the image is not limited to the method using the transformation matrix M, and may be, for example, a method of rotating the image so that the left eye and the right eye are horizontal.

[0066] In S1005, the face detection unit 303 detects a face area from the converted image. Here, the face detection unit 303 detects only the face area from the converted image, and does not detect the positions of face parts.

[0067] In S1006, the determination unit 304 determines whether or not a face area is detected from the converted image. If the determination unit 304 determines that a face area is detected from the converted image (Yes in S1006), the process proceeds to S1007. If the determination unit 304 determines that a face area is not detected from the converted image (No in S1006), the process ends.

[0068] In S1007, the determination unit 304 outputs the image (partial image) including the face candidate region detected in S1001 to an external device (for example, the storage unit 207).

[0069] Note that the determination unit 304 further determines whether the face candidate region is a face region based on whether a face region is detected from the converted image, but the above determination may be made using another determination method. For example, the determination unit 304 may determine that the face candidate region is a face region when a face region of a certain size or more is detected from the converted image. Furthermore, the determination unit 304 may determine that the face candidate region is a face region when the face region detected from the converted image is of a certain size or more and the distance between the center of the converted image and the center of the face region is within a threshold value.

[0070] In general, the face detection unit 303 can easily detect a face region from the converted image 900. In other words, if the face detection unit 303 cannot detect a face region from the converted image 900, it is presumed that the detection accuracy of the region 510 detected from the original image 400 or the positions of the face organs 610 to 650 is low. Therefore, the generation unit 302 generates the converted image 900 by geometrically transforming the region 510 detected from the original image 400 based on the positions of the face organs. Then, the determination unit 304 can determine whether each of the multiple face candidate regions is a face region or not based on whether a face region is detected from the converted image 900. This makes it possible to eliminate face candidate regions that do not include a face from the image.

[0071] As described above, according to the first embodiment, whether or not a face candidate region is a face region is determined based on whether or not a face region can be detected from a transformed image transformed based on the positions of face parts in the face candidate region of an image. This improves the accuracy of determining the face region from the face candidate region, thereby improving the accuracy of extracting an image (partial image) in which a face appears from the image.

[0072] Second embodiment In the first embodiment, if a face area is not detected from the converted image, it is determined that an image (partial image) including a face candidate area is not to be output. However, the quality required for a partial image may differ depending on the use of the image. In this way, the determination conditions for determining whether or not a face candidate area is a face area (hereinafter, the determination conditions for a face candidate area) may be changed so that the partial image has a quality according to the use of the image. In the second embodiment, the determination conditions for a face candidate area are changed depending on the use of the image. Below, the difference between the second embodiment and the first embodiment will be described.

[0073] For example, when creating learning data for training a DNN for face detection, face organ detection, or face authentication, and when creating a registration image to be registered in advance in a face authentication system, the image processing device sets strict judgment conditions for the face candidate region. On the other hand, when detecting a face from an image captured by an imaging device during operation of the face authentication system, the image processing device sets loose judgment conditions for the face candidate region. Hereinafter, an example will be described in which the image processing device judges whether or not a face candidate region is a face region based on judgment conditions for the face candidate region according to the purpose of the image.

[0074] When creating learning data for training a DNN and when creating registration images to be registered in advance in a face recognition system, training a DNN with low-quality images (large blur, etc.) can cause a decrease in the performance of the DNN and the face recognition system. However, images for training a DNN or for a face recognition system can be prepared in advance, spending a lot of time. Therefore, the image processing device may set stricter judgment conditions for face candidate regions.

[0075] The face authentication system has two types of authentication methods. The authentication methods are active authentication and non-active authentication. Active authentication is a method in which the user is authenticated while positioned in front of the imaging device of the user's own volition. In this case, the image processing device can generate appropriate image data by setting strict judgment conditions for the face candidate area.

[0076] Non-active authentication refers to a method in which a face authentication system autonomously authenticates a user when the user does not intend to be authenticated. Therefore, when the imaging device captures an image of the user, the user may move outside the imaging range of the imaging device. In addition, the orientation of the user's face relative to the imaging device (camera) varies. Thus, the number and quality of images captured during non-active authentication are largely dependent on the imaging environment and the settings of the imaging device. Therefore, during non-active authentication, it is difficult to obtain a sufficient number of images that capture the user's face and high-quality images. In this case, the image processing device can secure an appropriate amount of image data by loosely setting the judgment conditions for the face candidate area.

[0077] FIG. 11 is a block diagram illustrating an example of a functional configuration of an image processing device according to the second embodiment.

[0078] The purpose input unit 1101 receives the purpose of the image data from the user and transmits the purpose of the image data to the control unit 1102. The purposes of the image data are "learning", "registration", and "authentication". Learning refers to having an image learning model (DNN) learn the features of the image. Registration refers to registering the features and name of a person appearing in an image in a list. Authentication refers to identifying who the person appearing in the image is and outputting the name of the corresponding person.

[0079] The control unit 1102 receives the purpose of the image data from the purpose input unit 1101. The control unit 1102 controls the contents of processing to be performed by the face area detection unit 300, face organ detection unit 301, generation unit 302, face detection unit 303, and determination unit 304 according to the purpose of the image data. For example, when the purpose of the image data is "learning" or "registration", the control unit 1102 sets stricter determination conditions for the face candidate area. On the other hand, when the purpose of the learning data is "authentication", the control unit 1102 sets lenient determination conditions for the face candidate area.

[0080] 12 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are similar to those in the first embodiment, and therefore the description will be omitted.

[0081] In S1201, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that the judgment conditions for the face candidate region are to be set stricter (Yes in S1201), the process proceeds to S1202. If the control unit 1102 determines that the judgment conditions for the face candidate region are not to be set stricter (No in S1201), the process proceeds to S1002.

[0082] In S1202, the determination unit 304 determines whether or not to detect the positions of face parts from the face candidate region, based on whether or not the ratio of the part of the face candidate region that protrudes from the image to the face candidate region is equal to or less than a threshold. For example, the determination unit 304 calculates the ratio of the part of the region 550 that protrudes from the image 400 in Fig. 5 to the region 550. If the calculated ratio is not equal to or less than the threshold, the determination unit 304 determines not to detect the positions of face parts from the region 550, and ends the process.

[0083] 13 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are similar to those in the first embodiment, and therefore the description will be omitted.

[0084] In S1301, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that the judgment conditions for the face candidate region are to be set stricter (Yes in S1301), the process proceeds to S1302. If the control unit 1102 determines that the judgment conditions for the face candidate region are not to be set stricter (No in S1301), the process proceeds to S1005.

[0085] In S1302, the determination unit 304 determines whether or not to detect a face area from the converted image based on whether or not the difference between the positions of the face organs in the face candidate area and the reference positions in the converted image is within a threshold. If the determination unit 304 determines that the difference between the positions of the face organs in the face candidate area and the reference positions in the converted image is within the threshold (Yes in S1302), the process proceeds to S1005. If the determination unit 304 determines that the difference between the positions of the face organs in the face candidate area and the reference positions in the converted image is not within the threshold (No in S1302), the process ends.

[0086] 14 is a flowchart for explaining the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are the same as those in the first embodiment, and therefore the explanation will be omitted. At the start of the process, a variable i=0 is set. The number of the variable i indicates the number of times a human face area has been detected from the converted image.

[0087] In S1401, the control unit 1102 adds 1 to the variable i, and the process proceeds to S1402.

[0088] In S1402, the determination unit 304 determines whether the variable i is equal to or greater than the threshold. If the control unit 1102 determines that the variable i is equal to or greater than the threshold (Yes in S1401), the process proceeds to S1007. If the determination unit 304 determines that the variable i is not equal to or greater than the threshold (No in S1401), the process proceeds to S1403.

[0089] In S1403, the control unit 1102 detects one face candidate region from the converted image generated in S1004, and the process proceeds to S1002. Here, if the control unit 1102 sets stricter judgment conditions for face candidate regions, the control unit 1102 sets the threshold value in S1402 to a value equal to or greater than "2." The judgment conditions for face candidate regions are set stricter as the threshold value increases. On the other hand, if the control unit 1102 sets looser judgment conditions for face candidate regions, the control unit 1102 sets the threshold value in S1402 to "1."

[0090] 15 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes in S1000 and S1003 to S1007 are similar to those in the first embodiment, and therefore the description thereof will be omitted.

[0091] In S1501, the face area detection unit 300 calculates the likelihood of a face candidate area detected from an image.

[0092] In S1502, the face part detection unit 301 calculates the likelihood of the face part positions detected from the face candidate area.

[0093] In S1503, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that the judgment conditions for the face candidate region are to be set stricter (Yes in S1503), the process proceeds to S1004. If the control unit 1102 determines that the judgment conditions for the face candidate region are not to be set stricter (No in S1503), the process proceeds to S1504.

[0094] 16 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are similar to those in the first embodiment, and therefore the description will be omitted.

[0095] In S1601, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that stricter judgment conditions for the face candidate region are to be set (Yes in S1601), the process proceeds to S1002. If the control unit 1102 determines that stricter judgment conditions for the face candidate region are not to be set (No in S1601), the process proceeds to S1007.

[0096] 17 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are the same as those in the first embodiment, so the description will be omitted.

[0097] In S1701, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that the judgment conditions for the face candidate region are to be set stricter (Yes in S1701), the process proceeds to S1702. If the control unit 1102 determines that the judgment conditions for the face candidate region are not to be set stricter (No in S1701), the process proceeds to S1007.

[0098] In S1702, the determination unit 304 determines whether or not the size of the face area of ​​the converted image detected in S1006 is equal to or larger than the threshold. If the determination unit 304 determines that the size of the face area of ​​the converted image detected in S1006 is equal to or larger than the threshold (Yes in S1702), the process proceeds to S1007. If the determination unit 304 determines that the size of the face area of ​​the converted image detected in S1006 is not equal to or larger than the threshold (No in S1702), the process ends.

[0099] 18 is a flowchart illustrating the flow of image processing according to the second embodiment. The processes from S1000 to S1007 are similar to those in the first embodiment, and therefore the description will be omitted.

[0100] In S1801, the control unit 1102 determines whether or not to set stricter judgment conditions for the face candidate region based on the purpose of the image data received from the purpose input unit 1101. If the control unit 1102 determines that the judgment conditions for the face candidate region are to be set stricter (Yes in S1801), the process proceeds to S1802. If the control unit 1102 determines that the judgment conditions for the face candidate region are not to be set stricter (No in S1801), the process proceeds to S1007.

[0101] In S1802, the face organ detection unit 301 detects the positions of face organs from the face area of ​​the converted image detected in S1006, and the process proceeds to S1803.

[0102] In S1803, the determination unit 304 determines whether the difference between the position of the face organ in the face region of the converted image and the reference position in the converted image is within a threshold value. If the difference between the position of the face organ in the face image region of the converted image and the reference position in the converted image is within the threshold value (Yes in S1803), the process proceeds to S1007. If the difference between the position of the face organ in the face image region of the converted image and the reference position is not within the threshold value (No in S1803), the process ends.

[0103] As described above, according to the second embodiment, by changing the determination conditions for face candidate regions based on the purpose of the image, it is possible to generate partial images of quality suited to the purpose of the image.

[0104] Third embodiment The third embodiment performs a geometric transformation (e.g., affine transformation) on an image based on the positions of face organs in a face candidate region of the image. The third embodiment determines whether or not to output information indicating the face candidate region as information indicating a face region based on the result of detecting a face region from a transformed image. The third embodiment also performs registration, authentication, or learning processing of a partial image depending on the purpose of the partial image. Below, the differences between the third embodiment and the first and second embodiments will be explained.

[0105] FIG. 19 is a block diagram illustrating an example of a functional configuration of an image processing device according to the third embodiment.

[0106] The image processing device 110 includes a feature extraction unit 1901 , a feature matching unit 1902 , a registration unit 1903 , an authentication unit 1904 , a learning unit 1905 , and a name input unit 1906 .

[0107] (If the image is to be used for registration) The functions of the above-mentioned respective units when the input from the purpose input unit 1101 is for registration will be described below.

[0108] When the determining unit 304 determines that a face region has been detected from the converted image, the feature extracting unit 1901 receives the partial image from the generating unit 302. Next, the feature extracting unit 1901 extracts features from the partial image. For example, the feature extracting unit 1901 extracts a numerical vector from the partial image as a feature of the partial image. Note that the feature of the partial image is not limited to a numerical vector, and may be another feature. Then, the feature extracting unit 1901 transmits the feature extracted from the partial image to the registering unit 1903.

[0109] The registration unit 1903 registers the features of the partial image received from the feature extraction unit 1901 in a list in the storage unit 207, etc. In addition, the registration unit 1903 receives a name corresponding to the feature registered in the list in the storage unit 207, etc. from the name input unit 1906, and registers the name in the list in the storage unit 207, etc.

[0110] The name input unit 1906 includes a user interface (UI) for inputting a name corresponding to the characteristics of the partial image registered by the registration unit 1903. The user uses the name input unit 1906 to input the name of a person appearing in the partial image input to the image processing device 110.

[0111] (If the image is for authentication) The functions of the above-mentioned respective units when the input from the purpose input unit 1101 is authentication will be described below.

[0112] When the determining unit 304 determines that a face region has been detected from the converted image, the feature extracting unit 1901 receives the partial image from the generating unit 302. Next, the feature extracting unit 1901 extracts features from the partial image. For example, the feature extracting unit 1901 extracts a numerical vector from the partial image as a feature of the partial image. Note that the feature of the partial image is not limited to a numerical vector, and may be a feature having the same format as a feature already registered in the registering unit 1903.

[0113] The feature matching unit 1902 matches the feature of the partial image received from the feature extraction unit 1901 with the feature of the partial image already registered in the registration unit 1903. The feature matching unit 1902 transmits the matching result to the authentication unit 1904. Here, the feature matching unit 1902 calculates the difference between the feature extracted from the partial image and the feature already registered in the registration unit 1903. For example, the difference is the cosine similarity, the L1 distance, the L2 distance, etc. of two numerical vectors.

[0114] The authentication unit 1904 receives the feature differences of the partial images as the matching result from the feature matching unit 1902. Next, if the smallest difference among the feature differences of the partial images is within a threshold, the authentication unit 1904 outputs the name of the person appearing in the partial image corresponding to the smallest difference as the authentication result. On the other hand, if the smallest difference among the feature differences of the received partial images is not within the threshold, the authentication unit 1904 outputs an authentication result indicating "no matching person".

[0115] (If the image is for educational purposes) The functions of the above-mentioned respective units when the input from the purpose input unit 1101 is learning will be described below.

[0116] The name input unit 1906 includes a user interface (UI) for inputting a name corresponding to the feature of the partial image registered by the registration unit 1903. The user uses the name input unit 1906 to input the name of a person appearing in the image input to the image processing device 110.

[0117] If the determination unit 304 determines that a face area has been detected from the converted image, it transmits the partial image to the learning unit 1905. Furthermore, the determination unit 304 receives the name of the person appearing in the partial image from the name input unit 1906, and transmits the partial image and the name of the person appearing in the partial image to the learning unit 1905.

[0118] The learning unit 1905 learns the partial image and the name of the person appearing in the partial image received from the determination unit 304. For example, the learning unit 1905 is a DNN that extracts features from the partial image, but is not limited to this, and may be, for example, a DNN for detecting a face from a partial image.

[0119] As described above, according to the third embodiment, the process of registering, authenticating, or learning a partial image can be controlled depending on the purpose of the partial image.

[0120] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0121] The disclosure of this specification includes the following image information processing, image processing method, and program. (Item 1) a detection means for detecting a candidate face region from an image using a first face detector; an acquisition means for acquiring positions of face organs from an image of a first region detected as the face candidate region using a face organ detector; a generating means for generating a transformed image by transforming the image of the first region based on the positions of the facial features; a control means for controlling whether or not to output an image of the first region based on a detection result obtained by detecting a face from the converted image using a second face detector; An image processing device comprising: (Item 2) The first face detector is configured with a neural network different from that of the second face detector. 2. The image processing device according to item 1, (Item 3) the first face detector is a neural network trained using images showing human faces; the face organ detector is a neural network trained using images of the first region; the second face detector is a neural network trained using the transformed images. 3. The image processing device according to item 1 or 2. (Item 4) the first face detector is a neural network having a smaller size and a lower correct detection rate than the second face detector; The second face detector is a neural network having a larger scale and a higher correct detection rate than the first face detector. 4. The image processing device according to any one of items 1 to 3, (Item 5) The first face detector is configured with the same neural network as the second face detector. 2. The image processing device according to item 1, (Item 6) When the face is detected from the converted image, the control means outputs the image of the first region as a face image. 6. The image processing device according to any one of items 1 to 5, (Item 7) the control means, when the face is not detected from the converted image, suppresses output of the image of the first region as a face image. 7. The image processing device according to item 6, (Item 8) the generating means generates a converted image by converting the image of the first region based on a difference between a position of the face organ detected from the image of the first region and a reference position that defines the position of the face organ in the converted image. 8. The image processing device according to any one of items 1 to 7, (Item 9) An input means for inputting a use of the image; a change means for changing, based on a use of the image, at least one of a condition for detecting the position of the face organs from the image of the first region, a condition for detecting the face from the transformed image, a condition for generating the transformed image transformed based on the position of the face organs detected from the image of the first region, and a condition for outputting the image of the first region; 9. The image processing device according to any one of items 1 to 8, comprising: (Item 10) when the condition for detecting the position of the face organ from the image of the first region is changed by the change means, the control means determines whether or not to detect the position of the face organ from the image of the first region based on whether or not a proportion of the part of the first region that protrudes outside the image to the first region is equal to or smaller than a threshold value; 10. The image processing device according to item 9, (Item 11) the control means, when the condition for detecting the face from the converted image is changed by the change means, determines whether or not to detect the face from the converted image based on whether or not a difference between a position of the face organ detected from the image of the first region and a reference position defining the position of the face organ in the converted image is equal to or smaller than a threshold value; 11. The image processing device according to item 9 or 10, (Item 12) the control means, when a condition for outputting the information indicating the first region as the information indicating the face region is changed by the change means, determines whether or not to output an image of the first region based on whether or not the number of times the face is detected from the converted image is equal to or greater than a threshold value. 12. The image processing device according to any one of items 9 to 11, (Item 13) When the condition for generating the transformed image converted based on the position of the face organ detected in the first area is not changed by the change means, the control means determines whether or not to generate the transformed image converted based on the position of the face organ detected from the image of the first area based on whether the likelihood of the first area is equal to or less than a threshold and whether the likelihood of the position of the face organ detected in the first area is equal to or less than a threshold. 13. The image processing device according to any one of items 9 to 12, (Item 14) the control means determines to detect the position of the face organ from the image of the first region when a condition for detecting the position of the face organ from the image of the first region is changed by the change means; 14. The image processing device according to any one of items 9 to 13, (Item 15) the control means, when a condition for outputting the information indicating the first region as information indicating a face region is changed by the change means, determines whether or not to output an image of the first region based on whether or not a size of the face detected from the converted image is equal to or larger than a threshold value. 15. The image processing device according to any one of items 9 to 14, (Item 16) when the condition for outputting the information indicating the first region as information indicating a face region is changed by the change means, the control means determines whether or not to output an image of the first region based on whether or not a difference between a position of a face organ in the first region detected as a candidate face region from the converted image and a reference position defining the position of the face organ in the converted image is equal to or smaller than a threshold value; 16. The image processing device according to any one of items 9 to 15, (Item 17) an extraction means for extracting features from the converted image when the face is detected from the converted image; A registration means for registering the features of the converted image and the name of a person corresponding to the features of the converted image; an output means for outputting a name of a person in the converted image based on a comparison result between the feature of the converted image extracted by the extraction means and the feature of the converted image registered in the registration means; 2. The image processing device according to item 1, comprising: (Item 18) The use of the converted image includes at least one of: learning the converted image to a model that learns images; registering the features of the converted image and the name of a person corresponding to the features of the converted image; and authentication that outputs the name of a person in the converted image. 10. The image processing device according to item 9, (Item 19) A registration means for registering the features of the converted image and the name of a person corresponding to the features of the converted image; an output means for outputting a name of a person in the converted image based on a comparison result between the feature of the converted image extracted from the converted image and the feature of the converted image registered in the registration means, The registration means registers the features of the converted image extracted from the converted image when the converted image is used for the registration. 20. The image processing device according to item 18, characterized in that: (Item 20) the output means, when the converted image is used for authentication and the smallest difference between the features of the converted image extracted from the converted image and the features of the converted image registered in the registration means is within a threshold value, outputs the name of a person corresponding to the smallest difference. 20. The image processing device according to item 19, characterized in that (Item 21) When the converted image is used for learning, a learning means is provided for learning the converted image. 20. The image processing device according to item 18 or 19, (Item 22) the learning means learns the second face detector based on an image of the first region; 22. The image processing device according to item 21, (Item 23) The learning means further a first detection means for detecting a candidate region of a face from the image; a second detection means for detecting the positions of face features from the face candidate region; a third detection means for detecting the face from the converted image; 22. The image processing device according to item 21, comprising: (Item 24) An image processing method executed by an image processing device, comprising: a detection step of detecting candidate face regions from the image using a first face detector; an acquiring step of acquiring positions of face organs from an image of a first region detected as the face candidate region using a face organ detector; a generating step of generating a transformed image by transforming the image of the first region based on the positions of the facial features; a control step of controlling whether or not to output an image of the first region based on a detection result of detecting a face from the converted image using a second face detector; An image processing method comprising: (Item 25) A program for causing a computer to function as each of the means of the image processing device according to any one of items 1 to 23.

[0122] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0123] 10: image processing system, 100: imaging device, 110: image processing device, 120: network, 101: input section, 102: display section, 103: I / F, 104: CPU, 105: RAM, 106: ROM, 107: storage section, 108: data bus

Claims

1. A detection means for detecting a candidate region of a face from an image using a face detector; an acquisition means for acquiring positions of face organs from the image of the face candidate area detected by the detection means using a face organ detector; a generating means for generating a transformed image by transforming the image of the face candidate region based on the positions of the face features; a control means for controlling whether or not to output an image of the face candidate area as a face image based on a result of performing face detection processing on the converted image, a first face detector for detecting a face candidate region is configured with a neural network different from a second face detector used for performing face detection processing on the converted image; the first face detector is a neural network having a smaller size and a lower correct detection rate than the second face detector; The second face detector is a neural network having a larger scale and a higher correct detection rate than the first face detector.

13. An image processing device comprising:

2. A detection means for detecting a candidate region of a face from an image using a face detector; an acquisition means for acquiring positions of face organs from the image of the face candidate area detected by the detection means using a face organ detector; a generating means for generating a transformed image by transforming the image of the face candidate region based on the positions of the face features; a control means for controlling whether or not to output an image of the face candidate area as a face image based on a result of performing face detection processing on the converted image, An input means for inputting a use of the image; and a change means for changing, based on a use of the image, at least one of a condition for detecting the positions of the face organs from the image of the face candidate region, a condition for detecting the face from the transformed image, a condition for generating the transformed image transformed based on the positions of the face organs detected from the image of the face candidate region, and a condition for outputting the image of the face candidate region.

13. An image processing device comprising:

3. a first face detector for detecting a face candidate region is configured with a neural network different from a second face detector used for performing face detection processing on the converted image; 3. The image processing device according to claim 2.

4. the first face detector is a neural network trained using images showing human faces; the face organ detector is a neural network trained using images of the candidate face regions; the second face detector is a neural network trained using the transformed images.

4. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing apparatus.

5. the first face detector is a neural network having a smaller size and a lower correct detection rate than the second face detector; The second face detector is a neural network having a larger scale and a higher correct detection rate than the first face detector.

4. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing apparatus.

6. a first face detector for detecting a face candidate region, the first face detector being configured with the same neural network as a second face detector used for performing face detection processing on the converted image; 3. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing device.

7. When a face is detected from the converted image, the control means outputs an image of the candidate region of the face as a face image.

3. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing device.

8. the control means, when the face is not detected from the converted image, suppresses output of the image of the face candidate area as a face image.

8. The image processing device according to claim 7,

9. the generating means generates a converted image by converting the image of the face candidate region based on a difference between a position of the face organ detected from the image of the face candidate region and a reference position that defines the position of the face organ in the converted image.

3. The image processing device according to claim 1, wherein the first and second inputs are input to the image processing device.

10. An input means for inputting a use of the image; a change means for changing, based on a use of the image, at least one of a condition for detecting the positions of the facial organs from the image of the candidate face region, a condition for detecting the face from the transformed image, a condition for generating the transformed image transformed based on the positions of the facial organs detected from the image of the candidate face region, and a condition for outputting the image of the candidate face region; The image processing device according to claim 1 , further comprising:

11. the control means, when the condition for detecting the position of the face organ from the image of the face candidate area is changed by the change means, determines whether or not to detect the position of the face organ from the image of the face candidate area based on whether or not a proportion of the part of the face candidate area protruding outside the image to the whole of the face candidate area is equal to or smaller than a threshold value; 11. The image processing device according to claim 2 or 10.

12. the control means, when the condition for detecting the face from the converted image is changed by the change means, determines whether or not to detect the face from the converted image based on whether or not a difference between a position of the face organ detected from the image of the candidate region of the face and a reference position defining the position of the face organ in the converted image is equal to or smaller than a threshold value; 11. The image processing device according to claim 2 or 10.

13. the control means, when a condition for outputting the information indicating the candidate face region as the information indicating the face region is changed by the change means, determines whether or not to output the image of the candidate face region as a face image based on whether or not the number of times the face is detected from the converted image is equal to or greater than a threshold value. The image processing device according to claim 12 .

14. When the condition for generating the converted image converted based on the positions of the face organs detected from the face candidate area is not changed by the change means, the control means determines whether or not to generate the converted image converted based on the positions of the face organs detected from the image of the face candidate area, based on whether the likelihood of the face candidate area is equal to or less than a threshold and whether the likelihood of the positions of the face organs detected in the face candidate area is equal to or less than a threshold.

11. The image processing device according to claim 2 or 10.

15. the control means determines to detect the positions of the face organs from the image of the face candidate region when a condition for detecting the positions of the face organs from the image of the face candidate region is changed by the change means; 11. The image processing device according to claim 2 or 10.

16. the control means, when a condition for outputting the information indicating the candidate face region as information indicating a face region is changed by the change means, determines whether or not to output the image of the candidate face region as a face image based on whether or not a size of the face detected from the converted image is equal to or larger than a threshold.

11. The image processing device according to claim 2 or 10.

17. the control means, when a condition for outputting the information indicating the face candidate region as information indicating a face region is changed by the change means, determines whether or not to output the image of the face candidate region as a face image based on whether or not a difference between a position of a face organ in the face candidate region and a reference position defining the position of the face organ in the converted image is equal to or smaller than a threshold value; 11. The image processing device according to claim 2 or 10.

18. an extraction means for extracting features from the converted image when the face is detected from the converted image; A registration means for registering the features of the converted image and the name of a person corresponding to the features of the converted image; an output means for outputting a name of a person in the converted image based on a comparison result between the feature of the converted image extracted by the extraction means and the feature of the converted image registered in the registration means; 3. The image processing device according to claim 1, further comprising:

19. The use of the converted image includes at least one of: learning the converted image to a model that learns images; registering the features of the converted image and the name of a person corresponding to the features of the converted image; and authentication outputting the name of a person in the converted image.

11. The image processing device according to claim 2 or 10.

20. A registration means for registering the features of the converted image and the name of a person corresponding to the features of the converted image; an output means for outputting a name of a person in the converted image based on a comparison result between the feature of the converted image extracted from the converted image and the feature of the converted image registered in the registration means, The registration means registers the features of the converted image extracted from the converted image when the converted image is used for the registration.

20. The image processing device according to claim 19,

21. the output means, when the converted image is used for authentication and the smallest difference between the features of the converted image extracted from the converted image and the features of the converted image registered in the registration means is within a threshold value, outputs the name of a person corresponding to the smallest difference.

21. The image processing device according to claim 20 .

22. When the converted image is used for learning, a learning means is provided for learning the converted image.

20. The image processing device according to claim 19,

23. the learning means learns a second face detector used to perform face detection processing on the converted image based on the image of the face candidate region; 23. The image processing device according to claim 22.

24. The learning means further a first detection means for detecting a candidate region of a face from the image; a second detection means for detecting the positions of face features from the face candidate region; a third detection means for detecting the face from the converted image; The image processing device according to claim 22, further comprising:

25. An image processing method executed by an image processing device, comprising: a detection step of detecting candidate face regions from the image using a face detector; an acquisition step of acquiring positions of face organs from the image of the face candidate region detected by the detection step using a face organ detector; a generating step of generating a transformed image by transforming the image of the face candidate region based on the positions of the face features; a control step of controlling whether or not to output an image of the face candidate area as a face image based on a result of performing face detection processing on the converted image, a first face detector for detecting a face candidate region is configured with a neural network different from a second face detector used for performing face detection processing on the converted image; the first face detector is a neural network having a smaller size and a lower correct detection rate than the second face detector; The second face detector is a neural network having a larger scale and a higher correct detection rate than the first face detector.

13. An image processing method comprising:

26. An image processing method executed by an image processing device, comprising: a detection step of detecting candidate face regions from the image using a face detector; an acquisition step of acquiring positions of face organs from the image of the face candidate region detected by the detection step using a face organ detector; a generating step of generating a transformed image by transforming the image of the face candidate region based on the positions of the face features; a control step of controlling whether or not to output an image of the face candidate area as a face image based on a result of performing face detection processing on the converted image, an input step of inputting a use of the image; and a modification step of modifying, based on a use of the image, at least one of a condition for detecting the positions of the face organs from the image of the face candidate region, a condition for detecting the face from the transformed image, a condition for generating the transformed image transformed based on the positions of the face organs detected from the image of the face candidate region, and a condition for outputting the image of the face candidate region.

13. An image processing method comprising:

27. A program for causing a computer to function as each of the means of the image processing apparatus according to claim 1 or 2.

Images