Face discrimination device and face discrimination method

Abstract

To provide a technique that does not cause interface mismatch between a face detection function and a face recognition function when face recognition is performed, and appropriately defines a range for cutting out a face from an image, thereby preventing a decrease in accuracy of face recognition.SOLUTION: A face determination device includes a first processing unit A having a first face detection function A1 that identifies the face position of a person from an image captured by a camera, and a face image cutting function that cuts out the face image identified by the first face detection function A1 to a predetermined size, and a second processing unit B having a second face detection function B1 that identifies the face position from the face image of the person transmitted from the first processing unit A, and a face recognition function B2 that identifies who the person is from the face image identified by the second face detection function B1, and the face image cutting function of the first processing unit A cuts out a face image in a range larger than the actual face identified by the first face detection function A1.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a face discrimination device and a face discrimination method capable of improving a face detection function for detecting a face and a face recognition function for recognizing a feature portion of the face.

Background Art

[0002] In public spaces such as airports, railway stations, and shopping streets, a security system having a face detection function and a face recognition function is being used for crime prevention. Also, in offices, factories, etc., an entrance / exit management system having a similar function is being introduced for security enhancement.

[0003] As such a face discrimination device having a face detection function and a face recognition function, for example, the technique disclosed in Patent Document 1 is known. In the imaging device disclosed in Patent Document 1, a function of detecting and cutting out a region in which a face appears from image data in which various objects are reflected and transmitting it to a server such as a cloud, and information obtained by the server performing face recognition from this face image data are collated with data of a specific person on a database, and when they match, a function of notifying that the specific person is present or has passed is provided. Further, this imaging device has a determination step for determining whether face recognition is possible as a series of processes necessary for the above-described functions.

[0004] Specifically, in the determination step, face detection is performed using an image taken by subsampling reading, and when the detected face image does not have enough pixels required for face recognition, shooting is performed by full pixel reading, the face image is cut out, and transmission is performed to a device capable of face recognition. Also, in the step of cutting out and transmitting the face image, after cutting out an extra region from the region where the face is detected and compressing the image, a face image with a required amount of information is transmitted.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, in the technique disclosed in Patent Document 1, when performing face recognition by transmitting an image of a face portion detected from an image, there is a problem that an interface mismatch occurs between the face detection function and the face recognition function. In addition, in the technique disclosed in Patent Document 1, when detecting and cutting out a face from an image, since these ranges are not determined to be constant, there is also a problem that the accuracy of face recognition in subsequent processes is reduced.

[0007] The present invention has been made in view of the above circumstances, and provides a face discrimination device and a face discrimination method that do not cause an interface mismatch between a face detection function and a face recognition function when performing face recognition, and appropriately determine a range for cutting out a face from an image, thereby preventing a reduction in the accuracy of face recognition. MEANS FOR SOLVING THE PROBLEMS

[0008] In order to solve the above problems, the present invention proposes the following means. In the face discrimination device according to the first aspect of the present invention, a first processing unit having a first face detection function for specifying a face position of a person from a captured image of a camera and a face image cutting function for cutting out a face image specified by the first face detection function to a predetermined size, and a second processing unit having a second face detection function for specifying a face position from a face image of a person transmitted from the face image cutting function of the first processing unit and a face recognition function for specifying who the person is from the face image specified by the second face detection function, and the face image cutting function of the first processing unit cuts out a face image in a range larger than the actual face specified by the first face detection function.

[0009] In the face discrimination method according to the second aspect of the present invention, a first processing stage having a first face detection step of specifying the face position of a person from a captured image of a camera and a face image cutting step of cutting out the face image specified in the first face detection step to a predetermined size; A second processing stage having a second face detection step of specifying a face position from the face image of the person transmitted from the face image cutting step and a face recognition step of specifying who the person is from the face image specified in the second face detection step; and in the face image cutting step, a face image in a range larger than the actual face specified in the first face detection step is cut out.

Effects of the Invention

[0010] According to the present invention, when performing face recognition, it is possible to improve the accuracy of face recognition by preventing an interface mismatch between the face detection function and the face recognition function and appropriately determining the range for cutting out the face from the image.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0012] The minimum configuration of the face discrimination device 100 according to the present invention will be described with reference to FIG. 1. This face discrimination device 100 mainly includes a first processing unit A and a second processing unit B. For example, the first processing unit A is provided in a sub-computer on the camera side, and the second processing unit B is provided in a main computer connected via a server.

[0013] The first processing unit A has a first face detection function A1 that identifies the face position of a person from the captured image of the camera C, and a face image cutting function A2 that cuts out the face image identified by the first face detection function A1 to a predetermined size. Note that in the face image cutting function A2 of the first processing unit A, a face image in a range larger than the actual face identified by the first face detection function A1 is cut out.

[0014] The second processing unit B has a second face detection function B1 that identifies the face position from the face image of the person transmitted from the face image cutting function A2 of the first processing unit A, and a face recognition function B2 that identifies who the person is from the face image identified by the second face detection function B1.

[0015] And in the face discrimination device 100 configured as described above, after the first face detection function A1 of the first processing unit A identifies the face position of a person from the captured image of the camera C, the face image cutting function A2 of the first processing unit A cuts out the face image identified by the first face detection function A1 to a predetermined size. At this time, in the face image cutting function A2 of the first processing unit A, a face image in a range larger than the actual face identified by the first face detection function A1 is cut out. After that, in the second face detection function B1 of the second processing unit B, after identifying the face position from the face image of the person transmitted from the face image cutting function A2 of the first processing unit A, the face recognition function B2 identifies who the person is from the face image identified by the second face detection function B1.

[0016] In the face discrimination device 100 configured as described above, in the face image cutting function A2 of the first processing unit A, a face image in a range larger than the actual face identified by the first face detection function A1 is cut out and transmitted to the second face detection function B1, whereby the communication data volume can be reduced compared to transmitting all the input images.

[0017] Also, in the second face detection function B1 of the second processing unit B, based on the face image in a range larger than the actual face cut out by the first processing unit A, highly accurate face detection can be performed using the face contour and image information around the face.

[0018] Further, in the second processing unit B, by providing the second face detection function B1 and the face recognition function B2, if the interfaces of these second face detection function B1 and face recognition function B2 are compatible, there is no need to match the interface between the first face detection function A1 of the first processing unit A and the face recognition function B2 of the second processing unit B. That is, in the face discrimination device 100, by means of a simple configuration of adding the second face detection function B1 in the second processing unit B, the problem of interface incompatibility between the first face detection function A1 on the camera C side and the face recognition function B2 on the server side can be solved.

[0019] Note that in the face discrimination device 100, a face discrimination method including a first processing stage and a second processing stage is used. In the first processing stage, there are a first face detection step of specifying the face position of a person from the captured image of the digital camera CA, and a face image cutting step of cutting out the face image specified in the first face detection step to a predetermined size. Also, in the second processing stage, there are a second face detection step of specifying the face position from the face image of the person transmitted from the face image cutting step, and a face recognition step of specifying who the person is from the face image specified in the second face detection step. Also, in the face image cutting step of the first processing stage, a face image in a range larger than the actual face specified in the first face detection step is cut out.

[0020] (Embodiment) An embodiment of the present invention will be described with reference to FIGS. 2 to 4. FIG. 2 is a functional block diagram of a face discrimination device 101 according to the present embodiment, and FIG. 3 is a configuration diagram embodying FIG. 2. This face discrimination device 101 is provided in the software of a small power-saving computer 50 directly connected to the digital camera CA and in the software of a large high-performance computer 52 connected via a server 51, respectively. Also, the small power-saving computer 50 and the large high-performance computer 52 connected via the server 51 are in remote locations with a considerable distance between them and are connected by a network N. Here, the small power-saving computer 50 and the large high-performance computer 52 are defined as follows: for example, the one with relatively low power consumption is defined as the "small power-saving computer 50", and the one with high power consumption is defined as the "large high-performance computer 52". However, it may also be a system configuration in which the small power-saving computer 50 and the large high-performance computer 52 are defined according to the relative portability and processing capabilities.

[0021] The small power-saving computer 50 has a first processing unit 10 implemented by software. As shown in FIGS. 2 and 3, the first processing unit 10 has a first face detection function 11 for identifying the face position (shown as image P2) of a person from a captured image (shown as image P1) captured by the digital camera CA, and a face image cutting function 12 for cutting out the face image identified by the first face detection function 11 to a predetermined size. Note that in the face image cutting function 12 of the first processing unit 10, a face image in a range larger than the actual face identified by the first face detection function 11 (shown as image P3) is cut out.

[0022] In addition, a second processing unit 20 implemented by software is provided in a cloud (indicated by reference numeral CL) formed by the large high-performance computer 52. The second processing unit 20 has a second face detection function 21 for identifying the face position (shown as image P4) from the face image of a person transmitted from the face image cutting function 12 of the first processing unit 10, and a face recognition function 22 for identifying who the person is from the face image identified by the second face detection function 21 (shown as image P5).

[0023] Next, the operation of the face discrimination device 101 shown in FIGS. 2 and 3 will be described. First, the first face detection function 11 operating on the small power-saving computer 50 processes the image input from the digital camera CA to detect a face, and outputs the detection result as coordinates within the input image. For example, it outputs the coordinates (Xleft, Ytop) of the upper left vertex of a rectangle including the face and the coordinates (Xright, Ybottom) of the lower right vertex.

[0024] Here, it is desirable that the first face detection function 11 and the face image extraction function 12 be implemented as software with a low processing load that can operate on the small power-saving computer 50. Also, in the face image extraction function 12, the face detection result may differ depending on the processing status of the software. Examples thereof are indicated by reference numerals P2A, P2B, and P2C in FIGS. 4(A) to 4(C) below.

[0025] FIG. 4(A) is an example in which the face detection result (indicated by reference numeral P2A) is a rectangular portion having the upper end of the top of the head as the upper limit, the lower end of the chin as the lower limit, and the left and right most points of the face contour as the ends. FIG. 4(B) is an example in which the face detection result (indicated by reference numeral P2B) is a rectangular portion having the upper end of the eyebrows as the upper limit, the lower end of the lips as the lower limit, and the outer corners of the left and right eyes as the ends. FIG. 4(C) is an example in which the upper limit of the face detection result (indicated by reference numeral P2C) is above the top of the head, the lower limit is below the lower end of the chin, and the left and right ends are rectangular portions outside the left and right contours of the face.

[0026] However, in the first processing unit 10 of the present embodiment, in order to prevent the operations of the second face detection function 21 and the face recognition function 22 located at the subsequent stage from being hindered due to differences in the detection results in the first face detection function 11, the following processing is performed by the face image extraction function 12. That is, in the face image extraction function 12, an image (P2) in a range larger than the contour of the face detected by the first face detection function 11 is extracted as a partial image, and the upper limit / lower limit / left and right ends of the partial image are set to be located outside the top of the head / the lower end of the chin / the left and right contours of the face.

[0027] Specifically, in the face image extraction function 12, a rectangular portion enlarged outward by a predetermined offset amount from the coordinates of the upper limit / lower limit / left and right ends of the face indicated by the partial image is extracted as a new partial image. As another processing example of the face image extraction function 12, a rectangular portion enlarged by a certain similarity ratio from the center of the rectangular portion is extracted as a partial image. Such an offset amount and similarity ratio can be appropriately set and changed by the user.

[0028] Next, in the second face detection function 21 in the second processing unit 20 operating on the large high-performance computer 52, the partial image cut out by the face image extraction function 12 in the small power-saving computer 50 is processed to detect a face in the partial image.

[0029] The second face detection function 21 obtains detection results of the coordinates (Xleft’, Ytop’) of the upper left vertex and the coordinates (Xright’, Ybottom’) of the lower right vertex of the rectangle including the face. Also, in this second face detection function 21, an algorithm that detects a face using such coordinates together with feature points such as the top of the head, the lower end of the jaw, and the contour of the face may be used, or an algorithm that uses an image around the face may be used. Therefore, in the previous face image extraction function 12, the face image is extracted as a partial image in a large range including feature points such as the top of the head, the lower end of the jaw, and the contour of the face, so that the second face detection function 21 can be used.

[0030] Also, since the second face detection function 21 operates on the large high-performance computer 52, it is not necessary to use software with a low load like the first face detection function 11 installed in the small power-saving computer 50, and it is desired to have a higher function than the first face detection function 11. Therefore, in the second face detection function 21, for example, a high-function implementation that can output the coordinates of the eyes in addition to the coordinates indicating the rectangle including the face described above may be used. Also, in this second face detection function 21, a face recognition function 22 operating on the large high-performance computer 52 may process and output a detection result for identifying who the person of the face is.

[0031] Here, the face recognition function 22 may be an interface that requires, for example, the coordinates of the eyes as input data. The reason is that since the first face detection function 11 operates on the small and power-saving computer 50, it needs to be implemented as software with a low load, and thus the function of outputting the coordinates of eyes and the like is limited. And in such a case, an interface mismatch occurs between the first face detection function 11 of the first processing unit 10 and the face recognition function 22 of the second processing unit 20 in terms of the presence or absence of eye coordinates. However, according to the present embodiment, by providing a second face detection function 21 capable of outputting not only the coordinates indicating the face but also the eye coordinates within the second processing unit 20, the problem of interface mismatch between the first face detection function 11 on the digital camera CA side and the face recognition function 22 on the server 51 side can be solved.

[0032] And summarizing the operation of the face discrimination device 101 is as follows. First, after the first face detection function 11 of the first processing unit 10 specifies the face position of a person from the captured image of the digital camera CA, the face image extraction function 12 of the first processing unit 10 extracts the face image specified by the first face detection function 11 to a predetermined size. At this time, the face image extraction function 12 of the first processing unit 10 extracts a face image in a range larger than the actual face specified by the first face detection function 11. After that, the second face detection function 21 of the second processing unit 20 specifies the face position from the face image of the person transmitted from the face image extraction function 12 of the first processing unit 10, and then the face recognition function 22 identifies who the person is from the face image specified by the second face detection function 21.

[0033] And the face discrimination device 101 has the following effects. (1) The face image extraction function 12 of the first processing unit 10 extracts a face image in a range larger than the actual face specified by the first face detection function 11 and transmits it to the second face detection function 21, thereby reducing the communication data volume compared to the case of transmitting all of the input image to the large and high-performance computer 52.

[0034] (2) In the second face detection function 21 of the second processing unit 20, highly accurate face detection can be performed based on a face image in a range larger than the actual face cut out by the first processing unit 10, using the face contour and image information around the face.

[0035] (3) In the second processing unit 20, by providing the second face detection function 21 and the face recognition function 22, if the interfaces of these second face detection function 21 and face recognition function 22 are consistent, there is no need to align the interface between the first face detection function 11 of the first processing unit 10 and the face recognition function 22 of the second processing unit 20. That is, in the face discrimination device 101, with a simple configuration of adding the second face detection function 21 in the second processing unit 20, the problem of interface inconsistency between the first face detection function 11 on the digital camera CA side and the face recognition function 22 on the server 51 side can be solved.

[0036] (Modification example) The above embodiment may be modified as follows. In the above embodiment, a small power-saving computer 50 for operating the first processing unit 10 by software is provided. However, the small power-saving computer 50 may be equipped with hardware for operating the first face detection function 11 with lower power consumption. This hardware may be, for example, an additional semiconductor chip or an expansion board such as a GPGPU (General-purpose computing on graphics processing units) or what is called an AI (Artificial Intelligence) accelerator.

[0037] And with such hardware, the first face detection function 11 can be operated with lower power consumption. On the other hand, with such low-power hardware, compared with the case of implementing software operating on a CPU (Central Processing Unit) as adopted in the above-described embodiment, there may be a case where the functions required in the embodiment cannot be achieved. However, even in such a case, in the present embodiment, since the second face detection function 21 can be responsible for functions such as outputting the coordinates of the eyes, interface compatibility with the face recognition function 22 is possible.

[0038] As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and design changes and the like within the scope not departing from the gist of the present invention are also included.

Industrial Applicability

[0039] The present invention relates to a face discrimination device and a face discrimination method that can be used in public areas such as airports, railway stations, and shopping streets, work management in factories, etc., and can improve a face detection function for detecting a face and a face recognition function for recognizing characteristic parts of the face.

Explanation of Signs

[0040] 10 First processing unit 11 First face detection function 12 Face image extraction function 20 Second processing unit 21 Second face detection function 22 Face recognition function 50 Small power-saving computer 51 Server 52 Large high-performance computer 100 Face discrimination device 101 Face discrimination device A First processing unit A1 First face detection function A2 Face image extraction function B Second processing unit B1 Second face detection function B2 Face recognition function C Camera CA Digital camera CL Cloud N Network

Claims

1. A sub-computer comprising a first processing unit having a first face detection function for identifying the face position of a person from a captured image of a camera and a face image cutting function for generating a face cut-out image of a predetermined size including the entire face identified by the first face detection function and enlarged to the outside by a predetermined offset amount based on the upper / lower / left / right ends of the face, A main computer comprising a second processing unit having a second face detection function for identifying a range larger than the face range identified by the first face detection function in the face cut-out image transmitted from the face image cutting function of the first processing unit and including the face contour and the background image around the face, and for identifying the face position and the coordinates of a predetermined part on the face from that range, and a face recognition function for identifying who the person is from the face image identified by the second face detection function and the coordinates of the predetermined part, the main computer being communicatively connected to the sub-computer, The face discrimination device, wherein the face image cutting function of the first processing unit cuts out a face image in a range larger than the actual face identified by the first face detection function.

2. The first processing unit is provided in a sub-computer on the camera side, The second processing unit is provided in a main computer via a server, characterized in that The face discrimination device according to claim 1.

3. The sub-computer is a small power-saving computer, The main computer is a large high-performance computer that consumes relatively more power than the small power-saving computer, characterized in that the face discrimination device according to claim 2.

4. In the face image cutting function of the first processing unit, for the face identified by the first face detection function, the upper / lower / left / right ends are set outside the top of the head / lower end of the jaw / left / right contours of the face, characterized in that the face discrimination device according to any one of claims 1 to 3.

5. In the second face detection function and face recognition function of the second processing unit, data processing includes at least detecting the coordinates of a person's eyes, and data processing with a higher load than the first face detection function is performed. The face discrimination device according to any one of claims 1 to 4, characterized in that.

6. A first processing stage in which a subcomputer has a first face detection step of specifying a person's face position from a captured image of a camera and a face image cutting-out step of generating a face cut-out image cut out to a predetermined size including the entire face specified in the first face detection step and in a range enlarged by a predetermined offset amount based on the upper limit / lower limit / left and right ends of the face to the outside. A second processing stage in which a main computer communicatively connected to the subcomputer specifies a range including a face contour and a background image around the face larger than the face range specified in the first face detection step in the face cut-out image transmitted from the face image cutting-out step, and specifies a face position and coordinates of a predetermined part on the face from the range, and a face recognition step of specifying who the person is from the face image specified in the second face detection step and the coordinates of the predetermined part. In the face image cutting-out step, a face discrimination method characterized by cutting out a face image in a range larger than the actual face specified in the first face detection step.

7. The face discrimination method according to claim 6, characterized in that the first processing stage is executed by a processing device with relatively lower power consumption than the second processing stage, and then the processed face data is transmitted to the second processing stage.

Images