IMAGE PROCESSING APPARATUS, AUTHENTICATION SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM

Abstract

An image processing device (20) comprises: a setting unit (22) that sets a designated part of a subject except their face as a target part; an authentication unit (24) that determines whether a predetermined subject is included in a captured image obtained by capturing the region in a predetermined range with respect to a gate; a region identification unit (25) that, when the subject is included in the captured image, identifies a first region that is the face region of the subject and a second region that is the image region of the target part as a specific region that is excluded from abstraction processing; and an abstraction unit (26) that performs abstraction processing on the image region obtained by excluding the identified specific region.

Description

Image processing device, authentication system, image processing method, and non-transitory computer-readable medium

[0001] The present disclosure relates to an image processing device, an authentication system, an image processing method, and a non-transitory computer-readable medium.

[0002] Techniques for protecting the privacy of passersby who appear in an image of a subject have been proposed. For example, Patent Literature 1 discloses an image processing device that performs blurring on a background region of a target image, which is a region other than the main subject region.

[0003] Japanese Patent Application Laid-Open No. 2021-012597

[0004] In recent years, many experiments on facial recognition have been conducted at airports and train stations where the general public passes through. In these experiments, it is desirable to store the captured images for a certain period of time in order to verify the results. Therefore, there is a particularly high need to protect the privacy of passersby who appear in the images.

[0005] In such a face authentication system, it is required to flexibly set the areas to be concealed and the areas not to be concealed depending on the verification target or in accordance with the operation. However, with the technology described in the above-mentioned Patent Document 1, it is difficult to flexibly set the areas to be blurred.

[0006] In view of the above-mentioned problems, the object of the present disclosure is to provide an image processing device, an authentication system, an image processing method, and a non-transitory computer-readable medium that can flexibly set areas to be hidden or not hidden in a captured image.

[0007] An image processing device according to one aspect of the present disclosure includes a setting means for setting a specified part of a subject other than the face as a target part; an authentication means for determining whether a predetermined subject is included in a captured image of a predetermined range of an area based on a gate; an area identification means for, if the subject is included in the captured image, identifying a first area which is the face area of ​​the subject and a second area which is the image area of ​​the target part as specific areas that are not subject to abstraction processing; and an abstraction means for performing abstraction processing on the image area excluding the identified specific area.

[0008] An authentication system according to one aspect of the present disclosure includes an authentication device that performs face authentication based on a facial region of a subject, and an image processing device. The image processing device includes a setting unit that sets a specified region other than the face of the subject as a target region, an authentication unit that determines whether a predetermined subject is included in a captured image of a predetermined range based on a gate, a region specifying unit that, if the subject is included in the captured image, specifies a first region that is the facial region of the subject and a second region that is an image region of the target region as specific regions that are not subject to abstraction processing, and an abstraction unit that performs abstraction processing on the image region excluding the specified specific region.

[0009] An image processing method according to one aspect of the present disclosure sets a specified part of a subject other than the face as the target part, determines whether a predetermined subject is included in a captured image of a predetermined range of area based on a gate, and if the subject is included in the captured image, identifies a first area which is the subject's face area and a second area which is the image area of ​​the target part as specific areas that are not subject to abstraction processing, and performs abstraction processing on the image area excluding the identified specific area.

[0010] A non-transitory computer-readable medium according to one aspect of the present disclosure stores a program for causing a computer to implement the following functions: a function of setting a specified part of a body other than the face of a subject as a target part; a function of determining whether a predetermined subject is included in a captured image of a predetermined range of an area based on a gate; a function of identifying, if the subject is included in the captured image, a first area which is the face area of ​​the subject and a second area which is the image area of ​​the target part as specific areas that are not subject to abstraction processing; and a function of performing abstraction processing on the image area excluding the identified specific area.

[0011] The present disclosure can provide an image processing device, an authentication system, an image processing method, and a non-transitory computer-readable medium that can flexibly set areas to be hidden or not hidden in a captured image.

[0012] 1 is a block diagram showing the configuration of an image processing device according to a first embodiment. FIG. 2 is a flowchart showing an example of the flow of image processing according to the first embodiment. FIG. 3 is a block diagram showing the overall configuration of an authentication system according to a second embodiment. FIG. 4 is a block diagram showing the configuration of an authentication terminal according to the second embodiment. FIG. 5 is a block diagram showing the configuration of an authentication device according to the second embodiment. FIG. 6 is a flowchart showing an example of the flow of face information registration processing according to the second embodiment. FIG. 7 is a flowchart showing an example of the flow of authentication processing according to the second embodiment. FIG. 8 is a diagram for explaining a specific example of authentication processing according to the second embodiment. FIG. 9 is a block diagram showing the configuration of an image processing device according to the second embodiment. FIG. 10 is a flowchart showing an example of the flow of image processing according to the second embodiment. FIG. 11 is a diagram for explaining an example of a display on a display unit 250 according to the second embodiment. FIG. 12 is a diagram for explaining a specific example of image processing according to the second embodiment. FIG. 13 is a diagram for explaining a specific example of image processing according to the second embodiment.

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0014] <First Embodiment> Next, a first embodiment of the present disclosure will be described. Fig. 1 is a block diagram showing the configuration of an image processing device 20 according to the first embodiment. The image processing device 20 is one or more devices that perform abstraction processing on an image region other than a specific region in a captured image.

[0015] The specific region is a region that is not subject to abstraction processing. Abstraction processing is processing that makes it impossible to identify objects or people that appear in the image region that is the subject of abstraction processing. The abstraction processing is, for example, mosaic processing, blurring processing, filling processing, or transparency processing. In this embodiment 1, the specific region is a region that combines a first region included in the captured image and a second region included in the captured image.

[0016] The first area is a predetermined facial area of ​​a subject. For example, the subject is a person conducting or collaborating in a demonstration experiment of facial recognition. The second area is an image area of ​​a target part designated in advance by a user of the image processing device 20. The target part may be at least one of a part of the subject's body excluding the face, baggage carried by the subject, and accessories worn by the subject. In an experiment to authenticate the face of a subject passing through a gate, the target part may be a reading unit such as an IC card reader installed at the gate.

[0017] The image processing device 20 includes a setting unit 22 , an authentication unit 24 , an area specifying unit 25 , and an abstraction unit 26 .

[0018] The setting unit 22 is also referred to as a setting means. The setting unit 22 sets a specified body part as a target body part. For example, the setting unit 22 receives a designation of the type of target body part from a user of the image processing device 20 and sets the target body part. Furthermore, for example, in response to receiving information designating the type of target body part from another device, the setting unit 22 sets the target body part based on the information designating the type of target body part.

[0019] The authentication unit 24 is also referred to as an authentication means. The authentication unit 24 determines whether a predetermined subject is included in a captured image of a predetermined area based on the gate.

[0020] For example, the authentication unit 24 may determine whether the subject is included in the captured image by controlling face authentication for the facial region of each person included in the captured image. Controlling face authentication may mean that the image processing device 20 performs face authentication. Alternatively, controlling face authentication may mean that the image processing device 20 requests face authentication from another face authentication device (not shown) and receives the face authentication result from the face authentication device. Then, the authentication unit 24 may determine whether the subject is included in the detected persons based on the result of face authentication.

[0021] The region identification unit 25 is also referred to as a region identification means. When a subject is included in a captured image, the region identification unit 25 identifies a first region and a second region as identified regions. The identified regions are image regions that are not subject to abstraction processing.

[0022] The abstraction unit 26 is also referred to as an abstraction means. The abstraction unit 26 performs abstraction processing on the image region excluding the specific region by converting the pixel values ​​of pixels included in the image region excluding the specific region of the captured image. On the other hand, the abstraction unit 26 does not convert the pixel values ​​of the specific region. In other words, the abstraction unit 26 does not perform abstraction processing on the specific region.

[0023] 2 is a flowchart showing an example of the flow of image processing according to the first embodiment. First, the setting unit 22 of the image processing device 20 sets a specified body part as a target body part (S11). For example, the setting unit 22 receives a designation of the type of target body part from the user of the image processing device 20 and sets the received body part as the target body part. At this time, the user may designate the subject's hand, finger, ear, or posture as the type of target body part. The user may also designate the subject's baggage or an accessory worn by the subject as the type of target body part. The user may also designate a reading unit installed at a gate as the type of target body part.

[0024] Next, the authentication unit 24 controls face authentication for each person included in the captured image (S12). As a result, the authentication unit 24 obtains a face authentication result. Then, the authentication unit 24 determines whether the target person is included in the captured image (S13).

[0025] If the authentication unit 24 determines that the subject is included in the captured image (Yes in S13), the region identification unit 25 identifies the subject's face region (i.e., the first region) and the image region of the target body part (i.e., the second region) as identified regions (S14). Next, the abstraction unit 26 performs abstraction processing on the image region of the captured image other than the identified region (S15). The image processing device 20 then outputs the abstracted image (S16).

[0026] On the other hand, if the authentication unit 24 determines that the subject is not included in the captured image (No in S13), the image processing device 20 ends the process.

[0027] In S16, the image processing device 20 may store the abstracted image in a storage device.

[0028] As described above, according to the first embodiment, the facial region of the subject and the image region of the target body part specified by the user of the image processing device 20 are set as regions that are not concealed, and the abstraction process is performed on the other image regions. Therefore, it is possible to flexibly set regions that are concealed or not concealed according to the verification target or in line with the operation.

[0029] <Second Embodiment> Next, a second embodiment of the present disclosure will be described. Fig. 3 is a block diagram showing the overall configuration of an authentication system 1 according to the second embodiment. The authentication system 1 is a computer system that carries out a demonstration experiment to authenticate the identity of a person passing through a gate 60. The authentication system 1 stores the captured image used for the identity authentication for a predetermined period of time. In the second embodiment, the identity authentication is performed by facial authentication.

[0030] The authentication system 1 includes an authentication device 10, an image processing device 20a, an image storage device 30, an authentication terminal 40, and a gate 60. The authentication device 10, the image processing device 20a, the image storage device 30, and the authentication terminal 40 are communicably connected via a network N.

[0031] The network N is a wired or wireless communication network.

[0032] The gate 60 may be a ticket gate at a station where a demonstration experiment is conducted, a gate at an airport, or a gate at a concert hall or amusement park, etc. The gate 60 includes a first gate body 61 and a second gate body 62.

[0033] The first gate body 61 and the second gate body 62 are spaced apart at a predetermined distance in a direction perpendicular to the direction of travel of pedestrians. A camera 410 of the authentication terminal 40 is provided in the gate 60, and the camera 410 photographs pedestrians, and the authentication device 10 performs facial authentication based on the photographed image. The gate 60 opens or closes based on the results of the facial authentication.

[0034] Opening or closing the gate 60 means rotating or sliding the flap 63 to an open or closed state. However, opening or closing the gate 60 may also mean unlocking or locking the flap 63, or simply displaying or outputting a sound indicating whether passage is permitted or not. The flap 63 is not essential to the gate 60.

[0035] In this figure, the camera 410 is shown as being disposed on the second gate body 62, but this is not limiting and the camera 410 may be disposed on the first gate body 61. The camera 410 may also be provided at a distance from the gate 60.

[0036] The authentication device 10 is an example of a face authentication device that performs face authentication based on a person's face area. The authentication device 10 stores face information for each subject and performs face authentication on a person passing through the gate 60 using the stored face information.

[0037] For example, upon receiving a captured image from the authentication terminal 40, the authentication device 10 detects a person's face area from the captured image and performs face authentication on the face area. The authentication device 10 transmits the result of the face authentication to the authentication terminal 40.

[0038] The image processing device 20a is an example of the image processing device 20 of the first embodiment. The image processing device 20a performs abstraction processing on an image region other than the specific region of a captured image used by the authentication device 10 for face authentication, to generate an image in which at least a portion is abstracted. Hereinafter, an image in which at least a portion is abstracted will be referred to as an abstracted image. In the second embodiment, the abstracted image is used to verify the results of face authentication. The specific region, which is an area that is not abstracted, is composed of the subject's face region and an image region of the target region. The target region is an area that the verifier checks during verification to understand the situation at the time of face authentication. In the second embodiment, the target region may be at least one of a body region of the subject excluding the face, baggage carried by the subject, and an accessory worn by the subject.

[0039] The image processing device 20 a then stores the generated abstract image in the image storage device 30 .

[0040] The image storage device 30 is a storage device that stores the abstract image generated by the image processing device 20a. The abstract image stored in the image storage device 30 is deleted when a predetermined period of time has passed since the image was stored.

[0041] The authentication terminal 40 is a terminal device equipped with a camera 410 installed near the gate 60. The authentication terminal 40 transmits an image of a passerby taken by the camera 410 to the authentication device 10 and requests facial authentication from the authentication device 10. The authentication terminal 40 controls the opening and closing of the gate 60 based on the result of the facial authentication from the authentication device 10.

[0042] The gate 60 and the authentication terminal 40 may be configured as separate devices or may be configured as an integrated unit. The gate 60 may not have an opening / closing mechanism. For example, the gate 60 may be a pole-type gate rather than having two separate gate bodies, or may simply be a passageway. In this case, the authentication terminal 40 may output whether or not passage is permitted, instead of controlling the opening / closing of the gate 60.

[0043] 4 is a block diagram showing the configuration of the authentication terminal 40 according to embodiment 2. The authentication terminal 40 includes a camera 410, a storage unit 420, a communication unit 430, a display unit 440, and a control unit 460.

[0044] The camera 410 is an imaging device that takes images under the control of the control unit 460. The memory unit 420 is a storage device that stores programs for realizing each function of the authentication terminal 40. The communication unit 430 is a communication interface with the network N. The display unit 440 is a display device. The control unit 460 controls the hardware that the authentication terminal 40 has.

[0045] For example, the control unit 460 controls the camera 410 to capture an image of a predetermined range of area based on the gate, and transmits the captured image to the authentication device 10 via the communication unit 430. Furthermore, in response to the communication unit 430 receiving the result of face authentication from the authentication device 10, the control unit 460 controls the opening and closing of the gate 60 based on the result of face authentication.

[0046] 5 is a block diagram showing the configuration of the authentication device 10 according to the second embodiment. The authentication device 10 includes a storage unit 100, a memory 110, a communication unit 120, and a control unit 140.

[0047] The storage unit 100 is a storage device such as a hard disk, a flash memory, etc. The storage unit 100 stores a program 101 and a face information DB 102. The program 101 is a computer program in which the processing by the authentication device 10 is implemented.

[0048] The face information DB 102 stores a subject ID for identifying a subject and face information of the subject in association with each other. The face information is a set of feature points extracted from a face image.

[0049] The memory 110 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily storing information during operation of the control unit 140. The communication unit 120 is a communication interface with the network N.

[0050] The control unit 140 is a processor, i.e., a control device, that controls each component of the authentication device 10. The control unit 140 loads the program 101 from the storage unit 100 into the memory 110 and executes the program 101. In this way, the control unit 140 realizes the functions of a registration unit 141, a detection unit 143, and an authentication unit 144.

[0051] The registration unit 141 issues a new subject ID when registering face information. The registration unit 141 also detects a face area from a registration image for registering the face information, extracts feature points from the face area, and generates information on the extracted feature points as face information. The registration unit 141 associates the issued subject ID with the face information for registration extracted from the registration image and registers them in the face information DB 102.

[0052] The detection unit 143 detects a person's face area from the captured image for authentication received from the authentication terminal 40, and supplies the face area to the authentication unit 144. If the captured image includes multiple people, the detection unit 143 detects a face area for each person and supplies the face area to the authentication unit 144.

[0053] The authentication unit 144 extracts feature points from the face area detected by the detection unit 143 and uses this as face information for authentication. The authentication unit 144 then performs face authentication using the face information. Specifically, the authentication unit 144 compares the face information extracted from the captured image for authentication with the face information in the face information DB 102. As a result, the authentication unit 144 identifies the subject ID whose face information matches. Note that face information matching (matching) refers to a case where the degree of match is equal to or greater than a predetermined value.

[0054] The authentication unit 144 transmits the result of the face authentication to the authentication terminal 40 via the communication unit 120. The authentication unit 144 also transmits the captured image for authentication to the image processing device 20a via the communication unit 120.

[0055] FIG. 6 is a flowchart showing an example of the flow of face information registration processing according to the second embodiment. First, the registration unit 141 of the authentication device 10 acquires a registration image of the subject (S21). For example, the authentication device 10 acquires the registration image by receiving a face registration request including the registration image of the subject from a terminal used by the subject. Next, the registration unit 141 detects a face area included in the registration image (S22). Next, the registration unit 141 extracts face information from the face area detected in step S22 (S23). Finally, the registration unit 141 issues a subject ID and associates the subject ID with the face information and registers it in the face information DB 102 (S24). Note that the registration unit 141 may receive face information instead of the registration image from the source of the face registration request and register the face information in the face information DB 102 in association with the subject ID. In this case, the processes of S21 to S23 may be omitted.

[0056] FIG. 7 is a flowchart illustrating an example of the flow of authentication processing according to the second embodiment.

[0057] First, in response to receiving a photographed image from the authentication terminal 40 (S31), the detection unit 143 detects a person's face area from the photographed image (S32). If the detection unit 143 detects face areas of multiple people, it determines the person to be authenticated from the multiple people (S33).

[0058] FIG. 8 is a diagram illustrating a specific example of authentication processing according to the second embodiment. FIG. 8 shows a captured image 500 captured near the gate 60. The captured image 500 includes persons P1 to P5 who are about to pass through the gate 60. The detection unit 143 detects the facial areas of the multiple persons P1 to P5 in S32. Then, in S33, the detection unit 143 may determine the person P1 closest to the gate 60 as the person to be authenticated. Alternatively, in S33, the detection unit 143 may determine a person located within a predetermined range based on the gate 60 as the person to be authenticated. The authentication unit 144 performs face authentication on the person to be authenticated determined in this manner.

[0059] Returning to FIG. 7 , the explanation will be continued. The authentication unit 144 executes the processes shown in S34 to S38. First, the authentication unit 144 extracts facial information from the facial area of ​​the person to be authenticated (S34). Next, the authentication unit 144 compares the extracted facial information with the facial information in the facial information DB 102 (S35). If the degree of match between the facial information is equal to or greater than a predetermined value (Yes in S36), the authentication unit 144 outputs a message to the authentication terminal 40 indicating that facial authentication was successful (S37). On the other hand, if the degree of match between the facial information is less than the predetermined value (No in S36), the authentication unit 144 outputs a message to the authentication terminal 40 indicating that facial authentication was unsuccessful (S38).

[0060] Then, the authentication unit 144 outputs the captured image to the image processing device 20a (S39).

[0061] 9 is a block diagram showing the configuration of an image processing device 20a according to embodiment 2. The image processing device 20a includes a storage unit 200, a memory 210, a communication unit 220, an input unit 230, a control unit 240, and a display unit 250.

[0062] The storage unit 200 is a storage device such as a hard disk, a flash memory, etc. The storage unit 200 stores a program 201, a face information DB 202, and target body part information 203. The program 201 is a computer program that implements the processing by the image processing device 20a.

[0063] Similar to the face information DB 102, the face information DB 202 stores a subject ID and face information of the subject in association with each other.

[0064] The target part information 203 stores information about the target part. For example, the target part information 203 stores the type of target part in association with feature information about that target part. For example, if the type of target part is "bag," the feature information is information for detecting the image area of ​​the subject's bag. For example, the feature information may include information on feature amounts extracted from an image of a general bag. Alternatively, for example, the feature information may include the general relative position of the bag with respect to the face area. Note that the relative position may be a value normalized by the size of the face area.

[0065] The memory 210 is a volatile storage device such as a RAM, and is a storage area for temporarily storing information when the control unit 240 is operating. The communication unit 220 is a communication interface with the network N. The input unit 230 is an input device that accepts input from a user of the image processing device 20a.

[0066] The control unit 240 is a processor, i.e., a control device, that controls each component of the image processing device 20 a. The control unit 240 loads the program 201 from the storage unit 200 into the memory 210 and executes the program 201. In this way, the control unit 240 realizes the functions of a registration unit 241, a setting unit 242, a detection unit 243, an authentication unit 244, an area identification unit 245, and an abstraction unit 246.

[0067] The registration unit 241 is also referred to as a registration means. It has the same function as the registration unit 141. That is, the registration unit 241 associates a subject ID with face information for registration and registers the associated information in the face information DB 202. The registration unit 241 may extract face information for registration from a registration image, or may receive the face information extracted from the registration image together with the subject ID from the registration unit 141. In the former case, the registration unit 241 executes the same process as that shown in FIG. 6 .

[0068] The setting unit 242 is an example of the setting unit 22 in the first embodiment. The setting unit 242 receives a designation of the type of target region from the user of the image processing device 20a via the input unit 230. The setting unit 242 then sets the designated region as the target region.

[0069] The detection unit 243 is also referred to as a detection means. The detection unit 243 detects a person's face area from the captured image for authentication received from the authentication device 10, and supplies the face area to the authentication unit 244. The method of detecting the face area is the same as that of the detection unit 143. The face area detected by the detection unit 243 is also referred to as a first area. When the captured image includes multiple people, the detection unit 243 detects a face area for each person and supplies the face area to the authentication unit 244.

[0070] The authentication unit 244 extracts feature points from the face area detected by the detection unit 243 and uses this as face information for authentication. The authentication unit 244 then performs face authentication using the face information. The face authentication method is the same as that of the authentication unit 144. A person whose face is successfully authenticated here is one of the subjects registered in advance.

[0071] The authentication unit 244 supplies information on the face area (first area) of the subject whose face has been successfully authenticated to the area identification unit 245 .

[0072] The region identification unit 245 is an example of the region identification unit 25 of embodiment 1. First, the region identification unit 245 detects a second region. Specifically, the region identification unit 245 detects the second region, which is an image region of the target region, based on the position of the face region of the subject identified by the authentication unit 244 and the type of the specified target region. More specifically, the region identification unit 245 detects the second region using the position of the face region of the subject and feature information associated with the type of target region in the target region information 203.

[0073] The area specifying unit 245 then specifies the first area specified by the authentication unit 244 and the second area detected by itself as a specified area.

[0074] The abstraction unit 246 is an example of the abstraction unit 26 of embodiment 1. The abstraction unit 246 converts pixel values ​​of image regions other than the specific region in the captured image. In this way, the abstraction unit 246 performs abstraction processing on the image regions other than the specific region.

[0075] The display unit 250 is a display device. For example, the display unit 250 displays an input screen for receiving a designation of the type of target part from the user.

[0076] 10 is a flowchart showing an example of the flow of image processing according to embodiment 2. First, the setting unit 242 of the image processing device 20a accepts a designation of a type of target part from a user (S60), and sets the designated part as a target part (S61).

[0077] At this time, the display unit 250 may display an input screen 600 shown in FIG. 11 . FIG. 11 is a diagram illustrating an example of a display on the display unit 250 according to the second embodiment. For example, the input screen 600 includes a message prompting the user to select a target body part that is not to be subjected to the abstraction process, and further includes a selection area for the user to select one type from multiple target body part types. Examples of target body part types that can be selected include hand, whole body, bag, and hat. In response to the user selecting one type, the setting unit 242 accepts a designation of the type of target body part. The setting unit 242 then sets the target body part according to the accepted designation.

[0078] Returning to FIG. 10 , the explanation continues. Next, the communication unit 220 receives the captured image used for face authentication from the authentication device 10 (S62). Next, the detection unit 243 detects a person's face area from the captured image (S63). Next, the authentication unit 244 performs face authentication for each detected person (S64). Specifically, as shown in S34 to S36 of FIG. 7 , the authentication unit 244 extracts face information from the face area, compares the extracted face information with face information in the face information DB 202, and determines whether the degree of match between the face information is equal to or greater than a predetermined value. If the degree of match between the face information is equal to or greater than the predetermined value, the authentication unit 244 determines that face authentication has been successful, and determines that the person whose face authentication has been successful is a pre-registered subject. On the other hand, if the degree of match between the face information is less than the predetermined value, the authentication unit 144 determines that face authentication has failed.

[0079] The region identification unit 245 determines whether the target person is included in the detected people based on the face authentication results of all the detected people (S65). If the target person is not included in the detected people (No in S65), the image processing device 20a ends the process.

[0080] On the other hand, if the detected people include the target person (Yes in S65), the region identifying unit 245 detects the image region of the target part as the second region (S66). Specifically, the region identifying unit 245 detects the second region by using feature information associated with the type of the specified target part in the target part information 203.

[0081] The region identifying unit 245 identifies the face region of the person determined to be the target person as the first region. Then, the region identifying unit 245 identifies the first region, which is the face region of the target person, and the second region, which is the image region of the target body part, as identified regions (S67).

[0082] Next, the abstraction unit 246 performs abstraction processing on image regions other than the specific region included in the photographed image (S68).

[0083] The abstraction unit 246 then stores the abstracted image in the image storage device 30 (S69).

[0084] In the above description, the target part is set in S61, but the target part may be set at any time between S62 and S66. In this case, the target part may be set by setting the image area of ​​the target part (i.e., the second area) instead of setting the type of the target part. In the latter case, the process shown in S66 may be omitted.

[0085] For example, the display unit 250 of the image processing device 20 a may display the captured image, and the setting unit 242 may set a portion of the captured image designated by the user as the second region. As a method for designating the second region, the user may directly input the coordinates of the second region using a keyboard, or may surround the second region using a pointing device.

[0086] The setting unit 242 may accept the designation of the second region each time a captured image is received from the authentication device 10, but this is not limited to this. For example, before operation begins, the setting unit 242 may accept the designation of the second region from the user using one or more captured images and train a second region detection model that detects the second region from the captured image. Then, after operation begins, the setting unit 242 may detect the second region using the trained second region detection model. This reduces the burden of designation on the user.

[0087] 12 is a diagram illustrating a specific example of image processing according to the second embodiment. The abstract image 510 includes face regions of multiple people detected by the detection unit 243. The authentication unit 244 identifies the face region of person P1, whose degree of match with pre-registered face information of the target person is equal to or greater than a predetermined value, as a first region F1. The region identification unit 245 then detects the image region of the target region using feature information associated with the type of the specified target region in the target region information 203, and identifies this as a second region S1.

[0088] For example, if the feature information stored in the target part information 203 includes information on the feature amount of a bag, the region identification unit 245 may detect an image region in which the degree of match of the feature amount is equal to or greater than a predetermined threshold as the second region S1.

[0089] Alternatively, if the feature information includes the relative position of the bag with respect to the facial area, the area identification unit 245 may detect the second area S1 based on the position and size of the subject's facial area (first area F1) and the relative position.

[0090] Alternatively, if the feature information includes a relative position based on another part of the body, the region identification unit 245 may identify the second region S1 using the relative position and skeletal information. As an example, a bag is often located near the hand. Therefore, the region identification unit 245 may first extract skeletal information of the subject and identify the position of the subject's hand from the skeletal information. Then, the region identification unit 245 may detect an image region within a predetermined range from the identified hand position as the second region S1.

[0091] The region identification unit 245 may detect the second region S1 by combining some or all of the above-mentioned methods using the bag features, the method using the relative position based on the face region, and the method using the relative position based on other parts of the body.

[0092] The abstraction unit 246 then performs abstraction processing on the image region excluding the first region F1 and the second region S1. For example, the abstraction unit 246 abstracts the image region other than the specific region by lowering the resolution, converting multiple pixels to the same pixel value, or uniformly converting them to a specific pixel value. When converting multiple pixels to the same pixel value, the abstraction unit 246 may convert the pixel values ​​of the multiple pixels to an average pixel value of the multiple pixels, or may convert the pixel values ​​of any pixel among the multiple pixels.

[0093] In the above description, the target part is at least one of a body part of the subject excluding the face, baggage carried by the subject, and accessories worn by the subject. However, the target part may also be an IC card reader provided at the gate 60. In this case, the image processing device 20a may store a predetermined position and size in the captured image as the position and size of the IC card reader in the target part information 203.

[0094] A specific example of image processing in this case will be described with reference to Fig. 13. First, the region identification unit 245 identifies a first region F1 in the same manner as in Fig. 12. The region identification unit 245 also reads out characteristic information of the reading unit stored in the target part information 203 and determines the relative position and size of the reading unit in the captured image. Then, the region identification unit 245 detects a second region S1 from the captured image based on the relative position and size.

[0095] The image processing device 20a may store information on features extracted from an image captured by the IC card reader in the target body part information 203. In this case, the region identification unit 245 may detect, as the second region S1, an image region in which the degree of match of the features is equal to or greater than a predetermined value.

[0096] The abstraction unit 246 then performs abstraction processing on the image area excluding the first area F1 and the second area S1.

[0097] The target portion may also be a predetermined region near the gate 60. Fig. 14 is a diagram for explaining a specific example of image processing according to the second embodiment. In this case, the region specifying unit 245 may specify the second region S1 using pre-registered feature information in the same manner as when the target portion is a reading unit.

[0098] As described above, according to the second embodiment, the facial region of the subject and the image region of the target body part specified by the user of the image processing device 20 are set as regions that are not concealed, and the other image regions are subjected to abstraction processing. Therefore, it is possible to flexibly set regions that are concealed or not concealed depending on the verification target or in accordance with the operation. This allows the image processing device 20a to be used universally regardless of the authentication method of the authentication device 10.

[0099] The authentication device 10 may transmit the face area detection results and face authentication results to the image processing device 20a. This allows the image processing device 20a to use the face area detection results and face authentication results for image processing, so the face area detection process and face authentication process (S62 to S63 in FIG. 10) may be omitted.

[0100] The present disclosure is not limited to the above-described embodiments, and can be modified as appropriate within the scope of the present disclosure.

[0101] <First Modification> For example, two or more types of target areas may be specified. As an example, the user may specify the target person's baggage and the IC card reader as the target areas. In this case, the area identification unit 245 may identify the image area of ​​each specified target area as the second area.

[0102] <Second Modification> In the second embodiment, the region identification unit 245 uniformly defines the first and second regions as specific regions. However, the specific region may be limited by the time of capture. For example, the region identification unit 245 may identify the first and second regions within a predetermined period based on the estimated time of the subject's entry or exit through gate 60. In this case, the region identification unit 245 may estimate the entry time or exit time based on the arrival timing of the subject's IC card. As an example, the region identification unit 245 may estimate the entry time to be a predetermined time (e.g., 5 seconds) before the time the IC card reader detects a signal. The region identification unit 245 may also estimate the exit time to be a predetermined time (e.g., 2 seconds after the time the IC card reader detects a signal). The region identification unit 245 may then identify the first and second regions of the captured images captured between the entry time and the exit time as specific regions, and exclude the first and second regions of the captured images captured at other times from the specific region. This allows information unnecessary for verification to be hidden, making it possible to verify the authentication result more easily. In particular, when face authentication is performed using the walk-through method, the actions of entering and exiting the gate 60 can be verified more easily.

[0103] <Third Modification> In the above-described embodiment, the authentication device 10 authenticates the identity of the person to be authenticated by facial authentication. However, the identity authentication may be multi-factor authentication including a first authentication, which is facial authentication, and a second authentication other than facial authentication. For example, the second authentication may be authentication that determines whether the person to be authenticated is assuming a pre-registered pose when passing through a gate. Alternatively, the second authentication may be authentication that determines whether the person to be authenticated is looking in a pre-registered direction when passing through a gate. Alternatively, the second authentication may be authentication that determines whether the person to be authenticated is carrying pre-registered baggage or wearing pre-registered accessories. Alternatively, the second authentication may be authentication by reading an IC card. The body part used for the second authentication is the target body part.

[0104] In this case, the authentication device 10 may store target body part information that stores the type of target body part and feature information of the target body part. In addition to the first authentication, which is face authentication, the authentication device 10 may perform a second authentication on the image area of ​​the target body part using the target body part information. If the first authentication and the second authentication are successful, the authentication device 10 transmits to the authentication terminal 40 a message that the personal authentication was successful. On the other hand, if either authentication fails, the authentication device 10 transmits to the authentication terminal 40 a message that the personal authentication was unsuccessful.

[0105] Details of the personal authentication are as follows, for example. First, the detection unit 143 of the authentication device 10 detects a second region to be used for second authentication in addition to a facial region from the captured image. Next, the authentication unit 144 of the authentication device 10 extracts features from the second region detected by the detection unit 143 and uses this as feature information for authentication. The authentication unit 144 then compares the feature information for authentication with feature information linked to the subject ID in the target body part information 103. The authentication unit 144 authenticates the person based on the result of the first authentication based on the facial information and the result of the second authentication based on the feature information for authentication extracted from the second region.

[0106] When the authentication device 10 employs multi-factor authentication in this manner, the setting unit 242 of the image processing device 20a may set a body part corresponding to the authentication method of the authentication device 10 as a target body part. For example, the setting unit 242 of the image processing device 20a may receive the authentication method from the authentication device 10 and set a body part corresponding to the authentication method as a designated target body part. This eliminates the need for user input. Furthermore, for example, the setting unit 242 of the image processing device 20a may receive designation of a second authentication method from the user and set a body part corresponding to the received method as a designated target body part. This allows areas to be hidden or not hidden to be flexibly set in conjunction with the authentication method of the authentication device 10. This simplifies user input.

[0107] Note that a mode, such as whether or not second authentication is performed and the authentication method of the second authentication, may be set for each gate 60. For example, one-factor authentication using facial authentication may be set for the first gate, and two-factor authentication using facial authentication and IC card authentication may be set for the second gate. In this case, the setting unit 242 may set whether or not to set a target body part and which part to set as the target body part, depending on the mode set for each gate 60. In the above example, the setting unit 242 may not set a target body part for images captured at the first gate. In other words, the abstraction unit 246 may not perform abstraction processing on images captured at the first gate. Furthermore, the setting unit 242 may set the reading unit as the target body part for images captured at the second gate. In other words, the abstraction unit 246 may perform abstraction processing on an image area excluding the subject's face and the reading unit in images captured at the second gate.

[0108] <Other Modifications> In the above-described second embodiment, the image processing device 20a receives captured images from the authentication device 10, but may receive captured images from the authentication terminal 40. Furthermore, the image processing device 20a performs image processing on captured images used by the authentication device 10 for face authentication, but may also perform image processing on captured images other than the captured images used by the authentication device 10 for face authentication.

[0109] In the second embodiment, the authentication device 10 and the image processing device 20a are separate devices, but the authentication device 10 and the image processing device 20a may be configured as a single device.

[0110] Any processing in the present disclosure can be realized by causing a processor to execute a computer program.

[0111] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0112] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) An image processing device comprising: a setting means for setting a specified part other than the face of a subject as a target part; an authentication means for determining whether a predetermined subject is included in a captured image of a predetermined range of an area based on a gate; a region specifying means for specifying, if the subject is included in the captured image, a first region that is the subject's facial region and a second region that is an image region of the target part as specific regions that are not subject to abstraction processing; and an abstraction means for performing abstraction processing on image regions excluding the specified specific region. (Supplementary Note 2) The image processing device according to Supplementary Note 1, wherein the target part includes at least one of a part of the subject's body excluding the face, baggage carried by the subject, and accessories worn by the subject. (Supplementary Note 3) The image processing device according to Supplementary Note 2, wherein the region specifying means detects the second region based on the position of the identified facial region of the subject and the type of the target part. (Supplementary Note 4) The image processing device according to Supplementary Note 1 or 2, wherein the target region includes a reading unit installed at the gate. (Supplementary Note 5) The image processing device according to any one of Supplements 1 to 4, wherein the region identification means identifies the first region and the second region within a predetermined period based on a time when the subject is estimated to have entered or exited the gate as the specific region. (Supplementary Note 6) An authentication system comprising: an authentication device that performs face authentication based on a person's facial region; and an image processing device, wherein the image processing device has: a setting means for setting a specified region other than the subject's face as the target region; an authentication means for determining whether a predetermined subject is included in a captured image that captures a region within a predetermined range based on the gate; a region identification means for, when the subject is included in the captured image, identifying a first region that is the subject's facial region and a second region that is an image region of the target region as specific regions that are not subject to abstraction processing; and an abstraction means for performing abstraction processing on an image region excluding the identified specific region.(Supplementary Note 7) The authentication system according to Supplementary Note 6, wherein the authentication device detects the second region from the captured image, and authenticates the person further based on the result of the face authentication and the result of second authentication based on features extracted from the second region. (Supplementary Note 8) The authentication system according to Supplementary Note 7, wherein the setting means sets a part of the body according to the method of second authentication as the target part. (Supplementary Note 9) An image processing method comprising: setting a specified part of the body other than the face of a subject as the target part; determining whether a predetermined subject is included in a captured image capturing an area of ​​a predetermined range based on a gate; specifying, if the subject is included in the captured image, a first area that is the face area of ​​the subject and a second area that is the image area of ​​the target part as specific areas that are not subject to abstraction processing; and performing abstraction processing on the image area excluding the specified specific area. (Supplementary Note 10) A non-transitory computer-readable medium storing a program for causing a computer to implement the following functions: a function for setting a specified part of a body other than the face of a subject as a target part; a function for determining whether a predetermined subject is included in a captured image of a predetermined range of an area based on a gate; a function for identifying a first area, which is the face area of ​​the subject, and a second area, which is the image area of ​​the target part, as specific areas that are not subject to abstraction processing if the subject is included in the captured image; and a function for performing abstraction processing on image areas excluding the identified specific area.

[0113] REFERENCE SIGNS LIST 1 Authentication system 10 Authentication device 20, 20a Image processing device 22 Setting unit 23 Detection unit 24 Authentication unit 25 Area identification unit 26 Abstraction unit 30 Image storage device 40 Authentication terminal 60 Gate 61 First gate body 62 Second gate body 63 Flap 64 Reading unit 100 Storage unit 101 Program 102 Face information DB 110 Memory 120 Communication unit 140 Control unit 141 Registration unit 143 Detection unit 144 Authentication unit 200 Storage unit 201 Program 202 Face information DB 203 Target part information 210 Memory 220 Communication unit 230 Input unit 240 Control unit 241 Registration unit 242 Setting unit 243 Detection unit 244 Authentication unit 245 Region identification unit 246 Abstraction unit 250 Display unit 410 Camera 420 Storage unit 430 Communication unit 440 Display unit 450 Input unit 460 Control unit 500 Captured image 510 Abstraction image 600 Input screen

Claims

1. A setting means for setting a designated part other than the face of the subject as the target part; an authentication means for determining whether or not a predetermined target person is included in a captured image of a predetermined range of an area based on the gate; an area specifying means for specifying, when the photographed image includes the subject, a first area that is a face area of ​​the subject and a second area that is an image area of ​​the target part as specific areas that are not subject to abstraction processing; an abstraction means for performing abstraction processing on an image region other than the identified specific region; An image processing device comprising:

2. The target part includes at least one of a part of the target's body other than the face, baggage carried by the target, and an accessory worn by the target. The image processing device according to claim 1 .

3. The region specifying means detects the second region based on the specified position of the face region of the subject and the type of the subject part. The image processing device according to claim 2 .

4. The target portion includes a reading unit installed at the gate.

3. The image processing device according to claim 1 or 2.

5. The area specification means specifies, as the specified area, the first area and the second area within a predetermined period based on a time when the target person is estimated to have entered or exited the gate.

3. The image processing device according to claim 1 or 2.

6. An authentication device that performs face authentication based on a person's face area; Image processing device Equipped with The image processing device includes: A setting means for setting a designated part other than the face of the subject as the target part; an authentication means for determining whether or not a predetermined target person is included in a captured image of a predetermined range of an area based on the gate; an area specifying means for specifying, when the photographed image includes the subject, a first area that is a face area of ​​the subject and a second area that is an image area of ​​the target part as specific areas that are not subject to abstraction processing; an abstraction means for performing abstraction processing on an image region other than the identified specific region; have Authentication system.

7. The authentication device includes: Detecting the second region from the captured image; and performing authentication of the person based on a result of the face authentication and a result of a second authentication based on the feature amount extracted from the second region. The authentication system according to claim 6.

8. The setting means sets a body part according to a method of the second authentication as the target body part. The authentication system according to claim 7.

9. A designated part of the subject other than the face is set as the target part; determining whether or not a predetermined subject is included in a captured image of a predetermined range of an area based on the gate; When the subject is included in the photographed image, a first region that is a face region of the subject and a second region that is an image region of the target part are identified as specific regions that are not subject to abstraction processing; Abstraction processing is performed on the image area excluding the identified specific area. Image processing methods.

10. A function of setting a specified part of the subject other than the face as the target part; A function of determining whether or not a predetermined subject is included in a captured image of a predetermined range of an area based on the gate; a function of specifying, when the subject is included in the photographed image, a first region that is a face region of the subject and a second region that is an image region of the target part as specific regions that are not subject to abstraction processing; A function of performing abstraction processing on an image region other than the identified specific region; A program to make the above happen on a computer.