Information processing apparatus, information processing system, information processing method, and program

Abstract

An object of the present disclosure is to provide a service compliant with AI ethics and legal regulations in a service for registering feature information of a person appearing in an image.SOLUTION: An information processing apparatus according to the present disclosure includes an accepting unit configured to accept a first image including a person and a designation of a specific region in the first image from a user, and a registration unit configured to perform registration processing of registering feature information of a specific person included in the specific region among the persons included in the first image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a service for registering characteristic information of people appearing in images. [Background technology]

[0002] Services (hereafter referred to as facial recognition services) that accept image input from client services or applications and recognize the identity of the face in the input image are becoming widespread. The facial recognition service server outputs the facial recognition results based on the similarity between the features extracted from the face in the input image and the facial features of pre-registered people. Therefore, service users must register the facial features of the people to be recognized in the facial recognition service in advance.

[0003] Since facial image storage and facial feature extraction and registration involve handling personal information, such as a person's face, it is necessary to comply with AI (Artificial Intelligence) ethics and legal regulations. Patent Document 1 discloses the procedure for registering a facial image. In this procedure, on the initial screen where the user performs the facial image registration operation, an explanation regarding the handling of personal information in the system is first displayed and the user is asked to consent. Once the user's consent is obtained, a camera installed in the registration device captures the user's facial image, presents the captured facial image to the user, and, upon obtaining consent, transmits the facial image to a face management server. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-157932 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the method described in Patent Document 1, images of the user taken with a camera installed in a registration device are subject to registration processing, and consent is expected to be obtained from the user, who is the subject of the captured image. Therefore, arbitrary images, such as snapshots containing multiple faces, are not subject to processing. In recent years, services have been provided that manage various images taken by users on the cloud, and balancing diverse services with compliance with the above-mentioned AI ethics and legal regulations has become a challenge.

[0006] The purpose of this disclosure is to provide a service that registers characteristic information of people appearing in images, in accordance with AI ethics and legal regulations. [Means for solving the problem]

[0007] The information processing device disclosed herein includes a receiving means for receiving from a user a first image containing a person and a designation of a specific area within the first image, and a registration means for performing a registration process for registering characteristic information of a specific person included in the specific area among the people appearing in the first image, and consent to the registration of the characteristic information for the specific person is obtained from the user. [Effects of the Invention]

[0008] The information processing device of the present disclosure can provide a service that complies with AI ethics and legal regulations in a service for registering characteristic information of people appearing in images. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates an example of a system configuration of an information processing system. [Figure 2] 1A is a block diagram showing an example of the hardware configuration of an information processing device, and FIG. 1B is a block diagram showing an example of the hardware configuration of a client terminal. [Figure 3] FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing system according to the first embodiment. [Figure 4] 10 is a flowchart showing the flow of a feature registration process in the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of a person region detected from an image. [Figure 6] FIG. 10 is a diagram illustrating an example of a UI screen. [Figure 7] FIG. 4 is a diagram illustrating an example of a feature information table in the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a functional configuration of an information processing system according to a second embodiment. [Figure 9] 10 is a flowchart showing the flow of a feature registration process in the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a temporary storage table in the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of the functional configuration of an information processing system according to a third embodiment. [Figure 12] 11 is a flowchart showing the flow of a feature registration process in the third embodiment. [Figure 13] FIG. 11 is a diagram illustrating an example of a feature information table in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] The present invention will be described in detail below based on preferred embodiments thereof with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0011] First Embodiment The system configuration of an information processing system according to this embodiment will be described. Fig. 1 is a diagram showing an example of the system configuration of an information processing system 1 according to this embodiment. As shown in the figure, the information processing system 1 comprises an information processing device 100 and a client terminal 110 connected to each other so as to be able to communicate with each other via a network 120, forming a server-client system. The information processing device 100 functions as a server device of the information processing system 1, and the client terminal 110 functions as a client.

[0012] The information processing device 100, which is a server, may be implemented by a single computer or may include multiple computers. In the information processing system 1 shown in FIG. 1, the communication method used to connect the devices is, for example, the IEEE802.11 series communication standard (Wi-Fi (registered trademark)) or Bluetooth (registered trademark). Communication between the devices may be performed via Internet communication via a wireless LAN router, or the devices may communicate via mobile communication (3G, 4G, 5G). The client terminal 110 is an information processing terminal used by a user, and has a web browser function for browsing websites on the Internet and executing web applications provided by a server. The client terminal 110 may be implemented by, for example, a PC (personal computer), a smartphone, a tablet, or other information processing terminal, or a camera with a network communication function.

[0013] 2(a) is a diagram showing an example of the hardware configuration of the information processing device 100. The information processing device 100 is a computer, and includes a CPU 201, a RAM 202, a ROM 203, a network interface (I / F) 204, a storage device 205, a display device 206, and an input device 207. These components are connected to each other via a bus 209.

[0014] The CPU 201 controls the operation of each component of the information processing device 100 and is the main body that executes various processes (described later) performed by the information processing device 100. The RAM 202 is a memory that temporarily stores data and control information, and serves as a work area used by the CPU 101 when executing various processes. The ROM 203 stores operating parameters, operating programs, etc. that are fixedly used by the information processing device 100.

[0015] The network I / F 204 provides a function for connecting to and communicating with the network 120. The information processing device 100 transmits and receives data to and from external devices via the network I / F 204. The storage device 205 is a device for storing data and has an interface for receiving I / O commands for reading and writing data. The storage device 205 may be a hard disk drive (HDD), a solid state drive (SSD), an optical disk drive, a semiconductor storage device, or other storage device. The storage device 205 stores computer programs and data for causing the CPU 101 to execute each of the processes (described below) executed by the information processing device 100.

[0016] The display device 206 is, for example, an LCD (Liquid Crystal Display) or the like, and displays information output from the CPU 201 in a state that is visible to the user. The input device 207 includes, for example, a keyboard, a mouse, a touch panel, etc., and receives input of information according to user operations and inputs the information to the CPU 201.

[0017] FIG. 2(b) is a diagram illustrating an example of the hardware configuration of the client terminal 110. The client terminal 110 includes a CPU 211, a RAM 212, a ROM 213, a network I / F 214, a storage device 215, a display device 216, an input device 217, an imaging device 218, and the like. These components are connected to each other via a bus 219. In the example of FIG. 2(b), the client terminal 110 is shown as being configured as a smartphone with an imaging function, but the client terminal 110 is not limited to this configuration and may have other configurations. The CPU 211, RAM 212, ROM 213, network I / F 214, storage device 215, display device 216, and input device 217 are similar to the CPU 201, RAM 202, ROM 203, network I / F 204, storage device 205, display device 206, and input device 207 described above, and therefore description thereof will be omitted. The imaging device 218 has an imaging element such as a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS), and a lens, and inputs data of the captured image to the CPU 201.

[0018] In this embodiment, various functions provided by the information processing device 100 are provided to the client terminal 110 as web services. That is, the information processing device 100 provides a UI (user interface) screen via a web browser on the client terminal 110. The information processing device 100 displays various data on the client terminal 110 via the UI screen, and executes processing corresponding to the various functions provided by the information processing device 100 while accepting data input from the client terminal 110. Alternatively, the client terminal 110 may be equipped with a dedicated application. The client terminal 110 may execute the application to execute processing corresponding to the various functions with the information processing device 100 while transmitting and receiving data to and from the information processing device 100. The present disclosure is not limited to these embodiments, and the functions of the information processing device 100 may be realized by any method. Furthermore, various hardware components constituting the information processing device 100 may be virtual hardware resources on the cloud. In this case, the information processing device 100 transmits a function execution request to the corresponding hardware resource via the network I / F 204 and obtains the processing result via the network I / F 104.

[0019] Next, the functional configuration of the information processing system 1 according to this embodiment will be described. FIG. 3 is a diagram showing an example of the functional configuration of the information processing system 1 according to the first embodiment. As shown in FIG. 3, the information processing device 100 has a pre-processing unit 300, a receiving unit 305, and a registration unit 310. The pre-processing unit 300 has a pre-receiving unit 301, a pre-detection unit 302, and a response unit 303. The registration unit 310 has a detection unit 306, a determination unit 307, an extraction unit 308, and a storage unit 309. The client terminal 110 has a pre-transmission unit 311, a consent acquisition unit 312, and a transmission unit 313. The functions of each of these functional units are realized when a CPU calls a program stored in a ROM or a storage device and the CPU executes processing in accordance with the program.

[0020] The receiving unit 305 receives a first image containing a person and a designation of a specific area within the first image as a request to execute registration. The first image and the designation of the specific area within the first image are transmitted from the transmitting unit 313 of the client terminal 110. The first image is data of an arbitrarily captured image, and may be a still image or a video. The image may be captured using any imaging device. The first image includes one or more people as subjects. The specific area is an area including a specific person whose feature information is to be registered. The designation of the specific area is information that can identify the position and size of the specific area in the first image, and is expressed, for example, by the coordinate values ​​of two points within the first image. Specifically, if the shape of the specific area is rectangular on the image plane, the top left coordinate and bottom right coordinate of the rectangular area can be used as information related to the designation of the specific area. Note that the information designating the specific area is not limited to this and may be other information. For example, the information may be the top left coordinate of the area, and information indicating the size and shape of the area.

[0021] In the first embodiment, the receiving unit 305 receives a first image and a designation of a specific area within the first image after pre-processing by the pre-processing unit 300. The pre-processing unit 300 receives a second image corresponding to the first image before the receiving unit 305 receives the designation of the specific area. The pre-processing unit 300 detects one or more person areas from the second image and responds to the client terminal. The designation of the specific area within the first image received by the receiving unit 305 is made from among the person areas responded to by the pre-processing unit 300. It is assumed that the person area is detected as an area including a face. In the following description, the person area is also referred to as a face area.

[0022] The pre-reception unit 301 of the pre-processing unit 300 receives a second image from the pre-transmission unit 311 of the client terminal 110. In this embodiment, it is assumed that the second image is the same as the first image. However, the second image does not necessarily have to be completely identical to the first image. For example, the first image and the second image may differ due to a change in the color value of part of the image or image processing.

[0023] The advance detection unit 302 detects one or more person regions from the second image accepted by the advance acceptance unit 301. In the following description, one or more person regions detected from the second image are referred to as second person regions, and one or more person regions detected from the first image are referred to as first person regions. The response unit 303 responds (transmits) information about the second person regions detected by the advance detection unit 302 to the client terminal 110. The information about the second person regions transmitted to the client terminal 110 includes information specifying at least the position and size of the person region in the second image. For example, when the second person region is specified as the top left coordinate and the bottom right coordinate in the second image, the rectangular region whose diagonal is the line connecting the top left coordinate and the bottom right coordinate is the second person region in the second image.

[0024] The client terminal 110 receives information about the second person area from the information processing device 100 as a response to the pre-processing. The consent acquisition unit 312 of the client terminal 110 presents the second person area so that it can be visually recognized by the user. The client terminal 110 displays a screen showing the second person area acquired from the information processing device 100 on the display device 216 of the client terminal 110. For example, the consent acquisition unit 312 cuts out an area corresponding to the second person area from the second image transmitted to the information processing device 100 by the pre-transmission unit 311, and displays the person image enlarged or reduced to a predetermined size on the screen. The screen is displayed on the display device 216 of the client terminal 110. Examples of the screen will be described later.

[0025] The consent acquisition unit 312 inquires the user about the designation of one of the second person areas included in the response to the pre-processing and about consent to allow the information processing device 100 to register feature information of the person included in the designated area. This inquiry is made for each person. For example, on the above-mentioned screen, consent is sought for each person displayed on the screen. Note that the designation of the second person area may be performed by a user operation or may be performed automatically by processing of the client terminal 110. For example, if one second face area is obtained as the response to the pre-processing, the consent acquisition unit 312 may designate the one second person area and inquire about consent to register the person within the area from the user. Furthermore, if multiple second face areas are obtained as the response to the pre-processing, the consent acquisition unit 312 may designate each of the multiple second person areas one by one and display them on the screen, and, together with the display, inquire about consent to register the person within the designated area from the user. The following description will be given of an example in which an area is designated by a user operation.

[0026] When the consent acquisition unit 312 acquires consent from the user regarding the person included in the designated area, the transmission unit 313 of the client terminal 110 designates this area as a specific area and transmits designation information of the specific area together with the first image to the information processing device 100. As described above, the first image is usually the same image as the second image. Note that the transmission unit 313 may transmit information to the information processing device 100 indicating that consent has been acquired regarding the specific person included in the specific area, together with the designation information of the specific area and the first image.

[0027] Note that the user who responds to the consent inquiry does not necessarily have to be the person in the area. The user who responds to the consent inquiry is a user of the face registration service, and in addition to the person, it is assumed that the user may also be the photographer or owner of the first image (second image), an assistant who assists with registration, etc. Specifically, it may be a family member or friend of the person in the area, or a person who has received permission from the person in the area. Note that if consent from the user is not obtained, the transmission unit 313 of the client terminal 110 will not transmit the first image and the designation of the specific area.

[0028] The registration unit 310 of the information processing device 100 registers feature information of a specific person included in an area that matches a designated specific area among first person areas, which are one or more person areas included in the first image accepted by the acceptance unit 305. In this specification, "match" means that the positions, sizes, and shapes of the compared areas match or are similar. Similarity means that the overlapping degree value, which will be described later, is equal to or greater than a predetermined value.

[0029] The registration unit 310 first detects one or more person regions (first person regions) included in the first image using the detection unit 306. For example, a region including a person's face is detected. Next, the determination unit 307 determines whether or not the first person region detected by the detection unit 306 includes a region that matches a region (specific region) related to the specification accepted by the acceptance unit 305. If a matching region is found, the matching region is identified as the specific region. The extraction unit 308 extracts person features, here facial features, from the specific region. The match determination is performed, for example, based on the degree of overlap of the regions. The degree of overlap will be described later. The storage unit 309 stores the facial features extracted from the specific region in the storage device 205 as feature information of the specific person.

[0030] Next, a feature registration process executed by the information processing system 1 in the first embodiment will be described. FIG. 4 is a flowchart showing the flow of the feature registration process in the first embodiment, and registration of a person's facial feature will be described as an example. The processes shown in S402 to S404 and S409 to S413 of this flowchart are written in a Web application program stored in the ROM 203 or storage device 205 of the information processing device 100. The program is called by the CPU 201, loaded into the RAM 202, and executed by the CPU 201. Furthermore, the processes shown in S401 and S405 to S408 of this flowchart are written in a Web application program loaded into the RAM 312 of the client terminal 110, and executed by the CPU 211 of the client terminal 110.

[0031] The client terminal 110 accesses a website provided by the information processing device 100 via a web browser. When the login process is completed, the information processing device 100 transmits a top screen including a menu list to the client terminal 110. When the face registration menu is selected from the client terminal 110 on the top screen, the CPU 201 of the information processing device 100 starts the process of this flowchart. In the following description, the symbol "S" represents a step. In the feature registration process of FIG. 4, of the processes executed by the information processing device 100, S401 to S404 are referred to as pre-processing, and S409 to S413 are referred to as registration processing.

[0032] In S401, the CPU 211 of the client terminal 110 transmits an image to the information processing device 100 as a request to execute pre-processing for face registration. The image transmitted here is the second image described above. The second image is an image stored in the storage device 215 of the client terminal 110 or an image captured by the imaging device 218 of the client terminal 110. The second image may be a still image or a moving image. The user selects the second image and transmits it to the information processing device 100. Note that when the face registration menu is selected, the information processing device 100 may display a dialog screen on the client terminal for selecting the second image.

[0033] In S402, the CPU 201 of the information processing device 100 accepts the second image transmitted from the client terminal 110 as a request to execute pre-processing for face registration.

[0034] In S403, the CPU 201 detects a person's face area from the image received in S402. FIG. 5 is a diagram showing an example of a face area detected from a second image. Two people 501 and 502 appear in the second image 500, and face areas 503 and 504 are detected for these people 501 and 502, respectively. The CPU 201 acquires coordinate information 506 in the image coordinate system of the second image 500 for each of the face areas 503 and 504. The image coordinate system is a two-dimensional coordinate system having coordinate axes in two orthogonal directions. In the example of FIG. 5, the upper left corner of the image is the origin (0,0), the horizontal direction of the figure is the X axis, and the vertical direction is the Y axis. The positions and sizes of the face areas 503 and 504 are specified as rectangular areas specified, for example, by the upper left coordinate and the lower right coordinate of the area. In the example of coordinate information 506, a face area 503 with upper left coordinates (206, 205) and lower right coordinates (684, 416) is detected as area ID "01." Also, a face area 504 with upper left coordinates (1176, 310) and lower right coordinates (1458, 483) is detected as area ID "02."

[0035] In this embodiment, the CPU 201 uses a known technique such as an inference device trained by deep learning to detect an area estimated to be a face from an input image (second image), and outputs image information of the detected face area. Furthermore, if the input image is a video, an area estimated to be a face may be detected for a representative frame in the video using the inference device, as in the case of a still image. Note that the method for detecting a face area is not limited to a technique using deep learning, and any other method may be used.

[0036] In S404, the CPU 201 of the information processing device 100 responds with information about the face area detected in S403 to the client terminal 110. That is, the coordinate information 506 in FIG. 5 is transmitted to the client terminal 110. At this stage, the second image is not stored in the information processing device 100. This is to reduce data storage costs and comply with legal regulations.

[0037] In S405, the CPU 211 of the client terminal 110 displays, on the display device 216 of the client terminal 110, facial images corresponding to the facial areas 503 and 504 received as a response from the information processing device 100. FIG. 6 is a diagram showing an example of a screen displayed on the client terminal 110. First, in S405, the CPU 211 of the client terminal 110 displays, on the display device 216, a face selection screen 600 shown in FIG. 6(a). In FIG. 6(a), facial images 601 and 602 corresponding to the facial areas 503 and 504 detected in the preliminary detection process of S403 are displayed side by side. Based on the coordinate information of the facial areas 503 and 504 received as a response from the information processing device 100, the CPU 211 cuts out the facial areas from the second image transmitted in the preliminary process and displays them on the screen. Note that the facial images 601 and 602 do not necessarily need to be displayed in the same size as the second facial area in the second image. Each face may be enlarged or reduced to an appropriate size so that the user can view it on the face selection screen 600. In addition, although the example of Fig. 6(a) shows an example in which a face image in which the face area is cut out from the second image is displayed, the present invention is not limited to this, and the face area included in the response may be indicated by displaying a frame of an appropriate size at an appropriate position on the second image.

[0038] In S406, when the user designates an image of any face area and operates the OK button 604, the CPU 211 advances the process to S407 and transitions to the consent screen 610 in Fig. 6(b). If the cancel button 603 is operated, the process ends this flowchart or returns to face area detection (S403).

[0039] In S407, the CPU 211 displays the consent screen 610 of FIG. 6(b) on the display device 216. Then, the CPU 211 obtains consent from the user for the face included in the specific face area specified by the user in S405. The consent screen 610 of FIG. 6(b) displays the face image 601 specified in FIG. 6(a) and a message 611 requesting consent. The content of the message 611 may be, for example, "I will register this face. Do you agree that facial features will be extracted and stored in the system?" As shown in the example, the message 611 includes a sentence requesting consent for the information processing device 100 to extract and register facial features. The consent screen 610 also has a "Yes" button 612 to be pressed if the user consents and a "No" button 613 to be pressed if the user does not consent. When the user operates the "Yes" button 612, the CPU 211 proceeds to S407 and transitions to the confirmation screen 620 of FIG. 6(c). If the "No" button 613 is operated, the process ends this flowchart or returns to face area detection (S403).

[0040] In addition to the information displayed on the consent screen 610 shown in FIG. 6(b), the confirmation screen 620 of FIG. 6(c) displays a link 626 to the terms of use and a check box 623. The confirmation screen 620 also includes a "Next" button 624 and a "Back" button 625. When the user operates the link 626 to the terms of use, the screen transitions to a page of the terms of use for the face recognition service. When the check box 623 is pressed once, a check mark is displayed, and when the check box 623 is pressed again, the displayed check mark is cleared. In S407, when the user operates the "Next" button 624 with the check mark displayed in the check box 623, the CPU 211 stores the coordinate information of the face region corresponding to the specified face image 601 in the RAM 212 as consent acquisition completion. The CPU 211 then closes the confirmation screen 620 of FIG. 6(c) and proceeds to S408. When the "Back" button 625 is operated, the screen returns to the face selection screen 600 shown in FIG. 6(a).

[0041] In the example of FIG. 6(a), face images 601 and 602 corresponding to multiple face regions detected from the second image are displayed side by side, and the user selects one of the images of the face regions to transition to the next consent screen 610. However, other screen configurations may be used. The consent screens 610, each showing one face image corresponding to the detected multiple face regions, may be displayed sequentially. For example, when two face regions 503 and 504 are detected as shown in FIG. 5, the consent screen 610 of FIG. 6(b) first asks for consent for the face image 601 corresponding to the face region 503. When the user operates the "Yes" button 612, the screen transitions to the confirmation screen 620 of FIG. 6(c). When the "No" button 613 is operated, the confirmation screen 620 of FIG. 6(c) is skipped, and the consent screen 610 showing the face image 602 corresponding to the next face region 504 is displayed.

[0042] If the "Next" button 624 is operated with a check mark entered in the check box 623 on the confirmation screen 620, the CPU 211 stores the coordinate information of the face area corresponding to the face image in the RAM 212, indicating that consent has been obtained. Then, the CPU 211 displays the consent screen 610 displaying the face image 602 corresponding to the next face area 504. If the "Next" button 624 is operated with no check mark entered in the check box 623 on the confirmation screen 620, the CPU 211 determines that consent has not been obtained and displays the consent screen 610 displaying the face image 602 corresponding to the next face area 504. Once consent inquiries and answers have been obtained for all face areas included in the response, the process proceeds to S408.

[0043] The display format of the screens 600, 610, and 620 may be any display format as long as the facial image to be consented to is displayed so that the user can visually recognize it. For example, a frame indicating the detected facial area may be displayed on the second image. Furthermore, the selection of the face to be consented to from the facial area may be accepted by selecting one of these frames. Furthermore, on the face selection screen 600, a check mark may be displayed on the face to be consented to by a user's designation operation, allowing consent to be given collectively to all faces with a check mark. Furthermore, confirmation items required for consent other than the terms of use may be added to the screen.

[0044] In S407, if consent is not obtained for any of the face regions, the process from S408 onward is not executed and this flowchart ends. In S407, if consent is obtained for any of the face regions, the process proceeds to S408.

[0045] In S408, the CPU 211 of the client terminal 110 transmits the first image and designation information of the face area for which consent has been obtained from the user to the information processing device 100 as a request to execute face registration. The first image transmitted here is the same image as the second image transmitted in advance in S401. The designation information of the face area is coordinate information of the face area stored in the RAM 212 as having consent obtained.

[0046] In S409, the CPU 201 of the information processing device 100 receives, as a request to execute face registration, the first image and the face area for which consent has been obtained from the user, that is, designation information of the specific area.

[0047] In S410, the CPU 201 of the information processing device 100 detects one or more face areas from the first image accepted in S409. The one or more face areas detected from the first image by the information processing device 100 after accepting the first image and the specific area are also referred to as first face areas.

[0048] In S411, the CPU 201 of the information processing device 100 determines whether any of the first face regions detected in S410 matches the region designated in S409. In the determination process, the CPU 201 calculates the degree of overlap between each of the first face regions detected by the CPU 201 in S410 and the region designated in S409. The CPU 201 determines whether the regions match based on the value of the degree of overlap. The degree of overlap is an index representing the proportion of overlap between images, and an evaluation index called IoU (Intersection over Union) can be used, for example. The larger the IoU, the more the images overlap. The CPU 201 considers a first face region with the largest degree of overlap value and greater than a predetermined threshold to be a region that "matches" the designated region. Otherwise, the first face region is determined to "not match" the designated region.

[0049] The method for calculating the degree of overlap will now be described. The CPU 201 calculates the degree of overlap as the ratio of the "AND area between the target face area and the area related to the designation" to the "OR area between the target face area, which is any one of the detected first face areas, and the area related to the designation received from the client terminal 110." The CPU 201 calculates the degree of overlap for each of all first face areas detected from the first image. Then, as described above, the first face area with the largest overlap degree value and larger than a predetermined threshold value is determined to be the identified area. The degree of overlap is expressed by the following formula 1.

[0050] Overlap degree=(AND region of target face region and designated region) / (OR region of target face region and designated region) (1)

[0051] Furthermore, if there are multiple target face regions with the highest degree of overlap, the region with the highest likelihood of the target face region is prioritized and deemed to be the "matching" region (specific region). The likelihood is a score calculated during the face region detection process in S410. The higher the likelihood value, the higher the reliability that the region is a face; in other words, the higher the likelihood that the detected region is a "face." If there are multiple face regions with the same degree of overlap in terms of position, size, and shape, the region that is most reliable as a face is prioritized based on the likelihood.

[0052] In S411, if it is determined that there is no area in the detected first face area that matches the area related to the designation (S411; NO), this flowchart ends and the feature registration process ends. On the other hand, if it is determined that there is an area in the detected first face area that matches the area related to the designation (S411; YES), the process proceeds to S412.

[0053] In S412, the CPU 201 extracts facial features from a region (specific region) in the first image that matches the specified region. In this embodiment, the CPU 201 extracts the facial features as an N-dimensional vector using an inference model that has been trained in advance using a known technique such as deep learning. Note that the process of extracting facial features is not limited to this method, and any method may be used as long as it is capable of extracting features.

[0054] In S413, the CPU 201 stores the facial features extracted in S412 as feature information in the storage device 205. FIG. 7 is a diagram showing an example of a feature table 700 stored in the storage device 205. As shown in the figure, the feature table 700 accumulates and stores records each including an ID column 701, a feature column 702, a label information column 703, and a registration date and time column 704. The ID column 701 stores an ID that uniquely identifies a registered face. The feature column 702 stores facial features corresponding to the facial area identified in the ID column 701. The label information column 703 stores additional information for identifying the facial area. For example, a person's name is stored. The registration date and time column 704 stores the date and time when the facial features were registered. Note that the information stored in the feature table 700 is an example and is not limited to this.

[0055] As described above, when an image is transmitted from the client terminal 110, the information processing device 100 of the first embodiment responds by presenting the facial areas detected from the image to the client terminal 110 so that the user can visually recognize them. The client terminal 110 individually obtains consent for the registration of facial features in the face recognition service for each of the facial areas from the user. For each facial area for which consent has been obtained from the user, the client terminal 110 transmits a designation of the facial area and the image to the information processing device 100. The information processing device 100 detects the facial area from the received image, identifies an area from the detected facial area that matches the designation, extracts and registers facial feature information. This enables the information processing device 100 to register feature information for a specific person appearing in any image. Furthermore, even if an image contains multiple faces, it is possible to register facial feature information for a specific face to which the user has consented. Therefore, a service that registers feature information for people appearing in an image can be provided that complies with AI ethics and legal regulations.

[0056] By performing the feature registration process according to the above procedure, it is possible to safely register facial feature information, which is personal information, even when transmitting and receiving data over a network. In other words, since facial feature information is not directly transmitted and received between the client terminal 110 and the information processing device 100, the feature information is not leaked during data transmission and reception. Furthermore, since the information processing device 100 does not need to store facial images, data retention costs can be reduced and legal regulations can be met. Furthermore, there is a time lag between the reception of an image in the pre-processing and the reception of an image in the registration process. If the image is altered or changed to a different image during that time, it is highly likely that the area matching determination will not match, and the continuation of the process will be prevented.

[0057] In the first embodiment, a method for detecting a face region using a known technique, such as an inference device trained by deep learning, to detect a region estimated to be a face has been described. However, the present disclosure is not limited to this method. For example, face region detection may be achieved using an algorithm other than an inference model. Furthermore, in the above embodiment, a method for extracting facial features using an inference model trained in advance using a known technique, such as deep learning, to extract facial features as an N-dimensional vector has been described. However, the present disclosure is not limited to this method. For example, face feature extraction may be achieved using an algorithm other than an inference model. Furthermore, facial features may be extracted as information other than a vector. Furthermore, in the present embodiment, a method for calculating the degree of overlap has been described in which the ratio of the "AND region between the target face region and the designated region" to the "OR region between the target face region and the designated region" has been described. However, this calculation method is not limited to this. For example, the area of ​​the "AND region between the target face region and the designated region" may be calculated as the degree of overlap. Furthermore, the value of the "AND region between the target face region and the designated region" relative to the "target face region" may be calculated as the degree of overlap. Furthermore, the screen configurations and processing procedures in the flowcharts are merely examples, and the present disclosure is not limited to the above examples.

[0058] Second Embodiment In the first embodiment, a procedure for detecting a person area twice in a series of feature registration processes from pre-processing to registration processing is exemplified. In the second embodiment, a processing procedure for detecting a person area once will be described. In the second embodiment, as in the first embodiment, the information processing device 100 presents faces to be registered to the user, and registers facial feature information in a face recognition service after obtaining consent for each face individually. Note that the system configuration and hardware configuration of the information processing system of the second embodiment are the same as those of the first embodiment, and therefore description thereof will be omitted, and the same components will be described with the same reference numerals. Also, the following description will focus on differences from the first embodiment.

[0059] The functional configuration of an information processing device 100A according to the second embodiment will be described. Fig. 8 is a block diagram showing the functional configuration of the information processing device 100A according to the second embodiment. In Fig. 8, the same components as those in the functional configuration of the first embodiment are assigned the same reference numerals as those in the first embodiment. The functions of the functional components shown below are realized by the CPU calling up a program stored in a ROM or storage device and the CPU executing processing in accordance with the program.

[0060] The information processing device 100A of the second embodiment includes a pre-processing unit 800, a hash value determination unit 801, a reception unit 305, and a registration unit 810. The client terminal 110 includes a pre-transmission unit 311, a consent acquisition unit 312, and a transmission unit 313, similar to the first embodiment.

[0061] The pre-processing unit 800 includes a pre-accepting unit 301, a pre-detecting unit 302, a temporary storage unit 802, and a responding unit 303. The registration unit 810 includes a reading unit 803, a determining unit 307, an extracting unit 308, and a storage unit 309. The hash value determining unit 801 is used by both the pre-processing unit 800 and the registration unit 810.

[0062] The hash value determination unit 801 calculates a hash value for an image. The hash value determination unit 801 calculates a hash value for an image (second image) accepted by the pre-acceptance unit 301. The hash value determination unit 801 also calculates a hash value for an image (first image) accepted by the acceptance unit 305. The hash value determination unit 801 calculates a hash value for an image using the same algorithm when executed in the pre-processing unit 800 and when executed in the registration unit 810. In this embodiment, the hash value is calculated by calculation using, for example, the SHA-256 algorithm, but this is not limiting and any other algorithm can be used. If the first image and the second image are the same image, the same hash value is obtained; if not, different hash values ​​are obtained.

[0063] The temporary storage unit 802 stores the hash value calculated for the second image by the pre-processing unit 800 in association with one or more person regions detected from the second image by the pre-detection unit 302. Hereinafter, a person region detected from the second image is referred to as a person's face region. One or more face regions detected from the second image are referred to as second face regions.

[0064] The response unit 303 transmits to the client terminal 110 information about the second face region detected from the second image by the pre-detection unit 302.

[0065] In the registration unit 810, the hash value determination unit 801 calculates a hash value for the image (first image) accepted by the acceptance unit 305 and passes it to the reading unit 803. The reading unit 803 queries (reads) the second face area from the temporary storage unit 802 using the hash value calculated for the image accepted by the acceptance unit 305. The determination unit 307 determines whether or not the second face area read by the reading unit 803 includes an area that matches the area related to the specification accepted by the acceptance unit 305. The determination of the match is the same as in the first embodiment. If a matching area is found, the extraction unit 308 extracts facial features from the specific area. The storage unit 309 stores the facial features extracted from the specific area in the storage device 205.

[0066] Next, a feature registration process executed by the information processing system 1 in the second embodiment will be described. FIG. 4 is a flowchart showing the flow of the feature registration process in the second embodiment. The processes shown at S402, S403, S901, S902, S404, S409, S903 to S905, S412, and S413 in this flowchart are written in a Web application program stored in the ROM 203 or storage device 205 of the information processing device 100A. The program is called by the CPU 201, loaded into the RAM 202, and executed by the CPU 201. The processes shown at S401 and S405 to S408 in this flowchart are written in a Web application program loaded into the RAM 312 of the client terminal 110, and executed by the CPU 211 of the client terminal 110. In FIG. 9, the same processes as those in the feature registration process in the first embodiment are denoted by the same reference numerals. The following description will focus on the processes that differ from the first embodiment.

[0067] The processing of S401 to S403 is the same as in the first embodiment. That is, the CPU 211 of the client terminal 110 transmits an image (second image) to the information processing device 100A as a request to execute pre-processing for face registration. The CPU 201 of the information processing device 100A accepts the second image transmitted from the client terminal 110 and detects a person's facial area. Next, the process proceeds to S901.

[0068] In S901, the CPU 201 of the information processing device 100A calculates a hash value of the second image received in S402. In S902, the CPU 201 of the information processing device 100A associates the hash value calculated in S901 with one or more face areas (second face areas) detected in S403, and stores them in the temporary storage unit 802.

[0069] FIG. 10 is a diagram showing an example of a hash value table 1000 stored in the temporary storage unit 802. As shown in the figure, the hash value table 1000 stores records including an image hash value column 1001, a face area column 1002, a detection date and time column 1003, and an automatic deletion date and time column 1004. Note that one record targets one and the same image. The image hash value column 1001 stores a hash value calculated from the image. The face area column 1002 stores a list of one or more face areas detected from the image. The detection date and time column 1003 stores the date and time when face area detection was performed for the image. The automatic deletion date and time column 1004 stores the date and time when the record in question will be automatically deleted. Note that the information stored in the hash value table 1000 is an example and is not limited to this.

[0070] The first record in Figure 10 stores the image hash value "98abf72408..." in association with the area [{(260,205),(684,416)},{(1176,310),(1458,483)}] detected from the image. Two areas were detected: the coordinate range indicated by {(260,205),(684,416)} and the coordinate range indicated by {(1176,310),(1458,483)}.

[0071] Thereafter, in S404, the CPU 201 of the information processing device 100A transmits the face area detected in S403 to the client terminal 110 as a response.

[0072] The client terminal 110 executes the processes of S405 to S408 in the same manner as in the first embodiment. That is, the CPU 211 of the client terminal 110 displays the face area received as a response on the display device 216 of the client terminal 110. The CPU 211 designates a specific face area and obtains consent from the user to register feature information for that face area.

[0073] The CPU 211 of the client terminal 110 transmits the first image and designation information of the area for which consent has been obtained from the user to the information processing device 100A as a request to execute face registration. The first image transmitted here is the same as the second image transmitted in S401. However, the first image and the second image do not need to be completely identical.

[0074] In S409, the CPU 201 of the information processing device 100A receives, as a request to execute face registration, the first image and information specifying the face area for which consent has been obtained. Next, the process proceeds to S903.

[0075] In S903, the CPU 201 of the information processing apparatus 100A calculates the hash value of the first image received in S409 using the same algorithm as in S901.

[0076] In S904, the CPU 201 uses the hash value calculated in S903 to query the information on the face area stored in the temporary storage unit 802. That is, based on the hash value calculated in S903, the CPU 201 reads the information on the face area stored in association with the hash value from the hash value table 1000.

[0077] In S905, the CPU 201 determines whether any of the face regions queried in S904 matches the region designated in S409. In this determination process, the CPU 201 calculates the degree of overlap between each of the one or more face regions queried in S904 and the region designated in S409. The CPU 201 determines whether the regions match based on the overlap value. That is, the CPU 201 determines the first face region with the largest overlap value and greater than a predetermined threshold as a region that "matches" the designated region. Otherwise, the CPU 201 determines the first face region as a region that "does not match" the designated region. Furthermore, if there are multiple first face regions with the largest overlap, the region with the highest likelihood of being the first face region is prioritized and deemed to be a "matching" region. The overlap degree and likelihood are defined in the same way as in the first embodiment.

[0078] If it is determined in S411 that the queried second face region does not include an area that matches the designated area, this flowchart ends, and the feature registration process ends. On the other hand, if it is determined in S411 that the queried second face region includes an area that matches the designated area, the process proceeds to S412.

[0079] In S412, the CPU 201 extracts facial features from the area (specific area) that matches the designated area.

[0080] In S413, the CPU 201 stores the facial feature amounts extracted in S412 as feature information in the storage device 205. The method for extracting facial feature amounts in S412 and the method for storing feature information in S413 are the same as those in the first embodiment. After that, this flowchart ends.

[0081] As described above, the information processing device 100A of the second embodiment can register feature information about a specific person appearing in an arbitrary image by detecting a face area once. Furthermore, even if an image contains multiple faces, it can register facial feature information about a specific face that the user has consented to.

[0082] In the second embodiment, the SHA-256 algorithm is used to calculate the hash value of an image, but the present disclosure is not limited to this. For example, other hashing algorithms such as SHA-512, MD5, Perceptual Hash, and Average Hash may also be used.

[0083] <Third embodiment> In the first embodiment, an example was described in which person region detection was performed twice during a series of processes from pre-processing to registration processing. In the second embodiment, both the image used in pre-processing and the image used in registration processing were hashed using the same algorithm, and the hash value was used to query the person region detected in pre-processing, thereby limiting the number of person region detections to one, thereby registering feature information. Next, as a third embodiment, another example of a processing procedure in which person region detection is performed only once will be described. Similar to the first embodiment, the information processing device 100B of the third embodiment presents the faces of people to be registered to the user, and registers the person's feature information in the person recognition service after obtaining individual consent for each person. The system configuration and hardware configuration of the information processing system of the second embodiment are similar to those of the first embodiment, and therefore will not be described again, and the same components will be denoted by the same reference numerals. The following description will focus on differences from the first embodiment.

[0084] The functional configuration of an information processing device 100B according to the third embodiment will be described. Fig. 11 is a block diagram showing the functional configuration of the information processing device 100B according to the third embodiment. In Fig. 11, the same components as those in the functional configuration of the first embodiment are assigned the same reference numerals as those in the first embodiment. The functions of the functional components shown below are realized by the CPU calling up a program stored in a ROM or storage device and the CPU executing processing in accordance with the program.

[0085] The information processing device 100B of the third embodiment includes a pre-processing unit 1100, a receiving unit 1103, and a registration unit 1110. The client terminal 110 includes a pre-transmission unit 311, a consent acquisition unit 312, and a transmission unit 1111.

[0086] The pre-processing unit 1100 includes a pre-acceptance unit 301, a pre-detection unit 302, an extraction unit 1101, an encryption unit 1102, and a response unit 303. The registration unit 1110 includes a decryption unit 1104 and a storage unit 309.

[0087] Unlike the first embodiment, the pre-processing unit 1100 includes an extraction unit 1101 and an encryption unit 1102. That is, in the third embodiment, feature information is extracted in the pre-processing. The extraction unit 1101 extracts facial features for each of one or more face areas (second face areas) detected by the pre-detection unit 302 from the image (second image) accepted by the pre-acceptance unit 301. The encryption unit 1102 encrypts each of the facial features extracted by the extraction unit 1101.

[0088] The response unit 303 responds to the client terminal 110 with one or more second face areas detected by the pre-detection unit 302 and facial feature amounts corresponding to the second face areas each encrypted by the encryption unit 1102.

[0089] The client terminal 110 receives information on the second face region and the encrypted feature amounts of each face region from the information processing device 100B. The advance transmission unit 311 and the consent acquisition unit 312 of the client terminal 110 are the same as those in the first embodiment. In the third embodiment, the transmission unit 1111 transmits the encrypted face feature amounts to the information processing device 100B as a request to execute face registration. The transmission unit 1111 transmits the encrypted feature amounts linked to the face region for which consent has been obtained from the user to the information processing device 100B.

[0090] The reception unit 1103 of the information processing device 100B receives encrypted facial features as a request to execute face registration from the client terminal 110. The decryption unit 1104 decrypts the encrypted facial features received by the reception unit 1103. The storage unit 309 stores the facial features decrypted by the decryption unit 1104 in the storage device 205.

[0091] Next, a feature registration process executed by the information processing system 1 in the third embodiment will be described. FIG. 12 is a flowchart showing the flow of the feature registration process in the third embodiment. The processes shown in S402, S403, S1201 to S1203, S1205 to S1206, and S413 of this flowchart are written in a Web application program stored in the ROM 203 or storage device 205 of the information processing device 100B. The program is called by the CPU 201, loaded into the RAM 202, and executed by the CPU 201. The processes shown in S401, S405 to S407, and S1204 of this flowchart are written in a Web application program loaded in the RAM 312 of the client terminal 110, and executed by the CPU 211 of the client terminal 110. In FIG. 12, the same processes as those in the feature registration process in the first embodiment are denoted by the same reference numerals. The following description will focus on the processes that differ from the first embodiment.

[0092] The processing of S401 to S403 is the same as in the first embodiment. That is, the CPU 211 of the client terminal 110 transmits an image (second image) to the information processing device 100B as a request to execute pre-processing for face registration. The CPU 201 of the information processing device 100B accepts the second image transmitted from the client terminal 110 and detects the face area of ​​the person. Next, the process proceeds to S1201.

[0093] In S1201, the CPU 201 of the information processing device 100B extracts facial feature amounts for each of one or more facial regions detected in S403. The method for extracting facial feature amounts is the same as in the first embodiment.

[0094] In S1202, the CPU 201 encrypts each of the one or more facial feature amounts extracted in S1201. In this embodiment, the encryption method is to encrypt the feature amounts using the AES-256 algorithm.

[0095] In S1203, the CPU 201 associates the face area detected in S403 with the face feature amount encrypted in S1202, and sends a response to the client terminal 110.

[0096] The client terminal 110 executes the processes of S405 to S407 as in the first embodiment. That is, the CPU 211 of the client terminal 110 displays the face area received as a response on the display device 216 of the client terminal 110. The CPU 211 designates a specific face area and obtains consent from the user to register feature information for that face area. Next, the process proceeds to S1204.

[0097] In S1204, the CPU 211 of the client terminal 110 transmits the encrypted feature amount associated with the face area for which consent has been obtained from the user to the information processing device 100B as a request to execute face registration.

[0098] In S1205, the CPU 201 of the information processing device 100B receives the encrypted facial feature amount from the client terminal 110 as a request to execute face registration.

[0099] In S1206, the CPU 201 decrypts the encrypted facial feature amount received in S1205 using a decryption algorithm corresponding to the encryption algorithm and key used in S1202.

[0100] In S413, the CPU 201 stores the facial feature amount decoded in S1206 as feature information in the storage device 205. After that, this flowchart ends.

[0101] As described above, the information processing device 100B of the third embodiment can register feature information about a specific person appearing in an arbitrary image by detecting a face area once. Furthermore, even if an image contains multiple faces, it can register feature information about a specific face that the user has consented to.

[0102] Furthermore, since the information processing device 100B does not need to store images and facial feature information is encrypted before transmission and reception, data storage costs can be reduced, compliance with legal regulations can be achieved, and services can be provided without information leaks. Furthermore, in the third embodiment, an image only needs to be transmitted once from the client terminal 110 to the information processing device 100B, so the communication load can be reduced compared to the first and second embodiments.

[0103] In this embodiment, an example has been described in which the AES-256 algorithm is used to encrypt features, but the present disclosure is not limited to this. For example, other encryption algorithms such as RSA and ECC may also be used.

[0104] Although preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. For example, in the above-described embodiments, an information processing device functions as a server and performs feature registration processing while transmitting and receiving data to and from a client terminal. However, the present disclosure is not limited to this. An information processing device including server functions and client functions may perform the feature registration processing of each of the above-described embodiments. Furthermore, while feature information is registered for a person's face, the feature information is not limited to facial features, and features of other parts of the body other than the face may be extracted and registered. It is clear that a person skilled in the art could conceive of various modifications or alterations within the scope of the disclosed technical ideas, and it is understood that these naturally fall within the technical scope of the present disclosure.

[0105] <Other embodiments> The objectives of the present disclosure can also be achieved by providing a system or device with a storage medium storing a program that realizes the functions of the above-described embodiments, and having the computer of the system or device read and execute the program stored on the storage medium. In this case, the program itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program constitutes the present disclosure. Examples of storage media for providing the program include hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and DVDs. The functions of the above-described embodiments are not necessarily realized by a computer executing the read program. The functions of the above-described embodiments may also be realized by an operating system (OS) running on the computer performing some or all of the actual processing based on the program's instructions. Furthermore, the functions of the above-described embodiments may also be realized by the program being read from the storage medium and written to the memory of a function expansion unit connected to the computer, and the CPU of the function expansion unit performing some or all of the actual processing.

[0106] The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0107] The disclosure of the above-described embodiment includes the following configurations.

[0108] (Configuration 1) a receiving means for receiving from a user a first image including a person and a designation of a specific area within the first image; a registration means for performing a registration process to register characteristic information of a specific person included in the specific area among people appearing in the first image, An information processing device, wherein consent to the registration of the characteristic information of the specific person has been obtained from the user.

[0109] (Configuration 2) the accepting means accepts a second image corresponding to the first image before accepting the designation of the specific region; The image processing device further includes a pre-processing unit that detects a person area from the received second image and responds to the user, The specific area is designated from the person area detected by the pre-processing means. 2. The information processing device according to configuration 1,

[0110] (Configuration 3) 3. The information processing device according to configuration 2, wherein the second image is the same as the first image.

[0111] (Configuration 4) The information processing device according to any one of configurations 1 to 3, wherein the registration process includes a process of detecting a person area from the first image after the receiving means receives the first image and the specification, and determining whether the detected person area matches the specific area.

[0112] (Configuration 5) The information processing device described in configuration 4 is characterized in that, when the person area detected from the first image in the registration process matches the specific area, the feature information of the person included in the specific area is extracted.

[0113] (Configuration 6) the accepting means accepts the second image corresponding to the first image before accepting the designation; The pre-processing means further associates the person detected from the second image with a second hash value determined from the second image and stores the associated information in a storage unit; The registration process includes: a process of determining a first hash value from the first image using the same algorithm as that used to determine the second hash value, and acquiring a person area detected from the second image using the first hash value from the storage unit; a process of determining whether the detected person area matches the specific area; 4. The information processing device according to configuration 2 or 3, comprising:

[0114] (Configuration 7) The information processing device according to configuration 6, characterized in that when the person area detected from the first image in the registration process matches the specific area, the feature information of the person included in the specific area is extracted.

[0115] (Configuration 8) the receiving means receives the first image and the specification from a client terminal connected to the information processing device for communication; The information processing device according to any one of configurations 2, 3, 6, and 7, wherein the pre-processing means receives the second image from the client terminal and responds to the client terminal with a person area detected from the second image.

[0116] (Configuration 9) 9. The information processing device according to configuration 8, wherein the pre-processing means causes the client terminal to display a screen showing a person included in a person area detected from the second image.

[0117] (Configuration 10) a pre-processing means for receiving an image from a user and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; a receiving means for receiving the feature information associated with a specific area including a specific person among the person areas included in the response; a registration means for decrypting and registering the characteristic information received by the reception means, An information processing device, wherein consent to the registration of the characteristic information of the specific person has been obtained from the user.

[0118] (Configuration 11) the pre-processing means receives the image from a client terminal connected to the information processing device for communication, and transmits the response to the client terminal; 11. The information processing device according to configuration 10, wherein the accepting means accepts the feature information linked to the specific area designated on the client terminal.

[0119] (Configuration 12) 12. The information processing apparatus according to configuration 11, wherein the pre-processing means causes the client terminal to display a screen showing a person included in the person area detected from the image.

[0120] (Configuration 13) The information processing device according to configuration 9 or 12, wherein the screen includes, for each person displayed on the screen, an inquiry to the user regarding consent to the registration means registering the characteristic information.

[0121] (Configuration 14) 14. The information processing apparatus according to configuration 13, wherein the person who is the subject of the inquiry is displayed on the screen in a changing order.

[0122] (Configuration 15) 14. The information processing apparatus according to configuration 13, wherein an operation of specifying the person to be the subject of the inquiry is accepted from the user on the screen.

[0123] (Configuration 16) the specific region includes a face region, 16. The information processing device according to any one of configurations 1 to 15, wherein the feature information is a facial feature amount.

[0124] (Configuration 17) An information processing device that can be connected to a client terminal for communication, a receiving means for receiving an image transmitted from the client terminal; a response means for responding to information about the person detected from the image so that the user of the client terminal can visually recognize the information; a registration means for registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user; An information processing device comprising:

[0125] (Configuration 18) An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a receiving means for receiving, from the client terminal, a first image in which a person appears and a designation of a specific area within the first image; a registration means for performing a registration process to register characteristic information of a specific person included in the specific area among people appearing in the first image, The client terminal An information processing system characterized by comprising a transmission means for transmitting the first image and the designation of the specific area containing the specific person in the first image to the information processing device when consent to the registration of the characteristic information for the specific person is obtained from the user.

[0126] (Configuration 19) An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a pre-processing means for receiving an image from the client terminal, linking person regions included in the image with feature information extracted and encrypted from each of the person regions, and responding to the client terminal; a receiving means for receiving, from the client terminal, the feature information associated with a specific area including a specific person among the person areas included in the response; a registration means for decrypting and registering the characteristic information received by the reception means, The client terminal a transmitting means for transmitting the feature information associated with the specific area to the information processing device when consent to registration of the feature information is obtained from the user for the specific person designated from the person area included in the response; An information processing system comprising:

[0127] (Configuration 20) An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a receiving means for receiving an image transmitted from the client terminal; a response means for responding to the client terminal with information about the person detected from the image; a registration means for registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user, The client terminal an inquiry means for presenting people detected from the image based on the response so that the user can see them, and inquiring whether the user agrees to register the feature information for each of the presented people; a requesting means for requesting the information processing device to register the characteristic information of the specific person for whom the consent has been obtained by the inquiring means; An information processing system comprising:

[0128] (Configuration 21) An information processing method executed by an information processing device, receiving, from a user, a first image including a person and a designation of a specific area within the first image; performing a registration process of registering characteristic information of a specific person included in the specific area among people appearing in the first image; An information processing method, wherein consent to the registration of the characteristic information of the specific person included in the specific area is obtained from the user.

[0129] (Configuration 22) An information processing method executed by an information processing device, receiving an image from a user, and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; receiving the feature information associated with a specific area including a specific person from the person areas included in the response; and a step of decoding and registering the received characteristic information, An information processing method, wherein consent to the registration of the characteristic information of the specific person is obtained from the user.

[0130] (Configuration 23) An information processing method executed by an information processing device that can be connected to a client terminal for communication, receiving an image transmitted from the client terminal; a step of responding to a user of the client terminal with information about the person detected from the image so that the information can be visually recognized by the user; registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user; An information processing method comprising:

[0131] (Configuration 24) A program for causing a computer to execute an information processing method, The information processing method includes: receiving, from a user, a first image including a person and a designation of a specific area within the first image; performing a registration process of registering characteristic information of a specific person included in the specific area among people appearing in the first image; A program characterized in that consent to the registration of the characteristic information of the specific person included in the specific area is obtained from the user.

[0132] (Configuration 25) A program for causing a computer to execute an information processing method, The information processing method includes: receiving an image from a user, and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; receiving the feature information associated with a specific area including a specific person from the person areas included in the response; and a step of decoding and registering the received characteristic information, A program characterized in that consent to the registration of the characteristic information about the specific person is obtained from the user.

[0133] (Configuration 26) A program for causing a computer that can be connected to a client terminal to execute an information processing method, The information processing method includes: receiving an image transmitted from the client terminal; a step of responding to a user of the client terminal with information about the person detected from the image so that the information can be visually recognized by the user; a step of registering characteristic information of a specific person among the people detected from the image; Programs including.

Claims

1. a receiving means for receiving from a user a first image including a person and a designation of a specific area within the first image; a registration unit that performs a registration process to register characteristic information of a specific person included in the specific area among people appearing in the first image, An information processing device, wherein consent to the registration of the characteristic information of the specific person has been obtained from the user.

2. the accepting means accepts a second image corresponding to the first image before accepting the designation of the specific region; The image processing device further includes a pre-processing unit that detects a person area from the received second image and responds to the user, The specific area is designated from the person area detected by the pre-processing means.

2. The information processing apparatus according to claim 1, wherein:

3. 3. The information processing apparatus according to claim 2, wherein the second image is the same as the first image.

4. 4. The information processing device according to claim 1, wherein the registration process includes a process of detecting a person area from the first image after the reception means receives the first image and the specification, and determining whether the detected person area matches the specific area.

5. 5. The information processing apparatus according to claim 4, wherein, when a person area detected from the first image in the registration process matches the specific area, the feature information of the person included in the specific area is extracted.

6. the accepting means accepts the second image corresponding to the first image before accepting the designation; The pre-processing means further associates the person detected from the second image with a second hash value determined from the second image and stores the associated information in a storage unit; The registration process includes: a process of determining a first hash value from the first image using the same algorithm as that used to determine the second hash value, and acquiring a person area detected from the second image using the first hash value from the storage unit; a process of determining whether the detected person area matches the specific area; 4. The information processing device according to claim 2, further comprising:

7. 7. The information processing apparatus according to claim 6, wherein, when a person area detected from the first image in the registration process matches the specific area, the feature information of the person included in the specific area is extracted.

8. the receiving means receives the first image and the specification from a client terminal connected to the information processing device for communication; 3. The information processing apparatus according to claim 2, wherein the pre-processing means receives the second image from the client terminal and returns a person area detected from the second image to the client terminal.

9. The information processing apparatus according to claim 8 , wherein the pre-processing means causes the client terminal to display a screen showing a person included in a person area detected from the second image.

10. a pre-processing means for receiving an image from a user and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; a receiving means for receiving the feature information associated with a specific area including a specific person among the person areas included in the response; a registration means for decrypting and registering the characteristic information received by the reception means, An information processing device, wherein consent to the registration of the characteristic information of the specific person has been obtained from the user.

11. the pre-processing means receives the image from a client terminal connected to the information processing device for communication, and transmits the response to the client terminal; 11. The information processing apparatus according to claim 10, wherein the accepting unit accepts the feature information linked to the specific area designated on the client terminal.

12. The information processing apparatus according to claim 11 , wherein the pre-processing means causes the client terminal to display a screen showing the person included in the person area detected from the image.

13. 13. The information processing apparatus according to claim 9, wherein the screen includes, for each of the people displayed on the screen, an inquiry to the user regarding consent to the registration of the characteristic information by the registration means.

14. 14. The information processing apparatus according to claim 13, wherein the person to be inquired about is displayed on the screen in a sequentially changing manner.

15. 14. The information processing apparatus according to claim 13, wherein an operation of specifying the person to be inquired about is accepted from the user on the screen.

16. the specific region includes a face region, 11. The information processing apparatus according to claim 1, wherein the feature information is a facial feature amount.

17. An information processing device that can be connected to a client terminal for communication, a receiving means for receiving an image transmitted from the client terminal; a response means for responding to information about the person detected from the image so that the user of the client terminal can visually recognize the information; a registration means for registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user; An information processing device comprising:

18. An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a receiving means for receiving, from the client terminal, a first image including a person and a designation of a specific area within the first image; a registration unit that performs a registration process to register characteristic information of a specific person included in the specific area among people appearing in the first image, The client terminal An information processing system characterized by comprising a transmission means for transmitting the first image and the designation of the specific area containing the specific person in the first image to the information processing device when consent to the registration of the characteristic information for the specific person is obtained from the user.

19. An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a pre-processing means for receiving an image from the client terminal, linking person regions included in the image with feature information extracted and encrypted from each of the person regions, and responding to the client terminal; a receiving means for receiving, from the client terminal, the feature information associated with a specific area including a specific person among the person areas included in the response; a registration means for decrypting and registering the characteristic information received by the reception means, The client terminal a transmitting means for transmitting the feature information associated with the specific area to the information processing device when consent to registration of the feature information is obtained from the user for the specific person designated from the person area included in the response; An information processing system comprising:

20. An information processing system in which an information processing device and a client terminal are communicatively connected via a network, The information processing device includes: a receiving means for receiving an image transmitted from the client terminal; a response means for responding to the client terminal with information about the person detected from the image; a registration means for registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user, The client terminal an inquiry means for presenting people detected from the image based on the response so that the user can see them, and inquiring whether the user agrees to register the feature information for each of the presented people; a requesting means for requesting the information processing device to register the characteristic information of the specific person for whom the consent has been obtained by the inquiring means; An information processing system comprising:

21. An information processing method executed by an information processing device, receiving, from a user, a first image including a person and a designation of a specific area within the first image; performing a registration process of registering characteristic information of a specific person included in the specific area among people appearing in the first image, An information processing method, wherein consent to the registration of the characteristic information of the specific person included in the specific area is obtained from the user.

22. An information processing method executed by an information processing device, receiving an image from a user, and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; receiving the feature information associated with a specific area including a specific person from the person areas included in the response; and a step of decoding and registering the received characteristic information, An information processing method, wherein consent to the registration of the characteristic information of the specific person is obtained from the user.

23. An information processing method executed by an information processing device that can be connected to a client terminal for communication, receiving an image transmitted from the client terminal; a step of responding to a user of the client terminal with information about the person detected from the image so that the information can be visually recognized by the user; registering characteristic information of a specific person among the people detected from the image for whom consent has been obtained from the user; An information processing method comprising:

24. A program for causing a computer to execute an information processing method, The information processing method includes: receiving, from a user, a first image including a person and a designation of a specific area within the first image; performing a registration process of registering characteristic information of a specific person included in the specific area among people appearing in the first image, A program characterized in that consent to the registration of the characteristic information of the specific person included in the specific area is obtained from the user.

25. A program for causing a computer to execute an information processing method, The information processing method includes: receiving an image from a user, and responding by linking person areas included in the image with feature information extracted and encrypted from each of the person areas; receiving the feature information associated with a specific area including a specific person from the person areas included in the response; and a step of decoding and registering the received characteristic information, A program characterized in that consent to the registration of the characteristic information about the specific person is obtained from the user.

26. A program for causing a computer that can be connected to a client terminal to execute an information processing method, The information processing method includes: receiving an image transmitted from the client terminal; a step of responding to a user of the client terminal with information about the person detected from the image so that the information can be visually recognized by the user; a step of registering characteristic information of a specific person among the people detected from the image; Programs including.

Images