Information processing device, information processing system, information processing method, and recording medium

Abstract

This information processing device is provided with a first acquisition unit, a second acquisition unit, and a creation unit. The first acquisition unit acquires a first image including an authentication target. The second acquisition unit acquires a second image including the authentication target. The creation unit creates display information including a second image in which a result of authenticating the authentication target included in the first image is associated with the authentication target included in the second image.

Description

Information processing device, information processing system, information processing method, and recording medium

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a recording medium.

[0002] Recently, the introduction of gateless authentication systems that allow smooth entry without people having to stop has been considered as an alternative to the commonly used flapper gates for entrance control at station ticket gates, theme parks, event venues, etc.

[0003] For example, Patent Document 1 discloses a technique for identifying which index detected from an image corresponds to which index among indices arranged in real space. The information processing method described in Patent Document 1 includes a detection step and a first calculation step. In the detection step, the position of the index in an image captured by an imaging means that captures an image of real space in which the index is arranged is detected. In the first calculation step, when at least one of the position of the index in real space and the position and orientation of the imaging means has been determined based on measurement, the coordinate position of the index in real space when projected onto the captured image is determined based on the position of the index in real space and the position and orientation of the imaging means. Then, in the information processing method described in Patent Document 1, a process is performed to associate indices whose coordinate positions are close to each other in the captured image using the coordinate positions of each index detected in the detection step and the coordinate positions calculated in the first calculation step.

[0004] Japanese Patent Application Laid-Open No. 2005-83826

[0005] The technology described in Patent Document 1 uses a three-dimensional position and orientation sensor to obtain at least one of the position of the index in real space and the position and orientation of the imaging means, but does not disclose a technology for associating objects between different images captured at the same time of a common area. As a result, for example, it is difficult to confirm the authentication result of an authentication object performed using an image captured by one camera using an image captured by another camera.

[0006] One of the objectives of the present disclosure is to easily check the authentication result by using an image different from the image used for authentication.

[0007] The information processing device of the present disclosure includes a first acquisition means for acquiring a first image including an authentication object, a second acquisition means for acquiring a second image including the authentication object, and a creation means for creating display information including the second image in which the authentication result of the authentication object included in the first image is associated with the authentication object included in the second image.

[0008] The information processing method disclosed herein includes one or more computers acquiring a first image including an authentication object, acquiring a second image including the authentication object, and creating display information including the second image in which the authentication result of the authentication object included in the first image is associated with the authentication object included in the second image.

[0009] The program in the present disclosure is a program for causing one or more computers to execute the following steps: acquire a first image including an object to be authenticated; acquire a second image including the object to be authenticated; and create display information including the second image in which the authentication result of the object to be authenticated included in the first image is associated with the object to be authenticated included in the second image.

[0010] 1 is a diagram illustrating an overview of a first information processing system according to the present disclosure. FIG. 2 is a block diagram illustrating an overview of a first information processing device according to the present disclosure. FIG. 3 is a flowchart illustrating an overview of processing operations of the first information processing device according to the present disclosure. FIG. 4 is a block diagram illustrating a detailed configuration example of the first information processing system according to the present disclosure. FIG. 5 is a diagram illustrating an example of a first image captured by a first imaging device according to the present disclosure. FIG. 6 is a block diagram illustrating a detailed configuration example of the first information processing device according to the present disclosure. FIG. 7 is a flowchart illustrating a detailed example 1 of processing operations of the first information processing device according to the present disclosure. FIG. 8 is a flowchart illustrating a detailed example 2 of processing operations of the first information processing device according to the present disclosure. FIG. 9 is a flowchart illustrating a detailed example 3 of processing operations of the first information processing device according to the present disclosure. FIG. 10 is a diagram illustrating an installation example when a reference object is installed in a target area according to the present disclosure. FIG. 11 is a block diagram illustrating a configuration example of a first association unit according to the present disclosure. FIG. 12 is a diagram illustrating an example of a plan view of a target area estimated using a first image according to the present disclosure. FIG. 13 is a diagram illustrating an example of a second image associated with an authentication result according to the present disclosure. FIG. 14 is a flowchart illustrating an example of processing operations of a first terminal according to the present disclosure. FIG. 15 is a diagram illustrating an example of the physical configuration of a first information processing device according to the present disclosure. FIG. 16 is a diagram illustrating an example of the configuration of a second information processing system according to the present disclosure. FIG. 17 is a block diagram illustrating an example of the configuration of a second information processing device according to the present disclosure.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, similar components are denoted by similar reference numerals, and descriptions thereof will be omitted as appropriate. In the present disclosure, the drawings relate to one or more embodiments.

[0012] 1 , the information processing system S1 includes a first imaging device 31, a terminal 50, a first acquisition unit 110, a second acquisition unit 210, and a creation unit 320. The terminal 50 includes a second imaging unit 51 and a display unit 53.

[0013] The first image capturing device 31 captures an image of a target area. The second image capturing unit 51 captures an image of at least a part of the target area. The display unit 53 displays display information.

[0014] The first acquisition unit 110 acquires a first image including an authentication target. The second acquisition unit 210 acquires a second image including the authentication target. The creation unit 320 creates display information including a second image in which the authentication result of the authentication target included in the first image is associated with the authentication target included in the second image.

[0015] According to this information processing system S1, the authentication result can be easily confirmed by referring to the second image associated with the authentication result based on the first image. Therefore, it is possible to easily confirm the authentication result using an image different from the image used for authentication.

[0016] (Configuration Example of Information Processing Apparatus 100) As shown in FIG. 2, the information processing apparatus 100 includes a first acquisition unit 110, a second acquisition unit 210, and a creation unit 320.

[0017] The first acquisition unit 110 acquires a first image including an authentication target. The second acquisition unit 210 acquires a second image including the authentication target. The creation unit 320 creates display information including a second image in which the authentication result of the authentication target included in the first image is associated with the authentication target included in the second image.

[0018] According to this information processing system S1, the authentication result can be easily confirmed by referring to the second image associated with the authentication result based on the first image. Therefore, it is possible to easily confirm the authentication result using an image different from the image used for authentication.

[0019] (Example of Processing Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG.

[0020] The first acquisition unit 110 acquires a first image including an authentication target (step S110).

[0021] The second acquisition unit 210 acquires a second image including an authentication target (step S210).

[0022] The creation unit 320 creates display information including a second image in which the authentication result of the authentication object included in the first image is associated with the authentication object included in the second image (step S320).

[0023] According to this information processing, the authentication result can be easily confirmed by referring to the second image associated with the authentication result based on the first image, thereby making it possible to easily confirm the authentication result using an image different from the image used for authentication.

[0024] (Detailed Example) A detailed example of this embodiment will be described below.

[0025] The information processing system S1 includes a first image capturing device 31, a terminal 50, and an information processing device 100, as shown in FIG. 4, for example.

[0026] The first image capturing device 31, the terminal 50, and the information processing device 100 are connected to each other via, for example, a network NT, whereby the first image capturing device 31, the terminal 50, and the information processing device 100 transmit and receive information to and from each other.

[0027] (Regarding the First Image Capturing Device 31) The first image capturing device 31 is, for example, a camera equipped with a communication function. The first image capturing device 31 may be a camera with a fixed position, a camera equipped with at least one of tilt, pan, and zoom functions, or a movable camera.

[0028] The first image capturing device 31 captures an image of the target area to obtain a first image including the authentication target.

[0029] The target area is determined in advance as an area where authentication is performed. The target area is, for example, an area where authentication is performed for people entering and leaving a station ticket gate, a theme park, an event venue, etc. Note that the target area is not limited to the examples given here, and may be determined in advance as appropriate.

[0030] The authentication target is a person who is in the target area.

[0031] 5 is a diagram showing an example of a first image captured by the first image capturing device 31. The first image shown in the figure includes three authentication subjects A to C and an observer P. The observer P is a person who monitors the target area and is an example of a user of the terminal 50.

[0032] (Regarding the terminal 50) The terminal 50 is, for example, a portable terminal carried by the monitor P, and in this embodiment is a head-mounted display. Note that the portable terminal is not limited to a head-mounted display, and may be a smartphone, a tablet terminal, etc. Furthermore, the terminal 50 is not limited to a portable terminal, and may be a stationary terminal, etc.

[0033] As shown in FIG. 4 , the terminal 50 includes a second image capturing unit 51 , a communication unit 52 , and a display unit 53 .

[0034] The second image capturing unit 51 captures, for example, an image in the line of sight of the observer P. As a result, the second image capturing unit 51 captures an image of at least a part of the target area to obtain a second image including the authentication target. Such a second image is an image captured at substantially the same time as the first image by a different image capturing device, and is, for example, an image corresponding to the field of view of the observer P. The second image capturing unit 51 may be equipped with, for example, a zoom function.

[0035] The communication unit 52 communicates with other devices such as the information processing device 100. As a result, the communication unit 52 transmits and receives information to and from other devices. The communication unit 52 acquires, for example, display information.

[0036] The display unit 53 displays the display information. For example, the display unit 53 displays an image in the field of view of the observer P that corresponds to the authentication result included in the display information.

[0037] The image displayed by the display unit 53 may be an image using AR (Augmented Reality) technology, i.e., an image showing an indicator showing the authentication result in the real world viewed through a display provided on the head-mounted display. Furthermore, the image displayed by the display unit 53 may be an image in which an indicator showing the authentication result is included in the second image.

[0038] The image displayed by the display unit 53 may be an image using VR (Virtual Reality) technology, MR (Mixed Reality) technology, or the like.

[0039] (Regarding the information processing device 100) (Example of functional configuration of the information processing device 100) As shown in Fig. 6 , the information processing device 100 includes the above-mentioned first acquisition unit 110, second acquisition unit 210, and creation unit 320. The information processing device 100 further includes, for example, a first detection unit 120, a first tracking unit 130, an authentication unit 140, a second detection unit 220, a second tracking unit 230, an association unit 310, and a transmission unit 330.

[0040] The first detection unit 120 uses a first image to detect an authentication object included in the first image and a first position that is the position of the authentication object.

[0041] The first tracking unit 130 uses the result of detection from the first image to track the authentication target included in the first image.

[0042] The authentication unit 140 authenticates the authentication target using the first image.

[0043] The second detection unit 220 uses the second image to detect an authentication object included in the second image and a second position that is the position of the authentication object.

[0044] The second tracking unit 230 uses the detection result from the second image to track the authentication target included in the second image.

[0045] The matching unit 310 generates matching information for matching the authentication objects in the first image and the second image based on first information, which is information about the first image and includes a first position, and second information, which is information about the second image and includes a second position.

[0046] The transmitting unit 330 transmits the display information.

[0047] (Detailed Example 1 of Processing Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG. 7, for example.

[0048] The first acquisition unit 110 executes the above-mentioned step S110.

[0049] The first detection unit 120 uses the first image to detect an authentication object included in the first image and a first position that is the position of the authentication object (step S120).

[0050] The first tracking unit 130 uses the result of detection from the first image to track the authentication target included in the first image (step S130).

[0051] The authentication unit 140 authenticates the authentication target using the first image (step S140).

[0052] (Detailed Example 2 of Processing Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG. 8, for example.

[0053] The second acquisition unit 210 executes the above-mentioned step S210.

[0054] The second detection unit 220 uses the second image to detect the authentication object included in the second image and the second position, which is the position of the authentication object (step S220).

[0055] The second tracking unit 230 uses the result of detection from the second image to track the authentication target included in the second image (step S230).

[0056] (Detailed Example 3 of Processing Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG. 9, for example.

[0057] The matching unit 310 generates matching information for matching the authentication objects in the first image and the second image based on first information, which is information about the first image and includes a first position, and second information, which is information about the second image and includes a second position (step S310).

[0058] The creation unit 320 executes the above-mentioned step S320.

[0059] The transmitting unit 330 transmits the display information (step S330).

[0060] Hereinafter, each function provided in the information processing device 100 and the processing executed by the information processing device 100 will be described.

[0061] (Regarding the first acquisition unit 110) The first acquisition unit 110 acquires, for example, a first image from the first image capturing device 31. If the authentication target is present in the target area, the first image includes the authentication target. The first image is, for example, a video, and may be a real-time video.

[0062] (Regarding the first detection unit 120) The first detection unit 120 uses a first image to detect an authentication target included in the first image. Then, the first detection unit 120 detects a first position, which is the position of the detected authentication target. The first detection unit 120 may further detect at least one of an attribute and a feature amount of the detected authentication target.

[0063] The attributes are, for example, gender, physical attributes, posture, appearance attributes, etc. Physical attributes are attributes related to the body of the authentication target, such as height and build. Appearance attributes are attributes related to the appearance of the authentication target, and may include, for example, the color and pattern of clothing, whether or not accessories such as glasses or a tie are worn, or whether or not an umbrella is carried. Note that the attributes are not limited to those exemplified here.

[0064] The feature amount may be, for example, a facial feature amount extracted from a face image of the authentication target, a posture feature amount indicating the posture of the authentication target, an appearance feature amount indicating the appearance characteristics of the authentication target, etc. Such a feature amount may be a so-called feature vector. Note that the feature amount is not limited to those exemplified here.

[0065] The first detection unit 120 may, for example, input a first image into an image analysis model to detect the authentication target and the first position. The image analysis model is a model that, when an image is input, detects the authentication target from the input image and outputs the position of the detected authentication target. This position may be a position in the input image, or may be a position in an appropriate two-dimensional or three-dimensional coordinate system. When a three-dimensional coordinate system is used, the image analysis model may further use depth information of the image, such as the shooting direction of the device that captured the input image.

[0066] For example, the first detection unit 120 may input the first image to an image analysis model to acquire at least one of the attributes and features of the detected authentication target. In this case, the image analysis model may further output at least one of the attributes and features of the detected authentication target.

[0067] Such an image analysis model may be, for example, a trained machine learning model, which may be trained using training data including training images and correct answers related to the images.

[0068] The technology for realizing the first detection unit 120 is not limited to the machine learning model, and other image processing technologies may be used.

[0069] (First Tracking Unit 130) The first tracking unit 130 tracks the authentication target included in the first image, for example, using the result of detection from the first image by the first detection unit 120.

[0070] For example, the first tracking unit 130 tracks the authentication target included in the first image using at least one of the first position, attribute, and feature amount of the detected authentication target.

[0071] When the first position is used, the first tracking unit 130 may track the authentication target in the first images taken at different times, for example, by using a standard movement range according to the detected time difference. By using at least one of the attribute and the feature, the authentication target in the first images taken at different times can be tracked with higher accuracy.

[0072] For example, if the face can be photographed, the target of authentication in the first image can be tracked more accurately by using facial features. Also, if the target of authentication has its back turned or is looking down and the face cannot be photographed, other attributes and features can be used to track the target of authentication more accurately.

[0073] The method for tracking the authentication target included in the image is not limited to the example given here, and for example, a general technique for tracking a person in a video may be used.

[0074] (Authentication Unit 140) The authentication unit 140 authenticates the authentication target using, for example, the result of detection from the first image by the first detection unit 120 and pre-defined registered information.

[0075] For example, the authentication unit 140 compares the facial feature extracted from the first image with the facial feature registered in advance and calculates the similarity between them. This similarity may be, for example, but is not limited to, cosine similarity. For example, the authentication unit 140 compares the similarity with a predetermined authentication threshold and generates an authentication result indicating whether or not the authentication of the authentication target has been successful based on the comparison result. In this case, the authentication unit 140 may determine that the authentication has been successful if the similarity is equal to or greater than the authentication threshold, and may determine that the authentication has failed if the similarity is less than the authentication threshold.

[0076] The method of authenticating an object using an image is not limited to the example given here, and any general authentication technique using an image may be used.

[0077] (Regarding the second acquisition unit 210) The second acquisition unit 210 acquires a second image from, for example, the terminal 50. The second image is an image of a target area captured when, for example, a monitor P is looking at the target area. The target area included in the second image may be a part of the target area or the entire target area, depending on the angle of view of the second imaging unit 51 that captures the image. For example, if the authentication target is present in the target area included in the second image, the second image includes the authentication target. The second image may be, for example, a video, and may be a real-time video.

[0078] (Regarding the second detection unit 220) The second detection unit 220 uses the second image to detect an authentication target included in the second image. Then, the second detection unit 220 detects a second position, which is the position of the detected authentication target. The second detection unit 220 may further detect at least one of attributes and feature amounts of the detected authentication target. Attributes include, for example, the above-mentioned gender, physical attributes, posture, appearance attributes, etc. Feature amounts include, for example, the above-mentioned facial feature amounts, posture feature amounts, appearance feature amounts, etc.

[0079] At least one of the attributes and feature amounts detected by the second detection unit 220 may be the same as or may be partially or entirely different from at least one of the attributes and feature amounts detected by the above-described first tracking unit 130. Furthermore, the method by which the second detection unit 220 detects at least one of the authentication target, the second position, the attributes, and the feature amounts may be any of the methods described above as the method by which the first tracking unit 130 detects at least one of the authentication target, the second position, the attributes, and the feature amounts.

[0080] (Second Tracking Unit 230) The second tracking unit 230 tracks the authentication target included in the second image, for example, using the result of detection from the second image by the second detection unit 220.

[0081] For example, the second tracking unit 230 tracks the authentication target included in the second image by using at least one of the second position, attribute, and feature amount of the detected authentication target. The method by which the second tracking unit 230 tracks the authentication target included in the second image may be any of the methods described as the method by which the first tracking unit 130 tracks the authentication target included in the first image.

[0082] (Regarding the Corresponding Unit 310) The correlating unit 310 generates the correlating information based on the first information about the first image and the second information about the second image, for example, at a predetermined timing. The timing may be, for example, at a predetermined time interval, when the line of sight of the observer P changes, or when the shooting range of the second image changes.

[0083] The specified time interval may be determined, for example, in terms of time, or may be determined as an interval corresponding to the number of frames of the first image and the second image, respectively, which constitute the first video and the second video.

[0084] The change in the gaze direction may be detected using, for example, an acceleration sensor (not shown) provided in the terminal 50, the second image, or the like. Then, for example, when the amount of change in the gaze direction is greater than a predetermined gaze threshold, the association unit 310 may generate association information. Since the second image capturing unit 51 captures the gaze direction of the monitor P as described above, for example, the association unit 310 may estimate the amount of change in the gaze direction based on the second image. The amount of change in the gaze direction may be, for example, the acceleration at which the amount of change in the gaze direction has changed, the speed at which the amount of change in the gaze direction has changed, or the magnitude at which the amount of change in the gaze direction has changed.

[0085] The change in the capturing range of the second image may be caused by a change in the capturing direction of the second image capturing unit 51, or by a change in the angle of view using a zoom function or the like of the second image capturing unit 51. The change in the capturing direction of the second image capturing unit 51 may be caused, for example, by a change in the line of sight of the monitor P, or by a pan / tilt function of the second image capturing unit 51. The association unit 310 may generate association information, for example, when the similarity between the second images acquired at a predetermined time interval is equal to or less than a predetermined image threshold. Furthermore, for example, when a function for changing the capturing range, including at least one of a zoom function, a pan / tilt function, or the like, is used, the association unit 310 may acquire setting information related to the function for changing the capturing range from the second image capturing unit 51, and generate association information when the setting information is changed.

[0086] The first information includes a first location, and the second information includes a second location.

[0087] Each of the first information and the second information may further include at least one of information regarding the imaging device that captured the associated image, information regarding the object to be authenticated in the associated image, and information regarding the reference object in the associated image.

[0088] Here, the related image is the first image for the first information and the second image for the second information, and the photographing device that photographed the related image is the first photographing device 31 for the first information and the second photographing unit 51 (or the terminal 50) for the second information.

[0089] The information about the camera that captured the associated image may include, for example, at least one of the position of the camera, the direction of the image capture, and the like.

[0090] The position and imaging direction of the first imaging device 31 may be registered in advance, or may be acquired based on information from a sensor (not shown) provided in the first imaging device 31, for example.

[0091] The position of the second imaging unit 51 (terminal 50) is, for example, the current position of the terminal 50. The current position of the terminal 50 may be the current position detected by a current position detection function using a GPS (Global Positioning System) or the like of the terminal 50. Furthermore, the current position of the terminal 50 may be, for example, the current position of the observer P calculated from the first image.

[0092] The shooting direction of the second shooting unit 51 (terminal 50) may be acquired based on information from a sensor or the like (not shown) provided in the terminal 50.

[0093] The information on the authentication target in the related image is, for example, the detection result of the first detection unit 120 for the first information, and the detection result of the second detection unit 220 for the second information. That is, the information on the authentication target in the related image may include at least one of an attribute of the authentication target and a feature amount of the authentication target.

[0094] The reference object in the related image is, for example, a fixed object that is installed in advance in the target area as a marker indicating a positional reference. When such a reference object is installed in the target area, the association unit 310 can use the position of the reference object to generate association information. Figure 10 is a diagram showing an example of installation when reference objects are installed in the target area. Note that the number of installed reference objects, their positions in the target area, installation method, etc. are not limited to the example shown in the figure.

[0095] Such a reference object may be used to detect the first position and the second position in at least one of the first detection unit 120 and the second detection unit 220. The reference object may also be used to detect the current position of the second imaging unit 51 (or the terminal 50) from the second image.

[0096] Furthermore, for example, if the current position of the terminal 50 (i.e., the position of the monitor P) can be accurately determined using the GPS function or the like as described above, this position may be used as a position reference instead of the reference object or together with the reference object included in the first image or the second image. For example, if the first position in the first image is corrected as described below and the position of the authentication target can be accurately detected, the position of the authentication target may be used as a position reference instead of the reference object or together with the reference object included in the first image or the second image. For example, the position of such an authentication target may be used to detect the current position of the second imaging unit 51 (or the terminal 50) from the second image.

[0097] (Regarding Correspondence Information) The correspondence information includes information for correlating the authentication target in the first image with the authentication target in the second image. The correspondence information may further include an authentication result relating to the authentication target.

[0098] The correspondence information may, for example, associate first identification information for identifying an authentication target included in the first image with second identification information for identifying an authentication target included in the second image. More specifically, for example, the correspondence information may associate the authentication targets in the first image with those in the second image based on the first position and the second position. In this case, the first identification information may include a first corrected position obtained by correcting the first position detected by the first detection unit 120. Furthermore, the second identification information may include the second position detected by the second detection unit 220.

[0099] As shown in FIG. 11 , the association unit 310 includes a correction unit 311 and a generation unit 312 .

[0100] (Regarding the Correction Unit 311) The correction unit 311 generates correction information for correcting at least one of the first position and the second position so that the first position and the second position satisfy a predetermined association condition.

[0101] For example, the correction unit 311 generates correction information for associating a first position group consisting of the positions of one or more authentication targets included in the first image with a second position group consisting of the positions of one or more authentication targets included in the second image. The correction information is, for example, information used to collectively correct at least one of the first position group and the second position group so as to satisfy a predetermined association condition. The correction information may include, for example, at least one of an equation for correcting the position, elements of the equation, and parameters used in the equation, or may include values ​​such as a vector for correcting the position.

[0102] The association condition may be, for example, at least one of the following conditions A to E.

[0103] (Condition A) The association condition may be, for example, minimizing the difference (distance) between the first position and the second position, or making the sum of the differences (distances) equal to or less than a predetermined distance threshold. For example, the association condition may be minimizing the sum of the differences (distances) between a plurality of first positions and a plurality of second position groups, or making the sum equal to or less than a predetermined distance threshold. Here, the difference (distance) may be a distance in an image, or may be a distance in an appropriately defined two-dimensional coordinate system or three-dimensional coordinate system.

[0104] (Condition B) The association condition may be, for example, maximizing the overlap of the areas indicating the first position group and the second position group in the second image, or making the overlap equal to or greater than a predetermined value, etc. Here, the areas indicating the first position group and the second position group may be areas indicating only the authentication target, or may be areas surrounded by an indicator such as a frame indicating the authentication target.

[0105] (Condition C) The correspondence condition may be, for example, maximizing the degree of correspondence between the positional relationship of the first location group and the positional relationship of the second location group, or may be making the degree of correspondence equal to or greater than a predetermined threshold value. Here, the degree of correspondence may be the smallness of the sum of the differences (distances) between the first location group and the second location group (the reciprocal of the normalized distance or the difference from 1), or may be the sum of the cosine similarities between corresponding vectors in the first location group and the second location group.

[0106] For example, a plan view such as that shown in Fig. 12 may be used for the positional relationship. The plan view illustrated in Fig. 12 is an example of a plan view of the target area estimated using the first image, and shows the positional relationship of the authentication target as seen from above. The correction unit 311 may estimate an overhead view, such as a plan view, of the target area using the first image, and calculate the positional relationship of the first position group based on the positions of the authentication target included in the overhead view.

[0107] Here, the position of the authentication target (first position, second position) may be identified based on, for example, a horizontal position with respect to the ground and a depth position along the capture direction of each of the first image and the second image. The position of the authentication target may be, for example, the position of the subject's face included in the image. In this case, the position of the authentication target may be identified based on, for example, a horizontal component with respect to the ground and a depth component estimated from the size of the face. The size of the face may be, for example, at least one of the face size, interocular distance, etc. In such a case, an example of the association condition of condition A may be that the difference between the horizontal component and the depth component of the position of the authentication target included in each of the first image and the second image is minimized.

[0108] (Condition D) The association condition may include, for example, that at least one attribute of the authentication target included in each of the first image and the second image matches.

[0109] In more detail, for example, the matching condition may include a match in color attributes of the top and bottom clothing. In this case, for example, if the color attributes of the top and bottom clothing of the target for authentication detected from each of the first image and the second image match, the target for authentication included in each of the first image and the second image may be matched. Also, for example, if at least one of the color attributes of the top and bottom clothing of the target for authentication detected from each of the first image and the second image differs, the target for authentication included in each of the first image and the second image may not be matched.

[0110] (Condition E) The matching condition may be, for example, that the similarity of at least one of the feature quantities of the authentication target included in each of the first image and the second image is equal to or greater than a predetermined similarity threshold. This similarity may be, for example, cosine similarity, but is not limited to this. In this case, for example, if the similarity of the feature quantities extracted from the authentication target detected in each of the first image and the second image is equal to or greater than the similarity threshold, the authentication target included in each of the first image and the second image may be matched. Also, for example, if the similarity of the feature quantities extracted from the authentication target detected in each of the first image and the second image is less than the similarity threshold, the authentication target included in each of the first image and the second image may not be matched.

[0111] The association conditions are not limited to those exemplified here.

[0112] (Regarding the Generator 312) The generator 312 generates association information using the correction information.

[0113] For example, when any one of conditions A to C is used as the association condition, the generation unit 312 may calculate, as the correction information, a correction amount that satisfies the association condition. The correction amount may be expressed as a vector, for example.

[0114] For example, when any one of conditions D to E is used as the association condition, the generation unit 312 may calculate, as the correction information, a correction amount for aligning the positions of the authentication targets associated between the first image and the second image. The correction amount may be expressed, for example, as a vector. When there are multiple authentication targets associated under any one of conditions D to E, the generation unit 312 may use a value obtained by statistically processing (for example, averaging) the correction amounts of the multiple authentication targets as the correction amount (correction information).

[0115] For example, if any two or more of conditions A to E are used as the association conditions, the generation unit 312 may statistically process (for example, average) the correction amounts calculated under each condition and use the resulting value as the correction amount (correction information).

[0116] The generation unit 312, for example, corrects the first position using the calculated correction information and calculates a first corrected position obtained by correcting the first position. Then, the generation unit 312 generates association information including, for example, the first corrected position and the second position. In the association information, the authentication target closest to the first corrected position may be associated with the second position.

[0117] The association information may associate the authentication targets in the first image and the second image using the first detection unit 120 and labels assigned to the authentication targets detected by the first detection unit 120. In this case, the first identification information may include, for example, a label (first identification information) assigned to the authentication target detected by the first detection unit 120. Furthermore, the second identification information may include, for example, a label (second identification information) assigned to the authentication target detected by the second detection unit 220.

[0118] (Regarding Creation Unit 320) The creation unit 320 creates display information including a second image in which the authentication result of the authentication object included in the first image is associated with the authentication object included in the second image, for example, using association information.

[0119] 13 is a diagram showing an example of a display image included in the display information, i.e., a second image associated with an authentication result. The display image shown in the figure shows an example in which the display mode of a frame, which is an indicator indicating an authentication target, differs depending on the authentication result. In the display image shown in the figure, an authentication target that has been successfully authenticated is shown with an indicator surrounded by a single frame, and an authentication target that has failed authentication is shown with an indicator surrounded by a double frame. Note that the display method for associating the authentication results of the authentication targets in the second image is not limited to the method shown here, and the authentication results of the authentication targets may also be associated with the color of the indicator (e.g., frame) indicating the authentication target, whether it blinks, etc.

[0120] In the display image shown in the figure, the position of the index indicating the authentication result by authentication unit 140 using the first image is corrected from the display position corresponding to the first position indicated by the dotted line to a position corresponding to the first corrected position, so that the authentication result can be accurately displayed in correspondence with the authentication target in the second image.

[0121] As described above, for example, the association unit 310 generates association information at predetermined time intervals. The creation unit 320 may create display information, for example, using the tracking result of the second tracking unit 230 and the most recently generated association information until the next association information is generated. For example, the association unit 310 may create display information by causing the association between the second position (second identification information) and the first corrected position (first identification information) in the association information to follow the tracking result of the second position by the second tracking unit 230.

[0122] (Transmitting Unit 330) The transmitting unit 330 transmits, for example, display information to the terminal 50.

[0123] (Example of Processing Operation of Terminal 50) The terminal 50 executes information processing as shown in FIG. 14, for example.

[0124] The communication unit 52 acquires the display information via, for example, the network NT (step S52).

[0125] The display unit 53 displays the display information (step S53). As a result, a display image (i.e., a second image associated with the authentication result) such as that shown in FIG.

[0126] (Example of physical configuration of information processing device 100) The information processing device 100 may be, for example, a computer. Physically, the information processing device 100 includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070, as shown in FIG. 15 .

[0127] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0128] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0129] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0130] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing device 100. The processor 1020 reads each of these program modules into the memory 1030 and executes them to realize the function corresponding to the program module.

[0131] The network interface 1050 is an interface for connecting the information processing device 100 to a network NT. The network NT is a communication network for transmitting and receiving information to and from other devices (not shown), and may be configured as a wired or wireless network or a combination of these.

[0132] The input interface 1060 is an interface for the user to input information, and is composed of, for example, a touch panel, a keyboard, a mouse, and the like.

[0133] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, etc. The output interface 1070 may also include a speaker.

[0134] For example, the terminal 50 may physically include a camera in addition to the components included in the information processing device 100 .

[0135] The functions of the information processing system S1 may be provided by the information processing system S1 as a whole, and examples of the physical configuration of the device (e.g., an information processing device such as a computer) are not limited to the above examples. For example, the terminal 50 may be provided as an information processing device with some or all of the functions of the information processing device 100. Furthermore, for example, the information processing system S1 may be provided with one or more information processing devices other than the terminal 50 and the information processing device 100, and these may be provided with an appropriate share of the functions of the information processing system S1 as a whole.

[0136] (Actions and Effects) As described above, according to this embodiment, the information processing device 100 includes a first acquisition unit 110, a second acquisition unit 210, and a creation unit 320. The first acquisition unit 110 acquires a first image including an authentication target. The second acquisition unit 210 acquires a second image including the authentication target. The creation unit 320 creates display information including a second image in which the authentication result of the authentication target included in the first image is associated with the authentication target included in the second image.

[0137] This allows the authentication result to be easily confirmed by referring to the second image associated with the authentication result based on the first image, making it possible to easily confirm the authentication result using an image different from the image used for authentication.

[0138] According to this embodiment, the information processing device 100 includes an authentication unit 140 , a first detection unit 120 , a second detection unit 220 , and an association unit 310 .

[0139] The authentication unit 140 authenticates the authentication target using the first image. The first detection unit 120 uses the first image to detect the authentication target included in the first image and a first position which is the position of the authentication target. The second detection unit 220 uses the second image to detect the authentication target included in the second image and a second position which is the position of the authentication target.

[0140] The matching unit 310 generates matching information for matching the authentication objects in the first image and the second image based on first information, which is information about the first image and includes a first position, and second information, which is information about the second image and includes a second position.

[0141] This allows the authentication result to be accurately associated with the object to be authenticated in the second image based on the first position and the second position, and thus makes it possible to easily check the authentication result using an image different from the image used for authentication.

[0142] According to this embodiment, the association information includes information that associates information for identifying the authentication target included in the first image with information for identifying the authentication target included in the second image.

[0143] This allows the authentication result to be displayed in accurate correspondence with the authentication target in the second image, making it possible to easily check the authentication result using an image different from the image used for authentication.

[0144] According to this embodiment, the information processing device 100 includes a first tracking unit 130 and a second tracking unit 230. The first tracking unit 130 tracks the authentication target included in the first image using the result of detection from the first image. The second tracking unit 230 tracks the authentication target included in the second image using the result of detection from the second image.

[0145] The association unit 310 includes a correction unit 311 and a generation unit 312. The correction unit 311 generates correction information for correcting at least one of the first position and the second position so that the first position and the second position satisfy a predetermined association condition. The generation unit 312 generates association information using the correction information.

[0146] This allows the position of the indicator indicating the authentication result of authentication unit 140 using the first image to be corrected, and the authentication result to be displayed in accurate correspondence with the object to be authenticated in the second image. Therefore, it becomes possible to easily check the authentication result using an image different from the image used for authentication.

[0147] According to this embodiment, each of the first information and the second information further includes at least one of information regarding the imaging device that captured the associated image, information regarding the object to be authenticated in the associated image, and information regarding the reference object in the associated image.

[0148] This allows the first information and the second information to be used to accurately associate the authentication objects contained in the first image and the second image, respectively. Therefore, the authentication result can be accurately associated with the authentication object in the second image and displayed. Therefore, it is possible to easily check the authentication result using an image different from the image used for authentication.

[0149] According to this embodiment, the information about the authentication target includes at least one of the attributes of the authentication target and the feature amount of the authentication target.

[0150] This allows the authentication targets included in the first image and the second image to be accurately associated with each other using at least one of the attributes and features of the authentication targets. The authentication result can be accurately associated with the authentication target in the second image and displayed. Therefore, it is possible to easily check the authentication result using an image different from the image used for authentication.

[0151] Second Embodiment The association information may be information for associating the authentication targets in the first image and the second image, and may include, for example, correction information. In this embodiment, an example will be described in which the association information includes correction information.

[0152] 16, the information processing system S2 includes an information processing device 200 instead of the information processing device 100. Except for this, the information processing system S2 may be configured similarly to the information processing system S1 according to the first embodiment.

[0153] 17 , the information processing device 200 includes a first acquisition unit 110, a first detection unit 120, a first tracking unit 130, an authentication unit 140, a second acquisition unit 210, a second detection unit 220, and a transmission unit 330, which are similar to those in the first embodiment. The information processing device 200 further includes an association unit 410 and a creation unit 420, which replace the association unit 310 and the creation unit 320 in the first embodiment. The association unit 410 and the creation unit 420 execute steps S310 and S320 shown in FIG. 9 , instead of the association unit 310 and the creation unit 320 in the first embodiment, respectively.

[0154] The information processing device 200 may include a second tracking unit 230 similar to that in the first embodiment, but may not include the second tracking unit 230 as shown in the figure.

[0155] (Regarding the Corresponding Unit 410) The corresponding unit 410 generates corresponding information based on first information about the first image and second information about the second image, for example, at a predetermined timing. In this respect, the corresponding unit 410 may be similar to the corresponding unit 310 according to the first embodiment.

[0156] The correspondence information according to this embodiment includes correction information as information for correlating the authentication targets in the first image and the second image. The correction information is information for correcting a position in the second image corresponding to the first position so that the first position and the second position satisfy a predetermined correspondence condition. In other words, the correspondence information according to this embodiment includes correction information for correcting a position in the second image corresponding to the first position so that the first position and the second position satisfy a predetermined correspondence condition.

[0157] The association conditions may include one or more of the conditions A to E described in the first embodiment.

[0158] For example, when any one of conditions A to C is used as the association condition, the association unit 410 may calculate, as the correction information, a correction amount for correcting the first position to a corresponding position in the second image so as to satisfy the association condition. The correction amount may be expressed, for example, as a vector.

[0159] For example, when any one of conditions D to E is used as the association condition, the association unit 410 may calculate, as the correction information, a correction amount for correcting the first position of an authentication target associated between the first image and the second image to a corresponding position in the second image. The correction amount may be expressed, for example, as a vector. When there are multiple authentication targets associated under any one of conditions D to E, the association unit 410 may use a value obtained by statistically processing (for example, averaging) the correction amounts of the multiple authentication targets as the correction amount (correction information).

[0160] For example, if any two or more of conditions A to E are used as the association conditions, the association unit 410 may statistically process (for example, average) the correction amounts calculated under each condition and use the resulting value as the correction amount (correction information).

[0161] (Regarding the Creation Unit 420) The creation unit 420 creates display information including a second image in which the authentication result of the authentication target included in the first image is associated with the authentication target included in the second image, for example, by using association information. In this respect, the creation unit 420 may be similar to the creation unit 320 according to the first embodiment.

[0162] The creation unit 420 according to this embodiment creates display information including a second image in which the authentication result of the authentication target included in the first image is superimposed at a position corrected using the correction information. When the authentication result is displayed in the form of an index (e.g., a frame) indicating the authentication target as exemplified in Fig. 13, the display position of the index in the second image may be, for example, a position corresponding to the first position in the second image corrected using the correction information.

[0163] (Actions and Effects) As described above, according to this embodiment, the correspondence information includes correction information for correcting the position corresponding to the first position in the second image so that the first position and the second position satisfy predetermined correspondence conditions.

[0164] This allows the authentication result to be associated with the authentication target in the second image using the authentication result and tracking result using the first image, eliminating the need to track the authentication target using the second image. This reduces the processing load and allows the authentication result to be accurately associated with the authentication target in the second image and displayed. This makes it easier to check the authentication result using an image different from the image used for authentication, while also reducing the processing load.

[0165] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0166] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0167] Some or all of the above embodiments can be described as, but are not limited to, the following notes. 1. An information processing device comprising: a first acquisition means for acquiring a first image including an authentication object; a second acquisition means for acquiring a second image including the authentication object; and a creation means for creating display information including the second image in which an authentication result of the authentication object included in the first image is associated with the authentication object included in the second image. 2. The information processing device described in 1., further comprising: an authentication means for authenticating the authentication object using the first image; a first detection means for using the first image to detect the authentication object included in the first image and a first position which is a position of the authentication object; a second detection means for using the second image to detect the authentication object included in the second image and a second position which is a position of the authentication object; and an association means for generating association information for associating the authentication objects in the first image and the second image based on first information related to the first image and including the first position, and second information related to the second image and including the second position. 3. The information processing device described in 2., wherein the association information includes information for identifying the authentication target included in the first image and information for identifying the authentication target included in the second image. 4. The information processing device described in 3., further comprising: a first tracking means for tracking the authentication target included in the first image using a result detected from the first image; and a second tracking means for tracking the authentication target included in the second image using a result detected from the second image, wherein the association means includes: a correction means for generating correction information for correcting at least one of the first position and the second position so that the first position and the second position satisfy a predetermined association condition; and a generation means for generating the association information using the correction information. 5. The information processing device described in 2., wherein the association information includes correction information for correcting a position in the second image corresponding to the first position so that the first position and the second position satisfy a predetermined association condition.6. The information processing device described in any one of 2. to 5., wherein each of the first information and the second information further includes at least one of information related to an imaging device that captured the related image, information related to an authentication target in the related image, and information related to a reference object in the related image. 7. The information processing device described in 6., wherein the information related to the authentication target includes at least one of an attribute of the authentication target and a feature amount of the authentication target. 8. An information processing system comprising: a first imaging device that captures an image of a target area; and a terminal, wherein the terminal includes: a second imaging unit that captures an image of at least a part of the target area; and a display unit that displays display information, and further comprising: a first acquisition means that acquires a first image including the authentication target; a second acquisition means that acquires a second image including the authentication target; and a creation means that creates display information including the second image in which the authentication result of the authentication target included in the first image is associated with the authentication target included in the second image. 10. An information processing method in which one or more computers acquire a first image including an authentication target, acquire a second image including the authentication target, and create display information including the second image in which an authentication result of the authentication target included in the first image is associated with the authentication target included in the second image. 11. The information processing method described in 9., further including: authenticating the authentication target using the first image, detecting the authentication target included in the first image and a first position that is a position of the authentication target using the first image, detecting the authentication target included in the second image and a second position that is a position of the authentication target using the second image, and generating association information for associating the authentication target in the first image and the second image based on first information about the first image that includes the first position and second information about the second image that includes the second position. 12. The information processing method described in 10., in which the association information includes information for associating information for identifying the authentication target included in the first image with information for identifying the authentication target included in the second image.12. The information processing method described in 11., further including tracking the authentication target included in the first image using a result detected from the first image, and tracking the authentication target included in the second image using a result detected from the second image, and generating the association information includes generating correction information for correcting at least one of the first position and the second position so that the first position and the second position satisfy a predetermined association condition, and generating the association information using the correction information. 13. The information processing method described in 10., wherein the association information includes correction information for correcting a position in the second image corresponding to the first position so that the first position and the second position satisfy a predetermined association condition. 14. The information processing method described in any one of 10. to 13., wherein each of the first information and the second information further includes at least one of information on an imaging device that captured the associated image, information on the authentication target in the associated image, and information on a reference object in the associated image. 15. The information processing method according to 14., wherein the information relating to the authentication target includes at least one of an attribute of the authentication target and a feature amount of the authentication target. 16. A program for causing one or more computers to execute the information processing method according to any one of 9. to 15. 17. A recording medium having recorded thereon a program for causing one or more computers to execute the information processing method according to any one of 9. to 15.

[0168] This application claims priority based on Japanese Patent Application No. 2024-049281, filed March 26, 2024, the disclosure of which is incorporated herein by reference in its entirety.

[0169] S1, S2 Information processing system 31 First imaging device 50 Terminal 51 Second imaging unit 52 Communication unit 53 Display unit 100, 200 Information processing device 110 First acquisition unit 120 First detection unit 130 First tracking unit 140 Authentication unit 210 Second acquisition unit 220 Second detection unit 230 Second tracking unit 310, 410 Corresponding unit 311 Correction unit 312 Generation unit 320, 420 Creation unit 330 Transmission unit

Claims

1. An information processing device comprising: a first acquisition means for acquiring a first image including an object to be authenticated; a second acquisition means for acquiring a second image including the object to be authenticated; and a creation means for creating display information including the second image in which the authentication result of the object to be authenticated included in the first image is associated with the object to be authenticated included in the second image.

2. The information processing device according to claim 1, further comprising: an authentication means for authenticating the authentication target using the first image; a first detection means for detecting the authentication target included in the first image and a first position which is the position of the authentication target using the first image; a second detection means for detecting the authentication target included in the second image and a second position which is the position of the authentication target using the second image; and a matching means for generating matching information for matching the authentication target in the first image and the second image based on first information about the first image which includes the first position and second information about the second image which includes the second position.

3. The information processing device according to claim 2, wherein the association information includes information that associates information for identifying the authentication object contained in the first image with information for identifying the authentication object contained in the second image.

4. An information processing device as described in claim 3, further comprising: a first tracking means for tracking the authentication object included in the first image using the results detected from the first image; and a second tracking means for tracking the authentication object included in the second image using the results detected from the second image, wherein the matching means includes: a correction means for generating correction information for correcting at least one of the first position and the second position so that the first position and the second position satisfy predetermined matching conditions; and a generation means for generating the matching information using the correction information.

5. An information processing device as described in claim 2, wherein the correspondence information includes correction information for correcting a position in the second image corresponding to the first position so that the first position and the second position satisfy predetermined correspondence conditions.

6. An information processing device according to any one of claims 2 to 5, wherein each of the first information and the second information further includes at least one of information relating to the photographing device that captured the associated image, information relating to the object to be authenticated in the associated image, and information relating to the reference object in the associated image.

7. The information processing device according to claim 6, wherein the information relating to the authentication target includes at least one of an attribute of the authentication target and a feature amount of the authentication target.

8. An information processing system comprising: a first photographing device that photographs a target area; and a terminal, wherein the terminal includes: a second photographing unit that photographs at least a part of the target area; and a display unit that displays display information; a first acquisition means that acquires a first image including an authentication object; a second acquisition means that acquires a second image including the authentication object; and a creation means that creates display information including the second image in which the authentication result of the authentication object included in the first image is associated with the authentication object included in the second image.

9. An information processing method in which one or more computers acquire a first image including an object to be authenticated, acquire a second image including the object to be authenticated, and create display information including the second image in which the authentication result of the object to be authenticated included in the first image is associated with the object to be authenticated included in the second image.

10. A recording medium having recorded thereon a program for causing one or more computers to acquire a first image including an object to be authenticated, acquire a second image including the object to be authenticated, and create display information including the second image in which the authentication result of the object to be authenticated included in the first image is associated with the object to be authenticated included in the second image.

Images