Information processing apparatus, information processing method, and program

Abstract

To specify an image including the figure of a target user without imaging the target user in advance.SOLUTION: An information processing apparatus is configured to: acquire data on a captured image and position identifying information for identifying a position of a user; specify, based on the position identifying information, a candidate period which is a period in which the captured image may include the figure of the user; acquire features of a natural person included as a figure in the captured image corresponding to the candidate period; specify features of the user based on the acquired features of the natural person; associate the specified features of the user with identification information of the user; and specify a match period which is the candidate period in which the features of the user associated with the identification information of the user match or substantially match the features of the natural person.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to a technique for identifying an image including an image of a predetermined object.

Background Art

[0002] There is a technique for identifying an image including an image of a predetermined natural person. Patent Document 1 discloses a technique for identifying an image including an image of the user based on a user ID (identifier) which is an identifier of the user, position information indicating the position of the user, and feature amounts of the user based on an image of the user's face obtained in advance by photographing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique disclosed in Patent Document 1, it is necessary to photograph the user's face in advance and associate the data of the image obtained by the photographing with the user ID, which has a problem of imposing a burden on the user.

[0005] In order to solve the above problems, an object of the present disclosure is to provide a technique capable of identifying an image including an image of a user without photographing the target user in advance.

Means for Solving the Problems

[0006] The information processing apparatus according to the present disclosure includes an image acquisition unit that acquires data of a captured image, a position acquisition unit that acquires position identification information capable of identifying the position of a user, a first period identification unit that identifies a candidate period, which is a period during which the captured image may include an image of the user, based on the position identification information, a feature amount acquisition unit that acquires feature amounts of natural persons included as images in the captured image corresponding to the candidate period, a feature amount identification unit that identifies the feature amounts of the user based on the acquired feature amounts of the natural persons and associates the identified feature amounts of the user with identification information of the user, and a second period identification unit that identifies a matching period, which is the candidate period in which the feature amounts of the user associated with the identification information of the user match or substantially match the feature amounts of the natural persons.

Advantages of the Invention

[0007] According to the present disclosure, it is possible to identify an image including an image of a target user without previously capturing the target user.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the configurations for solving the problems. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential as means for solving the problems, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] [Embodiment 1] FIG. 1 is a diagram showing an example of the configuration of the information processing system according to Embodiment 1. The information processing system includes a transmitter 101, a photographing device 103, a receiver 104, and an information processing device 105. The user 100, who is the target natural person in this information processing system, holds the transmitter 101. The user 102, who is a companion of the user 100, may or may not hold the transmitter 101. Hereinafter, the user 102 will be described as not holding the transmitter 101. Also, in this embodiment, as an example, assuming that there are two users 102, one of the two users 102 will be denoted as user 102a and the other as user 102b for description.

[0011] The imaging device 103 is composed of a digital still camera, a digital video camera, or the like, captures users 100, 102, etc., and outputs data of an image obtained by the capture (hereinafter referred to as "captured image"). The captured image data output from the imaging device 103 is transmitted to the information processing device 105. In the present embodiment, as an example, assuming that there are three imaging devices 103, the three imaging devices 103 will be described as imaging device 103a, imaging device 103b, and imaging device 103c. Further, in the present embodiment, each of the three imaging devices 103 is installed at different locations in a facility such as an amusement park and will be described as capturing different imaging ranges from each other.

[0012] The transmitter 101 transmits a wireless signal indicating the transmitter ID registered in advance in the transmitter 101. The wireless signal is transmitted from the transmitter 101, for example, periodically. The transmitter 101 is composed of a proximity wireless communication device capable of transmitting a wireless signal such as a Bluetooth (registered trademark) beacon (hereinafter simply referred to as "beacon"). A wireless signal such as a beacon transmitted from such a proximity wireless communication device is a signal capable of roughly identifying the position of the transmitter 101 and is a signal difficult to accurately identify.

[0013] The receiver 104 receives the wireless signal transmitted from the transmitter 101 and outputs information indicating the reception time (hereinafter referred to as "reception time information"), information indicating the transmitter ID indicated by the received wireless signal, and information indicating the receiver ID which is the identifier of the receiver 104 that received the wireless signal. Hereinafter, the information indicating the transmitter ID will be referred to as "transmitter information", and the information indicating the receiver ID will be referred to as "receiver information". The reception time information, transmitter information, and receiver information output from the receiver 104 are transmitted to the information processing device 105. In the present embodiment, the receiver 104 and the imaging device 103 are communicably connected, and each information output from the receiver 104 will be described as being transmitted to the information processing device 105 via the imaging device 103, but the connection form is not limited to this.

[0014] For example, the receiver 104 and the information processing device 105 may be communicably connected, and each piece of information output from the receiver 104 may be directly transmitted to the information processing device 105. Also, in this embodiment, as an example, assuming that there are three receivers 104, the three receivers 104 will be described as the receiver 104a, the receiver 104b, and the receiver 104c. Further, in this embodiment, it will be described assuming that the receiver 104a and the imaging device 103a, the receiver 104b and the imaging device 103b, and the receiver 104c and the imaging device 103c are communicably connected to each other.

[0015] Based on the captured image data received from the imaging device 103 and the reception time information, transmitter information, and receiver information received from the receiver 104, the information processing device 105 identifies a captured image including the image of the user 100 or the users 100 and 102. Further, the information processing device 105 performs image editing using the identified captured image. Note that the numbers of the user 102, the imaging device 103, and the receiver 104 shown in FIG. 1 are merely examples and are not limited to these numbers.

[0016] As described above, the receiver 104 receives the radio signal from the transmitter 101. Therefore, based on whether or not the radio signal from the transmitter 101 is received, it is possible to determine whether or not the transmitter 101, that is, the user 100 who possesses the transmitter 101, exists within a predetermined area. For example, by installing the transmitter 101 such that the predetermined area corresponds to or is substantially equivalent to the imaging range of the imaging device 103, it is possible to determine whether or not the user 100 exists within the imaging range of the imaging device 103 based on whether or not the radio signal from the transmitter 101 is received. Specifically, for example, by installing the receiver 104 on the ground or underground at the center of the imaging range of the imaging device 103, the reception range of the radio signal at the receiver 104 and the imaging range of the imaging device 103 can be made to correspond to each other.

[0017] The position where the receiver 104 is set is not limited to this. For example, the receiver 104 may be installed at any position such as near the imaging device 103. In this case, it is preferable to apply known techniques such as the intensity (RSSI: Received Signal Strength Indicator), arrival time (ToA: Time of Arrival), arrival direction (AoA: Angle of Arrival), or radiation direction (AoD: Angle of Departure) of the received signal. By applying the said known technique, it becomes possible to specify approximately the position of the transmitter 101. Also, a plurality of receivers 104 may be arranged for one imaging range. When a radio signal indicating the same transmitter ID is received by a plurality of receivers 104, it can be determined that the said transmitter ID exists in the area common to the reception ranges of the radio signals in each receiver 104.

[0018] Also, when it is clear that the imaging range is passed at a predetermined timing, such as an attraction that is ridden by a small number of people in the theme park, the receiver 104 may be set at a position far from the imaging range of the imaging device 103. This is because the time when the user 100, or the users 100 and 102, exist in the imaging range of the imaging device 103 can be estimated from the information indicating the time when the receiver 104 received the radio signal and the information indicating the operating status of the attraction, etc.

[0019] Note that in the present embodiment, as an example, the transmitter 101 is described as being configured by a mobile terminal such as a smartphone capable of transmitting a radio signal such as a beacon, but the mobile terminal is merely an example and other devices may also be used. Also, the radio signal transmitted by the transmitter 101 is not limited to a beacon or the like. For example, instead of the receiver ID, the transmitter 101 may transmit to the receiver 104 a radio signal indicating a position specified based on a GNSS (Global Navigation Satellite System) signal such as GPS (Global Positioning System).

[0020] In addition, the imaging device 103 may be provided with a PTZ mechanism. In this case, it is preferable to install a plurality of receivers 104 for one imaging device 103. By installing a plurality of receivers 104 for one imaging device 103, the PTZ mechanism can be controlled based on the information on which receiver 104 has received the radio signal from the transmitter 101 so that the imaging range of the imaging device 103 includes the approximate position of the transmitter 101. Further, it is desirable that the imaging device 103 be a device capable of capturing a high-resolution imaging image for cropping and using the imaging image when editing the imaging image in the information processing device 105.

[0021] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 105 according to Embodiment 1. The information processing device 105 includes a CPU 200, a ROM 201, a RAM 202, an HDD 203, and a communication module 204 as its hardware configuration. The CPU 200 controls the information processing device 105 by reading a program stored in the ROM 201 or the HDD 203 or the like and expanding and executing it in the RAM 202. Specifically, for example, the CPU 200 executes an image analysis process for analyzing an imaging image and an editing process for editing the imaging image. A GPU (Graphics Processing Unit), not shown in FIG. 2, may be used for the image analysis process. Further, for the image analysis process, a processing circuit, not shown in FIG. 2, for image analysis constituted by an ASIC (application specific integrated circuit) or an FPGA (Field Programmable Gate Array) or the like may be used.

[0022] The communication module 204 performs information communication with external devices such as the imaging device 103. Specifically, for example, the communication module 204 receives captured image data transmitted from the imaging device 103. Also, the communication module 204 receives various types of information transmitted from the receiver 104. Specifically, for example, the communication module 204 acquires reception time information, transmitter information, and receiver information transmitted from the receiver 104 via the imaging device 103. In this case, the receiver 104 transmits the reception time information, transmitter information, and receiver information to the imaging device 103, and the imaging device 103 transfers the received information to the information processing device 105. The imaging device 103 may also transmit the transmitter information to the information processing device 105 by attaching the received transmitter information as metadata of the captured image data and transmitting the captured image data with the metadata to the information processing device 105.

[0023] Also, in this embodiment, the description is made assuming that the information processing device 105 performs image analysis processing. However, an edge server (not shown in FIGS. 1 and 2) may be provided between the imaging device 103 and the information processing device 105, and the edge server may perform image analysis processing.

[0024] FIG. 3 is a block diagram showing an example of the logical configuration of each device included in the information processing system according to Embodiment 1. The receiver 104 includes a receiving unit 300. The imaging device 103 includes an imaging unit 301. The information processing device 105 includes a data acquisition unit 302, a period specifying unit 303, a feature amount acquisition unit 304, and an editing unit 305. The receiving unit 300 receives the radio signal transmitted from the transmitter 101, and transmits information indicating the reception time of the radio signal (reception time information), information indicating the transmitter ID indicated by the received radio signal (transmitter information), and receiver information to the information processing device 105. The imaging unit 301 generates an image based on the electrical signal output from the image sensor, and transmits the data of the generated image to the information processing device 105 as captured image data. Note that it is assumed that the captured image data includes information indicating the shooting time, such as the generation time of the image, as metadata. For example, the imaging device 103 always captures a moving image during a predetermined period such as the available period of the facility, and transmits captured image data including information indicating the shooting time of each frame included in the moving image as metadata to the information processing device 105.

[0025] In the present embodiment, the imaging device 103 will be described as always capturing a moving image during a predetermined period. However, the imaging by the imaging device 103 does not necessarily have to be always performed. For example, the imaging device 103 may continuously receive a notification to that effect during the period in which the receiver 104 receives a radio signal from any of the transmitters 101, and perform imaging only during the period in which the notification is continuously received. Further, in the present embodiment, the imaging device 103 will be described as capturing a moving image and outputting the data of the moving image as captured image data. However, the imaging by the imaging device 103 is not limited to a moving image. For example, the imaging device 103 may periodically or irregularly and continuously capture a still image during the above-described predetermined period, and output the data of the still image as captured image data. Further, the imaging device 103 may capture both a moving image and a still image during the above-described predetermined period, and output the data of both as captured image data.

[0026] The data acquisition unit 302 acquires the reception time information, transmitter information, and receiver information output from the receiver 104, and the captured image data output from the imaging device 103. The data acquisition unit 302 registers the acquired transmitter information as metadata of the captured image data, and stores the transmitter information and the captured image data with metadata in the HDD 203. Specifically, the data acquisition unit 302 registers these pieces of information as metadata by associating the transmitter information and the reception time information with the information indicating the shooting time included as metadata in the captured image data.

[0027] The period specifying unit 303 generates validity period information (hereinafter referred to as "image validity period") indicating the valid period in the captured image for each transmitter ID by using the metadata of the captured image data. Specifically, the validity period information is generated by specifying the image validity period for each transmitter 101 by using the reception time information and the transmitter information included in the metadata of the captured image data. The validity period information generated by the period specifying unit 303 is transmitted to the feature amount acquisition unit 304 and the editing unit 305. An example of the transmitter information and the reception time information registered as the metadata 306 in the data acquisition unit 302 and the validity period information will be described later with reference to FIG. 4.

[0028] The feature quantity acquisition unit 304 generates association information based on the expiration date information. Specifically, the feature quantity acquisition unit 304 first refers to the expiration date information, performs image analysis processing on the captured images in the image expiration period corresponding to each transmitter ID, and acquires the feature quantities of natural persons shown in the captured images in the image expiration period. Here, it is assumed that each captured image obtained by capturing with a plurality of imaging devices 103 includes images of the user 100 who holds the transmitter 101 and the user 102 who is a companion of the user 100 in different image expiration periods. Subsequently, the feature quantity acquisition unit 304 compares the feature quantities corresponding to a plurality of natural persons acquired from each captured image in the image expiration period with each other, and identifies all or more feature quantities that are similar to each other. Subsequently, the feature quantity acquisition unit 304 generates association information by associating the identified feature quantities that are similar to each other with the information indicating the transmitter ID, assuming that the identified feature quantities that are similar to each other are the feature quantities of the user 100 or the user 102.

[0029] Specifically, for example, association information is generated by associating a statistical value such as an average value or a median value of a plurality of feature quantities that are similar to each other with the information indicating the transmitter ID as the feature quantity of the user 100 or the user 102. Note that the processing area of the image analysis processing for the captured images in the image expiration period may be the entire image area of the captured images, or may be a predetermined image area within the entire image area of the captured images. An example of the association information will be described later with reference to FIG. 5. Also, an example of the generation process of the association information in the feature quantity acquisition unit 304 will be described later with reference to the flowchart shown in FIG. 7. The association information generated by the feature quantity acquisition unit 304 is transmitted to the editing unit 305.

[0030] The editing unit 305 edits the captured images to generate edited images. Specifically, the editing unit 305 first extracts, for each transmitter ID, a plurality of periods including images of natural persons corresponding to features that match or substantially match the features of the association information from the captured images within the image validity period for each transmitter ID. Subsequently, the editing unit 305 generates an edited image by, for example, splicing together the captured images corresponding to the extracted plurality of periods along a timeline. An example of the editing process in the editing unit 305 will be described later with reference to the flowchart shown in FIG. 8. The data of the edited image generated by the editing unit 305 is stored, for example, in the HDD 203. The information processing apparatus 105 may transmit the data of the edited image to an external device via the communication module 204.

[0031] FIG. 4 is a diagram for explaining an example of the validity period information generated by the period specifying unit 303 according to Embodiment 1. FIG. 4(a) shows a timeline of metadata in a part of the shooting period of each imaging device 103. Each of the plurality of vertical solid lines on the timeline shown in FIG. 4(a) indicates the timing at which the receiver 104 receives a radio signal from the transmitter 101. Specifically, FIG. 4(a) shows, as an example, the timing at which the receiver 104 receives a radio signal from the transmitter 101 with a transmitter ID of "1234". When the radio signal transmitted by the transmitter 101 is a beacon, the transmitter 101 transmits a radio signal at a frequency of about once every several tens of seconds. The period specifying unit 303 specifies, as the image validity period corresponding to the transmitter ID, a predetermined period including the reception time of the beacon in the captured image of the imaging device 103 corresponding to the receiver 104 that has received the beacon transmitted by the transmitter 101 with the transmitter ID of "1234". Here, the predetermined period is, for example, a period from several tens of seconds before the reception time of the first beacon continuously received from one transmitter 101 to several tens of seconds after the reception time of the last beacon.

[0032] In the example shown in FIG. 4(a), the receiver 104a has received the first (initial) radio signal from the transmitter 101 with the transmitter ID "1234" around 10:30:20, and has received the second (last) radio signal from the transmitter 101 around 10:30:50. In this case, for example, the period specifying unit 303 specifies, for the captured image of the imaging device 103a corresponding to the receiver 104a, the period from 10:30:00 to 10:31:30 as the image valid period. FIG. 4(b) shows an example of the valid period information generated by the period specifying unit 303. The period specifying unit 303 generates valid period information for each transmitter ID. FIG. 4(b) shows, as an example, the valid period information corresponding to the transmitter 101 with the transmitter ID "1234". As shown in FIG. 4(b), the valid period information is, for example, information indicating the start and end times of the image valid period and information indicating the receiver 104 that has received the radio signal from the transmitter 101, which are associated with each other.

[0033] FIG. 5 is a diagram for explaining an example of the association information generated by the feature amount acquisition unit 304 according to Embodiment 1. FIG. 5(a) shows an example of the captured image 500. The captured image 500 includes the image 501 of the user 100, the image 502a of the user 102a, and the image 502b of the user 102b. The feature amount acquisition unit 304 acquires the respective feature amounts of the users 100 and 102 by performing image analysis processing on the captured image 500. Further, the feature amount acquisition unit 304 performs the same processing as the image analysis processing on the captured image 500 even for the captured image having an image valid period different from the image valid period including the captured image 500, thereby acquiring the respective feature amounts of the users 100 and 102. Furthermore, the feature amount acquisition unit 304 compares the plurality of acquired feature amounts with each other to specify the feature amounts (hereinafter referred to as "similar feature amounts") whose number or ratio of mutually similar feature amounts is equal to or greater than a predetermined threshold. Furthermore, the feature amount acquisition unit 304 generates association information by associating the specified similar feature amounts with the information indicating the transmitter ID, assuming that the specified similar feature amounts are the feature amounts of the user 100 or the user 102 who is a companion of the user 100.

[0034] The method for identifying the feature amounts of user 100 or user 102 by the feature amount acquisition unit 304 is not limited to the method described above. For example, as in the case of a group photo, one or more imaging devices 103 capable of imaging users 100 and 102 are installed so that the images of users 100 and 102 are included in a predetermined composition in the captured image. In this case, it is desirable to mark a sign 510 indicating the standing position on the floor or the like so that users 100 and 102 are imaged at a predetermined standing position. Further, in order to ensure that the beacon transmitted from the transmitter 101 is received by the receiver 104, it is desirable to issue a notification prompting users 100 and 102 to stay at the standing position for at least the period of the beacon transmission cycle. Further, measures such as providing a fence for restricting entry into the imaging range may be taken so that the captured image does not include images of natural persons other than users 100 and 102. Note that the image 511 included in the captured image 500 is an image of the fence. In this way, the feature amounts of user 100 or user 102 can be identified by performing image analysis processing on the images of natural persons included in the captured image 500.

[0035] FIG. 5(b) shows an example of the association information. As shown in FIG. 5(b), the association information is such that, for each transmitter 101, information indicating the transmitter ID (ID: 1234) and the feature amounts of users 100 and 102 are associated with each other.

[0036] FIG. 6 is a flowchart showing an example of a processing flow in the information processing apparatus 105 according to Embodiment 1. Note that the "S" attached to the beginning of the reference numerals means steps (processes). Further, the processing of each step shown in the flowchart of FIG. 6 is realized by the CPU 200 reading a predetermined program from the ROM 201 or the HDD 203 and expanding it in the RAM 202 and then the CPU 200 executing the program.

[0037] First, in S601, the data acquisition unit 302 acquires the captured image data output from each imaging device 103, and the reception time information, transmitter information, and receiver information output from each receiver 104. Next, in S602, the period specifying unit 303 specifies the image valid period in each captured image data acquired in S601 for each transmitter ID of the transmitter 101 based on the reception time information, transmitter information, and receiver information acquired in S601, and generates valid period information. Next, in S603, the feature quantity acquisition unit 304 performs image analysis processing on the captured image data in all the image valid periods specified in S602 among the captured image data acquired in S601, thereby acquiring the feature quantities of the natural persons captured in the captured images.

[0038] Next, in S604, the feature quantity acquisition unit 304 specifies the feature quantities of the user 100 who holds the transmitter 101 and the feature quantities of the user 102 who is a companion of the user 100 for each transmitter ID of the transmitter 101 based on the feature quantities acquired in S603. Specifically, in S604, first, the feature quantity acquisition unit 304 compares the feature quantities of the natural persons corresponding to the plurality of image valid periods specified in S602 and acquired in S603 for each transmitter ID of the transmitter 101 with each other. In S604, subsequently, the feature quantity acquisition unit 304 specifies the feature quantities (similar feature quantities) whose number or ratio of mutually similar feature quantities is equal to or greater than a predetermined threshold value as a result of the comparison of the feature quantities. In S604, subsequently, the feature quantity acquisition unit 304 generates association information associating the specified similar feature quantities with the information indicating the transmitter ID on the assumption that the specified similar feature quantities are the feature quantities of the user 100 or the user 102. Next, in S605, the editing unit 305 edits each captured image data acquired in S601 based on the feature quantities acquired in S603 and the feature quantities of the user 100 or the user 102 specified in S604, and generates an edited image. After S605, the information processing apparatus 105 ends the processing of the flowchart shown in FIG. 6.

[0039] FIG. 7 is a flowchart showing an example of the processing flow of the feature amount acquisition unit 304 according to Embodiment 1, and is a flowchart showing an example of the processing flow in S603 shown in FIG. 6. In S603, the feature amount acquisition unit 304 repeatedly executes a series of processes shown in the flowchart of FIG. 7 for each transmitter ID and for each image validity period. That is, the feature amount acquisition unit 304 executes the processes of the flowchart shown in FIG. 7 for each image validity period with respect to the captured image data of the imaging device 103 corresponding to each receiver 104 associated with each image validity period based on the validity period information generated in S602.

[0040] For example, when an image validity period from 10:30:00 to 10:31:30 is associated with the receiver 104a, the feature amount acquisition unit 304 executes the processes of the flowchart shown in FIG. 7 for the following captured image data. For example, in this case, the feature amount acquisition unit 304 performs the processes of the flowchart shown in FIG. 7 for each frame in the period from 10:30:00 to 10:31:30 in the captured image data of the imaging device 103a corresponding to the receiver 104a.

[0041] Note that the feature amount acquisition unit 304 does not need to execute the process for all frames in the period from 10:30:00 to 10:31:30, and may execute the process for one or a plurality of frames among the plurality of frames in the period. Specifically, for example, the feature amount acquisition unit 304 may execute the process only for the frame at the intermediate time point in the period, that is, the frame at 10:30:45, among the plurality of frames in the period. This is because the frame captured at the intermediate time point is likely to include the images of the user 100 and the user 102. Thus, by limiting the frames to be processed, the amount of calculation related to the process can be reduced. Hereinafter, as an example, the feature amount acquisition unit 304 will be described as performing the processes of the flowchart shown in FIG. 7 for all frames in the image validity period.

[0042] First, in S701, the feature amount acquisition unit 304 extracts the images of natural persons included in the captured images (each frame) during the image valid period by performing image analysis processing on the captured images. For example, the feature amount acquisition unit 304 performs tracking of the images of natural persons between frames to identify the same natural person, and associates the images of natural persons included in each frame with each other for each natural person. Next, in S702, the feature amount acquisition unit 304 selects an arbitrary image from among the one or more images of natural persons extracted in S701. Next, in S703, the feature amount acquisition unit 304 acquires the feature amounts of the natural person by performing image analysis processing on the series of images of natural persons included in each frame selected in S702. Specifically, for example, the feature amount acquisition unit 304 performs image analysis processing on the images in which the face part is captured more frontally among the series of images of natural persons included in each frame selected in S702, and acquires the feature amounts of the face part of the natural person. Next, in S704, the feature amount acquisition unit 304 associates the acquired feature amounts with the information indicating the transmitter ID and the information indicating the image valid period, and stores them in the HDD 203 or the like.

[0043] Next, at S705, the feature amount acquisition unit 304 determines whether all the images of natural persons have been selected at S702. If it is determined at S702 that at least a part of all the images of natural persons has not been selected, the feature amount acquisition unit 304 repeatedly executes the processes from S702 to S705 until it is determined at S705 that all the images of natural persons have been selected. In this case, at S702, the feature amount acquisition unit 304 selects any unselected image among the one or more images of natural persons extracted at S701. When it is determined at S705 that all the images of natural persons have been selected, the feature amount acquisition unit 304 ends the process of the flowchart shown in FIG. 7, that is, the process of S603 shown in FIG. 6. As described above, in S603, the feature amount acquisition unit 304 repeatedly executes a series of processes shown in the flowchart of FIG. 7 for each transmitter ID and for each image validity period in each captured image. Also, the processes of S602 and S603 shown in FIG. 6 may be executed after the acquisition of all the captured image data is completed after reaching the end of a predetermined period during which the imaging device 103 such as the available period of the facility continuously captures images, or may be continuously executed over the predetermined period.

[0044] FIG. 8 is a flowchart showing an example of the processing flow of the feature amount acquisition unit 304 according to Embodiment 1, and is a flowchart showing an example of the flow of the feature amount specification process in S604 shown in FIG. 6. The process of the flowchart shown in FIG. 8 is executed after the process of S603, or when the processes of S602 and S603 are continuously executed over the above-described predetermined period, after the processes of S602 and S603 for a part of the predetermined period are executed. Also, in S604, the process of the flowchart shown in FIG. 8 is repeatedly executed for each transmitter ID.

[0045] First, in S801, the feature amount acquisition unit 304 compares the feature amounts of a plurality of natural persons associated with information indicating the same transmitter ID in S704 shown in FIG. 7 with each other. Specifically, for example, the feature amount acquisition unit 304 compares the feature amount of each natural person with the feature amounts of each natural person associated with each image validity period different from the image validity period associated with the feature amount of the natural person. Based on this comparison, the feature amount acquisition unit 304 calculates the number of feature amounts of each natural person similar to the feature amount of each natural person, or the ratio of the number of feature amounts of each natural person similar to the feature amount of each natural person to the number of all image validity periods (hereinafter simply referred to as "ratio of feature amounts"). Next, in S802, based on the result of the comparison in S801, the feature amount acquisition unit 304 specifies, for example, feature amounts equal to or greater than a predetermined threshold such as 70% or more of the above-mentioned ratio of feature amounts as similar feature amounts. Next, in S803, assuming that the similar feature amounts specified in S802 are the feature amounts of user 100 or user 102, the feature amount acquisition unit 304 generates association information associating the specified similar feature amounts with the information indicating the transmitter ID. Note that the specific processing for specifying the feature amounts of user 100 or user 102 in S802 and S803 is an example, and this specific processing may be performed by the method using the above-mentioned group photo.

[0046] Next, in S804, the feature amount acquisition unit 304 uses a known image analysis technique to obtain the degree of association between the image of the natural person corresponding to the similar feature amount determined to be the feature amount of user 100 or user 102 in S803 and the images of other natural persons included in the same frame. Here, the degree of association refers to a value obtained by quantifying states such as the distance between natural persons, whether natural persons are lined up facing the same direction, whether they are facing each other and talking, whether they are present in the same area at the same time, and whether they are holding hands, etc. as indicators. The states to be quantified as the degree of association are not limited to those described above, and may be, for example, whether one natural person is carrying or hugging the other natural person. The feature amount acquisition unit 304 executes image analysis processing for each frame to detect the above-mentioned states and quantify them to obtain the degree of association.

[0047] Next, at S805, the feature quantity acquisition unit 304 identifies feature quantities that are a certain ratio or more, such as 30% or more of the above-described feature quantities, and whose relevance is higher than a certain value such as the average value of all relevance levels. The feature quantity acquisition unit 304 generates association information that associates the identified feature quantity with information indicating the transmitter ID, assuming that the identified feature quantity is a feature quantity of the companion user 102 of the user 100 who is closely related to the user 100 or the user 102. By associating such a feature quantity with the information indicating the transmitter ID, it is possible to more reliably associate the feature quantity of the user 102 who does not possess the transmitter 101, who is a companion of the user 100 who possesses the transmitter 101, with the information indicating the transmitter ID. After S805, the feature quantity acquisition unit 304 ends the processing of the flowchart shown in FIG. 8, that is, the processing of S604 shown in FIG. 6.

[0048] Note that if the process of S604 is executed after the processes of S602 and S603 for a part of the above-described predetermined period and association information is generated during the predetermined period, the information processing apparatus 105 may perform the following process. Specifically, for example, in this case, the information processing apparatus 105 transmits data of an image of the face of a natural person corresponding to the feature quantity of the natural person associated with the information indicating the transmitter ID of the transmitter 101 in the association information to the user 100 who possesses the transmitter 101. By checking the image before the process of S605, the user 100 can check whether the images of the user 100 and the user 102 are accurately identified in the information processing apparatus 105. Also, if the images of the user 100 and the user 102 are not accurately identified in the information processing apparatus 105, the user 100 may notify the information processing apparatus 105 that the images of the user 100 and the user 102 are not accurately identified. When the information processing apparatus 105 receives the notification, for example, the information processing apparatus 105 may perform the process of S604 again to generate the association information anew.

[0049] FIG. 9 is a flowchart showing an example of a processing flow in the editing unit 305 according to Embodiment 1, and is a flowchart showing an example of the processing flow in S605 shown in FIG. 6. The processing of the flowchart shown in FIG. 9 is executed after the processing of S604. Also, in S605, the processing of the flowchart shown in FIG. 9 is performed for each transmitter ID. First, in S901, the editing unit 305 selects an arbitrary image validity period corresponding to the same transmitter ID.

[0050] Next, in S902, the editing unit 305 compares the feature amount included in the association information with the feature amount acquired in S703 shown in FIG. 7 in the photographed image data of the image validity period selected in S901. Next, in S903, based on the result of the comparison in S902, the editing unit 305 determines whether or not at least one of the images of the user 100 and the user 102 is included in the photographed image during the image validity period selected in S901. Specifically, the editing unit 305 makes the above determination by determining whether or not any of the feature amounts included in the association information matches the feature amount in the photographed image during the image validity period selected in S901. Here, the match may include the meaning of a substantially match. If it is determined in S903 that it is included, in S904, the editing unit 305 stores the photographed image data corresponding to the image validity period selected in S901 as the photographed image data to be edited in the HDD 203 or the like.

[0051] Note that when the editing department 305 saves the captured image data to be edited in S904, the following processing may be performed. Specifically, for example, the editing department 305 first uses a known image analysis technique to detect a section in the captured images within the image validity period where a natural person shows a suitable reaction as a captured image. Subsequently, the editing department 305 extracts the captured image data corresponding to the detected section from the captured image data within the image validity period. Subsequently, the editing department 305 saves the extracted captured image data as the captured image data to be edited. Note that specifically, the section where a natural person shows a suitable reaction as a captured image is a section where the natural person is smiling, surprised, staring, or having a conversation, etc. Also, the length of the section detected by the editing department 305 where a natural person shows a suitable reaction as a captured image may be a predetermined length such as 4 seconds.

[0052] Also, when the editing department 305 saves the captured image data to be edited in S904, the following processing may be performed. Specifically, for example, the editing department 305 crops a part of the image area in the captured images within the image validity period to extract the captured image data after cropping, and saves the extracted captured image data as the captured image data to be edited. In this case, the editing department 305 may perform the following processing to determine the cropping range. Specifically, for example, the editing department 305 determines whether another user 100 or user 102 exists near the user 100 or user 102 who shows a suitable reaction.

[0053] When it is determined that there are other users 100 or 102 nearby, the editing unit 305 crops a part of the image area in the captured image of the image validity period as follows. For example, in this case, the editing unit 305 crops the image area in the captured image so that the image of the user 100 or 102 showing a suitable reaction and the image of other users 100 or 102 existing near the user 100 or 102 are included. When the image of the user 100 or 102 showing a suitable reaction moves significantly within the image area over time, the editing unit 305 may track and crop the image of the user 100 or 102 showing a suitable reaction. Also, when it is determined that there are no other users 100 or 102 nearby, the editing unit 305 crops the image area in the captured image so that only the image of the user 100 or 102 showing a suitable reaction is included. When the image of the user 100 or 102 showing a suitable reaction moves significantly within the image area over time, the editing unit 305 may track and crop the image of the user 100 or 102 showing a suitable reaction.

[0054] After S904, or when it is determined that it is not included in S903, in S905, the editing unit 305 determines whether all the image validity periods corresponding to the same transmitter ID in S901 have been selected. When it is determined in S905 that at least a part of all the image validity periods has not been selected, the editing unit 305 repeatedly executes the processing from S901 to S905 until it is determined in S905 that all the image validity periods have been selected. In this embodiment, it has been described that in S905, it is determined whether all the image validity periods have been selected, but the determination process in S905 is not limited to the above conditions. For example, the editing unit 305 may determine in S905 whether a sufficient amount of captured image data to be edited has been collected as a material for generating the edited image. When it is determined in S905 that a sufficient amount of captured image data to be edited has not been collected, the editing unit 305 repeatedly executes the processing from S901 to S905 until it is determined in S905 that a sufficient amount of captured image data to be edited has been collected.

[0055] Also, when there are many users 102, in order to prevent only the captured image data at the same shooting angle or similar positions from being saved as the captured image data to be edited, for example, the editing unit 305 may perform the following determination in S905. Specifically, in S905, the editing unit 305 determines whether the captured image data of each imaging device 103 with a sufficient length, such as 20 seconds or more, is saved as the captured image data to be edited. When it is determined in S905 that the captured image data of a certain imaging device 103 does not have a sufficient length, the editing unit 305 repeatedly executes the processes from S901 to S905 until the captured image data of the imaging device 103 satisfies the above conditions.

[0056] When it is determined in S905 that all the image validity periods have been selected, in S906, the editing unit 305 generates edited image data by arranging and joining the captured image data to be edited saved in S904 in chronological order. Note that when generating the edited image data, the editing unit 305 may generate the edited image data including the captured image data of an object symbolizing the entrance of the facility or the facility, the captured image data of the participated event, etc., other than the captured image data to be edited. Also, when an image obtained by cropping a part of the image area of the captured image is saved as the captured image data to be edited, the editing unit 305 may generate the edited image data including the captured image data before cropping so that the overall state can be understood.

[0057] Also, when the user 102 who is a companion of the user 100 having the transmitter 101 has a transmitter 101 with a transmitter ID different from that of the transmitter 101, the editing unit 305 may generate the edited image data as follows. In this case, for example, the editing unit 305 merges the captured image data corresponding to two or more transmitters 101 to generate one piece of edited image data. For example, when two children in a family of four have the transmitter 101, it is desirable to generate one piece of edited image data in which the captured image data to be edited corresponding to each of the two transmitters 101 is merged.

[0058] In this case, the editing unit 305 may execute the following processing before the processing of S901. For example, the editing unit 305 first identifies the association information corresponding to another transmitter 101 in which all the feature amounts included in the association information corresponding to a certain transmitter 101 match or substantially match. Subsequently, the editing unit 305 merges the expiration date information corresponding to all the transmitters 101 in which all the feature amounts included in the association information match or substantially match, and updates the expiration date information by filling in the image expiration dates that only correspond to any one of the transmitters 101. By the editing unit 305 executing the processing after S901 after the update of the expiration date information, one edited image data in which the photographed image data to be edited corresponding to each of the two transmitters 101 is merged is generated.

[0059] According to the information processing apparatus 105 configured as described above, an image including an image of a target user can be specified without previously photographing the target user. Further, according to the information processing apparatus 105, an edited image can be generated based on a photographed image including an image of at least one of the users 100 and 102 without previously photographing the target user.

[0060] [Embodiment 2] In the first embodiment, the form in which the transmitter 101 held by the user 100 communicates with the receiver 104 installed in the facility has been described. The transmitter 101 and the receiver 104 may be reversed. That is, a device corresponding to the transmitter 101 installed in the facility transmits by a radio signal indicating a pre-registered transmitter ID, the user 100 holds a device corresponding to the receiver 104, and the device receives the radio signal indicating the transmitter ID. The device corresponding to the receiver 104 held by the user 100 is configured by a mobile terminal such as a smartphone. In this case, the mobile terminal transmits information indicating the transmitter ID (transmitter information) indicated by the received radio signal, information indicating the reception time of the radio signal (reception time information), and information indicating the mobile terminal ID for identifying the mobile terminal (hereinafter referred to as "terminal information") to the information processing device 105. The data acquisition unit 302 of the information processing device 105 acquires the photographed image data output from each photographing device 103, as well as the reception time information, the transmitter information, and the terminal information output from the mobile terminal.

[0061] [Modification Example 1 of Embodiment 2] The facility may not install a device corresponding to the transmitter 101, and the mobile terminal held by the user may transmit information indicating a position specified based on a GNSS signal such as GPS (hereinafter referred to as "position information") to the information processing device 105. In this case, the data acquisition unit 302 of the information processing device 105 acquires the photographed image data output from each photographing device 103, as well as the position information, the time information corresponding to the position information, and the terminal information output from the mobile terminal. Further, the data acquisition unit 302 registers the time information and the terminal information in association with each other in the metadata of the photographed image data of the photographing device 103 whose photographing range includes the position indicated by the position information.

[0062] [Modification Example 2 of Embodiment 2] When it is clear that a user 100 passes through the shooting range of the shooting device 103 at a predetermined timing, for example, in an attraction where a small number of people ride in a theme park, the image processing system does not necessarily have to have the transmitter 101 and the receiver 104. In this case, for example, it may be substituted by reading, with a reading device, an image such as a barcode or a QR code (registered trademark) that can identify the user 100, formed on a recording medium such as a ticket held by the user 100. The image is not limited to the one formed on the recording medium, and may be an image displayed on the display device of the mobile terminal. The reading device estimates the time when the user 100 and the user 102 reach the shooting range of the shooting device 103 from the identification information indicated by the read image, the information indicating the time when the reading is performed, the information indicating the operating status of the attraction, and the like. The reading device transmits information indicating the estimated time (hereinafter referred to as "estimated time information"), information indicating a reading device ID that can identify the reading device (hereinafter referred to as "reading device information"), and the identification information to the information processing device 105. In this case, the data acquisition unit 302 of the information processing device 105 acquires the captured image data output from each shooting device 103, as well as the estimated time information, the identification information, and the reading device information output from the reading device. Further, the data acquisition unit 302 registers the estimated time information and the identification information in association with each other in the metadata of the captured image data of the shooting device 103 corresponding to the reading device.

[0063] [Modification Example 3 of Embodiment 2] When it is clear that the above-mentioned user 100 passes through the shooting range of the shooting device 103 at a predetermined timing, it may be substituted by using the shooting device of the mobile terminal possessed by the user 100 to read an image such as a barcode or a QR code installed in the facility. The image may be formed on a storage medium such as a signboard installed in the facility, or may be displayed on a display device installed in the facility. In this case, the mobile terminal transmits information such as the information indicated by the image, the terminal information, and the information indicating the time when the image was read to the information processing device 105. Further, the information processing device 105 receives these information and uses these information to estimate the time when the user 100 and the user 102 reach the shooting area of the shooting device 103.

[0064] [Other Embodiments] The information processing device 105 according to Embodiment 1 or 2 identifies the feature amounts corresponding to the user 100 and the user 102 by using the feature amounts of the facial parts of natural persons. However, the feature amounts used for the identification are not limited to the feature amounts of the facial parts of natural persons. For example, the information processing device 105 may acquire a plurality of feature amounts from the front and rear frames in the captured image, and select and acquire feature amounts having a certain or more difference so that only feature amounts having substantially the same feature amounts between the feature amounts of each frame are not registered. Further, in addition to the feature amounts of the facial parts, the information processing device 105 may acquire the feature amounts of the whole body such as the body part, and manage by associating the feature amounts of the facial parts with the feature amounts of the whole body. According to the information processing device 105 configured in this way, the identification accuracy of the feature amounts corresponding to the user 100 and the user 102 can be improved by identifying the feature amounts most similar to the feature amounts obtained from the captured images in other image valid periods.

[0065] The present disclosure can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device reading and executing the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0066] In addition, the numerical values, processing timings, processing orders, display screen configurations, information to be displayed, display forms, processing subjects, etc. used in the description of the above-described embodiments are given as examples for the purpose of specific description. Therefore, the description of the above-described embodiments is not intended to limit the scope of the present disclosure to the scope of the description. Further, with respect to the present disclosure, within the scope of the disclosure, a free combination of each embodiment, a modification of any component of each embodiment, or an omission of any component in each embodiment is possible.

[0067] [Configuration of the Present Disclosure] The present disclosure includes the following configurations, methods, and programs.

[0068] <Configuration 1> Image acquisition means for acquiring data of a captured image, Position acquisition means for acquiring position identification information capable of identifying the position of a user, First period identification means for identifying a candidate period, which is a period during which the captured image may include an image of the user, based on the position identification information, Feature amount acquisition means for acquiring a feature amount of a natural person included as an image in the captured image corresponding to the candidate period, Feature amount identification means for identifying the feature amount of the user based on the acquired feature amount of the natural person and associating the identified feature amount of the user with the identification information of the user, Second period identification means for identifying a matching period, which is the candidate period in which the feature amount of the user associated with the identification information of the user and the feature amount of the natural person match or substantially match, An information processing apparatus, characterized by comprising the above.

[0069] <Configuration 2> The feature amount identification means identifies, as the feature amount of the user, the feature amount of the natural person included as an image in a predetermined image area in the captured image among the feature amounts of the natural person acquired from the captured image corresponding to the candidate period. The information processing apparatus according to Configuration 1, characterized by the above.

[0070] <Configuration 3> The feature quantity specifying means specifies, as the feature quantity of the user, the feature quantity of the natural person in which the ratio of the feature quantities that are similar to each other is large, by comparing the feature quantities of the natural person obtained from the captured images corresponding to each of the plurality of candidate periods. The information processing apparatus according to Configuration 1, characterized by the above.

[0071] <Configuration 4> The feature quantity specifying means calculates the ratio by dividing the number of the feature quantities of the natural person in which the feature quantities are similar to each other by the number of the candidate periods. The information processing apparatus according to Configuration 3, characterized by the above.

[0072] <Configuration 5> The feature quantity specifying means acquires the degree of association between the feature quantity of the user specified based on the feature quantity of the natural person and the feature quantity of another natural person, and determines that the feature quantity of another natural person having a high degree of association with the feature quantity of the user belongs to a user group to which the user belongs, and specifies it as the feature quantity of another user different from the user, and associates the specified feature quantity of the other user with the identification information of the user. The second period specifying means specifies, as the matching period, the candidate period in which at least one of the feature quantity of the user and the feature quantity of the other user associated with the identification information of the user matches or substantially matches the feature quantity of the natural person. The information processing apparatus according to any one of Configurations 1 to 4, characterized by the above.

[0073] <Configuration 6> The feature quantity specifying means executes image analysis processing on the captured image, thereby detecting at least one of the distance between the user and the other natural person, the state in which the user and the other natural person are lined up side by side, the state in which the user and the other natural person are facing each other and having a conversation, the state in which the user and the other natural person are simultaneously present in the same area, the state in which the user and the other natural person are holding hands with each other, and the state in which one of the user and the other natural person is carrying or hugging the other, and acquiring the relevance based on the result of the detection. The information processing apparatus according to Configuration 5, characterized by the above.

[0074] <Configuration 7> Further comprising generation means for generating an edited image by splicing together the captured images corresponding to some or all of some or all of the plurality of matching periods. The information processing apparatus according to any one of Configurations 1 to 6, characterized by the above.

[0075] <Configuration 8> The generation means generates the edited image so as to include a period in which the user is smiling, a period in which the user is surprised, a period in which the user is staring, or a period in which the user is having a conversation, among the captured images corresponding to each of the plurality of matching periods. The information processing apparatus according to Configuration 7, characterized by the above.

[0076] <Configuration 9> The generation means crops an image area including the image of the user for at least a part of the captured images corresponding to each of the plurality of matching periods, and generates the edited image including the cropped captured images. The information processing apparatus according to Configuration 7 or 8, characterized by the above.

[0077] <Configuration 10> When all the feature amounts of all the users associated with each of the plurality of different identification information match or substantially match each other, the generation means merges the candidate periods corresponding to each of all the identification information, and among the candidate periods after merging, a plurality of candidate periods in which the feature amounts of the users associated with each of the plurality of identification information match or substantially match the feature amounts of the natural person are used as the matching periods when generating the edited image, and generates the edited image. The information processing apparatus according to any one of Configurations 7 to 9, characterized by the above.

[0078] <Configuration 11> The identification information is transmitter information capable of identifying a transmitter possessed by the user. The position specifying information is receiver information capable of identifying a receiver whose range of the reception area of a radio signal related to the transmitter information transmitted from the transmitter is known, and which receives the radio signal transmitted from the transmitter. The position acquisition means acquires the transmitter information and the receiver information output from the receiver when the receiver receives the radio signal transmitted from the transmitter. The information processing apparatus according to any one of Configurations 1 to 10, characterized by the above.

[0079] <Configuration 12> The identification information is receiver information capable of identifying a receiver possessed by the user and whose range of the reception area of a radio signal is known. The position specifying information is transmitter information capable of identifying a transmitter whose position is known and which transmits the radio signal to the receiver. The position acquisition means acquires the transmitter information and the receiver information output from the receiver when the receiver receives the radio signal transmitted from the transmitter. The information processing apparatus according to any one of Configurations 1 to 10, characterized by the above.

[0080] <Configuration 13> The identification information is information represented by an image printed on a recording medium held by the user or an image displayed on a mobile terminal held by the user, The position specifying information is reader device information that can identify a reader device with a known position for reading an image corresponding to the identification information, The position acquisition means acquires the identification information and the reader device information output from the reader device when the reader device reads an image representing the identification information, The information processing apparatus according to any one of Configurations 1 to 10, characterized in that.

[0081] <Configuration 14> The identification information is mobile terminal information that can identify a mobile terminal held by the user, The position specifying information is information represented by an image printed on a recording medium with a known position or an image displayed on a display device with a known position, The position acquisition means acquires the mobile terminal information and the position specifying information output from the mobile terminal when a photographing device included in the mobile terminal reads an image representing the identification information, The information processing apparatus according to any one of Configurations 1 to 10, characterized in that.

[0082] <Configuration 15> The identification information is mobile terminal information that can identify a mobile terminal held by the user, The position specifying information is information indicating the position of the mobile terminal, The position acquisition means acquires the mobile terminal information and information indicating the position of the mobile terminal output from the mobile terminal, The information processing apparatus according to any one of Configurations 1 to 10, characterized in that.

[0083] <Method> An image acquisition step of acquiring data of a photographed image, A position acquisition step of acquiring position specifying information capable of specifying the position of the user, A first period specifying step of specifying a candidate period, which is a period during which the captured image may include an image of the user, based on the position specifying information; A feature amount acquisition step of acquiring feature amounts of natural persons included as images in the captured images corresponding to the candidate period; A feature amount specifying step of specifying the feature amounts of the user based on the acquired feature amounts of the natural persons and associating the specified feature amounts of the user with the identification information of the user; A second period specifying step of specifying a matching period, which is the candidate period in which the feature amounts of the user associated with the identification information of the user match or substantially match the feature amounts of the natural persons; An information processing method characterized by including the above.

[0084] <Program> A program for causing a computer to function as the information processing apparatus according to any one of Configurations 1 to 15.

Explanation of Signs

[0085] 105 Information processing apparatus 302 Data acquisition unit 303 Period specifying unit 304 Feature amount acquisition unit

Claims

1. image acquisition means for acquiring data of a captured image; position acquisition means for acquiring position identification information capable of identifying the position of a user; first period identification means for identifying a candidate period which is a period during which the captured image may include an image of the user, based on the position identification information; feature amount acquisition means for acquiring feature amounts of natural persons included as images in the captured image corresponding to the candidate period; feature amount identification means for identifying the feature amount of the user based on the acquired feature amounts of the natural persons, and associating the identified feature amount of the user with identification information of the user; second period identification means for identifying a matching period which is the candidate period in which the feature amount of the user associated with the identification information of the user and the feature amount of the natural person match or substantially match; An information processing apparatus, characterized by comprising the above.

2. The feature amount identification means identifies, as the feature amount of the user, the feature amount of the natural person included as an image in a predetermined image area in the captured image, among the feature amounts of the natural persons acquired from the captured image corresponding to the candidate period. The information processing apparatus according to Claim 1, characterized by the above.

3. The feature amount identification means identifies, as the feature amount of the user, the feature amount of the natural person in which the ratio of the feature amounts being similar to each other is high, by comparing the feature amounts of the natural persons acquired from the captured images corresponding to each of the plurality of candidate periods. The information processing apparatus according to Claim 1, characterized by the above.

4. The feature amount identification means calculates the ratio by dividing the number of the feature amounts of the natural persons in which the feature amounts are similar to each other by the number of the candidate periods. The information processing apparatus according to Claim 3, characterized by the above.

5. The feature amount identification means acquires the degree of association between the feature amount of the user identified based on the feature amount of the natural person and the feature amounts of other natural persons, and identifies, as the feature amounts of other users belonging to a user group to which the user belongs, the feature amounts of other natural persons having a high degree of association with the feature amount of the user, and associates the identified feature amounts of the other users with the identification information of the user. The second period identification means identifies, as the matching period, the candidate period in which at least any one of the feature amount of the user and the feature amounts of the other users associated with the identification information of the user and the feature amount of the natural person match or substantially match. The information processing apparatus according to Claim 1, characterized by the above. The information processing apparatus according to claim 1, characterized by

6. The feature quantity specifying means executes image analysis processing on the captured image, thereby detecting at least one of the distance between the user and the other natural person, the state where the user and the other natural person are lined up side by side, the state where the user and the other natural person are facing each other and having a conversation, the state where the user and the other natural person are simultaneously present in the same area, the state where the user and the other natural person are holding hands with each other, and the state where one of the user and the other natural person is carrying or hugging the other, and obtaining the relevance based on the result of the detection. The information processing apparatus according to claim 5, characterized by

7. Further comprising generation means for generating an edited image by splicing together the captured images corresponding to some or all of some or all of the plurality of coincidence periods. The information processing apparatus according to claim 1, characterized by

8. The generation means generates the edited image so as to include a period in which the user is smiling, a period in which the user is surprised, a period in which the user is staring, or a period in which the user is having a conversation, among the captured images corresponding to each of the plurality of coincidence periods. The information processing apparatus according to claim 7, characterized by

9. The generation means crops an image area including the image of the user for at least a part of the captured images corresponding to each of the plurality of coincidence periods, and generates the edited image including the cropped captured images. The information processing apparatus according to claim 7, characterized by

10. When the feature quantities of all the users associated with each of the plurality of different identification information coincide or substantially coincide with each other, the generation means merges the candidate periods corresponding to each of all the identification information, and among the candidate periods after the merge, the candidate periods in which the feature quantities of the users associated with each of the plurality of identification information coincide or substantially coincide with the feature quantities of the natural person are used as the coincidence periods when generating the edited image to generate the edited image. The information processing apparatus according to claim 7, characterized by

11. The identification information is transmitter information capable of identifying a transmitter possessed by the user. The position specifying information includes receiver information that can identify a receiver whose range of the reception area of a radio signal regarding the transmitter information transmitted from the transmitter is known, and that receives the radio signal. The position acquisition means acquires the transmitter information and the receiver information output from the receiver when the receiver receives the radio signal transmitted from the transmitter. The information processing apparatus according to claim 1, characterized in that.

12. The identification information is receiver information that can identify a receiver whose range of the reception area of a radio signal is known and that the user possesses. The position specifying information is transmitter information that can identify a transmitter whose position is known and that transmits the radio signal to the receiver. The position acquisition means acquires the transmitter information and the receiver information output from the receiver when the receiver receives the radio signal transmitted from the transmitter. The information processing apparatus according to claim 1, characterized in that.

13. The identification information is information represented by an image printed on a recording medium possessed by the user or an image displayed on a portable terminal possessed by the user. The position specifying information is reader information that can identify a reader whose position is known and that reads an image corresponding to the identification information. The position acquisition means acquires the identification information and the reader information output from the reader when the reader reads an image representing the identification information. The information processing apparatus according to claim 1, characterized in that.

14. The identification information is portable terminal information that can identify a portable terminal possessed by the user. The position specifying information is information represented by an image printed on a recording medium whose position is known or an image displayed on a display device whose position is known. The position acquisition means acquires the portable terminal information and the position specifying information output from the portable terminal when an imaging device included in the portable terminal reads an image representing the identification information. The information processing apparatus according to claim 1, characterized in that.

15. The identification information is portable terminal information that can identify a portable terminal possessed by the user. The position specifying information is information indicating the position of the portable terminal. The position acquisition means acquires the portable terminal information and the information indicating the position of the portable terminal output from the portable terminal. The information processing apparatus according to claim 1, characterized in that.

16. An image acquisition step of acquiring data of a captured image; A position acquisition step of acquiring position identification information capable of identifying a user's position; A first period identification step of identifying a candidate period which is a period during which the captured image may include an image of the user based on the position identification information; A feature amount acquisition step of acquiring feature amounts of natural persons included as images in the captured image corresponding to the candidate period; A feature amount identification step of identifying the feature amounts of the user based on the acquired feature amounts of the natural persons, and associating the identified feature amounts of the user with the identification information of the user; A second period identification step of identifying a matching period which is the candidate period in which the feature amounts of the user associated with the identification information of the user match or substantially match the feature amounts of the natural persons; An information processing method, characterized by including the above steps.

17. A program for causing a computer to function as the information processing apparatus according to any one of Claims 1 to 15.

Images