Information processing device, information processing method, and recording medium

Abstract

An information processing device, method, and program that reduce the processing load of tracking are provided. [Solution] Information processing device 1 includes an acquisition unit 11 that acquires an image, a tracking unit 12 that tracks people included in the image, and a registration update unit 13 that registers or updates characteristic information of people that have been tracked by tracking unit 12. Based on an image that includes a person, registration update unit 13 determines whether or not to register or update the characteristic information of the person.

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium. [Background technology]

[0002] Patent document 1 describes a technology that detects multiple detection target positions indicating the position of a first detection target from multiple detection information obtained by multiple detection devices each periodically detecting a first detection target, converts the multiple detection target positions into coordinates based on the space in which the multiple detection devices are installed, stores first coordinates that are the position where the first detection target was located before the multiple detection devices detected it, obtains, from the multiple converted coordinates, converted coordinates into which the multiple detection target positions detected from the detection information detected up to a first time period longer than the period were converted, extracts, from the obtained converted coordinates, multiple second coordinates that are predicted to have a relationship with the first coordinates, calculates representative coordinates based on the multiple second coordinates, and determines the representative coordinates as the position to which the first detection target has moved from the first coordinates. Patent Document 2 describes a technology in which person images are detected from captured moving images, a first selection process is performed to select a best shot image from among the person images of the same person based on a first index, and the reliability of the best shot image is calculated, and if the calculated reliability of the best shot image is lower than a first threshold, a second selection process is performed to select a best shot image from among the person images of the same person in accordance with a second index, and if the calculated reliability of the best shot image is equal to or higher than the first threshold, a person authentication process is performed using the best shot image selected by the first selection process and a person image registered in advance, and if the calculated reliability of the best shot image is lower than the first threshold, a person authentication process is performed using the best shot image selected by the second selection process and a person image registered in advance, and the execution result of the person authentication process is displayed, and the optimal image for the person authentication process is selected. Patent Document 3 describes a technology for tracking multiple objects, in which a processing target area in an image or image area included in an image group is determined based on a position of another tracking target other than one tracking target at a past time or determined as a correct answer, and pixel modification processing is performed to change the processing target area to a pixel pattern that eliminates or reduces the features of the other tracking target, and the image or image area that has been subjected to pixel modification processing is output to a classifier for learning and / or identifying one tracking target.Patent document 4 describes a technology for tracking multiple moving objects, which involves acquiring multiple frames, detecting objects from the multiple frames, extracting for the detected objects a first movement trajectory that includes the first frame, a second movement trajectory that is made up of only frames before the first frame, and a third movement trajectory that is made up of only frames after the first frame, and associating the second movement trajectory with the third movement trajectory if the similarity between the second movement trajectory and the third movement trajectory is equal to or greater than the similarity between the first movement trajectory and the third movement trajectory. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 155727 [Patent Document 2] Japanese Patent Application Publication No. 2019-205002 [Patent Document 3] Japanese Patent Application Publication No. 2018-185724 [Patent Document 4] Japanese Patent Application Publication No. 2017-228303 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0005] One aspect of an information processing device includes an acquisition means for acquiring an image, a tracking means for tracking a person included in the image, and a registration / update means for registering or updating characteristic information of a person that has been tracked by the tracking means, the registration / update means determining whether or not to register or update the characteristic information of the person based on an image including the person.

[0006] One aspect of the information processing method involves acquiring an image, tracking a person included in the image, determining whether to register or update characteristic information about the person based on the image including the person, and registering or updating characteristic information about people who have been tracked.

[0007] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute an information processing method that acquires an image, tracks a person included in the image, determines whether or not to register or update characteristic information about the person based on an image including the person, and registers or updates characteristic information about people that have been tracked. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an information processing device according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of an information processing device according to the second embodiment. [Figure 3] FIG. 3 is a conceptual diagram showing an example of a situation in which the information processing device according to the second embodiment is applied. [Figure 4] FIG. 4 is a conceptual diagram showing an example of an information processing operation performed by the information processing device in the second embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of the ReID matching operation performed by the information processing device in the second embodiment. [Figure 6] FIG. 6 is a flowchart showing the flow of a registered feature update operation performed by the information processing device in the second embodiment. [Figure 7] FIG. 7 is a block diagram showing the configuration of an information processing device according to the third embodiment. [Figure 8] FIG. 8 is a flowchart showing the flow of the ReID matching operation performed by the information processing device in the third embodiment. [Figure 9] FIG. 9 is a block diagram showing the configuration of an information processing device according to the fourth embodiment. [Figure 10]FIG. 10 is a flowchart showing the flow of the ReID matching operation performed by the information processing device in the fourth embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of a registered feature update operation performed by an information processing device in the fourth embodiment. [Figure 12] FIG. 12 is a block diagram showing the configuration of an information processing device according to the fifth embodiment. [Figure 13] FIG. 13 shows an example of a display on the display in the fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings. [1: First embodiment]

[0010] An information processing device, an information processing method, and a recording medium according to a first embodiment will be described below. The information processing device, the information processing method, and the recording medium according to the first embodiment will be described below using an information processing device 1 to which the information processing device, the information processing method, and the recording medium according to the first embodiment are applied. [1-1: Configuration of information processing device 1]

[0011] The configuration of an information processing device 1 in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1 in the first embodiment.

[0012] As shown in FIG. 1, the information processing device 1 includes an acquisition unit 11, a tracking unit 12, and a registration update unit 13. The acquisition unit 11 acquires an image. The tracking unit 12 tracks a person included in the image. The registration update unit 13 determines whether to register or update characteristic information of the person based on the image including the person. The registration update unit 13 registers or updates characteristic information of people that the tracking unit 12 has tracked. [1-2: Technical Effects of Information Processing Device 1]

[0013] In the information processing device 1 according to the first embodiment, the tracking unit 12 determines whether to register or update the characteristic information of people who have been tracked, and therefore the processing load is smaller than when no determination is made. Furthermore, when it is determined to update, the tracking unit 12 updates the characteristic information of people who have been tracked, so that more preferable characteristic information can be registered. [2: Second embodiment]

[0014] A second embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the second embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 2 to which the second embodiment of the information processing device, the information processing method, and the recording medium is applied. [2-1: Configuration of information processing device 2]

[0015] The configuration of the information processing device 2 in the second embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the information processing device 2 in the second embodiment.

[0016] 2, the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a field programmable gate array (FPGA). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is provided in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing operations to be performed by the information processing device 2 are realized within the arithmetic device 21. That is, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) that the information processing device 2 should perform.

[0018] Fig. 2 shows an example of logical functional blocks realized in the arithmetic device 21 to execute information processing operations. As shown in Fig. 2, the arithmetic device 21 realizes an acquisition unit 211 which is a specific example of "acquisition means" described in the appendix to be described later, a tracing unit 212 which is a specific example of "tracing means" described in the appendix to be described later, a registration update unit 213 which is a specific example of "registration update means" described in the appendix to be described later, and a matching unit 214 which is a specific example of "matching means" described in the appendix to be described later. The registration update unit 213 may include a linking unit 2131, which is a specific example of the "linking means" described in the appendix to be described later, a registration unit 2132, which is a specific example of the "registration means" described in the appendix to be described later, a feature extraction determination unit 2133, which is a specific example of the "feature extraction determination means" described in the appendix to be described later, an extraction unit 2134, which is a specific example of the "extraction means" described in the appendix to be described later, an update determination unit 2135, which is a specific example of the "update determination means" described in the appendix to be described later, and an update unit 2136, which is a specific example of the "update means" described in the appendix to be described later. However, the matching unit 214 does not have to be realized within the calculation device 21. Furthermore, the registration update unit 213 may not include any of the linking unit 2131, the registration unit 2132, the feature extraction determination unit 2133, the extraction unit 2134, the update determination unit 2135, and the update unit 2136. The operations of the acquisition unit 211, tracking unit 212, registration update unit 213, and matching unit 214 will be described in detail later.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program. The storage device 22 may store data to be stored long-term by the information processing device 2. The storage device 22 may include at least one of a random access memory (RAM), a read-only memory (ROM), a hard disk device, a magneto-optical disk device, a solid-state drive (SSD), and a disk array device. In other words, the storage device 22 may include a non-transitory recording medium. The storage device 22 may store a ReID feature database RIDB. In the second embodiment, the ReID feature database RIDB may be a database that associates tracking IDs with ReID features on a one-to-one basis. The ReID feature database RIDB will be described in detail with reference to Table 1. However, the storage device 22 does not necessarily have to store the ReID feature database RIDB.

[0020] The communication device 23 can communicate with devices external to the information processing device 2 via a communication network (not shown). If the above-mentioned storage device 22 does not store the ReID feature database RIDB, the ReID feature database RIDB may be stored in a device external to the information processing device 2, and the communication device 23 may exchange information with the ReID feature database RIDB stored in the device external to the information processing device 2 via the communication network. Furthermore, the communication device 23 may acquire images from a camera CAM (described later) via the communication network.

[0021] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-2: Application example of information processing device 2]

[0023] Next, an application example of the information processing device 2 in the second embodiment will be described with reference to Fig. 3. Fig. 3 is a conceptual diagram showing an example of a situation in which the information processing device 2 in the second embodiment is applied. The information processing device 2 in the second embodiment may be used for real-time tracking of people in marketing, entrance / exit management, etc. Furthermore, this real-time tracking of people may be applied to "gateless authentication" in which the person to be authenticated does not stop and is unaware of authentication.

[0024] The information processing device 2 may track people present in the tracking area TA as tracking targets. The tracking area TA may be, for example, an area of ​​5 meters by 5 meters. Five to six people can pass through the 5-meter width at the same time, and 20 to 30 people may exist and pass through the tracking area TA.

[0025] The camera CAM may capture an image of a predetermined space to capture an image of a person. The camera CAM may be installed so as to capture an image of an area including the tracking area TA. The camera CAM may be installed at a height of, for example, 2.5 meters. If the camera CAM is installed at a position higher than the height of an average person, the image captured by the camera CAM is likely to include the head region and upper body region of the person. The head region and upper body region may be suitable regions for the tracking area of ​​the person. [2-3: Tracking behavior]

[0026] The head region of a person is a part that is easy to detect even when the person is facing away from the front, such as facing backwards or sideways. Furthermore, even when much of the area is obscured by other people, the head region is often successfully detected. Therefore, in the second embodiment, the tracking unit 212 may track a person based on the head region of the person detected from the image.

[0027] The tracking unit 212 may detect the head region of a person passing through the tracking area TA and track the person. The tracking unit 212 may continue to track the person passing through the tracking area TA.

[0028] When the tracking unit 212 detects people from an image, it may output the positions of head regions associated with tracking IDs for the number of people detected. The tracking ID does not have to be an ID that identifies a person, but may be an ID that associates the same person who appears in different images with each other.

[0029] The image used for tracking by the tracking unit 212 may be one image frame of the video data. That is, the acquisition unit 211 may sequentially acquire a plurality of image frames that make up the video data as images.

[0030] The tracking performed by the tracking unit 212 may be an operation of associating the same person between a past frame and a current frame. The tracking unit 212 may perform tracking by determining the position of the head region in the image. The movement of a person between previous and subsequent frames is often small, and the change in head position is often small. The tracking unit 212 may associate the same person based on the position of the head region in the image. The tracking unit 212 may perform tracking by, for example, determining the identity of the image pattern of the head region. Alternatively, the tracking unit 212 may perform tracking by determining the similarity of feature amounts extracted from the image of the head region. Alternatively, the tracking unit 212 may use optical flow to associate people between frames.

[0031] The tracking unit 212 may detect an upper body region including a head region of a person. The tracking unit 212 may also detect a region above the knees of a person. Furthermore, the tracking unit 212 may detect a region including major joints of a person. [2-4:ReID operation]

[0032] In tracking a person, it is ideal to continue tracking the same person as the same person. In other words, it is ideal to link the same person with a single ID. To achieve this, we implement reconnection through person re-identification (ReID).

[0033] ReID may be an operation of linking and managing the tracking IDs of a person tracked in different cases. The different cases may be at different times. The different cases may also be at different locations. Furthermore, the different cases may be when images acquired from different cameras CAM are used.

[0034] If the tracking unit 212 cannot detect the tracking portion of the person for a predetermined time, for example, one second, it may become difficult to track the person. That is, if 30 frames are captured per second, the tracking unit 212 may find it difficult to track the person if it cannot detect the tracking portion for all 30 frames. ReID may be an operation performed when tracking of a person is interrupted, when a new person appears, or the like. ReID may also be an operation performed when a tracking error occurs due to occlusion or the like, and different tracking IDs are assigned to the same person. ReID may be an operation that is good at determining whether the same person is present. In ReID, whether the same person is present may be determined by ReID matching, which matches feature amounts extracted from people. This feature amount may be called a ReID feature amount.

[0035] The ReID feature database RIDB may be a database in which a person's tracking ID and feature information of the person are registered in association with each other. The feature information may be a ReID feature for determining whether the person is the same. The ReID feature may be a feature extracted from an image of a rectangular area including the person. The ReID feature may be a feature extracted from a whole-body image of the person. The ReID feature may be a feature extracted from an upper body image of the person. The upper body image of the person may include at least the clothing worn by the person. In other words, in ReID, it may be determined that the people are the same because the clothing worn is the same. The ReID feature may be a feature extracted from a head image including the head of the person. The image area from which the ReID feature is extracted may be called a ReID feature extraction area.

[0036] The matching operation using the ReID feature may have a lighter operational load than a matching operation capable of identifying an individual, such as face authentication. That is, ReID can determine whether a person matches or does not match with higher accuracy than tracking by the tracking unit 212, and may have a smaller operational load than a matching operation that identifies an individual. In the second embodiment, even if tracking by the tracking unit 212 fails, tracking may be able to continue by an operation that has a lighter load than an operation that can identify an individual.

[0037] The matching operation performed by the matching unit 214 may be ReID matching. The matching operation by the matching unit 214 may be able to determine with higher accuracy whether or not the persons are the same as each other than the tracking operation by the tracking unit 212.

[0038] By implementing ReID, even if real-time tracking of a person in an image is interrupted, it is possible to reconnect the interrupted tracking. ReID is also an important technology when applying real-time tracking of a person to "gateless authentication," in which the person to be authenticated does not stop and is not aware of the authentication. [2-5: Information processing for each image]

[0039] The flow of information processing operations performed by the information processing device 2 in the second embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of information processing operations performed by the information processing device 2 in the second embodiment. The operations from "START" to "END" shown in Fig. 4 may be operations for each frame.

[0040] 4, the acquisition unit 211 acquires an image (step S21). The acquisition unit 211 may sequentially acquire a plurality of image frames constituting video data. The tracking unit 212 detects a head region as a tracking region from a person included in the image (step S22).

[0041] The tracking unit 212 determines whether or not a tracking target exists (step S23). If a tracking target exists, the registration update unit 213 tracks the person based on the head region detected from the person included in the image.

[0042] If a tracking target exists (step S23: Yes), the tracking unit 212 determines whether a new tracking target exists (step S24). If a tracking target whose tracking is not being continuously performed is included in the image, the tracking unit 212 may determine that the tracking target is a new tracking target.

[0043] If a new tracking target exists (step S24: Yes), a ReID operation is performed (step S25). Details of the ReID operation in step S25 will be described later with reference to FIG.

[0044] If no new tracking target exists (step S24: No), a registered feature update operation is performed (step S26). Details of the registered feature update operation in step S26 will be described later with reference to FIG.

[0045] After the ReID matching operation or the registered feature update operation is completed, the process proceeds to step S27. In step S27, the result of the information processing operation is displayed. Details of step S27 will be described in other embodiments below.

[0046] The tracking unit 212 determines whether or not an unprocessed tracking target exists (step S28). If an unprocessed tracking target exists (step S28: Yes), the process proceeds to step S24. If no tracking target exists in step S23, or if there is no unprocessed tracking target, the information processing operation for each image ends. [Step S25: ReID Verification Operation]

[0047] 5, the feature extraction determination unit 2133 receives the detection result (step S251). The detection result may include an image area of ​​a person included in the image.

[0048] The feature extraction determination unit 2133 determines whether to extract a ReID feature as feature information of the relevant person (step S252). The feature extraction determination unit 2133 may determine whether to extract a ReID feature based on the number of images in which the tracking unit 212 detected a head region of the same tracking target (also referred to as the "number of tracking frames"). The feature extraction determination unit 2133 may determine whether to extract a ReID feature based on whether the number of tracking frames of the tracking target exceeds a first threshold. For example, the feature extraction determination unit 2133 may determine to extract a ReID feature when the number of frames in which the relevant person being tracked is detected exceeds 15 frames. For example, if 30 frames are captured per second, the feature extraction determination unit 2133 may determine to extract a ReID feature 0.5 seconds after new tracking is started. Alternatively, the feature extraction determination unit 2133 may determine to extract a ReID feature when the position in the image in which the person is detected is not at the edge of the image. This is because, if a person is detected at the edge of an image, the person may not be included in the image in subsequent frames and may not be a tracking target. Alternatively, the feature extraction determination unit 2133 may determine to extract ReID features when the ReID feature extraction region of a person included in an image has a predetermined or higher quality. The case where the ReID feature extraction region in an image has a predetermined or higher quality may be when the quality is such that features appropriate for ReID matching can be extracted from the ReID feature extraction region in the image.

[0049] If it is determined that the ReID feature is not to be extracted (step S252: No), the registration unit 2132 may register the temporary ID "N" before the ReID process for the relevant person in the ReID feature database RIDB, an example of which is shown in Table 1 below. [Table 1: ReID feature database RIDB]

[0050] [Table 1]

[0051] As illustrated in Table 1 above, the ReID feature database RIDB may have a structure including a column of tracking IDs, a column of tracked IDs, a column of ReID features, a column of evaluation values ​​of ReID features, and a column of information indicating the image (also called the "registered image") from which the ReID features were extracted.

[0052] If it is determined that the ReID feature is to be extracted (step S252: Yes), the matching unit 214 extracts the ReID feature (step S253). The matching unit 214 matches the extracted ReID feature with the registered ReID feature registered in the ReID feature database RIDB, and determines whether the matching is successful or not (step S254). The matching unit 214 may calculate a matching score between the extracted ReID feature and each of all registered ReID features. The matching unit 214 may determine that the matching is successful when the maximum matching score among the calculated matching scores is greater than a second threshold. The matching unit 214 may determine that the tracking target corresponding to the registered ReID feature with the calculated maximum matching score is the same person as the tracking target. That is, if a ReID feature as feature information of the same person as the newly tracked person is registered in the ReID feature database RIDB, the registration unit 2132 associates the person with the registered person.

[0053] If the matching is successful (step S254: Yes), the linking unit 2131 links the new tracking target with the registered tracking target (step S255). For example, if it is determined that the new tracking target is the same person as the person with the tracking ID that has been tracked in the past, the linking unit 2131 may update the ReID feature amount database RIDB shown in Table 1 to the ReID feature amount database RIDB shown in Table 2. [Table 2: ReID feature database RIDB updated in step S255]

[0054] [Table 2]

[0055] That is, the column of tracked IDs may be updated with "A," the column of ReID features may be updated with "AAA," the column of evaluation values ​​of ReID features may be updated with "5," and the column of information indicating registered images may be updated with "An." "AAA" may be the ReID feature of "A," "5" may be the evaluation value of "AAA," and "An" may be information indicating the image from which "AAA" was extracted.

[0056] If the matching fails (step S24: No), the registration unit 2132 registers the ReID feature of the new tracking target (step S256). For example, the registration unit 2132 may update the ReID feature database RIDB illustrated in Table 1 to the ReID feature database RIDB illustrated in Table 3. [Table 3: ReID feature database RIDB updated in step S256]

[0057] [Table 3]

[0058] That is, the column of tracking ID may be updated with “D”, the column of ReID features may be updated with “DDD”, the column of evaluation values ​​of ReID features may be updated with “5”, and the column of information indicating the registered image may be updated with “Dm”. “DDD” may be the ReID feature extracted in step S253, “5” may be the evaluation value of “DDD”, and “Dm” may be information indicating the image from which “DDD” was extracted, i.e., the image acquired in step S251.

[0059] The registration update unit 213 outputs the result of the ReID operation (step S257). [Step S26: Registered feature update operation]

[0060] 6, the feature extraction determination unit 2133 receives the detection result (step S261). The detection result may include an image area of ​​a person included in the image.

[0061] The feature extraction determination unit 2133 acquires registered information of the tracking target (step S262). The feature extraction determination unit 2133 may acquire the number of tracking frames of the tracking target, an evaluation value of the registered image, and the registered image. The feature extraction determination unit 2133 determines whether to extract ReID features (step S263). The details of step S263 will be described later.

[0062] The extraction unit 2134 extracts ReID features from the ReID feature extraction region of the person (step S264). The ReID feature extraction region may be the upper body, or above the knees.

[0063] The update determination unit 2135 determines whether to update the ReID feature (step S265). Details of step S265 will be described later.

[0064] The update unit 2136 updates the ReID feature (step S266). Even after updating, the update unit 2136 does not need to discard the previously registered ReID feature, evaluation value, and registered image. The previously registered ReID feature, evaluation value, and registered image may be used to calculate the evaluation value.

[0065] The registration update unit 213 outputs the result of the registration feature update operation (step S267). [Determination in step S263]

[0066] In step S263, the feature extraction determination unit 2133 may determine whether or not to extract feature information of the person in question, based on at least one of the following (1) to (3).

[0067] (1) Number of tracking frames The feature extraction determination unit 2133 may determine whether to extract a ReID feature as feature information of the relevant person based on the number of tracking frames. The feature extraction determination unit 2133 may determine whether to extract a ReID feature based on whether the number of tracking frames of the tracking target exceeds a first threshold. For example, the feature extraction determination unit 2133 may determine to extract a ReID feature when the number of frames in which the relevant person being tracked by the tracking unit 212 is detected exceeds 15 frames. For example, if 30 frames are captured per second, the feature extraction determination unit 2133 may determine to extract a ReID feature 0.5 seconds after new tracking is started.

[0068] (2) Information indicating the specific joints (also called "visible joint points") of the person in the image By determining visible joint points, it is possible to determine how much of a person's area is included in an image with a small processing load. The feature extraction determination unit 2133 may determine whether specific joints of a person are included in an image. The specific joints may include facial features such as eyes and nose, as well as the neck, shoulders, elbows, wrists, waist, knees, and ankles. The specific joints may be joints above the knees. The ReID feature extraction region may be defined so that at least one of the person's head and torso overlaps. In other words, the image from which ReID features are extracted may be one in which the person's face is hidden, and it may be determined that ReID features are extracted from an image that includes the person's body from the neck down.

[0069] The feature extraction determination unit 2133 may make the determination based on the overlap between the visible joint points in the registered image and the visible joint points in the acquired image (current frame). The feature extraction determination unit 2133 may determine to extract feature information of the person in question when the overlap between the visible joint points in the registered image and the visible joint points in the acquired image (current frame) is greater than a third threshold. The registered image is an image from which ReID features have been extracted and registered, and is an image of a predetermined quality or higher, so if there is a large overlap between the visible joint points in the registered image and the visible joint points in the acquired image (current frame), it may be considered to be an image suitable for feature extraction. The third threshold may be set, for example, so that 50% or more of the entire body of the person in question overlaps.

[0070] For example, the feature extraction determination unit 2133 may determine to extract feature information of the person in question when the value calculated by the following formula 1 is greater than a third threshold value. [Formula 1] TIFF2026041985000005.tif11150J visible may be a set of visible joint points. may be an estimated score of the output visible joint points. The "1" on the right shoulder may indicate the current frame, and the "2" may indicate the previous frame. The feature extraction determination unit 2133 can further appropriately determine whether the frame is suitable for feature extraction by taking the estimated score into account.

[0071] (3) Evaluation value of ReID feature The feature extraction determination unit 2133 may determine whether to update the ReID feature database RIDB by calculating an evaluation value of the ReID feature registered in the ReID feature database RIDB. For example, if the evaluation value of the registered ReID feature is smaller than a fourth threshold, the feature extraction determination unit 2133 may determine to extract feature information of the person. In other words, if the registered ReID feature is not very desirable, the feature extraction determination unit 2133 may determine to update it. The calculation of the evaluation value of the ReID feature will be described in detail in the following description of step S265.

[0072] The feature extraction determination unit 2133 determines whether or not to extract ReID features based on the tracking history of the tracking target, registered information, and the like, thereby making it possible to reduce calculation time. [Determination in step S265]

[0073] The update determination unit 2135 may determine whether to update the registered feature information based on the evaluation value of the ReID feature extracted by the extraction unit 2134. The update determination unit 2135 may update the ReID feature when the evaluation value of the ReID feature extracted by the extraction unit 2134 exceeds the evaluation value of the ReID feature registered in the ReID feature database RIDB. The update determination unit 2135 may calculate the evaluation value by any of the following methods (i) to (iv).

[0074] (i) The update determination unit 2135 may calculate the evaluation value based on information indicating the overlap between visible joint points in the registered image and visible joint points in the acquired image. The information indicating the overlap between a predetermined joint of a person in the registered image and a predetermined joint of a person in the image may be the same as the information described in (2) of step S263. The update determination unit 2135 may perform the calculation using all registered images of the tracking target that were registered in the past, and use the maximum value as the evaluation value.

[0075] (ii) The update determination unit 2135 may evaluate the extracted ReID feature based on the result of matching the extracted ReID feature with the registered ReID feature, and calculate an evaluation value. The update determination unit 2135 may perform the calculation using all ReID feature values ​​of the tracking target that were registered in the past, and may use the maximum value as the evaluation value.

[0076] (iii) The update determination unit 2135 may calculate the evaluation value using a machine-learned computational model. The computational model may be a computational model that is machine-learned using the relationship between the evaluation value according to (i) above and the evaluation value according to (ii). The computational model may be a computational model that estimates the evaluation value of the extracted ReID feature when information indicating the overlap of visible joint points and a matching result with the ReID feature are input. The computational model may be a computational model that can be machine-learned, and a convolutional neural network is an example of a computational model that can be machine-learned.

[0077] (iv) The update determination unit 2135 may evaluate the extracted ReID features and calculate an evaluation value based on at least one of information indicating overlap between first predetermined regions of people in different images and information indicating overlap between second predetermined regions of different people in the same image. Each of the first predetermined region and the second predetermined region may be a ReID feature extraction region. The update determination unit 2135 may calculate the evaluation value based on an overlap index of the ReID feature extraction region. The update determination unit 2135 may employ IoU (Intersection over Union) to calculate the evaluation value using the following equation 2. [Formula 2] IoU with the same tracked object in a different image × (1 - IoU with other tracked objects in the current image)

[0078] The image acquired by the acquisition unit 211 may be supplemented with information about a person included in the image. This information may include information indicating a tracking area of ​​the person included in the image. In this case, the camera CAM may detect the person and the tracking area of ​​the person in addition to capturing an image. In other words, part of the operation of the tracking unit 212 described above may be performed outside the information processing device 2. [2-6: Technical Effects of Information Processing Device 2]

[0079] In person ReID matching, the accuracy of ReID matching can be improved by selecting which feature to use for matching, i.e., which feature to register. The information processing device 2 in the second embodiment selects an image from which features are extracted and further selects feature information to register and update, so that better features can be adopted while maintaining real-time performance. This can improve the accuracy of ReID matching and the tracking results.

[0080] For example, in the first frame in which a new tracking target appears, the area in which the tracking target is not occluded is often small. In response to this, the information processing device 2 determines whether to extract feature information based on the number of tracking frames, thereby making it possible to determine whether an image is one from which good feature information can be extracted without extracting feature information. Furthermore, the information processing device 2 determines whether to extract feature information based on visible joint points, thereby making it possible to determine whether an image is one from which good feature information can be extracted without extracting feature information. Furthermore, the information processing device 2 determines whether to update the ReID feature database RIDB based on the evaluation value of the feature information, thereby registering feature information with a high evaluation value and enabling good ReID matching using the registered feature information. Furthermore, the information processing device 2 makes a determination based on the result of matching with registered feature information already determined to be suitable for ReID matching, thereby making it possible to determine whether the feature information is good. Furthermore, the information processing device 2 makes a determination based on the overlap between the tracking targets in the past frame and the current frame, and the overlap between the tracking target and tracking targets other than the tracking target in the current frame, thereby making it possible to update the ReID feature database RIDB with feature information extracted from the tracking target with little occlusion. [3: Third embodiment]

[0081] An information processing device, an information processing method, and a recording medium according to a third embodiment will be described below. The third embodiment of the information processing device, the information processing method, and the recording medium will be described below using an information processing device 3 to which the third embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0082] The information processing device 3 in the third embodiment may be applied to a case where facial recognition is performed while tracking a moving person. In this case, the camera CAM may be a 4K camera and may be capable of capturing images of a quality that can be used for facial recognition. [3-1: Configuration of information processing device 3]

[0083] The configuration of the information processing device 3 in the third embodiment will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the information processing device 3 in the third embodiment.

[0084] As shown in FIG. 7 , the information processing device 3 of the third embodiment differs from the information processing device 2 of the second embodiment in that the calculation device 21 further includes an individual matching unit 315 and the storage device 22 further includes a facial feature database FC. The individual matching unit 315 collects biometric information of a person included in an image and matches the biometric information with registered biometric information. The facial feature database FC stores the registered ID of a registered person in association with the biometric information of the registered person. The facial feature database FC may also store the registered ID of a registered person in association with features extracted from the biometric information of the registered person. Note that if the storage device 22 does not store the facial feature database FC, the facial feature database FC may be stored in a device external to the information processing device 3, and the communication device 23 may exchange information with the facial feature database FC stored in a device external to the information processing device 3 via a communication network. Other features of the information processing device 3 may be the same as those of the information processing device 2. Therefore, in the following, only the parts that differ from the embodiments already described will be described in detail, and the description of other overlapping parts will be omitted as appropriate. [3-2: Facial Feature Database FC]

[0085] For example, the face feature amount database FC of the third embodiment may have the structure shown in Table 4 below. [Table 4: Facial feature database FC]

[0086] [Table 4] That is, the facial feature database FC may include a column of tracking IDs, a column of ReID features, a column of personal IDs, and a column of facial features. The personal ID may be an ID that identifies a person, or may be an ID of a person who has been identified as an individual. [3-3: ReID verification operation performed by information processing device 3]

[0087] The information processing device 3 in the third embodiment differs from the information processing device 2 in the second embodiment in the ReID matching operation in step S25 in Fig. 4. For this reason, the flow of the ReID matching operation performed by the information processing device 3 in the third embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of the ReID matching operation performed by the information processing device 3 in the third embodiment. The operations from "START" to "END" shown in Fig. 8 may be operations related to one tracking target.

[0088] 8, the matching unit 214 receives the detection result (step S251). The feature extraction determination unit 2133 determines whether to extract the ReID feature amount as feature information of the person (step S252).

[0089] If it is determined that the ReID feature is to be extracted (step S252: Yes), the matching unit 214 extracts the ReID feature (step S253). The individual matching unit 315 collects a face image as biometric information of the person included in the image, and extracts feature amounts of the face image (sometimes referred to as "facial feature amounts") from the face image (step S30).

[0090] The individual matching unit 315 matches the extracted facial feature amount with the facial feature amount registered in the facial feature amount database FC, and determines whether the matching is successful or not (step S31). The matching of the facial feature amount by the individual matching unit 315 may be an operation for identifying an individual.

[0091] If the matching of the facial features is successful (step S31: Yes), the linking unit 2131 links the person included in the image with the person corresponding to the registered biometric information (step S32). The matching unit 214 matches the extracted ReID features with all registered ReID features registered in the ReID feature database RIDB, and determines whether the matching is successful (step S33).

[0092] If the matching fails (step S33: No), the linking unit 2131 registers the extracted matching feature of the new tracking target as a registered matching feature (step S34). For example, if the new tracking target is the same person as the tracking ID "B" exemplified in Table 4 above, the facial feature is registered in the facial feature database FC, but the ReID feature is not registered. Therefore, the determination in step S31 is Yes, and the determination in step S33 is No. On the other hand, if the new tracking target is the same person as the tracking ID "A" exemplified in Table 4 above, the facial feature and the ReID feature are registered in the facial feature database FC. Therefore, the determination in step S31 is Yes, and the determination in step S33 is also Yes.

[0093] If the matching of the facial features fails (step S31: No), the matching unit 214 matches the extracted ReID features with all registered ReID features registered in the ReID feature database RIDB, and determines whether the matching is successful (step S254). A case where the facial matching fails may be when a facial image of a person is not registered. In other words, the tracking target person may be a person who has not been authenticated in the tracking area TA. If the matching is successful (step S254: Yes), the registration unit 2132 links the person included in the image with the person corresponding to the registered ReID features (step S255). If the matching fails (step S254: No), the linking unit 2131 registers the ReID features of the new tracking target as registered matching features (step S256). The registration update unit 213 outputs the result of the ReID matching operation (step S257).

[0094] In the operation of updating registered features in the third embodiment, if an image contains a face image suitable for extracting facial features, the facial features may be updated with the consent of the person. Since a person whose facial features are already registered is a registered and identified individual, for example, a notification asking whether the update is OK may be sent to a device such as a smartphone carried by the person, and consent may be obtained. Alternatively, if facial features worth updating are extracted when the person enters the tracking area TA, consent to the update may be obtained. In this case, too, a notification that the update has been completed may be sent to a device such as a smartphone carried by the person.

[0095] In the present embodiment, the personal matching unit 315 has been described taking as an example a case where a face image is collected as biometric information and a matching of face features is performed. However, other biometric information, for example, an iris image may be collected and a matching of features extracted from the iris image may be performed. [3-4: Technical Effects of Information Processing Device 3]

[0096] The information processing device 3 in the third embodiment links the tracking targets using biometric information, and therefore can link the tracking targets with high accuracy. [4: Fourth embodiment]

[0097] An information processing device, an information processing method, and a recording medium according to a fourth embodiment will be described below. The information processing device, the information processing method, and the recording medium according to the fourth embodiment will be described below using an information processing device 4 to which the information processing device, the information processing method, and the recording medium according to the fourth embodiment are applied. [4-1: Configuration of information processing device 4]

[0098] The configuration of the information processing device 4 in the fourth embodiment will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the information processing device 4 in the fourth embodiment.

[0099] 9, the information processing device 4 in the fourth embodiment differs from the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment in that the arithmetic device 21 includes an additional determination unit 416. Other features of the information processing device 4 may be the same as other features of the information processing device 2 or the information processing device 3. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [4-2: Information processing operation performed by information processing device 4]

[0100] The flow of information processing operations performed by the information processing device 4 in the fourth embodiment will be described with reference to Fig. 10 and Fig. 11. Fig. 10 is a flowchart showing the flow of ReID matching operations performed by the information processing device 4 in the fourth embodiment. Fig. 11 is a flowchart showing the flow of registered feature updating operations performed by the information processing device 4 in the fourth embodiment. The operations from "START" to "END" shown in Fig. 10 and Fig. 11 may be operations related to one tracking target.

[0101] As shown in FIG. 10, the collation unit 214 receives the detection result (step S251).

[0102] The additional determination unit 416 determines which of a plurality of predetermined attributes the person included in the image corresponds to (step S40). The plurality of predetermined attributes may be, for example, the orientation of the person's body. The feature amount that can be extracted from an image of a person captured from the front, the feature amount that can be extracted from an image of a person captured from the side, and the feature amount that can be extracted from an image of a person captured from the back are different. When the plurality of predetermined attributes is the orientation of the person's body, the additional determination unit 416 may determine the orientation of the person included in the image.

[0103] The feature extraction determination unit 2133 determines whether to extract the ReID feature of the person (step S252). If it is determined that the ReID feature is to be extracted (step S252: Yes), the matching unit 214 extracts the ReID feature (step S253).

[0104] The matching unit 214 matches the extracted ReID feature with the registered ReID feature of the attribute determined in step S40, which is registered in the ReID feature database RIDB, and determines whether the matching is successful (step S41). [4-3: ReID feature database RIDB]

[0105] For example, the ReID feature database RIDB of the fourth embodiment may have the structure shown in Table 5 below. [Table 5: ReID feature database RIDB]

[0106] [Table 5] That is, the ReID feature database RIDB of the fourth embodiment may include a column of tracking IDs, a column of front ReID features, a column of side ReID features, and a column of rear ReID features. The ReID feature database RIDB of the fourth embodiment may register ReID features of multiple attributes.

[0107] If the matching is successful (step S254: Yes), the registration unit 2132 associates the new tracking target with the registered tracking target (step S255). For example, if the attribute of the person included in the image is "back face" and the person is the same person as the person with tracking ID "A", the matching is successful because the back face ReID feature exists.

[0108] If the matching fails (step S41: No), the ReID feature corresponding to the attribute is not registered, so the registration unit 2132 registers the extracted ReID feature as feature information for the attribute (step S42). If the attribute of the person included in the image is "back face" and the person is the same person as the person with tracking ID "B", the matching fails because there is no back face ReID feature. In the example shown in Table 5, the back face ReID feature for tracking ID "B" may be updated from "-" to "BBBB" in step S42.

[0109] The registration update unit 213 outputs the result of the ReID matching operation (step S) 257).

[0110] As shown in FIG. 11, the feature extraction determination unit 2133 receives the detection result (step S261). The addition determination unit 416 determines which of a plurality of predetermined attributes the person included in the image corresponds to (step S43). The feature extraction determination unit 2133 acquires registered information of the tracking target (step S262). The feature extraction determination unit 2133 determines whether to extract ReID features (step S263). The extraction unit 2134 extracts matching features from a predetermined area of ​​the person (step S264).

[0111] The addition determination unit 416 determines whether the attribute determined in step S43 is a new attribute (step S44). The addition determination unit 416 may determine whether the attribute is a new attribute based on whether a ReID feature corresponding to the attribute is registered. The addition determination unit 416 may determine that the attribute is a new attribute when the column corresponding to the attribute is "-".

[0112] If the attribute is not a new attribute and a ReID feature corresponding to the attribute is registered (step S44: No), it is determined whether or not to update the registered ReID feature of the person (step S265). If it is determined that the ReID feature should be updated, the update unit 2136 updates the ReID feature (step S266).

[0113] If the ReID feature corresponding to the attribute has not been registered and the attribute is a new attribute (step S44: No), the registration unit 2132 registers feature information corresponding to the attribute (step S45).

[0114] The registration update unit 213 outputs the result of the registration feature update operation (step S267).

[0115] In this embodiment, the orientation of a person's body has been described as an example of the plurality of predetermined attributes. However, the plurality of predetermined attributes may be, for example, an unobstructed area of ​​a person. The unobstructed area of ​​a person may be, for example, a face area of ​​the person, an upper body area of ​​the person including the face, or a torso area from the shoulders to the waist of the person. That is, the ReID feature extraction area may be an attribute. For example, whether or not a person is carrying luggage may be an attribute. For example, an attribute may be determined based on whether luggage is included in the image together with the person. For example, the brightness of the person's area may be an attribute. The plurality of predetermined attributes may be, for example, the amount of light in the person's area. [4-4: Technical Effects of Information Processing Device 4]

[0116] The information processing device 4 in the fourth embodiment links the tracking targets using ReID feature amounts of a plurality of attributes, and therefore can link the tracking targets with high accuracy. [5: Fifth embodiment]

[0117] A fifth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the fifth embodiment of an information processing device, an information processing method, and a recording medium will be described using an information processing device 5 to which the fifth embodiment of the information processing device, the information processing method, and the recording medium is applied. [5-1: Configuration of information processing device 5]

[0118] The configuration of the information processing device 5 in the fifth embodiment will be described with reference to Fig. 12. Fig. 12 is a block diagram showing the configuration of the information processing device 5 in the fifth embodiment.

[0119] 12, the information processing device 5 in the fifth embodiment differs from the information processing device 2 in the second embodiment to the information processing device 4 in the fourth embodiment in that the arithmetic device 21 includes a display control unit 517. The display control unit 517 and other features of the information processing device 5 may be the same as at least one other feature of the information processing device 2 to the information processing device 4. Therefore, hereinafter, only the parts that differ from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [5-2: Display operation performed by information processing device 5]

[0120] The fifth embodiment may be an embodiment that explains a specific example of an operation of outputting the result of the information processing operation in the second embodiment described above (i.e., an operation corresponding to step S27 in FIG. 4). The information processing device 5 in the fifth embodiment may update the operation of outputting the result of the information processing operation every time one image is processed.

[0121] In the fifth embodiment, the output device 25 may include a display device (referred to as "display D") capable of displaying an image showing information to be output. FIG. 13 shows an example of a screen showing the results of an information processing operation displayed on the display D in the fifth embodiment. The display D in the fifth embodiment may display a screen that enables a person managing the tracking area TA to check the tracking status. The display control unit 517 may control the display of the display D.

[0122] The display control unit 517 displays the image by superimposing information indicating the tracking result on the relevant person included in the image. If tracking of the person is successful, the display control unit 517 may superimpose information indicating the success of tracking on the relevant person included in the image, and if tracking of the person is unsuccessful, the display control unit 517 may superimpose information indicating the failure of tracking on the relevant person included in the image and display the image. The display control unit 517 may superimpose the tracking ID of the person as information indicating the success of tracking. A case where tracking of a person is unsuccessful may mean a state in which the person is not linked to a person already being tracked and it is unclear whether the person is a new tracking target, that is, a case in which a ReID feature has not been extracted (step S252: No).

[0123] Fig. 13(a) shows an example of the display of the tracking result based on the nth acquired image. In the example shown in Fig. 13, the display control unit 517 may display an image in which a thick solid rectangle and a tracking ID are superimposed on a person who has been successfully tracked. The display control unit 517 may also display an image in which a dashed rectangle and "New" indicating that ReID matching has not yet been performed are superimposed on a person who has not been unsuccessfully tracked. Fig. 13(a) shows an example in which a person with a tracking ID "B" and a person with a tracking ID "C" are being tracked in the nth acquired image, and one person for whom ReID matching has not yet been performed is captured.

[0124] Fig. 13(b) shows an example of a display of tracking results based on the (n+k)th image. Fig. 13(b) illustrates an example in which a person who has not yet undergone ReID matching in the information processing operation based on the (n+k)th image has been determined to be a person who has been tracked in the past and whose tracking ID is "A" in the information processing operation based on the (n+k)th image.

[0125] Fig. 13(c) shows an example of a display of tracking results based on the (n+k)th image. Fig. 13(c) shows an example in which a person who has not yet undergone ReID matching in the information processing operation based on the (n+k)th image is determined to be a new tracking target in the information processing operation based on the (n+k)th image, and is assigned a tracking ID "D". [5-3: Technical Effects of Information Processing Device 5]

[0126] The person in charge of the tracking area TA can check the tracking status of each person in real time by viewing the display D. [6: Note]

[0127] The following additional notes are provided regarding the above-described embodiment. [Appendix 1] an acquisition means for acquiring an image; a tracking means for tracking a person included in the image; a registration and update means for registering or updating characteristic information of a person who has been tracked by the tracking means, the registration and update means determining whether or not to register or update the characteristic information of the person based on an image including the person; An information processing device comprising: [Appendix 2] further comprising a matching means for matching extracted feature information extracted from the person included in the image with registered feature information when the person included in the image is not a person being tracked by the tracking means, The registration update means a linking means for linking a person included in an image with a person corresponding to the registered feature information when the matching means has successfully matched the extracted feature information with the registered feature information; a registration means for registering the extracted feature information in association with a person included in an image when the matching means fails to match the extracted feature information with the registered feature information; a feature extraction determination means for determining, when a person included in an image is a person being tracked by the tracking means, whether or not to extract feature information of the person from the image based on at least one of the image and registered information about the person; an extraction means for extracting the feature information of the person from the image when the feature extraction determination means determines that feature information should be extracted; an update determination means for determining whether or not to update the registered feature information of the person based on at least one of the image, the registered information about the person, and the extracted feature information of the person extracted by the extraction means; and and an update means for updating the registered feature information of the person using the extracted feature information of the person extracted by the extraction means when the update determination means determines that the registered feature information should be updated. 10. The information processing device according to claim 1. [Appendix 3] The feature extraction determination means determines whether or not to extract feature information of the person in question based on the number of images including the person in question being tracked by the tracking means. 3. The information processing device according to claim 2. [Appendix 4] the registered information about the person includes a registered image from which the registered feature information is extracted; At least one of the feature extraction determination means and the update determination means performs the determination based on information indicating an overlap between a predetermined joint of the person in the registered image and a predetermined joint of the person in the image. 3. The information processing device according to claim 2. [Appendix 5] The information about the registered person includes an evaluation value obtained by evaluating the registered characteristic information about the person, At least one of the feature extraction determination means and the update determination means performs a determination based on the evaluation value. 3. The information processing device according to claim 2. [Appendix 6] The update determination means evaluates the extracted feature information based on a comparison result between the extracted feature information and the registered feature information, and determines whether or not to update the registered feature information. 3. The information processing device according to claim 2. [Appendix 7] The update determination means Information indicating an overlap between first predetermined regions of the person in the different images; and Information indicating overlap between second predetermined regions of different people in the same image. and evaluating the extracted feature information based on at least one of the above, and determining whether to update the registered feature information. 3. The information processing device according to claim 2. [Appendix 8] The device further includes an individual matching unit that collects biometric information of a person included in the image and matches the biometric information with registered biometric information, When the personal verification unit is successful in verifying the identity of the person, the linking unit links the person included in the image with the person corresponding to the registered biometric information. 3. The information processing device according to claim 2. [Appendix 9] The image processing device further includes an additional determination unit that determines to which of a plurality of predetermined attributes a person included in an image corresponds, and determines whether or not feature information corresponding to the attribute is registered; When the addition determination means determines that feature information corresponding to the attribute is not registered, the registration means registers the feature information corresponding to the attribute, When the addition determination means determines that characteristic information corresponding to the attribute is registered, the update determination means determines whether or not to update the registered characteristic information of the person. 3. The information processing device according to claim 2. [Appendix 10] a display means for displaying the image with information indicating the tracking result superimposed on the person included in the image; 3. The information processing device according to claim 1 or 2, further comprising: [Appendix 11] Acquire an image, Tracking people in the images; Based on an image including the person, it is determined whether or not to register or update characteristic information of the person, and characteristic information of people who have been tracked is registered or updated. Information processing methods. [Appendix 12] On the computer, Acquire an image, Tracking people in the images; Based on an image including the person, it is determined whether or not to register or update characteristic information of the person, and characteristic information of people who have been tracked is registered or updated. A recording medium on which a computer program for executing an information processing method is recorded.

[0128] At least some of the constituent elements of each of the above-described embodiments can be appropriately combined with at least some of the other constituent elements of each of the above-described embodiments. Some of the constituent elements of each of the above-described embodiments may not be used. Furthermore, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure are incorporated by reference as part of the description of this disclosure.

[0129] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0130] 1,2,3,4,5 Information processing equipment 11,211 Acquisition Department 12,212 Tracking Department 13,213 Registration and Update Department 2131 Attachment 2132 Registration Department 2133 Feature Extraction and Judgment Unit 2134 Extraction part 2135 Update determination section 2136 Update Department 214 Matching Unit 315 Personal Verification Department 416 Additional judgment section 517 Display control unit TA Tracking Area RIDB ReID feature database FC Facial Feature Database

Claims

1. an acquisition means for acquiring an image; a tracking means for tracking a person included in the image; extraction means for extracting, from the image, characteristic information of a person who has been tracked by the tracking means; an update determination means for determining whether or not to update the registered feature information of the person based on at least one of the image, the registered information about the person, and the extracted feature information of the person extracted by the extraction means; an updating means for updating the registered feature information of the person using the extracted feature information of the person extracted by the extracting means when the update determining means determines that the registered feature information should be updated; An information processing device comprising:

2. further comprising a feature extraction determination means for determining, when a person included in the image is a person being tracked by the tracking means, whether or not to extract feature information of the person from the image based on at least one of the image and registered information about the person; The extraction means extracts the feature information of the person from the image when the feature extraction determination means determines that the feature information should be extracted. The information processing device according to claim 1 .

3. a matching means for matching extracted feature information extracted from the person included in the image with registered feature information if the person included in the image is not a person being tracked by the tracking means; a linking means for linking a person included in the image with a person corresponding to the registered feature information when the matching means has successfully matched the extracted feature information with the registered feature information; and a registration unit that registers the extracted feature information in association with a person included in an image when the matching unit fails to match the extracted feature information with the registered feature information. The information processing device according to claim 1 .

4. The feature extraction determination means determines whether or not to extract feature information of the person in question based on the number of images including the person in question being tracked by the tracking means. The information processing device according to claim 2 .

5. the registered information about the person includes a registered image from which the registered feature information is extracted; At least one of the feature extraction determination means and the update determination means performs the determination based on information indicating an overlap between a predetermined joint of the person in the registered image and a predetermined joint of the person in the image. The information processing device according to claim 2 .

6. The information about the registered person includes an evaluation value obtained by evaluating the registered characteristic information about the person, At least one of the feature extraction determination means and the update determination means performs a determination based on the evaluation value. The information processing device according to claim 2 .

7. The update determination means evaluates the extracted feature information based on a comparison result between the extracted feature information and the registered feature information, and determines whether or not to update the registered feature information. The information processing device according to claim 1 .

8. The update determination means Information indicating an overlap between first predetermined regions of the person in the different images; and Information indicating overlap between second predetermined regions of different people in the same image. and evaluating the extracted feature information based on at least one of the above, and determining whether to update the registered feature information. The information processing device according to claim 1 .

9. acquiring an image; tracking people in said images; Extracting characteristic information of previously tracked people from the image; determining whether to update the registered feature information of the person based on at least one of the image, the registered information about the person, and the extracted extracted feature information of the person; and updating the registered feature information of the person using the extracted extracted feature information of the person when it is determined that the registered feature information of the person should be updated; A computer-implemented information processing method.

10. On the computer, acquiring an image; tracking people in said images; Extracting characteristic information of previously tracked people from the image; determining whether to update the registered feature information of the person based on at least one of the image, the registered information about the person, and the extracted extracted feature information of the person; and and updating the registered feature information of the person using the extracted extracted feature information of the person when it is determined that the registered feature information of the person is to be updated. A computer program for executing an information processing method.

Images