Person identification device

Abstract

To provide a person identification device and a program for improving identification accuracy of a person imaged by different cameras and imaged in different situations.SOLUTION: A person identification device is configured so that: object person image acquisition means 2 acquires an object person image; feature acquisition means 4 analyzes and acquires the image features of the object person image; change determination means 8 determines whether to change at least one element in a color space of the object person image, for improving identification accuracy in identification means 10, on the basis of the image features; identification image determination means 6 changes at least one element in the color space of the object person image when it is determined that the change is generated by the change determination means 8, and then generates an object person image for identification, and when it is determined that no change is generated, does not change the object person image and sets the object person image as the object person image for identification; the identification means 10 calculates a feature vector of the object person image for identification, compares the same with feature vectors of person images of multiple people to be identified and being recorded in a recording part, for identifying the person in the object person image for identification.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an apparatus for identifying a person. [Background technology]

[0002] There have been proposed devices that identify which person an image of a person is. For example, Patent Document 1 discloses a device that calculates a feature vector (feature amount) of a person's image and identifies the person based on the similarity of the feature vector.

[0003] This allows for accurate person identification. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication 2019-186955 Summary of the Invention [Problem to be solved by the invention]

[0005] However, because the device in Patent Document 1 simply uses feature vectors of person images, it does not have high accuracy in determining that the same person photographed under different conditions (such as a sunset or other conditions with different lighting conditions) or with different cameras is the same person.

[0006] In addition, since identification is performed based only on the feature vectors of the person image, there is a limit to how much the identification accuracy can be improved.

[0007] In view of the above-mentioned problems, an object of the present invention is to provide a person identification device that can improve the accuracy of identifying people captured in different situations or with different cameras, or a person identification device that aims to further improve identification accuracy. [Means for solving the problem]

[0008] The features of the present invention that can be applied independently or in combination are listed below.

[0009] (1)(2) The person identification device of the present invention comprises a recording unit which records feature vectors of images of multiple people to be identified, a target person image acquisition means which acquires a target person image of a target person, a feature acquisition means which acquires image features of the target person image, a change determination means which determines, based on the image features, whether it is better to change at least one element in the color space of the target person image in order to improve the identification accuracy in the identification means, an identification image determination means which obtains a target person image for identification with or without changing the target person image based on the decision of the change determination means, and an identification means which calculates the feature vector of the target person image for identification, compares it with the feature vectors of the multiple person images to be identified recorded in the recording unit, and identifies the person in the target person image for identification.

[0010] It is determined whether changing at least one element in the color space of the target person image will improve the identification accuracy, and if the identification accuracy will improve, the element is changed, so that the person can be identified with high accuracy even if the target person image is captured under different circumstances.

[0011] (3) The person identification device of the present invention is characterized in that the recording unit provides images of a plurality of people to be identified to a trained deep metric learning model that has been trained to calculate feature vectors of person images so that feature vectors of person images of the same person are similar and feature vectors of person images of different people are not similar, and records the calculated feature vectors.

[0012] Therefore, even if there are only a few person images to be identified, an appropriate person DB can be constructed.

[0013] (4) In the human identification device according to the present invention, the elements in the color space are L * a * b* The present invention is characterized in that it includes at least one of the above.

[0014] Therefore, by performing these image quality changes, it is possible to improve the identification accuracy.

[0015] (5) In the human identification device according to the present invention, the elements in the color space are the average brightness divided by the image size, or the average b * Multiply by the clarity and average a * It is characterized in that it is obtained by dividing by.

[0016] Therefore, by performing these image quality changes, it is possible to improve the identification accuracy.

[0017] (6) The person identification device according to the present invention is characterized in that the image features are at least the average brightness, average a * , a * Median, average b * , b * The features of the method include either the median value of the image, the number of pixels, or the sharpness.

[0018] Therefore, based on these features, it can be estimated whether or not image quality should be changed to improve identification accuracy.

[0019] (7) The person identification device of the present invention is characterized in that the change determination means plots a plurality of image features of the target person images in a multidimensional space of predetermined image features in which it is plotted whether or not the identification accuracy by the identification means improves when an element in the color space of a plurality of target person images having a predetermined plurality of image features is changed, and determines whether or not to change the element in the color space.

[0020] Therefore, based on these features, it can be estimated whether or not image quality should be changed to improve identification accuracy.

[0021] (8) The person identification device according to the present invention is characterized in that the person image includes a background, and the image features include the background.

[0022] Therefore, it is possible to determine whether or not to change the image quality based on image characteristics including the background.

[0023] (9) A person identification device according to the present invention is characterized in that it performs person identification on person images captured by cameras installed at different positions, and tracks the movement of people at different positions.

[0024] Therefore, person tracking can be performed based on the identification result.

[0025] (10)(11) The person identification device of the present invention is characterized in that the recording unit also records the types of belongings of each person, the target person image acquisition means extracts and acquires the target person image of the target person from the captured image, and the device further includes a possessions information acquisition means for estimating the type or presence or absence of possessions of the target person from the captured image, and the identification means identifies the person in the target person image taking into account the type or presence or absence of possessions of the person.

[0026] Therefore, the identification accuracy can be further improved.

[0027] (12)(13) The person identification device of the present invention includes a recording unit which records feature vectors of images of multiple people to be identified and the type of belongings of each person, a target person image acquisition means which extracts and acquires a target person image of the target person from a captured image, a possessions information acquisition means which estimates the type or presence or absence of possessions of the target person from the captured image, and an identification means which calculates the feature vector of the target person image, compares it with the feature vectors of the multiple person images to be identified recorded in the recording unit, and identifies the person in the target person image based on the type or presence or absence of possessions of the target person and the type or presence or absence of possessions of each person recorded in the recording unit.

[0028] Since person identification is performed taking into account information about possessions, the accuracy of identification can be improved.

[0029] In this embodiment, step S2 corresponds to the "target person image acquisition means."

[0030] In this embodiment, step S4 corresponds to the "characteristics acquisition means."

[0031] In this embodiment, step S5 corresponds to the "change determining means."

[0032] In the embodiment, steps S6, S7, and S8 correspond to the "identification image determining means."

[0033] In the embodiment, steps S9 and S10 correspond to the "identification means."

[0034] In this embodiment, step S20 corresponds to the "belongings information acquisition means."

[0035] The term "apparatus" is a concept that includes not only one consisting of one computer, but also one consisting of multiple computers connected via a network, etc. Therefore, when the means of the present invention (or a part of the means) is distributed among multiple computers, these multiple computers correspond to the apparatus.

[0036] The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also programs in source format, compressed programs, encrypted programs, programs that work in conjunction with an operating system to perform their functions, and the like. [Brief description of the drawings]

[0037] [Figure 1] 1 illustrates a functional configuration of a person identification device according to an embodiment of the present invention. [Diagram 2] 1 shows the hardware configuration of a person identification device. [Diagram 3] FIG. 1 is a diagram for explaining a trained deep metric learning model FlipReID. [Figure 4] This is a person image to be registered in the person DB 38. [Figure 5A] This is a feature vector recorded in the person DB 38. [Figure 5B] This is a feature vector recorded in the person DB 38. [Figure 6] 13 is a flowchart of a person identification program 36. [Figure 7] FIG. 7A is a diagram showing a captured image, and FIG. 7B is a diagram showing person extraction by YOLO. [Figure 8] FIG. 8A is an image of the target person, and FIG. 8B is an image with a lowered a*. [Figure 9] FIG. 13 is a diagram showing grouping of human images for pre-analysis based on whether or not there is improvement in accuracy when a* is lowered and when it is not. [Figure 10] FIG. 13 is a diagram showing a state in which person images of groups 2 and 3 are plotted in a multidimensional space according to image features. [Figure 11] FIG. 13 is a diagram showing the number of matches when k is changed in the pre-analysis. [Figure 12] 13 illustrates a functional configuration of a person identification device according to a second embodiment. [Figure 13] FIG. 13 is a diagram showing a person DB 38. [Figure 14] 13 is a flowchart of a person identification program 36. [Figure 15] 11A and 11B are diagrams showing the state of person recognition and belongings recognition in a captured image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] 1. First embodiment 1.1 Functional configuration A functional configuration diagram of a person identification device according to an embodiment of the present invention is shown in Fig. 1. A recording unit 12 pre-records feature vectors 14 of human images of a plurality of people to be identified.

[0039] A target person image acquisition means 2 acquires a target person image. A feature acquisition means 4 analyzes and acquires image features of the target person image. A change determination means 8 determines, based on the image features, whether or not it is advisable to change at least one element in the color space of the target person image in order to improve the identification accuracy of an identification means 10.

[0040] When the change determination means 8 determines that a change should be made, the image for identification determination means 6 changes at least one element in the color space of the target person image to generate a target person image for identification. When the change determination means 8 determines that no change should be made, the target person image is not changed and is used as the target person image for identification as is.

[0041] The identification means 10 calculates a feature vector of the image of the person to be identified, and compares it with feature vectors of images of a plurality of people to be identified that are recorded in a recording unit, thereby identifying the person in the image of the person to be identified.

[0042] In this embodiment, a determination is made as to whether changing at least one element in the color space of the target person image will improve the identification accuracy, and if the identification accuracy will improve, the element is changed, so that the person can be identified with high accuracy even if the target person image was captured under different circumstances.

[0043] 1.2 Hardware Configuration The hardware configuration of the person identification device is shown in Fig. 2. A CPU 20 is connected to a memory 22, a display 24, an SSD 26, a DVD-ROM 28, a keyboard / mouse 30, and a communication circuit 32.

[0044] The communication circuit 32 is a circuit for connecting to the Internet. In this example, a camera 42 installed inside or outside a building is connected via the Internet. The communication circuit 32 enables the CPU 20 to obtain images captured by the camera 42 via the Internet.

[0045] The SSD 26 stores an operating system 34, a person identification program 36, and a person DB 38. The person identification program 36 performs its functions in cooperation with the operating system 34. These programs were originally stored on a DVD-ROM 40 and were installed in the SSD 26 via the DVD-ROM drive 28.

[0046] The person DB 38 is a database in which feature vectors of images of multiple people to be identified are registered in advance. The person identification program 36 calculates the feature vector of the target person image acquired by the camera 42, compares it with the feature vector in the person DB 38, and identifies which person it is.

[0047] 1.3 Person Identification Processing The person identification process by the person identification device will be described below. In this embodiment, a plurality of people are registered in a person DB in advance, and it is determined which of the registered people a person captured in a target person image is.

[0048] Therefore, in this embodiment, it is necessary to record the person DB 38 in advance in the SSD 26. First, the person DB 38 will be described.

[0049] (1) Person DB38 The person DB 38 is a database that registers a large number of feature vectors of images of multiple known people to be identified. During person identification, the feature vector of an unknown person's image is calculated, and it is used to identify the person by determining whether it is similar to the feature vector of any person registered in the person DB 38.

[0050] In this embodiment, in order to construct the person DB 38, a deep metric learning model is used that is trained so that multiple feature amounts of images (including images of multiple angles and postures for each person) of multiple people (people other than the person to be identified) are not similar for images of different people, but are similar for images of the same person. For example, FlipReID can be used.

[0051] Even if the images are of the same person, the features (edges, etc.) extracted from the images will be different depending on the angle (front, back, side, etc.) and the posture (head up, head down, etc.). Deep metric learning learns that feature vectors represented by multiple features are similar for the same person and not similar for different people.

[0052] This is shown diagrammatically in Figure 3. When feature vectors represented by multiple feature amounts are arranged in a multidimensional space based on the feature amounts (for simplicity's sake, a two-dimensional space is used in Figure 3), feature extraction is learned so that the feature vectors of person α (person β) approach each other.

[0053] Therefore, when images of multiple known people to be identified, as shown in FIG. 4, are given to the deep metric learning model FlipReID trained in this way, it will output an approximate feature vector for each person. The feature vectors of each image are recorded in association with the person in the person DB 38. FIG. 5A shows an example of the person DB 38. FIG. 5B shows this plotted in a multidimensional space (two-dimensional space in the figure).

[0054] (2) Person Identification Processing 6 shows a flowchart of the person identification program 36. The CPU 20 acquires an image captured by the camera 42 (step S1). It is possible to acquire an image captured by the camera 42 each time it is acquired, or it is possible to acquire images captured by all the cameras 42 together, record them once in the SSD 26, and read them out sequentially from the SSD 26. Here, the description will be given on the assumption that an image such as that shown in FIG. 7A has been acquired.

[0055] The CPU 20 extracts the person from this captured image and surrounds it with a boundary box BX (step S2). The portion surrounded by the boundary box BX is extracted as a target person image. An example of the extracted person image is shown in Fig. 8A.

[0056] In addition, to extract people from captured images, a deep learning model trained to extract various objects including people can be used. For example, a deep learning model with the YOLO algorithm trained with images of many different objects can be used. In this case, the learning data is the captured image and the coordinate values ​​(coordinate positions in the captured image) of the boundary box surrounding the object (e.g., a person) captured in the captured image, with the type of object (e.g., a person) added as the correct answer data.

[0057] Next, the CPU 20 determines whether to perform identification using the extracted person image as is, or to change the image quality of the extracted person image before performing identification.

[0058] Here, first, a process of performing identification using the extracted person image as is will be described.

[0059] The CPU 20 provides the extracted person image (target person image) to the trained deep metric learning model FlipReID used when generating the person DB 38, and calculates a feature vector (step S9). The CPU 20 finds a feature vector that is closest to the feature vector of this person image from the person DB 38 (see FIG. 5). That is, as shown in FIG. 5b, in a multidimensional space formed by a plurality of feature amounts, the CPU 20 finds a feature vector that is closest to the feature vector U of the target person image from the person DB 38. In the example of FIG. 5b, the feature vector V2.1 of person 2 is closest to the feature vector U, so the person in the target person image is identified as person 2.

[0060] In this way, the person shown in the target person image can be identified as the person associated with the feature vector in the person DB 38 (step S10).

[0061] In this manner, the CPU 20 refers to the person DB 38 to identify the person.

[0062] As described above, before identifying a person, the CPU 20 determines whether to use the extracted person image (target person image) as is to identify the person, or to change the image quality of the extracted person image (target person image) (change elements in the color space) before identifying the person. The CPU 20 performs this determination process based on the image characteristics of the target person image. The process will be described below.

[0063] First, the CPU 20 calculates image features of the target person image (FIG. 8A) (step S4). In this embodiment, the average brightness, median brightness, and average a * , a * Median, average b * , b * The system calculates nine image features: median of pixel size, image width, image height, and sharpness.

[0064] The average brightness is the average brightness of each pixel over the entire image. The median brightness is the median brightness of each pixel over the entire image.* is the a in the CIELAB color space for each pixel * is averaged over the entire image. * The median of each pixel is * The median value of the entire image. * is the b in the CIELAB color space for each pixel * is averaged over the entire image. * The median of each pixel is * is the median value of the entire image. The width of the image is the number of pixels in the horizontal direction of the image. The height of the image is the number of pixels in the vertical direction of the image. Sharpness is calculated by converting the image to grayscale and applying a Laplacian filter to it to calculate the variance of pixel values. If the variance is large, more edges are visible and the image is sharp.

[0065] Next, the CPU 20 determines whether it is better to use the person image as it is to calculate a feature vector and perform identification by referring to the person DB 38, or whether it is better to use the person image a * A feature vector is calculated from an image (image with changed image quality) with its image quality generally lowered (10% lower in this example), and the person DB 38 is referenced to determine whether or not identification should be performed (step S5).

[0066] This determination is made using the k-nearest neighbor algorithm (k-nn) based on data prepared in advance as follows.

[0067] First, prepare multiple person images of people for pre-analysis (images of people other than the people registered in the person DB 38). Provide these multiple person images to the trained deep metric learning model FlipReID to generate a person DB for people for pre-analysis.

[0068] Next, a large number of target person images of the person for pre-analysis (images with different angles and postures from the above-mentioned registration image) are prepared, and are given to the trained deep distance learning model FlipReID to calculate feature vectors, and the person is identified by referring to the person DB. Note that here, since the people for pre-analysis are all known people, it is possible to determine whether or not this identification result is correct. In other words, the images can be divided into those for which identification was successful and those for which identification was unsuccessful.

[0069] Next, for the target person image for pre-analysis, * The image quality is changed by generally lowering the image quality (10% in this example) to obtain an image for person identification. This image for person identification is fed to the trained deep distance learning model FlipReID to calculate the feature vector, and the person is identified by referring to the person DB. In this case, too, the image for person identification can be divided into those for which identification was successful and those for which identification was unsuccessful.

[0070] Therefore, a * The person image without changing a * When the success and failure of the human images with reduced image quality are tabulated, they are classified into four groups, as shown in Figure 9.

[0071] In this embodiment, a * Group 2: Person images for which lowering a is deemed undesirable for improving identification accuracy; * Using the images of people in group 3, for which it is considered desirable to reduce the brightness of the images in order to improve the identification accuracy, we compared the image features of each person (average brightness, median brightness, average a * , a * Median, average b * , b * Calculate the median of the image, the image width, the image height, and clarity.

[0072] Then, the person images of groups 2 and 3 are arranged in a multi-dimensional space with these image features as axes. The arranged state is shown in Fig. 10A. Note that in this example, since there are nine image features, the arrangement is in a nine-dimensional space, but for simplicity's sake, Fig. 10A shows it in a two-dimensional space.

[0073] FIG. 10A shows a * Therefore, by examining which group the image features of the person image (FIG. 8A) of an unknown person calculated in step S4 are closest to, a * It can be determined whether the value should be decreased or not.

[0074] In this embodiment, k=3 and the K-nn method is used to determine the feature. FIG. 10B shows a plot T of image features of a person image where the person is unknown. Since k=3, three plots close to T are selected. The selected plots are in group 2 (a * Group 3 (a * Since there is only one option, by majority vote, a * It may be decided that the saturation voltage should not be reduced.

[0075] In this embodiment, k is an odd number, but if k is an even number, the decision may not be made by majority vote. In that case, it is better to decide based on which group the plot closest to plot T belongs to.

[0076] In this manner, the CPU 20 executes the following in step S5: * Should a be lowered? * Decide whether or not to lower the

[0077] Next, the CPU 20 * If it is determined that the value of a should not be reduced, the person image (FIG. 8A) extracted in step S2 is used as the person image for identification (step S7). *If it is determined that the image quality should be reduced, the image quality is reduced by reducing the overall image quality of the person image (FIG. 8A) extracted in step S2. * This is used as the person image for identification (step S8). * The image of a person for identification obtained by reducing the brightness is shown in Figure 8B. Although it is difficult to distinguish in a black and white image, the person has a slightly bluish color tone.

[0078] Next, the CPU 20 provides the person image for identification to the trained deep metric learning model FlipReID used when generating the person DB 38, and calculates a feature vector (step S9). The CPU 20 finds a feature vector that is closest to the feature vector of this person image from the person DB 38 (see FIG. 5). That is, as shown in FIG. 5B, in a multidimensional space formed by a plurality of feature amounts, the CPU 20 finds a feature vector that is closest to the feature vector of the target person image from the person DB 38 (step S10). In this way, the person is identified.

[0079] In this embodiment, a * Will the accuracy of identification improve if we generate a person image for identification without lowering a * The method determines whether generating an identification image for a person by lowering the image quality based on the image characteristics of the target person image. Therefore, the image quality can be changed based on the image characteristics of the target person image to improve the identification accuracy.

[0080] The CPU 20 repeats the above-mentioned processes of steps S4 to S10 the same number of times as the number of people captured in the captured image acquired in step S1 (steps S3, S11).

[0081] When the CPU 20 has identified people for all of the person images extracted from the captured images, it performs a tracking process (step S12). That is, it determines to which position the identified person has moved, taking into consideration the person identification results in captured images captured at the same time by other nearby cameras.

[0082] Next, the CPU 20 acquires captured images from the other cameras 42 and repeatedly executes step S2 and subsequent steps. When the processing for the captured images from all the cameras 42 is completed in this manner, the CPU 20 returns to the processing for acquiring the captured image from the first camera 42 and repeats the processing.

[0083] In this manner, person identification can be performed for images captured at each time by each camera 42.

[0084] (3) Determining k in the K-nn method The above step S5 (a * In the above description of the determination of whether or not to reduce the noise level, k is set to 3. It is preferable to determine the value of k as follows.

[0085] Using known person images for pre-analysis, * When a is lowered, * Figure 9 shows the results of investigating whether the identification accuracy changes when the value of a is not lowered (this point has already been mentioned). * Group 3 (a * The image features (average brightness, median brightness, average a * , a * Median, average b * , b * FIG. 10A shows a plot of the image quality on a multi-dimensional interval (two-dimensional space in the figure) based on the following three parameters: median value of the image quality, image width, image height, and sharpness.

[0086] As described above, the image features T of the target image of an unknown person are plotted in this multidimensional space (see FIG. 10B), and the three adjacent plots (k=3) are judged to belong to which group, and a * The IMF is currently deciding whether to lower the

[0087] Here, a question arises as to what value k should be set to. In this embodiment, the value of k is determined as follows.

[0088] First, we prepare a number of pre-analysis images of people whose personalities are known, and compare the images with the original image and the * Using the reduced-level person image, the trained deep distance learning model FlipReID is used to calculate a feature vector, and the person is identified by referring to the person DB.

[0089] As a result, these person images are grouped as shown in FIG. 9. Among these, only person images belonging to groups 2 and 3 are extracted. The person images belonging to groups 2 and 3 are * It is known whether it is better to lower the

[0090] Therefore, for each person image, image features are calculated and arranged in a multidimensional space as a plot T as shown in FIG. 10B. First, k=1 is set, and the above-mentioned previously known a * It is judged whether it is better to reduce k. Then, this process is repeated for k=2, k=3, etc.

[0091] An example of the results is shown in Figure 11. In this example, the number of matches is recorded when k is changed for 20 person images. Since k=3 has the highest number of matches, this is selected.

[0092] Since the value of k is determined in the above manner, the optimal value of k may vary depending on the camera 42, the imaging conditions, and the like.

[0093] 1.4 Variations (Other) (1) In the above embodiment, the camera 42 is connected to the person identification device via the Internet. However, it may be connected via a LAN or a dedicated line. Also, the captured image recorded by the camera 42 may be recorded on a portable recording medium and read by the person identification device.

[0094] (2) In the above embodiment, the person DB 38 is recorded in the person identification device. However, the person DB 38 may be recorded in another device connected to a network such as the Internet or a LAN.

[0095] (3) In the above embodiment, a trained deep learning model using the YOLO algorithm is used for object recognition. However, other object recognition algorithms, such as YOLACT, may be used.

[0096] In YOLO, the target person image includes the background image as shown in Figure 8A, and the feature vector and image features are calculated including the background image. In YOLACT, a boundary box that follows the outline of the extracted person instead of a rectangle can be used, so feature vectors and image features can be calculated for person images without a background.

[0097] (4) In the above embodiment, the image features include the average brightness, median brightness, and average a * , a * Median, average b * , b * However, image features based on any combination of these may be used.

[0098] For example, the average brightness divided by the image size may be used as the image feature. In this case, whether the image is small and bright is the feature. * × sharpness) ÷ average a * may be used as image features, in this case being vivid and greenish.

[0099] In addition, at least the feature that changes the image quality (a * ) image features (average a * , a * It is preferable to use the median value of the

[0100] (5) In the above embodiment, *The image quality is changed by reducing a certain percentage. * may be increased by a predetermined ratio. * Or L * Alternatively, a combination of these may be changed. Alternatively, R, G, B, etc. may be changed.

[0101] (6) In the above embodiment, in step S9, the person with the closest feature vector is selected as the person to be identified. However, it is also possible to extract multiple close feature vectors and identify the person based on which person has more features (if there are the same number of features, the person with the closest feature vector is identified).

[0102] (7) In the above embodiment, the person identification device is configured as a stand-alone PC. However, it may be configured as a server device. In this case, the identification result is transmitted to another device such as a terminal device.

[0103] (8) In the above embodiment, a person image to be identified is provided to the trained deep metric learning model FlipReID to calculate a feature vector, which is then registered as the person DB 38. However, an untrained deep metric learning model may be trained using a person image to be identified, and the person image to be identified may be provided to the trained deep metric learning model to calculate a feature vector, which may then be registered as the person DB 38.

[0104] (9) The above modifications may be implemented in combination with each other. Also, the modifications may be implemented in combination with other embodiments.

[0105] 2. Second embodiment 2.1 Overall configuration 12 shows the functional configuration of a person identification device according to the second embodiment. A target person image acquisition means 2 extracts a person image from a captured image. The extracted target person image is provided to a feature acquisition means 4, which calculates a feature vector.

[0106] The possession information acquisition means 18 extracts possessions from the captured image and estimates the presence or absence of possessions and the type of possessions.

[0107] The person DB 12 records feature vectors 14 of images of a person to be identified. In addition, if possessions are shown in the image at the time of registration, the types of the possessions are also recorded in association with the image.

[0108] The identification means 10 refers to the person DB 12 and identifies the person based on the feature vector of the target person image and the possession information.

[0109] In this embodiment, identification is performed taking into account information about possessions, thereby improving the accuracy of identification.

[0110] 2.2 Hardware Configuration The hardware configuration is the same as that shown in FIG. 2 in the first embodiment.

[0111] 2.3 Person Identification Processing (1) Person DB38 The person DB 38 has the same configuration as the first embodiment, except that when registering the feature vector of the person image to be identified, YOLO is used to determine whether the person has any possessions, and if so, the type of possessions identified by YOLO is also registered.

[0112] An example of a person DB 38 generated in this way is shown in Fig. 13. If the person's belongings are captured in the image for registration, the type of the belongings, such as "bag" or "backpack," is also recorded.

[0113] (2) Person Identification Processing 14 shows a flowchart of the person identification program 36. The CPU 20 acquires a captured image from the camera 42 (step S1). Then, the CPU 20 extracts a person image from the captured image using YOLO (step S2). The CPU 20 also estimates the type of belongings of the person using YOLO (step S20).

[0114] Figure 15 shows the extraction status of a person's image and the extraction status of the person's belongings. In YOLO, a label indicating the type (person, bag, backpack, handbag, etc.) is attached to the extracted boundary box, so it is possible to distinguish whether it is a person's image or a belonging. In addition, it is possible to estimate the type of belongings.

[0115] The possession item having the boundary box that overlaps most with the boundary box of a person is determined to be the possession of that person.

[0116] The CPU 20 calculates a feature vector of the extracted person image (step S9). Based on this feature vector, identification is performed by referring to the person DB 38, similarly to the first embodiment (step S10).

[0117] However, in this embodiment, if the target person has possessions, registered data in which the same type of possession is recorded is selected from the person DB 38. For example, if the target person has a "bag", only data in which a "bag" is recorded as a possession is selected from the data in the person DB 38 in Fig. 13.

[0118] The CPU 20 selects the person whose feature vector is most different from the selected data, and identifies the person.

[0119] In this manner, in this embodiment, identification is performed taking into account the type of possession, thereby improving the accuracy of identification.

[0120] 1.4 Variations (Other) (1) In the above embodiment, data in the person DB 38 is selected according to the type of possession of the target person. However, data in the person DB 38 may be selected according to the presence or absence of the possession, regardless of the type of possession.

[0121] (2) The identification method taking into account belongings according to the above embodiment can be implemented in combination with the first embodiment.

[0122] (3) The above-described modified examples may be implemented in combination with each other. Also, the modified examples may be implemented in combination with other embodiments.

Claims

1. a recording unit that records feature vectors of images of a plurality of people to be identified; A target person image acquisition means for acquiring a target person image of a target person; A feature acquisition means for acquiring image features of a target person image; a change determining means for determining whether or not at least one element in a color space of the target person image should be changed in order to improve the identification accuracy of the identifying means, based on the image features; an identification image determination means for obtaining a target person image for identification with or without changing the target person image based on the determination of the change determination means; an identification means for calculating a feature vector of the target person image for identification, and comparing the feature vectors of the target person images recorded in the recording unit to identify the person in the target person image for identification; A person identification device comprising:

2. A person identification program for implementing a person identification device by a computer, the program comprising: A target person image acquisition means for acquiring a target person image of a target person; A feature acquisition means for acquiring image features of a target person image; a change determining means for determining whether or not at least one element in a color space of the target person image should be changed in order to improve the identification accuracy of the identifying means, based on the image features; an identification image determination means for obtaining an identification image for a target person by changing or not changing the previous target person image based on the decision of the change determination means; A person identification program for calculating a feature vector of the target person image for identification, and comparing it with feature vectors of multiple person images that are the target of identification and are recorded in a recording unit, thereby functioning as an identification means for identifying the person in the target person image for identification.

3. In the device of claim 1 or the program of claim 2, The recording unit is a device or program characterized in that images of multiple people to be identified are provided to a trained deep metric learning model that has been trained to calculate feature vectors of person images so that feature vectors of person images of the same person are similar and feature vectors of person images of different people are not similar, and the calculated feature vectors are recorded.

4. In the device of claim 1 or the program of claim 2, The elements in the color space are L ＊ a ＊ b ＊ An apparatus or program comprising at least one of the above.

5. In the device of claim 1 or the program of claim 2, The change determination means plots a plurality of image features of the target person images on the multidimensional space of image features that plots whether or not the identification accuracy by the identification means improves when a single element in the color space of a plurality of target person images having a predetermined plurality of image features is changed, and determines whether or not to change the single element in the color space.

6. In the device of claim 1 or the program of claim 2, The image features include at least the average brightness, average a ＊ , a ＊ Median, average b ＊ , b ＊ A device or program comprising any one of the following: median value, number of pixels, or sharpness.

7. In the device of claim 1 or the program of claim 2, The elements in the color space are the average brightness divided by the image size, or the average b ＊ Multiply by the clarity and get the average a ＊ An apparatus or program for dividing a number by a factor of 1.

8. In the device of claim 1 or the program of claim 2, The person image includes a background, The image features include the background.

9. In the device of claim 1 or the program of claim 2, An apparatus or program for performing person identification on images of people captured by cameras installed at different positions, and tracking the movement of people at different positions.

10. 2. The apparatus of claim 1, The record section also records the types of possessions of each person, the target person image acquisition means extracts and acquires a target person image of a target person from a captured image, a possession information acquisition means for estimating the type or presence of possessions of the target person from the captured image, The apparatus is characterized in that the identification means identifies the person in the target person image while taking into consideration the type of possessions of the person or whether or not they are present.

11. In the program of claim 2, The record section also records the types of possessions of each person, the target person image acquisition means extracts and acquires a target person image of a target person from a captured image, The program further causes the computer to function as possession information acquisition means for estimating the type or presence of possessions of the target person from the captured image, The program, wherein the identification means identifies the person in the target person image while taking into consideration the type of possessions of the person or whether or not they are present.

12. a recording unit for recording feature vectors of images of a plurality of people to be identified and the types of possessions of each person; a target person image acquisition means for extracting and acquiring a target person image of a target person from a captured image; a possession information acquisition means for estimating the type or presence of possessions of the target person from the captured image; an identification means for calculating a feature vector of the target person image, comparing it with feature vectors of the images of a plurality of people to be identified and recorded in the recording unit, and identifying the person in the target person image based on the type or presence or absence of possessions of the target person and the type or presence or absence of possessions of each person recorded in the recording unit; A person identification device comprising:

13. A person identification program for implementing a person identification device by a computer, the program comprising: a target person image acquisition means for extracting and acquiring a target person image of a target person from a captured image; a possession information acquisition means for estimating the type or presence of possessions of the target person from the captured image; A person identification program that calculates a feature vector of the target person image, compares it with feature vectors of multiple person images to be identified that are recorded in the recording unit, and functions as an identification means for identifying the person in the target person image based on the type or presence or absence of belongings of the target person and the type or presence or absence of belongings of each person recorded in the recording unit.

Images