Verification device, verification method, and recording medium
Patent Information
- Application Number
- JP2024507215
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Priority Date
- 2022-03-14
- Filing Date
- 2022-03-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing verification devices face challenges in accurately matching query images with registered images due to factors like similar but not identical features, hidden face parts, and varying imaging conditions, leading to incorrect ranking of similar images, where non-matching images are often ranked higher than matching ones.
A verification device that corrects the query feature amount by calculating a new feature amount based on the statistical values of similar images, reducing the contribution of common features among similar images, thereby improving the matching accuracy by re-extracting similar images using the corrected feature amount.
The solution enhances the matching accuracy of query images by reducing the influence of common features among similar images, ensuring that matching images are ranked higher than non-matching ones, thus improving the overall verification process.
Abstract
Description
Verification device, verification method, and recording medium
[0001] The present disclosure relates to the technical field of a matching device, a matching method, and a recording medium that can extract data similar to query data from a plurality of registered data by matching query data with the plurality of registered data, for example.
[0002] An example of a matching device capable of matching query data with a plurality of registered data is described in Patent Document 1. Other prior art documents related to this disclosure include Patent Documents 2 to 3 and Non-Patent Document 1.
[0003] JP 2003-346149 A JP 2009-031991 A JP 2015-018401 A
[0004] Zhun Zhong et. al, “Re-ranking Person Re-identification with k-reciprocal Encoding”, arXiv 1701.08398, May 5, 2017
[0005] An object of this disclosure is to provide a matching device, a matching method, and a recording medium that aim to improve upon the techniques described in prior art documents.
[0006] One aspect of the matching device disclosed herein includes a matching means that extracts, from a plurality of registered data, a plurality of first similar data that are similar to the query data, based on query features that are features of the query data, and a correction means that corrects the query features based on calculated values of a plurality of registered features that are features of the plurality of first similar data, and the matching means extracts at least one second similar data that is similar to the query data from the plurality of registered data, based on the query features corrected by the correction means.
[0007] One aspect of the matching method disclosed herein includes extracting, from a plurality of registered data, a plurality of first similar data that are similar to the query data based on query features that are features of the query data; correcting the query features based on statistical values of a plurality of registered features that are features of the plurality of first similar data; and extracting, from the plurality of registered data, at least one second similar data that is similar to the query data based on the corrected query features.
[0008] One aspect of a recording medium of this disclosure is a recording medium having recorded thereon a computer program that causes a computer to execute a matching method, the matching method including: extracting, from a plurality of registered data, a plurality of first similar data that are similar to the query data based on query features that are features of the query data; correcting the query features based on statistical values of a plurality of registered features that are features of the plurality of first similar data; and extracting, from the plurality of registered data, at least one second similar data that is similar to the query data based on the corrected query features.
[0009] FIG. 1 is a block diagram showing the configuration of a matching device in a first embodiment. FIG. 2 schematically shows a query image, a plurality of registered images, and at least one similar image. FIG. 3 schematically shows a query image, a plurality of registered images, and at least one similar image. FIG. 4 is a block diagram showing the configuration of a matching device in a second embodiment. FIG. 5 is a flowchart showing the flow of matching operations performed by the matching device in the second embodiment. FIG. 6 schematically shows a query image, a plurality of registered images, and at least one similar image.
[0010] Hereinafter, embodiments of a verification device, a verification method, and a recording medium will be described with reference to the drawings.
[0011] (1) First Embodiment First, a first embodiment of a verification device, a verification method, and a recording medium will be described. Hereinafter, the first embodiment of the verification device, the verification method, and a recording medium will be described with reference to Fig. 1 using a verification device 1000 to which the first embodiment of the verification device, the verification method, and a recording medium is applied. Fig. 1 is a block diagram showing the configuration of the verification device 1000 in the first embodiment.
[0012] As shown in FIG. 1 , the matching device 1000 according to the first embodiment includes a matching unit 1001, which is a specific example of a "matching means" described in the appendix, and a correction unit 1002, which is a specific example of a "correction means" described in the appendix. The matching unit 1001 extracts, from a plurality of registered data sets, a plurality of first similar data sets that are registered data sets similar to the query data, based on query features that are features of the query data. The correction unit 1002 corrects the query features based on statistical values of a plurality of registered features that are features of the plurality of first similar data sets extracted by the matching unit 1001. When the correction unit 1002 corrects the query features, the matching unit 1001 extracts, from the plurality of registered data sets, at least one second similar data set that is registered data sets similar to the query data, based on the corrected query features.
[0013] The matching device 1000 according to the first embodiment described above corrects the query features based on the results of an extraction process that extracts registered data similar to the query data as first similar data based on the query features, and then performs the matching process again based on the corrected query features. As a result, the matching accuracy of the query data is improved compared to when the matching process is not performed again.
[0014] (2) Second Embodiment Next, a second embodiment of the verification device, the verification method, and the recording medium will be described. In the following, the second embodiment of the verification device, the verification method, and the recording medium will be described using the verification device 1 to which the second embodiment of the verification device, the verification method, and the recording medium is applied.
[0015] (2-1) Overview of the Matching Device 1 in the Second Embodiment As shown in FIG. 2 , the matching device 1 performs a matching operation in which a query image IMG_Q is matched with a plurality of registered images IMG_R, and at least one registered image IMG_R that is similar to the query image IMG_Q is extracted from the plurality of registered images IMG_R as at least one similar image IMG_S. Specifically, the matching device 1 performs a matching operation in which K (where K is a variable indicating an integer equal to or greater than 1) similar images IMG_S that are similar to the query image IMG_Q. For example, the matching device 1 may extract, as the similar image IMG_S, one registered image IMG_R that satisfies an extraction condition that the similarity between one registered image IMG_R extracted as the similar image IMG_S and the query image IMG_Q is higher than the similarity between other registered images IMG_R that are not extracted as the similar image IMG_S and the query image IMG_Q.
[0016] Note that if there is a registered image IMG_R that is identical to the query image IMG_Q, the similarity between the registered image IMG_R and the query image IMG_Q will be relatively high, and therefore the registered image IMG_R is likely to be extracted as a similar image IMG_S. Therefore, examples of the "registered image IMG_R similar to the query image IMG_Q" in the second embodiment include not only the "registered image IMG_R that is similar to but not identical to the query image IMG_Q" but also the "registered image IMG_R that is identical to the query image IMG_Q."
[0017] In the second embodiment, an example will be described in which a person's face is captured in each of a query image IMG_Q and a plurality of registered images IMG_R, as shown in FIG. 2 . In the following description, a person captured in the query image IMG_Q will be referred to as a query person, and a person captured in the registered image IMG_R will be referred to as a registered person. In this case, the matching operation of extracting a registered image IMG_R similar to the query image IMG_Q as a similar image IMG_S may be considered equivalent to the operation of extracting a registered image IMG_R in which a registered person similar to the query person is captured as a similar image IMG_S.
[0018] The verification device 1 may perform the verification operation to identify a registered person who is the same person as the query person (i.e., to perform person verification), or may perform the verification operation to search for registered images in which the query person appears as a registered person (i.e., to perform image search).
[0019] Note that if there is a registered image IMG_R that includes a registered person who is the same as the query person, the similarity between the registered image IMG_R and the query image IMG_Q will be relatively high, and therefore the registered image IMG_R is likely to be extracted as a similar image IMG_S. Therefore, examples of the "registered image IMG_R similar to the query image IMG_Q" in the second embodiment include not only "a registered image IMG_R that includes a registered person who is similar to but not the same as the query person," but also "a registered image IMG_R that includes a registered person who is the same as the query person."
[0020] (2-2) Technical Issues to be Solved by the Matching Device 1 in the Second Embodiment The matching device 1, which performs a matching operation to extract a registered image IMG_R similar to a query image IMG_Q as a similar image IMG_S, has the following technical issue. Below, the technical issue will be described using an example in which the query person appearing in the query image IMG_Q is person A, and the multiple registered images IMG_R include five registered images IMG_R in which person A, person B, person C, person D, and person E appear as the five registered people. In this case, as shown in FIG. 2 , the matching device 1 may typically extract, as similar images IMG_S, a registered image IMG_R in which a registered person (person A) who is the same as the query person (person A) appears, and a registered image IMG_R in which a registered person who is not the same as the query person (person A) but is similar to the query person (person A) appears. For example, FIG. 2 illustrates an example in which the matching device 1 extracts, as similar images IMG_S, a registered image IMG_R that includes a registered person (person A), a registered image IMG_R that includes a registered person (person B), and a registered image IMG_R that includes a registered person (person C). In this case, when multiple similar images IMG_S are sorted in order of similarity, as shown in FIG. 2, a similar image IMG_S that includes a registered person (person A) who is the same as the query person (person A) is typically listed higher than a similar image IMG_S that includes a registered person (person B or person C) who is not the same as the query person (person A). In this case, the matching operation extracts the registered person (person A) who is the same as the query person (person A) as the registered person who is most likely to be the same as the query person (person A). Therefore, it can be said that the matching accuracy of the query image IMG_Q is high.
[0021] On the other hand, depending on the state of the query person appearing in the query image IMG_Q and the state of the registered person appearing in the registered image IMG_R, a similar image IMG_S appearing in a registered person (person B, C, D, or E) who is not the same person as the query person (person A) may be listed higher than a similar image IMG_S appearing in a registered person (person A) who is the same person as the query person (person A).
[0022] As an example, if a registered person different from the query person (person A) is wearing the same accessory as the query person (person A), the similarity between the registered person different from the query person (person A) and the query person (person A) will be higher than if the registered person different from the query person (person A) is not wearing the same accessory as the query person (person A). In other words, the similarity between the registered image IMG_R in which a registered person different from the query person (person A) appears and the query image IMG_Q will be higher. As a result, a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) appears may be listed higher than a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) appears.
[0023] Examples of the article of wear include at least one of a mask, glasses, sunglasses, and a hat. For example, Figure 3 shows a query image IMG_Q and a registered image IMG_R acquired when a query person (person A) is wearing sunglasses, a registered person (person A) who is the same person as the query person (person A) is not wearing sunglasses, registered people (persons B and C) different from the query person (person A) are wearing sunglasses, and registered people (persons D and E) different from the query person (person A) are not wearing sunglasses. In this case, due to the fact that the query person (person A) is wearing sunglasses, the similarity between the query image IMG_Q and the registered images IMG_R(B) and IMG_R(C) in which the registered people wearing sunglasses (persons B and C) are captured is unintentionally high. As a result, as shown in Figure 3, a similar image IMG_S that includes a registered person (person A) who is the same person as the query person (person A) may be listed higher in rank than a similar image IMG_S that includes a registered person (person A) who is the same person as the query person (person A).
[0024] As another example, when a predetermined portion of the face of a query person (person A) is obscured by a predetermined occlusion, and a predetermined portion of the face of a registered person different from the query person (person A) is obscured by the same occlusion, the similarity between the registered person different from the query person (person A) and the query person (person A) is higher than when a predetermined portion of the face of the registered person different from the query person (person A) is not obscured by the same occlusion. In other words, the similarity between the registered image IMG_R in which a registered person different from the query person (person A) appears and the query image IMG_Q is higher. As a result, a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) appears may be listed higher than a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) appears. Note that an example of an occlusion is at least one of hair and beard.
[0025] As another example, when a predetermined portion of the face of a query person (person A) is not captured in the query image IMG_Q and the same portion of the face of a registered person different from the query person (person A) is not captured in the registered image IMG_R, the similarity between the registered person different from the query person (person A) and the query person (person A) is higher than when the predetermined portion of the face of the query person (person A) is not captured in the query image IMG_Q and the same portion of the face of a registered person different from the query person (person A) is captured in the registered image IMG_R. In other words, the similarity between the registered image IMG_R in which a registered person different from the query person (person A) is captured and the query image IMG_Q is higher. As a result, a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) is captured may be listed higher than a similar image IMG_S in which a registered person (person B, C, D, or E) who is not the same as the query person is captured. One of the reasons why a certain part of a person's face may not appear in the image is that a part of the face falls in the blind spot of the camera due to the orientation of the face relative to the camera.
[0026] As another example, if the imaging conditions under which a query person (person A) is imaged are similar to the conditions under which a registered person other than the query person (person A) is imaged, the similarity between the registered person other than the query person (person A) and the query person (person A) is higher than when the imaging conditions under which the query person (person A) is imaged are not similar to the conditions under which a registered person other than the query person (person A) is imaged. In other words, the similarity between the registered image IMG_R in which a registered person other than the query person (person A) is imaged and the query image IMG_Q is higher. As a result, a similar image IMG_S in which a registered person other than the query person (person B, C, D, or E) is imaged may be listed higher than a similar image IMG_S in which a registered person (person A) who is the same as the query person (person A) is imaged. Note that examples of imaging conditions include at least one of conditions related to the illumination light used to illuminate the query person or the registered person and conditions related to the type of camera used to illuminate the query person or the registered person.
[0027] In this way, when a similar image IMG_S that includes a registered person who is not the same as the query person (person A) is listed higher in rank than a similar image IMG_S that includes a registered person (person A) who is the same as the query person (person A), it can be said that the matching operation has extracted the registered person who is not the same as the query person (person A) as the registered person who is most likely to be the same as the query person (person A). For this reason, it cannot necessarily be said that the matching accuracy of the query image IMG_Q is high.
[0028] Therefore, in the second embodiment, the matching device 1 is configured to be able to achieve the above-mentioned technical solution. Specifically, the matching device 1 performs a matching operation so as to increase the matching accuracy of the query image IMG_Q. Such a matching device 1 will be described in further detail below.
[0029] (2-3) Configuration of the Verification Device 1 First, the configuration of the verification device 1 in the second embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the verification device 1 in the second embodiment.
[0030] 4, the verification device 1 includes a calculation device 11, a storage device 12, and a communication device 13. The verification device 1 may further include an input device 14 and an output device 15. However, the verification device 1 does not necessarily have to include at least one of the input device 14 and the output device 15. The calculation device 11, the storage device 12, the communication device 13, the input device 14, and the output device 15 may be connected via a data bus 16.
[0031] The arithmetic device 11 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 11 loads a computer program. For example, the arithmetic device 11 may load a computer program stored in the storage device 12. For example, the arithmetic device 11 may load a computer program stored in a computer-readable, non-transitory storage medium using a storage medium reading device (not shown) included in the verification device 1. The arithmetic device 11 may acquire (i.e., download or load) the computer program from a device (not shown) located outside the verification device 1 via the communication device 13 (or another communication device). The arithmetic device 11 executes the loaded computer program. As a result, a logical function block for executing the operation (for example, the above-described matching operation) that the verification device 1 should perform is realized within the calculation device 11. In other words, the calculation device 11 can function as a controller for realizing the logical function block for executing the operation (in other words, processing) that the verification device 1 should perform.
[0032] Fig. 4 shows an example of logical functional blocks realized in the arithmetic device 11 for executing the matching operation. As shown in Fig. 4, the arithmetic device 11 includes an image acquisition unit 111, a feature extraction unit 112, a matching unit 113 which is a specific example of "matching means" described in the appendix below, and a feature correction unit 114 which is a specific example of "correction means" described in the appendix below. Note that the respective operations of the image acquisition unit 111, the feature extraction unit 112, the matching unit 113, and the feature correction unit 114 will be described in detail later with reference to Fig. 4 etc.
[0033] The storage device 12 can store desired data. For example, the storage device 12 may temporarily store a computer program executed by the arithmetic device 11. The storage device 12 may temporarily store data that the arithmetic device 11 temporarily uses when the arithmetic device 11 is executing a computer program. The storage device 12 may also store data that the verification device 1 stores long-term. The storage device 12 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 12 may include a non-temporary recording medium.
[0034] The communication device 13 is capable of communicating with devices external to the verification device 1 .
[0035] The input device 14 is a device that accepts information input to the verification device 1 from outside the verification device 1. For example, the input device 14 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the verification device 1. For example, the input device 14 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the verification device 1.
[0036] The output device 15 is a device that outputs information to the outside of the verification device 1. For example, the output device 15 may output information as an image. That is, the output device 15 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 15 may output information as sound. That is, the output device 15 may include an audio device (a so-called speaker) that can output sound. For example, the output device 15 may output information on paper. That is, the output device 15 may include a printing device (a so-called printer) that can print desired information on paper.
[0037] (2-4) Verification Operation Performed by Verification Device 1 Next, the verification operation performed by the verification device 1 in the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the verification operation performed by the verification device 1 in the second embodiment.
[0038] 5 , the image acquisition unit 111 acquires a query image IMG_Q (step S101). For example, if the query image IMG_Q is generated by a camera that captures an image of a query person, the image acquisition unit 111 may acquire the query image IMG_Q from the camera using the communication device 13. For example, if the query image IMG_Q is stored in a device external to the matching device 1, the image acquisition unit 111 may acquire the query image IMG_Q from the device external to the matching device 1 using the communication device 13. For example, if the query image IMG_Q is stored in the storage device 12, the image acquisition unit 111 may acquire the query image IMG_Q from the storage device 12.
[0039] Thereafter, the feature extraction unit 112 extracts query features F_Q, which are features of the query image IMG_Q (step S102). Specifically, the feature extraction unit 112 extracts features of a query person appearing in the query image IMG_Q as the query features F_Q. For example, if the face of the query person appears in the query image IMG_Q, the feature extraction unit 112 may extract features of the query person's face as the query features F_Q.
[0040] In parallel with or before or after the processing from step S101 to step S102, the image acquisition unit 111 acquires a plurality of registered images IMG_R (step S103). For example, if the plurality of registered images IMG_R are stored in a device external to the matching device 1, the image acquisition unit 111 may acquire the plurality of registered images IMG_R from the device external to the matching device 1 using the communication device 13. For example, if the plurality of registered images IMG_R are stored in the storage device 12, the image acquisition unit 111 may acquire the plurality of registered images IMG_R from the storage device 12.
[0041] Thereafter, the feature extraction unit 112 extracts registered features F_R, which are features of each of the multiple registered images IMG_R (step S104). That is, the feature extraction unit 112 extracts multiple registered features F_R corresponding to the multiple registered images IMG_R, respectively. Specifically, the feature extraction unit 112 extracts the feature of a registered person appearing in the registered image IMG_R as the registered feature F_R. For example, if the face of the registered person appears in the registered image IMG_R, the feature extraction unit 112 may extract the feature of the registered person's face as the registered feature F_R.
[0042] If a device external to the matching device 1 or the storage device 12 stores, in addition to or instead of the plurality of registered images IMG_R, a plurality of registered features F_R corresponding to the plurality of registered images IMG_R, respectively, in step S103, the image acquisition unit 111 may acquire the plurality of registered features F_R in addition to or instead of the plurality of registered images IMG_R. In this case, in step S104, the feature extraction unit 112 does not need to extract the plurality of registered features F_R.
[0043] Thereafter, the matching unit 113 performs a matching process to extract K registered images IMG_R similar to the query image IMG_Q from the plurality of registered images IMG_R acquired in step S103 based on the query feature F_Q extracted in step S102, as K similar images IMG_R (step S105). In particular, in step S105, the matching unit 113 performs a matching process to extract a plurality of similar images IMG_R. That is, in step S105, a variable K indicating the number of similar images IMG_R extracted by the matching process is an integer equal to or greater than 2.
[0044] To extract K similar images IMG_S in step S105, the matching unit 113 matches the query feature F_Q extracted in step S102 with the multiple registered features F_R extracted in step S104. Specifically, the matching unit 113 calculates the similarity between the query feature F_Q and each of the multiple registered features F_R. The higher the similarity between the query feature F_Q and the registered feature F_R, the higher the similarity between the query image IMG_Q and the registered image IMG_R. Therefore, the similarity between the query feature F_Q and the registered feature F_R is equivalent to the similarity between the query image IMG_Q and the registered image IMG_R. In other words, the higher the similarity between the query feature F_Q and the registered feature F_R, the higher the similarity between the query person appearing in the query image IMG_Q and the registered person appearing in the registered image IMG_R. Therefore, the similarity between the query feature F_Q and the registered feature F_R is equivalent to the similarity between the query person and the registered person.
[0045] Thereafter, the matching unit 113 extracts K similar images IMG_S based on the calculated similarity. For example, the matching unit 113 may extract K similar images IMG_S with relatively high similarity. That is, the matching unit 113 may extract the K similar images IMG_S in descending order of similarity. In other words, the matching unit 113 may extract, as the K similar images IMG_S, K registered images IMG_R that satisfy an extraction condition that the similarity between each of the K registered images IMG_R extracted as similar images IMG_S and the query image IMG_Q is higher than the similarity between the remaining registered images IMG_R not extracted as similar images IMG_S and the query image IMG_Q.
[0046] Thereafter, the feature correcting unit 114 corrects the query feature F_Q extracted in step S102 based on the result of the matching process in step S105 (step S106). As a result, the feature correcting unit 114 generates a new query feature F_Q corresponding to the corrected query feature F_Q. Note that in the following description, the query feature F_Q corrected in step S106 will be referred to as a "query feature F_Q_correct" to distinguish it from the query feature F_Q that is not corrected in step S106 (i.e., the query feature F_Q extracted in step S102).
[0047] In the second embodiment, the feature amount correction unit 114 corrects the query feature amount F_Q based on K registered feature amounts F_R corresponding to the K similar images IMG_S (i.e., K registered images IMG_R) in step S105. Specifically, the feature amount correction unit 114 calculates a calculation value AV of the K registered feature amounts F_R by a predetermined calculation, and corrects the query feature amount F_Q based on the calculated calculation value AV.
[0048] The registered feature F_R is usually expressed as an M-dimensional feature vector (where M is a variable indicating an integer equal to or greater than 1). In this case, the feature correction unit 114 may calculate, as the calculated value AV, vector operation values of K feature vectors representing the K registered feature F_R, respectively.
[0049] The calculated value AV may be an index value indicating a feature component common to at least two of the K similar images IMG_S. In other words, the calculated value AV may be an index value indicating a feature component common to at least two of the K registered persons who appear in each of the K similar images IMG_S. In other words, the calculated value AV may be an index value indicating a feature component common to at least two of the K registered features F_R. The calculated value AV may be an index value in which the feature component common to at least two of the K similar images IMG_S is more strongly reflected than the feature component unique to each of the K similar images IMG_S. In other words, the calculated value AV may be an index value in which the feature component common to at least two of the K registered persons who appear in each of the K similar images IMG_S is more strongly reflected than the feature component unique to each of the K registered persons who appear in each of the K similar images IMG_S. In other words, the calculated value AV may be an index value in which a feature component common to at least two of the K registered feature amounts F_R is more strongly reflected than a feature component unique to each of the K registered feature amounts F_R. The calculated value AV may be an index value in which a feature component common to at least two of the K similar images IMG_S is a major feature component. In other words, the calculated value AV may be an index value in which a feature component common to at least two of the K registered persons who appear in each of the K similar images IMG_S is a major feature component. In yet other words, the calculated value AV may be an index value in which a feature component common to at least two of the K registered feature amounts F_R is a major feature component.
[0050] An example of such a calculated value AV is a simple average value (e.g., vector average). In this case, the feature amount correction unit 114 may calculate the simple average value of the K registered feature amounts F_R as the calculated value AV. In the simple average value of the K registered feature amounts F_R, a relatively large number of feature amount components common to at least two of the K registered feature amounts F_R remain. On the other hand, in the simple average value of the K registered feature amounts F_R, the norms of the feature amount components unique to each of the K registered feature amounts F_R are reduced by averaging. As a result, the simple average value of the K registered feature amounts F_R becomes an index value in which the feature amount components common to at least two of the K registered feature amounts F_R are more strongly reflected than the feature amount components unique to each of the K registered feature amounts F_R.
[0051] 3 , in a situation where a query person (person A) is wearing sunglasses, the following images are extracted as similar images IMG_S: a registered image IMG_R(A) that includes a registered person (person A) who is the same person as the query person (person A) but is not wearing sunglasses, a registered image IMG_R(B) that includes a registered person (person B) who is not the same person as the query person (person A) but is wearing sunglasses, and a registered image IMG_R(C) that includes a registered person (person C) who is not the same person as the query person (person A) but is wearing sunglasses. In this case, the feature amount correction unit 114 calculates, as the operation value AV, the simple average of the registered feature amount F_R of the registered image IMG_R(A), the registered feature amount F_R of the registered image IMG_R(B), and the registered feature amount F_R of the registered image IMG_R(C). In this case, the calculated value AV is an index value in which the feature component related to sunglasses that is common to two of the three registered feature amounts F_R is more strongly reflected than feature components other than the feature component related to sunglasses. In other words, the calculated value AV is an index value in which the feature component related to sunglasses that is common to two of the three registered feature amounts F_R is the main feature component.
[0052] In the example shown in FIG. 3 , the calculated value AV can also be considered to be an index value in which the feature components related to the region in which sunglasses are reflected, which is common to two of the three registered features F_R, are the main feature components. In this case, the calculated value AV may be considered to be an index value in which the feature components of a specific region of each of the K similar images IMG_S are more strongly reflected than the feature components of regions other than the specific region of each of the K similar images IMG_S. In the example shown in FIG. 3 , the calculated value AV may be considered to be an index value in which the feature components of the eye region (i.e., the eye region including sunglasses) of each of the K similar images IMG_S are more strongly reflected than the feature components of regions other than the eye region of each of the K similar images IMG_S. Therefore, the calculated value AV may be an index value in which the feature components of a specific region of each of the K similar images IMG_S are more strongly reflected than the feature components of regions other than the specific region of each of the K similar images IMG_S.
[0053] 3 , the calculated value AV can also be said to be an index value in which the feature components related to the object, sunglasses, which are common to two of the three registered features F_R, are the main feature components. Therefore, the calculated value AV may be an index value in which the feature components of a specific object common to at least two of the K similar images IMG_S are more strongly reflected than the feature components of objects other than the specific object in each of the K similar images IMG_S.
[0054] After the calculation value AV is calculated, the feature corrector 114 generates a query feature F_Q_correct by subtracting the calculated calculation value AV from the query feature F_Q extracted in step S102. That is, the feature corrector 114 generates the query feature F_Q_correct (that is, corrects the query feature F_Q) using the formula "query feature F_Q_correct=query feature F_Q-calculated value AV".
[0055] As described above, the calculated value AV is an index value in which the feature component common to at least two of the K registered features F_R is the main feature component. In this case, the calculation of subtracting the calculated value AV from the query feature F_Q may be considered to be the calculation of subtracting the feature component common to at least two of the K registered features F_R from the query feature F_Q. As a result, the calculation of subtracting the calculated value AV from the query feature F_Q may be considered to be essentially equivalent to the calculation of reducing the contribution of the feature component common to at least two of the K registered features F_R to the query feature F_Q. As a result, the contribution of the feature component common to at least two of the K registered features F_R to the corrected query feature F_Q_correct is smaller than the contribution of the feature component common to at least two of the K registered features F_R to the uncorrected query feature F_Q. 3 , the calculation of subtracting the calculation value AV from the query feature F_Q may be considered to be substantially equivalent to a calculation of reducing the contribution of the feature component of sunglasses to the query feature F_Q. As a result, the contribution of the feature component of sunglasses to the corrected query feature F_Q_correct becomes smaller than the contribution of the feature component of sunglasses to the uncorrected query feature F_Q.
[0056] Thereafter, the matching unit 113 performs a matching process to extract K similar images IMG_R similar to the query image IMG_Q from the multiple registered images IMG_R acquired in step S103 based on the query feature F_Q_correct corrected in step S106 (step S107). In particular, in step S107, the matching unit 113 performs a matching process to extract at least one similar image IMG_R. That is, in step S107, a variable K indicating the number of similar images IMG_R extracted by the matching process is an integer equal to or greater than 1.
[0057] To extract K similar images IMG in step S107, the matching unit 113 calculates the similarity between the query feature F_Q_correct extracted in step S106 and each of the multiple registered features F_R extracted in step S104. Then, the matching unit 113 extracts K similar images IMG_S based on the calculated similarity. For example, the matching unit 113 may extract K similar images IMG_S with relatively high similarity. That is, the matching unit 113 may extract K similar images IMG_S in descending order of similarity. In other words, the matching unit 113 may extract K registered images IMG_R as K similar images IMG_S that satisfy the extraction condition that the similarity between each of the K registered images IMG_R extracted as similar images IMG_S and the query image IMG_Q is higher than the similarity between the remaining registered images IMG_R not extracted as similar images IMG_S and the query image IMG_Q.
[0058] Here, one of the causes of the technical problem described above, that "a similar image IMG_S that includes a registered person who is not the same as the query person is listed higher in rank than a similar image IMG_S that includes a registered person who is the same as the query person," is that the matching process is performed using a query feature F_Q that has a high contribution level of a feature component common to at least two of the K registered features F_R. For example, in the example shown in FIG. 3 , one of the causes of the technical problem, that "a similar image IMG_S that includes a registered person who is not the same as the query person but who wears sunglasses is listed higher in rank than a similar image IMG_S that includes a registered person who is the same as the query person but who does not wear sunglasses," is that the matching process is performed using a query feature F_Q that has a high contribution level of a feature component common to at least two of the K registered features F_R.
[0059] On the other hand, in the second embodiment, the contribution of feature components common to at least two of the K registered features F_R to the corrected query feature F_Q_correct is relatively small. In the example shown in FIG. 3 , the contribution of feature components of sunglasses to the corrected query feature F_Q_correct is relatively small. Therefore, it can be said that the similarity between the query feature F_Q_correct and the registered feature F_R is a similarity with respect to a feature component different from the feature components common to at least two of the K registered features F_R. In the example shown in FIG. 3 , it can be said that the similarity between the query feature F_Q_correct and the registered feature F_R is a similarity with respect to a feature component different from the feature component of sunglasses. In this case, when the matching process is performed using a query feature F_Q_correct in which the contribution of feature components common to at least two of the K registered features F_R is low, the likelihood that a similar image IMG_S in which a registered person who is not the same person as the query person is listed higher than a similar image IMG_S in which a registered person who is the same person as the query person is included is lower than when the matching process is performed using a query feature F_Q in which the contribution of feature components common to at least two of the K registered features F_R is high.
[0060] In the example shown in FIG. 3 , because the matching process is performed using the query feature F_Q_correct, which has a low contribution level of the sunglasses feature component, it is expected that the similarity ranking of the registered image IMG_R, which includes a registered person wearing sunglasses but who is not the same person as the query person wearing sunglasses, will be lowered. Specifically, when the matching process is performed using the query feature F_Q, which has a high contribution level of the sunglasses feature component, a similar image IMG_S, which includes a registered person wearing sunglasses (person B or C) but who is not the same person as the query person wearing sunglasses (person A), will be listed relatively highly, as shown in FIG. 3 . On the other hand, when the matching process is performed using the query feature F_Q_correct, which has a low contribution level of the sunglasses feature component, it is unlikely that a similar image IMG_S, which includes a registered person wearing sunglasses (person B or C) but who is not the same person as the query person wearing sunglasses (person A), will be listed relatively highly, as shown in FIG. 6 . As a result, when matching is performed using the query feature F_Q_correct, in which the feature component of sunglasses has a low contribution, a similar image IMG_S that includes a registered person wearing sunglasses but who is not the same person as the query person wearing sunglasses is less likely to be listed higher than a similar image IMG_S that includes a registered person who is the same person as the query person wearing sunglasses but who is not wearing sunglasses, compared to when matching is performed using the query feature F_Q, in which the feature component of sunglasses has a high contribution. This increases the matching accuracy of the query image IMG_Q.
[0061] As such, in the second embodiment, the similarity between the registered feature F_R of a registered image IMG_R that includes a registered person other than the query person and the query feature F_Q_correct is smaller than the similarity between the registered feature F_R of a registered image IMG_R that includes a registered person other than the query person and the query feature F_Q. As a result, when the matching process is performed using the query feature F_Q_correct, the similarity ranking of the registered image IMG_R that includes a registered person other than the query person is expected to be lower than when the matching process is performed using the query feature F_Q. As a result, the similar image IMG_S that includes a registered person other than the query person is less likely to be listed higher than the similar image IMG_S that includes a registered person who is the same as the query person. This increases the matching accuracy of the query image IMG_Q.
[0062] (2-5) Technical Effects of the Verification Device 1 in the Second Embodiment As described above, the verification device 1 calculates a new query feature F_Q_correct by correcting the query feature F_Q based on the result of the verification process using the query feature F_Q, and performs the verification process again based on the new query feature F_Q_correct. As a result, the verification accuracy of the query image F_Q is improved compared to when the verification process is not performed again based on the new query feature F_Q_correct.
[0063] In the second embodiment, the feature amount correction unit 114 subtracts the calculation value AV from the query feature amount F_Q to correct the query feature amount F_Q. Therefore, as described above, the feature amount correction unit 114 can appropriately generate a query feature amount F_Q_correct in which the contribution of feature amount components common to at least two of the K registered features amount F_R is relatively small. As a result, the matching accuracy of the query image F_Q is improved.
[0064] In the second embodiment, the feature amount correction unit 114 calculates a simple average value of K registered features F_R as the calculation value AV to correct the query feature amount F_Q. Therefore, as described above, the feature amount correction unit 114 can appropriately generate a query feature amount F_Q_correct in which the contribution of feature amount components common to at least two of the K registered features amount F_R is relatively small. As a result, the matching accuracy of the query image F_Q is improved.
[0065] (2-6) Modification In the above description, the feature amount correction unit 114 calculates the simple average value of the K registered feature amounts F_R as the calculated value AV. However, the feature amount correction unit 114 may calculate other types of average values of the K registered feature amounts F_R as the calculated value AV. For example, the feature amount correction unit 114 may calculate the weighted average value of the K registered feature amounts F_R as the calculated value AV. As an example, the feature amount correction unit 114 may calculate the weighted average value of the K registered feature amounts F_R, where the weight is determined by the rank of the K registered feature amounts F_R (specifically, the rank according to the similarity calculated in the matching process). Specifically, the feature amount correction unit 114 may calculate the weighted average value of the K registered feature amounts F_R, where the weight is increased as the rank (i.e., the similarity) increases, as the calculated value AV.
[0066] Alternatively, the feature amount correction unit 114 may calculate a calculated value AV that is different from the average value. Examples of the calculated value AV that is different from the average value include at least one of the maximum value of the K registered feature amounts F_R, the minimum value of the K registered feature amounts F_R, and the median value of the K registered feature amounts F_R. In short, as described above, any calculated value may be used as the calculated value AV as long as the calculated value AV indicates a feature amount component common to at least two of the K registered images IMG_R each extracted as the K similar images IMG_S.
[0067] In the above description, the feature quantity correction unit 114 generates the query feature F_Q_correct by performing an operation to subtract the calculation value AV from the query feature F_Q. However, the feature quantity correction unit 114 may generate the query feature F_Q_correct by performing an operation different from the operation to subtract the calculation value AV from the query feature F_Q. For example, the feature quantity correction unit 114 may generate the query feature F_Q_correct by performing any operation using the calculation value AV. Considering that the reason for performing the calculation to subtract the calculated value AV from the query feature F_Q is to reduce the contribution of feature components common to at least two of the K registered features F_R to the query feature F_Q, the feature correction unit 114 may generate the query feature F_Q_correct by performing any calculation that can reduce the contribution of feature components common to at least two of the K registered features F_R to the query feature F_Q.
[0068] In the above description, the matching device 1 matches a query image IMG_Q that includes a query person with multiple registered images IMG_R that include multiple registered people, respectively, to extract K registered images IMG_R that include registered people similar to the query person that appears in the query image IMG_Q as K similar images IMG_S. However, the matching device 1 may also match a query image IMG_Q that includes an arbitrary query object other than a person with multiple registered images IMG_R that include multiple registered objects other than a person, respectively, to extract K registered images IMG_R that include registered objects similar to the query object that appears in the query image IMG_Q as K similar images IMG_S. In this case, the matching device 1 may perform a matching operation to search for registered images in which the query object appears as a registered object (i.e., to perform an image search). Even in this case, the likelihood that a similar image IMG_S that includes a registered object that is not the same as the query object will be ranked higher than a similar image IMG_S that includes a registered object that is the same as the query object is reduced, thereby improving the matching accuracy of the query image IMG_Q.
[0069] Alternatively, the matching device 1 may match arbitrary query data that is different from an image with multiple registered data that are different from images, thereby extracting K registered data that are similar to the query data as K similar data. In this case, the matching device 1 may perform the matching operation to search for registered data that is identical (or similar) to the query data (i.e., to perform a data search). Even in this case, it is unlikely that registered data that is not identical to the query data will be listed higher than registered data that is identical to the query data. This improves the matching accuracy of the query data.
[0070] Each of the query data and the registered data may be text data indicating text (i.e., sentence data). In this case, the matching device 1 may extract K pieces of registered data indicating registered text similar to the query text indicated by the query data as the K pieces of similar data. In this case, the matching device 1 may perform a matching operation to search for registered data indicating registered text that is identical (or similar) to the query text (i.e., to perform a text search). Even in this case, it is unlikely that registered data indicating registered text that is not identical to the query text will be listed higher than registered data indicating registered text that is identical to the query text. This improves the matching accuracy of text data.
[0071] The query data and the registered data may each be voice data indicating a voice. In this case, the matching device 1 may extract K pieces of registered data indicating a registered voice similar to the query voice indicated by the query data as the K similar data. In this case, the matching device 1 may perform a matching operation to search for registered data indicating a registered voice that is identical to (or similar to) the query voice (i.e., to perform a voice search). Even in this case, it is unlikely that registered data indicating a registered voice that is not identical to the query voice will be listed higher than registered data indicating a registered voice that is identical to the query voice. This improves the matching accuracy of voice data.
[0072] (3) Supplementary Notes The following supplementary notes are further disclosed regarding the above-described embodiment. [Supplementary Note 1] A matching device comprising: a matching means for extracting, from a plurality of registered data, a plurality of first similar data similar to the query data, based on query features that are features of the query data; and a correction means for correcting the query features based on calculated values of a plurality of registered features that are features of the plurality of first similar data, wherein the matching means extracts at least one second similar data similar to the query data from the plurality of registered data, based on the query features corrected by the correction means. [Supplementary Note 2] The matching device according to Supplementary Note 1, wherein the correction means corrects the query features by subtracting calculated values of the registered features of the plurality of similar data from the query features. [Supplementary Note 3] The matching device according to Supplementary Note 1 or 2, wherein the calculated value includes an average value. [Supplementary Note 4] The matching device according to any one of Supplementary Notes 1 to 3, wherein the query data includes query image data in which a query person appears, the query features include features of the query person, and each of the plurality of registered data includes registered image data in which a registered person appears, and the registered features include features of the registered person. [Supplementary Note 5] A matching method including: extracting a plurality of first similar data similar to the query data from a plurality of registered data based on query features that are features of the query data, correcting the query features based on statistics of a plurality of registered features that are features of the plurality of first similar data, and extracting at least one second similar data similar to the query data from the plurality of registered data based on the corrected query features.[Supplementary Note 6] A recording medium having recorded thereon a computer program for causing a computer to execute a matching method, the matching method including: extracting, from a plurality of registered data, a plurality of first similar data similar to the query data, based on query features that are features of the query data; correcting the query features based on statistical values of a plurality of registered features that are features of the plurality of first similar data; and extracting, from the plurality of registered data, at least one second similar data similar to the query data, based on the corrected query features.
[0073] At least some of the components of each of the above-described embodiments can be appropriately combined with at least some of the other components of each of the above-described embodiments. Some of the components of each of the above-described embodiments may not be used. Furthermore, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure are incorporated by reference as part of the description of this disclosure.
[0074] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Verification devices, verification methods, and recording media incorporating such modifications are also included in the technical idea of this disclosure.
[0075] REFERENCE SIGNS LIST 1 Matching device 11 Arithmetic device 111 Image acquisition unit 112 Feature extraction unit 113 Matching unit 114 Feature correction unit 1000 Matching device 1001 Matching unit 1002 Correction unit IMG_Q Query image IMG_R Registered image IMG_S Similar image F_Q Query feature F_R Registered feature
Claims
1. A matching means for extracting a plurality of first similar data items similar to the query data from a plurality of registered data items based on a query feature amount that is a feature amount of the query data; a correction means for correcting the query feature quantity based on a calculated value of a plurality of registered feature quantities which are feature quantities of the plurality of first similar data; Equipped with The matching means extracts at least one second similar data similar to the query data from among the plurality of registered data based on the query feature corrected by the correction means. Collation device.
2. The correction means corrects the query feature by subtracting a calculated value of the registered feature of the plurality of similar data from the query feature. The collation device according to claim 1 .
3. The calculated value includes an average value. The collation device according to claim 1 or 2.
4. the query data includes query image data in which a query person is photographed, the query feature includes a feature of the query person, Each of the plurality of registered data includes registered image data in which a registered person is photographed, The registered feature amount includes a feature amount of the registered person. The collation device according to any one of claims 1 to 3.
5. extracting a plurality of first similar data items similar to the query data from a plurality of registered data items based on a query feature amount that is a feature amount of the query data; correcting the query feature quantity based on a statistical value of a plurality of registered feature quantities which are feature quantities of the plurality of first similar data; extracting at least one second similar data similar to the query data from among the plurality of registered data based on the corrected query feature amount; Matching methods, including:
6. A computer program for causing a computer to execute a matching method, comprising: The collation method includes: extracting a plurality of first similar data items similar to the query data from a plurality of registered data items based on a query feature amount that is a feature amount of the query data; correcting the query feature quantity based on a statistical value of a plurality of registered feature quantities which are feature quantities of the plurality of first similar data; extracting at least one second similar data similar to the query data from among the plurality of registered data based on the corrected query feature amount; A computer program comprising:
