Verification device, verification method, and program

Abstract

In order to solve the problem of making it possible to provide a general-purpose comparison technology which improves reproducibility while maintaining comparison accuracy, this comparison device (20) is equipped with: a first reliability calculation means (21) for calculating a first reliability for each of a plurality of indicators pertaining to an object represented by target data; a second reliability calculation means (22) for calculating a plurality of second reliabilities from the plurality of first reliabilities; and a comparison means (23) for comparing the object on the basis of the plurality of second reliabilities.

Description

[Technical Field]

[0001] The present invention relates to a matching device, a matching method, and a program that can provide a general-purpose matching technique that improves recall while maintaining matching precision. [Background technology]

[0002] Image object matching techniques are well known. Image object matching is performed by calculating the reliability of a predefined index, such as "appearance" or "color." The reliability represents the similarity of the object with respect to the index. However, it is difficult to design a "single index" that can adapt to various environmental changes and changes in the target object itself.

[0003] Therefore, a combination of multiple indices is being used to respond to various environmental changes and changes in the target object itself.

[0004] For example, a technology has been proposed that uses two indices when performing fingerprint matching. In Patent Document 1, the first pattern matching is performed by matching the group delay spectrum with the registered feature amount of the fingerprint registrant. The second pattern matching is performed by matching the spatial spectrum with the registered spatial spectrum of the fingerprint registrant that has been subjected to similar processing in advance. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application No. Hei 6-60167 Summary of the Invention [Problem to be solved by the invention]

[0006] However, matching based on one of multiple indices reduces matching accuracy, while simply combining indices often results in mismatches even when the objects are the same, resulting in a lower recall rate.

[0007] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide a general-purpose matching technique that improves recall while maintaining matching accuracy. [Means for solving the problem]

[0008] A matching device according to one aspect of the present invention includes a first reliability calculation means for calculating a first reliability for each of a plurality of indicators relating to an object indicated by target data, a second reliability calculation means for calculating a plurality of second reliability from the plurality of first reliability, and a matching means for matching the object based on the plurality of second reliability.

[0009] A matching method according to one aspect of the present invention includes calculating a first reliability for each of a plurality of indicators relating to an object indicated by target data, calculating a plurality of second reliability from the plurality of first reliability, and matching the object based on the plurality of second reliability.

[0010] A program according to one aspect of the present invention causes a computer to function as a matching device including a first reliability calculation means for calculating a first reliability for each of a plurality of indicators relating to an object indicated by target data, a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities, and a matching means for matching the object based on the plurality of second reliabilities. [Effects of the Invention]

[0011] According to one aspect of the present invention, it is possible to provide a general-purpose matching technique that improves recall while maintaining matching accuracy. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a matching device according to a first exemplary embodiment of the present invention. [Figure 2]1 is a flowchart showing the flow of a matching method according to the first exemplary embodiment of the present invention. [Figure 3] FIG. 10 is a block diagram showing an example of the configuration of an information processing device according to a second exemplary embodiment of the present invention. [Figure 4] 10 is an image illustrating an example of an object tracking process. [Figure 5] 10 is an image illustrating an example of an object tracking process. [Figure 6] 10 is an image illustrating an example of an object tracking process. [Figure 7] 10 is a flowchart illustrating the flow of an object tracking process. [Figure 8] 10 is a flowchart illustrating a detailed example of an object matching process. [Figure 9] 10 is a flowchart illustrating a detailed example of a matching reliability calculation process. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of an information processing device according to a third exemplary embodiment of the present invention. [Figure 11] 10 is an image illustrating an example of biometric authentication processing. [Figure 12] 10 is a flowchart showing the flow of biometric authentication processing. [Figure 13] FIG. 1 illustrates an example of a computer that executes instructions of a program to realize each function. DETAILED DESCRIPTION OF THE INVENTION

[0013] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0014] <Outline of the collation device 20> The matching device 20 according to this exemplary embodiment is, in brief, a device that performs matching processing on one or more objects. Here, the matching device 20 can determine whether or not multiple objects correspond to each other, and can also compare each object with matching data to determine whether or not the object matches the matching data.

[0015] In this specification, when a first object and a second object are corresponding objects, or when these objects are objects that should be considered the same, these objects may be expressed as being "the same" or "matching each other."

[0016] In the matching device 20, objects that match each other are managed by, for example, the same ID. For example, if it is determined that an object detected in a certain frame and an object detected in the frame following the certain frame are the same object, the matching device 20 associates these objects with the same ID and manages them.

[0017] The collation device 20, for example, a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; a matching means for matching the objects based on the plurality of second reliabilities; It is equipped with:

[0018] <Configuration of the collation device 20> The configuration of the verification device 20 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the verification device 20.

[0019] As shown in Fig. 1, the matching device 20 includes a first reliability calculation unit 21, a second reliability calculation unit 22, and a matching unit 23. The first reliability calculation unit 21 is configured to realize first reliability calculation means in this exemplary embodiment. The second reliability calculation unit 22 is configured to realize second reliability calculation means in this exemplary embodiment. The matching unit 23 is configured to realize matching means in this exemplary embodiment.

[0020] The first reliability calculation unit 21 calculates a first reliability for each of a plurality of indices related to an object indicated by the target data. Here, the target data refers to data that is the target of matching processing by the matching device 20, and may include image data, audio data, point cloud data, other sensing data, etc., but these do not limit the present exemplary embodiment.

[0021] In addition, in this exemplary embodiment, an index related to an object may indicate an attribute related to the object, or may be a numerical evaluation of the attribute.

[0022] For example, if the target data is image data, the multiple indices are the color, shape, pattern, etc. of an object in the image represented by the image data.

[0023] In this exemplary embodiment, "reliability" refers to, for example, the degree of similarity between a certain object and a comparison target of the object. For example, when performing a process of tracking an object displayed in a video, a reliability indicating the degree of similarity between an object displayed in an image captured earlier in time and an object displayed in an image captured later in time is used. Also, for example, when performing a process of comparing pre-given matching data with an object, a reliability indicating the degree of similarity between the features indicated by the matching data and the features of the object is used.

[0024] For example, the first reliability calculation unit 21 calculates a value representing the degree of similarity between the color, shape, and pattern of an object indicated by the target data and the color, shape, and pattern of an object to which the object is compared, and uses this value as the first reliability.

[0025] The second reliability calculation unit 22 calculates a plurality of second reliabilities from a plurality of first reliabilities. Here, the second reliability calculation unit 22 calculates the plurality of second reliabilities by applying a predetermined calculation to some or all of the plurality of first reliabilities.

[0026] The second reliability may be, for example, a value obtained by multiplying some of the multiple first reliabilities calculated by the first reliability calculation unit 21. In this case, multiple second reliabilities are obtained depending on which of the multiple first reliabilities calculated by the first reliability calculation unit 21 are excluded from the product.

[0027] In the above example, three second reliabilities are calculated: the product of the object's color similarity and shape similarity, the product of the object's shape similarity and pattern similarity, and the product of the object's pattern similarity and color similarity.

[0028] For example, if the first reliability regarding the color of the object is r11, the first reliability regarding the shape of the object is r12, and the first reliability regarding the pattern of the object is r13, the following second reliability R21, R22, and R23 are calculated.

[0029] R21=r11*r12, R22=r11*r13, R23=r12*r13 The second reliability may be a value obtained by multiplying a plurality of first reliabilities calculated by the first reliability calculation unit 21 by a predetermined exponent. In this case, a plurality of second reliabilities are obtained by applying a plurality of combinations of exponents to the plurality of first reliabilities. The matching unit 23 matches objects based on a plurality of second reliabilities. For example, the matching unit 23 compares one of the above-mentioned three second reliabilities with a threshold value, and outputs a matching result of the object indicated by the target data.

[0030] <Flow of the verification method by the verification device 20> The flow of the matching method executed by the matching device 20 configured as above will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the flow of the matching method. As shown in the figure, the matching process includes steps S11, S12, and S13.

[0031] In step S11, the first reliability calculation unit 21 calculates a first reliability for each of a plurality of indices related to the object indicated by the target data.

[0032] In step S12, the second reliability calculation unit 22 calculates a plurality of second reliabilities from the plurality of first reliabilities.

[0033] In step S13, the matching unit 23 matches the objects based on the plurality of second reliabilities.

[0034] <Effects of the Verification Device 20 and Verification Method> According to the matching device 20 and matching method of this exemplary embodiment, a first reliability is calculated for each of a plurality of indicators relating to an object indicated by target data, a plurality of second reliability is calculated from the plurality of first reliability, and the object is matched based on the plurality of second reliability.

[0035] This makes it possible to provide a general-purpose matching technique that improves recall while maintaining matching accuracy. For example, it is possible to achieve appropriate matching accuracy and high recall regardless of the type of object being matched or the environment in which the object exists.

[0036] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will be omitted as appropriate.

[0037] <Configuration of information processing device 10> The configuration of the information processing device 10 according to this exemplary embodiment will be described with reference to the block diagram of FIG.

[0038] 3 is a block diagram illustrating an example of the functional configuration of the information processing device 10. As shown in FIG. 3, the information processing device 10 includes a verification device 20, a control unit 30, a storage unit 40, a communication unit 61, an input unit 62, and an output unit 63.

[0039] The matching device 20 is a functional block having the same functions as the matching device 20 described in exemplary embodiment 1. The matching device 20 includes a first reliability calculation unit 21, a second reliability calculation unit 22, and a matching unit 23, each having the functions described above with reference to FIG.

[0040] The storage unit 40 is configured, for example, by a semiconductor memory device, etc., and stores data. In this example, the storage unit 40 stores target data and index information.

[0041] Here, the target data according to this exemplary embodiment is data including an object to be matched, such as video image data, and the index information is information indicating a calculation method for calculating the first reliability.

[0042] The communication unit 61 is an interface for connecting the information processing device 10 to a network. The specific configuration of the network does not limit the present exemplary embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0043] The input unit 62 accepts various inputs to the information processing device 10. The specific configuration of the input unit 42 is not limited to this exemplary embodiment, but as an example, the input unit 42 may be configured to include input devices such as a keyboard and a touchpad. The input unit 42 may also be configured to include a data scanner that reads data via electromagnetic waves such as infrared rays or radio waves, and a sensor that senses the environmental state.

[0044] The output unit 63 is a functional block that outputs processing results by the information processing device 10. The specific configuration of the output unit 43 is not limited to this exemplary embodiment, but as an example, the output unit 43 is configured by a display, a speaker, a printer, etc., and displays various processing results by the information processing device 10 on a screen or outputs them as sounds or figures.

[0045] The control unit 30 controls the matching device 20, the storage unit 40, the communication unit 61, the input unit 62, and the output unit 63 to perform various processes. The control unit 30 also performs various image processes to, for example, detect objects to be matched by the matching device 20.

[0046] As described above, the information processing device 10 has the matching device 20, and therefore can perform the matching process described above with reference to Fig. 2 and use the processing result to perform further processing. Here, as an example, a case will be described in which the information processing device 10 performs object tracking processing using the processing result of the matching device 20.

[0047] Here, the object tracking process refers to a process of assigning the same ID to corresponding (identical) objects among objects present across multiple frames included in a video. In the object tracking process, as an example, a moving object is tracked by comparing an object displayed in a frame of an image captured earlier in time with an object displayed in a frame of an image captured later in time. Here, the object matching is performed by a matching device 20.

[0048] (Example of an image that can be used for object tracking processing) 4 to 6 are images illustrating the object tracking process executed by the information processing device 10. The flow of the object tracking process will be described later with reference to FIG.

[0049] The images in Figures 4 to 6 each show a ball bouncing on a road surface.

[0050] 4 to 6 are images corresponding to one of the frames constituting a moving image. For example, if a moving image is composed of N frames from the first frame to the Nth frame, image 102 shown in Fig. 5 and Fig. 6 is the image of the tth frame. Image 101 shown in Fig. 4 is an image captured temporally earlier than image 102 and is the image of the t-1th frame.

[0051] Image 101 in Fig. 4 shows 16 balls, each surrounded by a rectangular frame. Note that each ball in the image may be detected by image processing such as an existing object extraction process, or may be detected by a model trained by machine learning. Object detection may be performed, for example, from within an area indicated by a user's operation performed via input unit 42, or may be performed from within a pre-specified area excluding a background image.

[0052] As an example, object detection can be performed using the graph cut method. In the graph cut method, the boundary of the area that constitutes the foreground object image to be cut out is calculated from the color distribution and pixel color gradient of two types of images, namely, the foreground object image containing the object to be cut out and the background image. Then, the image is cut out along the calculated boundary, and the foreground object image to be cut out is extracted.

[0053] The 16 balls in FIG. 4 are each assigned an identification number (simply referred to as ID) of ID1 to ID16.

[0054] 5 also shows 16 balls, but the positions of the balls are slightly different in image 102 because image 102 was taken later than image 101. First, control unit 30 of information processing device 10 performs object extraction processing on image 102 to detect the 16 balls as objects.

[0055] In the example of Fig. 5, the control unit 30 has assigned rectangular frames to each of the 16 balls that are detected objects, but has not yet assigned an ID. The information processing device 10 determines whether each of the balls that are objects detected in image 102 of Fig. 5 is the same as the balls with IDs 1 to 16 detected in image 101. At this time, the matching device 20 executes a matching process, and the objects in image 102 are matched with the objects in image 101.

[0056] If the result of the comparison indicates that the ball in image 102 is identical to the ball in image 101, control unit 30 of information processing device 10 associates the same ID with the ball in image 102 that is identical to the ball in image 101. For example, ID1 is assigned to the ball in image 102 that is determined to be the same as the ball with ID1 in image 101, and ID2 is assigned to the ball in image 102 that is determined to be the same as the ball with ID2 in image 101. In the same manner, control unit 30 assigns IDs to each of the balls in image 102.

[0057] 6 shows 16 balls, each surrounded by a rectangular frame, and each ball is assigned an ID from ID1 to ID16. That is, the information processing device 10 identifies, in the image 102, each of the 16 balls shown in the image 101. In this way, the information processing device performs object tracking processing.

[0058] 4 to 6, an example has been described in which all 16 balls in image 101 appear in image 102. However, there may be cases in which, for example, a bouncing ball goes off screen and does not appear in image 102. For example, if the balls that go off screen are ball ID1 and ball ID9, information processing device 10 will not assign ID1 and ID9 to the balls in image 102, but will instead assign ID2 to ID8 and ID10 to ID16.

[0059] In addition, for example, a new ball may fly in. In this case, the information processing device 10 may assign a new ID (for example, ID 17) to the new ball in the image 102.

[0060] <Flow of Object Tracking Process by Information Processing Device 10> Next, an example of object tracking processing by the information processing device 10 will be described with reference to the flowchart of Fig. 7. Fig. 7 is a flowchart illustrating the flow of the object tracking processing.

[0061] In step S31, the control unit 30 of the information processing device 10 acquires image data of a moving image. The image data is stored, for example, as target data in the storage unit 40. Note that the image data acquired here is image data of a moving image and is composed of a plurality of frame images.

[0062] In step S32, the control unit 30 sets the value of the variable t to an initial value.

[0063] In step S33, the control unit 30 detects an object in the t-th frame. Specific details of the object detection process have been described above, and therefore will not be described here. In this step, the control unit 30 may, for example, attach a rectangular frame (bounding box) to the detected object, as shown in FIG. 5.

[0064] In step S34, the control unit 30 sets a matching target object in the t-1th frame. Here, the matching target object is an object to be matched from among the objects displayed in the image of the t-1th frame. For example, each of the balls with ID1 to ID16 described above with reference to FIG. 4 becomes a matching target object.

[0065] If there are multiple objects to be matched in the image of the t-1th frame, the process described below is repeatedly executed for each of the objects to be matched.

[0066] In step S35, the control unit 30 controls the matching device 20 to execute an object matching process, which will be described later. Details of the object matching process will be described later with reference to the flowchart of FIG.

[0067] In step S36, the control unit 30 determines whether or not there is a next frame in the image data acquired in step S31. If it is determined in step S36 that there is a next frame, the control unit 30 executes the process of step S37.

[0068] In step S37, the control unit 30 sets the value of the variable t to t + 1. Thereafter, the control unit 30 repeatedly executes the processes of steps S33 to S35.

[0069] If it is determined in step S36 that there is no next frame, the process of step S37 is skipped, and the control unit 30 executes the process of step S38.

[0070] In step S38, the control unit 30 outputs the matching result. In this manner, the object tracking process is executed. In this manner, in the object tracking process, the target data is image data of a video, and the matching device 20 matches an object included in an image of a first frame of the image data with an object included in an image of a second frame of the image data.

[0071] (Object matching process flow) Next, details of the object matching process in step S35 in FIG. 7 will be described with reference to the flowchart in FIG.

[0072] In step S51, the control unit 30 sets candidate objects from among the objects in the t-th frame. The candidate objects are objects that are likely to be the same as the object to be matched detected in the processing of step S34 in Fig. 7. As an example, the coordinates of the position where the object to be matched is detected in the image of the t-1-th frame are obtained, and objects located within a certain distance from the coordinates in the t-th frame are set as candidate objects.

[0073] In step S52, the verification device 20 executes a reliability calculation process, which will be described later. A detailed example of the reliability calculation process in step S52 in FIG. 8 will now be described with reference to the flowchart in FIG.

[0074] In step S71, the first reliability calculation unit 21 analyzes the matching target object and the candidate object for each index. Here, the index may be the color, shape, pattern, or speed of the object. Alternatively, the index may be the position, size, or acceleration of the object.

[0075] When the indicators are the color, shape, pattern, and speed of the object, the first reliability calculation unit 21 calculates, for example, the average value of pixel values ​​of the object to be matched and the candidate object for the indicator "color." Furthermore, the first reliability calculation unit 21 calculates, for example, the shape of the indicator "shape," a shape that approximates the contours of the object to be matched and the candidate object. Furthermore, the first reliability calculation unit 21 detects edges in the object to be matched and the candidate object for the indicator "pattern."

[0076] Furthermore, for example, with regard to the indicator "speed," the distance and direction of movement of the target object and the candidate object between one frame are identified. The distance and direction of movement of the target object are identified in advance by referring to the frame of the target object and the frame immediately preceding that frame, and the speed of the target object is also calculated. Assuming that the target object is the same as the candidate object, the distance and direction of movement of the candidate object are found.

[0077] Note that information for specifying what to use as an index and how to calculate the first reliability (referred to as information related to the index) may be set in advance or may be input via the communication unit 61 or the input unit 62. Alternatively, information related to the index may be provided as metadata or the like included in the image data as the target data.

[0078] In Figure 3, as an example, index information describing information about the index is stored in the memory unit 40, and the first reliability calculation unit 21 calculates the first reliability by referring to the index information stored in the memory unit 40.

[0079] The first reliability calculation unit 21 acquires information about the index, and calculates the first reliability based on the acquired information about the index.

[0080] In step S72, the first reliability calculation unit 21 calculates a first reliability for each index. For example, if the indexes are the color, shape, pattern, and speed of an object, the similarity of the color, shape, pattern, and speed of the object to be matched and the candidate object is calculated as the first reliability. Note that since there are multiple indexes, multiple (e.g., n) first reliabilities are calculated in step S72.

[0081] In step S73, the second reliability calculation unit 22 removes any m reliabilities from the n first reliabilities calculated in step S72, where m is a natural number less than n.

[0082] In step S74, the second reliability calculation unit 22 calculates n C m For example, if the number of indices is 4 and one reliability is removed in step S73, n is 4 and m is 3, so 4C3 = 4 sets are obtained.

[0083] More specifically, it is assumed that first reliability a, first reliability b, first reliability c, and first reliability d are calculated for each of index A, index B, index C, and index D. In this case, the four pairs obtained in step S74 are (a, b, c), (a, b, d), (a, c, d), and (b, c, d).

[0084] In step S75, the second reliability calculation unit 22 calculates the second reliability by calculating the product of the first reliability included in each pair obtained in step S74. In the above example, four second reliability values ​​are calculated: M1=a*b*c, M2=a*b*d, M3=a*c*d, and M4=b*c*d. Here, "*" indicates an operation representing a product.

[0085] In this way, the second reliability calculation unit 22 calculates the second reliability for each of nCm sets obtained by removing any m (m is a natural number less than n) first reliabilities from n (n is a natural number greater than or equal to 2) first reliabilities, by taking the product of all the first reliabilities included in the set.

[0086] In step S76, the matching unit 23 sets the maximum value max(M1, M2, M3, M4) of the second reliabilities calculated in step S75 as the matching reliability. That is, the matching unit 23 matches the objects based on the second reliabilities having the maximum value among the multiple second reliabilities.

[0087] In this way, the reliability calculation process is executed.

[0088] 8, after the process of step S52, in step S53, the matching unit 23 determines whether the matching reliability is equal to or greater than a threshold. If the matching reliability is equal to or greater than the threshold, the candidate object is considered to be the same as the object to be matched. In this way, the matching unit 23 matches the objects by comparing the second reliability having the largest value with the threshold.

[0089] If it is determined in step S53 that the reliability for matching is equal to or greater than the threshold, in step S54 the control unit 30 assigns the same ID to the candidate object and the object to be matched, which results in an ID being assigned to the candidate object surrounded by a rectangular frame, as described above with reference to FIG.

[0090] On the other hand, if it is determined in step S53 that the matching reliability is not equal to or greater than the threshold, the control unit 30 determines in step S55 whether or not there is a next candidate object. If it is determined in step S55 that there is a next candidate object, the next candidate object is set in step S51, and the processes of steps S52 and S53 are executed.

[0091] If it is determined in step S55 that there is no next candidate object, the object matching process ends.

[0092] If multiple objects to be matched are detected in the t-1th frame, the processes of steps S51 to S55 are executed multiple times. That is, the processes of steps S51 to S55 are executed repeatedly until matching is completed for all objects to be matched in the t-1th frame. For example, if 16 balls are detected as objects to be matched, the object matching process is executed for each of the 16 balls.

[0093] In this way, the object matching process is carried out.

[0094] 7 to 9, the information processing device 10 executes the object tracking process, thereby tracking a ball in an image, for example, as described above with reference to Figures 4 to 6. That is, if it is determined that the ball in image 102 of the t-th frame is the same as the ball in image 101 of the t-1-th frame, the same ID is assigned to the ball in image 102 that is the same as the ball in image 101, thereby tracking the object.

[0095] (Another example of the second reliability) The second reliability calculation unit 22 may calculate the second reliability using the logarithm of the first reliability.

[0096] Specifically, when the first reliability a, the first reliability b, the first reliability c, and the first reliability d are calculated for each of the indexes A, B, C, and D, the second reliabilities M1 to M4 are calculated as follows: M1=log a + log b + log c M2=log a + log b + log d M3=log a + log c + log d M3=log b + log c + log d may also be used.

[0097] (Another example of the second reliability) In the above example, it has been explained that in step S73, any m reliabilities are removed from the n first reliabilities calculated in step S72. However, instead of removing any m reliabilities, the second reliability calculation unit 22 may perform an operation on any m (m is a natural number less than n) first reliabilities with an exponent greater than 0 and less than 1.

[0098] For example, by performing a power operation with an index of 0, the value of the first reliability becomes 1, so when calculating the product of the first reliability, an operation equivalent to removing the first reliability with an index of 0 can be performed.

[0099] That is, the first reliability calculation unit 21 performs an exponent calculation on any m (m is a natural number less than n) first reliabilities out of n (n is a natural number equal to or greater than 2) first reliabilities, using a value greater than 0 and less than 1 as the exponent. n C m For each of the pairs, the matching unit 23 calculates a second reliability for the pair by taking the product of all the first reliability values ​​included in the pair, and n C m The objects may be matched based on the second reliability having the largest value among the second reliability values ​​for each of the pairs.

[0100] Specifically, when the first reliability a, the first reliability b, the first reliability c, and the first reliability d are calculated for each of the indicators A, B, C, and D, (the index to be applied to a, the index to be applied to b, the index to be applied to c, the index to be applied to d) may be set to (1, 1, 1, α), (1, 1, α, 1), (1, α, 1, 1), and (α, 1, 1, 1).

[0101] Here, α is a real number greater than 0 and less than 1. Then, the second reliability may be calculated by performing exponentiation using these exponents on the corresponding first reliability, and then taking the product of the first reliability after the exponentiation.

[0102] More specifically, M1=a * b * c * d α M2=a * b * c α * d M3=a * b α *c*d M4=a α * b * c * d A plurality of second reliabilities M1 to M4 may be calculated by the following formula.

[0103] (Another example of the second reliability) Furthermore, when performing the power calculation, the second reliability calculation section 22 may calculate the second reliability using the logarithm of the first reliability.

[0104] That is, as explained in other example 2, (the exponent to be applied to a, the exponent to be applied to b, the exponent to be applied to c, the exponent to be applied to d) are set to (1, 1, 1, α), (1, 1, α, 1), (1, α, 1, 1), (α, 1, 1, 1), and the exponents are powered by these exponents to the corresponding first reliabilities, and then the second reliabilities M1 to M4 are calculated as follows: M1=log a + log b + log c + α*log d M2=log a + log b + α*log c + log d M3=log a + α*log b + log c + log d M3=α*log a + log b + log c + log d may also be used.

[0105] Furthermore, the coefficients of each logarithm are not limited to the above examples, and more general coefficients can also be used.

[0106] <Advantages of Exemplary Embodiment 2> In this way, the information processing device 10 matches the candidate object with the matching target object without deliberately taking into consideration any m of the n first reliabilities.

[0107] For example, if matching is performed based on one of multiple indicators, there is a high possibility that multiple candidate objects will be determined to be identical to the object being matched. In other words, there is a high possibility that objects that are not actually identical will be mistakenly determined to be identical, which reduces matching accuracy.

[0108] On the other hand, if matching is performed by simply multiplying the indices, there is a high possibility that the objects will be judged as mismatched even though they are the same object, resulting in a decrease in recall.

[0109] Furthermore, depending on the object to be matched, the indices used for matching may vary, and the number of indices not taken into consideration may also vary.

[0110] According to the information processing device 10 of this exemplary embodiment, by matching a candidate object with an object to be matched without deliberately considering any m of the n first reliabilities, it is possible to perform general-purpose matching that improves the recall rate while maintaining the matching accuracy.

[0111] Exemplary Embodiment 3 Next, a third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and their description will be omitted as appropriate.

[0112] Fig. 10 is a block diagram showing an example of the configuration of the information processing device 10 according to this exemplary embodiment. In the example of Fig. 10, matching data is stored in a storage unit 40. The rest of the configuration is the same as the example described with reference to Fig. 3.

[0113] The matching data is, for example, information indicating the characteristics of a person to be matched, and the information processing device 10 in FIG. 10 performs biometric authentication processing by using, for example, the processing result of the matching device 20 using the matching data.

[0114] 11 is a diagram showing an example of an image captured in biometric authentication processing by the information processing device 10. Here, an example will be described in which an image of a person's face is captured and the features of the nose, mouth, and ears are compared with the features of the matching data.

[0115] 11, an image of the face of person 200 is captured. In this image, the area surrounded by a dotted ellipse indicates the person's nose, mouth, and ears. The control unit 30 of the information processing device 10 executes a facial image recognition process to identify the captured face of the person, extract feature points of the nose, mouth, and ears within the face, and identify the areas of the nose, mouth, and ears within the face.

[0116] 11 is the nose area of ​​the person 200, area P12 is the mouth area of ​​the person 200, area P13 is the left ear area of ​​the person 200, and area P14 is the right ear area of ​​the person 200. In FIG.

[0117] The matching data describes the features of each of the nose, mouth, and ears. The features are described with respect to a plurality of indices. Here, the plurality of indices are, for example, color, shape, and pattern. The "color" index describes the average pixel value of each region, etc. Furthermore, the "shape" index describes shapes that approximate the contours of each of the nose, mouth, and ears, etc. Furthermore, the "pattern" index describes the size and position of black and brown parts, etc. within each region.

[0118] The matching data may be images of the nose, mouth, and ears.

[0119] The matching device 20 analyzes each of the images in regions P11 to P14 for each index. For example, the first reliability calculation unit 21 calculates features related to multiple indices (e.g., color, shape, pattern) in the image of region P11 (nose). Then, the first reliability calculation unit 21 calculates a plurality of first reliabilities by calculating a similarity with the nose features described in the matching data. The first reliability calculation unit 21 calculates a plurality of first reliabilities by, for example, referring to index information stored in the storage unit 40.

[0120] Furthermore, the second reliability calculation unit 22 applies a predetermined calculation to some or all of the plurality of first reliabilities to calculate a plurality of second reliability values ​​related to the nose of the person 200. A specific example of the method for calculating the second reliability values ​​is as described in the second exemplary embodiment, and therefore a detailed description thereof will be omitted.

[0121] In addition, as a method for calculating the second reliability, it is also possible to apply "Another example 1 of the second reliability," "Another example 2 of the second reliability," or "Another example 2 of the second reliability" described in exemplary embodiment 2.

[0122] Then, the matching unit 23 determines the matching reliability and compares it with a threshold value to determine whether the nose features of the matching data match the nose features of the person 200.

[0123] The matching device 20 also matches the features of areas P12 to P14 (mouth, left ear, right ear) with the matching data, respectively. If the information processing device 10 determines that the features of the nose, mouth, and ears in the matching data all match the features of the nose, mouth, and ears of the person 200, it determines that the person 200 is the same as the person to be matched indicated by the matching data.

[0124] Next, an example of biometric authentication processing by the information processing device 10 in FIG. 10 will be described with reference to a flowchart in FIG.

[0125] In step S71, the control unit 30 of the information processing device 10 acquires image data. At this time, for example, image data of an image including the face of the person 200 in Fig. 11 is acquired. The image data may be stored in advance in the storage unit 40, or may be acquired by photographing the person 200 with a camera.

[0126] In step S72, the control unit 30 extracts feature points in the nose, mouth, and ear regions from the image data acquired in step S71, thereby extracting, for example, regions P11 to P14 in FIG.

[0127] In step S73, the control unit 30 reads and acquires the matching data from the storage unit 40.

[0128] In step S74, the matching device 20 executes a reliability calculation process. This process is similar to the process described above with reference to Fig. 9, and therefore detailed description will be omitted. However, matching is performed without taking into consideration any m of the n first reliabilities.

[0129] The process of step S74 is repeatedly executed corresponding to the number of areas extracted in step S72. For example, if areas P11 to P14 are extracted in step S72, the process of step S74 is executed four times.

[0130] In step S75, the matching unit 23 of the matching device 20 determines whether or not all of the matching reliability degrees calculated for each region are equal to or greater than a threshold value.

[0131] If it is determined in step S75 that the matching reliability of each region is equal to or greater than the threshold, then in step S76 the control unit 30 determines that the person in the image data matches the person to be matched.

[0132] In step S77, the control unit 30 outputs the determination result via the output unit 63.

[0133] In this way, the biometric authentication process is performed.

[0134] <Advantages of Exemplary Embodiment 3> In this way, the information processing device 10 matches the matching data with a person without deliberately taking into consideration any m of the n first reliabilities.

[0135] For example, if matching is performed based on one of multiple indicators, there is a high possibility that multiple people will be determined to be the same person. On the other hand, if matching is performed by simply multiplying indicators, there is a high possibility that two people will be determined to be a mismatch even though they are the same person.

[0136] According to the information processing device 10 of this exemplary embodiment, it is possible to perform biometric authentication that improves the recall rate while maintaining the matching accuracy.

[0137] [Software implementation example] Some or all of the functions of the information processing device 10 and the collation device 20 may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0138] In the latter case, the information processing device 10 and the collation device 20 are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG.

[0139] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as an information processing device 10 and a matching device 20. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of the information processing device 10 and the matching device 20.

[0140] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0141] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0142] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0143] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0144] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0145] (Appendix 1) a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; a matching means for matching the objects based on the plurality of second reliabilities; A matching device comprising:

[0146] (Appendix 2) The matching means matches the objects based on the second reliability having the largest value among the plurality of second reliability. 10. The verification device of claim 1.

[0147] (Appendix 3) The second reliability calculation means The first reliabilities are obtained by removing any m (m is a natural number less than n) first reliabilities from the n (n is a natural number equal to or greater than 2) first reliabilities. n C m for each of the sets, calculating a second confidence for the set by taking the product of all first confidences included in the set; The collation means The aforementioned n C m and matching the object based on the second reliability having the largest value among the second reliability for each of the pairs. 3. The verification device according to claim 1 or 2.

[0148] (Appendix 4) The second reliability calculation means Among the n (n is a natural number equal to or greater than 2) first reliabilities, an exponent greater than 0 and less than 1 is applied to any m (m is a natural number less than n) first reliabilities. n C m for each of the sets, calculating a second confidence for the set by taking the product of all first confidences included in the set; The collation means The aforementioned n C m and matching the object based on the second reliability having the largest value among the second reliability for each of the pairs. 3. The verification device according to claim 1 or 2.

[0149] (Appendix 5) The collation means The object is matched by comparing the second confidence having the largest value with a threshold. 5. The verification device of claim 3 or 4.

[0150] (Appendix 6) The first reliability calculation means Obtaining information about the indicator; Calculating the first reliability based on the acquired information about the index. 6. A verification device according to any one of appendices 1 to 5.

[0151] (Appendix 7) the target data is video data, The matching means matches an object included in an image of a first frame of the video data with an object included in an image of a second frame of the video data. 7. A verification device according to any one of appendices 1 to 6.

[0152] (Appendix 8) The matching means matches an object indicated by the image data as the target data with matching data. 7. A verification device according to any one of appendices 1 to 6.

[0153] (Appendix 9) The apparatus further includes a biometric authentication unit that executes biometric authentication processing by referring to the result of the matching by the matching unit. 10. The verification device according to claim 8.

[0154] (Appendix 10) Calculating a first reliability for each of a plurality of indicators related to the object indicated by the target data; calculating a plurality of second reliabilities from the plurality of first reliabilities; and matching the objects based on the plurality of second confidence levels. Matching methods, including:

[0155] (Appendix 11) Computer, a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; and a matching means for matching the objects based on the plurality of second reliabilities. program.

[0156] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0157] at least one processor, the processor comprising: A process of calculating a first reliability for each of a plurality of indicators related to the object indicated by the target data; calculating a plurality of second reliabilities from the plurality of first reliabilities; and performing a process of matching the objects based on the plurality of second reliabilities.

[0158] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process and the output sequence generation process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0159] 10. Information processing equipment 20. Verification device 21 First reliability calculation unit 22 Second reliability calculation unit 23 Matching Unit 30 Control Unit 40 Storage section 61 Communications Department 62 Input section 63 Output section

Claims

1. a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; a matching means for matching the objects based on the plurality of second reliabilities; Equipped with The second reliability calculation means for each of n C m sets obtained by removing any m (m is a natural number less than n) first reliabilities from the n (n is a natural number equal to or greater than 2) first reliabilities, calculating a second reliability for the set by taking the product of all the first reliabilities included in the set; The collation means The object is matched based on the second reliability having the largest value among the second reliability for each of the n C m pairs. Collation device.

2. A first reliability calculation means for calculating a first reliability for each of a plurality of indicators related to an object indicated by target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; a matching means for matching the objects based on the plurality of second reliabilities; Preparation, The second reliability calculation means For n C m sets obtained by performing an operation using an exponent greater than 0 and less than 1 on any m (m is a natural number less than n) first reliabilities among the n (n is a natural number equal to or greater than 2) first reliabilities, calculate a second reliability for each set by taking the product of all the first reliabilities included in the set; The collation means The object is matched based on the second reliability having the largest value among the second reliability for each of the n C m pairs. Collation device.

3. The matching means matches the objects based on the second reliability having the largest value among the plurality of second reliability. The collation device according to claim 1 or 2.

4. The collation means The object is matched by comparing the second confidence having the largest value with a threshold. The verification device according to any one of claims 1 to 3.

5. The first reliability calculation means Obtaining information about the indicator; Calculating the first reliability based on the acquired information about the index The verification device according to any one of claims 1 to 4.

6. the target data is video data, The matching means matches an object included in an image of a first frame of the video data with an object included in an image of a second frame of the video data. The verification device according to any one of claims 1 to 5.

7. The matching means matches an object indicated by the image data as the target data with matching data. The verification device according to any one of claims 1 to 5.

8. The apparatus further includes a biometric authentication unit that executes biometric authentication processing by referring to the result of the matching by the matching unit. The verification device according to claim 7.

9. Calculating a first reliability for each of a plurality of indicators related to the object indicated by the target data; calculating a plurality of second reliabilities from the plurality of first reliabilities; and matching the objects based on the plurality of second confidence levels. Including, Calculating the second reliability includes: for each of n C m sets obtained by removing any m (m is a natural number less than n) first reliabilities from the n (n is a natural number equal to or greater than 2) first reliabilities, calculating a second reliability for the set by taking the product of all the first reliabilities included in the set; The performing of the matching includes: and matching the object based on the second reliability having the largest value among the second reliability for each of the n C m sets. Matching method.

10. Calculating a first reliability for each of a plurality of indicators related to an object indicated by the target data; calculating a plurality of second reliabilities from the plurality of first reliabilities; and matching the objects based on the plurality of second confidence levels. Including, Calculating the second reliability includes: n C m sets are obtained by performing an operation using an exponent greater than 0 and less than 1 on any m (m is a natural number less than n) first reliabilities among n (n is a natural number equal to or greater than 2) first reliabilities, and for each of the n C m sets, a second reliability for the set is calculated by taking the product of all the first reliabilities included in the set; The performing of the matching includes: and matching the object based on the second reliability having the largest value among the second reliability for each of the n C m sets. Matching method.

11. Computer, a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; and a matching unit that matches the object based on the plurality of second reliabilities; The second reliability calculation means for each of n C m sets obtained by removing any m (m is a natural number less than n) first reliabilities from the n (n is a natural number equal to or greater than 2) first reliabilities, calculating a second reliability for the set by taking the product of all the first reliabilities included in the set; The collation means The object is matched based on the second reliability having the largest value among the second reliability for each of the n C m pairs. program.

12. A computer, a first reliability calculation means for calculating a first reliability for each of a plurality of indices related to the object indicated by the target data; a second reliability calculation means for calculating a plurality of second reliabilities from the plurality of first reliabilities; and a matching unit that matches the object based on the plurality of second reliabilities; The second reliability calculation means For n C m sets obtained by performing an operation using an exponent greater than 0 and less than 1 on any m (m is a natural number less than n) first reliabilities among the n (n is a natural number equal to or greater than 2) first reliabilities, calculate a second reliability for each set by taking the product of all the first reliabilities included in the set; The collation means The object is matched based on the second reliability having the largest value among the second reliability for each of the n C m pairs. program.

Images