Individual identification system and individual identification method

Abstract

To provide an individual identification system that can improve accuracy of identifying individuals.SOLUTION: An individual identification system 10 is configured to: estimate a geometric relation between feature points present in feature image data of an individual to be identified and feature points present in feature image data of each of a plurality of registered individuals; calculate a matching rate using a formula (1) for each of a plurality of partial areas 213 set in the feature image data of the individual to be identified; and determine whether or not the individual to be identified is an identical individual as any of the plurality of registered individuals based on whether or not the matching rate is equal to or greater than a threshold. Matching rate=(the number of feature points that have a geometric relation with feature points in the feature image data of the registered individual, among feature points present in the partial areas 213 of the feature image data of the individual to be identified) / (the number of feature points present in the partial areas 213 of the feature image data of the individual to be identified) Formula (1).SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an individual identification system and an individual identification method. [Background technology]

[0002] In recent years, ensuring the traceability of each individual item has become increasingly important in the production management and quality management of products, parts, and the like (hereinafter referred to as "items"). Methods for ensuring traceability include using a Kanban board and printing symbols (e.g., one-dimensional or two-dimensional codes) or characters (e.g., serial numbers) on each individual item to ensure traceability. However, the method using a Kanban board does not allow for tracking of each individual item. On the other hand, a configuration in which symbols or characters are printed on each individual item may lead to a deterioration of product performance. For this reason, a system is required that allows tracking of individual items without contact.

[0003] Patent Document 1 discloses a technology for identifying an item to be identified, which sets a first parameter and a second parameter by geometric verification, and identifies whether the item is the same or not using the two parameters. However, with the technology described in Patent Document 1, if feature points are not appropriately selected, the accuracy of identification may decrease. Also, due to circumstances specific to the item (e.g., the usage environment, etc.), there may be some parts where no reliable feature points remain. Furthermore, if subsequent deterioration or damage occurs, no feature points remain, which may result in a decrease in the accuracy of identification. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-528293 A Summary of the Invention

[0005] (Problem to be solved by the invention) In view of the above circumstances, an object of the present invention is to provide an individual identification system and an individual identification method that can improve the accuracy of identifying individual articles.

[0006] (Means for solving the problem) In order to achieve the above object, the individual identification system according to the present invention comprises: a first imaging device that captures an image of a specific area that is a part of a plurality of individual articles of the same kind and generates first image data that is data of an image in which the specific area is captured for each individual article; a first processing device that extracts, based on the first image data, a first feature point that is a feature point present in the specific region for each individual; a storage device that registers, as a registered individual, each of the individuals from which the first feature points have been extracted by the first processing device, together with information on the first feature points; a second imaging device that captures an image of the specific area of ​​an identification target individual that is one of the articles of the same kind, and generates second image data that is data of an image showing the specific area; a second processing device that extracts second feature points, which are feature points present in the specific region of the individual to be identified based on the second image data, and determines whether the individual to be identified is identical to any of the multiple registered individuals based on information of the extracted second feature points and information of the first feature points of the multiple registered individuals registered in the storage device; Equipped with The second processing device estimates a geometric relationship between the second feature points and the first feature points, sets a plurality of partial areas which are parts of the specific area of ​​the second image data, calculates a match rate between each registered individual and the individual to be identified for each of the set partial areas using the following formula (1), and if there is a registered individual for which the match rate is equal to or greater than a threshold, determines that the individual to be identified is the same individual as the registered individual for which the match rate is the highest among the registered individuals for which the match rate is equal to or greater than the threshold. The matching rate=(the number of the feature points among the second feature points that have the estimated geometric relationship with the first feature points) / (the number of the second feature points included in one of the partial regions set in the specific region of the second image data) Formula (1)

[0007] According to the present invention, a partial area smaller than a specific area is set in the specific area, a matching rate is calculated for each partial area, and whether or not the individual item to be identified is the same as the individual item registered is determined based on the matching rate for each partial area. Therefore, even if the specific area is locally scratched and the feature points have changed, if there is a partial area in which the feature points at the time of manufacture (registration) remain, the matching rate of the partial area can be used to improve the accuracy of identification. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an individual identification system. [Diagram 2] FIG. 2 is a diagram showing a schematic configuration of the characteristic image data. [Figure 3A] FIG. 3A is a schematic diagram illustrating feature point matching. [Figure 3B] FIG. 3B is a schematic diagram illustrating feature point matching. [Figure 4] FIG. 4 is a diagram illustrating the threshold value. [Figure 5A] FIG. 5A is a flowchart showing the process of the preparation stage of the individual identification method. [Figure 5B] FIG. 5B is a flowchart showing the preparatory process of the individual identification method. [Figure 6A] FIG. 6A is a flowchart showing the process of the identification stage of the individual identification method. [Figure 6B] FIG. 6B is a flowchart showing the process of the identification stage of the individual identification method. [Figure 7] FIG. 7 is a diagram showing an example of the present invention and a comparative example. [Figure 8A] FIG. 8A is a diagram showing an example of the present invention and a comparative example. [Figure 8B] FIG. 8B is a diagram showing an example of the present invention and a comparative example. [Figure 8C] FIG. 8C is a diagram showing an example of the present invention and a comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The individual identification system 10 according to this embodiment is configured to be able to register characteristic information of each individual of a plurality of articles of the same type, and is also configured to be able to identify which individual of the plurality of articles of the same type an individual belongs to among "a plurality of articles whose characteristic information is registered." Note that the individual to be identified may be referred to as the "individual to be identified," and the individual whose characteristic information is registered may be referred to as the "registered individual."

[0010] <Configuration of individual identification system> FIG. 1 is a diagram showing a schematic configuration of an individual identification system 10 according to this embodiment. The individual identification system 10 includes at least one common storage device 11 and multiple terminal devices 12. Each terminal device 12 includes an imaging device 121, a processing device 122, a terminal storage device 123, an output device 124, and an I / F (interface) 125. FIG. 1 shows an example in which the individual identification system 10 includes one common storage device 11 and three terminal devices 12, but the number of common storage devices 11 and terminal devices 12 is not limited. The common storage device 11 and the terminal devices 12 are connected to each other via a network 13 so as to be able to transmit and receive data. Each terminal device 12 can store data in the common storage device 11 via the network 13 and can read data stored in the common storage device 11 via the network 13. The specific standard of the network 13 is not particularly limited. The network 13 may be wired or wireless.

[0011] The common storage device 11 may be a device capable of storing data in a computer-readable format, such as a network storage device or a cloud storage device.

[0012] The imaging device 121 of each terminal device 12 is configured to capture an image of a part of each individual item of the same kind of article, and generate image data in which the part of each individual item is captured. A known digital camera is used as the imaging device 121.

[0013] The processing device 122 is configured to be able to control the imaging device 121, the terminal storage device 123, and the output device 124. The processing device 122 is also configured to be able to store data in the common storage device 11 and read data from the common storage device 11. The processing device 122 is also configured to be able to acquire image data generated by the imaging device 121 and to perform a predetermined process, which will be described later, on the acquired image data. The processing device 122 includes a computer including a CPU, a ROM, a RAM, and an I / F (interface). A computer program for executing the individual identification method is stored in the ROM of the computer beforehand. The CPU of the computer then reads out the computer program from the ROM, expands it in the RAM, and executes it. This executes the individual identification method according to the present embodiment.

[0014] The output device 124 is configured to be able to output the result of the classification (described later) by the processing device 122. A display device capable of displaying images, such as a liquid crystal display device, can be used as the output device 124. The output device 124 may also include a printing device capable of printing the result of the classification.

[0015] <Feature information> The feature information is information on feature points (hereinafter, sometimes simply referred to as "feature points") extracted by the A-KAZE (Accelerated KAZE) method. In this embodiment, the processing device 122 of the terminal device 12 uses image data showing a predefined part of each individual object of the same kind of article to extract feature points 212 (see FIG. 2) present in the predefined part of each individual object. The "predefined part" of each individual object may be referred to as a "specific region 211." The specific region 211 (see FIG. 2) is selected from an area whose surface properties are unlikely to change over time, for example, an area that is unlikely to be scratched or have a contact mark with another object during the manufacture of each article or during use after manufacture.

[0016] In the following description, a sun gear of a planetary gear mechanism is used as an example of the same type of article, and the end face of the sun gear shaft is used as the specific region 211 of each individual. Note that the sun gear shaft is a hollow shaft. Therefore, the specific region 211 shown in the following description is a surface having a circular ring shape.

[0017] Here, the feature image data 21 used for extracting the feature points 212 will be described. FIG. 2 is a diagram showing an example of the feature image data 21. The imaging device 121 connected to the terminal device 12 captures an image of a specific area 211 of one individual from which the feature points 212 are to be extracted, generates image data, and transmits the generated image data to the processing device 122. Hereinafter, the "image data showing the specific area 211 of one individual" generated by the imaging device 121 may be referred to as "original image data". The processing device 122 generates the feature image data 21 from the original image data. The feature image data 21 is data of an image in which the feature points 212 do not exist in areas other than the specific area 211. As described above, if the specific area 211 is a circular ring-shaped surface, the feature image data 21 is data of an image in which the feature points 212 do not exist in areas other than the circular ring-shaped area corresponding to the specific area 211. Specifically, the image data is such that all pixels in areas other than the circular ring-shaped area corresponding to the end face of the sun gear shaft are black pixels (pixels with a brightness value of 0).

[0018] First, the imaging device 121 captures an image of the specific region 211 so as to capture an image having a tendency that "the luminance of pixels included in the specific region 211 is higher than the luminance of pixels included in other regions (or regions adjacent to the specific region 211)." For example, the imaging device 121 captures an image of the specific region 211 in a state in which the specific region 211 is irradiated with light by a lighting device (not shown). Then, the imaging device 121 generates original image data and transmits the generated original image data to the processing device 122.

[0019] The processing device 122 grayscales the original image data acquired from the imaging device 121, binarizes the grayscaled original image data, and labels (assigns an identification number) the white areas included in the binarized original image data. The processing device 122 then detects the contour of the specific area 211 from the binarized original image data, and generates a mask image using the detected contour. The mask image is an image in which no feature points 212 exist in areas other than the specific area 211 (for example, an image in which the brightness of all pixels in areas other than the specific area 211 is 0). The processing device 122 then generates "image data in which the brightness of all pixels other than the area corresponding to the specific area 211 is 0" by overlaying the mask image on the original image data.

[0020] The processing device 122 grayscales the generated "image data in which all pixels other than the area corresponding to the specific area 211 have a brightness of 0", binarizes the grayscaled image data, and labels (assigns an identification number) the white areas included in the binarized image data. The processing device 122 then detects the contour of the specific area 211 from the binarized image data, and generates a mask image again using the detected contour. The processing device 122 then overlays the regenerated mask image on the original image data, thereby generating "image data in which all pixels other than the area corresponding to the specific area 211 have a brightness of 0". The processing device 122 performs histogram equalization on the generated image data. As a result, the characteristic image data 21 is generated.

[0021] <Individual Identification Method> The individual identification method according to this embodiment includes a preparation stage and an identification stage.

[0022] (Preparation stage) The preparation stage includes a process of extracting feature points 212 present in a specific region 211 of multiple individual items of the same kind, linking the feature information, which is information on the extracted feature points 212, with the tracking information of each individual item, and registering the feature information in the common storage device 11. Note that "registering the feature information in the common storage device 11" refers to storing the feature information in the common storage device 11 so that the processing device 122 of each terminal device 12 of the individual identification system 10 can read the feature information from the common storage device 11. The tracking information is information that can be used to track (trace) each individual item of the same kind. It can also be said that the tracking information is information for identifying each individual item. Examples of the tracking information that can be used include the manufacturing number (serial number) of each individual item, information indicating the date of manufacture, information indicating the manufacturer (manufacturing company), and information indicating the place of manufacture (manufacturing plant).

[0023] The imaging device 121 captures images of specific areas 211 of multiple individual articles of the same type under the control of the processing device 122 to generate original image data, and transmits the generated original image data to the processing device 122. The processing device 122 generates feature image data 21 from the original image data acquired from the imaging device 121, and extracts feature points 212 from the generated feature image data 21. The common storage device 11 then registers each individual article from which the feature points 212 have been extracted as a registered individual, linking it with information on the feature points 212 (feature information) and information identifying the individual article (tracking information). The A-KAZE method is applied as a method for extracting the feature points 212.

[0024] Next, the processing device 122 uses the feature information of the registered individual to set a "matching rate threshold" to be used in the identification stage described below. Methods for setting the matching rate threshold include feature point matching, geometric verification, partial region setting, matching rate calculation, and threshold setting.

[0025] Feature point matching is a process of matching "feature points 212 having the same or similar features" contained in two individuals selected from a plurality of registered individuals. This feature point matching can also be said to be a process of matching "the same feature points 212" present in each of the two individuals, assuming that the two selected individuals are the same individual. This feature point matching is a process included in the A-KAZE method. For ease of explanation, one of the two individuals may be distinguished by being referred to as the "first individual" and the other as the "second individual".

[0026] 3A and 3B are diagrams for explaining the concept of feature point matching. In FIG. 3A and FIG. 3B and the explanation of feature point matching, for convenience of explanation, the feature image data 21 of the first individual is given the reference symbol "21a", the specific area 211 of the feature image data 21a of the first individual is given the reference symbol "211a", the feature point 212 of the feature image data of the first individual is given the reference symbol "212a", the feature image data 21 of the second individual is given the reference symbol "21b", the specific area 211 of the feature image data 21b of the second individual is given the reference symbol "211b", and the feature point 212 of the feature image data of the second individual is given the reference symbol "212b". In this example, the feature point 212a existing in the specific area 211a of the feature image data 21a of the first individual is associated with the feature point 212b existing in the specific area 211b of the feature image data 21b of the second individual. The feature points 212a, 212b connected by a straight line are corresponding feature points 214a, 214b. The feature points 214a, 214b that are corresponding to each other may be referred to as "corresponding feature points 214a, 214b."

[0027] When the first individual and the second individual are the same individual, the feature image data 21a of the first individual and the feature image data 21b of the second individual contain the same feature points 212a, 212b, and these are associated with each other. However, even if the first individual and the second individual are the same individual, the feature points 212a extracted from the feature image data 21a of the first individual and the feature points 212b extracted from the feature image data 21b of the second individual may differ. In this case, the same feature points 212a, 212b may not be associated with each other, and different feature points 212a, 212b may be associated with each other.

[0028] The geometric verification is a process of verifying (or estimating) the geometric relationship between the corresponding feature points 214a present in the feature image data 21a of the first individual and the corresponding feature points 214b present in the feature image data 21b of the second individual. Specifically, the processing device 122 performs homography estimation for the corresponding feature points 214a of the feature image data 21a of the first individual and the corresponding feature points 214b of the second individual, and generates a homography transformation matrix. The relationship defined by this homography transformation matrix is ​​an example of a geometric relationship. At this time, if erroneous corresponding feature points 216a, 216b (corresponding feature points 216a, 216b that are not the same feature points 212a, 212b but are associated with each other) are included, the homography estimation cannot be performed correctly, and therefore a correct homography transformation matrix cannot be obtained. For this reason, in this embodiment, the processing device 122 of the terminal device 12 performs homography estimation using RANSAC (Random Sample Consensus) to remove the influence of the erroneous corresponding feature points 216a, 216b. Note that homography estimation using RANSAC is well known.

[0029] The partial region setting is a process of setting a partial region 213 in the specific region 211a of the feature image data 21a of the first individual. The partial region 213 is a region set within the specific region 211a and has an area smaller than that of the specific region 211a. In FIG. 3B, the hatched region is the partial region 213. When the specific region 211a is an annular region, an arc-shaped region having a central angle of a predetermined value is set as the partial region 213. Note that the value of the central angle of the partial region 213 in this case is not particularly limited, but a value in the range of 30 to 90°, for example, can be applied.

[0030] The matching rate calculation is a process of calculating the matching rate. The processing device 122 performs coordinate transformation of the feature points 212a included in one partial area 213 of the specific area 211a of the feature image data 21a of the first individual using the projective transformation matrix estimated by the geometric verification. As a result, the coordinates of each feature point 212a included in the partial area 213 set in the specific area 211a of the feature image data 21a of the first individual are transformed into coordinates in the feature image data 21b of the second individual. If the first individual and the second individual are the same individual, then in theory, the corresponding feature point 214b exists at the same position as the transformed coordinates. However, if the first individual and the second individual are not the same individual, or if the first individual and the second individual are the same individual but erroneous corresponding feature points 216a, 216b exist, the corresponding feature point 214b may not exist at the same position as the transformed coordinates.

[0031] The processing device 122 counts the number of feature points 212a present in one partial region 213 of the feature image data 21a of the first individual. Furthermore, as shown in FIG. 3B, the processing device 122 counts the number of "corresponding feature points 215b correctly associated with the feature points 212a present in the one partial region 213 of the feature image data 21a of the first individual" among the corresponding feature points 214b present in the feature image data 21a of the second individual. Note that the "corresponding feature points 215b correctly associated" are "feature points 215b located at the same positions as the coordinates obtained by performing coordinate transformation on the feature points 212a included in the partial region 213 of the feature image data 21a of the first individual using a projective transformation matrix." The processing device 122 then calculates the matching rate using the following formula (1). Matching rate=(the number of “corresponding feature points 215b that correctly correspond to the feature points 212a present in one partial region 213 of the feature image data 21a of the first individual” among the feature points 212b present in the feature image data 21b of the second individual) / (the number of feature points 212a present in one partial region 213 of the feature image data 21a of the first individual) Formula (1) In addition, the "one partial region 213" in the denominator of the formula (1) and the "one partial region 213" in the numerator are the same partial region.

[0032] The processing device 122 repeats changing the position of the partial region 213 of the feature image data 21a of the first individual and calculating the matching rate for the changed partial region 213 until there is no region in the specific region 211a that has not been set as a partial region 213. In this way, a plurality of partial regions 213 are set in the specific region 211a, and a matching rate is calculated for each of the set partial regions 213. For example, if the specific region 211a is a circular ring-shaped region and the partial region 213 is an arc-shaped region, the partial region 213 is shifted in the circumferential direction by a predetermined angle (for example, 1°) and the matching rate for the shifted partial region 213 is calculated. This process is repeated (360(°) / (predetermined angle(°))) times.

[0033] Furthermore, the processing device 122 weights the matching rate of the partial region 213 according to the value of the matching rate. In this embodiment, the larger the value of the matching rate, the larger the weight. For example, for a partial region 213 whose matching rate is less than 0.3, the processing device 122 multiplies the matching rate of the partial region 213 by a weighting factor of 0.1. For a partial region 213 whose matching rate is equal to or greater than 0.3 and less than 0.6, the processing device 122 multiplies the matching rate of the partial region 213 by a weighting factor of 0.7. For a partial region 213 whose matching rate is 0.6 or greater, the processing device 122 multiplies the matching rate of the partial region 213 by a weighting factor of 1. The reason for weighting will be described later.

[0034] Then, the processing device 122 executes the above-mentioned feature point matching, geometric verification, partial region setting, and matching rate calculation for all combinations of registered individuals. Specifically, the processing device 122 selects one individual from the registered individuals as a first individual, and selects each of the registered individuals as a second individual. Then, for the feature point 212a present in the feature image data 21a of the selected first individual and the feature point 212b present in the feature image data 21b of the second individual (i.e., for each combination of one first individual and multiple second individuals), the processing device 122 executes the above-mentioned feature point matching, geometric verification, partial region setting, and matching rate calculation. Then, the processing device 122 sequentially selects first individuals so that all registered individuals are selected as first individuals, and repeatedly executes such a series of processes for each combination of the selected first individual and multiple second individuals. Therefore, when the number of registered individuals is N (feature information of N individuals is registered) and M partial regions 213 are set for the specific region 211a of one individual, N×N×M matching rates are obtained.

[0035] In the threshold setting, the processing device 122 creates a histogram of the matching rate (weighted matching rate) when the first individual and the second individual are different individuals, and a histogram of the matching rate (weighted matching rate) when the first individual and the second individual are the same individual, and approximates each of the created histograms with a β distribution. FIG. 4 is a graph showing an example of a histogram and a β distribution. Then, the processing device 122 sets a threshold used in the identification stage using these β distributions. Specifically, a threshold is specified such that the "probability of a range equal to or greater than a threshold in a probability density function of a β distribution in which the first individual and the second individual are the same individual" and the "probability of a range equal to or less than a threshold in a probability density function of a β distribution in which the first individual and the second individual are not the same individual" are the same value. It can be said that this threshold is a value such that the "probability of misjudging different individuals as the same individual" and the "probability of misjudging the same individual as different individuals" are equal. The value of the probability when the probability of misjudging different individuals to be the same individual and the probability of misjudging the same individual to be different individuals are equal may be referred to as the equivalent error rate. Then, the processing device 122 registers the set threshold value in the common storage device 11.

[0036] Furthermore, the processing device 122 may set, as the threshold value, a value between the maximum value of the match rate (weighted match rate) when the first individual and the second individual are different individuals and the minimum value of the match rate (weighted match rate) when the first individual and the second individual are the same individual. For example, Threshold value=((maximum value of the matching rate when the first individual and the second individual are different individuals)+(minimum value of the matching rate when the first individual and the second individual are the same individual)) / 2 may be also possible.

[0037] The above is the preparation stage. The preparation stage may be automatically executed by the terminal device 12 included in the individual identification system 10, or may be executed by the terminal device 12 in response to an operation by a user of the terminal device 12. For example, a configuration in which the terminal device 12 included in the individual identification system 10 is incorporated in a manufacturing line for goods can be applied. In addition, the user of the terminal may operate the terminal device 12 to link the characteristic information with the tracking information and register them in the common storage device 11.

[0038] (Identification stage) The identification stage is a stage in which it is determined whether the individual to be identified (an individual of the same species as the registered individual) is the same individual as any of the registered individuals. The identification stage includes feature point extraction, feature point matching, geometric verification, partial region setting, match rate calculation, collation, identification, and output.

[0039] Feature point extraction is a process of extracting feature points 212 of an individual to be identified. The imaging device 121 captures an image of a specific region 211 of the individual to be identified under the control of the processing device 122 to generate original image data, and transmits the generated original image data to the processing device 122. The processing device 122 generates feature image data 21 from the acquired original image data, and extracts feature points 212 from the generated feature image data 21. The method of generating the feature image data 21 and the method of extracting the feature points 212 are the same as in the preparation stage.

[0040] Next, the processing device 122 sequentially executes feature point matching, partial area setting, and matching rate calculation for feature points 212 extracted from feature image data 21 obtained by capturing an image of a specific area 211 of the individual to be identified and feature points 212 of one individual selected from the multiple registered individuals. The contents of these processes are the same as those in the preparatory stage, except that the first individual is an "individual to be identified" (in the preparatory stage, the first individual is "one of the registered individuals"). Note that the second individual in these processes is "one of the registered individuals" as in the preparatory stage. For this reason, the matching rate in the identification stage is also calculated by the above formula (1). Then, the processing device 122 weights the matching rate calculated by formula (1) as in the preparatory stage.

[0041] Matching is a process of determining whether the identification target individual (first individual) and the selected registered individual (second individual) are the same individual. In this case, it can be said that the identification target individual is the matching target, and the selected registered individual is the matching source. The processing device 122 determines whether the matching rate of each partial region 213 includes a matching rate equal to or higher than a threshold set in the preparation stage. In other words, it determines whether the maximum value of the matching rates of the partial regions 213 is equal to or higher than a threshold. Then, if the maximum value of the matching rate is equal to or higher than the threshold (in other words, if there is a partial region 213 whose matching rate is equal to or higher than the threshold), the processing device 122 sets the selected registered individual as a "candidate individual". On the other hand, if the maximum value of the matching rate is less than the threshold, the processing device 122 determines that the identification target individual is not the same individual as the selected registered individual.

[0042] Then, the processing device 122 executes the above-mentioned feature point extraction, feature point matching, geometric verification, partial region setting, match rate calculation, and collation for the combination of the "identification target individual" and "each of all registered individuals."

[0043] Identification is a process of identifying an individual that is the same as the identification target individual from among the candidate individuals. When one candidate individual is set, the processing device 122 determines that "the identification target individual is the same as the one candidate individual." When multiple candidate individuals are set, the processing device 122 determines that "the identification target individual is the same as the candidate individual with the highest matching rate." When no candidate individual is set, the processing device 122 determines that "the identification target individual is not the same as any of the registered individuals (individuals whose characteristic information is registered in the common storage device 11)."

[0044] Output is a process of outputting the result of identification (identification). The processing device 122 causes the output device 124 to output the result of identification. For example, when the processing device 122 determines that the individual to be identified is the same individual as one of the registered individuals, it causes the output device 124 to output information indicating this, and also causes the output device 124 to output tracking information of the same individual. On the other hand, when the processing device 122 determines that the individual to be identified is not the same individual as any of the registered individuals, it causes the output device 124 to output information indicating this. This is the identification stage.

[0045] Here, a specific application example of the individual identification system 10 according to this embodiment will be described. One of the terminal devices 12 of the individual identification system 10 is installed in a factory (hereinafter referred to as the "first factory") that manufactures the item to be identified, the other is installed in a factory (hereinafter referred to as the "second factory") that manufactures a product incorporating the item to be identified, and still another is installed in a service factory, a store, or the like. Then, a preparation stage is executed in the terminal device 12 installed in the first factory. That is, the terminal device 12 installed in the first factory registers all the individual items manufactured as registered items in the common storage device 11 in association with their characteristic information and tracking information.

[0046] If a defect or the like is detected in a product manufactured in the second factory, an identification step is performed on the individual item incorporated in the product in which the defect or the like is detected in the terminal device 12 installed in the second factory. This allows users, etc., to refer to the tracking information of the individual item incorporated in the product in which the defect or the like is detected in the second factory.

[0047] Furthermore, after a product incorporating an item to be identified is released to the market, when the product is brought to a service factory, a store, etc. due to a malfunction or the like, the identification step is executed using the terminal device 12 installed in the service factory, the store, etc., so that the user, etc. can refer to the tracking information of the individual item incorporated in the product. Thus, according to this embodiment, the user, etc. can refer to the tracking information of the individual item incorporated in the product during the period from when the product is manufactured in the first factory to when it is incorporated in another product in the second factory, and during the period from when the product is incorporated in another product in the second factory to when it is actually used in the market.

[0048] According to the present embodiment, the partial regions 213 are set in the specific region 211, and the matching rate is calculated for each partial region 213, so that the accuracy of identifying individual articles can be improved. That is, the surface properties of the specific region 211 of each individual article may change due to contact with other objects or deterioration over time after the manufacture of each individual article. In particular, if the specific region 211 of each individual article is scratched or worn, the feature points 212 that existed in the specific region 211 at the time of manufacture (registration) may no longer exist, or new feature points 212 may appear. For this reason, in a configuration in which matching (identification) is performed using the matching rate calculated over the entire specific region 211, even if a part of the specific region 211 is locally scratched, the accuracy of identification may be reduced. In contrast, according to the present embodiment, the matching rate is calculated for each partial region 213 rather than for the entire specific region 211. Therefore, even if the specific region 211 is locally scratched or the like and the feature point 212 has changed, if there is a partial region 213 in which the feature point 212a from the time of manufacture (registration) remains, the matching rate of that partial region 213 can be used to improve the accuracy of identification.

[0049] Furthermore, the inventors have found that in the case of the same individual, the correct corresponding feature points 215a, 215b are locally abundant. For this reason, by setting a partial region 213 smaller than the specific region 211 in the specific region 211 and weighting the matching rate according to the number of corresponding feature points 215a, 215b that correctly correspond to the feature points 212a present in the partial region 213 (i.e., the density of the corresponding feature points 215a, 215b), the influence of the matching rate of the partial region 213 with a large number of correctly corresponding corresponding feature points 215a (which can be called the "partial region 213 with a high density of correctly corresponding corresponding feature points 215a") in the classification can be increased, and the influence of the partial region 213 with a small number of correctly corresponding corresponding feature points 215a, 215b (the partial region 213 with a low density of correctly corresponding corresponding feature points 215a, 215b) can be reduced. Therefore, the effect of improving the accuracy of classification can be enhanced.

[0050] In addition, in this embodiment, the threshold value of the matching rate is set to a value that makes the "probability of misjudging different individuals as the same individual" and the "probability of misjudging the same individual as different individuals" equal. With this configuration, it is possible to minimize the "probability of misjudging different individuals as the same individual" and the "probability of misjudging the same individual as different individuals". Therefore, it is possible to improve the accuracy of identification. In addition, as the threshold value of the matching rate, a value between the maximum value of the matching rate (weighted matching rate) when the first individual and the second individual are different individuals and the minimum value of the matching rate (weighted matching rate) when the first individual and the second individual are the same individual may be set as the threshold value. Even with this configuration, it is possible to reduce the "probability of misjudging different individuals as the same individual" and the "probability of misjudging the same individual as different individuals". Therefore, even with this configuration, it is possible to improve the accuracy of identification of individual information.

[0051] When the A-KAZE method is applied to the extraction of the feature points 212 and the feature point matching, the accuracy of the identification can be improved. Specifically, the A-KAZE method is a robust method that is highly resistant to enlargement / reduction, rotation, and illumination changes. Therefore, even if the imaging conditions (e.g., the position, orientation, and size of the individual in the image) when imaging each individual for registering feature information in the preparation stage are different from the imaging conditions when imaging the individual to be identified in the identification stage, the feature points 212 can be extracted and the feature points can be matched with high accuracy. In addition, by using homography estimation using RANSAC (Random Sample Consensus) for geometry verification, the geometric relationship can be estimated with high accuracy even if incorrect corresponding feature points 216a and 216b exist. Therefore, the accuracy of the identification can be improved.

[0052] <Specific operation of the individual identification system> 5A and 5B are flowcharts showing the specific operation of the individual identification system in the preparation stage. A computer program for executing the processing of this flowchart is stored in the ROM of the terminal device 12 or the terminal storage device 123 in advance.

[0053] In step S101, the CPU of the processing device 122 (hereinafter, sometimes simply referred to as the CPU) sets an individual for which feature information is to be registered (it can also be said that the individual for which feature information is to be registered is selected from multiple individuals of the same kind of article).Then, the CPU advances the process to step S102. Note that the user may select the individual for which feature information is to be registered.

[0054] In step S102, the CPU controls the imaging device 121 to capture an image of the specific region 211 of the individual whose feature information is to be registered. Then, the CPU acquires the original image data generated by the imaging device 121 from the imaging device 121. Then, the CPU advances the process to step S103.

[0055] In step S103, the CPU generates feature image data 21 using the acquired original image data. Next, in step S104, the CPU extracts feature points 212a from the generated feature image data 21. Next, in step S105, the CPU registers feature information, which is information on the extracted feature points 212a, in the common storage device 11 by linking it with tracking information of the individual to be registered.

[0056] In step S106, the CPU determines whether or not there is an individual for which characteristic information has not yet been registered. If the CPU determines that there is an individual for which characteristic information has not yet been registered, the process proceeds to step S107. In step S107, the CPU changes the individual for which characteristic information is to be registered to an individual for which characteristic information has not yet been registered. The CPU then returns the process to step S102. In this case, the CPU executes steps S102 to S105 for the changed individual. If the CPU has completed registration of characteristic information for all individuals to be registered, the process proceeds to step S108.

[0057] In step S108, the CPU sets (in other words, selects) one individual from the registered individuals as a first individual, and then the CPU advances the process to step S109.

[0058] In step S109, the CPU sets (in other words selects) one individual from the registered individuals as the second individual. Note that the CPU does not exclude the individual selected as the first individual in step S108 from the individuals selected as the second individual in step S109. In other words, the first individual and the second individual may be the same individual. The CPU then proceeds to step S110.

[0059] In step S110, the CPU performs feature point matching between the first individual set in step S108 and the second individual set in step S109. The CPU then performs geometric verification using the result of the feature point matching to estimate a homography matrix. The CPU then proceeds to step S111.

[0060] In step S111, the CPU sets the partial region 213 in the specific region 211a of the first individual. Then, the CPU advances the process to step S112.

[0061] In step S112, the CPU calculates the matching rate between the feature points 212a existing in the partial region 213 set in step S111 (or the partial region 213 changed in step S114 described later) and the feature points 212b of the second individual. Then, the CPU advances the process to step S113.

[0062] In step S113, the CPU determines whether or not an area that has not been set as a partial area 213 remains in the specific area 211a. If the CPU determines that an area that has not been set as a partial area 213 remains, the CPU advances the process to step S114. In step S114, the CPU sets "an area including an area that has not yet been set as a partial area 213" of the specific area 211a as the partial area 213 (in other words, changes the position of the partial area 213). The CPU then returns the process to step S112. In this case, the CPU executes the process of step S112 for the newly set partial area 213. On the other hand, if the CPU determines that no area that has not been set as a partial area 213 remains, the CPU advances the process to step S115.

[0063] In step S115, the CPU weights the matching rate of each partial region 213. The CPU then advances the process to step S116.

[0064] In step S116, the CPU determines whether or not there is a registered individual that has not been set as the second individual. If there is a registered individual that has not been set as the second individual, the CPU advances the process to step S117. In step S117, the CPU sets an individual (if there are multiple individuals, one of them) that has not yet been set as the second individual from the registered individuals as the second individual (in other words, changes the second individual). Then, the CPU returns the process to step S110. In this case, the CPU sets the registered individual selected in step S108 as the first individual and the registered individual set in step S117 as the second individual, and executes the processes of steps S110 to S115. On the other hand, if there is no registered individual that has not been set as the second individual (if the processes of steps S110 to S115 have been executed for all individuals), the CPU advances the process to step S118.

[0065] In step S118, the CPU determines whether or not there is an individual that has not yet been set as the first individual. If there is an individual that has not yet been set as the first individual, the CPU proceeds to step S119. In step S119, the CPU sets an individual that has not yet been set as the first individual from the registered individuals (one of them if there are multiple individuals) as the first individual (in other words, changes the first individual). Then, the CPU returns the process to step S109. In this case, the CPU executes the processes of steps S109 to S117 with the individual set in step S119 as the first individual. On the other hand, if there is no individual that has not yet been set as the first individual (if the processes of steps S109 to S117 have been executed for all individuals), the CPU proceeds to step S120.

[0066] In step S120, the CPU creates a histogram of the matching rates for the same individual and a histogram of the matching rates for different individuals, approximates each of these histograms with a β distribution, and sets a threshold value using the approximated β distribution.The CPU then registers the set threshold value in the common storage device 11.

[0067] Next, a specific operation of the individual identification system in the identification stage will be described. Figures 6A and 6B are flowcharts showing the operation of the individual identification system in the identification stage. A computer program for executing the processing of this flowchart is stored in the ROM of the terminal device 12 or the terminal storage device 123 in advance.

[0068] In step S201, the CPU causes the imaging device 121 to capture an image of the specific area 211 of the individual to be identified, and acquires original image data from the imaging device 121. Then, the CPU advances the process to step S202.

[0069] In step S202, the CPU uses the acquired original image data to generate feature image data 21. Next, in step S203, the CPU extracts feature points 212 from the generated feature image data 21. Then, the CPU advances the process to step S204.

[0070] In step S204, the CPU sets (in other words selects) one of the registered individuals as the individual to be matched, and then the CPU advances the process to step S205.

[0071] In step S205, the CPU sets the identification target individual extracted in step S203 as the matching target individual. Then, the CPU performs feature point matching between the feature points 212 of the individual set as the matching target and the feature points 212b of the individual set as the matching source in step S204. Furthermore, the CPU performs geometric verification using the result of the feature point matching to estimate a homography matrix. Then, the CPU proceeds to step S206.

[0072] In step S206, the CPU sets the partial region 213 in the specific region 211 of the individual to be matched (the individual imaged in step S201).Then, the CPU advances the process to step S207.

[0073] In step S207, the CPU calculates the matching rate between the feature points 212 present in the partial region 213 of the individual set as the matching target in step S206 (or step S209 described later) and the feature points 212 present in the registered individual set as the matching source in step S204. Then, the CPU proceeds to step S208.

[0074] In step S208, the CPU determines whether or not any area that has not been set as a partial area 213 remains in the specific area 211. If the CPU determines that any area that has not been set as a partial area 213 remains in the specific area 211, the CPU advances the process to step S209. In step S209, the CPU sets an area including an "area that has not yet been set as a partial area 213" in the specific area 211 as the partial area 213 (this can also be said to change the partial area 213). The CPU then returns the process to step S207. In this case, the CPU executes the process of step S207 for the newly set partial area 213. On the other hand, if the CPU determines that no area that has not been set as a partial area 213 remains in the specific area 211, the CPU advances the process to step S210.

[0075] In step S210, the CPU assigns weights to the matching rates of the partial regions 213. The CPU then advances the process to step S211.

[0076] In step S211, the CPU determines whether the maximum value of the matching rate is equal to or greater than the threshold (in other words, whether there is a partial region 213 whose matching rate is equal to or greater than the threshold). If the maximum value of the matching rate is equal to or greater than the threshold, the CPU proceeds to step S212, and if the maximum value of the matching rate is less than the threshold, the CPU proceeds to step S213. In step S212, if the maximum value of the matching rate is equal to or greater than the threshold, the CPU sets this registered individual as a "candidate individual". On the other hand, in step S213, the CPU determines that the individual whose image of the specific region 211 was captured in step S201 is not the same individual as this individual. The CPU then proceeds to step S214.

[0077] In step S214, the CPU determines whether or not there is an individual that has not been selected as a matching source among the registered individuals. If there is an individual that has not been selected as a matching source, the CPU proceeds to step S215. In step S215, the CPU selects an individual that has not yet been selected as a matching source from the registered individuals (one of them if there are multiple individuals) as the matching source individual. The CPU then returns the process to step S205. In this case, the CPU executes the processes of steps S205 to S213 with the identification target individual as the matching target and the registered individual selected in step S215 as the matching source. On the other hand, if there is no individual that has not been selected as a matching source (if the processes of steps S205 to S210 have been executed for all individuals), the CPU proceeds to step S216.

[0078] In step S216, the CPU selects the registered individual with the highest matching rate from among the candidate individuals. Then, the CPU determines that "the identification target individual (the individual whose image of the specific area 211 was captured in step S201) is the same individual as the individual with the highest matching rate among the registered individuals." If there is no candidate individual, the CPU determines that "the identification target individual is not the same individual as any of the registered individuals." Then, the CPU proceeds to step S217.

[0079] In step S217, the CPU outputs the classification result to the output device 124. Then, the CPU ends the series of processes.

[0080] By such an operation, the above-mentioned individual identification method is realized.

[0081] Next, an embodiment of the present invention will be described.

[0082] <Common conditions> The inventors prepared five sun gears as multiple individuals of the same kind of article, and (A) As is (B) After wiping with a cloth (C) After cleaning and then wiping with a cloth (D) Oil was applied, then washed, and then wiped off with a rag. The preparation stage was carried out in the four states above, and feature information was registered. On the other hand, the original image data of the individual to be identified is as follows: (a) The same original image data as (A) above (b) The same original image data as (B) above (c) Original image data in which the position of the specific area in the image has been changed under the same conditions as (B) above. (d) Original image data in which the position of the specific area in the image is changed and the specific area is rotated 180° under the same conditions as (B) above. (e) The same original image data as (C) above (f) Original image data in which the position of the specific area in the image has been changed under the same conditions as (C) above. (g) Original image data in which the position of the specific area in the image is changed and the specific area is rotated 180° under the same conditions as (C) above. (h) The same original image data as (D) above (i) Original image data in which the position of the specific area in the image has been changed under the same conditions as (D) above. (j) Original image data in which the position of the specific area in the image is changed and the specific area is rotated 180° under the same conditions as (D) above. Ten types of original image data were generated.

[0083] The present inventors carried out the following three examples. First embodiment: Assuming that each individual is identified within the manufacturing plant (in other words, for short-term use), the image (A) above is used as the original image data of the registered individual, and the image data (a), (b), (c), and (d) above are used as the original image data of the individual to be identified, and an identification step is performed. Second embodiment: Assuming identification after the product is released to the market (in other words, for long-term use), an embodiment in which the image (C) above is used as the original image data of the registered individual, and the image data (h), (i), and (j) above are used as the original image data of the individual to be identified, and an identification step is performed. Third Example: In order to verify the effect of the embodiment, the image (A) above is used as the original image data of the registered individual, and the image data (h), (i), and (j) above are used as the original image data of the individual to be recognized, and the recognition step is performed.

[0084] <First Verification> In all of the above first, second, and third examples, there were no erroneous judgments (judging the same individual to be a different individual, and different individuals to be the same individual), and all were correctly identified. The equivalent error rate in the first example was 1.2 × 10 ―9 and the equivalent error rate in the second embodiment is 4.6×10 ―1 3, and the equivalent error rate in the third embodiment is 8.8×10 -8 In these experiments, the results showed that the equivalent error rate was the lowest in the second embodiment. This shows that the accuracy of identification can be improved by cleaning a specific area of ​​the individual to be identified.

[0085] <Second Verification> The inventors measured the "matching rate between the same individual" and the "matching rate between different individuals" for the above-mentioned third embodiment, first comparative example, and second comparative example. The first comparative example is an example in which homography estimation and partial region setting are not performed. Specifically, it is an example of a method in which feature points present in the entire specific region of feature image data are matched by the A-KAZE method, and whether or not they are the same individual is determined based on the match rate. The second comparative example is an example of a method in which homography estimation is performed but partial region setting is not performed, and the match rate is calculated for the entire specific region.

[0086] FIG. 7 is a graph showing the measurement results of the "matching rate of different individuals" (value marked "different" in FIG. 7) and the "matching rate between the same individuals" (value marked "same" in FIG. 7) in the third embodiment, the first comparative example, and the second comparative example. As shown in FIG. 7, in the first comparative example, the match rate between different individuals may be higher than the match rate between the same individuals. In this case, there is a possibility of misidentification. On the other hand, in the second comparative example and the third embodiment, the match rate between different individuals may not be higher than the match rate between the same individuals. Therefore, misidentification does not occur in these cases. And, according to the third embodiment, it was confirmed that the difference in the match rate is larger than that of the second comparative example. The larger this difference, the less likely misidentification occurs. Therefore, it was confirmed that the method of this embodiment can improve the accuracy of identification.

[0087] <Third Verification> The inventors verified the difference in matching rate and the equivalent error rate for each of the Examples, the First Comparative Example, and the Second Comparative Example. FIG. 8A shows the verification result for short-term use, FIG. 8B shows the verification result for long-term use, and FIG. 8C shows the verification result for the combined period of short-term use and long-term use. As shown in FIG. 8A to FIG. 8C, it was confirmed that the difference in matching rate was larger in each of the Examples than in the First Comparative Example and the Second Comparative Example. It was also confirmed that the difference in matching rate was maximum when the angle was 30°. On the other hand, it was confirmed that the equivalent error rate was best when the angle was 90°.

[0088] <Summary of the embodiment> The individual identification system 10 according to this embodiment includes: A first imaging device (imaging device 121) that captures an image of a specific area 211 that is a part of a plurality of individual articles of the same kind, and generates first image data that is data of an image in which the specific area 211 is captured for each individual article; A first processing device (processing device 122) that extracts first feature points, which are feature points 212 present in the specific region 211 for each individual, based on the first image data; a storage device (common storage device 11) that registers, as a registered individual, each of the individuals from which the first feature points have been extracted by the first processing device (processing device 122) together with information on the first feature points; A second imaging device captures an image of the specific area 211 of an identification target individual, which is one of the articles of the same kind, and generates second image data, which is data of an image including the specific area 211; a second processing device (processing device 122) that extracts second feature points, which are feature points present in the specific region 211 of the individual to be identified based on the second image data, and determines whether the individual to be identified is identical to any one of the multiple registered individuals based on information of the extracted second feature points and information of the first feature points of the multiple registered individuals registered in the storage device (common storage device 11); Equipped with The second processing device (processing device 122) estimates the geometric relationship between the second feature points and the first feature points, sets a plurality of partial areas 213 which are portions of the specific area 211 in the specific area 211 of the second image data, calculates the matching rate between each registered individual and the individual to be identified for each of the set partial areas 213 using the following formula (1), and if there is a registered individual for which the matching rate is equal to or greater than a threshold, it determines that the individual to be identified is the same individual as the registered individual for which the matching rate is the highest among the registered individuals for which the matching rate is equal to or greater than the threshold. The matching rate=(the number of the second feature points that have the estimated geometric relationship with the first feature points among the second feature points) / (the number of the second feature points included in one of the partial regions set in the specific region of the second image data) Formula (1)

[0089] Further, the individual identification method according to the present embodiment is an individual identification method executed by the individual identification system 10, a first image data generation step of capturing an image of a specific area 211, which is a part of a plurality of individual articles of the same kind, and generating first image data, which is data of an image in which the specific area 211 is captured, for each individual article; a first feature point extraction step of extracting first feature points, which are feature points 212 present in the specific region 211 of each of the individuals, based on the first image data; a registration step of registering each of the individuals from which the first feature points have been extracted as a registered individual together with information on the first feature points; a second image data generation step of capturing an image of the specific area 211 of an identification target individual, which is an individual of the same kind of article, and generating second image data which is data of an image including the specific area 211; a classification step of extracting second feature points, which are feature points 212 present in the specific region 211 of the individual to be classified, based on the second image data, and determining whether the individual to be classified is identical to any one of the multiple registered individuals based on information on the extracted second feature points and information on the first feature points of the registered individuals; Equipped with The identification step includes a step of estimating a geometric relationship between the second feature points and the first feature points; a step of setting a plurality of partial areas which are parts of the specific area of ​​the second image data, and calculating a match rate between each registered individual and the individual to be identified for each of the set partial areas using the following formula (3); and a step of determining, when there is a registered individual for which the match rate is equal to or greater than a threshold, that the individual to be identified is the same individual as the individual with the highest match rate among the registered individuals for which the match rate is equal to or greater than the threshold. The matching rate=(the number of the feature points among the second feature points that have the estimated geometric relationship with the first feature points) / (the number of the second feature points included in one of the partial regions set in the specific region of the second image data) Equation (3)

[0090] According to this embodiment, the matching rate is calculated for each partial region 213 smaller than the specific region 211, rather than for the entire specific region 211. Therefore, even if the specific region 211 is locally scratched or the like and the feature points 212 have changed, if there is a partial region 213 in which the feature points 212 from the time of manufacture (time of registration) remain (or a partial region 213 with a large number of remaining feature points 212), the matching rate of that partial region 213 will be higher than the matching rates of the other partial regions 213. Therefore, by using a partial region 213 with a high matching rate, the accuracy of identification can be improved.

[0091] In the individual identification system 10 according to this embodiment, The first processing device (processing device 122) estimating the geometric relationship between the first feature points present in each of the same two first image data captured for one of the plurality of individuals from which the first feature points have been extracted, setting a plurality of partial regions 213 in a specific region 211 of the first image data for the one individual, calculating the matching rate of the same individual for each of the set plurality of partial regions 213 using the following formula (2), and calculating a first probability density function that approximates a histogram of the calculated matching rates; estimating the geometric relationship between the first feature points present in two different first image data in which two different individuals from the plurality of individuals from which the first feature points have been extracted are captured, setting a plurality of the partial regions 213 in one of the two first image data, calculating the matching rate of the different individuals for each of the set plurality of partial regions 213 using the following formula (2), and calculating a second probability density function that approximates a histogram of the calculated matching rates; The threshold is set so that the probability of the first probability density function in a range equal to or greater than the threshold and the probability of the second probability density function in a range equal to or less than the threshold are the same value. The matching rate=(the number of the second feature points that have the estimated geometric relationship with the second feature points that exist in one of the partial regions of one of the two second image data, among the second feature points that exist in the other of the two second image data) / (the number of the second feature points that exist in one of the partial regions of one of the two second image data)

[0092] The individual identification method according to the present embodiment includes the steps of: estimating the geometric relationship between the first feature points present in each of the same two first image data captured for one of the individuals from which the first feature points have been extracted, setting a plurality of partial regions 213 in a specific region 211 of the first image data for the one individual, calculating the matching rate of the same individual for each of the set plurality of partial regions 213 using the following formula (2), and calculating a first probability density function that approximates a histogram of the calculated matching rates; estimating the geometric relationship between the first feature points present in two different first image data in which two different individuals from the plurality of individuals from which the first feature points have been extracted are captured, setting a plurality of the partial regions in one of the two first image data, calculating the matching rate of the different individuals for each of the set partial regions by the following formula (4), and calculating a second probability density function that approximates a histogram of the calculated matching rates; The method includes a threshold setting step of setting the threshold so that the probability of the first probability density function in a range equal to or greater than the threshold and the probability of the second probability density function in a range equal to or less than the threshold are the same value. The matching rate=(the number of the second feature points that have the estimated geometric relationship with the second feature points that exist in one of the partial regions of one of the two second image data, among the second feature points that exist in the other of the two second image data) / (the number of the second feature points that exist in one of the partial regions of one of the two second image data)

[0093] In this embodiment, the threshold value of the matching rate is set to a value that makes the "probability of misjudging different individuals to be the same individual" equal to the "probability of misjudging the same individual to be different individuals." With this configuration, it is possible to minimize the "probability of misjudging different individuals to be the same individual" and the "probability of misjudging the same individual to be different individuals." Therefore, it is possible to improve the accuracy of identification.

[0094] In the individual identification system 10 according to this embodiment, the first processing device (processing device 122) multiplies the match rate calculated by the formula (2) by a weighting coefficient that increases as the match rate increases, and approximates the first probability density function (probability density function of a β distribution) and the second probability density function (probability density function of a β distribution) from a histogram of the match rate multiplied by the weighting coefficient; The second processing device (processing device 122) multiplies the matching rate calculated by the equation (1) by the weighting coefficient, and if there is a registered individual whose matching rate multiplied by the weighting coefficient is greater than or equal to a threshold value, it determines that the individual to be identified is the same individual as the individual whose matching rate multiplied by the weighting coefficient is the largest among the registered individuals whose matching rate multiplied by the weighting coefficient is greater than or equal to the threshold value.

[0095] In the individual identification method according to the present embodiment, In the threshold setting step, the matching rate calculated by the formula (4) is multiplied by a weighting coefficient that increases as the matching rate increases, and the first probability density function and the second probability density function are approximated from a histogram of the matching rate multiplied by the weighting coefficient; In the identification step, the matching rate calculated by the formula (3) is multiplied by the weighting coefficient, and if there is a registered individual whose matching rate multiplied by the weighting coefficient is equal to or greater than a threshold value, it is determined that the individual to be identified is the same individual as the individual whose matching rate multiplied by the weighting coefficient is the highest among the registered individuals whose matching rate multiplied by the weighting coefficient is equal to or greater than the threshold value.

[0096] It has been found that, in the case of the same individual, correct corresponding feature points 215a, 215b are locally abundant. For this reason, by weighting the match rate according to the number of correct corresponding feature points 215a, 215b present in the partial region 213 (i.e., the density of the correct feature points 215a, 215b), it is possible to increase the influence of the match rate of partial regions 213 having a large number of correct corresponding feature points 215a, 215b (which can be referred to as "partial regions 213 having a high density of feature points 212") in identification, and reduce the influence of specific parts having a small number of correct corresponding feature points 215a, 215b.

[0097] the first feature point (feature point 212) and the second feature point are feature points extracted by the A-KAZE method, The geometric relationship is a projective transformation matrix calculated by homography estimation.

[0098] The A-KAZE method is a robust method that is highly resistant to changes in enlargement / reduction, rotation, and lighting. Therefore, even if the imaging conditions (e.g., the position, orientation, and lighting conditions of each individual relative to the imaging device when imaging) when imaging each individual to be registered differ from the imaging conditions when imaging the individual to be identified, it is possible to extract feature points 212 and match feature points with high accuracy.

[0099] Although the embodiments and examples of the present invention have been described above, the technical scope of the present invention is not limited to the above-mentioned embodiments and examples. Various modifications of the present invention are possible without departing from the spirit of the present invention, and such modifications are also included in the technical scope of the present invention.

[0100] For example, in each of the above embodiments, the sun gear of the planetary gear mechanism is shown as the target object, but the target object of the present invention is not limited to the sun gear of the planetary gear mechanism. The target object and target area (area from which features are extracted) of the present invention can be set appropriately. [Explanation of symbols]

[0101] 10: individual identification system, 11: common storage device, 12: terminal device, 20: original image, 21: characteristic image, 121: imaging device, 122: processing device, 211: specific region, 212: characteristic point, 213: partial region

Claims

1. a first imaging device that captures an image of a specific area that is a part of a plurality of individual articles of the same kind and generates first image data that is data of an image in which the specific area is captured for each of the individual articles; a first processing device that extracts, based on the first image data, a first feature point that is a feature point present in the specific region for each of the individuals; a storage device that registers, as a registered individual, each of the individuals from which the first feature points have been extracted by the first processing device, together with information on the first feature points; a second imaging device that captures an image of the specific area of ​​an identification target individual that is one of the articles of the same kind, and generates second image data that is data of an image showing the specific area; a second processing device that extracts second feature points, which are feature points present in the specific region of the individual to be identified based on the second image data, and determines whether the individual to be identified is identical to any of the multiple registered individuals based on information of the extracted second feature points and information of the first feature points of the multiple registered individuals registered in the storage device; Equipped with The second processing device estimates a geometric relationship between the second feature points and the first feature points, sets a plurality of partial areas which are parts of the specific area of ​​the second image data, calculates a match rate between each of the registered individuals and the individual to be identified for each of the set partial areas using the following formula (1), and if there is a registered individual for which the match rate is equal to or greater than a threshold, determines that the individual to be identified is the same individual as the registered individual for which the match rate is the highest among the registered individuals for which the match rate is equal to or greater than the threshold. The matching rate=(the number of the feature points among the second feature points that have the estimated geometric relationship with the first feature points) / (the number of the second feature points included in one of the partial regions set in the specific region of the second image data) Formula (1)

2. 2. The individual identification system according to claim 1, The first processing device includes: estimating the geometric relationship between the first feature points present in each of the same two first image data captured for one of the plurality of individuals from which the first feature points have been extracted, setting a plurality of partial regions in the specific region of the first image data for the one individual, calculating the matching rate of the same individual for each of the set partial regions by the following formula (2), and calculating a first probability density function that approximates a histogram of the calculated matching rates; estimating the geometric relationship between the first feature points present in two different first image data in which two different individuals among the plurality of individuals from which the first feature points have been extracted are captured, setting a plurality of the partial regions in one of the two first image data, calculating the matching rate of the different individuals for each of the set partial regions by the following formula (2), and calculating a second probability density function that approximates a histogram of the calculated matching rates; an individual identification system, wherein the threshold is set so that a probability in a range above the threshold of the first probability density function and a probability in a range below the threshold of the second probability density function are the same value. The matching rate=(the number of the second feature points that have the estimated geometric relationship with the second feature points that exist in one of the partial regions of one of the two second image data, among the second feature points that exist in the other of the two second image data) / (the number of the second feature points that exist in the one of the partial regions of the one of the two second image data)

3. 3. The individual identification system according to claim 2, the first processing device multiplies the match rate calculated by the formula (2) by a weighting coefficient that increases as the match rate increases, and approximates the first probability density function and the second probability density function from a histogram of the match rates multiplied by the weighting coefficient; The second processing device multiplies the match rate calculated by the formula (1) by the weighting coefficient, and if there is a registered individual whose match rate multiplied by the weighting coefficient is greater than or equal to the threshold, it determines that the individual to be identified is the same individual as the individual whose match rate multiplied by the weighting coefficient is the highest among the registered individuals whose match rate multiplied by the weighting coefficient is greater than or equal to the threshold.

4. The individual identification system according to any one of claims 1 to 3, the first feature point and the second feature point are feature points extracted by an A-KAZE method, An individual identification system, wherein the geometric relationship is a projective transformation matrix calculated by homography estimation.

5. An individual identification method executed by an individual identification system, comprising: a first image data generating step of capturing an image of a specific area that is a part of a plurality of individual articles of the same kind to generate first image data that is data of an image in which the specific area is captured for each of the individual articles; a first feature point extraction step of extracting first feature points that are feature points present in the specific region of each of the individuals based on the first image data; a registration step of registering each of the individuals from which the first feature points have been extracted as the registered individuals together with information on the first feature points; a second image data generating step of capturing an image of the specific area of ​​an identification target individual, which is one of the articles of the same kind, and generating second image data that is data of an image showing the specific area; an identification step of extracting second feature points, which are feature points present in the specific region of the individual to be identified based on the second image data, and determining whether the individual to be identified is identical to any one of the multiple registered individuals based on information on the extracted second feature points and information on the first feature points of the registered individuals; Equipped with The identification step is an individual identification method comprising the steps of: estimating a geometric relationship between the second feature points and the first feature points; setting a plurality of partial areas that are parts of the specific area of ​​the second image data, and calculating a match rate between each of the registered individuals and the individual to be identified for each of the set partial areas using the following formula (3); and, if there is a registered individual for which the match rate is equal to or greater than a threshold, determining that the individual to be identified is the same individual as the individual with the highest match rate among the registered individuals for which the match rate is equal to or greater than the threshold. The matching rate=(the number of the feature points among the second feature points that have the estimated geometric relationship with the first feature points) / (the number of the second feature points included in one of the partial regions set in the specific region of the second image data) Formula (3)

6. The individual identification method according to claim 5, estimating the geometric relationship between the first feature points present in each of two identical first image data captured for one of the individuals from which the first feature points have been extracted, setting a plurality of partial regions in the specific region of the first image data for the one individual, calculating the matching rate of the same individual for each of the set partial regions by the following formula (2), and calculating a first probability density function that approximates a histogram of the calculated matching rates; estimating the geometric relationship between the first feature points present in two different first image data in which two different individuals among the plurality of individuals from which the first feature points have been extracted are captured, setting a plurality of partial regions in one of the two first image data, calculating the matching rate of the different individuals for each of the set partial regions by the following formula (4), and calculating a second probability density function that approximates a histogram of the calculated matching rates; An individual identification method comprising a threshold setting step of setting the threshold so that a probability in a range above the threshold of the first probability density function and a probability in a range below the threshold of the second probability density function are the same value. The matching rate=(the number of the second feature points that have the estimated geometric relationship with the second feature points that exist in one of the partial regions of one of the two second image data, among the second feature points that exist in the other of the two second image data) / (the number of the second feature points that exist in the one of the partial regions of the one of the two second image data)

7. The individual identification method according to claim 6, In the threshold setting step, the matching rate calculated by the formula (4) is multiplied by a weighting coefficient that increases as the matching rate increases, and the first probability density function and the second probability density function are approximated from a histogram of the matching rate multiplied by the weighting coefficient; In the identification step, the matching rate calculated by the formula (3) is multiplied by the weighting coefficient, and if there is a registered individual whose matching rate multiplied by the weighting coefficient is greater than or equal to the threshold, it is determined that the individual to be identified is the same individual as the individual whose matching rate multiplied by the weighting coefficient is the highest among the registered individuals whose matching rate multiplied by the weighting coefficient is greater than or equal to the threshold.

8. The individual identification method according to any one of claims 5 to 7, the first feature point and the second feature point are feature points extracted by an A-KAZE method, The individual identification method, wherein the geometric relationship is a projective transformation matrix calculated by homography estimation.

Images