A method for identifying individual animals, and a computer program for doing so.

Abstract

This technology provides a method for individual identification using features other than nose prints. [Solution] The disclosed method includes: (a) acquiring a target image relating to the back of the animal to be identified; (b) obtaining an embedding vector from the target image using a deep distance learning model; (c) calculating the distance between the registered embedding vectors using registration data that includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) determining whether the animal to be identified is one of the multiple registered individuals that is a judgment registered individual, using the distance; and (e) adding the embedding vector relating to the animal to be identified to the registration data if the confidence level of the determination determined according to the distance is equal to or greater than a confidence threshold.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a method for performing individual identification of animals and a computer program. BACKGROUND ART

[0002] Patent Document 1 discloses a technique for collating muzzle patterns of cattle. In this conventional technique, a muzzle pattern image is extracted from a facial image of a cow, a feature vector of the muzzle pattern image is acquired using a neural network for muzzle pattern image classification, and the muzzle pattern image is collated by calculating the similarity between the feature vector and a known feature vector. PRIOR ART DOCUMENT PATENT DOCUMENT

[0003] Patent Document 1 Japanese Unexamined Patent Publication No. 2022-48464 SUMMARY OF THE INVENTION PROBLEM TO BE SOLVED BY THE INVENTION

[0004] However, the conventional technique requires a high-resolution facial image that allows identification of the fine structure of the muzzle pattern, and there has been a problem that it is difficult to acquire such a facial image. For example, for cattle that are walking continuously, it is difficult to stop the movement of the cattle for photography. Such a problem is not limited to cattle, and has been a problem common to individual identification of other animals such as pigs. Therefore, a technique capable of performing individual identification using features other than muzzle patterns is desired. MEANS FOR SOLVING THE PROBLEM

[0005] A first embodiment of this disclosure provides a method for individual identification of an animal to be identified. This method includes: (a) acquiring a target image relating to the back of the animal to be identified; (b) obtaining an embedding vector from the target image using a deep distance learning model; (c) calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) determining whether the animal to be identified corresponds to one of the plurality of registered individuals using the distance; and (e) adding the embedding vector relating to the animal to be identified to the registration data if the confidence level of the determination, determined according to the distance, is equal to or greater than a confidence threshold.

[0006] A second embodiment of this disclosure provides a computer program that causes a processor to perform a process for individual identification of an animal to be identified. This computer program causes the processor to perform the following: (a) a process for acquiring a target image relating to the back of the animal to be identified; (b) a process for obtaining an embedding vector from the target image using a deep distance learning model; (c) a process for calculating the distance between the registered embedding vector and the embedding vector using registration data that includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) a process for determining whether the animal to be identified corresponds to one of the plurality of registered individuals using the distance; and (e) a process for adding the embedding vector relating to the animal to be identified to the registration data if the confidence level of the determination, determined according to the distance, is equal to or greater than a confidence threshold. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing the configuration of the individual identification system of the first embodiment. [Figure 2] A flowchart illustrating the steps for training a deep metric learning model and creating registration data. [Figure 3] A flowchart showing the detailed steps for step S20. [Figure 4] This diagram illustrates the process of generating the target image in step S20. [Figure 5] An explanatory diagram showing an example of registered data. [Figure 6] A flowchart illustrating the procedure for individual identification processing in the first embodiment. [Figure 7] An explanatory diagram showing an example of an embedding vector calculated during individual identification processing. [Figure 8] An explanatory diagram showing an example of the distance between an embedding vector and a registered embedding vector. [Figure 9] A flowchart showing the detailed procedure for step S170 in the first embodiment. [Figure 10] A flowchart showing the procedure for deleting registered data. [Figure 11] A block diagram showing the configuration of the individual identification system of the second embodiment. [Figure 12] An explanatory diagram showing an example of historical data in the second embodiment. [Figure 13] A flowchart showing the detailed procedure for step S170 in the second embodiment. [Figure 14] A flowchart showing the detailed procedure for step S170 in the third embodiment. [Figure 15] A block diagram showing the configuration of the individual identification system of the fourth embodiment. [Figure 16] An explanatory diagram showing an example of registered data in the fourth embodiment. [Figure 17] A flowchart illustrating the procedure for individual identification processing in the fourth embodiment. [Figure 18] An explanatory diagram showing an example of an embedding vector in the fourth embodiment. [Figure 19] An explanatory diagram showing an example of the distance between the embedding vector and the registered embedding vector in the fourth embodiment. [Figure 20] An explanatory diagram showing examples of various determination distances in the fourth embodiment. [Figure 21] A flowchart showing the detailed procedure for step S150a in the fourth embodiment. [Figure 22]A flowchart showing the detailed procedure of step S150a in the fifth embodiment. [Figure 23] A flowchart showing the detailed procedure of step S150a in the sixth embodiment. MODES FOR CARRYING OUT THE INVENTION

[0008] A. First Embodiment: FIG. 1 is an explanatory diagram showing the configuration of an individual identification system according to the first embodiment. This individual identification system includes an information processing device 300 and an image sensor 400. In the present embodiment, the processing target of individual identification is a cow CW. However, instead of cows, other animals such as pigs, dogs, and cats may be used as processing targets.

[0009] The image sensor 400 is a camera that captures an image of the cow CW, which is the target of individual identification processing. The image sensor 400 is preferably installed so as to capture the back of the cow CW from above the cow CW. As the image sensor 400, a video camera that captures moving images may be used, or a still camera that captures still images may be used. Further, as the image sensor 400, various sensors exemplified below can be used. (1) Depth sensor When a depth image captured by a depth sensor is used, individual identification can be performed from the unevenness of the back of a cow. (2) RGB sensor (color image sensor) When a color image captured by an RGB sensor is used, individual identification can be performed from the black-and-white pattern, which is the mottled pattern on the back of a cow. If there is dirt on the back of the cow, the color image changes, so performing individual identification using only an RGB sensor may cause misrecognition. On the other hand, a depth image captured by a depth sensor is not affected by dirt on the back of the cow, so it has the advantage of a low possibility of misrecognition due to dirt. (3) Spectral sensor When a spectral image captured by a spectral sensor is used, individual identification can be performed from the black-and-white pattern, which is the mottled pattern for each wavelength on the back of a cow. (4) Thermosensor By using thermal images captured with a thermal sensor, it is possible to identify individual cows based on the infrared intensity distribution on their backs. The infrared intensity distribution on the cow's back reflects how the degree of infrared scattering changes depending on the length of the hair, and how the degree of infrared diffusion changes depending on the distance due to unevenness.

[0010] The information processing device 300 performs individual identification of cattle CW using an image of the dorsal side of the cattle CW captured by the image sensor 400. The information processing device 300 includes a processor 310, a memory 320, an interface circuit 330, and an input device 340 and a display device 350 connected to the interface circuit 330. The image sensor 400 is also connected to the interface circuit 330. The processor 310 not only has the function of performing the processing detailed below, but also has the function of displaying the data obtained by the processing and the data generated in the process of the processing on the display device 350. The information processing device 300 can be implemented by a computer such as a personal computer.

[0011] The processor 310 has the functions of a target image acquisition unit 510, a learning unit 520, and an individual identification unit 530. The target image acquisition unit 510 acquires multiple target images from the back image captured by the image sensor 400. The learning unit 520 performs distance learning of the deep distance learning model 620. The individual identification unit 530 performs individual identification of cattle CW using the trained deep distance learning model 620. The functions of each of these units are realized by the processor 310 executing a computer program stored in the memory 320. However, some of these functions may be realized by hardware circuits. The term "processor" in this disclosure includes such hardware circuits. Furthermore, one or more processors that perform various processes may be processors included in one or more remote computers connected via a network.

[0012] The distance used for training in the deep metric learning model 620 can also be similarity. A high similarity score corresponds to a small distance. In other words, "small distance" is equivalent to "high similarity." In the following explanation, "distance" is used as a term that includes "similarity."

[0013] Memory 320 stores the object recognition model 610, the deep distance learning model 620, and the registered data 630. The object recognition model 610 is a machine learning model that takes an image of a cow CW as input and outputs multiple keypoints, which are the morphological feature points of the cow CW. In this embodiment, it is assumed that the object recognition model 610 has been trained. The object recognition model 610 can also be called a "feature recognition model".

[0014] The deep metric learning model 620 is a machine learning model that takes images of cow CW as input and outputs embedding vectors. These embedding vectors are also called "feature vectors." The deep metric learning model 620 can be constructed, for example, using FaceNet.

[0015] The registration data 630 is a database in which, for each of several registered individuals, the individual ID and multiple embedding vectors obtained using the deep distance learning model 620 are registered. The individual ID is, for example, the individual identification number displayed on the ear tag of a cow CW. The embedding vectors registered in the registration data 630 are called "registered embedding vectors". In addition to registered embedding vectors, the registration data 630 may also include images of each registered individual.

[0016] Figure 2 is a flowchart showing the procedure for training the deep distance learning model 620 and creating the registration data 630. In the following description, we assume that n is an integer greater than or equal to 2, and that n target cows are registered in the registration data 630. In this embodiment, the n target cows are of the same breed.

[0017] In step S10, the target image acquisition unit 510 uses the image sensor 400 to capture multiple dorsal images of each of the n target cows to be registered and the k target cows to be trained. Here, n and k are integers of 2 or greater. This capture can be performed, for example, as each cow sequentially passes below the image sensor 400. If a video is captured, multiple frame images are selected from the video as dorsal images. This selection can be performed, for example, by automatically selecting frame images that include the rear portion of the target cows to be registered using an annotation tool. Alternatively, the operator may make the selection manually.

[0018] In step S20, the target image acquisition unit 510 acquires multiple target images of the backs of each of the n registered target cows and k training target cows from the back images captured by the image sensor 400. "Training target cows" refers to cows used to create training data for the deep distance learning model 620. In deep distance learning, from the viewpoint of robustness, it is desirable to train the neural network using training data different from the registered data 630. It is also desirable to create training data that contains more data than the registered data 630. Therefore, it is preferable that k be set to a number greater than n. However, some of the training target cows may be registered cows. Alternatively, n=k may be used, and the same cow may be used as both the training target cows and the registered target cows. The number of target images acquired from each cow may be set to a constant value N, or it may be set to a different number for each individual. Here, N is an integer of 1 or more, but it is preferable that it be 2 or more.

[0019] Figure 3 is a flowchart showing the detailed procedure of step S20, and Figure 4 is an explanatory diagram showing the processing content. In the first embodiment, a depth sensor is used as the image sensor 400. Some of the step numbers are included in Figure 4.

[0020] The dorsal image BG shown in Figure 4 includes the posterior portion of the cattle coww, including the lumbar horns and the base of the tail. Since individual cattle cowws have distinctive features in the shape of their posterior portion, it is preferable that the dorsal image BG includes the posterior portion of the cattle coww, including the lumbar horns and the base of the tail.

[0021] In step S21, the target image acquisition unit 510 selects one cow to be processed from n registered cows and k training cows. In step S22, the target image acquisition unit 510 selects one back image BG to be processed from multiple back images BG. In step S23, the target image acquisition unit 510 creates a feature detection image CG by performing a grayscale conversion process on the back image BG. This grayscale conversion process extracts a grayscale range from the total grayscale of the back image BG that makes it easy to detect the contour of the cow, and converts it into an image with a predetermined number of grayscales. For example, if the back image BG, which is a depth image, has 2400 grayscales, the feature detection image CG may be created by extracting 512 intermediate grayscales and compressing them to 256 grayscales. However, step S23 is optional.

[0022] In step S24, the target image acquisition unit 510 uses the object recognition model 610 to detect multiple keypoints from the feature detection image CG. In the example in Figure 4, two keypoints KP1 and KP2 are detected at the hip horns of the cow CW, and one keypoint KP3 is detected at the base of the tail. These keypoints KP1 to KP3 are feature points present on the outline of the cow CW. The object recognition model 610 is a machine learning model that takes the feature detection image CG as input and outputs multiple keypoints KP1 to KP3. However, the object recognition model 610 may be configured to recognize keypoints indicating other locations. The object recognition model 610 can be configured using, for example, a deep learning model such as DeepLabCut or ResNet. The object recognition model 610 may also be configured using other machine learning models.

[0023] In step S25, the target image acquisition unit 510 determines whether a predetermined number of keypoints KP1 to KP3 have been detected. If the predetermined number of keypoints KP1 to KP3 have not been detected, the process returns to step S22, a new back image BG is selected as the processing target, and the processing from step S23 onwards is executed again. If the predetermined number of keypoints KP1 to KP3 have been detected, the process proceeds to step S26.

[0024] In step S26, the target image acquisition unit 510 extracts the target image TG from the back image BG based on a plurality of key points KP1 to KP3. The target image TG is an image that includes the characteristic parts of the cow's back. In the example in Figure 4, the back image BG is shown with a cropping frame CF set based on key points KP1 to KP3, and the target image TG extracted according to the cropping frame CF is shown. At this time, it is preferable to rotate and resize the target image TG so that the direction and size of the cow CW's hip horns within the target image TG become the desired values. Alternatively, the pixel values ​​of the target image TG may be normalized. This normalization is, for example, normalizing the pixel value of the minimum depth value to 1.0 and the pixel value of the maximum depth value to 0, and further converting the pixel values ​​in the range of 0 to 1.0 to a grayscale of 0 to 255. Steps S23 to S26 may be omitted, and the back image BG may be used as the target image TG as is.

[0025] In step S27, the target image acquisition unit 510 determines whether or not N target images TG have already been created for a single cow. If N target images TG have not been created, the process returns to step S22, a new back image BG is selected as the processing target, and the processing from step S22 onward is executed again. If N target images TG have already been created, the process proceeds to step S28.

[0026] In step S28, the target image acquisition unit 510 determines whether processing has been completed for all n registered target cows and k training target cows. If processing is not completed for all cows, the process returns to step S21, a new cow is selected as the target for processing, and the processing from step S22 onwards is executed again. If processing is completed for all cows, the process in step S20 is terminated.

[0027] In step S30 of Figure 2, the learning unit 520 creates distance learning data for deep distance learning by associating N target images TG acquired for each individual cow with an individual ID. The individual ID is usually entered by the user. When the user determines the individual ID of a cow, it is preferable to display a color image of the cow, captured using an RGB sensor, on the display device 350, as it is easier to identify cows using a color image than a depth image.

[0028] In step S40, the learning unit 520 performs distance learning on the deep distance learning model 620 using the distance learning data. This distance learning is a process of adjusting the internal parameters of the deep distance learning model 620 so that the distance between the same individual and the distance between different individuals are small for the embedding vectors output from the deep distance learning model 620. For example, the distance between the embedding vectors may be the Euclidean distance between the vectors or the angle between the vectors.

[0029] In step S50, the learning unit 520 sequentially inputs N target images TG for each registered cow into the trained deep distance learning model 620 and obtains an embedding vector for each target image TG. As a result, N embedding vectors are obtained for each registered cow. In step S60, the learning unit 520 creates registration data 630 for each of the n registered cows by associating the N embedding vectors with individual IDs.

[0030] In step S70, the learning unit 520 determines whether processing has been completed for all back images captured in step S10. If processing is complete, the process shown in Figure 2 is terminated. If processing is not complete, the process returns to step S20, and steps S20 to S70 are repeated for the next back image.

[0031] By performing the learning process shown in Figure 2 above, a trained deep distance learning model 620 and registration data 630 for n target cows are obtained. In the following explanation, the target cows will be referred to as "registered individuals." Also, the embedding vectors registered in the registration data 630 will be referred to as "registration embedding vectors."

[0032] Figure 5 is an explanatory diagram showing an example of registration data 630. In this example, the number of registered cows n is 4, and the number of target images TG N is 3. Registration data 630 contains registration embedding vectors Vr for each of the four registered individuals, along with their registration dates, for each of the three target images TG. In this example, the registration embedding vector Vr is a 5-dimensional vector with 5 elements. The individual IDs of the four registered individuals are ID1 to ID4. In addition, the three target images TG obtained for each individual are assigned different image IDs. However, image IDs do not necessarily have to be registered. Normally, a larger number of registration embedding vectors Vr are registered for each registered individual, but in Figure 5, the number of registration embedding vectors Vr is reduced for illustrative purposes.

[0033] Figure 6 is a flowchart showing the procedure for individual identification processing in the first embodiment. The individual identification processing is preferably performed periodically, for example, on all cattle raised in the same farm as the cattle to be identified.

[0034] In step S110, the individual identification unit 530 acquires N1 target images TG of the back of one target cow using the image sensor 400. Here, N1 is an integer of 1 or more. N1 may be set to a value equal to the number of registered embedding vectors for each registered individual registered in the registration data 630, or it may be set to a different value. Also, N1 may not be a pre-set value, but the number of target images TG that have been acquired for the target cow at that time. In the following description, the number of registered embedding vectors for each registered individual will be referred to as "N2" and distinguished from the number of target images TG N1 of the target cow. The integer N2 is the same as the integer N used in Figures 2 and 3, and is an integer of 2 or more. In the example in Figure 5, the number of registered embedding vectors N2 is 3. The specific processing content of step S110 is the same as the target image TG creation process for registered cows described in Figures 3 and 4.

[0035] In step S120, the individual identification unit 530 uses the trained deep distance learning model 620 to obtain N1 embedding vectors for N1 target images TG.

[0036] Figure 7 is an explanatory diagram showing an example of an embedding vector calculated in the individual identification process. In this example, the number of target images TG N1 is 3. That is, the embedding vector Vt is calculated for each of the three target images TG obtained for the cow to be identified. The embedding vector Vt is a vector of the same dimension as the registered embedding vector Vr shown in Figure 5.

[0037] In step S130, the individual identification unit 530 calculates the distance between N1 embedding vectors and N2 registration embedding vectors for each registered individual. As a result, N1 × N2 distances are calculated for each registered individual. Since there are n registered individuals, n × N1 × N2 distances are calculated for each identified cow.

[0038] Figure 8 is an explanatory diagram illustrating an example of the distance between an embedding vector and a registered embedding vector. For the registered embedding vector Vr shown in Figure 5, n=4 and N2=3, and for the embedding vector Vt shown in Figure 7, N1=3, so 36 distances L are calculated for one identifiable cow.

[0039] In step S140, the individual identification unit 530 determines a determination distance Lj(ID) that represents N1 × N2 distances for each of the n registered individuals. The "ID" in the code of the determination distance Lj(ID) means the individual ID of the registered individual. The determination distance Lj(ID) is determined by one of the following methods, for example.

[0040] <Method for determining the judgment distance Lj(ID) JM1> For each registered individual, M distances L are selected in ascending order from N1 × N2 distances L, and a value proportional to the average of these M distances L is determined as the determination distance Lj(ID). Here, M is an integer greater than or equal to 2 and less than or equal to N1 × N2, and preferably less than N1 × N2. It is also preferable that integers N1 and N2 are set such that N1 × N2 is 3 or greater. In this disclosure, a value proportional to a specific value means a value obtained by multiplying the specific value by a positive coefficient, and may also be the specific value itself.

[0041] <Method for determining the judgment distance Lj(ID) JM2> For each registered individual, the determination distance Lj(ID) is determined to be a value proportional to the minimum value among N1 × N2 distances L.

[0042] In the first embodiment, the determination distance Lj(ID) is determined according to the determination method JM1 described above. Figure 8 above shows the determination distance Lj(ID) calculated using M=3 for each registered individual.

[0043] In step S150 of Figure 6, the individual identification unit 530 uses the determination distance Lj(ID) to determine whether the registration condition is met or the unregistered condition is met. The "registration condition" is a condition indicating that the cow to be identified corresponds to one of several registered individuals registered in the registration data 630. The "unregistered condition" is a condition indicating that the cow to be identified is not registered in the registration data 630. Specifically, in step S150, if the minimum value of the determination distance Lj(ID) is smaller than the predetermined determination threshold Th1, the registration condition is considered to be met, and the process proceeds to step S160. In step S160, the individual identification unit 530 identifies the registered individual corresponding to the minimum value of the determination distance Lj(ID) as the cow to be identified. In the example in Figure 8, the minimum value of the determination distance Lj(ID) corresponds to the registered individual whose individual ID is ID3, so this registered individual is identified as the cow to be identified. In the determination process in steps S150 and S160, registered individuals identified as belonging to the group of cattle to be identified are called "determined registered individuals." After step S160, the registration data update process is performed in step S170, which will be described later.

[0044] On the other hand, if the minimum value of the determination distance Lj(ID) is greater than the determination threshold Th1, the unregistered condition is considered to be met, and the process proceeds to step S180. The minimum value of the determination distance Lj(ID) becomes greater than the determination threshold Th1 when the cow to be identified is an unregistered individual, or when there is dirt on the back of the cow to be identified. In this embodiment, in step S180, the individual identification unit 530 determines that the cow to be identified is an unregistered individual.

[0045] Furthermore, if the minimum value of the judgment distance Lj(ID) is equal to the judgment threshold Th1, the process proceeds to one of the two pre-selected branch destinations from step S150. This is also true for other judgment steps that use thresholds.

[0046] In step S150, it is also possible to determine which registered individual the target cow belongs to, or whether it is an unregistered individual, using conditions different from those described above. That is, if N1 × N2 distances satisfy the pre-set unregistered condition, it can be determined that the target cow is an unregistered individual. Examples of other unregistered conditions will be described in the second embodiment and subsequent embodiments.

[0047] Figure 9 is a flowchart showing the detailed procedure of step S170 in the first embodiment. In step S171, the individual identification unit 530 determines whether the minimum value of the determination distance Lj(ID) is smaller than a preset second determination threshold Th2. Preferably, the second determination threshold Th2 is smaller than the determination threshold Th1 used in step S150 of Figure 6. If the minimum value of the determination distance Lj(ID) is smaller than the second determination threshold Th2, the confidence level RL of the determination in the individual identification process is considered to be the first level RL1, and the process proceeds to step S172. The first level RL1 means that the confidence level RL of the determination is sufficiently high.

[0048] In step S172, the individual identification unit 530 adds the embedded vector Vt of the cow to be identified to the registration data 630. In step S160 described above, in the example in Figure 8, the cow to be identified is identified as the registered individual whose individual ID is ID3, so the embedded vector Vt of the cow to be identified shown in Figure 7 is added as the registration embedded vector Vr of that registered individual.

[0049] On the other hand, if the minimum value of the judgment distance Lj(ID) is greater than the second judgment threshold Th2, the confidence level RL of the judgment is considered to be the second level RL2, which is lower than the first level RL1, and step S172 is skipped, ending the process shown in Figure 9. The second level RL2 means that the confidence level RL of the judgment is low.

[0050] The second judgment threshold Th2 used in Figure 9 can be considered as being used to determine the confidence level RL of the judgment in the individual identification process. The first level of confidence level RL1 corresponds to a level where the confidence level of the judgment, which is determined according to the distance, is equal to or greater than a preset distance confidence threshold, and the second level RL2 corresponds to a level where the confidence level of the judgment is lower than the distance confidence threshold. Note that it is not necessary to explicitly determine the first level RL1 and the second level RL2 of confidence level RL as processing parameters, and it is acceptable to perform processing that is substantially equivalent to that in Figure 9.

[0051] According to the processing shown in Figures 6 and 9 above, it is possible to identify individual cows from N1 × N2 distances for each registered individual. Furthermore, if the confidence level of the determination, which is determined according to the distance, is equal to or greater than the confidence threshold, an embedded vector Vt is added to the registration data 630, thereby increasing the reliability of the registration data 630.

[0052] Furthermore, when data is added to the registered data 630 according to the processes shown in Figures 6 and 9, the amount of data for each registered individual gradually increases. Therefore, it is preferable to delete a portion of the data in the registered data 630 to prevent it from growing excessively. For example, one of the following methods can be applied to delete data registered in the registered data 630. <Deletion process for registered data D1> If the registration period for a registration embedding vector Vr related to a specific registered individual exceeds a predetermined allowable registration period, the registration embedding vector Vr is deleted from the registration data 630. <Deletion process for registered data D2> If the number of registration embedding vectors Vr for a particular registered individual exceeds a preset allowable number of registrations, some of those registration embedding vectors Vr are deleted from the registration data 630. In this case, it is preferable that the registration embedding vectors Vr be deleted in order of oldest registration date. <Deletion process for registered data D3> If a specific registered individual is not determined to be a determination-registered individual within a predetermined allowable period, the data relating to the determination-registered individual is deleted from the registration data 630. Here, "determination-registered individual" refers to a registered individual that has been identified as a target cow in the determination process in steps S150 and S160 of Figure 6.

[0053] Figure 10 is a flowchart showing the procedure for deleting registered data. This process is an example of the deletion process D1 described above. It is preferable that this process be performed periodically.

[0054] In step S310, the individual identification unit 530 selects one registered individual registered in the registration data 630. In step S320, the individual identification unit 530 obtains the registration period for each registration embedding vector Vr related to the selected registered individual. The "registration period" is the elapsed time from the time the registration embedding vector Vr was registered in the registration data 630 until the time the process in Figure 10 is executed. In this embodiment, the registration period is the number of days elapsed from the registration date shown in Figure 5.

[0055] In step S330, the individual identification unit 530 determines whether the registration period of the registration embedding vector Vr exceeds a preset allowable registration period. The allowable registration period is the period during which the registration embedding vector Vr is allowed to remain registered in the registration data 630. The reason for setting an allowable registration period is that without such a period, there is a possibility that the number of old registration embedding vectors Vr will increase excessively.

[0056] In step S330, if the registration period of the registration embedding vector Vr exceeds the allowable registration period, the process proceeds to step S340, where the individual identification unit 530 deletes the registration embedding vector Vr from the registration data 630. On the other hand, if the registration period of the registration embedding vector Vr does not exceed the allowable registration period, step S340 is skipped and the process proceeds to step S350. In step S350, the individual identification unit 530 determines whether the processing in steps S310 to S340 has been completed for all registered individuals registered in the registration data 630. If the processing has not been completed, the process returns to step S310, and the processing in steps S310 to S350 is executed again. According to the deletion process in Figure 10, unnecessary registration embedding vectors Vr can be deleted from the registration data 630.

[0057] According to the first embodiment described above, an embedding vector Vt can be obtained from a target image of the back of the animal to be identified using the deep distance learning model 620, and the individual can be identified using the distance between this embedding vector Vt and the registered embedding vector Vr. Furthermore, if the confidence level of the individual identification determination is above the confidence threshold, the embedding vector Vt is added to the registered data 630, thereby increasing the reliability of the registered data 630. For example, it is possible to update the registered contents of the registered data 630 in accordance with changes over time, such as the body shape of the target cow to be identified.

[0058] B. Second Embodiment: Figure 11 is an explanatory diagram showing the configuration of the individual identification system in the second embodiment. The individual identification system of the second embodiment differs from the first embodiment in that history data 640 is added to the memory 320, but the other configurations are the same as those of the first embodiment. Also, the processing in Figures 2 and 3 is the same as that of the first embodiment. The individual identification processing in Figure 6 is also almost the same as that of the first embodiment, but as will be described later, the detailed procedure of step S170 differs from that of the first embodiment.

[0059] The history data 640 is data in which the history of each of the multiple registered individuals registered in the registration data 630 has been determined to be a registered individual for determination during the individual identification process. As mentioned above, a "registered individual for determination" is a registered individual that has been identified as corresponding to an identifiable cattle in the determination process in steps S150 and S160 of Figure 6.

[0060] Figure 12 shows an example of history data 640 in the second embodiment. The history data 640 includes a history registration number, individual ID, history ID, history judgment count Nc1, history match count Nc2, image ID, registration date, and history embedding vector Vh. The history embedding vector Vh is the same as the embedding vector Vt of the cow to be identified.

[0061] The history registration number is a number that identifies each individual registered in the history database. In the example in Figure 12, data for three registered individuals with history registration numbers #1 to #3 is registered in history data 640. One history registration number is assigned for each combination of registered individual ID and history ID. For example, the registered individual with history registration number #1 has a registered individual ID of ID1 and a history ID of HID1. Similarly, the registered individual with history registration number #2 has a registered individual ID of ID1 and a history ID of HID2.

[0062] The history ID is an identifier assigned when data for a target cow is newly registered in history data 640 if it is determined that data corresponding to that cow does not exist in history data 640. The history ID is assigned to the registered individual ID of a registered individual that has been determined to be a target cow during the individual identification process. One or more history IDs can be set for a single registration ID. The method for setting history IDs will be described later.

[0063] The history determination count Nc1 is the number of times it was determined that the registered individual ID of a registered individual that was determined to be a target cow for identification was already registered in the history data 640. The history match count Nc2 is the number of times it was determined that the data concerning the target cow matched the data concerning the registered individual in the history data 640. The method for determining counts Nc1 and Nc2 will be described later.

[0064] For a registered individual with history registration number #1, the history judgment count Nc1 is 5, and the history match count Nc2 is 4. Similarly, for a registered individual with history registration number #2, the history judgment count Nc1 is 5, and the history match count Nc2 is 1. For a registered individual with history registration number #3, both the history judgment count Nc1 and the history match count Nc2 are 0. When a registered individual is first registered in history data 640, Nc1=0 and Nc2=0 are set.

[0065] In the example in Figure 12, for each registered individual, N1 (=3) embedding vectors used in a single individual identification process are registered as a representative history embedding vector Vh. However, each time the history of a registered individual is determined, the N1 embedding vectors used at that time may be added and registered as a new history embedding vector Vh.

[0066] Figure 13 is a flowchart showing the detailed procedure for step S170 in the second embodiment. In step S211, the individual identification unit 530 determines whether the registered individual ID of the individual determined to be a target cow in step S160 of Figure 6 is already registered in the history data 640. If the registered individual ID of the individual determined to be a target cow is not already registered in the history data 640, the process proceeds to step S218.

[0067] In step S218, the individual identification unit 530 registers new data about the cattle to be identified into the history data 640. When step S170 in Figure 13 is executed for the first time, the history data 640 has not been created, so the process proceeds to step S218 and new registration is performed. The registered individual ID of the judged registered individual is registered as data for the newly registered history registered individual, and a new history ID is assigned. In addition, the history judgment count Nc1 and the history match count Nc2 are both set to zero.

[0068] In step S211, if the registered individual ID of the individual to be judged is already registered in the history data 640, proceed to step S212.

[0069] The processing from step S212 onward involves adding data about the registered individual to the history data 640 or transferring data from the history data 640 to the registration data 630. This processing is performed for one or more registered individuals that have the same registered individual ID as the cow to be identified. In step S212, the individual identification unit 530 increments the history determination count Nc1 of the registered individuals that have the same registered individual ID as the determined registered individual. For example, if the registered individual ID of the determined registered individual is ID1, the history determination count Nc1 is incremented for the two registered individuals whose history registration numbers are #1 and #2 as shown in Figure 12.

[0070] In step S213, the individual identification unit 530 calculates the distance between the history embedding vector Vh of the registered individual and the embedding vector Vt of the cow to be identified. For example, when two registered individuals with registered individual ID ID1, as shown in Figure 12, are to be processed, 18 distances are calculated between their six history embedding vectors Vh and the three embedding vectors Vt of the cow to be identified, as shown in Figure 7.

[0071] In step S214, the individual identification unit 530 determines the minimum distance Lmin calculated in step S213.

[0072] In step S215, the individual identification unit 530 determines whether the minimum distance Lmin is less than a preset determination threshold Th3. This determination threshold Th3 is used to determine whether the data concerning the cow to be identified matches the data concerning the registered individual in the history. It is preferable that this value be smaller than the determination threshold Th1 used in step S150 in Figure 6. If the minimum distance Lmin is less than the determination threshold Th3, the cow to be identified is considered to match the registered individual in the history, and the process proceeds to step S216.

[0073] In step S216, the individual identification unit 530 increments the history match count Nc2 of the history registered individual that matches the cow to be identified by one, and proceeds to step S219, which will be described later. The individuals subject to this increment are the history registered individuals for which the history registered vector Vh that gives the minimum distance Lmin calculated in step S213 is registered.

[0074] On the other hand, if the minimum distance Lmin is greater than the judgment threshold Th3, the cow to be identified is considered not to match the registered history individual, and the process proceeds to step S217. In step S217, the cow to be identified is added to the history data 640 as a new registered history individual. The registered individual ID of the added registered history individual is set to the same ID as the judged registered individual that was determined to be the cow to be identified, and a new history ID is assigned. In addition, the history judgment count Nc1 is set to the value counted up in step S212, and the history match count Nc2 is set to zero.

[0075] In step S219, the individual identification unit 530 determines whether the history judgment count Nc1 is equal to or greater than a preset history judgment count threshold Tc1. The history judgment count threshold Tc1 is preferably set to a value of 2 or more. If the history judgment count Nc1 is equal to or greater than the history judgment count threshold Tc1, the confidence level RL of the judgment is considered to be the first level RL1, and the process proceeds to step S220, which will be described later. On the other hand, if the history judgment count Nc1 is less than the history judgment count threshold Tc1, the confidence level RL of the judgment is considered to be the second level RL2, and the process in Figure 13 is terminated. As explained in Figure 9 of the first embodiment, the first level RL1 of the confidence level RL corresponds to a level in which the confidence level of the judgment in the individual identification process is equal to or greater than a preset confidence threshold, and the second level RL2 corresponds to a level in which the confidence level of the judgment is lower than the confidence threshold.

[0076] In step S220, the individual identification unit 530 determines whether the history match count Nc2 is equal to or greater than a preset history match count threshold Tc2. Preferably, the history match count threshold Tc2 is set to a value of 2 or more and less than or equal to the history determination count threshold Tc1, and also preferably to a value smaller than the history determination count threshold Tc1. If the history match count Nc2 is equal to or greater than the history match count threshold Tc2, the confidence level RL of the determination is maintained at the first level RL1 and the process proceeds to step S221. In step S221, the individual identification unit 530 moves the history embedding vector Vh related to the history registered individual from the history data 640 to the registration data 630. That is, data related to the history registered individual is added to the registration data 630 and deleted from the history data 640.

[0077] On the other hand, if the history match count Nc2 is less than the history match count threshold Tc2, the confidence level RL of the judgment in the individual identification process is considered to be the second level RL2, and the process proceeds to step S222. In step S222, the individual identification unit 530 deletes the data relating to the registered individual from the history data 640. In this way, if the history judgment count Nc1 is greater than or equal to the history judgment count threshold Tc1, and the history match count Nc2 is less than the history match count threshold Tc2, the registered individual is considered unnecessary and deleted from the history data 640. This is because there is a high possibility that the data relating to that registered individual is incorrect. The processing in steps S219 to S222 prevents an excessive number of registered individuals from being registered in the history data 640.

[0078] For example, let's assume that the history judgment count threshold Tc1 is set to 5 and the history match count threshold Tc2 is set to 3. In this case, for the history registered individual with history registration number #1 shown in Figure 12, the history judgment count Nc1 (=5) is greater than or equal to the history judgment count threshold Tc1 (=5), and the history match count Nc2 (=4) is greater than or equal to the history match count threshold Tc2 (=3), so step S221 is executed and the data of that history registered individual is moved from history data 640 to registration data 630. Also, for the history registered individual with history registration number #2, the history judgment count Nc1 (=5) is greater than or equal to the history judgment count threshold Tc1 (=5), and the history match count Nc2 (=1) is less than the history match count threshold Tc2 (=3), so step S222 is executed and the data of that history registered individual is deleted from history data 640.

[0079] It can be understood that in the processes of steps S219 to S222 described above, the history determination count Nc1 and the history matching count Nc2 are used to determine whether the reliability RL of the determination is the first level RL1 or the second level RL2. Specifically, whether the reliability RL of the determination is the first level RL1 or the second level RL2 is determined depending on whether the history determination count Nc1 is greater than or equal to the history determination count threshold Tc1. Further, when the history determination count Nc1 is greater than or equal to the history determination count threshold Tc1, it is further determined whether the history matching count Nc2 is greater than or equal to the history matching count threshold Tc2. That is, when Tc2 ≦ Nc2, the reliability RL of the determination is maintained at the first level RL1, and when Nc2 < Tc2, the reliability RL of the determination is changed to the second level RL2. As a result, appropriate reliability can be determined using the history determination count Nc1 and the history matching count Nc2 of the history data. However, the reliability RL resetting process using the history matching count Nc2 may be omitted.

[0080] The second embodiment also achieves substantially the same effects as the first embodiment. Further, in the second embodiment, the reliability RL of the determination can be determined to an appropriate level using the history data 640.

[0081] It should be noted that unnecessary data may also be deleted from the history data 640 by using the same process as the registered data 630 deletion process described in the first embodiment.

[0082] C. Third Embodiment: FIG. 14 is a flowchart showing the detailed procedure of step S170 in the third embodiment. The configuration of the individual identification system according to the third embodiment is the same as that of the second embodiment. Also, the individual identification processing in FIG. 2, FIG. 3 and FIG. 6 is also substantially the same as that of the second embodiment, and the detailed procedure of step S170 is different from that of the second embodiment.

[0083] The process in Figure 14 is obtained by adding steps S211 to S224 shown in Figure 13 of the second embodiment after step S171 shown in Figure 9 of the first embodiment. According to the third embodiment, the confidence level of the judgment can be set to an appropriate level by combining the method for determining the confidence level of the judgment in step S171 with the method for determining the confidence level of the judgment in steps S219 and S220.

[0084] D. Fourth Embodiment: Figure 15 is an explanatory diagram showing the configuration of the individual identification system in the fourth embodiment. The individual identification system of the fourth embodiment differs from the second embodiment in that it is provided with multiple image sensors 400(p), deep distance learning models 620(p), registered data 630(p), and historical data 640(p), but the other configurations are the same as those of the second embodiment. The 'p' at the end of these symbols is an ordinal number from 1 to P, and P is an integer of 2 or greater.

[0085] Multiple image sensors 400(p) can be combinations of various different image sensors as described in the first embodiment, or combinations of the same type of image sensors. When using combinations of the same type of image sensors, it is preferable to set at least one attribute of the multiple image sensors, such as the installation angle or field of view, to be different from each other. It is preferable that the relative positions of the respective sensor coordinate systems of the multiple image sensors 400(p) are known, and that the coordinate transformation matrices for any two sensor coordinate systems are known.

[0086] The number P of the image sensors 400(p) is an integer of 2 or more. In the fourth embodiment, P=2, and two image sensors 400, a depth sensor and an RGB sensor, are used to capture a video of the back of a cow CW. Preferably, the depth sensor and the RGB sensor have substantially the same shooting area and are configured to capture images at the same shooting timing. For example, it is possible to use one RGBD sensor including a depth sensor and an RGB sensor as the image sensor 400(p).

[0087] In the following explanation, a symbol with (p) appended to the end indicates that it corresponds to the ordinal p of the deep metric learning model 620(p). Similarly, the prefix "kind p" also indicates that it corresponds to the ordinal p of the deep metric learning model 620. For example, the kind p embedding vector registered in the p-th registered data 630(p) is called the "kind p registered embedding vector".

[0088] The learning process described in Figures 2 to 4 in the first embodiment is applied similarly to each individual deep metric learning model 620(p), so its explanation is omitted.

[0089] Furthermore, in the processing shown in Figure 3 for images captured by the RGB sensor, it is possible to detect multiple keypoints KP1 to KP3 from the back image BG captured by the RGB sensor, similar to the processing for images captured by the depth sensor, and set the cropping frame CF based on these keypoints. Alternatively, a coordinate transformation matrix between the depth sensor and the RGB sensor may be used to transform the multiple keypoints KP1 to KP3 detected from the back image BG of the depth sensor into coordinates in the sensor coordinate system of the RGB sensor, and then set the cropping frame CF from the transformed keypoints KP1 to KP3. The latter method is particularly useful when the depth sensor and RGB sensor are configured as a single RGBD sensor.

[0090] Figure 16 is an explanatory diagram showing an example of registration data 630(p) in the fourth embodiment. In this example, two types of image sensors 400(p) are used: a depth sensor and an RGB sensor. The number of target cows n is 4, and the number of target images TG N(p) is N(1)=N(2)=3. The registration data 630(p) includes a first registration data 630(1) created using the depth sensor and a second registration data 630(2) created using the RGB sensor. The first registration data 630(1) is the same as the registration data 630 of the first embodiment shown in Figure 5. However, the registration date is omitted for illustrative purposes. The second registration data 630(2) contains registration embedding vectors Vr(2) for three target images TG for each of the same four individuals as in the first registration data 630(1).

[0091] Figure 17 is a flowchart showing the procedure for individual identification processing in the fourth embodiment. Steps S110a to S160a are modified versions of steps S110 to S160 of the individual identification processing in the first embodiment shown in Figure 6. The same processing as in the first to third embodiments can be applied to steps S170 and S180.

[0092] In step S110a, the individual identification unit 530 uses two image sensors 500(p) to acquire N1 target images TG of the back of one target cow. Here, N1 is an integer of 1 or more. Similar to the first embodiment, the number of p-th type registration embedding vectors for each registered individual is defined as "N2" and distinguished from the number of target images TG of the target cow N1. The integer N2 is an integer of 2 or more. It is preferable that each of the integers N1 and N2 be a constant value independent of the ordinal number p.

[0093] In step S120a, the individual identification unit 530 uses the P trained deep distance learning models 620(p) to obtain N1 p-th kind embedding vectors for N1 target images TG.

[0094] Figure 18 is an explanatory diagram showing an example of an embedding vector in the fourth embodiment. The first type embedding vector Vt(1) is calculated using an image captured with a depth sensor and is the same as the embedding vector Vt shown in Figure 7. The second type embedding vector Vt(2) is calculated using an image captured with an RGB sensor.

[0095] In step S130a, the individual identification unit 530 calculates the distance between N1 p-th species embedding vectors Vt(p) and N2 p-th species registration embedding vectors Vr(p) for each registered individual. As a result, N1 × N2 p-th species distances are calculated for each registered individual. Since there are n registered individuals, n × N1 × N2 p-th species distances are calculated for each identified cow.

[0096] Figure 19 is an explanatory diagram showing an example of the distance between the embedding vector and the registered embedding vector in the fourth embodiment. For the registered embedding vector Vr(p) shown in Figure 16, n=4 and N2=3, and for the embedding vector Vt(p) shown in Figure 18, N1=3, so for one target cow, 36 p-th distances L(p) are calculated from the images captured by each individual image sensor 400(p).

[0097] In step S140a, the individual identification unit 530 determines an integrated determination distance Ljt(ID) for each of the n registered individuals by integrating N1 × N2 p-th type distances. The integrated determination distance Ljt(ID) can be determined by, for example, one of the following methods.

[0098] <Method for determining the integrated judgment distance Ljt(ID) DM1> For each registered individual, the integrated determination distance Ljt(ID) is determined to be a value proportional to the sum of the following: the average value of the first type distance obtained by averaging M first type distances L(1) selected in ascending order from N1 × N2 first type distances L(1), and the average value of the second type distance obtained by averaging M second type distances L(2) selected in ascending order from N1 × N2 second type distances L(2). M is an integer between 2 and N1 × N2, and preferably an integer less than N1 × N2. Furthermore, it is preferable that integers N1 and N2 are set such that N1 × N2 is 3 or greater.

[0099] <Method for determining the integrated judgment distance Ljt(ID) DM2> For each registered individual, N1 × N2 first-type distances L(1) and corresponding N1 × N2 second-type distances L(2) are added together to obtain N1 × N2 summation results. From these N1 × N2 summation results, M summation results are selected in ascending order, and a value proportional to the average value of the selected M summation results is determined as the integrated determination distance Ljt(ID). The term "corresponding" means that the results are obtained using images taken at substantially the same shooting timing using two image sensors 400(p). Specifically, the first-type distance L(1) and second-type distance L(2) obtained using images with the same frame number taken by the depth sensor and RGB sensor, respectively, correspond to the distances that "correspond" to each other.

[0100] Figure 20 is an explanatory diagram showing examples of various determination distances in the fourth embodiment. The upper part of Figure 20 shows the integrated determination distance Ljt(ID) calculated by applying the determination method DM1 described above to the p-th type distance shown in Figure 19 and using M=3.

[0101] In step S150a of Figure 17, the individual identification unit 530 uses the integrated determination distance Ljt(ID) to determine whether the registration condition is met or the unregistered condition is met. If the registration condition is met, the process proceeds to step S160a, where it is determined whether the cow to be identified is one of several registered individuals. On the other hand, if the unregistered condition is met, the process proceeds to step S170, where the cow to be identified is determined to be unregistered and registered in the history data 640(p).

[0102] The process in step S170 in the fourth embodiment can be carried out according to the detailed procedure in any of the following: Figure 9 of the first embodiment, Figure 13 of the second embodiment, or Figure 14 of the third embodiment. In this case, the "distance" used in steps S171 and S215 can be at least one of the following: an integrated distance which is a combination of the first type distance L(1) and the second type distance L(2), or the first type distance L(1) and the second type distance L(2). For example, as the determination process in step S171, one of the following can be used. <First judgment process JP11> If the combined decision value determined using the first type distance L(1) and the second type distance L(2) is smaller than the combined threshold, the result is determined as Yes; if it is larger than the combined threshold, the result is determined as No. The combined decision value can be determined according to the same method as either of the determination methods DM1 or DM2 described above. <Second judgment process JP12> If two or more of the following are smaller than their respective thresholds, the result is determined to be Yes; otherwise, the result is determined to be No. <Third judgment process JP13> If the combined decision value determined using the first type distance L(1) and the second type distance L(2), the first type decision value determined using the first type distance L(1), and the second type decision value determined using the second type distance L(2) are all smaller than their respective thresholds, the result is determined as Yes; otherwise, the result is determined as No.

[0103] Furthermore, for the determination process in step S215, one of the following can be used, for example. <First judgment process JP21> The result is determined as Yes if the minimum value of the combined distance, obtained by integrating the first kind distance L(1) and the second kind distance L(2), is less than the threshold, and as No if it is greater than the threshold. The combined distance can be calculated, for example, by adding N1 × N2 first kind distances L(1) and the corresponding N1 × N2 second kind distances L(2). <Second judgment process JP22> If two or more of the following are smaller than their respective thresholds: the minimum value of the combined distance obtained by integrating the first type distance L(1) and the second type distance L(2), the minimum value of the first type distance L(1), and the minimum value of the second type distance L(2), then the result is Yes; otherwise, the result is No. <Third judgment process JP23> The result is determined to be Yes if the minimum value of the combined distance obtained by combining the first type distance L(1) and the second type distance L(2), the minimum value of the first type distance L(1), and the minimum value of the second type distance L(2) are all smaller than their respective thresholds; otherwise, the result is determined to be No.

[0104] Figure 21 is a flowchart showing the detailed procedure of step S150a in the fourth embodiment. In step S151, the individual identification unit 530 determines whether the minimum value of the integrated determination distance Ljt(ID) is smaller than a preset integrated threshold Tht. If the minimum value of the integrated determination distance Ljt(ID) is smaller than the integrated threshold Tht, the process proceeds to step S161, where the registered individual corresponding to the minimum value of the integrated determination distance Ljt(ID) is identified as the individual of the cattle to be identified. In the example in Figure 20, the minimum value of the integrated determination distance Ljt(ID) corresponds to the registered individual whose individual ID is ID3, so this registered individual is identified as the individual of the cattle to be identified.

[0105] On the other hand, if the minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht, the process proceeds to step S152. The minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht when the cow to be identified is an unregistered individual, or when there is dirt on the back of the cow to be identified. In this case, the identification determination is performed using a determination value different from the integrated determination distance Ljt(ID).

[0106] Furthermore, if the minimum value of the integrated determination distance Ljt(ID) is equal to the integrated threshold Tht, the process may proceed from step S151 to step S161, or to step S152. This also applies to the other determination steps described later.

[0107] In step S152, the individual identification unit 530 determines a first-class determination distance Lj1(ID) that represents N1 × N2 first-class distances L(1) for each of the n registered individuals. The first-class determination distance Lj1(ID) is determined by, for example, one of the following methods.

[0108] <Method for determining the Type 1 classification distance Lj1 (ID) EM1> For each registered individual, M1 values ​​are selected in ascending order from N1 × N2 first-type distances L(1), and the first-type determination distance Lj1(ID) is determined to be a value proportional to the average value of the M1 first-type distances L(1). Here, M1 is an integer greater than or equal to 2 and less than or equal to N1 × N2, and preferably less than N1 × N2. Furthermore, it is preferable that the integers N1 and N2 are set such that N1 × N2 is 3 or greater.

[0109] <Method for determining the Type 1 classification distance Lj1 (ID) EM2> For each registered individual, the first type determination distance Lj1(ID) is determined to be a value proportional to the minimum value among N1 × N2 first type distances L(1).

[0110] In the fourth embodiment, the determination method EM1 described above is applied to the p-th type distance shown in Figure 19, and the first type determination distance Lj1(ID) is determined using M1=3. The first type determination distance Lj1(ID) thus determined is shown at the bottom of Figure 20.

[0111] In step S153, the individual identification unit 530 determines whether the minimum value of the first type determination distance Lj1(ID) is smaller than a preset first threshold Th1. If the minimum value of the first type determination distance Lj1(ID) is smaller than the first threshold Th1, the process proceeds to step S162, where the registered individual corresponding to the minimum value of the first type determination distance Lj1(ID) is identified as the individual of the cattle to be identified.

[0112] On the other hand, if the minimum value of the first type determination distance Lj1(ID) is greater than the first threshold Th1, the process proceeds to step S154. In step S154, the individual identification unit 530 determines a second type determination distance Lj2(ID) that represents N1 × N2 second type distances L(2) for each of the n registered individuals. The second type determination distance Lj2(ID) is determined in the same way as the first type determination distance Lj1(ID) determination methods EM1 and EM2 described above. The lower part of Figure 20 shows the second type determination distance Lj2(ID) calculated according to the same method as the determination method EM1 described above.

[0113] In step S155, the individual identification unit 530 determines whether the minimum value of the second type determination distance Lj2(ID) is smaller than the pre-set second threshold Th2. If the minimum value of the second type determination distance Lj2(ID) is smaller than the second threshold Th2, the process proceeds to step S163, where the registered individual corresponding to the minimum value of the second type determination distance Lj2(ID) is identified as the individual of the cattle to be identified.

[0114] On the other hand, if the minimum value of the second type determination distance Lj2(ID) is greater than the second threshold Th2, the process proceeds to step S156. In step S156, it is determined that the unregistered condition is met. In this case, step S180 in Figure 17 is executed.

[0115] In step S160a of Figure 17, it is determined whether the cow to be identified is one of several registered individuals registered in registration data 630(p). Note that the processing in steps S161 to S163 of Figure 21 can be considered equivalent to step S160a of Figure 16.

[0116] According to the processing shown in Figures 17 and 21 above, the individual cow to be identified can be determined from N1 × N2 p-th distances for each registered individual. In particular, in the fourth embodiment, individuals can be identified using various determination distances, including the integrated determination distance Ljt(ID). Furthermore, if the minimum value of each of the three determination distances Ljt(ID), Lj1(ID), and Lj2(ID) is greater than the respective thresholds Tht, Th1, and Th2, it can be determined that the cow to be identified is an unregistered individual.

[0117] E. Fifth Embodiment: Figure 22 is a flowchart showing the detailed procedure of step S150a in the fifth embodiment. The configuration of the apparatus in the fifth embodiment and the processing content in Figures 2, 3, and 17 are the same as in the fourth embodiment. The processing procedure in Figure 22 omits steps S154, S155, and S163 in Figure 21, and the other steps are the same as in the fourth embodiment.

[0118] In step S153 of the fifth embodiment, if the minimum value of the first type determination distance Lj1(ID) is greater than the first threshold Th1, the process proceeds to step S156, and it is determined that the unregistered condition is met.

[0119] The fifth embodiment also produces almost the same effect as the fourth embodiment. Furthermore, according to the process shown in Figure 22, if the minimum values ​​of the integrated determination distance Ljt(ID) and the first determination distance Lj1(ID) are greater than the threshold, it can be determined that the cow to be identified is an unregistered individual.

[0120] F. Sixth Embodiment: Figure 23 is a flowchart detailing the procedure for step S150a in the sixth embodiment. The configuration of the apparatus in the sixth embodiment and the processing content in Figures 2, 3, and 17 are the same as in the fourth and fifth embodiments. The processing procedure in Figure 23 further omits steps S152, S153, and S162 in Figure 22, and the other steps are the same as in the fifth embodiment.

[0121] In step S151 of the sixth embodiment, if the minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht, the process proceeds to step S156, and it is determined that the unregistered condition is met.

[0122] The sixth embodiment also produces substantially the same effects as the fourth and fifth embodiments. Furthermore, according to the process shown in Figure 23, if the minimum value of the integrated determination distance Ljt(ID) is greater than the threshold, it can be determined that the cow to be identified is an unregistered individual.

[0123] Furthermore, the processes shown in Figures 21, 22, and 23 are the same in that they determine that the target cow is an unregistered individual when it satisfies the unregistered condition, which includes the minimum value of the integrated determination distance Ljt(ID) being greater than the integrated threshold Tht. The individual identification processes of the fourth to sixth embodiments are applicable not only to the first embodiment but also to the second and third embodiments.

[0124] Other forms: This disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms (aspects). The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure, or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate.

[0125] (1) According to a first embodiment of the present disclosure, a method is provided for individual identification of an animal to be identified. This method includes (a) acquiring a target image relating to the back of the animal to be identified; (b) obtaining an embedding vector from the target image using a deep distance learning model; (c) calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) determining whether the animal to be identified corresponds to one of the plurality of registered individuals using the distance; and (e) adding the embedding vector relating to the animal to be identified to the registration data if the confidence level of the determination determined according to the distance is equal to or greater than a confidence threshold. This method uses a deep distance learning model to obtain embedding vectors from target images of the back of the target animal, and the distance between these embedding vectors and registered embedding vectors can be used to identify individuals. Furthermore, if the confidence level of the individual identification determination is above a confidence threshold, the embedding vectors are added to the registered data, thereby improving the reliability of the registered data.

[0126] (2) In the above method, step (e) may include (e1) the step of considering the confidence level to be at a first level equal to or greater than the confidence threshold if the distance is less than a predetermined distance threshold, and considering the confidence level to be at a second level lower than the confidence threshold if the distance is equal to or greater than the distance threshold. This method allows us to determine an appropriate level of confidence using distance.

[0127] (3) In the above method, step (e) may include (e1) using history data in which data relating to a history registered individual corresponding to the animal to be identified is registered in order to register the history of the determination in step (d), if it is determined that the registered individual ID of the registered individual determined to be the animal to be identified in step (d) is already registered in the history data, the step of counting up the number of history determinations relating to the history registered individual having the registered individual ID; and (e2) if the number of history determinations is equal to or greater than a first threshold, the confidence level is considered to be a first level equal to or greater than the confidence threshold, and if the number of history determinations is less than the first threshold, the confidence level is considered to be a second level lower than the confidence threshold. This method allows us to determine an appropriate level of confidence using the number of historical judgments performed on the historical data.

[0128] (4) In the above method, step (e) may further include (e3) a step of counting up the number of history matches for the history registered individual when it is determined that the data for the animal to be identified matches the data for the history registered individual, and (e4) a step of maintaining the confidence level at the first level when the number of history matches is equal to or greater than the second threshold when the confidence level is deemed to be at the first level in step (e2), and changing the confidence level to the second level when the number of history matches is less than the second threshold. This method allows us to determine an appropriate level of confidence using the number of historical matches in the historical data.

[0129] (5) In the above method, step (e4) may further include, if the number of history matches is less than the second threshold, a step of deleting from the history data the data relating to the registered individual in the history that is registered in the history data. This method allows you to delete unnecessary entries from the history data.

[0130] (6) The above method may further include (f) a step of deleting data relating to a specific registered individual that is one of the plurality of registered individuals if the deletion condition is met for that specific registered individual from the registration data. This method allows you to remove unnecessary data from your registered data.

[0131] (7) In the above method, step (f) may include at least one of the following steps: (i) deleting the registration embedding vector from the registration data when the registration period of the registration embedding vector relating to the specific registered individual exceeds a predetermined allowable registration period; (ii) deleting a portion of the registration embedding vector from the registration data when the number of registration embedding vectors relating to the specific registered individual exceeds a predetermined allowable registration number; and (iii) deleting data relating to the specific registered individual from the registration data when the specific registered individual is not determined to be an identifiable target animal over a predetermined allowable period. This method allows you to remove unnecessary data from your registered data.

[0132] (8) In the above method, step (a) may include the step of acquiring N1 p-th type target images with respect to the back of the animal to be identified using a p-th type image sensor, when p is an ordinal number from 1 to 2 and N1 is an integer of 1 or more; step (b) may include the step of obtaining N1 p-th type embedding vectors from the N1 p-th type target images using a p-th deep distance learning model; and step (c) may include the step of calculating N1 × N2 p-th type distances between the N1 p-th type embedding vectors and the N2 p-th type registration embedding vectors using N2 p-th type registration embedding vectors that have been generated in advance for each of the n registered individuals, when n and N2 are integers of 2 or more. Step (d) may include (d1) a step of determining an integrated determination distance for each of the n registered individuals by integrating N1 × N2 first-type distances and N1 × N2 second-type distances; (d2) a step of determining that the registered individual corresponding to the minimum value of the integrated determination distance is the target animal if the minimum value of the integrated determination distance is smaller than a preset integration threshold; and (d3) a step of determining that the target animal is an unregistered individual if it satisfies the unregistered condition, which includes the minimum value of the integrated determination distance being larger than the integration threshold. This method uses two deep distance learning models to obtain two types of embedding vectors from two types of target images of the back of the target animal. The integrated decision distance, determined from the distance between these two embedding vectors and two registered embedding vectors, can then be used to identify the individual. Furthermore, if the unregistered condition is met, the target animal can be determined to be an unregistered individual.

[0133] (9) In the above method, when M is an integer of 2 or more and less than or equal to N1 × N2, the integrated determination distance for each of the n registered individuals may be a value proportional to the sum of the first type distance average value obtained by averaging the M first type distances selected in ascending order from the N1 × N2 first type distances and the second type distance average value obtained by averaging the M second type distances selected in ascending order from the N1 × N2 second type distances. This method allows for the calculation of an appropriate overall judgment distance.

[0134] (10) In the above method, step (d3) may include (d3-1) determining a first type determination distance that represents the N1 × N2 first type distances for each of the n registered individuals, and (d3-2) determining that the registered individual corresponding to the minimum value of the n first type determination distance is an individual of the animal to be identified, when the minimum value of the n first type determination distances is smaller than a preset first type threshold.

[0135] (11) In the above method, step (d3) may further include: (d3-3) when the minimum value of the first type determination distance is greater than the first type threshold, a step of determining a second type determination distance that represents the N1 × N2 second type distances for each of the n registered individuals; (d3-4) when the minimum value of the n second type determination distances is less than a preset second type threshold, a step of determining that the registered individual corresponding to the minimum value of the second type determination distance is an individual of the animal to be identified; and (d3-5) when the minimum value of the second type determination distance is greater than the second type threshold, a step of determining that the animal to be identified is an unregistered individual. This method allows for the determination of whether the animal being identified is a registered individual or an unregistered individual, based on the Type 1 and Type 2 determination distances.

[0136] (12) In the above method, step (d3) may further include (d3-3) a step of determining that the animal to be identified is an unregistered individual when the minimum value of the first type determination distance is greater than the first type threshold. This method allows for the determination of whether the animal being identified is a registered individual or an unregistered individual, based on the first type of identification distance.

[0137] (13) In the above method, when M is an integer of 2 or more and less than or equal to N1 × N2, the first species determination distance for each of the n registered individuals may be a value proportional to the average first species distance obtained by averaging the M first species distances selected in ascending order from the N1 × N2 first species distances. This method allows for the calculation of an appropriate Class 1 classification distance.

[0138] (14) According to a second embodiment of the present disclosure, a computer program is provided which causes a processor to perform a process for individual identification of an animal to be identified. The computer program causes the processor to perform the following: (a) a process for acquiring a target image relating to the back of the animal to be identified; (b) a process for obtaining an embedding vector from the target image using a deep distance learning model; (c) a process for calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) a process for determining whether the animal to be identified corresponds to one of the plurality of registered individuals using the distance; and (e) a process for adding the embedding vector relating to the animal to be identified to the registration data if the confidence level of the determination determined according to the distance is equal to or greater than a confidence threshold.

[0139] This disclosure can also be implemented in various forms other than those described above. For example, it can be implemented in the form of a device that performs individual identification processing, or a non-transitory storage medium on which a computer program is recorded. [Explanation of Symbols]

[0140] 300…Information processing device, 310…Processor, 320…Memory, 330…Interface circuit, 340…Input device, 350…Display device, 400…Image sensor, 510…Target image acquisition unit, 520…Learning unit, 530…Individual identification unit, 610…Object recognition model, 620…Deep distance learning model, 630…Registered data, 640…History data

Claims

1. A method for identifying individual animals, (a) A step of acquiring a target image relating to the dorsal side of the animal to be identified, (b) A step of obtaining an embedding vector from the target image using a deep distance learning model, (c) A step of calculating the distance between a registration embedding vector and an embedding vector using registration data that includes a registration embedding vector generated in advance for each of a plurality of registered individuals, (d) A step of determining whether the animal to be identified is one of the multiple registered individuals, using the distance, (e) If the confidence level of the determination determined according to the distance is equal to or greater than the confidence threshold, the step of adding the embedding vector relating to the animal to be identified to the registration data, Methods that include...

2. The method according to claim 1, The aforementioned step (e) is, (e1) If the distance is less than a predetermined distance threshold, the confidence level is considered to be at a first level equal to or greater than the confidence threshold; if the distance is equal to or greater than the distance threshold, the confidence level is considered to be at a second level lower than the confidence threshold. Methods that include...

3. The method according to claim 1, The aforementioned step (e) is, (e1) Using the history data in which data relating to the history registered individual corresponding to the animal to be identified is registered in order to register the history of the determination in step (d), if it is determined that the registered individual ID of the determined registered individual that was determined to be the animal to be identified in step (d) is already registered in the history data, the number of history determinations relating to the history registered individual having the registered individual ID is counted up. (e2) A step of considering the reliability to be at a first level, which is at or above the reliability threshold, when the number of history judgments is at or above a first threshold, and considering the reliability to be at a second level, which is lower than the reliability threshold, when the number of history judgments is less than the first threshold, Methods that include...

4. The method according to claim 3, The above step (e) further includes, (e3) If it is determined that the data relating to the animal to be identified matches the data relating to the registered individual, the step of counting up the number of history matches relating to the registered individual, (e4) If the confidence level is deemed to be at the first level in step (e2), the confidence level is maintained at the first level if the number of historical matches is equal to or greater than the second threshold, and the confidence level is changed to the second level if the number of historical matches is less than the second threshold, Methods that include...

5. The method according to claim 4, The above step (e4) is, If the number of history matches is less than the second threshold, the further step is to delete the data relating to the registered individual in the history that is registered in the history data from the history data. Methods that include...

6. The method according to claim 1, further, (f) When the deletion condition is met for a specific registered individual which is one of the multiple registered individuals, the step of deleting the data relating to the specific registered individual for which the deletion condition has been met from the registration data. Methods that include...

7. The method according to claim 6, The aforementioned step (f) is, (i) If the registration period of the registration embedding vector relating to the specific registered individual exceeds a predetermined allowable registration period, the step of deleting the registration embedding vector from the registration data; (ii) If the number of registration embedding vectors relating to the specific registered individual exceeds a preset allowable number of registrations, a step of deleting a portion of the registration embedding vectors from the registration data; (iii) If the specified registered individual is not determined to be the animal to be identified within a predetermined allowable period, the step of deleting the data relating to the specified registered individual from the registration data, A method comprising at least one of the following steps.

8. A method according to any one of claims 1 to 7, Step (a) includes the step of acquiring N1 type p target images of the dorsal side of the animal to be identified using a type p image sensor, where p is an ordinal number from 1 to 2 and N1 is an integer of 1 or more. Step (b) includes the step of obtaining N1 p-th type embedding vectors from the N1 p-th type target images using a p-th deep distance learning model, Step (c) includes a step of calculating N1 × N2 p-th type distances between the N1 p-th type embedding vectors and the N2 p-th type embedding vectors, using N2 p-th type registration embedding vectors that have been generated in advance for each of the n registered individuals, where n and N2 are integers of 2 or more. The aforementioned step (d) is, (d1) A step of determining an integrated determination distance for each of the n registered individuals by integrating N1 × N2 first type distances and N1 × N2 second type distances, (d2) A step of determining that the registered individual corresponding to the minimum value of the integrated determination distance is the animal to be identified when the minimum value of the integrated determination distance is smaller than a preset integrated threshold, (d3) A step of determining that the animal to be identified is an unregistered individual when the unregistered condition is met, which includes the minimum value of the integrated determination distance being greater than the integrated threshold, Methods that include...

9. The method according to claim 8, A method in which, when M is an integer of 2 or more and less than or equal to N1 × N2, the integrated determination distance for each of the n registered individuals is a value proportional to the sum of the first type distance mean value obtained by averaging M first type distances selected in ascending order from the N1 × N2 first type distances and the second type distance mean value obtained by averaging M second type distances selected in ascending order from the N1 × N2 second type distances.

10. The method according to claim 8, The aforementioned step (d3) is, (d3-1) A step of determining a first type determination distance that represents the N1 × N2 first type distances for each of the n registered individuals, (d3-2) A step in which, if the minimum value of the n first type determination distances is smaller than a preset first type threshold, the registered individual corresponding to the minimum value of the first type determination distance is determined to be an individual of the animal to be identified. Methods that include...

11. The method according to claim 10, The above step (d3) further, (d3-3) When the minimum value of the first type determination distance is greater than the first type threshold, a step is taken to determine a second type determination distance that represents the N1 × N2 second type distances for each of the n registered individuals, (d3-4) A step in which, if the minimum value of the n second type determination distances is smaller than a preset second type threshold, the registered individual corresponding to the minimum value of the second type determination distance is determined to be an individual of the animal to be identified. (d3-5) A step in which, if the minimum value of the second type determination distance is greater than the second type threshold, the animal to be identified is determined to be an unregistered individual, Methods that include...

12. The method according to claim 10, The above step (d3) further, (d3-3) A step in which, if the minimum value of the first type determination distance is greater than the first type threshold, the animal to be identified is determined to be an unregistered individual. Methods that include...

13. The method according to claim 10, A method wherein the first species determination distance for each of the n registered individuals is a value proportional to the minimum value among the N1 × N2 first species distances.

14. A computer program that causes a processor to perform a process for identifying individual animals, (a) A process to obtain a target image relating to the back of the animal to be identified, (b) A process to obtain an embedding vector from the target image using a deep distance learning model, (c) A process to calculate the distance between a registration embedding vector and another embedding vector using registration data that includes registration embedding vectors generated in advance for each of a plurality of registered individuals, (d) A process of determining whether the animal to be identified is one of the multiple registered individuals, using the distance, (e) If the confidence level of the determination determined according to the distance is equal to or greater than the confidence threshold, the process of adding the embedding vector relating to the animal to be identified to the registration data, A computer program that causes the aforementioned processor to execute.

Images