A method for identifying individual animals, and a computer program for doing so.

Abstract

This invention provides a method and computer program that can perform individual identification using characteristics other than nose prints. [Solution] The method includes the steps of: acquiring a target image of the back of the animal to be identified S110; obtaining an embedding vector from the target image using a deep distance learning model S120; calculating the distance between the embedding vectors using registration data that includes pre-generated registration embedding vectors for each of the multiple registered individuals S130; determining whether the animal to be identified is one of the multiple registered individuals or an unregistered individual not registered in the registration data using a determination distance Li(ID) S150; registering data about the animal to be identified as provisional registration data if the animal to be identified is an unregistered individual S170; and transferring the provisional registration data to registered data if the provisional registration data satisfies pre-set transfer conditions.

Description

[[Technical Field]]

[0001] The present disclosure relates to a method for individual animal identification and a computer program. [[Background Art]]

[0002] Patent Document 1 discloses a technique for collating nose print images of cattle. In this conventional technique, a nose print image is extracted from a facial image of a cow, a feature vector of the nose print image is acquired using a neural network for nose print image classification, and the nose print image is collated by calculating the similarity between the feature vector and a known feature vector. [[Prior Art Documents]] [[Patent Documents]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2022-48464 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] However, the conventional technique requires a high-resolution facial image that can identify the fine structure of the nose print, and there has been a problem that it is difficult to acquire such a facial image. For example, it is difficult to stop a cow that is walking continuously for photographing. Such a problem is not limited to cattle, but is common to individual identification of other animals such as pigs. Therefore, a technique capable of performing individual identification using features other than nose prints is desired. [[Means for Solving the Problem]]

[0005] A first embodiment of this disclosure provides a method for individual identification of an animal to be identified. This method includes: (a) acquiring a target image relating to the dorsal side of the animal to be identified; (b) obtaining an embedding vector from the target image using a deep distance learning model; (c) calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) determining, using the distance, whether the animal to be identified is one of the plurality of registered individuals or an unregistered individual not registered in the registration data; (e) registering data relating to the animal to be identified as provisional registration data if the animal to be identified is an unregistered individual; and (f) transferring the provisional registration data to registered data if the provisional registration data satisfies pre-set transition conditions.

[0006] A second embodiment of the present disclosure provides a computer program that causes a processor to perform a process for individual identification of an animal to be identified. The computer program causes the processor to perform the following: (a) a process for acquiring a target image relating to the back of the animal to be identified; (b) a process for obtaining an embedding vector from the target image using a deep distance learning model; (c) a process for calculating the distance between the registered embedding vector and the embedding vector using registration data that includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) a process for determining, using the distance, whether the animal to be identified is one of the plurality of registered individuals or an unregistered individual not registered in the registration data; (e) a process for registering data relating to the animal to be identified as provisional registration data if the animal to be identified is an unregistered individual; and (f) a process for transferring the provisional registration data to the registered data if the provisional registration data satisfies pre-set transfer conditions. [Brief explanation of the drawing]

[0007] [Figure 1] A block diagram showing the configuration of the individual identification system of the first embodiment. [Figure 2] A flowchart illustrating the steps for training a deep metric learning model and creating registration data. [Figure 3] A flowchart showing the detailed steps for step S20. [Figure 4] This diagram illustrates the process of generating the target image in step S20. [Figure 5] An explanatory diagram showing an example of registered data. [Figure 6] A flowchart illustrating the procedure for individual identification processing in the first embodiment. [Figure 7] An explanatory diagram showing an example of an embedding vector calculated during individual identification processing. [Figure 8] An explanatory diagram showing an example of the distance between an embedding vector and a registered embedding vector. [Figure 9] A flowchart showing the detailed procedure of step S170 in the first embodiment. [Figure 10] An explanatory diagram showing an example of provisional registration data. [Figure 11] An explanatory diagram showing an example of the distance between an embedded vector and a provisionally registered embedded vector. [Figure 12] A flowchart illustrating the procedure for updating registered data. [Figure 13] A flowchart showing the procedure for deleting provisionally registered individuals. [Figure 14] A block diagram showing the configuration of the individual identification system of the second embodiment. [Figure 15] An explanatory diagram showing an example of registered data in the second embodiment. [Figure 16] A flowchart illustrating the procedure for individual identification processing in the second embodiment. [Figure 17] An explanatory diagram showing an example of an embedding vector in the second embodiment. [Figure 18] An explanatory diagram showing an example of the distance between the embedding vector and the registered embedding vector in the second embodiment. [Figure 19] An explanatory diagram showing examples of various determination distances in the second embodiment. [Figure 20]A flowchart showing the detailed procedure of step S150a in the second embodiment. [Figure 21] A flowchart showing the detailed procedure of step S150a in the third embodiment. [Figure 22] A flowchart showing the detailed procedure of step S150a in the fourth embodiment. MODES FOR CARRYING OUT THE INVENTION

[0008] A. First Embodiment: Fig. 1 is an explanatory diagram showing the configuration of an individual identification system according to the first embodiment. This individual identification system includes an information processing device 300 and an image sensor 400. In the present embodiment, the processing target of individual identification is cattle CW. However, instead of cattle, other animals such as pigs, dogs, and cats may be used as the processing target.

[0009] The image sensor 400 is a camera that captures images of cattle CW, which is the target of individual identification processing. The image sensor 400 is preferably installed so as to capture an image of the back of cattle CW from above the cattle CW. As the image sensor 400, a video camera that captures moving images may be used, or a still image camera that captures still images may be used. Further, various sensors exemplified below can be used as the image sensor 400. (1) Depth sensor When a depth image captured by a depth sensor is used, individual identification can be performed from the unevenness of the back of cattle. (2) RGB sensor (color image sensor) When a color image captured by an RGB sensor is used, individual identification can be performed from the black-and-white pattern, which is the mottling on the back of cattle. If there is dirt on the back of cattle, the color image changes, so misrecognition may occur when individual identification is performed using only an RGB sensor. On the other hand, a depth image captured by a depth sensor is not affected by dirt on the back of cattle, so it has the advantage of a low possibility of misrecognition due to dirt. (3) Spectral sensor By using spectral images captured with a spectroscopic sensor, it is possible to identify individual cows based on the black and white patterns on their backs, which are the wavelength-specific markings. (4) Thermosensor By using thermal images captured with a thermal sensor, it is possible to identify individual cows based on the infrared intensity distribution on their backs. The infrared intensity distribution on the cow's back reflects how the degree of infrared scattering changes depending on the length of the hair, and how the degree of infrared diffusion changes depending on the distance due to unevenness.

[0010] The information processing device 300 performs individual identification of cattle CW using an image of the dorsal side of the cattle CW captured by the image sensor 400. The information processing device 300 includes a processor 310, a memory 320, an interface circuit 330, and an input device 340 and a display device 350 connected to the interface circuit 330. The image sensor 400 is also connected to the interface circuit 330. The processor 310 not only has the function of performing the processing detailed below, but also has the function of displaying the data obtained by the processing and the data generated in the process of the processing on the display device 350. The information processing device 300 can be implemented by a computer such as a personal computer.

[0011] The processor 310 has the functions of a target image acquisition unit 510, a learning unit 520, and an individual identification unit 530. The target image acquisition unit 510 acquires multiple target images from the back image captured by the image sensor 400. The learning unit 520 performs distance learning of the deep distance learning model 620. The individual identification unit 530 performs individual identification of cattle CW using the trained deep distance learning model 620. The functions of each of these units are realized by the processor 310 executing a computer program stored in the memory 320. However, some of these functions may be realized by hardware circuits. The term "processor" in this disclosure includes such hardware circuits. Furthermore, one or more processors that perform various processes may be processors included in one or more remote computers connected via a network.

[0012] The distance used for training in the deep metric learning model 620 can also be similarity. A high similarity score corresponds to a small distance. In other words, "small distance" is equivalent to "high similarity." In the following explanation, "distance" is used as a term that includes "similarity."

[0013] Memory 320 stores the object recognition model 610, the deep distance learning model 620, the registered data 630, and the provisional registered data 640. The object recognition model 610 is a machine learning model that takes an image of a cow CW as input and outputs multiple keypoints, which are the morphological feature points of the cow CW. In this embodiment, it is assumed that the object recognition model 610 has been trained. The object recognition model 610 can also be called a "feature recognition model".

[0014] The deep metric learning model 620 is a machine learning model that takes images of cow CW as input and outputs embedding vectors. These embedding vectors are also called "feature vectors." The deep metric learning model 620 can be constructed, for example, using FaceNet.

[0015] The registration data 630 is a database in which, for each of several registered individuals, the individual ID and multiple embedding vectors obtained using the deep distance learning model 620 are registered. The individual ID is, for example, the individual identification number displayed on the ear tag of a cow CW. The embedding vectors registered in the registration data 630 are called "registered embedding vectors". In addition to registered embedding vectors, the registration data 630 may also include images of each registered individual.

[0016] The provisional registration data 640 is data in which unregistered individuals that are not registered in the registration data 630 are provisionally registered during the individual identification process described later. The contents of the provisional registration data 640 will be described later.

[0017] Figure 2 is a flowchart showing the procedure for training the deep distance learning model 620 and creating the registration data 630. In the following description, we assume that n is an integer greater than or equal to 2, and that n target cows are registered in the registration data 630. In this embodiment, the n target cows are of the same breed.

[0018] In step S10, the target image acquisition unit 510 uses the image sensor 400 to capture multiple dorsal images of each of the n target cows to be registered and the k target cows to be trained. Here, n and k are integers of 2 or greater. This capture can be performed, for example, as each cow sequentially passes below the image sensor 400. If a video is captured, multiple frame images are selected from the video as dorsal images. This selection can be performed, for example, by automatically selecting frame images that include the rear portion of the target cows to be registered using an annotation tool. Alternatively, the operator may make the selection manually.

[0019] In step S20, the target image acquisition unit 510 acquires multiple target images of the backs of each of the n registered target cows and k training target cows from the back images captured by the image sensor 400. "Training target cows" refers to cows used to create training data for the deep distance learning model 620. In deep distance learning, from the viewpoint of robustness, it is desirable to train the neural network using training data different from the registered data 630. It is also desirable to create training data that contains more data than the registered data 630. Therefore, it is preferable that k be set to a number greater than n. However, some of the training target cows may be registered cows. Alternatively, n=k may be used, and the same cow may be used as both the training target cows and the registered target cows. The number of target images acquired from each cow may be set to a constant value N, or it may be set to a different number for each individual. Here, N is an integer of 1 or more, but it is preferable that it be 2 or more.

[0020] Figure 3 is a flowchart showing the detailed procedure of step S20, and Figure 4 is an explanatory diagram showing the processing content. In the first embodiment, a depth sensor is used as the image sensor 400. Some of the step numbers are included in Figure 4.

[0021] The dorsal image BG shown in Figure 4 includes the posterior portion of the cattle coww, including the lumbar horns and the base of the tail. Since individual cattle cowws have distinctive features in the shape of their posterior portion, it is preferable that the dorsal image BG includes the posterior portion of the cattle coww, including the lumbar horns and the base of the tail.

[0022] In step S21, the target image acquisition unit 510 selects one cow to be processed from n registered cows and k training cows. In step S22, the target image acquisition unit 510 selects one back image BG to be processed from multiple back images BG. In step S23, the target image acquisition unit 510 creates a feature detection image CG by performing a grayscale conversion process on the back image BG. This grayscale conversion process extracts a grayscale range from the total grayscale of the back image BG that makes it easy to detect the contour of the cow, and converts it into an image with a predetermined number of grayscales. For example, if the back image BG, which is a depth image, has 2400 grayscales, the feature detection image CG may be created by extracting 512 intermediate grayscales and compressing them to 256 grayscales. However, step S23 is optional.

[0023] In step S24, the target image acquisition unit 510 uses the object recognition model 610 to detect multiple keypoints from the feature detection image CG. In the example in Figure 4, two keypoints KP1 and KP2 are detected at the hip horns of the cow CW, and one keypoint KP3 is detected at the base of the tail. These keypoints KP1 to KP3 are feature points present on the outline of the cow CW. The object recognition model 610 is a machine learning model that takes the feature detection image CG as input and outputs multiple keypoints KP1 to KP3. However, the object recognition model 610 may be configured to recognize keypoints indicating other locations. The object recognition model 610 can be configured using, for example, a deep learning model such as DeepLabCut or ResNet. The object recognition model 610 may also be configured using other machine learning models.

[0024] In step S25, the target image acquisition unit 510 determines whether a predetermined number of keypoints KP1 to KP3 have been detected. If the predetermined number of keypoints KP1 to KP3 have not been detected, the process returns to step S22, a new back image BG is selected as the processing target, and the processing from step S23 onwards is executed again. If the predetermined number of keypoints KP1 to KP3 have been detected, the process proceeds to step S26.

[0025] In step S26, the target image acquisition unit 510 extracts the target image TG from the back image BG based on a plurality of key points KP1 to KP3. The target image TG is an image that includes the characteristic parts of the cow's back. In the example in Figure 4, the back image BG is shown with a cropping frame CF set based on key points KP1 to KP3, and the target image TG extracted according to the cropping frame CF is shown. At this time, it is preferable to rotate and resize the target image TG so that the direction and size of the cow CW's hip horns within the target image TG become the desired values. Alternatively, the pixel values ​​of the target image TG may be normalized. This normalization is, for example, normalizing the pixel value of the minimum depth value to 1.0 and the pixel value of the maximum depth value to 0, and further converting the pixel values ​​in the range of 0 to 1.0 to a grayscale of 0 to 255. Steps S23 to S26 may be omitted, and the back image BG may be used as the target image TG as is.

[0026] In step S27, the target image acquisition unit 510 determines whether or not N target images TG have already been created for a single cow. If N target images TG have not been created, the process returns to step S22, a new back image BG is selected as the processing target, and the processing from step S22 onward is executed again. If N target images TG have already been created, the process proceeds to step S28.

[0027] In step S28, the target image acquisition unit 510 determines whether processing has been completed for all n registered target cows and k training target cows. If processing is not completed for all cows, the process returns to step S21, a new cow is selected as the target for processing, and the processing from step S22 onwards is executed again. If processing is completed for all cows, the process in step S20 is terminated.

[0028] In step S30 of Figure 2, the learning unit 520 creates distance learning data for deep distance learning by associating N target images TG acquired for each individual training target cow with an individual ID. The individual ID is usually entered by the user. When the user determines the individual ID of a cow, it is preferable to display a color image of the cow taken using an RGB sensor on the display device 350, as it is easier to identify the cow from a color image than from a depth image. As mentioned above, the individual identification number displayed on the ear tag of the cow CW is used as the individual ID.

[0029] In step S40, the learning unit 520 performs distance learning on the deep distance learning model 620 using the distance learning data. This distance learning is a process of adjusting the internal parameters of the deep distance learning model 620 so that the distance between the same individual and the distance between different individuals are small for the embedding vectors output from the deep distance learning model 620. For example, the distance between the embedding vectors may be the Euclidean distance between the vectors or the angle between the vectors.

[0030] In step S50, the learning unit 520 sequentially inputs N target images TG for each registered cow into the trained deep distance learning model 620 and obtains an embedding vector for each target image TG. As a result, N embedding vectors are obtained for each registered cow. In step S60, the learning unit 520 creates registration data 630 for each of the n registered cows by associating the N embedding vectors with individual IDs.

[0031] In step S70, the learning unit 520 determines whether processing has been completed for all back images captured in step S10. If processing is complete, the process shown in Figure 2 is terminated. If processing is not complete, the process returns to step S20, and steps S20 to S70 are repeated for the next back image.

[0032] By performing the learning process shown in Figure 2 above, a trained deep distance learning model 620 and registration data 630 for n target cows are obtained. In the following explanation, the target cows will be referred to as "registered individuals." Also, the embedding vectors registered in the registration data 630 will be referred to as "registration embedding vectors."

[0033] Figure 5 is an explanatory diagram showing an example of registration data 630. In this example, the number of registered cows n is 4, and the number of target images TG N is 3. Registration data 630 contains registration embedding vectors Vr for each of the four registered individuals, for each of the three target images TG. In this example, the registration embedding vector Vr is a 5-dimensional vector with 5 elements. The individual IDs of the four registered individuals are ID1 to ID4. Also, the three target images TG obtained for each individual are assigned different image IDs. However, image IDs do not necessarily have to be registered. Normally, a larger number of registration embedding vectors Vr are registered for each registered individual, but in Figure 5, the number of registration embedding vectors Vr is reduced for illustrative purposes.

[0034] Figure 6 is a flowchart showing the procedure for individual identification processing in the first embodiment. The individual identification processing is preferably performed periodically, for example, on all cattle raised in the same farm as the cattle to be identified.

[0035] In step S110, the individual identification unit 530 acquires N1 target images TG of the back of one target cow using the image sensor 400. Here, N1 is an integer of 1 or more. N1 may be set to a value equal to the number of registered embedding vectors for each registered individual registered in the registration data 630, or it may be set to a different value. Also, N1 may not be a pre-set value, but the number of target images TG that have been acquired for the target cow at that time. In the following description, the number of registered embedding vectors for each registered individual will be referred to as "N2" and distinguished from the number of target images TG N1 of the target cow. The integer N2 is the same as the integer N used in Figures 2 and 3, and is an integer of 2 or more. In the example in Figure 5, the number of registered embedding vectors N2 is 3. The specific processing content of step S110 is the same as the target image TG creation process for registered cows described in Figures 3 and 4.

[0036] In step S120, the individual identification unit 530 uses the trained deep distance learning model 620 to obtain N1 embedding vectors for N1 target images TG.

[0037] Figure 7 is an explanatory diagram showing an example of an embedding vector calculated in the individual identification process. In this example, the number of target images TG N1 is 3. That is, the embedding vector Vt is calculated for each of the three target images TG obtained for the cow to be identified. The embedding vector Vt is a vector of the same dimension as the registered embedding vector Vr shown in Figure 5.

[0038] In step S130, the individual identification unit 530 calculates the distance between N1 embedding vectors and N2 registration embedding vectors for each registered individual. As a result, N1 × N2 distances are calculated for each registered individual. Since there are n registered individuals, n × N1 × N2 distances are calculated for each identified cow.

[0039] Figure 8 is an explanatory diagram illustrating an example of the distance between an embedding vector and a registered embedding vector. For the registered embedding vector Vr shown in Figure 5, n=4 and N2=3, and for the embedding vector Vt shown in Figure 7, N1=3, so 36 distances L are calculated for one identifiable cow.

[0040] In step S140, the individual identification unit 530 determines a determination distance Lj(ID) that represents N1 × N2 distances for each of the n registered individuals. The "ID" in the code of the determination distance Lj(ID) means the individual ID of the registered individual. The determination distance Lj(ID) is determined by one of the following methods, for example.

[0041] <Method for determining the judgment distance Lj(ID) JM1> For each registered individual, M distances L are selected in ascending order from N1 × N2 distances L, and a value proportional to the average of these M distances L is determined as the determination distance Lj(ID). Here, M is an integer greater than or equal to 2 and less than or equal to N1 × N2, and preferably less than N1 × N2. It is also preferable that integers N1 and N2 are set such that N1 × N2 is 3 or greater. In this disclosure, a value proportional to a specific value means a value obtained by multiplying the specific value by a positive coefficient, and may also be the specific value itself.

[0042] <Method for determining the judgment distance Lj(ID) JM2> For each registered individual, the determination distance Lj(ID) is determined to be a value proportional to the minimum of N1 × N2 distances L.

[0043] In the first embodiment, the determination distance Lj(ID) is determined according to the determination method JM1 described above. Figure 8 above shows the determination distance Lj(ID) calculated using M=3 for each registered individual.

[0044] In step S150 of Figure 6, the individual identification unit 530 uses the determination distance Lj(ID) to determine whether the registration condition is met or the non-registration condition is met. The "registration condition" is a condition indicating that the cow to be identified corresponds to one of several registered individuals registered in the registration data 630. The "non-registration condition" is a condition indicating that the cow to be identified is not registered in the registration data 630. Specifically, in step S150, if the minimum value of the determination distance Lj(ID) is smaller than the predetermined determination threshold Th, the registration condition is considered to be met, and the process proceeds to step S160. In step S160, the individual identification unit 530 identifies the registered individual corresponding to the minimum value of the determination distance Lj(ID) as the cow to be identified. In the example in Figure 8, the minimum value of the determination distance Lj(ID) corresponds to the registered individual whose individual ID is ID3, so this registered individual is identified as the cow to be identified.

[0045] On the other hand, if the minimum value of the determination distance Lj(ID) is greater than the determination threshold Th, the unregistered condition is considered to be met, and the process proceeds to step S170. The minimum value of the determination distance Lj(ID) becomes greater than the determination threshold Th when the cow to be identified is an unregistered individual, or when there is dirt on the back of the cow to be identified. In this case, in step S170, the individual identification unit 530 determines that the cow to be identified is unregistered and registers the provisional registration data 640. "Unregistered" means that it is not registered in the registration data 630.

[0046] Furthermore, if the minimum value of the determination distance Lj(ID) is equal to the determination threshold Th, the process proceeds to one of the two pre-selected branch destinations from step S150. This is also true for other determination steps that use thresholds.

[0047] In step S150, it is also possible to determine which registered individual the target cow belongs to, or whether it is an unregistered individual, using conditions different from those described above. That is, if N1 × N2 distances satisfy the pre-set unregistered condition, it can be determined that the target cow is an unregistered individual. Examples of other unregistered conditions will be described in the second embodiment and subsequent embodiments.

[0048] Figure 9 is a flowchart showing the detailed procedure for step S170 in the first embodiment. In step S171, the individual identification unit 530 obtains the contents of the provisional registration data 640 and the number of provisionally registered individuals Npr registered in the provisional registration data 640.

[0049] Figure 10 is an explanatory diagram showing an example of provisional registration data 640. In this example, the number of provisionally registered individuals Npr is 2. For each provisionally registered individual, provisional registration data 640 contains the provisional registration individual ID, provisional registration count Mpr, initial registration date, provisional registration image ID, and provisional registration embedding vector Vpr obtained from the provisional registration image. However, the provisional registration image ID is optional. Alternatively, the target image used to create the provisional registration embedding vector Vpr may also be registered in provisional registration data 640. The provisional registration individual ID is unrelated to the ear tag of the provisionally registered individual and is a sequential number that is automatically determined when, for example, a provisionally registered individual is registered in provisional registration data 640.

[0050] In the example in Figure 10, three temporary registration embedding vectors Vpr are registered for each temporary registration, corresponding to three temporary registration images. For example, a temporary registration individual with temporary registration individual ID PID1 has a temporary registration count Mpr of 2, so six temporary registration embedding vectors Vpr are registered. Similarly, a temporary registration individual with temporary registration individual ID PID2 has a temporary registration count Mpr of 1, so three temporary registration embedding vectors Vpr are registered.

[0051] In step S172, the individual identification unit 530 determines whether the number of provisionally registered individuals Npr registered in the provisional registration data 640 is 1 or greater. If the number of provisionally registered individuals Npr is less than 1, that is, if no provisionally registered individuals are registered in the provisional registration data 640, the process proceeds to step S177, where the individual identification unit 530 determines that the unregistered individual is a new provisionally registered individual. After step S177, the process proceeds to step S178, which will be described later.

[0052] On the other hand, if the number of provisionally registered individuals Npr registered in the provisional registration data 640 is 1 or more, the process proceeds to step S173, where the individual identification unit 530 calculates the distance between an unregistered individual and each provisionally registered individual. If N1 is the number of embedding vectors Vt for unregistered individuals, and N3 is the number of provisional registration embedding vectors Vpr for each provisionally registered individual, then N1 × N3 distances are calculated.

[0053] Figure 11 is an explanatory diagram illustrating an example of the distance between an embedding vector and a provisional registration embedding vector. In this example, it is assumed that in the individual identification process using the embedding vector Vt shown in Figure 7, the target cow is determined to be an unregistered individual, and the distance between the embedding vector Vt in Figure 7 and the provisional registration embedding vector Vpr in Figure 10 is calculated. For the first provisionally registered individual, the number of provisional registration embedding vectors Vpr N3 is 6, so a distance of N1 × N3 = 18 vectors is calculated. For the second provisionally registered individual, the number of provisional registration embedding vectors Vpr N3 is 3, so a distance of N1 × N3 = 9 vectors is calculated.

[0054] In step S174, the individual identification unit 530 determines a determination value Lpj(PID) that represents the N1 × N3 distances obtained in step S174 for each provisionally registered individual. The "PID" in the sign of the determination value Lpj(PID) means the provisionally registered individual ID. The determination value Lpj(PID) is determined by a method similar to either of the determination distance Lj(ID) determination methods JM1 or JM2 described above. In the example in Figure 11, according to determination method JM1, M distances L are selected in ascending order from the N1 × N2 distances L, and the determination value Lpj(PID) is determined to be a value proportional to the average value of the M distances L.

[0055] In step S175, the individual identification unit 530 determines whether the minimum value of the determination value Lpj(PID) is smaller than a preset threshold Tph. If the minimum value of the determination value Lpj(PID) is larger than the threshold Tph, the process proceeds to step S177, and the individual identification unit 530 determines that the unregistered individual is a new provisionally registered individual.

[0056] On the other hand, if the minimum value of the judgment value Lpj(PID) is less than the threshold Tph, the process proceeds to step S176, and the provisional registration count Mpr of the provisionally registered individual corresponding to the minimum value of the judgment value Lpj(PID) is incremented by one. In the example in Figure 11, the minimum value of the judgment value Lpj(PID) is 0.1, which is assumed to be less than the threshold Tph. In this case, the provisional registration count Mpr of the provisionally registered individual whose provisional registration individual ID is PID1 is incremented by one.

[0057] In step S178, the individual identification unit 530 updates the provisional registration data 640. Specifically, for provisionally registered individuals whose provisional registration count Mpr was incremented in step S176, the provisional registration embedding vector Vpr is added to the provisional registration data 640. Also, if it is determined in step S177 that an individual is a new provisionally registered individual, the provisional registration individual ID, provisional registration count Mpr (=1), initial registration date, and provisional registration embedding vector Vpr of that individual are registered in the provisional registration data 640.

[0058] According to the process shown in Figure 6 above, the individual cow to be identified can be determined from N1 × N2 distances related to each registered individual. Furthermore, if the N1 × N2 distances satisfy the pre-set unregistered condition, the cow to be identified can be determined to be an unregistered individual. In addition, if the cow to be identified is determined to be an unregistered individual, the data related to the unregistered individual can be registered in the provisional registration data 640 by the process shown in Figure 9.

[0059] Figure 12 is a flowchart showing the procedure for updating registered data. This process is preferably performed periodically.

[0060] In step S210, the individual identification unit 530 selects one of the provisionally registered individuals registered in the provisional registration data 640. In step S220, the individual identification unit 530 obtains the provisional registration count Mpr of the selected provisionally registered individual. In step S230, the individual identification unit 530 determines whether the provisional registration count Mpr is equal to or greater than the count threshold Tmpr. The count threshold Tmpr is an integer of 2 or more, and is preferably set to 3 or more.

[0061] If the number of provisional registrations Mpr is equal to or greater than the threshold Tmpr, the pre-set transition conditions are deemed to have been met, and the process proceeds to step S240, where the individual identification unit 530 transfers the data relating to the provisionally registered individual from the provisional registration data 640 to the registered data 630. That is, the data relating to the provisionally registered individual is deleted from the provisional registration data 640 and registered in the registered data 630. When transferring from the provisional registration data 640 to the registered data 630, it is preferable for the user to input and register the registered individual ID. The registered individual ID is the individual identification number displayed on the ear tag of the cattle. However, the operation of inputting the individual identification number of the ear tag as the registered individual ID into the registered data 630 may be performed at any time after step S240. As mentioned above, when inputting the registered individual ID, it is preferable to display a color image of the cattle captured using an RGB sensor on the display device 350.

[0062] Furthermore, other conditions besides "the number of provisional registrations Mpr is equal to or greater than the number threshold Tmpr" may be used as transition conditions for moving provisionally registered individuals from provisional registration data 640 to registered data 630. For example, the transition condition "the number of provisional registration embedding vectors Vpr is equal to or greater than the number threshold" may be used.

[0063] If the number of provisional registrations Mpr in step S230 is less than the number threshold Tmpr, step S240 is skipped and the process proceeds to step S250. In step S250, the individual identification unit 530 determines whether the processing in steps S210 to S240 has been completed for all provisionally registered individuals registered in the provisional registration data 640. If the processing has not been completed, the process returns to step S210, and the processing in steps S210 to S250 is executed again.

[0064] In steps S220 and S230, different conditions may be used to determine whether or not to transfer the data of a provisionally registered individual to the registered data 630. For example, if the number of provisional registration embedding vectors Vpr for a single provisionally registered individual exceeds a predetermined threshold, it may be decided to transfer the data of that provisionally registered individual to the registered data 630. As can be seen from these examples, a provisionally registered individual can be registered as a registered individual if the provisional registration data 640 satisfies the predetermined registration conditions.

[0065] According to the registration update process shown in Figure 12, when the number of provisional registrations Mpr exceeds the threshold Tmpr, the data for the provisionally registered individual is moved from the provisional registration data 640 to the registered data 630, so that the appropriate registered individual can be registered in the registered data 630. In addition, it is not necessary to register the ear tag as the registered individual ID for all unregistered individuals; the ear tag only needs to be registered as the registered individual ID when the number of provisional registrations Mpr exceeds the threshold Tmpr, thus reducing the number of ear tag registrations.

[0066] Figure 13 is a flowchart showing the procedure for deleting provisionally registered individuals. This process is preferably performed periodically.

[0067] In step S310, the individual identification unit 530 selects one of the provisionally registered individuals registered in the provisional registration data 640. In step S320, the individual identification unit 530 obtains the registration period of the selected provisionally registered individual. The "registration period" is the elapsed time from the time the provisionally registered individual was first registered in the provisional registration data 640 until the time the process shown in Figure 13 is executed. In this embodiment, the registration period is the number of days elapsed from the initial registration date shown in Figure 10.

[0068] In step S330, the individual identification unit 530 determines whether the registration period exceeds a predetermined allowable registration period. The allowable registration period is the period during which data on a provisionally registered individual is allowed to be registered in the provisional registration data 640. The reason for setting an allowable registration period is that if such a period is not set, the number of provisionally registered individuals registered over a long period of time may increase, potentially leading to an excessive amount of data in the provisional registration data 640.

[0069] In step S330, if the registration period of a provisionally registered individual exceeds the allowable registration period, the process proceeds to step S340, where the individual identification unit 530 deletes the data relating to that provisionally registered individual from the provisional registration data 640. In the process shown in Figure 12 above, when the number of provisional registrations Mpr becomes equal to or greater than the number threshold Tmpr, the data relating to that provisionally registered individual is moved from the provisional registration data 640 to the registration data 630. Therefore, for provisionally registered individuals being processed in step S330, it is presumed that the number of provisional registrations Mpr has not yet reached the number threshold Tmpr. Accordingly, the processing in steps S330 and S340 corresponds to the process of deleting the data relating to a provisionally registered individual from the provisional registration data 640 if the registration period of a provisionally registered individual exceeds the allowable registration period before the number of provisional registrations Mpr reaches the number threshold Tmpr.

[0070] In step S330, if the registration period of a provisionally registered individual has not exceeded the allowable registration period, step S340 is skipped and the process proceeds to step S350. In step S350, the individual identification unit 530 determines whether the processing in steps S310 to S340 has been completed for all provisionally registered individuals registered in the provisional registration data 640. If the processing has not been completed, the process returns to step S310, and the processing in steps S310 to S350 is executed again for the next provisionally registered individual.

[0071] According to the deletion process in Figure 13, if the registration period of a provisionally registered individual exceeds the allowable registration period before the number of provisional registrations reaches the threshold, the data related to that provisionally registered individual is deleted from the provisional registration data 640, thus eliminating the need to delete unnecessary provisionally registered individuals from the provisional registration data 640. Note that other conditions may be used as deletion conditions. For example, in the process in Figure 9, the date on which the provisional registration count Mpr of a provisionally registered individual was incremented may be stored, and if the period from that increment date exceeds the allowable period, the data related to that provisionally registered individual may be deleted.

[0072] According to the first embodiment described above, an embedding vector Vt can be obtained from a target image of the back of the animal to be identified using the deep distance learning model 620, and the individual can be identified using the distance between this embedding vector Vt and the registered embedding vector Vr. Furthermore, if the animal to be identified is an unregistered individual, it can be registered as a provisionally registered individual in the provisional registration data 640, and if the provisional registration data 640 satisfies the pre-set registration conditions, the provisionally registered individual can be registered as a registered individual in the registered data 630.

[0073] B. Second Embodiment: Figure 14 is an explanatory diagram showing the configuration of the individual identification system in the second embodiment. The individual identification system of the second embodiment differs from the first embodiment in that it is provided with multiple image sensors 400(p), deep distance learning models 620(p), registered data 630(p), and provisional registered data 640(p), but the other configurations are the same as those of the first embodiment. The 'p' at the end of these symbols is an ordinal number from 1 to P, and P is an integer of 2 or greater.

[0074] Multiple image sensors 400(p) can be combinations of various different image sensors as described in the first embodiment, or combinations of the same type of image sensors. When using combinations of the same type of image sensors, it is preferable to set at least one attribute of the multiple image sensors, such as the installation angle or field of view, to be different from each other. It is preferable that the relative positions of the respective sensor coordinate systems of the multiple image sensors 400(p) are known, and that the coordinate transformation matrices for any two sensor coordinate systems are known.

[0075] The number P of the image sensors 400(p) is an integer of 2 or more. In the second embodiment, P=2, and two image sensors 400, a depth sensor and an RGB sensor, are used to capture a video of the back of a cow CW. Preferably, the depth sensor and the RGB sensor have substantially the same shooting area and are configured to capture images at the same shooting timing. For example, it is possible to use one RGBD sensor including a depth sensor and an RGB sensor as the image sensor 400(p).

[0076] In the following explanation, a symbol with (p) appended to the end indicates that it corresponds to the ordinal p of the deep metric learning model 620(p). Similarly, the prefix "kind p" also indicates that it corresponds to the ordinal p of the deep metric learning model 620. For example, the kind p embedding vector registered in the p-th registered data 630(p) is called the "kind p registered embedding vector".

[0077] The learning process described in Figures 2 to 4 in the first embodiment is applied similarly to each individual deep metric learning model 620(p), so its explanation is omitted.

[0078] Furthermore, in the processing shown in Figure 3 for images captured by the RGB sensor, it is possible to detect multiple keypoints KP1 to KP3 from the back image BG captured by the RGB sensor, similar to the processing for images captured by the depth sensor, and set the cropping frame CF based on these keypoints. Alternatively, a coordinate transformation matrix between the depth sensor and the RGB sensor may be used to transform the multiple keypoints KP1 to KP3 detected from the back image BG of the depth sensor into coordinates in the sensor coordinate system of the RGB sensor, and then set the cropping frame CF from the transformed keypoints KP1 to KP3. The latter method is particularly useful when the depth sensor and RGB sensor are configured as a single RGBD sensor.

[0079] Figure 15 is an explanatory diagram showing an example of registration data 630(p) in the second embodiment. In this example, two types of image sensors 400(p) are used: a depth sensor and an RGB sensor. The number of target cows n is 4, and the number of target images TG N(p) is N(1)=N(2)=3. The registration data 630(p) includes a first registration data 630(1) created using the depth sensor and a second registration data 630(2) created using the RGB sensor. The first registration data 630(1) is the same as the registration data 630 in the first embodiment shown in Figure 5. The second registration data 630(2) contains registration embedding vectors Vr(2) for three target images TG for each of the same four individuals as in the first registration data 630(1).

[0080] Figure 16 is a flowchart showing the procedure for individual identification processing in the second embodiment. Steps S110a to S160a are modified versions of steps S110 to S160 of the individual identification processing in the first embodiment shown in Figure 6, and the processing in step S170 is almost the same as in the first embodiment.

[0081] In step S110a, the individual identification unit 530 uses two image sensors 500(p) to acquire N1 target images TG of the back of one target cow. Here, N1 is an integer of 1 or more. Similar to the first embodiment, the number of p-th type registration embedding vectors for each registered individual is defined as "N2" and distinguished from the number of target images TG of the target cow N1. The integer N2 is an integer of 2 or more. It is preferable that each of the integers N1 and N2 be a constant value independent of the ordinal number p.

[0082] In step S120a, the individual identification unit 530 uses the P trained deep distance learning models 620(p) to obtain N1 p-th kind embedding vectors for N1 target images TG.

[0083] Figure 17 is an explanatory diagram showing an example of an embedding vector in the second embodiment. The first type embedding vector Vt(1) is calculated using an image captured with a depth sensor and is the same as the embedding vector Vt shown in Figure 7. The second type embedding vector Vt(2) is calculated using an image captured with an RGB sensor.

[0084] In step S130a, the individual identification unit 530 calculates the distance between N1 p-th species embedding vectors and N2 p-th species registration embedding vectors for each registered individual. As a result, N1 × N2 p-th species distances are calculated for each registered individual. Furthermore, since there are n registered individuals, for each identified cow, n × N1 × N2 p-th species distances are calculated from the images captured by each image sensor 400(p).

[0085] Figure 18 is an explanatory diagram showing an example of the distance between the embedding vector and the registered embedding vector in the second embodiment. For the registered embedding vector Vr(p) shown in Figure 15, n=4 and N2=3, and for the embedding vector Vt(p) shown in Figure 17, N1=3, so for one target cow, 36 p-th distances L(p) are calculated from the images captured by each individual image sensor 400(p).

[0086] In step S140a, the individual identification unit 530 determines an integrated determination distance Ljt(ID) for each of the n registered individuals by integrating N1 × N2 p-th type distances. The integrated determination distance Ljt(ID) can be determined by, for example, one of the following methods.

[0087] <Method for determining the integrated judgment distance Ljt(ID) DM1> For each registered individual, the integrated determination distance Ljt(ID) is determined to be a value proportional to the sum of the following: the average value of the first type distance obtained by averaging M first type distances L(1) selected in ascending order from N1 × N2 first type distances L(1), and the average value of the second type distance obtained by averaging M second type distances L(2) selected in ascending order from N1 × N2 second type distances L(2). M is an integer between 2 and N1 × N2, and preferably an integer less than N1 × N2. Furthermore, it is preferable that integers N1 and N2 are set such that N1 × N2 is 3 or greater.

[0088] <Method for determining the integrated judgment distance Ljt(ID) DM2> For each registered individual, N1 × N2 first-type distances L(1) and corresponding N1 × N2 second-type distances L(2) are added together to obtain N1 × N2 summation results. From these N1 × N2 summation results, M summation results are selected in ascending order, and a value proportional to the average value of the selected M summation results is determined as the integrated determination distance Ljt(ID). The term "corresponding" means that the results are obtained using images taken at substantially the same shooting timing using two image sensors 400(p). Specifically, the first-type distance L(1) and second-type distance L(2) obtained using images with the same frame number taken by the depth sensor and RGB sensor, respectively, correspond to the distances that "correspond" to each other.

[0089] Figure 19 is an explanatory diagram showing examples of various determination distances in the second embodiment. The upper part of Figure 19 shows the integrated determination distance Ljt(ID) calculated by applying the determination method DM1 described above to the p-th type distance shown in Figure 18 and using M=3.

[0090] In step S150a, the individual identification unit 530 uses the integrated determination distance Ljt(ID) to determine whether the registration condition is met or whether the unregistered condition is met. If the registration condition is met, the process proceeds to step S160a, where it is determined whether the cow to be identified is one of several registered individuals. On the other hand, if the unregistered condition is met, the process proceeds to step S170, where the cow to be identified is determined to be unregistered and registered in the provisional registration data 640(p).

[0091] The process in step S170 in the second embodiment can be performed according to the detailed procedure in Figure 9 of the first embodiment. In this case, the "distance" used in steps S174 and S175 can be at least one of the following: an integrated distance combining the first type distance L(1) and the second type distance L(2), the first type distance L(1), and the second type distance L(2). That is, for example, one of the following can be used as the determination process in step S175. <First judgment process JP1> If the combined decision value determined using the first type distance L(1) and the second type distance L(2) is smaller than the combined threshold, the result is determined as Yes; if it is larger than the combined threshold, the result is determined as No. The combined decision value can be determined according to the same method as either of the determination methods DM1 or DM2 described above. <Second judgment process JP2> If two or more of the following are smaller than their respective thresholds, the result is determined to be Yes; otherwise, the result is determined to be No. <Third judgment process JP3> If the combined decision value determined using the first type distance L(1) and the second type distance L(2), the first type decision value determined using the first type distance L(1), and the second type decision value determined using the second type distance L(2) are all smaller than their respective thresholds, the result is determined as Yes; otherwise, the result is determined as No.

[0092] Figure 20 is a flowchart showing the detailed procedure of step S150a in the second embodiment. In step S151, the individual identification unit 530 determines whether the minimum value of the integrated determination distance Ljt(ID) is smaller than a preset integrated threshold Tht. If the minimum value of the integrated determination distance Ljt(ID) is smaller than the integrated threshold Tht, the process proceeds to step S161, where the registered individual corresponding to the minimum value of the integrated determination distance Ljt(ID) is identified as the individual of the cattle to be identified. In the example in Figure 19, the minimum value of the integrated determination distance Ljt(ID) corresponds to the registered individual whose individual ID is ID3, so this registered individual is identified as the individual of the cattle to be identified.

[0093] On the other hand, if the minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht, the process proceeds to step S152. The minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht when the cow to be identified is an unregistered individual, or when there is dirt on the back of the cow to be identified. In this case, the identification determination is performed using a determination value different from the integrated determination distance Ljt(ID).

[0094] Furthermore, if the minimum value of the integrated determination distance Ljt(ID) is equal to the integrated threshold Tht, the process may proceed from step S151 to step S161, or to step S152. This also applies to the other determination steps described later.

[0095] In step S152, the individual identification unit 530 determines a first-class determination distance Lj1(ID) that represents N1 × N2 first-class distances L(1) for each of the n registered individuals. The first-class determination distance Lj1(ID) is determined by, for example, one of the following methods.

[0096] <Method for determining the Type 1 classification distance Lj1 (ID) EM1> For each registered individual, M1 values ​​are selected in ascending order from N1 × N2 first-type distances L(1), and the first-type determination distance Lj1(ID) is determined to be a value proportional to the average value of the M1 first-type distances L(1). Here, M1 is an integer greater than or equal to 2 and less than or equal to N1 × N2, and preferably less than N1 × N2. Furthermore, it is preferable that the integers N1 and N2 are set such that N1 × N2 is 3 or greater.

[0097] <Method for determining the Type 1 classification distance Lj1 (ID) EM2> For each registered individual, the first type determination distance Lj1(ID) is determined to be a value proportional to the minimum value among N1 × N2 first type distances L(1).

[0098] In the second embodiment, the determination method EM1 described above is applied to the p-th type distance shown in Figure 18, and the first type determination distance Lj1(ID) is determined using M1=3. The first type determination distance Lj1(ID) thus determined is shown at the bottom of Figure 19.

[0099] In step S153, the individual identification unit 530 determines whether the minimum value of the first type determination distance Lj1(ID) is smaller than a preset first threshold Th1. If the minimum value of the first type determination distance Lj1(ID) is smaller than the first threshold Th1, the process proceeds to step S162, where the registered individual corresponding to the minimum value of the first type determination distance Lj1(ID) is identified as the individual of the cattle to be identified.

[0100] On the other hand, if the minimum value of the first type determination distance Lj1(ID) is greater than the first threshold Th1, the process proceeds to step S154. In step S154, the individual identification unit 530 determines a second type determination distance Lj2(ID) that represents N1 × N2 second type distances L(2) for each of the n registered individuals. The second type determination distance Lj2(ID) is determined in the same way as the first type determination distance Lj1(ID) determination methods EM1 and EM2 described above. The lower part of Figure 19 shows the second type determination distance Lj2(ID) calculated according to the same method as the determination method EM1 described above.

[0101] In step S155, the individual identification unit 530 determines whether the minimum value of the second type determination distance Lj2(ID) is smaller than the pre-set second threshold Th2. If the minimum value of the second type determination distance Lj2(ID) is smaller than the second threshold Th2, the process proceeds to step S163, where the registered individual corresponding to the minimum value of the second type determination distance Lj2(ID) is identified as the individual of the cattle to be identified.

[0102] On the other hand, if the minimum value of the second type determination distance Lj2(ID) is greater than the second threshold Th2, the process proceeds to step S156. In step S156, it is determined that the unregistered condition is met. In this case, step S170 in Figure 16 is executed.

[0103] In step S160a of Figure 16, it is determined whether the cow to be identified is one of several registered individuals registered in registration data 630(p). Note that the processing in steps S161 to S163 of Figure 20 can be considered equivalent to step S160a of Figure 16.

[0104] According to the processing shown in Figures 16 and 20 above, the individual cow to be identified can be determined from N1 × N2 p-th distances for each registered individual. In particular, in the second embodiment, individuals can be identified using various determination distances, including the integrated determination distance Ljt(ID). Furthermore, if the minimum value of each of the three determination distances Ljt(ID), Lj1(ID), and Lj2(ID) is greater than their respective thresholds Tht, Th1, and Th2, it can be determined that the cow to be identified is an unregistered individual.

[0105] C. Third Embodiment: Figure 21 is a flowchart showing the detailed procedure of step S150a in the third embodiment. The configuration of the apparatus in the third embodiment and the processing content in Figures 2, 3, and 6 are the same as in the second embodiment. The processing procedure in Figure 21 omits steps S154, S155, and S163 in Figure 20, and the other steps are the same as in the second embodiment.

[0106] In step S153 of the third embodiment, if the minimum value of the first type determination distance Lj1(ID) is greater than the first threshold Th1, the process proceeds to step S156, and it is determined that the unregistered condition is met.

[0107] The third embodiment also produces almost the same effect as the second embodiment. Furthermore, according to the process shown in Figure 21, if the minimum values ​​of the integrated determination distance Ljt(ID) and the first type determination distance Lj1(ID) are greater than the threshold, it can be determined that the target cow is an unregistered individual.

[0108] D. Fourth Embodiment: Figure 22 is a flowchart showing the detailed procedure of step S150a in the fourth embodiment. The configuration of the apparatus in the fourth embodiment and the processing content in Figures 2, 3, and 6 are the same as in the second and third embodiments. The processing procedure in Figure 22 further omits steps S152, S153, and S162 in Figure 21, and the other steps are the same as in the third embodiment.

[0109] In step S151 of the fourth embodiment, if the minimum value of the integrated determination distance Ljt(ID) is greater than the integrated threshold Tht, the process proceeds to step S156, and it is determined that the unregistered condition is met.

[0110] The fourth embodiment also produces substantially the same effects as the second and third embodiments. Furthermore, according to the process shown in Figure 22, if the minimum value of the integrated determination distance Ljt(ID) is greater than the threshold, it can be determined that the cow to be identified is an unregistered individual.

[0111] Note that the processing in Figures 20, 21, and 22 is the same in that it determines that the target cow is an unregistered individual if it satisfies the unregistered condition, which includes the minimum value of the integrated determination distance Ljt(ID) being greater than the integrated threshold Tht.

[0112] Other forms: This disclosure is not limited to the embodiments described above, and can be implemented in various forms without departing from its spirit. For example, this disclosure can also be implemented in the following forms (aspects). The technical features in the embodiments described above that correspond to the technical features in each of the forms described below can be replaced or combined as appropriate in order to solve some or all of the problems of this disclosure, or to achieve some or all of the effects of this disclosure. Furthermore, if such technical features are not described as essential in this specification, they can be deleted as appropriate.

[0113] (1) According to a first embodiment of the present disclosure, a method is provided for individual identification of an animal to be identified. This method includes (a) acquiring a target image relating to the dorsal side of the animal to be identified; (b) obtaining an embedding vector from the target image using a deep distance learning model; (c) calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) determining, using the distance, whether the animal to be identified is one of the plurality of registered individuals or an unregistered individual; (e) registering data relating to the animal to be identified as provisional registration data if the animal to be identified is an unregistered individual; and (f) transferring the provisional registration data to registered data if the provisional registration data satisfies pre-set transition conditions. This method uses a deep distance learning model to obtain embedding vectors from target images of the back of the target animal, and the distance between these embedding vectors and registered embedding vectors can be used to identify the individual. Furthermore, if the target animal is an unregistered individual, provisional registration data can be registered, and if the provisional registration data meets the pre-set registration conditions, the unregistered individual can be registered as a registered individual.

[0114] (2) In the above method, step (e) may include registering the animal to be identified as a provisionally registered individual in the provisional registration data, registering the embedding vector as provisional registration embedding vector data, and registering the number of provisional registrations of the provisionally registered individual, and step (f) may include transferring the data relating to the provisionally registered individual registered in the provisional registration data to the registered data when the number of provisional registrations of the provisionally registered individual exceeds a threshold. This method allows for the transfer of data about provisionally registered individuals from provisional registration data to registered data when the number of provisional registrations exceeds a threshold, thus enabling the registration of appropriate individuals into the registered data.

[0115] (3) The above method may further include (g) a step of deleting data relating to the provisionally registered individual from the provisionally registered data if the provisionally registered individual satisfies the predetermined deletion conditions. This method allows you to delete unnecessary provisionally registered individuals.

[0116] (4) In the above method, step (g) may include a step of deleting data relating to the provisionally registered individual from the provisional registration data if the registration period of the provisionally registered individual exceeds a predetermined allowable registration period before the number of provisional registrations reaches the threshold number. This method allows you to delete unnecessary provisionally registered individuals.

[0117] (5) In the above method, step (a) may include the step of acquiring N1 p-th type target images with respect to the back of the animal to be identified using a p-th type image sensor, when p is an ordinal number from 1 to 2 and N1 is an integer of 1 or more; step (b) may include the step of obtaining N1 p-th type embedding vectors from the N1 p-th type target images using a p-th deep distance learning model; and step (c) may include the step of calculating N1 × N2 p-th type distances between the N1 p-th type embedding vectors and the N2 p-th type registration embedding vectors using N2 p-th type registration embedding vectors that have been generated in advance for each of the n registered individuals, when n and N2 are integers of 2 or more. Step (d) may include (d1) a step of determining an integrated determination distance for each of the n registered individuals by integrating N1 × N2 first type distances and N1 × N2 second type distances; (d2) a step of determining that the registered individual corresponding to the minimum value of the integrated determination distance is an individual of the target animal if the minimum value of the integrated determination distance is smaller than a preset integration threshold; and (d3) a step of determining that the target animal is an unregistered individual if it satisfies an unregistered condition including that the minimum value of the integrated determination distance is larger than the integration threshold. This method uses two deep distance learning models to obtain two types of embedding vectors from two types of target images of the back of the target animal. The integrated decision distance, determined from the distance between these two embedding vectors and two registered embedding vectors, can then be used to identify the individual. Furthermore, if the unregistered condition is met, the target animal can be determined to be an unregistered individual.

[0118] (6) In the above method, when M is an integer of 2 or more and less than or equal to N1 × N2, the integrated determination distance for each of the n registered individuals may be a value proportional to the sum of the first type distance average value obtained by averaging the M first type distances selected in ascending order from the N1 × N2 first type distances and the second type distance average value obtained by averaging the M second type distances selected in ascending order from the N1 × N2 second type distances. This method allows for the calculation of an appropriate overall judgment distance.

[0119] (7) In the above method, step (d3) may include (d3-1) determining a first type determination distance that represents the N1 × N2 first type distances for each of the n registered individuals, and (d3-2) determining that the registered individual corresponding to the minimum value of the n first type determination distance is an individual of the animal to be identified, when the minimum value of the n first type determination distances is smaller than a preset first type threshold.

[0120] (8) In the above method, step (d3) may further include: (d3-3) when the minimum value of the first type determination distance is greater than the first type threshold, a step of determining a second type determination distance that represents the N1 × N2 second type distances for each of the n registered individuals; (d3-4) when the minimum value of the n second type determination distances is less than a preset second type threshold, a step of determining that the registered individual corresponding to the minimum value of the second type determination distance is an individual of the animal to be identified; and (d3-5) when the minimum value of the second type determination distance is greater than the second type threshold, a step of determining that the animal to be identified is an unregistered individual that does not fall under any of the plurality of registered individuals. This method allows for the determination of whether the animal being identified is a registered individual or an unregistered individual, based on the Type 1 and Type 2 determination distances.

[0121] (9) In the above method, step (d3) further, (d3-3) A step in which, if the minimum value of the first type determination distance is greater than the first type threshold, the animal to be identified is determined to be an unregistered individual. It may also include According to this method, it is possible to determine whether the animal being identified is a registered individual or an unregistered individual, depending on the Type 1 determination distance.

[0122] (10) In the above method, when M is an integer of 2 or more and less than or equal to N1 × N2, the first species determination distance for each of the n registered individuals may be a value proportional to the average first species distance obtained by averaging the M first species distances selected in ascending order from the N1 × N2 first species distances. This method allows for the calculation of an appropriate Class 1 classification distance.

[0123] (11) According to a second embodiment of the present disclosure, a computer program is provided which causes a processor to perform a process for individual identification of an animal to be identified. The computer program causes the processor to perform the following: (a) a process for acquiring a target image relating to the back of the animal to be identified; (b) a process for obtaining an embedding vector from the target image using a deep distance learning model; (c) a process for calculating the distance between the registered embedding vector and the embedding vector using registration data which includes pre-generated registration embedding vectors for each of a plurality of registered individuals; (d) a process for determining, using the distance, whether the animal to be identified is one of the plurality of registered individuals or an unregistered individual not registered in the registration data; (e) a process for registering data relating to the animal to be identified as provisional registration data if the animal to be identified is an unregistered individual; and (f) a process for transferring the provisional registration data to the registered data if the provisional registration data satisfies pre-set transfer conditions.

[0124] This disclosure can also be implemented in various forms other than those described above. For example, it can be implemented in the form of a device that performs individual identification processing, or a non-transitory storage medium on which a computer program is recorded. [Explanation of symbols]

[0125] 300... Information processing device, 310... Processor, 320... Memory, 330... Interface circuit, 340... Input device, 350... Display device, 400... Image sensor, 510... Target image acquisition unit, 520... Learning unit, 530... Individual identification unit, 610... Object recognition model, 620... Deep distance learning model, 630... Registered data, 640... Provisional registered data.

Claims

1. A method for identifying individual animals, (a) A step of acquiring a target image relating to the back of the animal to be identified, (b) A step of obtaining an embedding vector from the target image using a deep distance learning model, (c) A step of calculating the distance between a registration embedding vector and an embedding vector using registration data that includes registration embedding vectors generated in advance for each of a plurality of registered individuals, (d) A step of determining, using the distance, whether the animal to be identified is one of the multiple registered individuals or an unregistered individual, (e) If the animal to be identified is an unregistered individual, the step of registering data relating to the animal to be identified as provisional registration data, (f) A step of transferring the provisional registration data to the registered data when the provisional registration data satisfies the pre-set transfer conditions, Methods that include...

2. The method according to claim 1, The aforementioned step (e) is, The provisional registration data includes the steps of registering the animal to be identified as a provisionally registered individual, registering the embedding vector as provisional registration embedding vector data, and registering the number of times the provisionally registered individual has been provisionally registered. The aforementioned step (f) is, When the number of times a provisionally registered individual has been provisionally registered exceeds a threshold, the process includes transferring the data relating to the provisionally registered individual registered in the provisionally registered data to the registered data. method.

3. The method according to claim 2, further, (g) A step of deleting data relating to the provisionally registered individual from the provisional registration data when the provisionally registered individual satisfies the predetermined deletion conditions, Methods that include...

4. The method according to claim 3, The method includes step (g) of deleting data relating to the provisionally registered individual from the provisional registration data if the registration period of the provisionally registered individual exceeds a predetermined allowable registration period before the number of provisional registrations reaches the threshold number.

5. A method according to any one of claims 1 to 4, The above step (a) includes the step of acquiring N1 type p target images of the dorsal side of the animal to be identified using a type p image sensor, where p is an ordinal number from 1 to 2 and N1 is an integer of 1 or more. Step (b) includes the step of obtaining N1 p-th type embedding vectors from the N1 p-th type target images using a p-th deep distance learning model, Step (c) includes a step of calculating N1 × N2 p-th type distances between the N1 p-th type embedding vectors and the N2 p-th type embedding vectors, using N2 p-th type registration embedding vectors that have been generated in advance for each of the n registered individuals, where n and N2 are integers of 2 or more. The aforementioned step (d) is, (d1) A step of determining an integrated determination distance for each of the n registered individuals by integrating N1 × N2 first type distances and N1 × N2 second type distances, (d2) When the minimum value of the integrated determination distance is smaller than a preset integrated threshold, the step of determining that the registered individual corresponding to the minimum value of the integrated determination distance is an individual of the animal to be identified, (d3) A step of determining that the animal to be identified is an unregistered individual when the unregistered condition is met, which includes the minimum value of the integrated determination distance being greater than the integrated threshold, Methods that include...

6. The method according to claim 5, A method in which, when M is an integer of 2 or more and less than or equal to N1 × N2, the integrated determination distance for each of the n registered individuals is a value proportional to the sum of the first type distance mean value obtained by averaging M first type distances selected in ascending order from the N1 × N2 first type distances and the second type distance mean value obtained by averaging M second type distances selected in ascending order from the N1 × N2 second type distances.

7. The method according to claim 5, The aforementioned step (d3) is, (d3-1) A step of determining a first type determination distance that represents the N1 × N2 first type distances for each of the n registered individuals, (d3-2) A step in which, if the minimum value of the n first type determination distances is smaller than a preset first type threshold, the registered individual corresponding to the minimum value of the first type determination distance is determined to be an individual of the animal to be identified. Methods that include...

8. The method according to claim 7, The above step (d3) further, (d3-3) When the minimum value of the first type determination distance is greater than the first type threshold, a step is taken to determine a second type determination distance that represents the N1 × N2 second type distances for each of the n registered individuals, (d3-4) A step in which, if the minimum value of the n second type determination distances is smaller than a preset second type threshold, the registered individual corresponding to the minimum value of the second type determination distance is determined to be an individual of the animal to be identified. (d3-5) A step in which, if the minimum value of the second type determination distance is greater than the second type threshold, the animal to be identified is determined to be an unregistered individual that does not correspond to any of the multiple registered individuals, Methods that include...

9. The method according to claim 7, The above step (d3) further, (d3-3) A step in which, if the minimum value of the first type determination distance is greater than the first type threshold, the animal to be identified is determined to be an unregistered individual. Methods that include...

10. The method according to claim 7, A method wherein the first species determination distance for each of the n registered individuals is a value proportional to the minimum value among the N1 × N2 first species distances.

11. A computer program that causes a processor to perform a process for identifying individual animals, (a) A process to obtain a target image relating to the back of the animal to be identified, (b) A process to obtain an embedding vector from the target image using a deep distance learning model, (c) A process to calculate the distance between a registration embedding vector and another embedding vector using registration data that includes registration embedding vectors generated in advance for each of a plurality of registered individuals, (d) A process to determine, using the distance, whether the animal to be identified is one of the multiple registered individuals or an unregistered individual not registered in the registration data, (e) If the animal to be identified is an unregistered individual, the process of registering data relating to the animal to be identified as provisional registration data, (f) A process to transfer the provisional registration data to the registered data when the provisional registration data satisfies the pre-set transfer conditions, A computer program that causes the aforementioned processor to execute.

Images