Individual management system, individual management method, and program

Abstract

To provide an individual management system capable of easily identifying and managing an animal in each individual, and an individual management method and a program.SOLUTION: In an individual management system 1, an imaging part 20 images an animal. A display part 21 displays imaging data picked up by the imaging part 20. A segmentation part 30 segments individual imaging data of the animal from the imaging data picked up by the imaging part 20. A feature amount extraction part 31 extracts a feature amount of the individual on the basis of the imaging data segmented by the segmentation part 30. An identification part 33 inputs the feature amount extracted by the feature amount extraction part 31, identifies the individual having the feature amount, and outputs identification information on the individual. An adjustment part 36 adjusts a display state in the display part 21 about the individual identified by the identification part 33 on the basis of the position and size of the imaging data segmented by the segmentation part 30.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an individual management system, an individual management method, and a program.

Background Art

[0002] As the number of cows raised per household in Japan increases year by year, the increasing burden on workers for livestock raising and health management has become a problem. As the number of cows raised in large numbers increases, it is becoming increasingly difficult to perform customized individual management according to the growth or health status of the cows.

[0003] The bottleneck in individual management is the task of identifying the target cow. In order to distinguish individuals, tags with 10-digit ear tag numbers are attached to the ears of cows. Since the ear tag numbers on the ear tags are small and difficult to read, in order to read them, it is necessary to approach the cow, hold the ear in place, and visually confirm it. However, there are some cows whose ear hair gets in the way and the ear tag numbers cannot be recognized, or the tags are dirty and it is difficult to confirm the ear tag numbers. In addition, there are not a few cows with the tags fallen off and the ear tag numbers unknown.

[0004] In addition to checking the ear tag numbers, methods of reading two-dimensional barcodes or QR codes (registered trademarks) formed on the livestock's accessories have been proposed. In addition, technologies for identifying cows using implanted microchip-type electronic tags implanted subcutaneously in cows or bolus-type (ingestible type) electronic tags that are made to be ingested by cows and retained in the rumen or reticulum have been used. Methods of identifying individuals using non-contact wireless tags have also been proposed (see Patent Document 1). In these methods, the barcode or electronic tag is read using a reader, or the cow is induced to a special gate to recognize the electronic tag to identify the individual.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, both methods require specialized equipment and still involve randomly examining cattle. Furthermore, there's the challenge of difficulty in reading the electronic labels of individual cattle when multiple cattle are close together. Therefore, the key to solving this problem lies in efficiently finding and identifying a target cattle from a large group without the need for specialized equipment. Even if an individual is identified, some may carry pathogens, so it's necessary to quickly obtain information about the identified individual. These problems can arise not only with cattle but also with other livestock such as pigs, and even with companion animals like dogs or cats when identifying them individually.

[0007] This invention was made under the circumstances described above, and aims to provide an individual animal management system, an individual animal management method, and a program that can easily identify and manage animals on an individual basis. [Means for solving the problem]

[0008] To achieve the above objective, the individual management system according to the first aspect of the present invention is: The imaging unit for imaging animals, A display unit that displays the imaging data captured by the imaging unit, From the imaging data captured by the imaging unit, an extraction unit extracts imaging data of individual animals, A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit, An identification unit identifies an individual by pattern recognition using the features extracted by the feature extraction unit and outputs identification information for that individual. An identification information input unit for inputting identification information of the target object, A determination unit that determines whether the identification information of an individual identified by the identification unit matches the identification information input to the identification information input unit, A search unit searches for an individual corresponding to the identification information input to the identification information input unit based on the determination result of the determination unit, Based on the position and size of the image data extracted by the extraction unit, the identification unit identifies the data. The target searched by the search unit An adjustment unit for adjusting the display state in the display unit for an individual, It is equipped with.

[0010] The adjustment unit is, Based on the position and size of the image data extracted by the extraction unit, the display state of the display unit is adjusted so that the individual searched by the search unit is highlighted. It would be acceptable to do so.

[0011] An individual management system according to a second aspect of the present invention is: The imaging unit for imaging animals, A display unit that displays the imaging data captured by the imaging unit, From the imaging data captured by the imaging unit, an extraction unit extracts imaging data of individual animals, A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit, An identification unit identifies an individual by pattern recognition using the features extracted by the feature extraction unit and outputs identification information for that individual. An attribute information database that stores individual identification information and attribute information in association, An attribute information reading unit reads attribute information corresponding to the identification information of an individual identified by the identification unit when the features extracted by the feature extraction unit are input to the identification unit, from the attribute information database. An adjustment unit adjusts the display state in the display unit for the individual identified by the identification unit based on the position and size of the image data extracted by the extraction unit, Equipped with, The adjustment unit is, Based on the position and size of the image data extracted by the extraction unit, the image representing the attribute information read from the attribute information database for the individual identified by the identification unit is aligned with the image data. 。

[0012] The aforementioned cutting unit extracts the image data of the individual's face and the image data of its whole body. The adjustment unit detects the orientation of the individual's body based on the position of the image data of the individual's face and the position of the image data of the entire body, obtained by the cutting unit. Based on the orientation of the detected individual's body, an image showing attribute information is aligned with the imaging data. It would be acceptable to do so.

[0013] The imaging unit and the display unit are implemented in a mobile terminal. This may also be the case.

[0014] The individual management system according to the second aspect of the present invention includes an imaging unit for imaging an animal, a cutting unit for cutting out the imaging data of an individual animal from the imaging data captured by the imaging unit, a feature quantity extraction unit for extracting the feature quantity of the individual based on the imaging data cut out by the cutting unit, an identification unit for identifying an individual by pattern recognition using the feature quantity extracted by the feature quantity extraction unit and outputting the identification information of the individual, a moving body on which the imaging unit is mounted and which is movable, a command generation unit for moving the moving body to near the individual identified by the identification unit based on the position of the imaging data cut out from the cutting unit, generating a command to perform an operation on the individual, and outputting the command to the moving body, and is provided with.

[0015] a feed information database for associating and storing the identification information of an individual and feed information, a feed information reading unit for reading out the feed information corresponding to the identification information of the individual identified by the identification unit from the feed information database, and is provided with, The moving body is provided with an automatic feeding unit for feeding the individual identified by the identification unit based on the feed information read out by the feed information reading unit. This may also be the case.

[0016] is provided with a feature quantity database for associating and storing the identification information of an individual and the feature quantity extracted by the feature quantity extraction unit, The identification unit performs machine learning for distinguishing an individual animal based on the relationship between the feature quantity and the identification information stored in the feature quantity database, and outputs the identification information corresponding to the input feature quantity when a feature quantity is input. This may also be the case.

[0017] The aforementioned cropping unit uses a first deep learning model trained for general object recognition to crop the imaging data of an individual, The feature extraction unit extracts individual features using a second deep learning model trained for object recognition. The aforementioned identification unit outputs identification information corresponding to the feature quantities using a support vector machine that performs one-to-other classification. It would be acceptable to do so.

[0018] The first deep learning model described above is YOLO (You Only Look Once), The second deep learning model mentioned above is VGG16. It would be acceptable to do so.

[0019] The aforementioned imaging data is video data, The imaging unit sends frame images of the video data to the extraction unit as imaging data, either through an input operation or intermittently. The extraction unit, each time it receives the frame image, extracts the imaging data of an individual animal from the frame image. Each time the image data is extracted by the cropping unit, the feature extraction unit extracts the features of that individual based on the image data extracted by the cropping unit. The identification unit identifies an individual possessing a feature that is extracted by the feature extraction unit each time that feature is extracted. It would be acceptable to do so.

[0020] The feature extraction unit extracts features related to the movements of individual animals based on the body movements of the individual animals detected from the multiple frame images. It would be acceptable to do so.

[0021] This invention 4 Individual management methods from this perspective are: A method of individual management performed by an individual management system, The imaging unit captures an image of an animal, and the imaging data captured by the imaging unit is displayed on the display unit. A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which identifies an individual by pattern recognition using the features extracted in the feature extraction step and outputs the identification information of that individual, An identification information input step in which identification information of the target object is entered, A determination step to determine whether the identification information of the individual identified in the above identification step matches the identification information entered in the above identification information input step, A search step that searches for an individual corresponding to the identification information entered in the identification information input step, based on the determination result in the determination step, Based on the position and size of the image data extracted in the extraction step, the data identified in the identification step , the target searched in the above search step An adjustment step to adjust the display state in the display unit for an individual, Includes. The individual management method according to the fifth aspect of the present invention is: A method of individual management performed by an individual management system, The imaging unit captures an image of an animal, and the imaging data captured by the imaging unit is displayed on the display unit. A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which identifies an individual by pattern recognition using the features extracted in the feature extraction step and outputs the identification information of that individual, An attribute information reading step involves reading attribute information corresponding to the identification information of the individual identified in the aforementioned identification step from an attribute information database that stores the individual's identification information and attribute information in association. The adjustment step includes adjusting the display state on the display unit for the individual identified in the identification step, based on the position and size of the imaging data extracted in the extraction step, In the adjustment step described above, Based on the position and size of the image data extracted in the extraction step, the image representing the attribute information read from the attribute information database for the individual identified in the identification step is aligned with the image data.

[0022] This invention 6 Individual management methods from this perspective are: A method of individual management performed by an individual management system, An imaging step in which an imaging unit mounted on a mobile body is used to image an animal, A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which takes the features extracted in the feature extraction step as input, identifies an individual possessing those features, and outputs identification information for that individual, A command generation step which generates a command to move the mobile body to the vicinity of the individual identified in the identification step and to perform an operation on that individual, based on the position of the image data extracted in the extraction step, and outputs this command to the mobile body, Includes.

[0023] This invention 7 The program relating to this perspective is Computers, From the imaging data captured by the imaging unit that images animals, an extraction unit extracts the imaging data of individual animals. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. Identification information input unit for inputting identification information of the target object. A determination unit that determines whether the identification information of an individual identified by the identification unit matches the identification information input to the identification information input unit. A search unit searches for an individual corresponding to the identification information input to the identification information input unit, based on the determination result of the determination unit. Based on the position and size of the image data extracted by the cropping unit, the display unit that displays the image data captured by the imaging unit is identified by the identification unit. The target searched by the search unit Adjustment unit for adjusting the display status of individual items. To make it function as such. A program relating to the eighth aspect of the present invention is: Computers, From the imaging data captured by the imaging unit that images animals, an extraction unit extracts the imaging data of individual animals. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. An attribute information database that stores individual identification information and attribute information in association. When the feature quantities extracted by the feature quantity extraction unit are input to the identification unit, the attribute information reading unit reads attribute information corresponding to the identification information of the individual identified by the identification unit from the attribute information database. Based on the position and size of the image data extracted by the cropping unit, an adjustment unit adjusts the display state for the individual identified by the identification unit in the display unit that displays the image data captured by the imaging unit. To make it function as, The adjustment unit is, Based on the position and size of the image data extracted by the cropping unit, the image representing the attribute information read from the attribute information database for the individual identified by the identification unit is aligned with the image data.

[0024] This invention 9 The program relating to this perspective is Computers, An imaging unit mounted on a mobile vehicle captures images of animals, and an extraction unit extracts the image data of individual animals from the captured image data. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. A command generation unit generates a command to move the moving body near the individual identified by the identification unit and to perform an operation on that individual, based on the position of the image data extracted from the cutting unit, and outputs this command to the moving body. To make it function as such. [Effects of the Invention]

[0025] According to the present invention, individual animals can be automatically identified based on their external characteristics obtained from imaging data of those animals, and the display state of the identified animals can be adjusted. This makes it easy to identify and manage each animal individually. [Brief explanation of the drawing]

[0026] [Figure 1] This is a block diagram showing the configuration of the individual management system according to Embodiment 1 of the present invention. [Figure 2] (A) is a diagram showing an example of imaging data captured by the imaging unit. (B) is a diagram showing an example of imaging data extracted by the extraction unit. [Figure 3] This figure shows an example of image data containing multiple cows. [Figure 4] The block diagram shows the hardware configuration of the mobile terminals that make up the individual management system in Figure 1. [Figure 5] Figure 1 is a flowchart of the processing in the registration mode of the individual management system. [Figure 6]Figure 1 is a flowchart of the processing in the exploration mode of the individual management system. [Figure 7] This is a block diagram showing the configuration of the individual management system according to Embodiment 2 of the present invention. [Figure 8] Figure 1 is a flowchart of the processing in attribute display mode of the individual management system. [Figure 9] This figure shows an example of the attribute information that will be displayed. [Figure 10] This is a block diagram showing the configuration of the individual management system according to Embodiment 3 of the present invention. [Figure 11] Figure 10 is a flowchart of the automated feeding process in the individual management system. [Modes for carrying out the invention]

[0027] Embodiments of the present invention will be described in detail below with reference to the drawings. In each drawing, the same or equivalent parts are denoted by the same reference numerals.

[0028] Embodiment 1 First, Embodiment 1 of the present invention will be described. The individual management system 1 according to this embodiment identifies individual cattle, for example, that are raised on a farm. The cattle may be dairy cows, beef cattle, or both.

[0029] As shown in Figure 1, the individual management system 1 according to this embodiment comprises a mobile terminal 2 and a server system 3. The mobile terminal 2 is a terminal small enough for an operator to carry, such as a smartphone. The server system 3 is an information processing device installed in a predetermined location and capable of data communication with the mobile terminal 2.

[0030] The mobile terminal 2 comprises an imaging unit 20, a display unit 21, and an identification information input unit 22. That is, the imaging unit 20, the display unit 21, and the identification information input unit 22 are implemented in the mobile terminal 2. The server system 3 comprises an extraction unit 30, a feature quantity calculation unit 31, a feature quantity database 32, an identification unit 33, a judgment unit 34, a search unit 35, and an adjustment unit 36. In other words, the individual management system 1 comprises an imaging unit 20, a display unit 21, an identification information input unit 22, an extraction unit 30, a feature quantity extraction unit 31, a feature quantity database 32, an identification unit 33, a judgment unit 34, a search unit 35, and an adjustment unit 36.

[0031] (Imaging Department) The imaging unit 20 captures images of the cow, which is the animal to be identified. Since the imaging unit 20 is implemented in the mobile terminal 2, it can capture images of the cow from various angles and distances. The imaging unit 20 is capable of capturing images at various magnifications. By changing the magnification, the imaging unit 20 can capture images of cows individually or in herds. The imaging unit 20 captures, for example, a facial image of a cow viewed from the front, as shown in Figure 2(A).

[0032] The imaging data captured by the imaging unit 20 is sent to the server system 3. The captured imaging data may be still image data or video data. If the imaging data is video data, the imaging unit 20 may send each frame image of the video data to the server system 3, or it may send frame images of the video data to the server system 3 as imaging data in response to an input operation or continuously. The mode in which frame images are sent to the server system 3 when an input operation is performed is called step mode, and the mode in which frame images are sent to the server system 3 continuously is called seamless mode.

[0033] (Display) The display unit 21 displays the imaging data captured by the imaging unit 20. The operator operating the mobile terminal 2 can view the displayed data, position the target to be imaged within the imaging field of view, and focus to capture the image. As will be described later, the display state of the imaging data on the display unit 21 can be adjusted by the adjustment unit 36 ​​of the server system 3.

[0034] (Identification information input section) The identification information input unit 22 inputs the identification information of the target cow, namely the ear tag number. The ear tag number is a number attached to the ear tag assigned to each individual cow. The ear tag number allows for the identification of which cow it is. For cows registered by their ear tag number, records such as birth date, sex, and movement history (location and date) are registered in the management database. In addition, some farms register records such as the feed menu tailored to each cow's developmental stage or calving history in the management database. Furthermore, veterinarians create medical records for each individual cow and manage treatment history using the ear tag number. In this way, if the ear tag number is known, information about the individual cow can be accessed. The identification information entered into the identification information input unit 22 is sent to the server system 3.

[0035] (Cut-out section) The cropping unit 30 extracts individual cow image data from the image data captured by the imaging unit 20. If multiple cows are captured in the image data, each cow is extracted individually. For example, the cropping unit 30 extracts the frontal face image of the cow shown in Figure 2(B) from the image shown in Figure 2(A). Furthermore, when frame images of the image data are transmitted sequentially from the imaging unit 20, the cropping unit 30 extracts individual cow image data for each frame image. In other words, each time the cropping unit 30 receives a frame image, it extracts the image data of the individual animal from the frame image.

[0036] The extraction unit 30 extracts image data of individual animals using a first deep learning model trained for general object recognition. "General object recognition" refers to the recognition of objects or scenes contained in an image of an unconstrained real-world scene by their general names. In contrast, "specific object recognition" is a recognition technique for objects with exactly the same shape. Specific object recognition can be used for objects whose shape is somewhat fixed, such as a human face, even if the shape is not completely fixed, such as an industrial product. When detecting markers or human faces, specific object recognition using natural feature points is mainly used. However, when targeting the entire body of a cow, the appearance varies greatly depending on the breed's pattern, coat color, and posture, and is not as well known as a human face, so specific object recognition is often unable to recognize it. For this reason, in this embodiment, a model trained for general object recognition is employed.

[0037] Furthermore, the first deep learning model is pre-trained. Generally, deep learning requires a large amount of training data. However, preparing large amounts of training data requires a lot of effort. Therefore, in this embodiment, transfer learning using a pre-trained first deep learning model is employed. The first deep learning model consists of a convolutional neural network trained using a large dataset. By performing additional training on a convolutional neural network that has already been trained with large amounts of training data, it becomes possible to achieve high accuracy with less training data.

[0038] In this embodiment, the first deep learning model is YOLO (You Only Look Once). YOLO works by inputting the entire input image into a convolutional neural network, extracting objects from the image data, and directly calculating their position and size. YOLO is characterized by its high processing speed. In this embodiment, the extracted image data from YOLO is rectangular, as shown in Figure 2(B).

[0039] In this way, the cropping unit 30 also detects the position and size of the image data cropped from the image data sent from the imaging unit 20. The position and size of the cropped image data are transmitted as supplementary information to the feature extraction unit 31, the identification unit 33, the determination unit 34, the search unit 35, and the adjustment unit 36. The detected position and size of the cropped image data are used in the adjustment unit 36 ​​to adjust the display state of the display unit 21.

[0040] (Feature extraction unit) The feature extraction unit 31 extracts the features of an individual based on the imaging data extracted by the cropping unit 30. The feature extraction unit 31 extracts the features of an individual animal using a second deep learning model trained to recognize objects. In this embodiment, the second deep learning model can be, for example, VGG16 for general object recognition. However, the present invention is not limited to this. As the second deep learning model, VGG19 or ResNet50, which are also general object recognition models, may be used. These are models trained on a large dataset called ImageNet.

[0041] Furthermore, it is possible to use FaceNet and VGGFace, which are face recognition models for recognizing human faces. When FaceNet is used, the output of the layer designed for feature extraction is used as the feature vector. In addition, with VGG16, VGG19, ResNet50, and VGGFace, the output of the pooling layer immediately preceding the fully connected layer is used as the feature vector.

[0042] Each time the image data is extracted by the cropping unit 30, the feature extraction unit 31 extracts the features of that individual based on the image data cropped by the cropping unit 30. The extracted features are normalized so that the mean is 0 and the variance is 1. From one image data, for example, 512 features are extracted as one set of features. Since the second deep learning model is also trained using a large dataset, the output features can be of high quality.

[0043] (Feature Database) The feature database 32 stores the identification information input to the identification information input unit 22 and the features extracted by the feature extraction unit 31 in association with each other. In this embodiment, the individual management system 1 has two operating modes for the feature database 32: a registration mode and a search mode. In registration mode, the feature database 32 stores the features extracted by the feature extraction unit 31 and the identification information input to the identification information input unit 22 in association with each other. In search mode, the feature database 32 reads the stored information from the identification unit 33. In registration mode, the feature database 32 stores the identification information and the features each time a feature is extracted by the feature extraction unit 31. One set of features is not limited to one set of identification information. Multiple sets of features can be stored to correspond to multiple sets of imaging data.

[0044] (Identification unit) The identification unit 33 corresponds to the classifier in the transfer learning described above. The identification unit 33 identifies individuals by pattern recognition using the features extracted by the feature extraction unit 31. More specifically, the identification unit 33 performs machine learning to distinguish individual cows based on the relationship between the features stored in the feature database 32 and the identification information. Through this pattern recognition machine learning, the identification unit 33 is trained to take the features of a cow's face as input and output the identification information corresponding to those features, i.e., the cow's ear tag number.

[0045] When operating in search mode, the identification unit 33 receives the features extracted by the feature extraction unit 31, performs classification based on those features, and outputs the ear tag number corresponding to those features. The identification unit 33 outputs identification information corresponding to the features, i.e., the ear tag number of the cow, using a support vector machine that performs one-to-other classification, i.e., multi-class classification. If there are C cows, i.e., C classes, the identification unit 33 can have C classifiers that perform binary classification between the correct class and other classes. During inference, all classifiers are made to make predictions, and the class with the highest prediction probability of being correct is taken as the final prediction result, thereby achieving multi-class classification. Each correct class is associated with a corresponding ear tag number, and the ear tag number of the correct class is output. The identification unit 33 outputs identification information of the individual possessing the feature each time a feature is extracted by the feature extraction unit 31.

[0046] (Judgment Department) When operating in search mode, the determination unit 34 inputs the features extracted by the feature extraction unit 31 to the identification unit 33 to determine whether the identification information of the individual identified by the identification unit 33 matches the identification information input to the identification information input unit 22. The determination unit 34 performs this determination each time the identification information of an individual is output from the identification unit 33.

[0047] (Exploration Department) The search unit 35 searches for the target cow corresponding to the identification information input to the identification information input unit 22, based on the determination result of the determination unit 34. Each time the determination unit 34 performs a determination, the search unit 35 checks whether the determination result of the determination unit 34 matches, that is, whether the target cow has been found. If the identification information of the cow identified by the identification unit 33 in the determination unit 34 matches the identification information input to the identification information input unit 22, the search unit 35 outputs to the adjustment unit 36 ​​that the target cow has been found.

[0048] As shown in Figure 3, if the image data contains multiple cows, the cropping unit 30 crops the image data for each individual cow, and the feature extraction unit 31 extracts features for each individual cow. The identification unit 33 takes the extracted features as input to each individual from which features have been extracted by the feature extraction unit 31 and outputs the corresponding individual identification number for that cow. The determination unit 34 determines whether the identification information of each individual matches the identification information input to the identification information input unit 22, for each individual from which an identification number is output by the identification unit 33. Based on the determination result for each individual by the determination unit 34, the search unit 35 identifies the individual whose identification information matches as the individual corresponding to the identification information input to the identification information input unit 22.

[0049] (adjustment section) The adjustment unit 36 ​​adjusts the display state in the display unit 21 for the individual identified by the identification unit 33, based on the position and size of the image data of the individual extracted by the cropping unit 30. More specifically, the adjustment unit 36 ​​adjusts the display state of the display unit 21 so that the target individual searched by the search unit 35 is highlighted in the displayed image data, based on the position and size of the image data extracted by the cropping unit 30. For example, as shown in Figure 3, in the image data P1, a dotted line frame is displayed surrounding the image data P2 of the target cow's face. This dotted line frame locks onto the target cow, i.e., the target individual, and continues to highlight it. The highlighting may be expressed in a way other than a dotted line frame, for example, a solid line frame, or the area around the target cow may be color-coded. The method of highlighting is not limited to this.

[0050] The adjustment unit 36 ​​may simply display the ear tag number of the searched individual. Each time an individual is identified by the identification unit 33, the adjustment unit 36 ​​adjusts the display state on the display unit 21 for the individual identified by the identification unit 33, based on the position and size of the image data extracted by the cropping unit 30.

[0051] (Hardware configuration) The mobile terminal 2 shown in Figure 1 can be realized, for example, by a computer having the hardware configuration shown in Figure 4 implementing a software program.

[0052] Specifically, the mobile terminal 2 comprises a CPU (Central Processing Unit) 11 that controls the entire device, a main memory unit 12 that operates as a workspace for the CPU 11, an external memory unit 13 that stores the operating program for the CPU 11, a camera 14, a display 15, an operation input unit 16, a communication interface 17, and an internal bus 18 that connects these.

[0053] The CPU 11 is a processor (arithmetic unit) that executes software programs (hereinafter simply referred to as "programs"). The program 19 is read into the main memory unit 12 from the external memory unit 13. The CPU 11 executes the program 19 stored in the main memory unit 12. This enables the functions of the imaging unit 20, the display unit 21, and the identification information input unit 22.

[0054] The main memory unit 12 consists of RAM (Random Access Memory), etc. The CPU 11's program 19 is loaded into the main memory unit 12 from the external memory unit 13. The main memory unit 12 is also used as the CPU 11's work area (temporary data storage area).

[0055] The external storage unit 13 is composed of non-volatile memory such as flash memory or a hard disk. The external storage unit 13 has a program 19 pre-stored in it for the CPU 11 to execute.

[0056] Camera 14 performs imaging. Camera 14 can capture still images or videos. The functions of the imaging unit 20 are realized through the coordinated operation of the CPU 11 and camera 14.

[0057] The display 15 is a display device that displays images. The functions of the display unit 21 are realized through the coordinated operation of the CPU 11 and the display 15.

[0058] The operation input unit 16 is a man-machine interface that inputs operation information from the operator operating the mobile terminal 2. The operation of the identification information input unit 22 is realized through the coordinated operation of the CPU 11 and the operation input unit 16.

[0059] The communication interface 17 is an interface to the communication network. Data communication with communication terminals and with other server computers takes place via the communication interface 17. In this embodiment, data communication with the server system 3 is possible via the communication interface.

[0060] The hardware configuration of server system 3 is the same as that of mobile terminal 2 shown in Figure 4, in that it includes a CPU 11, main memory 12, external memory 13, and communication interface 17. The functions of the extraction unit 30, feature extraction unit 31, feature database 32, identification unit 33, determination unit 34, search unit 35, and adjustment unit 36 ​​are realized through the coordinated operation of the CPU 11, main memory 12, external memory 13, and communication interface 17. Server system 3 does not need to include a camera 14, display 15, and operation input unit 16. Server system 3 is connected to mobile terminal 2 via the communication interface 17, enabling data communication.

[0061] The program 19 executed on the mobile terminal 2 is a program that causes the computer having the above configuration to function as an imaging unit 20, a display unit 21, and an identification information input unit 22. The program 19 executed on the server system 3 is a program that causes the computer having the above configuration to function as a cropping unit 30, a feature extraction unit 31, a feature database 32, an identification unit 33, a determination unit 34, a search unit 35, and an adjustment unit 36.

[0062] Next, the operation of the individual management system 1 according to this embodiment, that is, the individual management method performed by the individual management system 1, will be described. The operation of the individual management system 1 is divided into the registration mode processing shown in Figure 5 and the search mode processing shown in Figure 6.

[0063] (Processing in registration mode) First, let's explain the processing in registration mode. As shown in Figure 5, in the individual management system 1, the imaging unit 20 captures an image of a cow on the mobile terminal 2, and the display unit 21 displays the image data captured by the imaging unit 20 (step S1; imaging and display step). In registration mode, only one cow is imaged at a time, and in this embodiment, as shown in Figure 2(A), a frontal image of the cow's face is captured. Here, multiple image data can be obtained for the same individual.

[0064] Next, in the mobile terminal 2, the identification information input unit 22 inputs the identification information of the captured cow (step S2; identification information input step). Here, the ear tag number attached to the cow's ear tag is read and the read ear tag number is input. The input identification information is sent to the server system 3. Note that the order of steps S1 and S2 may be reversed. The imaging data and identification information captured by the imaging unit 20 are sent to the server system 3.

[0065] Next, in the server system 3, the extraction unit 30 extracts individual cow image data from the image data captured by the imaging unit 20 (step S3; extraction step). If multiple image data sets are obtained for the same individual, individual cow image data is extracted for each image data set.

[0066] Next, in the server system 3, the feature extraction unit 31 extracts the features of the individual based on the imaging data extracted in the above extraction step (step S4; feature extraction step). If multiple imaging data sets are obtained for the same individual, the features of the individual cow are extracted for each imaging data set.

[0067] Next, in the server system 3, the feature database 32 stores the individual identification information and the features extracted by the feature extraction unit 31 in association, and registers the individual cow (step S5; registration step). If multiple imaging data are obtained for the same individual, multiple sets of features are stored for the same identification information.

[0068] Next, in the server system 3, the identification unit 33 determines whether or not the storage of data into the feature database 32 is complete (step S6). This determination is made based on whether or not the storage of identification information and features for all cows to be identified is complete. If there are still individuals whose data has not yet been stored in the feature database 32, this determination is rejected. If the determination is rejected (step S6; No), the individual management system 1 returns to step S1, and the processing in steps S1 to S5 is executed for cows that have not yet been registered.

[0069] Once registration of all individuals into the feature database 32 is complete (Step S6; Yes), the identification unit 33 performs machine learning to distinguish between animal individuals based on the relationship between the features stored in the feature database 32 and the identification information (Step S7; Learning Step). Through this machine learning, the internal parameters of the identification unit 33 are adjusted so that when it receives features extracted by the feature extraction unit 31 as input, it outputs the identification information of the individual corresponding to those features.

[0070] (Processing in exploration mode) Next, the processing in the search mode will be described. As shown in Figure 6, the individual management system 1, on the mobile terminal 2, has an imaging unit 20 that images a cow, and a display unit 21 that displays the image data captured by the imaging unit 20 (step S11; imaging and display step). The image data captured by the imaging unit 20 is sent to the server system 3. One cow or multiple cows may be imaged at the same time.

[0071] Next, in the mobile terminal 2, the identification information input unit 22 inputs the identification information of the individual cow to be searched (step S12; identification information input step). The input identification information is sent to the server system 3. Note that the order of steps S11 and S12 may be reversed.

[0072] Next, in the server system 3, the extraction unit 30 extracts image data of individual cows from the image data captured by the imaging unit 20 (step S13; extraction step). If the image data contains multiple individual cows, image data is extracted for each individual.

[0073] Next, in the server system 3, the feature extraction unit 31 extracts the features of each individual based on the imaging data extracted in the above extraction step (step S14; feature extraction step). If the imaging data contains images of multiple cows, features are calculated for each individual.

[0074] Next, in the server system 3, the identification unit 33 identifies individuals by pattern recognition using the features extracted in the feature extraction step (step S15; identification step). If the imaging data contains images of multiple cows, identification information is obtained for each individual.

[0075] Next, in the server system 3, the determination unit 34 determines whether the identification information of the individual identified by the identification unit 33 matches the identification information input to the identification information input unit 22 (step S16; determination step). If multiple individual cows are captured in the imaging data, the determination is performed for each individual.

[0076] Next, in the server system 3, the search unit 35 determines whether the identification number matches in the determination unit 34 and whether the target cow has been found (step S17). Here, the determination is made for each individual captured in the imaging data, and if even one matches the target cow, the determination is affirmed. If the target cow has not been found (step S17; No), the individual management system 1 returns to the imaging and display step of step S1. Subsequently, as long as the target cow has not been found (step S17; No), steps S11 to S17 are repeated, and the search for the target cow captured in the imaging data continues.

[0077] When the target cow is found (Step S17; Yes), the adjustment unit 36 ​​in the server system 3 adjusts the display state on the display unit 21 for the individual identified in the identification step, based on the position and size of the image data extracted in the extraction step (Step S18; Adjustment step). As a result, the display unit 21 of the mobile terminal 2 displays the adjusted display state. For example, the target cow is highlighted.

[0078] After step S18 is completed, the individual management system 1 makes a decision to end the search (step S19). This decision is made according to the operation input on the mobile terminal 2 or the server system 3. If the search is not to be ended (step S19; No), the individual management system 1 returns to step S11 and repeats steps S11 to S17 to search for the target cow. If found, steps S18 and S19 are executed to adjust the display state of the target cow. This continues the highlighting of the target individual. By continuing the highlighting, the display can be set to a state where the system is locked onto that individual.

[0079] If the search is terminated (step S19; Yes), the individual management system 1 terminates the processing in the search mode.

[0080] As described in detail above, the individual management system 1 according to this embodiment can automatically distinguish individual animals based on their external characteristics obtained from imaging data of those animals, and adjust the display state of the distinguished animals. This makes it easy to identify and manage animals on an individual basis.

[0081] According to this embodiment, individual cattle can be identified and managed without the need for special machines such as tag readers.

[0082] According to this embodiment, it is possible to locate the target cow without approaching it, making it suitable for preventing infectious diseases. Examples of infectious diseases include bovine leukemia.

[0083] In this embodiment, individual identification is performed using the cow's face. However, the present invention is not limited to this. For example, individual identification may be performed using the cow's entire body. Alternatively, the cow's face and entire body may be identified separately to identify the individual.

[0084] Alternatively, individual cows may be identified using features related to their individual movements. In this case, the feature extraction unit 31 needs to calculate features related to the individual cow's movements, such as gait, and these features are calculated based on the body movements of the individual detected from multiple frame images.

[0085] Furthermore, in this embodiment, the system searches for a single cow whose identification information has been entered. However, it is not limited to this. Multiple cows may be searched simultaneously.

[0086] Embodiment 2 Next, Embodiment 2 of the present invention will be described. In the individual management system 1 according to this embodiment, the configuration and operation for registering data in the feature database 32 in registration mode are the same as those of the individual management system 1 according to the above embodiment. The individual management system 1 according to this embodiment uses augmented reality technology to display a superimposed display of the searched individual cow and the attribute information about that individual.

[0087] As shown in Figure 7, the individual management system 1 according to this embodiment differs from the individual management system 1 according to Embodiment 1 in that the server system 3 includes an attribute information database 40, an attribute information reading unit 41, and an adjustment unit 42 instead of a determination unit 34 and a search unit 35.

[0088] The attribute information database 40 stores individual identification information and attribute information in association. Individual attribute information includes, for example, information about the pathogens that the individual carries.

[0089] When the attribute information reading unit 41 is operating in attribute information display mode, it reads attribute information corresponding to the identification information output from the identification unit 33 based on the input of features extracted by the feature extraction unit 31 from the attribute information database 40.

[0090] Based on the position and size of the image data extracted by the cropping unit 30, the adjustment unit 42 aligns the image data with the image data displaying the attribute information read from the attribute information database 40 for the individual identified by the identification unit 33.

[0091] The cropping unit 30 may also crop out the image data of the individual's face and the image data of its whole body. In this case, the adjustment unit 42 detects the orientation of the individual's body based on the positions of the image data of the individual's face and the image data of its whole body cropped by the cropping unit 30. The adjustment unit 42 may also adjust the image showing attribute information based on the detected orientation of the individual's body.

[0092] Next, the operation of the individual management system 1 according to this embodiment, that is, the individual management method executed by the individual management system 1, will be described. The individual management system 1 is divided into a registration mode process and an attribute display mode process as shown in Figure 8. The registration mode process is the same as the process shown in Figure 5 of the individual management system 1 according to the above embodiment.

[0093] (Processing in attribute display mode) The processing of the attribute display mode will now be explained. As shown in Figure 8, the individual management system 1, on the mobile terminal 2, has an imaging unit 20 that images a cow, and a display unit 21 that displays the image data captured by the imaging unit 20 (step S21; imaging and display step). The image data captured by the imaging unit 20 is sent to the server system 3. One cow or multiple cows may be imaged at the same time.

[0094] Next, in the server system 3, the extraction unit 30 extracts image data of individual cows from the image data captured by the imaging unit 20 (step S22; extraction step). If the image data contains multiple individual cows, image data is extracted for each individual.

[0095] Next, in the server system 3, the feature extraction unit 31 extracts the features of each individual based on the imaging data extracted in the above extraction step (step S23; feature extraction step). If the imaging data contains images of multiple cows, features are calculated for each individual.

[0096] Next, in the server system 3, the identification unit 33 identifies individuals by pattern recognition using the features extracted in the feature extraction step (step S24; identification step). If multiple cows are captured in the imaging data, each individual is identified, and their identification information is obtained.

[0097] Next, in the server system 3, the attribute information reading unit 41 reads attribute information corresponding to the identification information output from the identification unit 33 based on the input of features extracted by the feature extraction unit 31 from the attribute information database 40 (step S25; attribute information reading step).

[0098] Next, in the server system 3, the attribute information reading unit 41 determines whether or not there is any attribute information to read (step S26). If there is no attribute information (step S26; No), the individual management system 1 returns to step S21. Steps S21 to S26 are repeated until there is attribute information corresponding to the identification information of the individual identified in the identification step.

[0099] If attribute information is available (Step S26; Yes), the adjustment unit 42 adjusts the image showing the attribute information read from the attribute information database 40 for the individual identified by the identification unit 33, based on the position and size of the image data extracted by the cropping unit 30, so as to align it with the image data (Step S27; Adjustment step). As a result, the display unit 21 displays the individual and its attribute information superimposed on each other.

[0100] After step S27 is completed, the individual management system 1 makes a decision to terminate the display (step S28). This decision is made according to the operation input on the mobile terminal 2 or the server system 3. If the display is not terminated (step S28; No), the individual management system 1 returns to step S21 and repeats steps S21 to S28. This allows the display of the attribute information of the target cow to continue.

[0101] If the search is terminated (step S28; Yes), the individual management system 1 terminates the processing in attribute display mode.

[0102] Attribute information can take many forms. For example, as shown in Figure 9, it is possible to overlay a 3D computer graphics (CG) model of the cow (3D cow model) onto the target cow. In this case, as mentioned above, the cropping unit 30 extracts the image data of the individual's face and the image data of its entire body. Basically, the position and size of the 3D cow model are determined so as to match the rectangular area of ​​the image data of the entire body.

[0103] The information required for alignment is the scaling factor of the 3D cow model, the 3D centroid coordinates, and the rotation angle. Regarding the scaling factor, the height, length, and width of the 3D cow model are determined to match the length and width of the rectangular area. Of the 3D centroid coordinates, the X and Y coordinates are the centroid coordinates of the extracted rectangular imaging area. Since the depth of the 3D cow model is represented by changing the scaling factor, the Z coordinate is kept as a constant fixed value.

[0104] The rotation angle is used to determine whether the 3D cow model is facing right or left. The orientation of the 3D cow model is determined by whether the part of the imaging area where the face is detected is closer to the centroid coordinate of the rectangular imaging area where the whole body is detected, relative to the centroid coordinate of the rectangular imaging area where the whole body is detected, and whether it is to the left or right of the imaging area where the whole body is detected. In other words, the adjustment unit 42 detects the orientation of the individual's body based on the position of the imaging data of the individual's face and the position of the imaging data of the whole body, which are extracted by the extraction unit 30. Specifically, the adjustment unit 42 uses the position of the imaging data of the whole body as the position of the cow, and detects the orientation of the body based on the position of the face relative to the position of the imaging data of the whole body. Based on the detected orientation of the individual's body, the adjustment unit 42 adjusts the position of the 3D cow model so that it is superimposed on the individual.

[0105] In augmented reality, it is desirable that the superimposed information be three-dimensional in order to match the actual three-dimensional space, but the information obtained through general object recognition is two-dimensional. As described above, in this embodiment, the actual target's position, size, and body orientation are detected from the obtained two-dimensional information, and the display state of the image showing attribute information is adjusted based on the detection results to achieve spatial consistency between the real space and the virtual information, thereby realizing a display with less sense of incongruity.

[0106] Augmented reality allows attribute information to be displayed as an image around the cow via a mobile device 2, making the information intuitively understandable (infographics). For example, an image showing the virus emerging from around the cow is more impactful and makes it easier to recognize the risk of infection than simply displaying the text "infected cow."

[0107] According to this embodiment, by using general object recognition to detect cows in the image data and aligning the superimposed attribute information, it is possible to provide augmented reality that does not require specific markers.

[0108] Augmented reality can be classified into location-based AR and vision-based AR based on how location information is acquired. Location-based AR presents information to the user by linking it to location information acquired using GPS, etc. Vision-based AR presents information to the user by analyzing information from images acquired using cameras, etc. Vision-based AR can be classified into marker-based AR and markerless AR.

[0109] Marker-based AR places arbitrary markers in real space and aligns the presented information based on the detected markers. Markerless AR aligns the presented information using pattern matching or self-localization with natural feature points without using markers. The augmented reality used in the individual management system 1 according to this embodiment is classified as markerless AR, but it differs from known markerless AR. While known markerless AR aligns superimposed information by performing self-localization, this embodiment aligns the superimposed image using the relative positional relationship with the object detected by general object recognition. This is the point of difference from known markerless AR.

[0110] Embodiment 3 Next, Embodiment 3 of the present invention will be described. The individual management system 1 according to this embodiment provides feed to cattle.

[0111] As shown in Figure 10, the individual animal management system 1 according to this embodiment includes a mobile unit 4. The mobile unit 4 is a vehicle that can move around within the farm. The imaging unit 20 is mounted on the mobile unit 4, not on the mobile terminal 2. The mobile unit 4 includes an automatic feeding unit 24. The automatic feeding unit 24 automatically feeds the cattle according to commands from the server system 3.

[0112] The server system 3 is the same as the server system 3 that constitutes the individual management system 1 according to Embodiment 1 above, in that it comprises a segmentation unit 30, a feature extraction unit 31, a feature database 32, an identification unit 33, a determination unit 34, and a search unit 35.

[0113] The cropping unit 30 extracts image data of individual animals from the image data captured by the imaging unit 20. The feature extraction unit 31 extracts the features of the individual animal based on the image data cropped by the cropping unit 30. The feature database 32 stores the individual identification information in association with the features extracted by the feature extraction unit 31. The identification unit 33 identifies the individual animal by pattern recognition using the features extracted by the feature extraction unit 31. The determination unit 34 determines whether the identification information of the individual animal identified by the identification unit 33 matches the identification information entered into the identification information input unit 22. The search unit 35 searches for the individual animal corresponding to the identification information entered into the identification information input unit 22 based on the determination result of the determination unit 34.

[0114] The server system 3 further includes a feed information database 45, a feed information reading unit 46, and a command generation unit 47. The feed information database 45 stores individual cattle identification information and feed information in association. The feed information is, for example, information indicating individual feeds tailored to the growth stage or health condition of the cattle. The feed information reading unit 46 reads feed information from the feed information database 45 that corresponds to the identification information of the individual cattle searched by the search unit 35. The command generation unit 47 generates a command to move the mobile unit 4 near the individual identified by the identification unit 33 and searched by the search unit 35, based on the position of the image data cut out from the cut-out unit 30, and outputs this command to the mobile unit 4 to perform an operation on that individual. The mobile unit 4 feeds the individual identified by the identification unit 33 based on the feed information read out by the feed information reading unit 46.

[0115] The operation of the individual management system 1 according to this embodiment, namely the automatic feeding process, will now be described. As shown in Figure 11, first, the imaging unit 20 mounted on the movable mobile body 4 images the animal (step S31; imaging step).

[0116] Next, the identification information input unit 22 inputs the identification information (step S32; identification information input step).

[0117] The extraction unit 30 extracts image data of individual animals from the image data captured by the imaging unit 20 (step S33; extraction step).

[0118] The feature extraction unit 31 extracts the features of the individual based on the imaging data extracted in the extraction step (step S34; feature extraction step).

[0119] The identification unit 33 receives the features extracted in the feature extraction step, identifies individuals possessing those features, and outputs their identification information (step S35; identification step).

[0120] Next, in the server system 3, the determination unit 34 determines whether the identification information output from the identification unit 33 matches the identification information input to the identification information input unit 22 (step S36; determination step). If the imaging data contains images of multiple cows, the determination is performed for each individual.

[0121] Next, in the server system 3, the search unit 35 determines whether the identification number matches in the determination unit 34 and whether the target cow has been found (step S37). Here, the determination is made for each individual captured in the imaging data, and if even one match is found, the determination is affirmed. If the target cow has not been found (step S37; No), the individual management system 1 returns to the imaging step in step S31. Subsequently, as long as the target cow has not been found (step S37; No), steps S31 to S37 are repeated, and the search for the target cow captured in the imaging data continues.

[0122] When the target cow is found (Step S37; Yes), the feed information reading unit 46 reads feed information from the feed information database 45, using the identification information of the individual found by the search unit 35 as the key (Step S40; Feed Information Reading Step).

[0123] Next, the command generation unit 47 generates a command to move the mobile body 4 closer to the individual identified in the identification step (step S35) and to feed that individual, based on the position of the image data extracted in the extraction step, and outputs this command to the mobile body 4 (step S41; command generation step). The generated command is transmitted to the mobile body 4.

[0124] Upon receiving the command, the automatic feeding unit 24 of the mobile unit 4 feeds the individual cattle that have been searched for based on the feed information (step S42; feeding step).

[0125] After step S42 is completed, the individual management system 1 makes a determination to end feeding (step S43). This determination is made by the automatic feeding unit 24. If feeding has not ended (step S43; No), the individual management system 1 returns to step S31 and repeats steps S31 to S37 to search for the target cow. Once the target cow is found, steps S40 to S42 are performed to feed that cow. This ensures that feeding of the target cow continues.

[0126] If feeding is to be terminated (Step S43; Yes), the individual management system 1 terminates the automatic feeding process.

[0127] In dairy and beef cattle farming, feeding has a significant impact on the health and productivity of cattle. However, feeding is a demanding task that requires a considerable amount of time and effort each day. The heavy workload on dairy farmers, including daily milking or feeding, and calving monitoring which may require nighttime work, is one of the reasons why farmers leave the industry or why succession by new farmers is not progressing. The individual animal management system 1 according to this embodiment can reduce the workload and labor savings of producers, support the advancement of livestock management techniques, and improve working conditions.

[0128] In this embodiment, the system searches for individual cows corresponding to specific identification information and feeds them based on the feed information of those cows. However, the present invention is not limited to this. In addition to searching for individual cows corresponding to specific identification information, the system may also identify individual cows from those that have been imaged, read the feed information of the identified individual from the feed information database 45, and feed them based on that feed information.

[0129] By operating a mobile unit 4 equipped with an imaging unit 20 for remote monitoring of cattle within the farm, it becomes possible to reduce the number of rounds and save labor, and even people with little work experience can understand the health status of the cattle regardless of time or location. Furthermore, such remote monitoring can lead to the early detection of diseases in cattle. Remote monitoring is possible if the imaging data captured by the imaging unit 20 of the mobile unit 4 is displayed on the display of a terminal at the management center connected to the server system 3.

[0130] Furthermore, in the individual management system 1 according to this embodiment, the target cow that has been searched for can be continuously tracked using a mobile device. This individual management system 1 may also be configured to store the movement trajectory of the target cow being tracked. By storing the movement trajectory of the cow, it becomes possible to analyze the cow's behavioral patterns.

[0131] In the above embodiment, the mobile terminal 2 is a smartphone. However, the present invention is not limited to this. A tablet computer or a mobile phone may be used as the mobile terminal 2. Alternatively, a head-mounted display capable of overlaying the view in front of the user with an augmented reality image may be used as the mobile terminal 2.

[0132] In the above embodiment, the cattle's ear tag number is used as identification information. However, the present invention is not limited to this. If the cattle's tag is missing and the ear tag number is unknown, a unique identification number having a numbering system that does not overlap with the ear tag number may be used as the cattle's identification information.

[0133] In the above embodiment, cattle are used as the target of identification. However, the present invention is not limited to this. Other livestock animals such as pigs may be used as the target of identification. Companion animals such as dogs and cats may also be used as the target of identification.

[0134] According to the above embodiment, it is expected that work style reforms will be realized in the cattle farming industry and productivity will be improved through efficient livestock management. The same applies to the raising of other livestock animals.

[0135] The hardware and software configurations of Individual Management System 1 are examples only and can be changed and modified as needed.

[0136] The core processing part of the individual management system 1, which consists of a CPU 11, main memory unit 12, external memory unit 13, camera 14, display 15, operation input unit 16, communication interface 17, etc., can be implemented using a normal computer system, not a dedicated system. For example, the individual management system 1 that performs the above processing may be configured by distributing a computer program for executing the above operations on a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.) and installing the computer program on a computer. Alternatively, the individual management system 1 may be configured by storing the computer program on a storage device of a server device on a communication network such as the Internet and downloading it using a normal computer system.

[0137] When the functions of the individual management system 1 are realized through a division of labor between the OS (operating system) and application programs, or through collaboration between the OS and application programs, only the application program portion may be stored on a recording medium or storage device.

[0138] It is also possible to superimpose a computer program onto a carrier wave and distribute it via a communication network. For example, a computer program could be posted on a bulletin board system (BBS) on a communication network and distributed via the network. This computer program could then be launched and executed under the control of the OS, similar to other application programs, thereby enabling the aforementioned processing.

[0139] This invention allows for various embodiments and modifications without departing from the broad spirit and scope of the invention. Furthermore, the embodiments described above are for illustrative purposes only and do not limit the scope of the invention. In other words, the scope of this invention is indicated not by the embodiments, but by the claims. Various modifications made within the scope of the claims and the equivalent scope of the meaning of the invention are considered to be within the scope of this invention. [Industrial applicability]

[0140] This invention can be used in agricultural fields such as livestock management, veterinary medicine, and data management technology. In particular, it can be used for identifying individual livestock animals or companion animals. [Explanation of Symbols]

[0141] 1 Individual Management System, 2 Mobile Terminal, 3 Server System, 4 Mobile Unit, 11 CPU, 12 Main Memory Unit, 13 External Memory Unit, 14 Camera, 15 Display, 16 Operation Input Unit, 17 Communication Interface, 18 Internal Bus, 19 Program, 20 Imaging Unit, 21 Display Unit, 22 Identification Information Input Unit, 24 Automatic Feeding Unit, 30 Extraction Unit, 31 Feature Extraction Unit, 32 Feature Database, 33 Identification Unit, 34 Judgment Unit, 35 Search Unit, 36 Adjustment Unit, 40 Attribute Information Database, 41 Attribute Information Reading Unit, 42 Adjustment Unit, 45 Feed Information Database, 46 Feed Information Reading Unit, 47 Command Generation Unit

Claims

1. The imaging unit for imaging animals, A display unit that displays the imaging data captured by the imaging unit, From the imaging data captured by the imaging unit, an extraction unit extracts imaging data of individual animals, A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit, An identification unit identifies an individual by pattern recognition using the features extracted by the feature extraction unit and outputs identification information for that individual. An identification information input unit for inputting identification information of the target object, A determination unit that determines whether the identification information of an individual identified by the identification unit matches the identification information input to the identification information input unit, A search unit searches for an individual corresponding to the identification information input to the identification information input unit based on the determination result of the determination unit, An adjustment unit adjusts the display state in the display unit for the target individual identified by the identification unit and searched by the search unit, based on the position and size of the image data extracted by the extraction unit. An individual management system equipped with [specific features / features].

2. The adjustment unit is, Based on the position and size of the image data extracted by the extraction unit, the display state of the display unit is adjusted so that the individual searched by the search unit is highlighted. The individual management system according to claim 1.

3. The imaging unit for imaging animals, A display unit that displays the imaging data captured by the imaging unit, From the imaging data captured by the imaging unit, an extraction unit extracts imaging data of individual animals, A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit, An identification unit identifies an individual by pattern recognition using the features extracted by the feature extraction unit and outputs identification information for that individual. An attribute information database that stores individual identification information and attribute information in association, An attribute information reading unit reads attribute information corresponding to the identification information of an individual identified by the identification unit when the features extracted by the feature extraction unit are input to the identification unit, from the attribute information database. The system includes an adjustment unit that adjusts the display state in the display unit for the individual identified by the identification unit based on the position and size of the image data extracted by the extraction unit, The adjustment unit is, Based on the position and size of the image data extracted by the extraction unit, the image representing the attribute information read from the attribute information database for the individual identified by the identification unit is aligned with the image data. Individual management system.

4. The aforementioned cutting unit extracts the image data of the individual's face and the image data of its whole body. The adjustment unit detects the orientation of the individual's body based on the position of the image data of the individual's face and the position of the image data of the entire body, obtained by the cutting unit. Based on the orientation of the detected individual's body, an image showing attribute information is aligned with the imaging data. The individual management system according to claim 3.

5. The imaging unit and the display unit are implemented in a mobile terminal. The individual management system according to any one of claims 1 to 4.

6. The imaging unit for imaging animals, From the imaging data captured by the imaging unit, an extraction unit extracts imaging data of individual animals, A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit, An identification unit identifies an individual by pattern recognition using the features extracted by the feature extraction unit and outputs identification information for that individual. The aforementioned imaging unit is mounted on a movable body, A command generation unit generates a command to move the moving body near the individual identified by the identification unit and perform an operation on that individual, based on the position of the image data extracted from the cutting unit, and outputs this command to the moving body. An individual management system equipped with [specific features / features].

7. A feed information database that stores individual identification information and feed information in association, The system includes a feed information reading unit that reads feed information corresponding to the identification information of the individual identified by the identification unit from the feed information database, The aforementioned moving body is The system includes an automatic feeding unit that feeds individuals identified by the identification unit based on the feed information read out by the feed information reading unit. The individual management system according to claim 6.

8. The system includes a feature database that stores individual identification information and the features extracted by the feature extraction unit in association with each other, The aforementioned identification unit is By performing machine learning that distinguishes individual animals based on the relationship between features stored in the aforementioned feature database and identification information, when a feature is input, the corresponding identification information is output. The individual management system according to any one of claims 1 to 7.

9. The aforementioned cropping unit uses a first deep learning model trained for general object recognition to crop the imaging data of an individual, The feature extraction unit extracts individual features using a second deep learning model trained for object recognition. The aforementioned identification unit outputs identification information corresponding to the feature quantities using a support vector machine that performs one-to-other classification. The individual management system according to claim 8.

10. The first deep learning model described above is YOLO (You Only Look Once), The second deep learning model is VGG16. The individual management system according to claim 9.

11. The aforementioned imaging data is video data, The imaging unit sends frame images of the video data to the extraction unit as imaging data, either through an input operation or intermittently. The extraction unit, each time it receives the frame image, extracts the imaging data of an individual animal from the frame image. Each time the image data is extracted by the cropping unit, the feature extraction unit extracts the features of that individual based on the image data extracted by the cropping unit. The identification unit identifies an individual possessing a feature that is extracted by the feature extraction unit each time that feature is extracted. The individual management system according to any one of claims 1 to 10.

12. The feature extraction unit extracts features related to the movements of individual animals based on the body movements of the individual animals detected from the multiple frame images. The individual management system according to claim 11.

13. A method of individual management performed by an individual management system, The imaging unit captures an image of an animal, and the imaging data captured by the imaging unit is displayed on the display unit. A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which identifies an individual by pattern recognition using the features extracted in the feature extraction step and outputs the identification information of that individual, An identification information input step in which identification information of the target object is entered, A determination step to determine whether the identification information of the individual identified in the above identification step matches the identification information entered in the above identification information input step, A search step that searches for an individual corresponding to the identification information entered in the identification information input step, based on the determination result in the determination step, An adjustment step adjusts the display state on the display unit for the target individual identified in the identification step and searched in the search step, based on the position and size of the image data extracted in the extraction step. Individual management methods including those mentioned above.

14. A method of individual management performed by an individual management system, The imaging unit captures an image of an animal, and the imaging data captured by the imaging unit is displayed on the display unit. A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which identifies an individual by pattern recognition using the features extracted in the feature extraction step and outputs the identification information of that individual, An attribute information reading step involves reading attribute information corresponding to the identification information of the individual identified in the aforementioned identification step from an attribute information database that stores the individual's identification information and attribute information in association. The adjustment step includes adjusting the display state on the display unit for the individual identified in the identification step, based on the position and size of the imaging data extracted in the extraction step, In the adjustment step described above, Based on the position and size of the image data extracted in the extraction step, for the individual identified in the identification step, an image showing attribute information read from the attribute information database is aligned with the image data. Individual management methods.

15. A method of individual management performed by an individual management system, An imaging step in which an imaging unit mounted on a mobile body is used to image an animal, A cutting step in which imaging data of individual animals is extracted from the imaging data captured by the imaging unit, A feature extraction step is performed to extract the characteristic features of the individual based on the imaging data extracted in the aforementioned extraction step, An identification step which takes the features extracted in the feature extraction step as input, identifies an individual possessing those features, and outputs identification information for that individual, A command generation step which generates a command to move the mobile body to the vicinity of the individual identified in the identification step and to perform an operation on that individual, based on the position of the image data extracted in the extraction step, and outputs this command to the mobile body, Individual management methods including those mentioned above.

16. Computers, From the imaging data captured by the imaging unit that images animals, an extraction unit extracts the imaging data of individual animals. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. Identification information input unit for inputting identification information of the target object. A determination unit that determines whether the identification information of an individual identified by the identification unit matches the identification information input to the identification information input unit. A search unit searches for an individual corresponding to the identification information input to the identification information input unit, based on the determination result of the determination unit. Based on the position and size of the image data extracted by the cropping unit, an adjustment unit adjusts the display state of the display unit that displays the image data captured by the imaging unit, relating to the target individual identified by the identification unit and searched by the search unit. A program that makes it function as such.

17. Computers, From the imaging data captured by the imaging unit that images animals, an extraction unit extracts the imaging data of individual animals. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. An attribute information database that stores individual identification information and attribute information in association. When the feature quantities extracted by the feature quantity extraction unit are input to the identification unit, the attribute information reading unit reads attribute information corresponding to the identification information of the individual identified by the identification unit from the attribute information database. Based on the position and size of the image data extracted by the cropping unit, an adjustment unit adjusts the display state for the individual identified by the identification unit in the display unit that displays the image data captured by the imaging unit. To make it function as, The adjustment unit is, Based on the position and size of the image data extracted by the extraction unit, the image representing the attribute information read from the attribute information database for the individual identified by the identification unit is aligned with the image data. program.

18. Computers, An imaging unit mounted on a mobile vehicle captures images of animals, and an extraction unit extracts the image data of individual animals from the captured image data. A feature extraction unit extracts the characteristic features of an individual based on the imaging data extracted by the aforementioned cutting unit. An identification unit receives the features extracted by the feature extraction unit, identifies individuals possessing those features, and outputs identification information for those individuals. A command generation unit generates a command to move the moving body near the individual identified by the identification unit and to perform an operation on that individual, based on the position of the image data extracted from the cutting unit, and outputs this command to the moving body. A program that makes it function as such.

Images