Body weight estimation device, machine learning device, body weight estimation method, and body weight estimation program

Abstract

To provide a weight estimation device which can easily and safely estimate a weight of a cow.SOLUTION: A weight estimation device 10 comprises an image acquisition unit 300 which acquires at least one of a two-dimensional image and a depth image obtained by capturing a cow, a body orientation determination unit 301 which inputs the acquired image to a first learning model 11A to determine a body orientation of the cow, a body part image generation unit 302 which inputs an image obtained by capturing the cow in a prescribed body orientation to a second learning model 11B to generate a body part image, and a weight estimation unit 303 which inputs the body part image to a third learning model 11C to estimate the weight of the cow.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a body weight estimation device, a body orientation determination device, a body part image generation device, a machine learning device, an inference device, a body weight estimation method, and a body weight estimation program.

Background Art

[0002] Conventionally, various devices and instruments have been used as means for measuring the weight of cows. For example, Patent Document 1 discloses a body weight measurement device that measures the body weight of a cow when the cow walks through a weighing plate provided between a pair of fences. Patent Document 2 also discloses a tape measure with graduations for estimating the body weight of a cow marked at positions corresponding to the graduations of the chest girth of the cow.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] The body weight measurement device disclosed in Patent Document 1 is not only expensive and large-scale, but also requires inducing the cow to pass between the fences, resulting in a heavy workload for the operator and taking time for the measurement work. In addition, when estimating the body weight of a cow using the tape measure disclosed in Patent Document 2, the operator needs to come into contact with the cow to measure the chest girth, so there is a risk of accidents such as being sandwiched by the cow.

[0005] The present invention has been made in view of the above-described problems, and an object thereof is to provide a weight estimation device, a body orientation determination device, a body part image generation device, a machine learning device, an inference device, a weight estimation method, and a weight estimation program that can easily and safely estimate the weight of a cow.

Means for Solving the Problems

[0006] In order to achieve the above object, a weight estimation device according to an aspect of the present invention includes: an image acquisition unit that acquires at least one of a two-dimensional image and a depth image in which a cow is imaged; a body orientation determination unit that determines the body orientation of the cow imaged in the image by inputting the image into a first learning model obtained by machine learning the correlation between the first learning image corresponding to the image and the body orientation of the cow imaged in the first learning image; a body part image generation unit that generates a body part image by removing a removal region other than the body part region from the image by inputting the image in which the cow is imaged in the predetermined body orientation into a second learning model obtained by machine learning the correlation between the second learning image corresponding to the image and the body part region including a predetermined body part of the cow imaged in the second learning image; and a weight estimation unit that estimates the weight of the cow imaged in the image by inputting the body part image into a third learning model obtained by machine learning the correlation between the third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image.

Effects of the Invention

[0007] According to the weight estimation device according to an aspect of the present invention, since the weight of the cow is estimated from an image in which the cow is imaged in a predetermined body orientation, the weight of the cow can be easily and safely estimated. Other problems, configurations, and effects will be clarified in the embodiments for carrying out the invention described later.

[0008] The above-described problems, configurations, and effects will be clarified in the embodiments for carrying out the invention described later.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. Hereinafter, the scope necessary for the description for achieving the object of the present invention will be schematically shown, and the scope necessary for the description of the corresponding part of the present invention will be mainly described, and the parts where the description is omitted will be based on known techniques.

[0011] (First Embodiment) FIG. 1 is an overall configuration diagram showing an example of a cattle management system 1 according to a first embodiment. The cattle management system 1 is a system for estimating (measuring) the body weight of cattle C and managing the growth status and health status of cattle C.

[0012] The cattle management system 1 mainly includes a machine learning device 2, a terminal device 3, and a management device 4. Each of the devices 2 to 4 is configured by, for example, a general-purpose or dedicated computer (see FIG. 3 described later), and is connected to a wired or wireless network 5 so as to be able to transmit and receive various data to and from each other. Note that the number of each of the devices 2 to 4 and the connection configuration to the network 5 are not limited to the example of FIG. 1.

[0013] Cattle C is, for example, a livestock cattle raised in a livestock facility. Note that the number and breed of cattle C are not particularly limited.

[0014] The machine learning device 2 operates as the main body in the learning phase of machine learning, and generates first to third learning models 11A to 11C used in the terminal device 3 by machine learning. The machine learning device 2 adopts, for example, supervised learning as a machine learning method.

[0015] The learned first to third learning models 11A to 11C are provided to the terminal device 3 via, for example, the network 5 or a recording medium or the like. At this time, the first to third learning models 11A to 11C may be provided as data integrated with the weight estimation program 100 by being incorporated into the weight estimation program 100 which is one of the apps executed on the terminal device 3, or may be provided as data that can be referenced by the weight estimation program 100 and is data different from the weight estimation program 100.

[0016] The terminal device 3 is used, for example, by an operator (such as a facility manager or a breeder) of a livestock facility. The terminal device 3 is a device owned by the operator and is configured by, for example, a portable computer such as a smartphone or a tablet terminal. The terminal device 3 receives various input operations and outputs various information via a display screen or voice by an app or a browser. Further, the terminal device 3 transmits and receives various data to and from the management device 4 via the network 5.

[0017] The terminal device 3 includes a camera 35 for imaging the cow C, and functions as a weight estimation device 10 for estimating (measuring) the weight of the cow C by executing the weight estimation program 100 installed in the terminal device 3. The weight estimation device 10 operates as the main body in the inference phase of machine learning, and estimates the weight of the cow C based on the image of the cow C imaged by the camera 35 and the first to third learning models 11A to 11C generated by the machine learning device 2.

[0018] The camera 35 is an imaging device capable of imaging at least one of a two-dimensional image and a depth image. The two-dimensional image is an image obtained by projecting a subject and its surroundings onto a two-dimensional plane orthogonal to the optical axis of the camera 35, and is, for example, a color image, a grayscale image, or the like. The depth image is an image that records the depth data of a subject and its surroundings in the direction of the optical axis of the camera 35, and may be, for example, either an active method or a passive method, and may be a grayscale image (an image representing perspective by brightness and darkness), a color image (an image representing perspective by color), or the like. When the camera 35 images both a two-dimensional image and a depth image, it is preferable that the imaging ranges (angle of view) of both images be approximately the same.

[0019] In the present embodiment, the camera 35 will be described as being configured as an RGB-D (Red Green Blue Depth) camera capable of imaging both a two-dimensional image (color image) and a depth image (grayscale image).

[0020] The management device 4 includes a cattle management database 40 that manages various information about cattle C. Based on the information received from the terminal device 3, the management device 4 edits (adds, changes, deletes, etc.) the information in the cattle management database 40 or transmits the information stored in the cattle management database 40 to the terminal device 3.

[0021] FIG. 2 is a data configuration diagram showing an example of the cattle management database 40. The cattle management database 40 is composed of a cattle information table 400 and a measurement information table 401.

[0022] The cattle information table 400 has a plurality of records specified by cattle IDs, and each record registers a cattle name, a calving date, and a parity. The measurement information table 401 has a plurality of records specified by measurement IDs, and each record registers a cattle ID, an imaging date and time, a two-dimensional image, a depth image, and a body weight.

[0023] In the cattle management database 40, each record in the measurement information table 401 is associated with the cattle information table 400 by the cattle ID, so that the measurement results (estimation results) of the weight of cattle C estimated (measured) by the terminal device 3 (weight estimation device 10) are managed for each cattle C. Therefore, based on the information stored in the cattle management database 40, analysis information such as the change in weight and the average value can be obtained for each cattle C.

[0024] (Configuration of Computer 900) FIG. 3 is a hardware configuration diagram showing an example of the computer 900. Each of the machine learning device 2, the terminal device 3, and the management device 4 is configured by a general-purpose or dedicated computer 900.

[0025] As shown in FIG. 3, the main components of the computer 900 include a bus 910, a processor 912, a memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication I / F (interface) unit 922, an external device I / F unit 924, an I / O (input / output) device I / F unit 926, and a media input / output unit 928. Note that the above components may be appropriately omitted according to the application for which the computer 900 is used.

[0026] The processor 912 is composed of one or more arithmetic processing units (CPU (Central Processing Unit), MPU (Micro-processing unit), DSP (digital signal processor), GPU (Graphics Processing Unit), etc.) and operates as a control unit that controls the entire computer 900. The memory 914 stores various data and programs 930 and is composed of, for example, a volatile memory (DRAM, SRAM, etc.) that functions as a main memory and a non-volatile memory (ROM), a flash memory, etc.

[0027] The input device 916 is composed of, for example, a keyboard, a mouse, a numeric keypad, an electronic pen, etc., and functions as an input unit. The output device 917 is composed of, for example, a sound (voice) output device, a vibration device, etc., and functions as an output unit. The display device 918 is composed of, for example, a liquid crystal display, an organic EL display, an electronic paper, a projector, etc., and functions as an output unit. The input device 916 and the display device 918 may be integrally configured like a touch panel display. The storage device 920 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and functions as a storage unit. The storage device 920 stores various data necessary for the execution of the operating system and the program 930.

[0028] The communication I / F unit 922 is connected to a network 940 (which may be the same as the network 5 in FIG. 1) such as the Internet or an intranet, either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from other computers according to a predetermined communication standard. The external device I / F unit 924 is connected to an external device 950 such as a camera, a printer, a scanner, a reader / writer, etc., either wired or wirelessly, and functions as a communication unit that transmits and receives data to and from the external device 950 according to a predetermined communication standard. The I / O device I / F unit 926 is connected to an I / O device 960 such as various sensors and actuators, and functions as a communication unit that transmits and receives various signals and data such as detection signals from sensors and control signals to actuators to and from the I / O device 960. The media input / output unit 928 is composed of, for example, drive devices such as a DVD ( Digital Versatile Disc) drive, a CD (Compact Disc) drive, etc., and reads and writes data to and from media (non-volatile storage media) 970 such as DVDs and CDs.

[0029] In the computer 900 having the above configuration, the processor 912 calls and executes the program 930 stored in the storage device 920 in the memory 914, and controls each part of the computer 900 via the bus 910. Note that the program 930 may be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded on the medium 970 in an installable file format or an executable file format, and provided to the computer 900 via the media input / output unit 928. The program 930 may be provided to the computer 900 by downloading it via the network 940 through the communication I / F unit 922. Also, the computer 900 may implement various functions realized when the processor 912 executes the program 930 with hardware such as an FPGA (field-programmable gate array) or an ASIC (application specific integrated circuit).

[0030] The computer 900 is composed of, for example, a stationary computer or a portable computer, and is an electronic device in any form. The computer 900 may be a client-type computer, or a server-type computer or a cloud-type computer. The computer 900 may be applied to devices other than the machine learning device 2, the terminal device 3, and the management device 4.

[0031] (Machine learning device 2) FIG. 4 is a block diagram showing an example of the machine learning device 2 according to the first embodiment. The machine learning device 2 includes a control unit 20, an input unit 21, an output unit 22, a communication unit 23, a learning data storage unit 24, and a learned model storage unit 25.

[0032] By executing a machine learning program (not shown), the control unit 20 functions as a learning data acquisition unit 200 and first to third machine learning units 201A to 201C. The input unit 21 accepts various input operations, and the output unit 22 outputs various information via a display screen or voice, thereby functioning as a user interface. The communication unit 23 is connected to an external device (for example, the terminal device 3, the management device 4, etc.) via the network 5 and functions as a communication interface for transmitting and receiving various data.

[0033] The learning data acquisition unit 200 acquires first to third learning data 12A to 12C by receiving various data from an external device via the communication unit 23 and the network 5. In addition, the learning data acquisition unit 200 acquires first to third learning data 12A to 12C by accepting an operator's input operation via the input unit 21 and the output unit 22.

[0034] The learning data storage unit 24 is a database that stores a plurality of sets of first to third learning data 12C to 12C acquired by the learning data acquisition unit 200. The first to third learning data 12C to 12C are used as training data, verification data, and test data in supervised learning. Note that the specific configuration of the database constituting the learning data storage unit 24 may be designed as appropriate.

[0035] The first to third machine learning units 201A to 201C respectively generate first to third learning models 11A to 11C by performing machine learning using the first to third learning data 12A to 12C stored in the learning data storage unit 24.

[0036] The learned model storage unit 25 is a database that stores the learned first to third learning models 11A to 11C that have been machine-learned by the first to third machine learning units 201A to 201C, respectively. The first to third learning models 11A to 11C stored in the learned model storage unit 25 are provided to the actual system (for example, the terminal device 3) via the network 5, a recording medium, or the like. In the figure 4 In 4 , the learning data storage unit 24 and the learned model storage unit 25 are shown as separate storage units, but these may be configured as a single storage unit.

[0037] FIG. 5 is a diagram showing an example of the first learning model 11A and the first learning data 12A. The first learning data 12A is composed of a first learning image and the body orientation of cow C imaged in the first learning image.

[0038] The first learning image is an image of cow C captured by the camera 35 and corresponds to at least one of a two-dimensional image and a depth image. In the present embodiment, the case where the first learning image is a two-dimensional image will be described, but it may be a depth image.

[0039] The body orientation of cow C corresponds to the correct label in supervised learning. The body orientation of cow C represents the body orientation when cow C is imaged, and may be represented by multi-class classification according to the azimuth (4 azimuths, 8 azimuths, 16 azimuths) when the entire circumference (360 degrees) is divided by a predetermined number (4, 8, 16, etc.), or may be represented by two-class classification as to whether it is a predetermined body orientation or not.

[0040] For example, when the orientation of cow C is represented by an 8-class classification based on 8 directions as shown in FIG. 5, by imaging from the front of the cow, the "front" where the head side of cow C is imaged frontally in the image, the "front left", "left side", and "rear left" where the left side of the body of cow C is imaged in the image by imaging from the front left, left side, and rear left of the cow respectively, the "front right", "right side", and "rear right" where the right side of the body of cow C is imaged in the image by imaging from the front right, right side, and rear right of the cow respectively, and the "rear" where the hip side of cow C is imaged frontally in the image. Also, when the orientation of cow C is represented by a 2-class classification, it is represented by whether the image is taken from the rear left or left side of the cow, that is, whether it is a predetermined orientation ("rear left" or "left side") where the left side of the body of cow C is imaged in the image.

[0041] The learning data acquisition unit 200 receives, for example, an image (a two-dimensional image in this embodiment) of cow C taken in the past from an external device as a first learning image, and receives an input operation for specifying the orientation of cow C imaged in the first learning image from the input unit 21, and associates them to obtain the first learning data 12A.

[0042] The first learning model 11A adopts, for example, the structure of a convolutional neural network (CNN), and includes an input layer 110A, an intermediate layer 111A, and an output layer 112A. Synapses (not shown) connecting each neuron are provided between each layer, and weights are associated with each synapse.

[0043] The input layer 110A has a number of neurons corresponding to the number of pixels of the first learning image as input data, and the pixel value of each pixel is input to each neuron respectively. The intermediate layer 111A is composed of, for example, a convolutional layer, a pooling layer, and a fully connected layer, extracts feature amounts from the first learning image, and outputs them as a feature vector of a one-dimensional array. The output layer 112A outputs, as output data, a determination result obtained by determining (inferring) the body orientation of the cow C imaged in the first learning image based on the feature vector output from the intermediate layer 111A. In the example of FIG. 5, the output layer 112A has a number of neurons corresponding to 8 classes consisting of "front", "front left", "left side", "rear left", "front right", "right side", "rear right", and "rear", and each neuron outputs a score (reliability) when classified into each class as a value in a predetermined range (for example, 0 to 1).

[0044] The first machine learning unit 201A causes the first learning model 11A to perform machine learning on the correlation between the first learning image and the body orientation of the cow C imaged in the first learning image by inputting a plurality of sets of the first learning data 12A to the first learning model 11A. The plurality of sets of the first learning data 12A are composed of the first learning images captured in various body orientations corresponding to each class.

[0045] Specifically, the first machine learning unit 201A uses the first learning image that constitutes the first learning data 12A as input data, inputs it into the input layer 110A, and uses an error function that compares the body orientation (output data) of cow C output as a determination result from the output layer 112A with the body orientation (correct label) of cow C that constitutes the first learning data 12A to adjust the weights associated with each synapse. Then, when the first machine learning unit 201A determines that a predetermined learning end condition is satisfied, it ends the machine learning and stores a weight parameter group composed of the weights associated with each of the synapses at that time in the learned model storage unit 25 as the learned first learning model 11A. Note that the first machine learning unit 201A may perform predetermined image adjustment (for example, image format, image size, color correction, etc.) on the first learning image as preprocessing when inputting the first learning image into the input layer 110A.

[0046] FIG. 6 is a diagram showing an example of the second learning model 11B and the second learning data 12B. The second learning data 12B is composed of a second learning image and a body part region including a predetermined body part of cow C imaged in the second learning image.

[0047] The second learning image is an image of cow C captured by the camera 35 and corresponds to at least one of a two-dimensional image and a depth image. In the present embodiment, the case where the second learning image is a depth image captured with the body orientation of cow C being a predetermined body orientation ("left rear") will be described, but it may also be a two-dimensional image captured under the same conditions.

[0048] The body part region corresponds to the correct label in supervised learning. The body part region represents the region in the second learning image where a predetermined body part of Cow C is imaged, and does not include the removal region outside the predetermined body part, that is, the region where body parts other than the predetermined body part or the background are imaged. The predetermined body part may be only the torso of Cow C, or may be the torso and limbs of Cow C. When the predetermined body part is the "torso" of Cow C, the body part region is the region where only the torso is imaged, and does not include the region where body parts other than the torso or the background are imaged (removal region). Also, when the predetermined body part is the "torso and limbs" of Cow C, the body part region is, as shown in FIG. 6, the region obtained by combining the region where the torso is imaged and the region where the limbs are imaged, and does not include the region where body parts other than the torso and limbs or the background are imaged (removal region).

[0049] The learning data acquisition unit 200 receives, for example, an image captured by Cow C in the past (in this embodiment, a depth image captured in the "left rear" body orientation) from an external device as the second learning image, and receives an input operation for specifying the body part region and the removal region in the second learning image from the input unit 21, and associates them to obtain the second learning data 12B. At this time, the learning data acquisition unit 200 may receive, as an input operation for specifying the body part region and the removal region, for example, information specifying the regions where each part of the body part of Cow C (torso, head, neck, limbs) is imaged and the region where the background is imaged.

[0050] The second learning model 11B is a learning model that performs segmentation. For example, it adopts the structure of a fully convolutional network (FCN), and includes an input layer 110B, an intermediate layer 111B, and an output layer 112B. Synapses (not shown) connecting each neuron are formed between each layer, and weights are associated with each synapse are.

[0051] The input layer 110B has a number of neurons corresponding to the number of pixels of the second learning image as input data, and the pixel value of each pixel is input to each neuron respectively. The intermediate layer 111B is composed of, for example, a convolutional layer and a deconvolutional layer. The output layer 112B has a number of neurons corresponding to the number of images of the second learning image, and outputs, as output data, the classification result of classifying (inferring) into a body part region and a removal region other than the body part region. Each neuron in the output layer 112B outputs, for example, a score (reliability) indicating whether each pixel is a body part region or not as a value within a predetermined range (for example, 0 to 1). In the example of FIG. 6, the output layer 112B generates and outputs a body part image in which pixels corresponding to "torso and limbs" are used as the body part region and pixels corresponding to the rest are used as the removal region.

[0052] The second machine learning unit 201B inputs a plurality of sets of the second learning data 12B to the second learning model 11B, and causes the second learning model 11B to perform machine learning on the correlation between the second learning image and the body part region including a predetermined body part of the cow C imaged in the second learning image.

[0053] Specifically, the second machine learning unit 201B uses an error function that inputs the second learning image constituting the second learning data 12B as input data to the input layer 110B and compares the body part region (output data) output as the classification result from the output layer 112B with the body part region (correct label) constituting the second learning data 12B, and adjusts the weights associated with each synapse. Then, when the second machine learning unit 201B determines that a predetermined learning end condition is satisfied, it ends the machine learning, and stores, in the learned model storage unit 25, a weight parameter group composed of the weights associated with each of the synapses at that time as the learned second learning model 11B.

[0054] FIG. 7 is a diagram showing an example of the third learning model 11C and the third learning data 12C. The third learning data 12C is composed of the third learning image and the body weight of the cow C imaged in the third learning image.

[0055] The third learning image is an image captured by the camera 35 of Cow C, and corresponds to at least one of a two-dimensional image and a depth image. In the present embodiment, the third learning image is a depth image captured with the body orientation of Cow C being a predetermined body orientation ("left rear"), and the following will describe the case where it is a depth image (body part image) obtained by removing a removal area other than the body part area including a predetermined body part ("trunk and limbs") from the depth image, but it may also be a two-dimensional image captured and processed under the same conditions.

[0056] The weight of Cow C corresponds to the correct label in supervised learning. When the third learning model 11C is a classification model, as shown in FIG. 7, the weight of Cow C may be represented by multi-class classification for each predetermined range (for example, every 10 kg), or when the third learning model 11C is a regression model, it may be represented by a numerical value with the unit of kilogram.

[0057] The learning data acquisition unit 200 receives, for example, an image captured by Cow C in the past (in the present embodiment, a depth image (body part image) obtained by removing a removal area other than the body part area including "trunk and limbs" from the depth image captured with the body orientation of "left rear") as the third learning image from an external device, and receives an input operation for inputting the weight of Cow C captured in the third learning image from the input unit 21, and associates them to obtain the third learning data 12C. At this time, the third learning image may be generated by editing an image with image editing software or the like. Also, for example, a measured value measured by a weighing scale is input as the weight of Cow C.

[0058] The third learning model 11C adopts, for example, the structure of a convolutional neural network, and includes an input layer 110C, an intermediate layer 111C, and an output layer 112C. Synapses (not shown) connecting each neuron are provided between each layer, and weights are associated with each synapse.

[0059] The input layer 110C has a number of neurons corresponding to the number of pixels of the third training image as input data, and the pixel value of each pixel is input to each neuron respectively. The intermediate layer 111C is composed of, for example, a convolutional layer, a pooling layer, and a fully connected layer, extracts feature amounts from the third training image, and outputs them as a feature vector of a one-dimensional array. The output layer 112C outputs, as output data, an estimation result obtained by estimating (inferring) the weight of the cow C imaged in the third training image based on the feature vector output from the intermediate layer 111C. In the example of FIG. 7, the output layer 112C has a number of neurons corresponding to each weight for every 10 kg, and each neuron outputs a score (reliability) when classified into each class as a value in a predetermined range (for example, 0 to 1).

[0060] The third machine learning unit 201C inputs a plurality of sets of the third training data 12C to the third training model 11C, and causes the third training model 11C to perform machine learning on the correlation between the third training image and the weight of the cow C imaged in the third training image.

[0061] Specifically, the third machine learning unit 201C uses an error function that inputs the third training image constituting the third training data 12C as input data to the input layer 110C and compares the weight of the cow C (output data) output as an estimation result from the output layer 112C with the weight of the cow C (correct label) constituting the third training data 12C, and adjusts the weights associated with each synapse. Then, when the third machine learning unit 201C determines that a predetermined learning end condition is satisfied, it ends the machine learning, and stores a weight parameter group composed of the weights associated with each of the synapses at that time in the learned model storage unit 25 as the learned third training model 11C.

[0062] (Machine learning method) FIG. 8 is a flowchart showing an example of a machine learning method by the machine learning apparatus 2 according to the first embodiment. Hereinafter, a case where the third machine learning unit 201C generates the third learning model 11C using a plurality of sets of third learning data 12C will be described. In FIG. 8, the description “third” in “third learning model 11C” and “third learning data 12C” is omitted. Also, since the machine learning methods by the first machine learning unit 201A and the second machine learning unit 201B are the same as those of the third machine learning unit 201C, their descriptions are omitted.

[0063] First, in step S100, the learning data acquisition unit 200 acquires a plurality of sets of third learning data 12C as a preliminary preparation for starting machine learning, and stores the acquired plurality of sets of third learning data 12C in the learning data storage unit 24.

[0064] Next, in step S110, the third machine learning unit 201C prepares the third learning model 11C before learning in order to start machine learning. In the third learning model 11C before learning, the weight of each synapse is set to an initial value. Each pixel of the third learning image constituting the third learning data 12C is associated with each neuron in the input layer 110C. Each class corresponding to each weight of 10 kg is associated with each neuron in the output layer 112C.

[0065] Next, in step S120, the third machine learning unit 201C randomly selects, for example, one set of the third learning data 12C from the plurality of sets of third learning data 12C stored in the learning data storage unit 24. third learning data 12C is acquired.

[0066] Next, in step S130, the third machine learning unit 201C inputs the third learning image (input data) included in a set of third learning data 12C to the input layer 110C of the prepared pre-learning (or in-learning) third learning model 11C. As a result, an estimated result (output data) of the weight of cow C is output from the output layer 112C of the third learning model 11C, and this estimated result is estimated by the pre-learning (or in-learning) third learning model 11C. Therefore, in the pre-learning (or in-learning) state, the weight of cow C output as the estimated result indicates information different from the weight (correct label) of cow C included in the third learning data 12C.

[0067] Next, in step S140, the third machine learning unit 201C uses an error function that compares the weight of cow C (correct label) included in a set of third learning data 12C acquired in step S120 with the weight of cow C (output data) output as the estimated result from the output layer 112C in step S130, and adjusts the weights associated with each synapse so that the evaluation value of the error function becomes smaller, thereby performing machine learning. As a result, the third machine learning unit 201C causes the third learning model 11C to machine-learn the correlation between the third learning image and the weight of cow C imaged in the third learning image.

[0068] Next, in step S150, the third machine learning unit 201C determines whether a predetermined learning end condition is satisfied, for example, based on whether the evaluation value of the error function is smaller than the allowable value, whether the number of repetitions of steps S120 to S140 has reached a predetermined number, and the like.

[0069] In step S150, when the third machine learning unit 201C determines that the learning end condition is not satisfied and machine learning is to be continued (No in step S150), the process returns to step S120, and the processes of steps S120 to S140 are performed on the third learning model 11C being learned using the unlearned third learning data 12C. On the other hand, in step S150, when the third machine learning unit 201C determines that the learning end condition is satisfied and machine learning is to end (Yes in step S150), the process proceeds to step S160.

[0070] Then, in step S160, the third machine learning unit 201C stores the learned third learning model 11C (adjusted weight parameter group) that has been machine-learned by adjusting the weights associated with each synapse in the learned model storage unit 25, and ends the series of machine learning methods shown in FIG. 8. In the above machine learning method, step S100 corresponds to the learning data storage step, steps S110 to S150 correspond to the machine learning step, and step S160 corresponds to the learned model storage step.

[0071] As described above, according to the machine learning apparatus 2 and the machine learning method according to the present embodiment, it is possible to provide a third learning model 11C that can accurately estimate (measure) the weight of the cow C from the image captured by the camera 35. Further, according to the machine learning apparatus 2 and the machine learning method according to the present embodiment, it is possible to provide a first learning model 11A that can accurately determine the body orientation of the cow C from the image captured by the camera 35. Furthermore, according to the machine learning apparatus 2 and the machine learning method according to the present embodiment, it is possible to provide a second learning model 11B that can generate a body part image from the image captured by the camera 35.

[0072] (Weight Estimation Device 10) FIG. 9 is a block diagram showing an example of a terminal device 3 that functions as the weight estimation device 10 according to the first embodiment. FIG. 10 is a functional explanatory diagram showing an example of the weight estimation device 10 according to the first embodiment. The terminal device 3 includes a control unit 30, an input unit 31, an output unit 32, a communication unit 33, and a memory It includes a memory unit 34 and a camera 35.

[0073] The control unit 30 functions as a weight estimation device 10 by executing the weight estimation program 100 stored in the memory unit 34. Specifically, the control unit 30 functions as an image acquisition unit 300, a body orientation determination unit 301, a body part image generation unit 302, a weight estimation unit 303, and an input / output processing unit 304 included in the weight estimation device 10.

[0074] The input unit 31 receives various input operations, and the output unit 32 outputs various information via a display screen or voice, thereby functioning as a user interface that cooperates with the input / output processing unit 304. The communication unit 33 is connected to an external device (for example, a machine learning device 2, a management device 4, etc.) via the network 5 and functions as a communication interface for transmitting and receiving various data. In addition to storing the weight estimation program 100, the memory unit 34 stores the learned first to third learning models 11A to 11C (for example, adjusted weight parameter groups) generated by the machine learning device 2, an operating system (OS), other programs (applications), various data, etc.

[0075] The image acquisition unit 300 acquires an image of the cow C to be the target of weight estimation captured by the camera 35. In this embodiment, the case where the image acquisition unit 300 acquires both a two-dimensional image and a depth image will be described.

[0076] The body orientation determination unit 301 determines the body orientation of the cow C imaged in the image by inputting the image (in this embodiment, a two-dimensional image) acquired by the image acquisition unit 300 into the first learning model 11A. In this embodiment, since the first learning model 11A outputs scores for 8 classes (8 orientations), the body orientation determination unit 301 selects the class that outputs the maximum score among the scores for each class and outputs the body orientation corresponding to that class as the determination result.

[0077] The body part image generation unit 302 inputs the image of the cattle body orientation determined by the body orientation determination unit 301 and captured at a predetermined body orientation (in this embodiment, the depth image captured at the "left rear" body orientation) into the second learning model 11B, and generates, as a body part image, an image obtained by removing the removal area other than the body part area including a predetermined body part from the image. In this embodiment, the second learning model 11B generates, as a body part image, an image obtained by removing the removal area (head, neck, background, etc.) other than the body part area including "the body and four limbs" as a predetermined body part. Therefore, the body part image generation unit 302 extracts only the body and four limbs and outputs a body part image with the head, neck, background, etc. removed.

[0078] The weight estimation unit 303 inputs the body part image generated by the body part image generation unit 302 (in this embodiment, the depth image obtained by removing the removal area other than the body part area including "the body and four limbs" from the depth image captured at the "left rear" body orientation) into the third learning model 11C, and estimates the weight of the cattle C imaged in the body part image. In this embodiment, since the third learning model 11B outputs scores for each multi-class (weight every 10 kg), the weight estimation unit 303 selects the class that outputs the score with the maximum value among the scores for each class, and outputs the weight corresponding to that class as the estimation result.

[0079] The input / output processing unit 304 controls various tasks by displaying various display screens (imaging screen, weight estimation result screen, etc.), controlling the screen transition of the display screen, and receiving input operations for each display screen. Examples of tasks include imaging of cattle C by the camera 35, estimation of the weight of the imaged cattle C, display of the estimated result, editing (addition, modification, deletion, etc.) and reference of the cattle management database 40, data storage for storing the information stored in the cattle management database 40 as data in a predetermined format, acquisition of analysis information (weight trend, average value, etc.) based on the information stored in the cattle management database 40, and setting of various parameters in the weight estimation program 100. Examples thereof include acquisition of analysis information (weight trend, average value, etc.) based on the information stored in the cattle management database 40, and setting of various parameters in the weight estimation program 100.

[0080] Note that the number of the first to third learning models 11A to 11C stored in the storage unit 34 is not limited to one each. For example, a plurality of learned models with different conditions such as machine learning methods, body orientations of cows C, and breeds may be stored and selectively used. Further, the first to third learning models 11A to 11C may be substituted by a storage unit of an external computer (which may be the machine learning device 2 or other server-type computers or cloud-type computers). In that case, the body orientation determination unit 301, the body part image generation unit 302, and the weight estimation unit 303 may access the external computer.

[0081] Also, in the present embodiment, the image acquisition unit 300 will be described for the case of acquiring both a two-dimensional image and a depth image as shown in FIG. 10. However, only one of the two-dimensional image and the depth image may be acquired. In that case, the body orientation determination unit 301, the body part image generation unit 302, and the weight estimation unit 303 may use only one image acquired by the image acquisition unit 300. When the image acquisition unit 300 acquires a depth image, for example, as one image, the body orientation determination unit 301 inputs the depth image into the first learning model 11A to determine the body orientation of cow C imaged in the depth image. The body part image generation unit 302 inputs the depth image in which cow C is imaged in a predetermined body orientation into the second learning model 11B to generate a depth image with the removal area removed as the body part image. The weight estimation unit may estimate the weight of cow C imaged in the depth image by inputting the body part image into the third learning model. At this time, the first learning model 11A is obtained by machine learning the correlation between the depth image as the first learning image and the body orientation of cow C imaged in the depth image.

[0082] Furthermore, even when the image acquisition unit 300 acquires both a two-dimensional image and a depth image, the body orientation determination unit 301, the body part image generation unit 302, and the weight estimation unit 303 may use one of the images (for example, the depth image) acquired by the image acquisition unit 300. In that case, for the other image (for example, the two-dimensional image), the input / output processing unit 304 may use it for various tasks, or may display the other image (for example, the two-dimensional image) on, for example, the imaging screen 13 (see FIG. 12 described later) or the weight estimation result screen 14 (see FIG. 13 described later). Thereby, when the other image is a two-dimensional image, the visibility of the image in the user interface can be improved.

[0083] (Weight Estimation Method) FIG. 11 is a flowchart showing an example of a weight estimation method by the weight estimation device 10 according to the first embodiment. The weight estimation method shown in FIG. 11 is executed when the weight estimation program 100 is started on the terminal device 3. Hereinafter, it will be described on the assumption that the setting parameters of the weight estimation program 100 are set, and the information of the cow C to be the target of weight estimation is registered in the cow information table 400 of the cow management database 40.

[0084] First, in step S200, when the input / output processing unit 304 receives, as an input operation of the operator, a weight measurement start operation to start the weight measurement of the cow C, in step S201, it displays the imaging screen 13 for imaging the cow C by the camera 35.

[0085] FIG. 12 is a screen configuration diagram showing an example of the imaging screen 13. The imaging screen 13 includes an image display area 130 that displays a real-time image by the camera 35, and an imaging button 131 that instructs imaging by the camera 35. Note that a message prompting to image the cow C in a predetermined body orientation of "left rear" may be displayed on the imaging screen 13. Also, either a two-dimensional image or a depth image may be displayed in the image display area 130, both images may be displayed side by side, or may be displayed in a switchable manner.

[0086] In step S202, when the input / output processing unit 304 receives a pressing operation of the imaging button 131, the camera 35 images the cow C. Then, in step S210, the image acquisition unit 300 acquires a two-dimensional image and a depth image as the images captured by the camera 35.

[0087] Next, in step S220, the body orientation determination unit 301 inputs the two-dimensional image among the two-dimensional image and the depth image acquired in step S210 into the first learning model 11A, thereby determining the body orientation of the cow C imaged in the image.

[0088] Then, in step S221, if the body orientation of the cow C determined in step S220 is a predetermined body orientation ("left rear"), the process proceeds to step S230; otherwise, a message indicating that the body orientation of the cow C imaged by the camera 35 is not the predetermined body orientation ("left rear") is displayed, and the process returns to step S201. Note that the body orientation determination unit 301 may determine the body orientation of the cow C at a predetermined cycle by inputting the real-time image captured by the camera 35 into the first learning model 11A at a predetermined cycle. In this case, the input / output processing unit 304 may enable the imaging button 131 on the imaging screen 13 or display a message prompting the imaging of the cow C at the timing when the body orientation of the cow C is the predetermined body orientation ("left rear").

[0089] Next, in step S230, the body part image generation unit 302 inputs the depth image among the two-dimensional image and the depth image acquired in step S210 and captured with the "left rear" body orientation into the second learning model 11B, thereby generating a depth image with the removal area outside the body part area including the "trunk and limbs" removed from the depth image as the body part image.

[0090] Next, in step S240, the weight estimation unit 303 inputs the body part image generated in step S230 into the third learning model 11C, thereby estimating the weight of the cow C imaged in the body part image.

[0091] Next, in step S250, the input / output processing unit 304 displays a weight estimation result screen 14 for outputting the weight of the cow C to be the target of weight estimation as the estimation result estimated in step S240.

[0092] FIG. 13 is a screen configuration diagram showing an example of the weight estimation result screen 14. The weight estimation result screen 14 includes an image display area 140 for displaying an image captured by the camera 35 in step S202, a cow information display column 141 for displaying information on the cow C that is the target of weight estimation, an estimated result display column 142 for displaying the estimation result of the weight of the cow C estimated by the weight estimation unit 303 in step S240, and a registration button 143 for registering the estimation result of the weight of the cow C in the cow management database 40.

[0093] In step S251, when the input / output processing unit 304 receives a weight estimation result registration operation by pressing the registration button 143, in step S252, the input / output processing unit 304 transmits registration information for registering the weight of the cow C estimated in step S240 to the management device 4, and ends the series of weight estimation methods shown in FIG. 11. Note that when the input / output processing unit 304 does not receive the pressing operation of the registration button 143, the weight estimation method may be ended without transmitting the registration information to the management device 4. Further, when there are a plurality of cows C that are the targets of weight estimation, the above weight estimation method may be repeatedly executed. In the above weight estimation method, step S210 is an image acquisition step, step S220 is a body orientation determination step, step S230 is a body part image generation step, and step S240 is a body weight estimation step.

[0094] As described above, according to the weight estimation device 10 and the weight estimation method according to the present embodiment, by inputting an image of the cow C captured in a predetermined body orientation into the first to third learning models 11A to 11C, the weight of the cow C can be estimated from the image, so that the weight of the cow C can be estimated simply and safely.

[0095] At this time, the body orientation determination unit 301 determines the body orientation of cow C using the first learning model 11A, and the weight of cow C is estimated based on an image captured in a predetermined body orientation. Therefore, when using an image captured from, for example, "left rear" or "left side" as the predetermined body orientation of cow C, the entire side of cow C including the left abdomen is captured in the image, and the imaging range of the body part of cow C becomes larger. Thus, the estimation accuracy when estimating the weight of cow C can be improved.

[0096] Also, the body part image generation unit 302 generates a body part image using the second learning model 11B, and the weight of cow C is estimated based on a body part image from which a removal region other than the body part region including a predetermined body part of cow C is removed. Therefore, when using a body part image from which a removal region other than, for example, "trunk" or "trunk and limbs" is removed as the predetermined body part of cow C, in that body part image, the head and neck, which are body parts that cow C frequently moves, are removed. Thus, by reducing the influence due to changes in the posture of cow C, a decrease in the estimation accuracy when estimating the weight of cow C can be suppressed.

[0097] Furthermore, the weight estimation unit 303 estimates the weight of cow C from the body part image using the third learning model 11C. As the weight estimation device 10, for example, a portable computer such as a smartphone or a tablet terminal can be used. Thus, the weight of cow C can be estimated simply and safely without using an expensive and large-scale device.

[0098] (Second Embodiment) FIG. 14 is a block diagram showing an example of a terminal device 3a that functions as the weight estimation device 10a according to the second embodiment. FIG. 15 is a functional explanatory diagram showing an example of the weight estimation device 10a according to the second embodiment.

[0099] By executing the body weight estimation program 100a, the control unit 30 of the terminal device 3a functions as a body weight estimation device 10a including an image acquisition unit 300, a body part image generation unit 302, a body weight estimation unit 303, and an input / output processing unit 304. That is, the body weight estimation device 10a according to the second embodiment omits the body orientation determination unit 301 that determines the body orientation of cow C using the first learning model 11A compared to the body weight estimation device 10 according to the first embodiment.

[0100] In the body weight estimation device 10a shown in FIG. 15, the image acquisition unit 300 acquires a depth image, the body part image generation unit 302 generates a body part image by inputting the depth image into the second learning model 11B, and the body weight estimation unit 303 estimates the body weight of cow C by inputting the body part image into the third learning model 11C. The depth image acquired by the image acquisition unit 300 is preferably a depth image captured when the body orientation of cow C is "left rear" or "left side", but is not particularly limited. Since the other basic configurations and operations in the cattle management system 1 are the same as those in the first embodiment, the description is omitted.

[0101] As described above, according to the body weight estimation device 10a and the body weight estimation method according to the present embodiment, by inputting an image of cow C captured in a predetermined body orientation into the second and third learning models 11B and 11C, the body weight of cow C can be estimated from the image, so that the body weight of cow C can be estimated simply and safely.

[0102] (Third Embodiment) FIG. 16 is a block diagram showing an example of a terminal device 3b that functions as a body weight estimation device 10b according to the third embodiment. FIG. 17 is a functional explanatory diagram showing an example of the body weight estimation device 10b according to the third embodiment.

[0103] The control unit 30 of the terminal device 3b functions as a weight estimation device 10b including an image acquisition unit 300, a body orientation determination unit 301, a weight estimation unit 303, and an input / output processing unit 304 by executing the weight estimation program 100b. That is, the weight estimation device 10b according to the third embodiment is obtained by omitting the body part image generation unit 302 that generates a body part image using the second learning model 11B from the weight estimation device 10 according to the first embodiment.

[0104] In the weight estimation device 10b shown in FIG. 17, the image acquisition unit 300 acquires a two-dimensional image and a depth image, the body orientation determination unit 301 determines the body orientation of the cow C by inputting the two-dimensional image into the first learning model 11A, and the weight estimation unit 303 inputs the depth image captured when the body orientation of the cow C is at a predetermined body orientation ("left rear") into the third learning model 11C to estimate the weight of the cow C. Note that the third learning model 11C is preferably one obtained by machine learning the correlation between the depth image and the weight of the cow C, rather than the correlation between the body part image and the weight of the cow C, but is not particularly limited. Since the other basic configurations and operations in the cow management system 1 are the same as those in the first embodiment, the description thereof is omitted.

[0105] As described above, according to the weight estimation device 10b and the weight estimation method according to the present embodiment, by inputting the image of the cow C captured at a predetermined body orientation into the first and third learning models 11A and 11C, the weight of the cow C can be estimated from the image, so that the weight of the cow C can be estimated simply and safely.

[0106] (Other Embodiments) The present invention is not limited to the above-described embodiments, and various modifications can be made and implemented without departing from the gist of the present invention. And all of them are included in the technical idea of the present invention.

[0107] In the above embodiment, the case where the machine learning device 2 includes the first to third machine learning units 201A to 201C has been described. However, for example, three machine learning devices may include the first to third machines You may provide each of the learning units 201A to 201C.

[0108] In the above embodiment, the weight estimation devices 10, 10a, and 10b realized by causing the computer 900 (terminal device 3) to execute the weight estimation programs 100, 100a, and 100b have been described as including at least one of the body orientation determination unit 301 and the body part image generation unit 302 in addition to the image acquisition unit 300 and the weight estimation unit 303. However, the weight estimation device may include at least the image acquisition unit 300 and the weight estimation unit 303. Further, by causing the computer 900 (terminal device 3) to execute the body orientation determination program, the computer 900 may function as a body orientation determination device including the image acquisition unit 300 and the body orientation determination unit 301. Furthermore, by causing the computer 900 (terminal device 3) to execute the body part image generation program, the computer 900 may function as a body part image generation device including the image acquisition unit 300 and the body part image generation unit 302.

[0109] In the above embodiment, as a specific method of machine learning by the first to third machine learning units 201A to 201C, the case of adopting a convolutional neural network or a fully convolutional network has been described. However, the first to third machine learning units 201A to 201C may adopt any other machine learning method (including not only supervised learning but also unsupervised learning and reinforcement learning). Examples of other machine learning methods include tree types such as decision trees and regression trees, ensemble learning such as bagging and boosting, other neural network types (including deep learning) such as recurrent neural networks, hierarchical clustering, non-hierarchical clustering, clustering types such as the k-nearest neighbor method and the k-means method, multivariate analysis such as principal component analysis, factor analysis, and logistic regression, and support vector machines.

[0110] In the above embodiment, the output data of the third learning model 11C has been described in the case where it is the weight of cow C. However, in addition to the weight of cow C, the output data of the third learning model 11C may adopt at least one of a body condition score (an index of obesity), a rumen fill score (an index of satiety), and a lameness score (an index of leg disease). In that case, the weight estimation unit 303 may further estimate at least one of a body condition score, a rumen fill score, and a lameness score in addition to the weight of cow C by inputting the image into the third learning model 11C.

[0111] (Inference device, inference method, or inference program) Instead of the aspect of the weight estimation device 10 according to the above embodiment, it may be provided in an aspect of an inference device (inference method or inference program) used to estimate the weight of cow C. In this case, the inference device includes a memory and a processor, and the processor among them executes a series of processes.

[0112] The first aspect of a series of processes includes an image acquisition process for acquiring at least one of a two-dimensional image and a depth image of a cow, and an inference process for inferring the body orientation of the cow imaged in the image when the image is acquired by the image acquisition process. The second aspect of a series of processes includes an image acquisition process for acquiring at least one of a two-dimensional image and a depth image of a cow, and an inference process for inferring a body part region including a predetermined body part of the cow imaged in the image when the image is acquired by the image acquisition process. The third aspect of a series of processes includes an image acquisition process for acquiring at least one of a two-dimensional image and a depth image of a cow, and an inference process for inferring the weight of the cow imaged in the image when the image is acquired by the image acquisition process. By providing in the manner of the above-described inference device (inference method or inference program), it becomes possible to easily apply to various devices as compared with the case of implementing the weight estimation device 10. It should be naturally understood by those skilled in the art that the inference device (inference method or inference program) may use the learned first to third learning models 11A to 11C generated by the machine learning device 2 according to the above embodiment when executing the inference process.

Explanation of Signs

[0113] 1... Cow management system, 2... Machine learning device, 3, 3a, 3b... Terminal device, 4... Management device, 5... Network, 10, 10a, 10b... Weight estimation device, 11A... First learning model, 11B... Second learning model, 11C... Third learning model, 12A... First learning data, 12B... Second learning data, 12C... Third learning data, 13... Imaging screen, 14... Weight estimation result screen, 20... Control unit, 21... Input unit, 22... Output unit, 23... Communication unit, 24... Learning data storage unit, 25... Learned model storage unit, 30... Control unit, 31... Input unit, 32... Output unit, 33... Communication unit, 34... Storage unit, 35... Camera, 40... Cow management database, 100, 100a, 100b... Body weight estimation program, 110A~110C... Input layer, 111A~111C... Intermediate layer, 112A~112C... Output layer, 130... Image display area, 131... Imaging button, 140... Image display area, 141... Cattle information display column, 142... Estimation result display column, 143... Registration button, 200... Learning data acquisition unit, 201A... First machine learning unit, 201B... Second machine learning unit, 201C... Third machine learning unit 300... Image acquisition unit, 301... Body orientation determination unit, 302... Body part image generation unit, 303... Body weight estimation unit, 304... Input / output processing unit, 400... Cattle information table, 401... Measurement information table 900... Computer

Claims

1. An image acquisition unit that acquires a depth image of a cow; By inputting the depth image into a first learning model obtained by machine learning the correlation between the first learning image corresponding to the depth image and the body orientation of the cow imaged in the first learning image, a body orientation determination unit that determines the body orientation of the cow imaged in the depth image; By inputting the depth image of the cow imaged in the predetermined body orientation into a second learning model obtained by machine learning the correlation between the second learning image corresponding to the depth image and the body part region including the predetermined body part of the cow imaged in the second learning image, a body part image generation unit that generates the depth image with the removal region other than the body part region removed from the depth image as a body part image; A weight estimation unit that estimates the weight of the cow imaged in the depth image by inputting the body part image into a third learning model obtained by machine learning the correlation between the third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image, and The predetermined body orientation is The body orientation in which the left side of the cow's torso is imaged in the depth image by imaging the depth image from the left rear or left side of the cow; The predetermined body part is The torso of the cow, or the torso and limbs of the cow, A weight estimation device.

2. An image acquisition unit that acquires a depth image of a cow imaged in a predetermined body orientation; By inputting the depth image into a second learning model obtained by machine learning the correlation between the second learning image corresponding to the depth image and the body part region including the predetermined body part of the cow imaged in the second learning image, a body part image generation unit that generates the depth image with the removal region other than the body part region removed from the depth image as a body part image; A weight estimation unit that estimates the weight of the cow imaged in the depth image by inputting the body part image into a third learning model obtained by machine learning the correlation between the third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image, and The predetermined body orientation is The body orientation in which the left side of the cow's torso is imaged in the depth image by imaging the depth image from the left rear or left side of the cow, and The predetermined body part is The torso of the cow, or the torso and limbs of the cow, A weight estimation device.

3. An image acquisition unit that acquires a depth image of a predetermined body part of a cow; ​ By inputting the depth image into a first learning model that has learned the correlation between the first learning image corresponding to the depth image and the body orientation of the cow imaged in the first learning image, a body orientation determination unit that determines the body orientation of the cow imaged in the depth image; A weight estimation unit that estimates the weight of the cow imaged in the depth image by inputting the depth image in which the body orientation of the cow is imaged in a predetermined body orientation into a third learning model that has learned the correlation between the third learning image corresponding to the depth image and the weight of the cow imaged in the third learning image, and The predetermined body orientation is The body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow, The predetermined body part is The body of the cow, or the body and limbs of the cow, A weight estimation device.

4. An image acquisition unit that acquires a depth image in which a predetermined body part of a cow is imaged in a predetermined body orientation, and A weight estimation unit that estimates the weight of the cow imaged in the depth image by inputting the depth image into a third learning model that has learned the correlation between the third learning image corresponding to the depth image and the weight of the cow imaged in the third learning image, and The predetermined body orientation is The body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow, The predetermined body part is The body of the cow, or the body and limbs of the cow, A weight estimation device.

5. The image acquisition unit Further acquires a two-dimensional image captured simultaneously with the depth image, and The body orientation determination unit Instead of the depth image, inputs the two-dimensional image into a first learning model that has learned the correlation between the first learning image corresponding to the two-dimensional image and the body orientation of the cow imaged in the first learning image, thereby determining the body orientation of the cow imaged in the depth image captured simultaneously with the two-dimensional image. The weight estimation device according to claim 1 or claim 3.

6. A machine learning device that generates a third learning model for estimating the weight of a cow based on a body part image obtained by removing a removal region other than the body part region including the predetermined body part from a depth image in which the predetermined body part of the cow is imaged in a predetermined body orientation. A learning data storage unit that stores a plurality of sets of third learning data composed of a third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image. A machine learning unit that inputs a plurality of sets of the third learning data into the third learning model to machine-learn the correlation between the third learning image and the weight in the third learning model. A learned model storage unit that stores the third learning model machine-learned by the machine learning unit. It is provided with. The predetermined body orientation is The body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow. The predetermined body part is The cow's body, or the cow's body and limbs. Machine learning device.

7. A machine learning device for generating a third learning model for estimating the weight of a cow based on a depth image of a predetermined body part of the cow imaged in a predetermined body orientation, A learning data storage unit that stores a plurality of sets of third learning data composed of a third learning image corresponding to the depth image and the weight of the cow imaged in the third learning image. A machine learning unit that inputs a plurality of sets of the third learning data into the third learning model to machine-learn the correlation between the third learning image and the weight in the third learning model. It is provided with a learned model storage unit that stores the third learning model machine-learned by the machine learning unit. The predetermined body orientation is The body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow. The predetermined body part is The cow's body, or the cow's body and limbs. Machine learning device.

8. An image acquisition step of acquiring a depth image of the cow, A body orientation determination step of determining the body orientation of the cow imaged in the depth image by inputting the depth image into a first learning model that has machine-learned the correlation between the first learning image corresponding to the depth image and the body orientation of the cow imaged in the first learning image. Inputting the depth image of the cow imaged in the predetermined body orientation into a second learning model that has learned the correlation between the depth image and a second learning image corresponding to the depth image and a body part region including a predetermined body part of the cow imaged in the second learning image, thereby generating a body part image by removing a removal region other than the body part region from the depth image. A weight estimation step of estimating the weight of the cow imaged in the depth image by inputting the body part image into a third learning model that has learned the correlation between the body part image and a third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image. The predetermined body orientation is a body orientation in which the depth image is taken from the left rear or left side of the cow, so that the left side of the cow's body is imaged in the depth image. The predetermined body part is the cow's body, or the cow's body and limbs. Weight estimation method.

9. An image acquisition step of acquiring a depth image of a cow imaged in a predetermined body orientation. Inputting the depth image into a second learning model that has learned the correlation between the depth image and a second learning image corresponding to the depth image and a body part region including a predetermined body part of the cow imaged in the second learning image, thereby generating a body part image by removing a removal region other than the body part region from the depth image. The body part image is input into a third learning model that has learned the correlation between the body part image and a third learning image corresponding to the body part image and the weight of the cow imaged in the third learning image, thereby estimating the weight of the cow imaged in the depth image. The weight estimation step is provided. The predetermined body orientation is a body orientation in which the depth image is taken from the left rear or left side of the cow, so that the left side of the cow's body is imaged in the depth image. The predetermined body part is the cow's body, or the cow's body and limbs. Weight estimation method.

10. An image acquisition step of acquiring a depth image of a predetermined body part of a cow. An orientation determination step of determining the body orientation of the cow imaged in the depth image by inputting the depth image into a first learning model that has learned the correlation between the depth image and a first learning image corresponding to the depth image and the body orientation of the cow imaged in the first learning image. A weight estimation step of estimating the weight of the cow imaged in the depth image by inputting the depth image in which the body orientation of the cow is imaged in a predetermined body orientation into a third learning model obtained by machine learning the correlation between the third learning image corresponding to the depth image and the weight of the cow imaged in the third learning image, The predetermined body orientation is a body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow, The predetermined body part is the body of the cow, or the body and limbs of the cow, A weight estimation method.

11. An image acquisition step of acquiring a depth image in which a predetermined body part of a cow is imaged in a predetermined body orientation, and a weight estimation step of estimating the weight of the cow imaged in the depth image by inputting the depth image into a third learning model obtained by machine learning the correlation between the third learning image corresponding to the depth image and the weight of the cow imaged in the third learning image, The predetermined body orientation is a body orientation in which the left side of the cow's body is imaged in the depth image by imaging the depth image from the left rear or left side of the cow, The predetermined body part is the body of the cow, or the body and limbs of the cow, A weight estimation method.

12. The image acquisition step further acquires a two-dimensional image captured simultaneously with the depth image, and The body orientation determination step determines the body orientation of the cow imaged in the depth image captured simultaneously with the two-dimensional image by inputting the two-dimensional image, instead of the depth image, into a first learning model obtained by machine learning the correlation between the first learning image corresponding to the two-dimensional image and the body orientation of the cow imaged in the first learning image. The weight estimation method according to claim 8 or claim 10.

13. A weight estimation program that causes a computer to function as each part included in the weight estimation device according to any one of claims 1 to 5. A weight estimation program.

Images