Program, information processing method, and information processing device

Abstract

To provide a program, etc. capable of estimating a body weight on the basis of a peripheral length of an object animal.SOLUTION: A program causes a computer to execute processing for acquiring image data capturing an object animal and three-dimensional coordinate data on the object animal, specifying a first feature point 11a and a second feature point 11b of the object animal in the image data, specifying a plurality of pieces of three-dimensional coordinate data corresponding to point groups between the first feature point 11a and the second feature point 11b, and deriving a peripheral length of the object animal on the basis of the plurality of pieces of specified three-dimensional coordinate data and deriving a body weight of the object animal on the basis of the derived peripheral length.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] In recent years, there has been a technology for estimating the weight of a cow. Patent Document 1 discloses a weight estimation device that estimates the weight of a cow from an image captured from a predetermined direction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2023-014766 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention of Patent Document 1 has a problem in that it is not possible to estimate the weight of a target animal (for example, a cow) based on its circumference.

[0005] One aspect of the present invention is to provide a program or the like that is capable of estimating the weight of a target animal based on its circumference. [Means for solving the problem]

[0006] A program relating to one aspect causes a computer to execute processes to acquire image data of a target animal and three-dimensional coordinate data of the target animal, identify a first feature point and a second feature point of the target animal in the image data, identify a plurality of three-dimensional coordinate data corresponding to a point cloud between the first feature point and the second feature point, derive the circumference of the target animal based on the identified plurality of three-dimensional coordinate data, and derive the weight of the target animal based on the derived circumference. Effect of the Invention

[0007] In one aspect, it is possible to estimate the body weight of a subject animal based on its circumference. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram showing an example of the configuration of a computer. [Diagram 2] FIG. 11 is an explanatory diagram showing an example of the record layout of a measurement result DB and a training data DB. [Diagram 3] FIG. 13 is an explanatory diagram for explaining a process for deriving the weight of a cow. [Figure 4] 10 is a flowchart showing a processing procedure for deriving the weight of a cow. [Diagram 5] FIG. 11 is a block diagram showing an example of the configuration of a computer according to a second embodiment. [Figure 6] FIG. 13 is an explanatory diagram illustrating an example of a record layout of a template data DB. [Figure 7] FIG. 1 is an explanatory diagram showing an example of a plurality of reference feature points of a cow. [Figure 8] FIG. 4 is an explanatory diagram for explaining a process of identifying a first feature point and a second feature point. [Figure 9] FIG. 11 is an explanatory diagram for explaining a process of identifying feature points. [Figure 10] 10 is a flowchart showing a processing procedure for identifying a first feature point and a second feature point. [Figure 11] 13 is a flowchart showing a procedure of a subroutine for correcting a mask image. [Figure 12] 13 is a flowchart showing a procedure of a subroutine for adjusting the size of a mask image. [Figure 13] 13 is a flowchart showing a procedure of a subroutine for processing to identify feature points. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof.

[0010] (Embodiment 1) The first embodiment relates to a form for deriving the weight of a target animal. The target animals include elephants, cows, horses, sheep, pigs, chickens, ducks, dogs, cats, mice, tuna, etc. It is to be noted that the present invention is not limited to target animals, and can be applied to objects, human beings, animal skeletons, internal organs, etc. in the same manner. The objects include manufactured products such as motorcycles, automobiles, or railroad cars, equipment, utility poles, or components, etc. In the following, a cow will be described as the target animal, but the present invention can be applied to other types of target animals in the same manner.

[0011] The system of this embodiment includes an information processing device 1. The information processing device 1 is an information processing device that processes, stores, transmits, receives, and otherwise processes various types of information. The information processing device 1 is an information processing device such as a smartphone, a mobile phone, a wearable device such as an Apple Watch (registered trademark), a wearable camera, a tablet, or a personal computer.

[0012] The information processing device 1 may be a server device or a cloud server device that provides functions as a cloud service. For simplicity, the information processing device 1 will be hereinafter referred to as a computer 1.

[0013] A computer 1 according to this embodiment acquires image data of a cow and three-dimensional coordinate data of the cow. The computer 1 identifies a first characteristic point and a second characteristic point of the cow in the acquired image data. The computer 1 identifies a plurality of three-dimensional coordinate data corresponding to a point cloud between the identified first characteristic point and second characteristic point. The computer 1 derives the circumference of the cow based on the identified plurality of three-dimensional coordinate data. The computer 1 derives the weight of the cow based on the derived circumference.

[0014] 1 is a block diagram showing an example of the configuration of a computer 1. The computer 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, a large-capacity storage unit 17, an imaging unit 18, and a 3D sensor 19. Each component is connected by a bus B.

[0015] The control unit 11 includes an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or a quantum processor, and performs various information processing, control processing, etc. related to the computer 1 by reading and executing a control program 1P (program product) stored in the memory unit 12.

[0016] The control program 1P can be deployed to be executed on a single computer, or on multiple computers located at one site, or distributed across multiple sites and interconnected by a communications network. Although the control unit 11 is described in FIG. 1 as being a single processor, it may be a multiprocessor.

[0017] The storage unit 12 includes memory elements such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing processing related to communication.

[0018] The input unit 14 may be a keyboard, a mouse, or a touch panel integrated with the display unit 15. The display unit 15 is a liquid crystal display, an organic EL (electroluminescence) display, or the like, and displays various information according to instructions from the control unit 11.

[0019] The reading unit 16 reads the portable storage medium 1a including a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the mass storage unit 17. The control unit 11 may also download the control program 1P from another computer via a network N or the like and store it in the mass storage unit 17. Furthermore, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0020] The mass storage unit 17 includes a recording medium such as a hard disk drive (HDD), a solid state drive (SSD), etc. The mass storage unit 17 includes a measurement result database (DB) 171, a training data DB 172, a circumference estimation model 173, and a weight estimation model 174.

[0021] Measurement result DB171 stores measurement results such as the circumference and weight of cows. Training data DB172 stores training data for constructing (creating) a circumference estimation model 173 and a weight estimation model 174. The circumference estimation model 173 is an estimator that estimates the circumference of cows, and is a trained model generated by machine learning. The weight estimation model 174 is an estimator that estimates the weight of cows, and is a trained model generated by machine learning.

[0022] In this embodiment, the storage unit 12 and the large-capacity storage unit 17 may be configured as an integrated storage device. The large-capacity storage unit 17 may be configured with a plurality of storage devices. Furthermore, the large-capacity storage unit 17 may be an external storage device connected to the computer 1.

[0023] The imaging unit 18 is, for example, an imaging device such as a CCD (Charge Coupled Device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, etc. The imaging unit 18 may not be built into the computer 1, but may be externally connected directly to the computer 1 and configured to be capable of imaging.

[0024] The 3D sensor 19 is a sensor capable of aligning an image of a target animal such as a cow captured by the imaging unit 18 with three-dimensional coordinate data. That is, the three-dimensional coordinate data is associated with each pixel of the image data.

[0025] The 3D sensor 19 may be, for example, a LiDAR sensor. The LiDAR sensor irradiates laser light and detects reflected or scattered light, and can detect three-dimensional point cloud data (X, Y, Z). In addition to the LiDAR sensor, a depth sensor or an infrared sensor may be used. A sensor in which the imaging unit 18 and the 3D sensor 19 are integrated may be used.

[0026] The computer 1 is not limited to being integrated with the 3D sensor 19, and the computer 1 and the 3D sensor 19 may be provided as separate devices.

[0027] The computer 1 may execute various information processing and control processing, etc., as a single computer, or may execute the processing in a distributed manner among multiple computers. Furthermore, the computer 1 may be realized by multiple virtual machines provided in a single computer.

[0028] The system is not limited to the above-mentioned configuration. For example, an information processing terminal (tablet, etc.) captures an image of a cow. The information processing terminal acquires three-dimensional coordinate data of the cow by a LiDAR sensor mounted on the information processing terminal. The information processing terminal transmits the acquired image data and three-dimensional coordinate data of the cow to a server device. The server device derives the weight of the cow based on the image data and three-dimensional coordinate data of the cow transmitted from the information processing terminal. The server device transmits the derived weight of the cow to the information processing terminal. The information processing terminal may receive the weight of the cow transmitted from the server device and display the received weight of the cow. Note that a cloud server may be used instead of the server device. That is, the information processing terminal captures an image of the cow, transmits the image data, and receives and displays the weight of the cow. The server device performs derivation processing (calculation processing) of the circumference, weight, etc. of the cow.

[0029] FIG. 2 is an explanatory diagram showing an example of the record layout of the measurement result DB 171 and the training data DB 172. As shown in FIG. The measurement result DB 171 includes an image ID string, an image data string, a three-dimensional coordinate data string, a measurement part string, a feature point name string, a feature point coordinate string, a circumference string, and a weight string. The image ID string stores a unique image data ID for identifying the image data of each cow.

[0030] The image data column stores image data of the cow. The three-dimensional coordinate data column stores three-dimensional coordinate data corresponding to each pixel of the cow's image data (two-dimensional image data). The measurement part column stores the names of the measurement parts of the cow. The feature point name column stores the names of the feature points in the cow's image data. The feature point coordinate column stores the coordinate values ​​of the feature points in the cow's image data. The circumference column stores the circumference of the cow. The weight column stores the weight of the cow.

[0031] The training data DB 172 includes a training ID column, a type column, an input column, and an output column. The training ID column stores a unique ID of each training data to identify the training data. The type column stores the type of the learning model including the circumference estimation model 173 and the weight estimation model 174.

[0032] When the type of learning model is "circumference estimation model 173", the input column stores the total length of the line segments of the cow, and the output column stores the circumference of the cow. The total length of the line segments will be described later. When the type of learning model is "weight estimation model 174", the input column stores the circumference of the cow, and the output column stores the weight of the cow.

[0033] The above-mentioned storage formats of each DB are merely examples, and other storage formats may be used as long as the relationships between the data are maintained.

[0034] Next, the process of deriving the weight of a cow will be described. First, the computer 1 acquires image data of an image of the cow and three-dimensional coordinate data of the cow. The image data of the cow is image data of the cow captured from the side. For example, the computer 1 acquires image data of the cow captured from the side via the imaging unit 18. Note that the computer 1 may acquire image data of the cow captured by an external information processing terminal having an imaging function.

[0035] The computer 1 acquires three-dimensional coordinate data of the cow. Specifically, the computer 1 acquires the three-dimensional coordinate data of the cow via the 3D sensor 19, which is a LiDAR sensor for detecting three-dimensional point cloud data. The three-dimensional coordinate data is data having three-dimensional coordinate values ​​for each pixel of the image (three-dimensional point cloud data).

[0036] Next, the computer 1 derives the weight of the cow. Fig. 3 is an explanatory diagram for explaining the process of deriving the weight of the cow. Fig. 3A is an explanatory diagram for explaining the process of identifying the first feature point and the second feature point. Fig. 3B is an explanatory diagram for explaining the process of deriving the total length. Fig. 3C is an explanatory diagram for explaining the process of deriving the circumference and weight of the cow.

[0037] The computer 1 identifies a first feature point 11a and a second feature point 11b of the cow in the acquired image data. As shown in the figure, the first feature point 11a is the upper coordinate of the chest depth, and the second feature point 11b is the lower coordinate of the chest depth. The chest depth is the linear distance from the upper edge of the thoracic vertebrae to the lower edge of the chest on the line along which the chest circumference is measured.

[0038] The positions of the first characteristic point 11a and the second characteristic point 11b are not limited to the positions exemplified in Fig. 3. For example, the positions of the first characteristic point 11a and the second characteristic point 11b may be the apex of the withers for measuring the withers height, and a ground measurement point that is vertically connected to the ground by the withers apex. Alternatively, the positions of the first characteristic point 11a and the second characteristic point 11b may be the cross for measuring the cross height, and a ground measurement point that is vertically connected to the ground by the cross.

[0039] Furthermore, the positions of the first characteristic point 11a and the second characteristic point 11b may be measurement points at both ends of the waist part passing through the base of the eighth rib behind the shoulder to measure the chest width (rib width at the chest measurement part). Furthermore, the positions of the first characteristic point 11a and the second characteristic point 11b may be measurement points at the outer edge of the left waist angle and measurement points at the outer edge of the right waist angle to measure the waist angle width (straight line distance between the outer edges of the left and right waist angles).

[0040] Regarding the process of identifying the first feature point 11a and the second feature point 11b, for example, the computer 1 may accept designation (input) of the first feature point 11a and the second feature point 11b by the user for the image data. Note that the first feature point 11a and the second feature point 11b are not limited to being manually identified by the user, and can be automatically identified, for example, by using the template data in the second embodiment.

[0041] The computer 1 acquires (specifies) from the measurement result DB 171, three-dimensional coordinate data corresponding to the received first feature point 11a and second feature point 11b.

[0042] In the cow's contour region, the brightness or phase is unstable due to the influence of noise, etc., so if the three-dimensional coordinate data of the contour region acquired by the 3D sensor 19 is used, the derivation process of the total length described below may become inaccurate due to the influence of noise, etc. By using the first offset point 11c and the second offset point 11d that are offset toward the inside of the cow's body, it is possible to prevent an inaccurate derivation of the total length. The first offset point 11c is a point offset downward from the first characteristic point 11a, and the second offset point 11d is a point offset upward from the second characteristic point 11b.

[0043] The first offset point 11c and the second offset point 11d are obtained by equally dividing the line segment 12a at a predetermined interval. Specifically, the computer 1 identifies a line segment 12a connecting the identified first feature point 11a and second feature point 11b. The computer 1 divides the identified line segment 12a at a predetermined interval. As an example, the predetermined interval may be an interval obtained by dividing the line segment 12a into five equal parts. As shown in the figure, the line segment 12a is divided into five equal parts, into intervals 13a, 13b, 13c, 13d, and 13e at equal intervals.

[0044] The computer 1 identifies a first offset point 11c belonging to interval 13b adjacent to the lower side of interval 13a to which the first feature point 11a belongs. The computer 1 identifies a second offset point 11d belonging to interval 13d adjacent to the upper side of interval 13e to which the second feature point 11b belongs.

[0045] The computer 1 identifies a plurality of three-dimensional coordinate data corresponding to a plurality of point groups between the identified first offset point 11c and the second offset point 11d. Specifically, the computer 1 acquires (identifies) three-dimensional coordinate data corresponding to each point between the first offset point 11c and the second offset point 11d from the measurement result DB 171. As shown in FIG. 3B, three-dimensional coordinate data corresponding to each pixel (e.g., point 11e and point 11f) is identified between the first offset point 11c and the second offset point 11d. The computer 1 derives the total length of the line segments between each identified three-dimensional coordinate data.

[0046] The computer 1 derives the circumference of the cow from the total length of the derived line segment lengths. Specifically, the computer 1 inputs the derived total length to a circumference estimation model 173, and derives (outputs) the circumference of the cow. The circumference estimation model 173 is used as a program module that is part of artificial intelligence (AI) software.

[0047] The circumference estimation model 173 may be realized by using an artificial intelligence technique such as machine learning or deep learning. The circumference estimation model 173 is an estimator that has been constructed (generated) based on the correlation between the total length of the line segments of the measurement target (measurement site) area (e.g., chest depth) of the cow and the circumference of the cow. For example, the circumference estimation model 173 uses training data accumulated in the training data DB 172 to learn the correlation between the total length of the line segments of the measurement target area of ​​the cow and the circumference of the cow. The circumference estimation model 173 estimates the circumference of the cow from the total length of the line segments of the measurement target area of ​​the cow that has been input based on the learned learning result.

[0048] The training data is data generated by associating the total length of the line segments of the measurement target area (e.g., chest depth) of the cow with the circumference of the cow obtained from past measurements. The circumference estimation model 173 may be generated by the computer 1, an external information processing device, a cloud server, or the like.

[0049] When the computer 1 acquires the total length of the line segments in the measurement target area of ​​the cow, the computer 1 inputs the acquired total length into the circumference estimation model 173 and outputs the estimation result of the circumference of the cow. For example, the estimated circumference of the cow is 129 cm.

[0050] The perimeter estimation model 173 may be a trained model constructed using any learning algorithm, such as linear regression analysis, multivariate adaptive regression splines (MARS), support vector regression (SVR), regression tree, model tree, genetic program, binary classification, logistic regression, k-nearest neighbor method, support vector machine, decision tree, or random forest.

[0051] In addition, the circumference of a cow can be calculated using an estimation formula that shows the relationship between the total length and the circumference. For example, the circumference can be calculated using the following formula (1). Estimated circumference (cm) = k1 × total length (cm) … (1) k1 is a coefficient (e.g., a positive number) for calculating the circumference, and is calculated, for example, by multivariate curve fitting. Note that k1 may be set by a system administrator or the like. For example, k1 may be 2.5. The computer 1 derives (calculates) the circumference of the cow using the above-mentioned formula (1) based on the total length of the line segments of the cow's chest depth.

[0052] The computer 1 derives the weight of the cow from the derived circumference of the cow. Specifically, the computer 1 inputs the derived circumference of the cow into a weight estimation model 174, and derives (outputs) the weight of the cow. The weight estimation model 174 is used as a program module that is part of artificial intelligence software.

[0053] The weight estimation model 174 may be realized by using an artificial intelligence technique such as machine learning or deep learning. The weight estimation model 174 is an estimator that has been constructed (generated) based on the correlation between the circumference of a measurement target area of ​​a cow (e.g., chest depth) and the weight of the cow. For example, the weight estimation model 174 uses training data accumulated in the training data DB 172 to learn the correlation between the circumference of a measurement target area of ​​a cow and the weight of the cow. The weight estimation model 174 estimates the weight of the cow from the circumference of the measurement target area based on the learned learning result.

[0054] The training data is data generated by associating the circumference of a measurement target area (e.g., chest depth) of a cow with the weight of the cow obtained from past measurements. The weight estimation model 174 may be generated by the computer 1, an external information processing device, a cloud server, or the like.

[0055] When the computer 1 acquires the perimeter of the measurement target region of the cow, it inputs the acquired perimeter to the weight estimation model 174 and outputs the estimation result of the weight of the cow. For example, the estimated weight of the cow is 382 kg.

[0056] The weight estimation model 174 may be a trained model constructed using any learning algorithm, such as linear regression analysis, MARS, support vector regression, regression tree, model tree, genetic program, binary classification, logistic regression, k-nearest neighbor method, support vector machine, decision tree, or random forest.

[0057] In addition, the weight of a cow can be calculated using an estimation formula that shows the relationship between the circumference and weight of the cow. For example, the estimation formula for weight is expressed as the following formula (2). Estimated weight (kg) = k2-k3 x circumference (cm) + k4 x [perimeter (cm)] 2 …(2) k2, k3 and k4 are coefficients (e.g., positive numbers) for calculating the weight, and are calculated, for example, by multivariate curve fitting. Note that k2, k3 and k4 may be set by a system administrator or the like. For example, k2 may be 55, k3 may be 2.2, and k4 may be 0.02. The computer 1 derives (calculates) the weight of the cow based on the circumference of the cow using the above-mentioned formula (2).

[0058] 4 is a flow chart showing the processing procedure for deriving the weight of a cow. The control unit 11 of the computer 1 acquires image data of the cow taken from the side by the imaging unit 18 (step S101). The control unit 11 acquires three-dimensional coordinate data corresponding to each pixel of the acquired image data via the 3D sensor 19 (step S102).

[0059] The control unit 11 stores the acquired image data of the cow and the three-dimensional coordinate data corresponding to each pixel of the image data as one record in the measurement result DB 171 of the mass storage unit 17 in association with the image ID (step S103).

[0060] The control unit 11 identifies the first feature point 11a and the second feature point 11b of the cow in the acquired image data (step S104). For example, the control unit 11 may accept a user's designation (input) of the first feature point 11a and the second feature point 11b for the image data through the input unit 14. The control unit 11 identifies three-dimensional coordinate data corresponding to the identified first feature point 11a and second feature point 11b from the measurement result DB 171 of the mass storage unit 17 (step S105).

[0061] The control unit 11 identifies a first offset point 11c offset from the identified first feature point 11a and a second offset point 11d offset from the identified second feature point 11b (step S106). Specifically, the control unit 11 identifies a line segment 12a connecting the identified first feature point 11a and the second feature point 11b. The control unit 11 equally divides the identified line segment 12a into a predetermined interval (for example, an interval obtained by dividing into five equal parts). The control unit 11 identifies a first offset point 11c belonging to an interval adjacent to the lower side of the interval to which the first feature point 11a belongs. The control unit 11 identifies a second offset point 11d belonging to an interval adjacent to the upper side of the interval to which the second feature point 11b belongs.

[0062] The control unit 11 identifies a plurality of three-dimensional coordinate data corresponding to a plurality of points between the identified first offset point 11c and the second offset point 11d (step S107). Specifically, the control unit 11 identifies (acquires) three-dimensional coordinate data corresponding to each point between the identified first offset point 11c and the second offset point 11d from the measurement result DB 171 of the mass storage unit 17. The control unit 11 derives the total length of the line segments between the three-dimensional coordinate data of each identified point (step S108).

[0063] The control unit 11 derives the circumference of the cow from the derived total length (step S109). Specifically, the control unit 11 inputs the derived total length to a circumference estimation model 173, and outputs (derives) the circumference of the cow. The control unit 11 derives the weight of the cow from the derived circumference of the cow (step S110). Specifically, the control unit 11 inputs the derived circumference of the cow to a weight estimation model 174, and outputs (derives) the weight of the cow.

[0064] The control unit 11 stores the measurement results in the measurement result DB 171 of the mass storage unit 17 (step S111), and ends the process. Specifically, the control unit 11 stores in the measurement result DB 171 the measurement site (e.g., chest depth), the names (e.g., upper chest deep and lower chest deep) and coordinates of the feature points (first feature point 11a and second feature point 11b), as well as the derived circumference and weight of the cow, in association with the image ID.

[0065] According to this embodiment, it is possible to derive the perimeter of a target animal based on image data of the target animal and three-dimensional coordinate data of the target animal.

[0066] According to this embodiment, it is possible to derive the weight of a target animal based on the circumference of the target animal.

[0067] According to this embodiment, it is possible to derive the perimeter of a target animal based on equation (1) showing the relationship between the total length and perimeter of the target animal, or based on the perimeter estimation model 173.

[0068] According to this embodiment, it is possible to derive the weight of the target animal based on the equation (2) showing the relationship between the circumference and the weight of the target animal, or based on the weight estimation model 174.

[0069] (Embodiment 2) The second embodiment relates to an embodiment in which the first feature point 11a and the second feature point 11b are automatically specified by using image data of a target animal and template data of a measurement target area of ​​the target animal. Note that a description of the contents overlapping with the first embodiment will be omitted. Note that, although the following description will be given assuming that a cow is a target animal, the same can be applied to other types of target animals.

[0070] Fig. 5 is a block diagram showing an example of the configuration of the computer 1 in the embodiment 2. Note that the same reference numerals are used for the same contents as in Fig. 1, and the description thereof will be omitted. The large-capacity storage unit 17 includes a template data DB 175. The template data DB 175 stores template data of the measurement target area of ​​the cow.

[0071] 6 is an explanatory diagram showing an example of a record layout of template data DB 175. Template data DB 175 includes a template ID (Identifier) ​​string, an image data string, a measurement site string, a reference feature point name string, and a reference feature point coordinate string.

[0072] The template ID column stores a unique ID of template data to identify the template data of the measurement target area of ​​each cow. The image data column stores reference image data of the measurement target area of ​​the cow.

[0073] The measurement part column stores the names of measurement parts of the cow. Measurement parts include chest depth, body length, etc. The measurement parts may also include withers height, horizontal body length, cross height, etc. The reference feature point name column stores the names of reference feature points of the cow in the template data. The reference feature point coordinate column stores the coordinate values ​​of reference feature points of the cow in the template data. The template data will be described later.

[0074] 7 is an explanatory diagram showing an example of a plurality of reference characteristic points of a cow. In order to measure the body size of the cow, a plurality of reference characteristic points are provided in advance in the measurement target area of ​​the cow. As shown in the figure, there is a reference characteristic point 14a indicating the upper deep chest area, and a reference characteristic point 14b indicating the lower deep chest area. The cow's chest circumference can be measured based on the reference characteristic point 14a (upper deep chest area) and the reference characteristic point 14b (lower deep chest area).

[0075] It should be noted that the reference characteristic points are not limited to those shown in Fig. 7. For example, a reference characteristic point indicating the shoulder end, a reference characteristic point indicating the ischial end, or a reference characteristic point indicating the cross part (the intersection of a line connecting both hip angles of the cow and the spine) may be provided. The body length of the cow can be measured based on the reference characteristic point indicating the shoulder end and the reference characteristic point indicating the ischial end. The cross height of the cow can be measured based on the reference characteristic point indicating the cross part.

[0076] FIG. 8 is an explanatory diagram for explaining the process of identifying the first feature point 11a and the second feature point 11b. First, the computer 1 acquires image data of the cow to be measured. The computer 1 extracts segmentation data (hereinafter, referred to as a mask image) of the measurement target area of ​​the cow from the acquired image data of the cow. The measurement target area is, for example, the body of the cow excluding the legs, tail, and head. For example, the computer 1 may extract a mask image of the measurement target area of ​​the cow using an image segmentation model constructed by an instance segmentation network.

[0077] The image segmentation model is trained using image data of a cow captured from the side. The image segmentation model is set to output an image in which the cow's torso is white and the other areas are black, so training is performed so that the output image approaches the correct image. Computer 1 inputs the acquired cow image data into the image segmentation model, and outputs a mask image of the area to be measured. In the output mask image, the area corresponding to the cow's torso (foreground pixel area) is white (pixel value 1) and the other areas (background pixel area) are black (pixel value 0).

[0078] The image segmentation model may be a SegNet model, or U-NET (Convolutional Networks for Biomedical Image Segmentation), which is one of the models of semantic segmentation. U-NET is a type of FCN (Fully Convolutional Networks) and includes an encoder that performs downsampling and a decoder that performs upsampling. U-NET is a neural network that does not have a fully connected layer and is composed only of a convolution layer and a pooling layer.

[0079] In addition to the above-mentioned image segmentation model, a background subtraction method or the like can be used. By applying the background subtraction method, a mask image is generated as a result of the background subtraction.

[0080] Next, if there is a white portion in the background pixel portion of the extracted mask image, the computer 1 corrects the mask image based on the three-dimensional coordinate data. As shown in the figure, since there is a white area 20a in the background pixel portion, it is necessary to correct the mask image using the three-dimensional coordinate data.

[0081] Specifically, the computer 1 acquires three-dimensional coordinate data by a 3D sensor 19 that measures depth (distance) information to the cow. The computer 1 corrects the extracted mask image based on the acquired three-dimensional coordinate data. As shown in the figure, the three-dimensional coordinate data is data having three-dimensional coordinate values ​​including an X axis (first coordinate axis), a Y axis (second coordinate axis), and a Z axis.

[0082] The X-axis indicates the horizontal direction (left-right direction) of the cow, with the right direction on the paper as the positive direction. The Y-axis indicates the vertical direction (up-down direction) of the cow, with the downward direction on the paper as the positive direction. The Z-axis indicates the viewing direction of the pixels (white pixels) that are the cow's torso (perpendicular to the paper in Figure 8).

[0083] For example, the computer 1 calculates a depth difference value that indicates the difference in depth between the depth (Z-axis value) in the viewpoint direction of a pixel (white pixel) that is the cow's body and the depth in the viewpoint direction of a pixel that is the target (white region 20a). If the calculated depth difference value is equal to or greater than a predetermined threshold, the computer 1 sets the pixel value of the target pixel to 0, thereby making the pixel black. If the depth difference value is less than the predetermined threshold, the pixel value of the target pixel is maintained as is.

[0084] The threshold value of the depth difference value may be, for example, 50 centimeters (cm). The threshold value of the depth difference value may be set appropriately depending on the distance between the imaging device and the cow at the time of imaging. Alternatively, a learning model that has been trained to output the threshold value of the depth difference value when image data and three-dimensional coordinate data of the cow are input may be used. If the depth difference value is outside a predetermined threshold range (for example, 30 cm to 50 cm), the pixel may be determined to be a pixel other than the cow's torso, and the pixel value of the pixel may be set to 0.

[0085] The computer 1 identifies a plurality of feature points (such as the first feature point 11a and the second feature point 11b) in the mask image. Fig. 9 is an explanatory diagram for explaining the process of identifying feature points.

[0086] The computer 1 adjusts the size of the corrected mask image. Specifically, the computer 1 acquires template data (hereinafter, referred to as a template mask image) of a measurement target area (for example, a cow's trunk) from the template data DB 175.

[0087] The template mask image is image data of a measurement target area including predetermined names and coordinate values ​​of a plurality of reference feature points. The template mask image may be created manually by a user, or may be generated using a template mask image output model that has been trained to output a plurality of reference feature points in the measurement target area of ​​a cow when image data of the cow is input.

[0088] The computer 1 obtains a rectangle 15a based on the maximum connected component of the obtained measurement target area from the template mask image of the measurement target area. The maximum connected component refers to the largest area among areas having pixel-to-pixel continuity. As shown in the figure, the maximum connected component is a white area in the cow's body.

[0089] Specifically, the computer 1 acquires a rectangle 15a based on the maximum and minimum values ​​of the X-axis and the maximum and minimum values ​​of the Y-axis in the maximum connected component of the measurement target area. The computer 1 calculates the size of the acquired rectangle 15a based on the coordinate values ​​(x1, y1) of the upper left pixel and the coordinate values ​​(x2, y2) of the lower right pixel of the rectangle 15a.

[0090] Based on the pixel values ​​of each pixel of the mask image to be measured, the computer 1 identifies a measurement target region in which the pixel value of the pixels is 1. The computer 1 acquires a rectangle 15b obtained based on the maximum connected component of the identified measurement target region. The computer 1 adjusts (changes) the size of the mask image based on the calculated size of the rectangle 15a. Specifically, the computer 1 adjusts the size of the mask image so that the width and height of the acquired rectangle 15b are the same as the width and height of the rectangle 15a.

[0091] The computer 1 identifies a plurality of feature points in the resized mask image. Specifically, the computer 1 determines the X-axis value of each feature point in the resized mask image based on the X-axis value of each reference feature point in the template mask image. As shown in the figure, the X-axis value (x) of the reference feature point P in the template mask image is set to the X-axis value (x) of the feature point Q in the mask image.

[0092] The computer 1 calculates a division ratio based on the Y-axis value of each reference feature point in the template mask image and the maximum and minimum Y-axis values ​​of the maximum connected component in the measurement target area. The computer 1 determines the Y-axis value of each feature point in the resized mask image based on the calculated division ratio.

[0093] As shown in the figure, the computer 1 calculates distances d1 and d2 in the maximum connected component of the measurement target region of the template mask image. Distance d1 is the distance between the Y-axis value (y) of the reference feature point P and the minimum Y-axis value (ymin) of the maximum connected component of the measurement target region corresponding to the reference feature point P. Distance d2 is the distance between the Y-axis value (y) of the reference feature point P and the maximum Y-axis value (ymax) of the maximum connected component of the measurement target region corresponding to the reference feature point P.

[0094] The computer 1 calculates a division ratio at the reference feature point P based on the vertical width (ymax-ymin) of the maximum connected component of the measurement target region corresponding to the reference feature point P and the calculated distances d1 and d2. For example, the calculated division ratio is 2:3.

[0095] The computer 1 acquires the maximum (y'max) and minimum (y'min) values ​​on the Y axis of the maximum connected component in the measurement target region of the mask image after adjusting the size, in association with the X axis value (x) of the determined mask image feature point Q. The computer 1 determines the Y axis value (y') of feature point Q based on the calculated division ratio and the maximum (y'max) and minimum (y'min) values ​​on the Y axis of the maximum connected component in the measurement target region of the acquired mask image. As shown in the figure, the coordinate values ​​of the determined feature point Q are (x, y').

[0096] Next, returning to FIG. 8, the computer 1 stores the identified multiple feature points (first feature point 11a and second feature point 11b) in the measurement result DB 171. Specifically, the computer 1 acquires the coordinate values ​​of each of the identified multiple feature points. The computer 1 assigns an image ID to the image data of the cow. The computer 1 stores the image data of the cow, the name of the measurement site, and the name and coordinate values ​​of each feature point in the measurement result DB 171 in association with the assigned image ID. The computer 1 displays the identified multiple feature points (first feature point 11a and second feature point 11b) by superimposing them on the image data of the cow.

[0097] 10 is a flowchart showing the processing procedure for identifying the first feature point 11a and the second feature point 11b. The control unit 11 of the computer 1 acquires image data of a cow captured from the side by the imaging unit 18 (step S121). The control unit 11 extracts a mask image of the measurement target area of ​​the cow using an image segmentation model constructed by U-NET or the like (step S122).

[0098] The control unit 11 executes a subroutine for correcting the mask image (step S123). The control unit 11 executes a subroutine for adjusting the size of the mask image (step S124). The control unit 11 executes a subroutine for identifying feature points (first feature point 11a and second feature point 11b) (step S125). The subroutines for correcting the mask image, adjusting the size of the mask image, and identifying the feature points will be described later.

[0099] The control unit 11 stores the measurement results in the measurement result DB 171 of the mass storage unit 17 (step S126). Specifically, the control unit 11 assigns an image ID to the image data of the cow. The control unit 11 stores the image data of the cow, the name of the measurement site, and the name and coordinate values ​​of each characteristic point in the measurement result DB 171 in association with the assigned image ID. The control unit 11 displays the identified first characteristic point 11a and second characteristic point 11b superimposed on the image data of the cow via the display unit 15 (step S127), and ends the process.

[0100] 11 is a flow chart showing the processing procedure of a subroutine of the process of correcting a mask image. The control unit 11 of the computer 1 acquires three-dimensional coordinate data by the 3D sensor 19 that measures depth information to the cow (step S01). The control unit 11 acquires three-dimensional coordinate data of a pixel (white pixel) that is a cow from the acquired three-dimensional coordinate data (step S02). The pixel may be, for example, any pixel in the area that is the cow's body.

[0101] The control unit 11 acquires one pixel from the mask image to be measured (step S03). The control unit 11 determines whether the acquired pixel is black or not (step S04). For example, if the pixel value of a pixel is 0, the control unit 11 determines that the pixel is black. Alternatively, if the pixel value of a pixel is 1, the control unit 11 determines that the pixel is white.

[0102] When the control unit 11 determines that the acquired pixel is black (YES in step S04), the process returns to step S03. When the control unit 11 determines that the acquired pixel is not black (NO in step S04), the control unit 11 determines whether or not three-dimensional coordinate data of the pixel exists based on the three-dimensional coordinate data acquired in the process of step S01 (step S05). When the control unit 11 determines that three-dimensional coordinate data of the pixel does not exist (NO in step S05), the process proceeds to step S08, which will be described later.

[0103] When the control unit 11 determines that the three-dimensional coordinate data of the pixel exists (YES in step S05), it calculates a depth difference value between the depth (Z-axis value) in the viewpoint direction of the pixel (white pixel) that is the cow (e.g., the cow's body) acquired in the process of step S02 and the depth in the viewpoint direction of the pixel (step S06).The control unit 11 determines whether the calculated depth difference value is equal to or greater than a predetermined threshold value (e.g., 50 cm) (step S07).

[0104] If the calculated depth difference value is less than a predetermined threshold value (NO in step S07), the control unit 11 proceeds to processing in step S09, which will be described later. If the calculated depth difference value is equal to or greater than a predetermined threshold value (YES in step S07), the control unit 11 sets the pixel value of the pixel to 0, thereby setting the pixel to black (step S08). The control unit 11 determines whether the pixel is the last pixel in the mask image (step S09).

[0105] If the control unit 11 determines that the pixel is not the last pixel (NO in step S09), the process returns to step S03. If the control unit 11 determines that the pixel is the last pixel (YES in step S09), the control unit 11 ends the subroutine of the correction process of the mask image and returns.

[0106] 12 is a flowchart showing the procedure of a subroutine for adjusting the size of a mask image. The control unit 11 of the computer 1 acquires a template mask image of a measurement target area (e.g., a cow's trunk) from the template data DB 175 of the mass storage unit 17 (step S11).

[0107] 9, the control unit 11 acquires a first rectangle (rectangle 15a) based on the maximum connected component of the acquired measurement target region from the template mask image of the measurement target region (step S12). The control unit 11 calculates the size of the acquired first rectangle (rectangle 15a) based on the pixel coordinate values ​​(x1, y1) of the upper left and pixel coordinate values ​​(x2, y2) of the lower right of the first rectangle (rectangle 15a) (step S13).

[0108] Based on the pixel values ​​of each pixel of the mask image to be measured, the control unit 11 identifies a measurement target region in which the pixel value is 1 (step S14). The control unit 11 acquires a second rectangle (rectangle 15b) obtained based on the maximum connected component of the identified measurement target region (step S15).

[0109] The control unit 11 adjusts the size of the measurement target area in the mask image based on the calculated size of the first rectangle (rectangle 15a) (step S16). Specifically, the control unit 11 adjusts the size of the mask image so that the width and height of the second rectangle (rectangle 15b) obtained based on the maximum connected component of the measurement target area in the mask image are the same as the width and height of the first rectangle (rectangle 15a) obtained based on the maximum connected component of the measurement target area in the template mask image. The control unit 11 ends the subroutine of the mask image size adjustment process and returns.

[0110] 13 is a flowchart showing the procedure of a subroutine for identifying feature points. The control unit 11 of the computer 1 acquires a plurality of reference feature points in a corresponding template mask image from the template data DB 175 of the mass storage unit 17 based on the template ID (step S21). The control unit 11 acquires the maximum connected component of a measurement target region (e.g., a cow's trunk) in the template mask image (step S22).

[0111] The control unit 11 acquires the coordinate value of one of the acquired reference feature points (step S23). The control unit 11 determines the value of a first coordinate axis (e.g., X-axis) of the feature point to be identified (step S24). Specifically, the control unit 11 sets the value of the first coordinate axis of the acquired reference feature point to the value of the first coordinate axis of the feature point.

[0112] The control unit 11 acquires the maximum and minimum values ​​of the second coordinate axis (e.g., Y axis) of the maximum connected component of the measurement target region in association with the value of the first coordinate axis (e.g., X axis) of the reference feature point for the measurement target region of the template mask image (step S25). The control unit 11 calculates a division ratio at the reference feature point based on the value of the second coordinate axis of the reference feature point and the maximum and minimum values ​​of the second coordinate axis of the acquired maximum connected component (step S26).

[0113] The control unit 11 determines the value of the second coordinate axis of the feature point based on the calculated division ratio (step S27). Specifically, the control unit 11 acquires the maximum and minimum values ​​of the second coordinate axis of the maximum connected component of the measurement target region of the mask image in association with the value of the first coordinate axis of the feature point determined in the process of step S24. The control unit 11 determines the value of the second coordinate axis of the feature point based on the calculated division ratio and the maximum and minimum values ​​of the second coordinate axis of the maximum connected component of the measurement target region of the mask image acquired.

[0114] The control unit 11 determines whether or not the reference feature point in question is the last reference feature point from among the multiple reference feature points in the template mask image (step S28). If the control unit 11 determines that the reference feature point in question is not the last reference feature point (NO in step S28), the process returns to step S23. If the control unit 11 determines that the reference feature point in question is the last reference feature point (YES in step S28), the control unit 11 outputs the identified multiple feature points (step S29). The control unit 11 ends the subroutine for identifying feature points and returns.

[0115] According to this embodiment, it is possible to specify (automatically extract) the first feature point 11a and the second feature point 11b based on image data of the target animal and template data of the measurement target area of ​​the target animal.

[0116] According to this embodiment, the first feature point 11a and the second feature point 11b are automatically extracted, thereby improving the efficiency of deriving the weight of the target animal.

[0117] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the scope and meaning equivalent to the claims.

[0118] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. The claims may also be written in a format in which a multiple claim cites at least one other multiple claim (multi-multi claim). [Explanation of symbols]

[0119] 1. Information processing equipment (computer) 11 Control section 12 Storage section 13. Communications Department 14 Input section 15 Display 16 Reading unit 17 Mass storage 171 Measurement result DB 172 Training Data DB 173 Circumference Estimation Model 174 Body Weight Estimation Model 175 Template Data DB 18 Imaging section 19 3D Sensor 1a Removable storage media 1b Semiconductor memory 1P control program

Claims

1. Acquiring image data of a target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; Identifying a plurality of three-dimensional coordinate data corresponding to a point group between the first feature point and the second feature point; Deriving a perimeter of the target animal based on the identified three-dimensional coordinate data; The weight of the target animal is derived based on the derived circumference. A program that causes a computer to carry out processing.

2. Identifying a plurality of three-dimensional coordinate data corresponding to a plurality of points equally divided at a predetermined interval on a line segment connecting a first offset point offset from the first feature point and a second offset point offset from the second feature point. The program according to claim 1.

3. The total length of the line segments between each of the identified three-dimensional coordinate data is calculated.

3. The program according to claim 1 or 2.

4. The circumference of the target animal is derived from the derived total length based on an estimation equation showing the relationship between the total length and the circumference of the target animal, or based on a first learning model trained to output the circumference of the target animal when the total length of the target animal is input. The program according to claim 3.

5. The weight of the target animal is derived from the derived circumference based on an estimation equation showing the relationship between the circumference and weight of the target animal, or a second learning model trained to output the weight of the target animal when the circumference of the target animal is input.

3. The program according to claim 1 or 2.

6. Extracting segmentation data of a measurement target area of ​​the target animal from the image data; acquiring template data of the measurement target area and a plurality of reference feature points of the target animal in the template data; Identifying a first feature point and a second feature point of the target animal in the extracted segmentation data based on the acquired template data and a plurality of reference feature points.

3. The program according to claim 1 or 2.

7. Acquiring image data of a target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; Identifying a plurality of three-dimensional coordinate data corresponding to a point group between the first feature point and the second feature point; Deriving a perimeter of the target animal based on the identified three-dimensional coordinate data; The weight of the target animal is derived based on the derived circumference. Information processing methods.

8. An information processing device including a control unit, The control unit is Acquiring image data of a target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; Identifying a plurality of three-dimensional coordinate data corresponding to a point group between the first feature point and the second feature point; Deriving a perimeter of the target animal based on the identified three-dimensional coordinate data; The weight of the target animal is derived based on the derived circumference. Information processing device.

Images