Program, information processing method and information processing device

Abstract

To provide a program, etc. capable of estimating a body weight on the basis of a peripheral length of an object animal.SOLUTION: A program causes a computer to execute processing for acquiring image data capturing an object animal and three-dimensional coordinate data on the object animal, specifying a first feature point 11a and a second feature point 11b of the object animal in the image data, specifying a plurality of pieces of three-dimensional coordinate data corresponding to point groups between the first feature point 11a and the second feature point 11b, and deriving a peripheral length of the object animal on the basis of the plurality of pieces of specified three-dimensional coordinate data and deriving a body weight of the object animal on the basis of the derived peripheral length.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] In recent years, there has been a technology for estimating the weight of cattle. Patent Document 1 discloses a weight estimation device that estimates the weight of a cattle from an image captured from a predetermined direction. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-014766 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the invention of Patent Document 1 has a problem in that it is not possible to estimate the weight of a target animal (for example, a cow) based on its circumference.

[0005] One aspect of the present invention is to provide a program or the like that can estimate the weight of a target animal based on its circumference. [Means for solving the problem]

[0006] A program according to one aspect includes acquiring image data of an image of a target animal and three-dimensional coordinate data of the target animal, identifying a first feature point and a second feature point of the target animal in the image data, and a first offset point offset from the Second feature a plurality of equal divisions at predetermined intervals on a line segment connecting the first offset point and the second offset point offset from the first offset point; The computer is caused to execute a process of identifying a plurality of three-dimensional coordinate data corresponding to the point cloud, deriving the perimeter of the target animal based on the identified plurality of three-dimensional coordinate data, and deriving the weight of the target animal based on the derived perimeter. [Effects of the Invention]

[0007] In one aspect, it is possible to estimate the body weight of a subject animal based on its circumference. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of a computer. [Figure 2] FIG. 10 is an explanatory diagram showing an example of the record layout of a measurement result DB and a training data DB. [Figure 3] FIG. 10 is an explanatory diagram illustrating a process for deriving the weight of a cow. [Figure 4] 10 is a flowchart showing a processing procedure for deriving the weight of a cow. [Figure 5] FIG. 10 is a block diagram showing an example of the configuration of a computer according to a second embodiment. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of a record layout of a template data DB. [Figure 7] FIG. 10 is an explanatory diagram showing an example of a plurality of reference feature points of a cow. [Figure 8] FIG. 10 is an explanatory diagram illustrating a process of identifying a first feature point and a second feature point. [Figure 9] FIG. 10 is an explanatory diagram illustrating a process of identifying feature points. [Figure 10] 10 is a flowchart showing a processing procedure for identifying first feature points and second feature points. [Figure 11] 10 is a flowchart showing the procedure of a subroutine for processing to correct a mask image. [Figure 12] 10 is a flowchart showing the procedure of a subroutine for processing to adjust the size of a mask image. [Figure 13] 10 is a flowchart showing the procedure of a subroutine for processing to identify feature points. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention will be described in detail below with reference to the drawings showing embodiments thereof.

[0010] (Embodiment 1) The first embodiment relates to a form for deriving the weight of a target animal. Target animals include elephants, cows, horses, sheep, pigs, chickens, ducks, dogs, cats, mice, and tuna. The present invention is not limited to target animals, and can be applied to objects, humans, animal skeletons, internal organs, and the like. Objects include manufactured products such as motorcycles, automobiles, and railroad cars, equipment, utility poles, and components. While the following description focuses on cows as the target animal, the present invention can be applied to other types of target animals as well.

[0011] The system of this embodiment includes an information processing device 1. The information processing device 1 processes, stores, transmits, and receives various types of information. The information processing device 1 is an information processing device such as a smartphone, a mobile phone, a wearable device such as an Apple Watch (registered trademark), a wearable camera, a tablet, or a personal computer.

[0012] The information processing device 1 may be a server device or a cloud server device that provides functions as a cloud service. For simplicity, the information processing device 1 will be referred to as a computer 1 below.

[0013] A computer 1 according to this embodiment acquires image data of a cow and three-dimensional coordinate data of the cow. The computer 1 identifies a first feature point and a second feature point of the cow in the acquired image data. The computer 1 identifies a plurality of three-dimensional coordinate data corresponding to a point cloud between the identified first feature point and second feature point. The computer 1 derives the circumference of the cow based on the identified plurality of three-dimensional coordinate data. The computer 1 derives the weight of the cow based on the derived circumference.

[0014] 1 is a block diagram showing an example of the configuration of a computer 1. The computer 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, a large-capacity storage unit 17, an imaging unit 18, and a 3D sensor 19. Each component is connected by a bus B.

[0015] The control unit 11 includes an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or a quantum processor, and performs various information processing, control processing, etc. related to the computer 1 by reading and executing a control program 1P (program product) stored in the memory unit 12.

[0016] The control program 1P can be deployed to run on a single computer, or on multiple computers located at one site, or distributed across multiple sites and interconnected by a communications network. Although the control unit 11 is illustrated in FIG. 1 as a single processor, it may also be a multiprocessor.

[0017] The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data required for the control unit 11 to execute processing. The storage unit 12 also temporarily stores data required for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing processing related to communication.

[0018] The input unit 14 may be a keyboard, a mouse, or a touch panel integrated with the display unit 15. The display unit 15 is a liquid crystal display, an organic EL (electroluminescence) display, or the like, and displays various information according to instructions from the control unit 11.

[0019] The reading unit 16 reads a portable storage medium 1a including a CD (Compact Disc)-ROM or a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the mass storage unit 17. Alternatively, the control unit 11 may download the control program 1P from another computer via a network N or the like and store it in the mass storage unit 17. Furthermore, the control unit 11 may read the control program 1P from the semiconductor memory 1b.

[0020] The mass storage unit 17 includes a recording medium such as a hard disk drive (HDD), a solid state drive (SSD), etc. The mass storage unit 17 includes a measurement result database (DB) 171, a training data DB 172, a circumference estimation model 173, and a weight estimation model 174.

[0021] Measurement result DB 171 stores measurement results such as the circumference and weight of cattle. Training data DB 172 stores training data for constructing (creating) a circumference estimation model 173 and a weight estimation model 174. The circumference estimation model 173 is an estimator that estimates the circumference of cattle, and is a trained model generated by machine learning. The weight estimation model 174 is an estimator that estimates the weight of cattle, and is a trained model generated by machine learning.

[0022] In this embodiment, the storage unit 12 and the large-capacity storage unit 17 may be configured as an integrated storage device. Furthermore, the large-capacity storage unit 17 may be configured with a plurality of storage devices. Furthermore, the large-capacity storage unit 17 may be an external storage device connected to the computer 1.

[0023] The imaging unit 18 is an imaging device such as a CCD (Charge Coupled Device) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, etc. The imaging unit 18 may not be built into the computer 1, but may be externally connected directly to the computer 1 and configured to be capable of capturing images.

[0024] The 3D sensor 19 is a sensor that can align the image of a target animal such as a cow captured by the imaging unit 18 with three-dimensional coordinate data. That is, the three-dimensional coordinate data is associated with each pixel of the image data.

[0025] The 3D sensor 19 may be, for example, a LiDAR sensor. The LiDAR sensor emits laser light and detects reflected or scattered light, and can detect three-dimensional point cloud data (X, Y, Z). In addition to the LiDAR sensor, a depth sensor or an infrared sensor may also be used. A sensor that integrates the imaging unit 18 and the 3D sensor 19 may also be used.

[0026] The computer 1 is not limited to being integrated with the 3D sensor 19, and the computer 1 and the 3D sensor 19 may be provided as separate devices.

[0027] The computer 1 may perform various information processing and control processing, etc., as a single computer, or may perform the processing in a distributed manner across multiple computers. Furthermore, the computer 1 may be realized by multiple virtual machines provided within a single computer.

[0028] The system configuration is not limited to the above. For example, an information processing terminal (such as a tablet) captures an image of a cow. The information processing terminal acquires three-dimensional coordinate data of the cow using a LiDAR sensor mounted on the information processing terminal. The information processing terminal transmits the acquired image data and three-dimensional coordinate data of the cow to a server device. The server device derives the weight of the cow based on the image data and three-dimensional coordinate data of the cow transmitted from the information processing terminal. The server device transmits the derived weight of the cow to the information processing terminal. The information processing terminal may receive the weight of the cow transmitted from the server device and display the received weight of the cow. Note that a cloud server may be used instead of the server device. In other words, the information processing terminal captures an image of the cow, transmits the image data, and receives and displays the weight of the cow. The server device performs processing (arithmetic processing) to derive the circumference, weight, etc. of the cow.

[0029] FIG. 2 is an explanatory diagram showing an example of the record layout of the measurement result DB 171 and the training data DB 172. The measurement result DB 171 includes an image ID string, an image data string, a three-dimensional coordinate data string, a measurement part string, a feature point name string, a feature point coordinate string, a circumference string, and a weight string. The image ID string stores a unique image data ID to identify the image data of each cow.

[0030] The image data column stores image data of the cow. The three-dimensional coordinate data column stores three-dimensional coordinate data corresponding to each pixel of the cow's image data (two-dimensional image data). The measurement part column stores the names of the measurement parts of the cow. The feature point name column stores the names of feature points in the cow's image data. The feature point coordinate column stores the coordinate values ​​of feature points in the cow's image data. The circumference column stores the circumference of the cow. The weight column stores the weight of the cow.

[0031] The training data DB 172 includes a training ID column, a type column, an input column, and an output column. The training ID column stores a unique ID of each training data to identify the training data. The type column stores the type of learning model, including the circumference estimation model 173 and the weight estimation model 174.

[0032] When the type of learning model is "perimeter estimation model 173," the input string stores the total length of the line segments of the cow, and the output string stores the circumference of the cow. The total length of the line segments will be described later. When the type of learning model is "weight estimation model 174," the input string stores the circumference of the cow, and the output string stores the weight of the cow.

[0033] The storage format of each DB described above is an example, and other storage formats may be used as long as the relationships between the data are maintained.

[0034] Next, the process of deriving the weight of a cow will be explained. First, the computer 1 acquires image data of an image of the cow and three-dimensional coordinate data of the cow. The image data of the cow is image data of the cow captured from the side. For example, the computer 1 acquires image data of the cow captured from the side via the imaging unit 18. Note that the computer 1 may also acquire image data of the cow captured by an external information processing terminal with an imaging function.

[0035] The computer 1 acquires three-dimensional coordinate data of the cow. Specifically, the computer 1 acquires the three-dimensional coordinate data of the cow via a 3D sensor 19, which is a LiDAR sensor for detecting three-dimensional point cloud data. The three-dimensional coordinate data is data (three-dimensional point cloud data) that has three-dimensional coordinate values ​​for each pixel of the image.

[0036] Next, the computer 1 derives the weight of the cow. Fig. 3 is an explanatory diagram illustrating the process of deriving the weight of the cow. Fig. 3A is an explanatory diagram illustrating the process of identifying the first feature point and the second feature point. Fig. 3B is an explanatory diagram illustrating the process of deriving the total length. Fig. 3C is an explanatory diagram illustrating the process of deriving the circumference and weight of the cow.

[0037] The computer 1 identifies a first feature point 11a and a second feature point 11b of the cow in the acquired image data. As shown in the figure, the first feature point 11a is the upper coordinate of chest depth, and the second feature point 11b is the lower coordinate of chest depth. Chest depth is the linear distance from the upper edge of the thoracic vertebrae to the lower edge of the chest on the line along which chest circumference is measured.

[0038] The positions of the first feature point 11a and the second feature point 11b are not limited to the positions illustrated in Fig. 3. For example, the positions of the first feature point 11a and the second feature point 11b may be the apex of the withers for measuring withers height and a ground measurement point that extends vertically from the withers apex to the ground. Alternatively, the positions of the first feature point 11a and the second feature point 11b may be the cross for measuring cross height and a ground measurement point that extends vertically from the cross to the ground.

[0039] Furthermore, the positions of the first characteristic point 11a and the second characteristic point 11b may be measurement points at both ends of the waist part passing through the base of the eighth rib behind the shoulders to measure chest width (rib width at the chest circumference measurement part). Furthermore, the positions of the first characteristic point 11a and the second characteristic point 11b may be measurement points at the outer edge of the left waist corner and measurement points at the outer edge of the right waist corner to measure waist angle width (straight-line distance between the outer edges of the left and right waist corners).

[0040] Regarding the process of identifying the first feature points 11a and the second feature points 11b, for example, the computer 1 may accept designation (input) of the first feature points 11a and the second feature points 11b by the user for the image data. Note that the first feature points 11a and the second feature points 11b are not limited to being manually identified by the user, but can also be automatically identified by using the template data in the second embodiment, for example.

[0041] The computer 1 acquires (specifies) from the measurement result DB 171 three-dimensional coordinate data corresponding to the received first feature point 11a and second feature point 11b.

[0042] In the cow's contour region, the brightness or phase is unstable due to the influence of noise, etc., so if the 3D coordinate data of the contour region acquired by the 3D sensor 19 is used, the noise may cause the derivation process of the total length, which will be described later, to be inaccurate. By using the first offset point 11c and the second offset point 11d, which are offset toward the inside of the cow's body, it is possible to prevent an inaccurate derivation of the total length. The first offset point 11c is a point offset downward from the first feature point 11a, and the second offset point 11d is a point offset upward from the second feature point 11b.

[0043] The first offset point 11c and the second offset point 11d are obtained by equally dividing the line segment 12a at a predetermined interval. Specifically, the computer 1 identifies a line segment 12a connecting the identified first feature point 11a and second feature point 11b. The computer 1 divides the identified line segment 12a at a predetermined interval. As an example, the predetermined interval may be an interval obtained by dividing the line segment 12a into five equal parts. As shown in the figure, the line segment 12a is divided into five equal parts, namely, intervals 13a, 13b, 13c, 13d, and 13e.

[0044] The computer 1 identifies a first offset point 11c belonging to interval 13b, which is adjacent to the lower side of interval 13a to which first feature point 11a belongs. The computer 1 identifies a second offset point 11d belonging to interval 13d, which is adjacent to the upper side of interval 13e to which second feature point 11b belongs.

[0045] The computer 1 identifies a plurality of three-dimensional coordinate data corresponding to a plurality of points between the identified first offset point 11c and second offset point 11d. Specifically, the computer 1 acquires (identifies) three-dimensional coordinate data corresponding to each point between the first offset point 11c and the second offset point 11d from the measurement result DB 171. As shown in FIG. 3B, three-dimensional coordinate data corresponding to each pixel (e.g., point 11e and point 11f) is identified between the first offset point 11c and the second offset point 11d. The computer 1 derives the total length of the line segments between each identified three-dimensional coordinate data.

[0046] The computer 1 derives the circumference of the cow from the total length of the derived line segment lengths. Specifically, the computer 1 inputs the derived total length into a circumference estimation model 173, and derives (outputs) the circumference of the cow. The circumference estimation model 173 is used as a program module that is part of artificial intelligence (AI) software.

[0047] The circumference estimation model 173 may be realized using artificial intelligence techniques such as machine learning or deep learning. The circumference estimation model 173 is an estimator that has been constructed (generated) based on the correlation between the total length of the line segments of the measurement target (measurement site) area (e.g., chest depth) of a cow and the circumference of the cow. For example, the circumference estimation model 173 uses training data accumulated in the training data DB 172 to learn the correlation between the total length of the line segments of the measurement target area of ​​the cow and the circumference of the cow. Based on the learned learning results, the circumference estimation model 173 estimates the circumference of the cow from the total length of the line segments of the measurement target area of ​​the cow that has been input.

[0048] The training data is data generated by associating the total length of line segments in the measurement target area of ​​the cow (for example, chest depth) with the circumference of the cow obtained from past measurements. Note that the circumference estimation model 173 may be generated by the computer 1, an external information processing device, a cloud server, or the like.

[0049] When the computer 1 acquires the total length of the line segments in the measurement target area of ​​the cow, it inputs the acquired total length into the circumference estimation model 173 and outputs the estimation result of the circumference of the cow. For example, the estimated circumference of the cow is 129 cm.

[0050] The perimeter estimation model 173 may be a trained model constructed using any learning algorithm, such as linear regression analysis, MARS (Multivariate Adaptive Regression Splines), support vector regression (SVR), regression tree, model tree, genetic program, binary classification, logistic regression, k-nearest neighbor method, support vector machine, decision tree, or random forest.

[0051] In addition, the circumference of a cow can be calculated using an estimation formula that shows the relationship between the total length and circumference. For example, the circumference estimation formula is expressed as the following formula (1). Estimated circumference (cm) = k1 × total length (cm) ... (1) k1 is a coefficient (e.g., a positive number) for calculating the circumference, and is calculated, for example, by multivariate curve fitting. Note that k1 may be set by a system administrator or the like. For example, k1 may be 2.5. The computer 1 derives (calculates) the circumference of the cow using the above-mentioned formula (1) based on the total length of the line segments of the cow's chest depth.

[0052] The computer 1 derives the weight of the cow from the derived circumference of the cow. Specifically, the computer 1 inputs the derived circumference of the cow into a weight estimation model 174, and derives (outputs) the weight of the cow. The weight estimation model 174 is used as a program module that is part of artificial intelligence software.

[0053] The weight estimation model 174 may be realized using artificial intelligence techniques such as machine learning or deep learning. The weight estimation model 174 is an estimator that has been constructed (generated) based on the correlation between the circumference of a measurement target area of ​​a cow (e.g., chest depth) and the weight of the cow. For example, the weight estimation model 174 uses training data accumulated in the training data DB 172 to learn the correlation between the circumference of a measurement target area of ​​a cow and the weight of the cow. The weight estimation model 174 estimates the weight of the cow from the circumference of the measurement target area based on the learned learning results.

[0054] The training data is data generated by associating the circumference of a measurement target area (for example, chest depth) of a cow with the weight of the cow obtained from past measurements. Note that the weight estimation model 174 may be generated by the computer 1, an external information processing device, a cloud server, or the like.

[0055] When the computer 1 acquires the perimeter of the measurement target area of ​​the cow, it inputs the acquired perimeter into the weight estimation model 174 and outputs the estimation result of the weight of the cow. For example, the estimated weight of the cow is 382 kg.

[0056] The weight estimation model 174 may be a trained model constructed using any learning algorithm, such as linear regression analysis, MARS, support vector regression, regression tree, model tree, genetic program, binary classification, logistic regression, k-nearest neighbor method, support vector machine, decision tree, or random forest.

[0057] In addition, the weight of a cow can be calculated using an estimation formula that shows the relationship between the circumference and weight of the cow. For example, the weight estimation formula is expressed as the following formula (2). Estimated weight (kg) = k2-k3 x circumference (cm) + k4 x [perimeter (cm)] 2 …(2) k2, k3, and k4 are coefficients (e.g., positive numbers) for calculating weight, and are calculated, for example, by multivariate curve fitting. Note that k2, k3, and k4 may be set by a system administrator, etc. For example, k2 may be 55, k3 may be 2.2, and k4 may be 0.02. Computer 1 derives (calculates) the weight of the cow based on the circumference of the cow using the above-mentioned formula (2).

[0058] 4 is a flowchart showing the processing steps for deriving the weight of a cow. The control unit 11 of the computer 1 acquires image data of the cow taken from the side using the imaging unit 18 (step S101). The control unit 11 acquires three-dimensional coordinate data corresponding to each pixel of the acquired image data via the 3D sensor 19 (step S102).

[0059] The control unit 11 stores the acquired image data of the cow and the three-dimensional coordinate data corresponding to each pixel of the image data as one record in the measurement result DB 171 of the mass storage unit 17 in association with the image ID (step S103).

[0060] The control unit 11 identifies the first feature point 11a and the second feature point 11b of the cow in the acquired image data (step S104). For example, the control unit 11 may accept, via the input unit 14, a user's designation (input) of the first feature point 11a and the second feature point 11b for the image data. The control unit 11 identifies three-dimensional coordinate data corresponding to the identified first feature point 11a and second feature point 11b from the measurement result DB 171 of the mass storage unit 17 (step S105).

[0061] The control unit 11 identifies a first offset point 11c offset from the identified first feature point 11a and a second offset point 11d offset from the identified second feature point 11b (step S106). Specifically, the control unit 11 identifies a line segment 12a connecting the identified first feature point 11a and second feature point 11b. The control unit 11 equally divides the identified line segment 12a into predetermined intervals (for example, intervals obtained by dividing the line segment 12a into five equal parts). The control unit 11 identifies a first offset point 11c that belongs to an interval adjacent to the lower side of the interval to which the first feature point 11a belongs. The control unit 11 identifies a second offset point 11d that belongs to an interval adjacent to the upper side of the interval to which the second feature point 11b belongs.

[0062] The control unit 11 identifies a plurality of three-dimensional coordinate data corresponding to a plurality of points between the identified first offset point 11c and second offset point 11d (step S107). Specifically, the control unit 11 identifies (acquires) three-dimensional coordinate data corresponding to each point between the identified first offset point 11c and second offset point 11d from the measurement result DB 171 of the mass storage unit 17. The control unit 11 derives the total length of the line segments between the three-dimensional coordinate data of each identified point (step S108).

[0063] The control unit 11 derives the circumference of the cow from the derived total length (step S109). Specifically, the control unit 11 inputs the derived total length into the circumference estimation model 173, and outputs (derives) the circumference of the cow. The control unit 11 derives the weight of the cow from the derived circumference of the cow (step S110). Specifically, the control unit 11 inputs the derived circumference of the cow into the weight estimation model 174, and outputs (derives) the weight of the cow.

[0064] The control unit 11 stores the measurement results in the measurement result DB 171 of the mass storage unit 17 (step S111), and ends the process. Specifically, the control unit 11 stores in the measurement result DB 171 the measurement site (for example, chest depth), the names (for example, upper chest deep and lower chest deep) and coordinates of the feature points (first feature point 11a and second feature point 11b), as well as the derived circumference and weight of the cow, in association with the image ID.

[0065] According to this embodiment, it is possible to derive the perimeter of a target animal based on image data of the target animal and three-dimensional coordinate data of the target animal.

[0066] According to this embodiment, it is possible to derive the weight of a target animal based on the circumference of the target animal.

[0067] According to this embodiment, it is possible to derive the perimeter of the target animal based on equation (1) showing the relationship between the total length and perimeter of the target animal, or based on the perimeter estimation model 173.

[0068] According to this embodiment, it is possible to derive the weight of the target animal based on the equation (2) that shows the relationship between the circumference and weight of the target animal, or based on the weight estimation model 174.

[0069] (Embodiment 2) The second embodiment relates to an embodiment in which the first feature point 11a and the second feature point 11b are automatically identified by using image data of a target animal and template data of a measurement target area of ​​the target animal. Note that a description of the same content as in the first embodiment will be omitted. Note that the following description will be given assuming that the target animal is a cow, but the same can be applied to other types of target animals.

[0070] Figure 5 is a block diagram showing an example of the configuration of a computer 1 in embodiment 2. Note that the same reference numerals are used to designate the same parts as in Figure 1, and descriptions thereof will be omitted. The large-capacity storage unit 17 includes a template data DB 175. The template data DB 175 stores template data of the measurement target area of ​​the cow.

[0071] 6 is an explanatory diagram showing an example of a record layout of template data DB 175. Template data DB 175 includes a template ID (Identifier) ​​string, an image data string, a measurement region string, a reference feature point name string, and a reference feature point coordinate string.

[0072] The template ID column stores a unique ID for template data to identify the template data for each cow's measurement area. The image data column stores reference image data for the cow's measurement area.

[0073] The measurement part column stores the names of measurement parts of the cow. Measurement parts include chest depth or body length, etc. Note that measurement parts may also include withers height, horizontal body length, or hip height, etc. The reference feature point name column stores the names of reference feature points of the cow in the template data. The reference feature point coordinate column stores the coordinate values ​​of reference feature points of the cow in the template data. Note that template data will be described later.

[0074] Fig. 7 is an explanatory diagram showing an example of a plurality of reference feature points of a cow. In order to measure the body dimensions of the cow, a plurality of reference feature points are set in advance in the measurement target area of ​​the cow. As shown in the figure, there is a reference feature point 14a indicating the upper deep chest area and a reference feature point 14b indicating the lower deep chest area. The cow's chest circumference can be measured based on the reference feature point 14a (upper deep chest area) and the reference feature point 14b (lower deep chest area).

[0075] It should be noted that the reference feature points are not limited to those shown in Fig. 7. For example, a reference feature point indicating the shoulder end, a reference feature point indicating the ischial end, or a reference feature point indicating the cross part (the intersection of the line connecting both hip angles of the cow and the spine) may be set. The body length of the cow can be measured based on the reference feature point indicating the shoulder end and the reference feature point indicating the ischial end. The cross part height of the cow can be measured based on the reference feature point indicating the cross part.

[0076] FIG. 8 is an explanatory diagram illustrating the process of identifying the first feature point 11a and the second feature point 11b. First, the computer 1 acquires image data of the cow to be measured. From the acquired image data of the cow, the computer 1 extracts segmentation data (hereinafter referred to as a mask image) of the measurement target area of ​​the cow. The measurement target area is, for example, the cow's body excluding the legs, tail, and head. For example, the computer 1 may extract a mask image of the measurement target area of ​​the cow using an image segmentation model constructed by an instance segmentation network.

[0077] The image segmentation model is trained using image data of a cow captured from the side. The image segmentation model is set to output an image in which the cow's torso is white and the other areas are black, so training is carried out so that the output image approaches the correct image. Computer 1 inputs the acquired cow image data into the image segmentation model and outputs a mask image of the area to be measured. In the output mask image, the area corresponding to the cow's torso (foreground pixel area) is white (pixel value 1) and the other areas (background pixel area) are black (pixel value 0).

[0078] The image segmentation model may be a SegNet model or a U-NET (Convolutional Networks for Biomedical Image Segmentation), which is a semantic segmentation model. U-NET is a type of FCN (Fully Convolutional Networks) that includes an encoder that performs downsampling and a decoder that performs upsampling. U-NET is a neural network that does not have a fully connected layer and is composed only of convolutional layers and pooling layers.

[0079] In addition to the image segmentation model described above, a background subtraction method or the like can be used. By applying the background subtraction method, a mask image is generated as a result of the background subtraction.

[0080] Next, if there is a white portion in the background pixel portion of the extracted mask image, the computer 1 corrects the mask image based on the three-dimensional coordinate data. As shown in the figure, there is a white area 20a in the background pixel portion, so it is necessary to correct the mask image using the three-dimensional coordinate data.

[0081] Specifically, the computer 1 acquires three-dimensional coordinate data using a 3D sensor 19 that measures depth (distance) information to the cow. The computer 1 corrects the extracted mask image based on the acquired three-dimensional coordinate data. As shown in the figure, the three-dimensional coordinate data is data having three-dimensional coordinate values ​​including an X axis (first coordinate axis), a Y axis (second coordinate axis), and a Z axis.

[0082] The X-axis indicates the horizontal direction (left and right) of the cow, with the right direction on the paper as the positive direction. The Y-axis indicates the vertical direction (up and down) of the cow, with the downward direction on the paper as the positive direction. The Z-axis indicates the viewing direction (perpendicular to the paper in Figure 8) of the pixels that are the cow's body (white pixels).

[0083] For example, the computer 1 calculates a depth difference value that indicates the difference in depth between the depth (Z-axis value) in the viewing direction of a pixel (white pixel) that is the cow's body and the depth in the viewing direction of a pixel that is the target (white region 20a). If the calculated depth difference value is equal to or greater than a predetermined threshold, the computer 1 sets the pixel value of the target pixel to 0, thereby making the pixel black. If the depth difference value is less than the predetermined threshold, the pixel value of the target pixel is left unchanged.

[0084] The depth difference threshold may be, for example, 50 centimeters (cm). The depth difference threshold may be set appropriately depending on the distance between the imaging device and the cow when capturing an image. Alternatively, a learning model may be used that is trained to output a depth difference threshold when image data and three-dimensional coordinate data of the cow are input. If the depth difference value is outside a predetermined threshold range (for example, 30 cm to 50 cm), the pixel may be determined to be a pixel other than the cow's torso, and the pixel value of that pixel may be set to 0.

[0085] The computer 1 identifies a plurality of feature points (first feature points 11a, second feature points 11b, etc.) in the mask image. Fig. 9 is an explanatory diagram for explaining the process of identifying feature points.

[0086] The computer 1 adjusts the size of the corrected mask image. Specifically, the computer 1 acquires template data (hereinafter referred to as a template mask image) of the measurement target area (for example, the body of a cow) from the template data DB 175.

[0087] The template mask image is image data of a measurement target area that includes predetermined names and coordinate values ​​of a plurality of reference feature points. The template mask image may be created manually by a user, or may be generated using a template mask image output model that has been trained to output a plurality of reference feature points in the measurement target area of ​​a cow when image data of the cow is input.

[0088] From the acquired template mask image of the measurement target area, the computer 1 acquires a rectangle 15a based on the maximum connected component of the measurement target area. The maximum connected component refers to the largest area among areas with pixel-to-pixel continuity. As shown in the figure, the maximum connected component is the white area of ​​the cow's body.

[0089] Specifically, the computer 1 acquires a rectangle 15a based on the maximum and minimum values ​​of the X axis and the Y axis in the largest connected component of the measurement target area. The computer 1 calculates the size of the acquired rectangle 15a based on the coordinate values ​​(x1, y1) of the upper left pixel and the coordinate values ​​(x2, y2) of the lower right pixel.

[0090] Based on the pixel values ​​of each pixel in the mask image to be measured, the computer 1 identifies a measurement target region where the pixel value of the pixel is 1. The computer 1 acquires a rectangle 15b obtained based on the maximum connected component of the identified measurement target region. The computer 1 adjusts (changes) the size of the mask image based on the calculated size of the rectangle 15a. Specifically, the computer 1 adjusts the size of the mask image so that the width and height of the acquired rectangle 15b are the same as the width and height of the rectangle 15a.

[0091] The computer 1 identifies a plurality of feature points in the resized mask image. Specifically, the computer 1 determines the X-axis value of each feature point in the resized mask image based on the X-axis value of each reference feature point in the template mask image. As shown in the figure, the X-axis value (x) of reference feature point P in the template mask image is set to the X-axis value (x) of feature point Q in the mask image.

[0092] The computer 1 calculates a division ratio based on the Y-axis value of each reference feature point in the template mask image and the maximum and minimum Y-axis values ​​of the largest connected component in the measurement target area. The computer 1 determines the Y-axis value of each feature point in the resized mask image based on the calculated division ratio.

[0093] As shown in the figure, the computer 1 calculates distances d1 and d2 in the largest connected component of the measurement target region of the template mask image. Distance d1 is the distance between the Y-axis value (y) of the reference feature point P and the minimum Y-axis value (ymin) of the largest connected component of the measurement target region corresponding to the reference feature point P. Distance d2 is the distance between the Y-axis value (y) of the reference feature point P and the maximum Y-axis value (ymax) of the largest connected component of the measurement target region corresponding to the reference feature point P.

[0094] The computer 1 calculates the division ratio at the reference feature point P based on the vertical width (ymax-ymin) of the maximum connected component in the measurement target region corresponding to the reference feature point P and the calculated distances d1 and d2. For example, the calculated division ratio is 2:3.

[0095] For the resized mask image, the computer 1 obtains the maximum value (y'max) and minimum value (y'min) of the Y axis of the largest connected component in the measurement target region of the mask image, corresponding to the X axis value (x) of the determined mask image feature point Q. The computer 1 determines the Y axis value (y') of feature point Q based on the calculated division ratio and the maximum value (y'max) and minimum value (y'min) of the Y axis of the largest connected component in the measurement target region of the obtained mask image. As shown in the figure, the coordinate values ​​of the determined feature point Q are (x, y').

[0096] Next, returning to FIG. 8, the computer 1 stores the identified multiple feature points (first feature point 11a and second feature point 11b) in the measurement result DB 171. Specifically, the computer 1 acquires the coordinate values ​​of each of the identified multiple feature points. The computer 1 assigns an image ID to the image data of the cow. The computer 1 stores the image data of the cow, the name of the measurement site, and the name and coordinate values ​​of each feature point in association with the assigned image ID in the measurement result DB 171. The computer 1 displays the identified multiple feature points (first feature point 11a and second feature point 11b) superimposed on the image data of the cow.

[0097] 10 is a flowchart showing the processing steps for identifying the first feature point 11a and the second feature point 11b. The control unit 11 of the computer 1 acquires image data of a cow imaged from the side using the imaging unit 18 (step S121). The control unit 11 extracts a mask image of the measurement target area of ​​the cow using an image segmentation model constructed by, for example, U-NET or the like (step S122).

[0098] The control unit 11 executes a subroutine for processing to correct the mask image (step S123). The control unit 11 executes a subroutine for processing to adjust the size of the mask image (step S124). The control unit 11 executes a subroutine for processing to identify feature points (first feature points 11a and second feature points 11b) (step S125). The subroutines for processing to correct the mask image, adjust the size of the mask image, and identify the feature points will be described later.

[0099] The control unit 11 stores the measurement results in the measurement result DB 171 of the mass storage unit 17 (step S126). Specifically, the control unit 11 assigns an image ID to the image data of the cow. The control unit 11 stores the image data of the cow, the name of the measurement site, and the name and coordinate values ​​of each feature point in association with the assigned image ID in the measurement result DB 171. The control unit 11 displays the identified first feature point 11a and second feature point 11b superimposed on the image data of the cow via the display unit 15 (step S127), and ends the processing.

[0100] 11 is a flowchart showing the processing steps of a subroutine for processing to correct a mask image. The control unit 11 of the computer 1 acquires three-dimensional coordinate data using the 3D sensor 19 that measures depth information to the cow (step S01). The control unit 11 acquires three-dimensional coordinate data of pixels (white pixels) that represent the cow from the acquired three-dimensional coordinate data (step S02). The pixel in question may be, for example, any pixel in the area that represents the cow's body.

[0101] The control unit 11 acquires one pixel from the mask image to be measured (step S03). The control unit 11 determines whether the acquired pixel is black (step S04). For example, if the pixel value of a pixel is 0, the control unit 11 determines that the pixel is black. Alternatively, if the pixel value of a pixel is 1, the control unit 11 determines that the pixel is white.

[0102] If the control unit 11 determines that the acquired pixel is black (YES in step S04), the process returns to step S03. If the control unit 11 determines that the acquired pixel is not black (NO in step S04), the control unit 11 determines whether or not three-dimensional coordinate data for the pixel exists from the three-dimensional coordinate data acquired in the process of step S01 (step S05). If the control unit 11 determines that three-dimensional coordinate data for the pixel does not exist (NO in step S05), the process proceeds to step S08, which will be described later.

[0103] If the control unit 11 determines that three-dimensional coordinate data for the pixel exists (YES in step S05), it calculates a depth difference value between the depth (Z-axis value) in the viewpoint direction of the pixel (white pixel) that represents the cow (e.g., the cow's body) acquired in the process of step S02 and the depth in the viewpoint direction of the pixel (step S06).The control unit 11 determines whether the calculated depth difference value is equal to or greater than a predetermined threshold value (e.g., 50 cm) (step S07).

[0104] If the calculated depth difference value is less than a predetermined threshold value (NO in step S07), the control unit 11 proceeds to the processing of step S09, which will be described later. If the calculated depth difference value is equal to or greater than a predetermined threshold value (YES in step S07), the control unit 11 sets the pixel value of the pixel to 0, thereby setting the pixel to black (step S08). The control unit 11 determines whether the pixel is the last pixel in the mask image (step S09).

[0105] If the control unit 11 determines that the pixel is not the last pixel (NO in step S09), the process returns to step S03. If the control unit 11 determines that the pixel is the last pixel (YES in step S09), the control unit 11 ends the subroutine for the correction process of the mask image and returns.

[0106] 12 is a flowchart showing the procedure of a subroutine for adjusting the size of a mask image. The control unit 11 of the computer 1 acquires a template mask image of a measurement target area (e.g., a cow's body) from the template data DB 175 in the mass storage unit 17 (step S11).

[0107] 9, the control unit 11 acquires a first rectangle (rectangle 15a) based on the maximum connected component of the acquired measurement target region from the template mask image of the measurement target region (step S12).The control unit 11 calculates the size of the acquired first rectangle (rectangle 15a) based on the pixel coordinate values ​​(x1, y1) of the upper left and pixel coordinate values ​​(x2, y2) of the lower right of the acquired first rectangle (rectangle 15a) (step S13).

[0108] Based on the pixel values ​​of each pixel of the mask image to be measured, the control unit 11 identifies a measurement target region in which the pixel value is 1 (step S14). The control unit 11 acquires a second rectangle (rectangle 15b) obtained based on the maximum connected component of the identified measurement target region (step S15).

[0109] Based on the calculated size of the first rectangle (rectangle 15a), control unit 11 adjusts the size of the measurement target region in the mask image (step S16). Specifically, control unit 11 adjusts the size of the mask image so that the width and height of the second rectangle (rectangle 15b) obtained based on the maximum connected component of the measurement target region in the mask image become the same as the width and height of the first rectangle (rectangle 15a) obtained based on the maximum connected component of the measurement target region in the template mask image. Control unit 11 ends the subroutine for adjusting the mask image size and returns.

[0110] 13 is a flowchart showing the processing steps of a subroutine for processing to identify feature points. Based on the template ID, the control unit 11 of the computer 1 acquires a plurality of reference feature points in the corresponding template mask image from the template data DB 175 in the mass storage unit 17 (step S21). The control unit 11 then acquires the maximum connected component of the measurement target region (e.g., the body of a cow) in the template mask image (step S22).

[0111] The control unit 11 acquires the coordinate value of one of the acquired reference feature points (step S23). The control unit 11 determines the value of a first coordinate axis (e.g., X-axis) of the feature point to be identified (step S24). Specifically, the control unit 11 sets the value of the first coordinate axis of the acquired reference feature point to the value of the first coordinate axis of the feature point.

[0112] The control unit 11 acquires the maximum and minimum values ​​of the second coordinate axis (e.g., Y axis) of the largest connected component of the measurement target region in association with the value of the first coordinate axis (e.g., X axis) of the reference feature point for the measurement target region of the template mask image (step S25).The control unit 11 calculates the division ratio at the reference feature point based on the value of the second coordinate axis of the reference feature point and the maximum and minimum values ​​of the second coordinate axis of the acquired largest connected component (step S26).

[0113] The control unit 11 determines the value of the second coordinate axis of the feature point based on the calculated division ratio (step S27). Specifically, the control unit 11 acquires the maximum and minimum values ​​of the second coordinate axis of the largest connected component in the measurement target region of the mask image in association with the value of the first coordinate axis of the feature point determined in the processing of step S24. The control unit 11 determines the value of the second coordinate axis of the feature point based on the calculated division ratio and the maximum and minimum values ​​of the second coordinate axis of the largest connected component in the measurement target region of the acquired mask image.

[0114] The control unit 11 determines whether the reference feature point in question is the last reference feature point from among the multiple reference feature points in the template mask image (step S28). If the control unit 11 determines that the reference feature point in question is not the last reference feature point (NO in step S28), the process returns to step S23. If the control unit 11 determines that the reference feature point in question is the last reference feature point (YES in step S28), the control unit 11 outputs the identified multiple feature points (step S29). The control unit 11 ends the subroutine for feature point identification processing and returns.

[0115] According to this embodiment, it is possible to identify (automatically extract) the first feature points 11a and the second feature points 11b based on image data of the target animal and template data of the measurement target area of ​​the target animal.

[0116] According to this embodiment, the first feature point 11a and the second feature point 11b are automatically extracted, thereby improving the efficiency of deriving the weight of the target animal.

[0117] The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0118] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0119] 1. Information processing equipment (computer) 11 Control section 12 Storage section 13 Communications Department 14 Input section 15 Display 16 Reading unit 17 Mass storage 171 Measurement result DB 172 Training Data DB 173 Circumference Estimation Model 174 Weight Estimation Model 175 Template Data DB 18 Imaging unit 19 3D Sensor 1a Portable storage media 1b semiconductor memory 1P control program

Claims

1. Acquiring image data of the target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; identifying a plurality of three-dimensional coordinate data corresponding to a plurality of points equally divided at predetermined intervals on a line segment connecting a first offset point offset from the first feature point and a second offset point offset from the second feature point; Deriving the perimeter of the target animal based on the identified plurality of three-dimensional coordinate data; Deriving the weight of the target animal based on the derived circumference. A program that causes a computer to perform a process.

2. The total length of the line segments between the identified three-dimensional coordinate data is calculated. The program according to claim 1.

3. The circumference of the target animal is derived from the derived total length based on an estimation formula indicating the relationship between the total length and circumference of the target animal, or a first learning model that has been trained to output the circumference of the target animal when the total length of the target animal is input. The program according to claim 2.

4. The weight of the target animal is derived from the derived circumference based on an estimation formula showing the relationship between the circumference and weight of the target animal, or a second learning model that has been trained to output the weight of the target animal when the circumference of the target animal is input. The program according to claim 1 or 2.

5. Extracting segmentation data of a measurement target region of the target animal from the image data; acquiring template data of the measurement target area and a plurality of reference feature points of the target animal in the template data; Identifying a first feature point and a second feature point of the target animal in the extracted segmentation data based on the acquired template data and a plurality of reference feature points. The program according to claim 1 or 2.

6. Acquiring image data of the target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; identifying a plurality of three-dimensional coordinate data corresponding to a plurality of points equally divided at predetermined intervals on a line segment connecting a first offset point offset from the first feature point and a second offset point offset from the second feature point; Deriving the perimeter of the target animal based on the identified plurality of three-dimensional coordinate data; Deriving the weight of the target animal based on the derived circumference. Information processing methods.

7. An information processing device including a control unit, The control unit Acquiring image data of the target animal and three-dimensional coordinate data of the target animal; Identifying a first feature point and a second feature point of the target animal in the image data; identifying a plurality of three-dimensional coordinate data corresponding to a plurality of points equally divided at predetermined intervals on a line segment connecting a first offset point offset from the first feature point and a second offset point offset from the second feature point; Deriving the perimeter of the target animal based on the identified plurality of three-dimensional coordinate data; Deriving the weight of the target animal based on the derived circumference. Information processing device.

8. Acquiring image data of the target animal and three-dimensional coordinate data of the target animal; Extracting segmentation data of a measurement target region of the target animal from the image data; acquiring template data of the measurement target area and a plurality of reference feature points of the target animal in the template data; Identifying a first feature point and a second feature point of the target animal in the extracted segmentation data based on the acquired template data and a plurality of reference feature points; Identifying a plurality of three-dimensional coordinate data corresponding to a point group between the first feature point and the second feature point; Deriving the perimeter of the target animal based on the identified plurality of three-dimensional coordinate data; Deriving the weight of the target animal based on the derived circumference. A program that causes a computer to perform a process.

Images