Parturition detection device, parturition detection method, and parturition detection program

Abstract

To improve accuracy of detecting a sign of delivery of livestock.SOLUTION: A delivery detection device according to one aspect of the present invention acquires a plurality of captured images generated by continuously capturing images of livestock with an image capturing device, calculates a center-of-gravity position of the livestock from each of the acquired captured images, calculates a difference between the calculated center-of-gravity positions as a movement amount between the respective captured images, calculates a difference pixel number between the respective captured images, determines whether or not the calculated difference pixel number exceeds a first threshold, further determines whether or not the calculated movement amount of the center-of-gravity positions exceeds a second threshold when it exceeds the first threshold, totals the difference pixel number as a turning pixel number when it is less than the second threshold, totals the difference pixel number as a movement pixel number when it exceeds the second threshold, and determines whether or not a sign of delivery exists in the livestock based on the totaled turning pixel number and the movement pixel number.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a parturition detection device, a parturition detection method, and a parturition detection program. [Background technology]

[0002] Manually detecting all signs of birth in livestock such as cows, horses, sheep, and pigs is costly. In addition, human errors can occur. Therefore, in recent years, there has been progress in the development of technology that uses cameras to automate the detection of signs of birth in livestock.

[0003] For example, Patent Document 1 proposes a method for detecting signs of livestock birth using images of the livestock. Specifically, Patent Document 1 proposes calculating the amount of movement of the livestock from the obtained captured images, and detecting signs of livestock birth based on an increase in the calculated amount of movement of the livestock. According to this method, it is possible to automate the detection of signs of livestock birth, and as a result, it is possible to reduce the cost of monitoring the livestock. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 158698 Summary of the Invention [Problem to be solved by the invention]

[0005] The inventors of the present invention found that the above-mentioned conventional method for detecting signs of parturition has the following problems. That is, it is known that an increase in the amount of movement of livestock is a sign of parturition. However, this behavior is not limited to movement. As another example of a predictive behavior, livestock that is a sign of parturition may turn around. Turning is an act of turning almost on the spot. Therefore, the amount of parallel movement in the turning is not very large or is almost nonexistent. With the above-mentioned conventional method, such turning is recognized as an act with a small amount of movement, so there is a problem that the sign of parturition due to an increase in turning is overlooked, and the accuracy of detecting the sign of parturition may be reduced.

[0006] In one aspect, the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for improving the accuracy of detecting signs of parturition in livestock. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the present invention employs the following configuration.

[0008] That is, a parturition detection device according to one aspect of the present invention comprises an image acquisition unit that acquires a plurality of captured images generated by continuously capturing images of livestock using an imaging device, a position calculation unit that calculates the position of the center of gravity of the livestock from each of the acquired captured images, a movement amount calculation unit that calculates the difference in the calculated center of gravity position between each of the captured images as a movement amount, a difference pixel number calculation unit that calculates the number of differential pixels between each of the captured images, a counting unit that determines whether the difference pixel number calculated between each of the captured images exceeds a first threshold, and if the difference pixel number exceeds the first threshold, further determines whether the amount of movement of the center of gravity position calculated between each of the captured images exceeds a second threshold, and if the amount of movement of the center of gravity position is less than the second threshold, counts the difference pixel number as a number of rotation pixels, and if the amount of movement of the center of gravity position exceeds the second threshold, counts the difference pixel number as a number of movement pixels, and a determination unit that determines whether the livestock is showing signs of parturition based on the counted number of rotation pixels and number of movement pixels.

[0009] When livestock turn, the area of ​​the livestock shown in the captured image changes. Therefore, when the livestock turns, a certain amount of pixel change occurs in the captured image. In this configuration, the pixel change is evaluated by the pixel difference number, and the change (amount of movement) in the center of gravity of the livestock is evaluated. In this way, when the livestock performs an action with a large pixel difference number (exceeding the first threshold), the action can be classified as a moving action or a turning action based on the amount of movement of the center of gravity. Therefore, in this configuration, the amount of turning action can be counted as well as the amount of moving action of the livestock, and as a result, in determining whether or not the livestock has a sign of parturition, not only the amount of moving action of the livestock but also the amount of turning action, which was overlooked in the conventional method, can be taken into consideration. Therefore, according to this configuration, the accuracy of detecting a sign of parturition in livestock can be improved.

[0010] In the parturition detection device according to the above aspect, the determining step may be configured to determine whether or not the livestock has a sign of parturition based on whether or not the counted number of turning pixels and the number of moving pixels exceed a threshold. With this configuration, it is possible to detect the sign of parturition in the livestock in a simple manner.

[0011] In the parturition detection device according to the above aspect, the determining may be configured to determine whether or not the livestock has a sign of parturition from the aggregated number of turning pixels and number of moving pixels using a trained determination model. According to this configuration, in a situation where a machine learning technique is employed, by adopting not only the amount of movement behavior of the livestock but also the amount of turning behavior (number of turning pixels) as an explanatory variable, it is possible to improve the accuracy of detecting the sign of parturition in the livestock.

[0012] In the delivery detection device according to the above aspect, the livestock may be either a cow or a horse. With this configuration, when detecting a sign of delivery in either a cow or a horse, the detection accuracy can be improved.

[0013] In the parturition detection device according to the above aspect, the imaging device may be a thermography camera. A thermography camera can easily capture the movements of livestock even in dark places. Therefore, according to this configuration, it is possible to make it difficult for light to affect the detection of signs of parturition in livestock.

[0014] Livestock that may show signs of parturition may be kept in a farrowing cage. In this case, in the parturition detection device according to the above aspect, the imaging device may be installed at a predetermined height in one of the four corners of the farrowing cage. If the livestock moves out of the imaging range of the imaging device, an image showing the livestock cannot be obtained, resulting in inadequate monitoring of the livestock. With this configuration, blind spots are less likely to occur in the imaging range of the imaging device, making it less likely for the livestock to move out of the imaging range of the imaging device. As a result, it is possible to reduce the probability of inadequate monitoring of the livestock caused by the livestock moving out of the imaging range.

[0015] In the parturition detection device according to the above aspect, the imaging device may be arranged so that the range in the parturition box where the livestock may be present falls within the imaging range of the imaging device, and only one livestock may be present in the parturition box. As an example of the arrangement, the imaging device may be arranged so that the center of the angle of view of the imaging device is at the center of the parturition box, so that the range in the parturition box where the livestock may be present falls within the imaging range of the imaging device. These configurations can prevent the livestock from moving out of the imaging range of the imaging device. In addition, since there is only one livestock present in the parturition box, it is possible to prevent the process of calculating the center of gravity of the livestock from becoming complicated. Therefore, these configurations can facilitate the process of detecting signs of parturition in livestock, and further improve the detection accuracy.

[0016] As another embodiment of the parturition detection device according to each of the above-mentioned configurations, one aspect of the present invention may be an information processing method or a program that realizes all or part of the above-mentioned components, or may be a storage medium that stores such a program and is readable by a computer or other device or machine. Here, a storage medium that is readable by a computer or the like is a medium that stores information such as a program by electrical, magnetic, optical, mechanical, or chemical action.

[0017] For example, a delivery detection method according to one aspect of the present invention is an information processing method in which a computer executes the steps of: acquiring a plurality of captured images generated by continuously capturing images of livestock using an imaging device; calculating the position of the center of gravity of the livestock from each of the acquired captured images; calculating the difference in the calculated center of gravity position between each of the captured images as an amount of movement; calculating the number of differential pixels between each of the captured images; determining whether the number of differential pixels calculated between each of the captured images exceeds a first threshold; if the number of differential pixels exceeds the first threshold, determining whether the amount of movement of the center of gravity position calculated between each of the captured images exceeds a second threshold; when the amount of movement of the center of gravity position is less than the second threshold, counting the number of differential pixels as a number of rotation pixels; when the amount of movement of the center of gravity position exceeds the second threshold, counting the number of differential pixels as a number of movement pixels; and determining whether the livestock is showing signs of delivery based on the counted number of rotation pixels and number of movement pixels.

[0018] Furthermore, for example, a parturition detection program according to one aspect of the present invention is a program for causing a computer to execute the steps of: acquiring a plurality of captured images generated by continuously capturing images of livestock using an imaging device; calculating the position of the center of gravity of the livestock from each of the acquired captured images; calculating the difference in the calculated center of gravity position between each of the captured images as an amount of movement; calculating the number of differential pixels between each of the captured images; determining whether the number of differential pixels calculated between each of the captured images exceeds a first threshold; if the number of differential pixels exceeds the first threshold, determining whether the amount of movement of the center of gravity position calculated between each of the captured images exceeds a second threshold; when the amount of movement of the center of gravity position is less than the second threshold, collecting the number of differential pixels as a number of rotation pixels; when the amount of movement of the center of gravity position exceeds the second threshold, collecting the number of differential pixels as a number of movement pixels; and determining whether the livestock is showing signs of parturition based on the collected number of rotation pixels and number of movement pixels. Effect of the Invention

[0019] According to the present invention, it is possible to improve the accuracy of detecting signs of parturition in livestock. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 shows a schematic diagram of an example of a situation in which the present invention is applied. [Diagram 2] FIG. 2 is a schematic diagram illustrating an example of a hardware configuration of the parturition detection device according to the embodiment. [Figure 3A] FIG. 3A is a plan view illustrating an example of a livestock farrowing pen according to an embodiment. [Figure 3B] FIG. 3B is a side view that illustrates an example of a livestock farrowing pen according to an embodiment. [Figure 4] FIG. 4 is a schematic diagram showing an example of a software configuration of the parturition detection device according to the embodiment. [Diagram 5] FIG. 5 is a flowchart illustrating an example of a processing procedure of the parturition detection device according to the embodiment. [Figure 6] FIG. 6 illustrates a schematic diagram of an example process for detecting signs of labor using a trained decision model. [Figure 7] FIG. 7 illustrates a schematic diagram of an example process for generating a trained decision model that can be used to detect signs of labor. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] An embodiment according to one aspect of the present invention (hereinafter, also referred to as "the present embodiment") will be described below with reference to the drawings. However, the present embodiment described below is merely an example of the present invention in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present invention. In other words, in implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted. Note that while the data appearing in this embodiment is described in natural language, more specifically, it is specified in pseudo-language, commands, parameters, machine language, etc. that can be recognized by a computer.

[0022] §1 Examples of application 1 is a schematic diagram illustrating an example of a scene in which the present invention is applied. A parturition detection device 1 according to this embodiment is a computer configured to monitor the behavior of livestock D using an imaging device S and detect signs of parturition in the livestock D.

[0023] First, the parturition detection device 1 according to this embodiment acquires a plurality of captured images 3 generated by continuously capturing images of the livestock D using the imaging device S. Next, the parturition detection device 1 calculates the position of the center of gravity of the livestock D from each of the acquired captured images 3. Next, the parturition detection device 1 calculates the difference in the calculated center of gravity positions between each of the captured images 3 as the amount of movement of the livestock D. In addition, the parturition detection device 1 calculates the difference in the number of pixels between each of the captured images 3.

[0024] The parturition detection device 1 judges whether the difference in pixel count calculated between each captured image 3 exceeds a first threshold value. The first threshold value may be appropriately set so as to detect the moving behavior and turning behavior of the livestock D. If the difference in pixel count exceeds the first threshold value, the parturition detection device 1 further judges whether the movement amount of the center of gravity position calculated between each captured image 3 (the movement amount of the center of gravity position calculated between the captured images 3 in which the difference in pixel count exceeding the first threshold value is calculated) exceeds a second threshold value. The second threshold value may be appropriately set so as to distinguish between the moving behavior and turning behavior of the livestock D. If the movement amount of the center of gravity position is less than the second threshold value, the parturition detection device 1 counts the corresponding difference in pixel count as the turning pixel count. On the other hand, if the movement amount of the center of gravity position exceeds the second threshold value, the parturition detection device 1 counts the corresponding difference in pixel count as the moving pixel count. Then, the parturition detection device 1 judges whether there is a sign of parturition in the livestock D based on the counted turning pixel count and moving pixel count.

[0025] As described above, in this embodiment, the change in the image accompanying the behavior of livestock D is evaluated using the difference in pixel count, and the amount of movement of the center of gravity of livestock D is also evaluated. As a result, when livestock D performs a behavior that results in a large number of difference pixels, the behavior performed by livestock D can be classified as a moving behavior or a turning behavior based on the amount of movement of the center of gravity. Therefore, in this embodiment, the amount of turning behavior of livestock D can be tallied as well as the amount of moving behavior of livestock D, and as a result, not only the amount of moving behavior of livestock D but also the amount of turning behavior can be taken into consideration when determining whether livestock D is about to give birth. Therefore, according to this embodiment, it is possible to improve the accuracy of detecting signs of livestock D's giving birth.

[0026] In addition, the type of livestock D is not particularly limited as long as it is an animal that can move and turn as a sign of parturition, and may be appropriately selected depending on the embodiment. As an example, the livestock D may be either a cow or a horse. Furthermore, the number of livestock D kept within the imaging range of the imaging device S may be arbitrary. As an example, multiple livestock D may be kept within the imaging range of the imaging device S. However, from the viewpoint of facilitating image analysis of the livestock D captured in the captured image 3, it is preferable that the number of livestock D kept within the imaging range of the imaging device S is only one.

[0027] §2 Configuration Example [Hardware configuration] Fig. 2 illustrates a schematic example of a hardware configuration of the parturition detection device 1 according to this embodiment. The parturition detection device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, an input device 15, an output device 16, and a drive 17 are electrically connected. In Fig. 2, the communication interface and the external interface are described as a "communication I / F" and an "external I / F".

[0028] The control unit 11 includes a hardware processor such as a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM), and is configured to execute information processing based on programs and various data.

[0029] The storage unit 12 is an example of a memory, and is configured, for example, by a hard disk drive, a solid state drive, etc. In this embodiment, the storage unit 12 stores various information such as a delivery detection program 81.

[0030] The parturition detection program 81 is a program for causing the parturition detection device 1 to execute information processing (FIG. 5) described below regarding detection of signs of parturition in the livestock D. The parturition detection program 81 includes a series of instructions for the information processing. Details will be described later.

[0031] The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc., and is an interface for performing wired or wireless communication via a network. Using the communication interface 13, the parturition detection device 1 can perform data communication with other information processing devices via the network.

[0032] The external interface 14 is, for example, a USB (Universal Serial Bus) port, a dedicated port, etc., and is an interface for connecting to an external device. The type and number of the external interfaces 14 may be selected arbitrarily.

[0033] In this embodiment, the imaging device S may be connected to the parturition detection device 1 via the communication interface 13 or the external interface 14. As long as the imaging device S is capable of observing the livestock D, the type of the imaging device S is not particularly limited and may be appropriately selected depending on the embodiment. In one example, the imaging device S may be a general RGB camera, a depth camera (depth sensor), a stereo camera, etc. In another example, the imaging device S may be a thermography camera. A thermography camera can easily capture the movements of the livestock D even in dark places. Therefore, by employing a thermography camera as the imaging device S, it is possible to reduce the influence of light when detecting signs of parturition in the livestock D.

[0034] The input device 15 is a device for inputting, for example, a mouse, a keyboard, etc. The output device 16 is a device for outputting, for example, a display, a speaker, etc. An operator such as a user can operate the parturition detection device 1 by using the input device 15 and the output device 16. The input device 15 and the output device 16 may be integrally configured, for example, by a touch panel display, etc.

[0035] The drive 17 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading various information such as a program stored in a storage medium 91. The storage medium 91 is a medium that accumulates information such as a program by electrical, magnetic, optical, mechanical, or chemical action so that a computer or other device, machine, or the like can read the various information such as the stored program. The parturition detection program 81 may be stored in the storage medium 91. The parturition detection device 1 may acquire the parturition detection program 81 from the storage medium 91. Note that, in FIG. 2, a disk-type storage medium such as a CD or a DVD is illustrated as an example of the storage medium 91. However, the type of the storage medium 91 is not limited to the disk type, and may be other than the disk type. An example of a storage medium other than the disk type may be a semiconductor memory such as a flash memory. The type of the drive 17 may be selected arbitrarily according to the type of the storage medium 91.

[0036] Regarding the specific hardware configuration of the parturition detection device 1, components can be omitted, replaced, and added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processor may be configured with a microprocessor, an FPGA (field-programmable gate array), or the like. The storage unit 12 may be configured with a RAM and a ROM included in the control unit 11. At least one of the communication interface 13, the external interface 14, the input device 15, the output device 16, and the drive 17 may be omitted. The parturition detection device 1 may include an imaging device S. The parturition detection device 1 may be configured with multiple computers. In this case, the hardware configurations of the computers may or may not be the same. In addition, the parturition detection device 1 may be an information processing device designed specifically for the service provided, as well as a general-purpose server device, a general-purpose PC (Personal Computer), a mobile terminal (e.g., a smartphone), a tablet PC, a microcomputer, or the like.

[0037] [An example of livestock breeding conditions] 3A and 3B are a plan view and a side view showing an example of a farrowing room 5 for livestock D according to this embodiment. As an example of a rearing method, livestock D that may show signs of parturition may be reared in a farrowing room 5 as shown in the figure.

[0038] In the example shown in Figures 3A and 3B, the birthing room 5 is formed in a rectangular shape in a plan view. The width 5W and depth 5D of the birthing room 5 may be any value. As an example, the width 5W and depth 5D may each be 3 m to 5 m. The area of ​​the birthing room 5 in a plan view is 9 m. 2 ~25m 2 It may be.

[0039] In this embodiment, one imaging device S is installed at a predetermined height 5H in one of the four corners of the farrowing room 5 in a plan view. As an example, the predetermined height 5H may be determined to be higher than the upper end of the livestock D in order to monitor the livestock D from above. As a specific example, the predetermined height 5H may be 2.5 m to 3 m from the floor. By disposing the imaging device S at one of the four corners of the farrowing room 5, it is possible to reduce the occurrence of blind spots in the imaging range of the imaging device S. This makes it possible to reduce the likelihood of the livestock D moving outside the imaging range of the imaging device S. As a result, it is possible to reduce the probability of inadequate monitoring of the livestock D due to the livestock D moving outside the imaging range in the farrowing room 5.

[0040] The orientation of the imaging device S may be appropriately determined according to the embodiment so that the livestock D can be observed. The number of livestock D present in the farrowing room 5 may be arbitrary. In one example, the imaging device S may be arranged so that the range in which the livestock D may exist in the farrowing room 5 falls within the imaging range of the imaging device S, and only one livestock D may exist in the farrowing room 5. As long as at least a part of the livestock D falls within the imaging range of the imaging device S, "the range in which the livestock D may exist in the farrowing room 5 falls within the imaging range of the imaging device S" may include a part of the farrowing room 5 being out of the imaging range of the imaging device S. Preferably, the imaging device S may be arranged so that the center of the angle of view of the imaging device S is located at the center of the farrowing room 5 so that the range in which the livestock D may exist in the farrowing room 5 falls within the imaging range of the imaging device S. According to these monitoring methods, it is possible to prevent the livestock D from moving outside the imaging range of the imaging device S. In addition, since the number of livestock D present in the farrowing room 5 is one, it is possible to prevent the process of detecting the livestock D from becoming complicated. Therefore, according to these monitoring methods, the process of detecting signs of parturition in livestock D can be facilitated and the detection accuracy can be further improved.

[0041] The shape of the farrowing chamber 5 is not limited to the examples in Figures 3A and 3B, and may be appropriately selected depending on the embodiment. As another example, the shape of the farrowing chamber 5 in a plan view may be a square, another parallelogram, a circle, an ellipse, a triangle, another polygon, or a combination of these. Each corner of the farrowing chamber 5 may be rounded or may not be exactly 90 degrees. When the farrowing chamber 5 is configured in a shape other than a rectangle, the four corners of the farrowing chamber 5 may be identified by regarding the shape of the farrowing chamber 5 as a rectangle (forming an appropriate rectangle within the farrowing chamber).

[0042] [Software configuration] 4 illustrates an example of a software configuration of the parturition detection device 1 according to this embodiment. The control unit 11 of the parturition detection device 1 loads a parturition detection program 81 stored in the storage unit 12 into the RAM. The control unit 11 then executes instructions included in the parturition detection program 81 loaded into the RAM by the CPU to control each component. As a result, as shown in FIG. 4, the parturition detection device 1 according to this embodiment operates as a computer including an image acquisition unit 111, a position calculation unit 112, a movement amount calculation unit 113, a difference pixel number calculation unit 114, a counting unit 115, a determination unit 116, and an output unit 117 as software modules.

[0043] The image acquisition unit 111 is configured to acquire a plurality of captured images 3 generated by continuously capturing images of the livestock D by the imaging device S. The position calculation unit 112 is configured to calculate the position of the center of gravity of the livestock D from each of the acquired captured images 3. The movement amount calculation unit 113 is configured to calculate the difference in the calculated position of the center of gravity of the livestock D between each of the captured images 3 as the movement amount of the livestock D. The difference pixel number calculation unit 114 is configured to calculate the difference pixel number between each of the captured images 3.

[0044] The counting unit 115 is configured to determine whether the differential pixel number calculated between each captured image 3 exceeds a first threshold value, and if the differential pixel number exceeds the first threshold value, further determine whether the movement amount of the center of gravity position calculated between each captured image 3 exceeds a second threshold value, and if the movement amount of the center of gravity position is less than the second threshold value, count the differential pixel number as the number of turning pixels, and if the movement amount of the center of gravity position exceeds the second threshold value, count the differential pixel number as the number of moving pixels. The determination unit 116 is configured to determine whether the livestock D has a sign of parturition based on the counted number of turning pixels and number of moving pixels. The output unit 117 is configured to output a determination result of the livestock D's sign of parturition.

[0045] Each software module of the parturition detection device 1 will be described in detail in an operation example described later. In this embodiment, an example is described in which each software module of the parturition detection device 1 is realized by a general-purpose CPU. However, some or all of the above software modules may be realized by one or more dedicated processors. In other words, each of the above modules may be realized as a hardware module. Furthermore, with regard to the software configuration of the parturition detection device 1, software modules may be omitted, replaced, or added as appropriate depending on the embodiment.

[0046] §3 Example of operation FIG. 5 is a flowchart showing an example of the processing procedure of the parturition detection device 1 according to this embodiment. The following processing procedure is an example of a parturition detection method. However, the following processing procedure is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0047] (Step S101) In step S101, the control unit 11 operates as the image acquisition unit 111 and acquires a plurality of captured images 3 generated by continuously capturing images of the livestock D by the imaging device S.

[0048] In one example, the imaging device S may be connected to the parturition detection device 1 via the communication interface 13 or the external interface 14, and the control unit 11 may acquire the plurality of captured images 3 directly from the imaging device S. In another example, the control unit 11 may acquire the plurality of captured images 3 indirectly from the imaging device S via, for example, a network, a storage medium 91, an external storage device, or the like. The external storage device may be connected to the parturition detection device 1 via the communication interface 13 or the external interface 14.

[0049] The multiple captured images 3 may be generated by capturing images of the livestock D at predetermined intervals for a certain period of time. The certain period of time and the predetermined interval may be appropriately set so as to enable detection of signs of parturition in the livestock D. As an example, the certain period of time may be on the order of several minutes (e.g., 10 minutes), and the predetermined interval may be on the order of several milliseconds to several seconds (e.g., 1 second). After acquiring the multiple captured images 3, the control unit 11 proceeds to the next step S102.

[0050] (Step S102) In step S102, the control unit 11 operates as the position calculation unit 112 and calculates the position of the center of gravity of the livestock D from each captured image 3 that has been acquired.

[0051] The position of the center of gravity of the livestock D may be calculated by any image processing. As an example, an area in which the livestock D appears may be extracted from each captured image 3 by a method such as pattern matching or segmentation using a trained model. Furthermore, when extracting the area in which the livestock D appears, each captured image 3 may be binarized. Then, the position of the center of gravity may be calculated from the extracted area in which the livestock D appears. After calculating the position of the center of gravity of the livestock D in each captured image 3, the control unit 11 proceeds to the next step S103.

[0052] (Step S103) In step S103, the control unit 11 operates as the movement amount calculation unit 113, and calculates the difference in the calculated center of gravity positions between the captured images 3 as the movement amount.

[0053] The plurality of captured images 3 are arranged in a time series. Typically, the control unit 11 may calculate the difference in the calculated center of gravity position between the captured images 3 adjacent in the time series as the amount of movement. As a specific example, it is assumed that three images, captured image 3_1, captured image 3_2, and captured image 3_3, are arranged in a time series in order from the earliest to the latest. The control unit 11 may calculate the difference in the center of gravity position between each captured image (3_1, 3_2) (3_2, 3_3). However, the method of calculating the amount of movement is not limited to this example as long as it is a method of calculating the difference between previous and next images in the time series, and may be appropriately determined according to the embodiment. After calculating the amount of movement of the livestock D between each captured image 3, the control unit 11 proceeds to the next step S104.

[0054] (Step S104) In step S104, the control unit 11 operates as the pixel difference calculation unit 114 and calculates the pixel difference between each captured image 3.

[0055] As an example, the control unit 11 extracts pixels having differences in pixel values ​​by taking image differences between the captured images 3, and counts the extracted pixels. In this way, the control unit 11 may calculate the number of differential pixels between the captured images 3. The amount of difference (threshold) in pixel values ​​that is used to determine that a "difference exists" may be set appropriately depending on the embodiment.

[0056] The pair of captured images 3 for which the difference in pixel count is to be calculated may be the same as the pair of captured images 3 for which the difference (movement amount) in the center of gravity position is to be calculated. That is, typically, the control unit 11 may calculate the difference in pixel count between captured images adjacent in time series among the multiple captured images 3. In the above specific example, the control unit 11 may calculate the difference in pixel count between each captured image (3_1, 3_2) (3_2, 3_3). However, the method of calculating the difference in pixel count is not limited to this example as long as it is a method of calculating the difference between previous and next images in time series, and may be appropriately determined according to the embodiment. After calculating the difference in pixel count between each captured image 3, the control unit 11 proceeds to the next step S105.

[0057] (Step S105) In step S105, the control unit 11 operates as the counting unit 115 and determines whether or not the differential pixel number calculated between the target captured images 3 exceeds a first threshold value. As an example, when the process from step S105 onwards is executed for the first time, the pair of target captured images 3 may be the pair of captured images 3 that are earliest in time series. When the determination process is completed, the control unit 11 proceeds to the next step S106. Note that in this embodiment, the processes from step S105 to step S111 described below are performed as the operation of the counting unit 115.

[0058] (Step S106) In step S106, the control unit 11 determines where the process should branch depending on the result of the determination in the process of step S105. If the difference pixel number exceeds the first threshold, the control unit 11 advances the process to the next step S107. On the other hand, if the difference pixel number is less than the first threshold, the control unit 11 skips the processes of steps S107 to S110 and advances the process to step S111.

[0059] When the difference pixel number is equal to the first threshold, the process may branch to either step S107 or step S111. That is, the case in which the difference pixel number is equal to the first threshold may be assigned to either the first case in which the difference pixel number exceeds the first threshold or the second case in which the difference pixel number is less than the first threshold.

[0060] (Steps S107 and S108) In step S107, the control unit 11 judges whether the movement amount of the center of gravity position calculated between the target captured images 3 exceeds a second threshold. In step S108, the control unit 11 determines the branch destination of the process according to the judgment result by the process of step S107. If the movement amount is less than the second threshold, the control unit 11 advances the process to the next step S109. On the other hand, if the movement amount exceeds the second threshold, the control unit 11 advances the process to the next step S110. Note that, if the movement amount is equal to the second threshold, the branch destination of the process may be either step S109 or step S110. That is, the case where the movement amount is equal to the second threshold may be assigned to either the first case where the movement amount exceeds the second threshold or the second case where the movement amount is less than the second threshold.

[0061] (Steps S109 and S110) In step S109, the control unit 11 counts up the number of pixel differences calculated between the target captured images 3 as the number of turning pixels. On the other hand, in step S110, the control unit 11 counts up the number of pixel differences calculated between the target captured images 3 as the number of moving pixels. When the counting of the number of pixel differences is completed in any step, the control unit 11 advances the process to step S111.

[0062] (Step S111) In step S111, the control unit 11 determines whether or not to end the tallying process from step S105 to step S110. If there are any pairs of captured images 3 to which the tallying process is to be applied, the control unit 11 determines not to end the tallying process, returns the process to step S105, and executes the tallying process for any of the remaining pairs of captured images 3. On the other hand, if there are no pairs of captured images 3 to which the tallying process is to be applied, the control unit 11 determines to end the tallying process, and proceeds to the next step S112.

[0063] In this embodiment, the pair of captured images 3 for which the movement amount and the difference in pixel count are calculated are the targets for which the tallying process from step S105 to step S110 is executed. Due to the conditional branch in step S111, the control unit 11 repeats the execution of the tallying process until it is completed for all pairs of captured images 3 for which the movement amount and the difference in pixel count are calculated. This allows the control unit 11 to execute the tallying process between each pair of captured images 3.

[0064] The order of performing the processes is not particularly limited and may be appropriately determined depending on the embodiment. As an example, the control unit 11 may perform the tallying process from step S105 to step S110 in order from the pair of captured images 3 with the earliest time in the chronological order. In the above specific example, the control unit 11 may perform the tallying process from step S105 to step S110 for the pair of captured images (3_1, 3_2). Thereafter, the control unit 11 may return the process to step S105 by the process of step S111, and perform the tallying process from step S105 to step S110 for the next pair of captured images (3_2, 3_3). When the tallying process is completed for all pairs, the control unit 11 advances the process to the next step S112.

[0065] (Step S112) In step S112, the control unit 11 operates as the determination unit 116 and determines whether or not the livestock D has a sign of parturition based on the counted number of turning pixels and number of moving pixels. The method of using the number of turning pixels and number of moving pixels in determining the sign of parturition may be appropriately selected depending on the embodiment. Two methods are shown below as examples.

[0066] (A) First Method In the first method, the control unit 11 compares the number of turning pixels and the number of moving pixels with a threshold value. That is, the control unit 11 determines whether or not the livestock D has a sign of parturition based on whether or not the counted number of turning pixels and the number of moving pixels exceed the threshold value.

[0067] As an example of a comparison method, the control unit 11 may determine whether the number of turning pixels exceeds a third threshold and the number of moving pixels exceeds a fourth threshold. The third threshold and the fourth threshold may each be appropriately set so as to be able to determine signs of parturition. The third threshold and the fourth threshold may be the same as each other, or may be different from each other. If the number of turning pixels exceeds the third threshold and the number of moving pixels exceeds the fourth threshold, the control unit 11 may determine that the livestock D has signs of parturition. On the other hand, if the number of turning pixels is less than the third threshold or the number of moving pixels is less than the fourth threshold, the control unit 11 may determine that the livestock D does not have signs of parturition. According to the first method, signs of parturition in the livestock D can be detected in a simple manner.

[0068] In addition, the case where the number of turning pixels is equal to the third threshold value may be assigned to either the first case where the number of turning pixels exceeds the third threshold value or the second case where the number of turning pixels is less than 3. Similarly, the case where the number of moving pixels is equal to the fourth threshold value may be assigned to either the first case where the number of moving pixels exceeds the fourth threshold value or the second case where the number of moving pixels is less than the fourth threshold value.

[0069] (B) Second Method Fig. 6 illustrates an example of a process for detecting signs of parturition in livestock D using a trained judgment model 6. As shown in Fig. 6, in the second method, the control unit 11 uses the trained judgment model 6 to judge whether or not there are signs of parturition in livestock D from the aggregated number of turning pixels and number of moving pixels. In other words, the aggregated number of turning pixels and number of moving pixels are used as explanatory variables of the trained judgment model 6.

[0070] The judgment model 6 is configured by a machine learning model having one or more calculation parameters used in a calculation to derive an inference result (i.e., a judgment result as to whether or not there is a sign of labor). As long as it is possible to execute a process of judging a sign of labor from the number of turning pixels and the number of moving pixels, the type of machine learning model is not particularly limited and may be appropriately selected depending on the embodiment. As an example, the judgment model 6 may be configured by a neural network, a support vector machine, a regression model, a decision tree model, or the like.

[0071] FIG. 7 shows a schematic example of a process for generating a trained judgment model 6 that can be used to detect signs of parturition. A plurality of learning datasets 7 are used to train the judgment model 6 in machine learning. Each learning dataset 7 is composed of a combination of training data 71 and a correct answer label 72. The training data 71 is configured to include the number of turning pixels and the number of moving pixels for learning. The correct answer label 72 is configured to include a true value of whether or not the livestock D has a sign of parturition in the situation shown by the corresponding training data 71.

[0072] Training the judgment model 6 is performed by adjusting (optimizing) values ​​of calculation parameters so as to derive true values ​​of corresponding correct labels 72 from the training data 71 using each learning data set 7. The machine learning method may be appropriately selected according to the type of machine learning model to be adopted. As an example, the machine learning method may be a method such as backpropagation, solving an optimization problem, performing regression analysis, or random forest. Any computer (hereinafter also referred to as a model generating device) may be used for the machine learning. In one example, the model generating device may be the parturition detection device 1. In another example, the model generating device may be a computer other than the parturition detection device 1.

[0073] In the examples of Figs. 6 and 7, the judgment model 6 is configured by a neural network. When a neural network is adopted to configure the judgment model 6, the judgment model 6 is typically configured to include an input layer, one or more intermediate layers (hidden layers), and an output layer. For each layer, any type of layer such as a fully connected layer may be adopted. The number of layers included in the judgment model 6, the number of nodes (neurons) in each layer, and the connection relationship of the nodes may be appropriately determined according to the embodiment. The weight of the connection between each node, the threshold value of each node, etc. are examples of the above-mentioned calculation parameters.

[0074] As an example of a training process when a neural network is adopted, the model generation device inputs the training data 71 of each learning data set 7 into the judgment model 6 and executes a forward propagation calculation process of the judgment model 6. As a calculation result of this forward propagation, the model generation device acquires an output value corresponding to the result of judging whether or not there is a sign of parturition in the livestock D based on the training data 71 (the number of turning pixels and the number of moving pixels for learning) of each learning data set 7. The model generation device calculates an error between the acquired output value and the true value indicated by the corresponding correct answer label 72, and further calculates the gradient of the calculated error. Next, the model generation device calculates an error of the value of the calculation parameter of the judgment model 6 by backpropagating the gradient of the calculated error by the error backpropagation method. Then, the model generation device updates the value of the calculation parameter based on the calculated error.

[0075] Through this series of update processes, the model generation device adjusts the parameter values ​​of the judgment model 6 so as to reduce the sum of errors between the judgment result (output value) and the true value for each learning data set 7. This adjustment of the parameter values ​​may be repeated until a predetermined condition is satisfied, such as, for example, performing the adjustment a specified number of times or until the calculated sum of errors becomes equal to or less than a threshold value. Furthermore, machine learning conditions, such as, for example, a loss function and a learning rate, may be set appropriately depending on the embodiment. Through this machine learning process, a trained judgment model 6 that has acquired the ability to judge the signs of parturition of the livestock D based on the number of turning pixels and the number of moving pixels can be generated.

[0076] The trained judgment model 6 generated by the above machine learning may be stored as learning result data 125. As long as it is possible to hold information for executing the calculation of the trained judgment model 6, the configuration of the learning result data 125 is not particularly limited and may be appropriately determined depending on the embodiment. As an example, the learning result data 125 may be configured to include information indicating the configuration of the judgment model 6 (e.g., the structure of a neural network, etc.) and the values ​​of calculation parameters adjusted by machine learning.

[0077] The learning result data 125 may be stored in any storage area. The storage destination of the learning result data 125 may be, for example, the RAM in the control unit 11, the storage unit 12, an external storage device, a storage medium, a storage unit of another computer, or a combination of these. The storage medium may be, for example, a CD, a DVD, a flash memory, or the like. The external storage device may be, for example, a data server such as a NAS (Network Attached Storage). Alternatively, the external storage device may be an external storage device directly connected to the computer.

[0078] The learning result data 125 may be referred to as appropriate to set the trained judgment model 6 in a usable state on the computer. When the trained judgment model 6 is generated by another computer, the learning result data 125 may be provided to the parturition detection device 1 in any manner and at any timing. As one example, the learning result data 125 may be provided to the parturition detection device 1 via a network, a storage medium 91, etc. As another example, the learning result data 125 may be incorporated in advance into the parturition detection device 1. The learning result data 125 may be stored in the memory unit 12 of the parturition detection device 1.

[0079] Returning to FIG. 6, when the second method is adopted, the judgment unit 116 has a trained judgment model 6 by holding the learning result data 125. In step S112, the control unit 11 sets the trained judgment model 6 by referring to the learning result data 125. Then, the control unit 11 inputs the aggregated number of turning pixels and number of moving pixels to the trained judgment model 6, and executes a calculation process of the trained judgment model 6. As a result of executing this calculation process, the control unit 11 acquires an output value from the trained judgment model 6 corresponding to the result of judging whether or not there are signs of parturition in the livestock D.

[0080] (C) Summary By employing at least one of the above two methods, the control unit 11 can determine whether or not there are signs of parturition in the livestock D based on the counted number of turning pixels and number of moving pixels. When the determination of the signs of parturition is completed, the control unit 11 proceeds to the next step S113.

[0081] The information used to determine signs of parturition does not have to be limited to the aggregated number of turning pixels and number of moving pixels. Other information than the number of turning pixels and number of moving pixels may be further used to determine signs of parturition. In this case, the control unit 11 further acquires the other information before executing the processing of step S113. In step S113, the control unit 11 further uses the other information to determine whether or not the livestock D has signs of parturition. When the second method described above is adopted as the determination method, the training data 71 of each learning data set 7 is configured to further include other information for learning.

[0082] (Step S113) Returning to FIG. 5, in step S113, the control unit 11 operates as the output unit 117 and outputs the determination result of the sign of parturition of the livestock D.

[0083] The output destination of the judgment result and the content of the information to be output may be appropriately determined according to the embodiment. For example, the control unit 11 may directly output the judgment result obtained in step S112. Furthermore, the control unit 11 may execute some information processing based on the obtained judgment result. The control unit 11 may then output the result of executing the information processing as information related to the judgment result. As an example, when it is determined that the livestock D has a sign of parturition, the control unit 11 may generate notification information for notifying the same and output the generated notification information. The output destination may be, for example, the output device 16, an output device of another computer (for example, a terminal of a zookeeper, veterinarian, etc.), etc. The output format may be, for example, a screen output, an audio output, a printout, etc.

[0084] In one example, the output process of step S113 may be executed in both cases where it is determined that there are signs of labor and where it is determined that there are no signs of labor. In another example, the output process of step S113 may be executed only when it is determined that there are signs of labor, and when it is determined that there are no signs of labor, the output process may be omitted.

[0085] When the output of information related to the determination result is completed, the control unit 11 ends the processing procedure according to this operation example. The control unit 11 may repeatedly execute the series of information processing from step S101 to step S113 for any period and at any timing. This allows the parturition detection device 1 to continuously monitor the signs of parturition of the livestock D.

[0086] [Features] As described above, in this embodiment, in the processing of steps S101 to S112, the change in the image accompanying the behavior of the livestock D is evaluated using the difference in pixel number, and the amount of movement of the center of gravity of the livestock D is also evaluated. As a result, when the livestock D performs a behavior that results in a large difference in pixel number, the behavior performed by the livestock D can be classified as a moving behavior or a turning behavior based on the amount of movement of the center of gravity. Therefore, in this embodiment, the amount of turning behavior of the livestock D can be counted as well as the amount of moving behavior of the livestock D, and as a result, in the process of determining the signs of parturition of the livestock D in step S112, not only the amount of moving behavior of the livestock D but also the amount of turning behavior can be taken into consideration. Therefore, according to this embodiment, it is possible to improve the accuracy of detecting signs of parturition of the livestock D.

[0087] § 4 Variations Although the embodiment of the present invention has been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements or modifications can be made without departing from the scope of the present invention. For example, the following modifications are possible. In the following, the same reference numerals are used for the same components as in the above embodiment, and the description of the same points as in the above embodiment is omitted as appropriate. The following modifications can be combined as appropriate.

[0088] In the above embodiment, one livestock D is raised in the farrowing room 5. However, the environment in which the livestock D is raised does not have to be limited to this example. In another example, a plurality of livestock D may be raised in one farrowing room 5. In yet another example, one or more livestock D may be raised in a location other than the farrowing room 5.

[0089] Furthermore, in the above embodiment, the imaging device S may be placed at any of the four corners of the farrowing room 5. However, as long as the imaging device S is capable of capturing images of the livestock D, the placement of the imaging device S is not limited to this example and may be determined appropriately depending on the embodiment. In another example, the imaging device S may be placed on any side of the farrowing room 5. If the livestock D are raised in a place other than the farrowing room 5, the imaging device S may be placed appropriately in a place where it is possible to capture images of the livestock D.

[0090] In the above embodiment, the order of the steps in the process procedure shown in Fig. 5 may be changed as desired as long as there is no dependency. In another example, the process of step S104 may be performed before the process of step S102 or step S103. The process order of steps S102 to S104 may be determined as desired.

[0091] Furthermore, in the above embodiment, the unit of repetition by step S111 need not be limited to the example in Fig. 5. That is, the return destination of step S111 need not be limited to step S105. In another example, the unit of repetition may be steps S102 to S110. That is, the control unit 11 may execute the processes of steps S102 to S110 for each pair of captured images 3. [Explanation of symbols]

[0092] 1...Parturition detection device, 11: control unit, 12: storage unit, 13: communication interface, 14...External interface, 15...input device, 16...output device, 17...drive, 81... Parturition detection program, 91... Storage medium, 111: image acquisition unit; 112: movement calculation unit; 113: movement amount calculation unit; 114: difference pixel number calculation unit; 115: counting unit; 116: judgment unit; 117: output unit; 3…Image capture, 5…Birthing room, S...imaging device, D...livestock

Claims

1. an image acquisition unit that acquires a plurality of captured images generated by continuously capturing images of livestock using an imaging device; a position calculation unit that calculates a center of gravity position of the livestock from each of the acquired captured images; a movement amount calculation unit that calculates a difference between the calculated center positions of the centers of gravity between the captured images as a movement amount; a pixel difference calculation unit for calculating a pixel difference between the captured images; a counting unit that determines whether the difference pixel number calculated between each of the captured images exceeds a first threshold value, and if the difference pixel number exceeds the first threshold value, further determines whether the movement amount of the center of gravity position calculated between each of the captured images exceeds a second threshold value, and counts the difference pixel number as a number of turning pixels when the movement amount of the center of gravity position is less than the second threshold value, and counts the difference pixel number as a number of moving pixels when the movement amount of the center of gravity position exceeds the second threshold value; A determination unit that determines whether or not the livestock has a sign of parturition based on the counted number of turning pixels and the number of moving pixels; Equipped with Parturition detection device.

2. The determining step includes determining whether or not the livestock has a sign of parturition based on whether or not the counted number of turning pixels and the number of moving pixels exceed a threshold value. The delivery detection device of claim 1.

3. The determining step includes using a trained determination model to determine whether the livestock has a sign of parturition from the aggregated number of turning pixels and the number of moving pixels. The delivery detection device of claim 1.

4. The livestock is either a cow or a horse. A delivery detection device according to any one of claims 1 to 3.

5. The imaging device is a thermography camera. A delivery detection device according to any one of claims 1 to 4.

6. The imaging device is installed at a predetermined height in one of the four corners of the birthing room. A delivery detection device according to any one of claims 1 to 5.

7. the imaging device is disposed so that an area in the farrowing pen where the livestock may be present is within an imaging range of the imaging device; There is only one livestock animal in the farrowing pen; 7. The delivery detection device of claim 6.

8. the imaging device is disposed such that a center of an angle of view of the imaging device is located at a center of the farrowing pen, so that an area in the farrowing pen where the livestock may be present is within an imaging range of the imaging device; The delivery detection device of claim 7.

9. The computer acquiring a plurality of captured images generated by continuously capturing images of livestock using an imaging device; calculating a center of gravity position of the livestock from each of the acquired captured images; calculating a difference between the calculated center positions of the centers of gravity between the captured images as a movement amount; calculating a difference in pixel number between the captured images; determining whether the difference in pixel number calculated between each of the captured images exceeds a first threshold; determining whether or not the movement amount of the center of gravity position calculated between the captured images exceeds a second threshold value when the difference pixel number exceeds the first threshold value; When the movement amount of the center of gravity position is less than the second threshold value, counting the difference pixel number as a turning pixel number; counting the difference pixel number as a movement pixel number when the movement amount of the center of gravity position exceeds the second threshold value; A step of determining whether or not the livestock has a sign of parturition based on the counted number of turning pixels and the number of moving pixels; Execute Methods for detecting labor.

10. On the computer, acquiring a plurality of captured images generated by continuously capturing images of livestock using an imaging device; calculating a center of gravity position of the livestock from each of the acquired captured images; calculating a difference between the calculated center positions of the centers of gravity between the captured images as a movement amount; calculating a difference in pixel number between the captured images; determining whether the difference in pixel number calculated between each of the captured images exceeds a first threshold; determining whether or not the movement amount of the center of gravity position calculated between the captured images exceeds a second threshold value when the difference pixel number exceeds the first threshold value; When the movement amount of the center of gravity position is less than the second threshold value, counting the difference pixel number as a turning pixel number; counting the difference pixel number as a movement pixel number when the movement amount of the center of gravity position exceeds the second threshold value; A step of determining whether or not the livestock has a sign of parturition based on the counted number of turning pixels and the number of moving pixels; In order to execute Parturition detection program.

Images