Information processing device, information processing method, and program

Abstract

To more easily determine the position of a fetus in a mother cow's womb. [Solution] The information processing device (1) comprises an acquisition unit (101) that acquires an image of the end of a fetus's limb at the beginning of labor, an estimation unit (102) that inputs the image acquired by the acquisition unit (101) into a learning model that has been trained using learning limb images that are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of labor and a newborn calf within five days after labor, and estimates whether the imaged end of the limb is a forelimb or a hindlimb, and an output unit (103) that outputs the estimation result by the estimation unit (102).

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In bovine births, there are cases where natural birth is difficult (difficult labor), and assistance is required when necessary. Accurately understanding the fetal position (fetal posture) is important in determining whether assistance is required. Fetal posture during bovine birth can be roughly predicted by determining whether the fetal limbs peeking out from the mother cow's genitals after the second rupture of the membranes are the forelimbs or hindlimbs. However, it is difficult to distinguish between the forelimbs and hindlimbs based on the direction and shape of the hooves alone, and the fetal position is generally determined by inserting a hand into the vagina to check the tip of the fetus's nose or by feeling the limb joints.

[0003] As an example, Non-Patent Document 1 discloses a technique for inserting a hand into the birth canal of a mother cow to check the position of the head, limbs, etc. of the fetus. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Mie Prefecture Wagyu Breeding Council et al., Wagyu Breeding and Rearing Manual (1st Edition), June 2020, https: / / mie.lin.gr.jp / gijutu / siyou / manual / all.pdf Summary of the Invention [Problem to be solved by the invention]

[0005] However, even for an experienced veterinarian, it is hard work and difficult to find the fetal joints inside the vagina of a cow in labor due to the intense uterine contractions. Also, at breeding sites, livestock owners sometimes begin pulling the fetus by visually determining the direction of the fetus's hooves alone, but if the position of the fetus is not suitable for delivery, the fetus may become stuck inside the mother's body, putting the lives of both the fetus and the mother at risk.

[0006] Fig. 1 is a diagram showing an example of the position of a single fetus at the time of delivery. As shown in Fig. 1, the position of a fetus at the time of delivery can be, for example, Head-up position: The fetus is exposed from the front legs out of the mother's body, with the soles of the exposed front legs facing downwards. Tail-up position: The fetus is exposed from the hind legs out of the mother's body, with the soles of the exposed hind legs facing upwards. Head-down position: The fetus is exposed from the front legs out of the mother's body, with the soles of the exposed front legs facing upwards. Tail-down position: The fetus is exposed from the hind legs out of the mother's body, with the soles of the exposed hind legs facing downwards. As shown in Figure 1, head-up and tail-up presentations are fetal positions that allow for delivery as is, while head-down and tail-down presentations are fetal positions that do not allow for delivery as is.

[0007] Here, when a cow gives birth, the user can relatively easily determine the direction of the hoof of the exposed limbs of the fetus by visual inspection, etc. However, as mentioned above, it is difficult for the user to determine whether the exposed limbs of the fetus are forelimbs or hind limbs.

[0008] Therefore, if it is possible to determine whether the fetal limbs are forelimbs or hindlimbs when they become visible, it is thought that the fetal position of a single fetus can be predicted, enabling appropriate delivery.

[0009] An object of one aspect of the present invention is to realize a technology that can more easily determine the posture of a single fetus inside a mother cow's womb. [Means for solving the problem]

[0010] In order to solve the above problem, an information processing device according to one embodiment of the present invention comprises an acquisition unit that acquires an image of the end of a fetus's limb at the beginning of labor; an estimation unit that inputs the image acquired by the acquisition unit into a learning model that has been trained using training limb images that are images of the forelimbs and hind limbs of at least one individual fetus after the beginning of labor and a newborn calf within five days after labor, and estimates whether the imaged end of the limb is a forelimb or a hind limb; and an output unit that outputs the estimation result by the estimation unit.

[0011] In order to solve the above problem, an information processing method according to one embodiment of the present invention includes an acquisition process for acquiring an image of the end of a fetal limb at the beginning of labor; an estimation process for inputting the image acquired in the acquisition process into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one individual fetus after the beginning of labor and a newborn calf within five days after labor, to estimate whether the imaged end of the limb is a forelimb or a hind limb; and an output process for outputting the estimation result obtained by the estimation process.

[0012] In order to solve the above problem, a program according to one embodiment of the present invention causes a computer to perform an acquisition process of acquiring an image of the end of a fetal limb at the beginning of labor; an estimation process of inputting the image acquired in the acquisition process into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one individual fetus after the beginning of labor and a newborn calf within five days after labor, to estimate whether the imaged end of the limb is a forelimb or a hind limb; and an output process of outputting the estimation result obtained by the estimation process. [Effects of the Invention]

[0013] According to one aspect of the present invention, the position of a fetus in a mother cow's womb can be more easily determined. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a diagram showing an example of fetal posture at delivery according to the present invention. [Figure 2] 1 is a block diagram showing an example of the configuration of an information processing device according to the present invention; [Figure 3] FIG. 1 is a diagram showing an example of the state of a mother and her calf at the time of delivery of a cow according to the present invention. [Figure 4] 1 is an example image of the end of a limb of a newborn calf according to the present invention. [Figure 5] 1 is an example image of the end of a limb of a newborn calf according to the present invention. [Figure 6] 1 is an example image of the end of a limb of a newborn calf according to the present invention. [Figure 7] FIG. 1 is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 8] FIG. 1 is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 9] 1 is a block diagram showing an example of the configuration of an information processing device according to the present invention; [Figure 10] FIG. 1 is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 11] FIG. 1 is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 12] FIG. 10 is a diagram showing the number of images showing the end of a limb used in an example of the present invention. [Figure 13] FIG. 10 is a diagram showing a processing result in an embodiment according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] [Embodiment 1] Hereinafter, one embodiment of the present invention will be described in detail.

[0016] (Overview of information processing device 1) The information processing device 1 is a device for estimating, using a learning model, whether the end of a fetal limb (limb tip) imaged at the beginning of labor is a forelimb or a hindlimb. Note that in this embodiment, the direction of the hoof of the fetal limb may be determined by, for example, visual inspection by a user.

[0017] An example of the information processing device 1 is a server. In this case, the information processing device 1 may be connected to, for example, a terminal owned by a user via a network N described below. Here, the terminal owned by the user may have, for example, an imaging function. Specific examples of the terminal owned by the user include a digital camera, a smartphone, and a tablet computer. Each process in the information processing device 1 may be executed, for example, from a browser running on the terminal owned by the user, or may be executed from an application on a server via an API (Application Programming Interface).

[0018] Note that each component of the information processing device 1 may be provided entirely by the server as described above, or may be provided partially between the server and the user's terminal connected via a network N, etc. Also, as an example, each component of the information processing device 1 may be provided entirely by the user's terminal.

[0019] (Configuration of information processing device 1) The configuration of the information processing device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example configuration of the information processing device 1. As shown in Fig. 2, the information processing device 1 includes a control unit 10, a storage unit 20, a communication unit 30, and an input / output unit 40.

[0020] (Control unit 10) The control unit 10 comprehensively controls each unit of the information processing device 1. The control unit 10 includes, for example, an acquisition unit 101, an estimation unit 102, an output unit 103, a limb image acquisition unit 104, and a learning unit 105, as shown in FIG.

[0021] (Acquisition part 101) The acquisition unit 101 acquires images of the ends of the limbs of the fetus at the beginning of delivery.

[0022] The processing target of the information processing device 1 according to this embodiment may be, for example, all breeds of cattle. The breeds of cattle that may be processed by the information processing device 1 may be, for example, any of beef cattle, dairy cattle, working cattle, and ornamental / pet cattle. Specific examples of the breeds of cattle include, but are not limited to, Holstein, Japanese Black, other local breeds, and crossbreeds. Furthermore, the mother cow and fetus that may be processed by the information processing device 1 may be of the same breed or different breeds.

[0023] Figure 3 is a diagram showing an example of the state of a mother and her fetus during parturition. As shown in image PREG in Figure 3, the "fetus at the beginning of parturition" may be from the time when the tips of the fetus's limbs become visible from the mother cow's genitals during parturition until the time when the fetus's head or tail becomes visible.

[0024] As one example, the end of a fetal limb may be a region including the tip of the limb to the fetlock. That is, in this case, the end of a fetal limb includes the hoof at the tip of the limb. Also, as another example, the end of a fetal limb may be a region including the tip of the limb to the hoof. Here, for example, if only a portion of the region included in the end of a fetal limb is exposed outside the mother's body, the user may pull the fetal limb out of the mother's body with the user's hand or the like until the entire region is exposed, and then image the end of the fetal limb.

[0025] The acquiring unit 101 may acquire, for example, an image of the end of a limb of a fetus at the beginning of labor via a terminal having an imaging function. Specific examples of terminals having an imaging function include a digital camera, a smartphone, and a tablet computer. The acquiring unit 101 may acquire, for example, an image captured via the terminal having an imaging function via a network N connected to the information processing device 1. Here, the terminal having an imaging function may be, for example, the terminal owned by the user.

[0026] (Example image of the end of a calf's limb) Figures 4 to 6 are example images of the end of a leg of a newborn calf within five days after calving. Image ORI in Figure 4 is an example of an image captured including both the end of a leg of a newborn calf within five days after calving and objects other than the end of the leg. Image TR1 in Figure 5 is an example of an image showing the range from the tip of the leg of a newborn calf within five days after calving to roughly the fetlock. Image TR2 in Figure 6 is an example of an image showing the range from the tip of the leg of a newborn calf within five days after calving to roughly the hoof. In each of the images in Figures 4 to 6, a transparent plate with a grid of the same size is placed over the hoof.

[0027] For example, the acquiring unit 101 may acquire an image capturing both the end of a fetal limb and an object other than the end of the limb, as shown in image ORI in Fig. 4. Alternatively, for example, the acquiring unit 101 may acquire an image showing a range from the tip of a fetal limb to roughly the fetlock, as shown in image TR1 in Fig. 5. Alternatively, for example, the acquiring unit 101 may acquire an image showing a range from the tip of a fetal limb to roughly the hoof, as shown in image TR2 in Fig. 6.

[0028] As an example, the acquisition unit 101 may acquire an image that has been cropped by the user so that the proportion of the area of ​​the image showing the ends of the fetus's limbs is larger, as shown in image TR1 of FIG. 5 or image TR2 of FIG. 6, in an image captured including the ends of the fetus's limbs and objects other than the ends of the limbs, as shown in image ORI of FIG. 4.

[0029] (Estimation section 102) The estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model trained using training limb images, which are images of the forelimbs and hindlimbs of at least one individual of a fetus at or after the start of calving and a newborn calf within five days after calving, and estimates whether the end of the captured limb is a forelimb or a hindlimb. The training limb images are image data including one or more pairs of "images of the forelimbs of at least one individual of a fetus at or after the start of calving and a newborn calf within five days after calving, and a label indicating that the image is of a forelimb," and "pairs of images of the hindlimbs of at least one individual of a fetus at or after the start of calving and a newborn calf within five days after calving, and a label indicating that the image is of a hindlimb." For example, the fetus at or after the start of calving and the newborn calf within five days after calving shown in the training limb images may be different from the fetus at the start of calving shown in the image acquired by the acquisition unit 101.

[0030] Here, the individual shown in the learning limb image may be, for example, as described above, at least one of a fetus after the start of parturition and a newborn calf within five days after delivery. That is, as a specific example, the individual shown in the learning limb image may be: It may be a fetus at the onset of labor, or It may be a fetus at a stage after the onset of labor, or - It may be a newborn calf within 5 days of calving, or The animal may be a fetus after the start of parturition and / or a newborn calf within 5 days of parturition, or - It may be any combination of the above types of individuals.

[0031] The newborn calf shown in the learning limb image may be, for example, a newborn calf born within five days after delivery that is stillborn or has died.

[0032] The learning model is a machine learning model for estimating whether the end of a fetal limb imaged at the beginning of labor is a forelimb or a hindlimb. Here, the learning model is a machine learning model that has been trained in advance by machine learning using training limb images. That is, the training limb images may include, for example, training data, validation data, and test data for the learning model. In this case, the training limb images may include, for example, "training data, validation data, and test data for the forelimbs" and "training data, validation data, and test data for the hindlimbs."

[0033] The learning model may also be a machine learning model that has been deep-learned using training limb images. Specific examples of deep learning frameworks include PyTorch. Specific examples of learning models include VGG (Visual Geometry Group) Net, ResNet, AlexNet, DenseNet, SqueezeNet, SwinTransformer, and VisionTransformer.

[0034] The estimation unit 102 according to this embodiment inputs the image acquired by the acquisition unit 101 into a learning model that has been trained in advance using training limb images, and estimates whether the end of the limb of the fetus photographed at the beginning of labor is a forelimb or a hind limb.

[0035] Here, as an example, the learning model may be learned in advance by a learning unit 105 described later. Also, as another example, the learning model may be learned in advance by a device external to the information processing device 1.

[0036] The estimation result by the estimation unit 102 may include, for example, one or more pairs of an image showing the end of the limb of the fetus that was the subject of estimation processing by the estimation unit 102 and information indicating whether the end of the limb corresponds to a "front limb" or a "hind limb."

[0037] (output unit 103) The output unit 103 outputs the estimation result from the estimation unit 102.

[0038] With the above-described configuration, it is possible to use a machine learning model to more accurately determine whether the end of a fetal limb at the beginning of labor is a forelimb or a hind limb.

[0039] (limb image acquisition unit 104) The limb image acquisition unit 104 acquires learning limb images.

[0040] (Learning Section 105) Learning unit 105 trains a learning model using the training limb images acquired by limb image acquisition unit 104. For example, the processes in limb image acquisition unit 104 and learning unit 105 may be executed in advance of the processes in estimation unit 102. That is, estimation unit 102 may, for example, input the images acquired by acquisition unit 101 into the learning model trained by learning unit 105, and estimate whether the end of the captured limb is a forelimb or a hind limb.

[0041] With the above-described configuration, it is possible to train a machine learning model for estimating whether the end of a fetal limb at the beginning of labor is a forelimb or a hind limb.

[0042] (Storage unit 20) The storage unit 20 stores various data referenced by the control unit 10 and various data generated by the control unit 10. Specific examples of data stored in the storage unit 20 include: ·Acquired image data ORI ·Learning image data TD Learning model LM Estimation result ER ·Extremity site information DE Examples include:

[0043] The acquired image data ORI is image data acquired by the acquiring unit 101, and is an image of the end of a fetal limb at the beginning of labor. The acquired image data ORI may be, for example, an image of only the end of a fetal limb at the beginning of labor, or may be an image of at least the end of a fetal limb at the beginning of labor. The image ORI in FIG. 3 is an example of the acquired image data ORI.

[0044] The training limb image data TD is image data that includes one or more of the following: "a pair of an image showing the forelimbs of at least one individual of a fetus after the start of labor and a newborn calf within five days after labor, and a label indicating that the image is of a forelimb," and "a pair of an image showing the hindlimbs of at least one individual of a fetus after the start of labor and a newborn calf within five days after labor, and a label indicating that the image is of a hindlimb." For example, the fetus after the start of labor and the newborn calf within five days after labor shown in the training limb image data TD may be a different individual from the fetus at the start of labor shown in the acquired image data ORI.

[0045] Here, the individual shown in the learning limb image data TD may be, for example, as described above, at least one of a fetus after the start of parturition and a newborn calf within five days after delivery. That is, as a specific example, the individual shown in the learning limb image data TD may be: It may be a fetus at the onset of labor, or It may be a fetus at a stage after the onset of labor, or - It may be a newborn calf within 5 days of calving, or The animal may be a fetus after the start of parturition and / or a newborn calf within 5 days of parturition, or - It may be any combination of the above types of individuals.

[0046] The newborn calf shown in the learning limb image data TD may be, for example, a stillborn or dead calf born within five days after delivery.

[0047] The learning model LM is a machine learning model for estimating whether the end of a fetal limb imaged at the beginning of labor is a forelimb or a hindlimb. Here, the learning model LM is a machine learning model that has been trained in advance by machine learning using training limb image data TD. That is, the training limb image data TD may include, for example, training data, verification data, and test data for the learning model LM. In this case, the training limb image data TD may include, for example, "training data, verification data, and test data for the forelimbs" and "training data, verification data, and test data for the hindlimbs."

[0048] The learning model LM may also be a machine learning model that has undergone deep learning using the training limb image data TD. Specific examples of deep learning frameworks include PyTorch. Specific examples of the learning model LM include VGG (Visual Geometry Group) Net, ResNet, AlexNet, DenseNet, SqueezeNet, SwinTransformer, and VisionTransformer.

[0049] The estimation result ER is a result of estimation by the estimation unit 102 as to whether the end of the imaged fetal limb is a forelimb or a hindlimb. The estimation result ER may include, for example, one or more pairs of an image showing the end of the fetal limb that was the subject of estimation processing by the estimation unit 102 and information indicating whether the end of the limb corresponds to a "forelimb" or a "hindlimb."

[0050] The limb end position information DE is information indicating a part included in the limb end of a fetus. As an example, the limb end of a fetus may be a part including the tip of the limb to the fetlock. In other words, in this case, the limb end of a fetus includes the hoof located at the limb end. Also, as an example, the limb end of a fetus may be a part including the tip of the limb to the hoof.

[0051] (Communication unit 30) The communication unit 30 communicates with devices external to the information processing device 1. As an example, the communication unit 30 communicates with each external device connected to the information processing device 1 via a network N (not shown). The communication unit 30 transmits data supplied from the control unit 10 to the outside, and supplies data received from each external device to the control unit 10. Note that the specific configuration of the network N does not limit the present exemplary embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0052] (Input / output section 40) The input / output unit 40 is configured to include at least one of input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel. Alternatively, the input / output unit 40 may be configured to have input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel connected to it. In this configuration, the input / output unit 40 accepts various types of information input to the information processing device 1 from the connected input devices. Furthermore, the input / output unit 40 outputs various types of information to connected output devices under the control of the control unit 10. An example of the input / output unit 40 is an interface such as a USB (Universal Serial Bus).

[0053] (Flow of information processing method S1) The flow of the information processing method S1 executed by the information processing device 1 will be described with reference to Fig. 7. Fig. 7 is a flow diagram showing an example of the flow of the information processing method S1. For example, as shown in Fig. 7, the information processing method S1 includes an acquisition process (step) S11, an estimation process (step) S12, and an output process (step) S13.

[0054] (Step S11) In step S11, the acquisition unit 101 acquires an image of the end of the limb of the fetus at the beginning of delivery.

[0055] (Step S12) In step S12, the estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one of a fetus after the start of delivery and a newborn calf within five days of delivery, and estimates whether the end of the imaged limb is a forelimb or a hind limb.

[0056] (Step S13) In step S13, the output unit 103 outputs the estimation result obtained by the estimation unit 102.

[0057] (Flow of information processing method S2) The flow of the information processing method S2 executed by the information processing device 1 will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing an example of the flow of the information processing method S2. For example, as shown in Fig. 8, the information processing method S2 includes a learning limb image acquisition process (step) S21 and a learning process (step) S22.

[0058] (Step S21) In step S21, the limb image acquisition unit 104 acquires learning limb images that are images showing the forelimbs and hind limbs of at least one of a fetus after the start of delivery and a newborn calf within five days after delivery.

[0059] (Step S22) In step S22, the learning unit 105 uses the training limb images acquired by the limb image acquisition unit 104 to train a learning model.

[0060] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0061] (Overview of information processing device 1A) The information processing device 1A is a device that has the same configuration as the information processing device 1 according to embodiment 1, and further has a configuration for trimming an image captured including the ends of the limbs of the fetus and objects other than the ends of the limbs so that the proportion of the area of ​​the image showing the ends of the limbs of the fetus in the image is increased.

[0062] (Configuration of information processing device 1A) The configuration of the information processing device 1A will be described with reference to Fig. 9. Fig. 9 is a block diagram showing an example configuration of the information processing device 1A. As shown in Fig. 9, the information processing device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30, and an input / output unit 40. The communication unit 30 and the input / output unit 40 have the same configurations as the components of the same names in the information processing device 1, and therefore descriptions thereof will be omitted here.

[0063] (Control unit 10A) Control unit 10A controls all the units of information processing device 1A. As shown in Fig. 9, control unit 10A includes, for example, acquisition unit 101, estimation unit 102, output unit 103, limb image acquisition unit 104, learning unit 105, determination unit 106, range estimation unit 107, range output unit 108, extremity image acquisition unit 109, and range learning unit 110. Acquisition unit 101, limb image acquisition unit 104, and learning unit 105 have the same configurations as the same components in information processing device 1, and therefore will not be described here.

[0064] (Estimation section 102) The estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one of a fetus after the start of delivery and a newborn calf within five days of delivery, and estimates whether the end of the imaged limb is a forelimb or a hind limb.

[0065] The training limb image may further indicate, for example, the orientation of the hoof of the leg. In this case, the estimation unit 102 may, for example, input the image acquired by the acquisition unit 101 into a learning model trained using the training limb image to further estimate the orientation of the hoof of the captured limb. The orientation of the hoof of the fetus's limb is indicated by whether the orientation of the sole of the fetus's limb is generally upward or downward in the vertical direction when the mother cow is assumed to be standing upright, as shown in image PREG in FIG. 3, for example. In this case, the training limb image may be image data including, for example, one or more pairs of "images showing the soles of the hoofs facing upward on the limbs of at least one individual fetus after the start of calving and a newborn calf within five days after calving, and a label indicating that the image shows the sole of the hoof facing upward" and "pairs of images showing the soles of the hoofs facing downward on the limbs of at least one individual fetus after the start of calving and a newborn calf within five days after calving, and a label indicating that the image shows the sole of the hoof facing downward."

[0066] The estimation unit 102 according to this embodiment may, for example, input the image acquired by the acquisition unit 101 into a learning model trained using training limb images, which are images that further indicate the orientation of the hooves of the legs of at least one of a fetus after the beginning of parturition and a newborn calf within five days after delivery, and further estimate the orientation of the hooves of the legs of the fetus photographed at the beginning of parturition.

[0067] With the above-described configuration, the orientation of the hooves of the fetus's limbs in an image taken at the beginning of labor can be determined using a machine learning model.

[0068] The other components of the estimation unit 102 are the same as those of the estimation unit 102 in the information processing device 1, and therefore will not be described here.

[0069] (Judgment unit 106) The determination unit 106 determines whether or not assistance is available for delivery of the fetus based on the estimation result by the estimation unit 102 and hoof orientation information indicating the orientation of the hoof of the imaged leg. The hoof orientation information indicates whether the hoof of the leg of the fetus is facing upward or downward. The hoof orientation information may be estimated by the estimation unit 102, or may be obtained by the information processing device 1 based on the result of a user's visual inspection or other judgment.

[0070] The determination unit 106 may determine which of the four types of fetal position the fetus falls into, for example, based on the result of estimation by the estimation unit 102 as to whether the end of the leg is a forelimb or a hind leg, and on hoof orientation information indicating whether the hoof of the imaged leg faces upward or downward. The determination unit 106 may then determine whether delivery is possible based on the fetal position shown in the image acquired by the acquisition unit 101. That is, the determination unit 106 may determine, for example, whether the fetal position of the fetus is suitable for delivery as is, or whether delivery is not possible as is.

[0071] Here, if the position of the fetus is such that it can be delivered as is, the user can assist in the delivery. On the other hand, if the position of the fetus is such that it cannot be delivered as is, the user cannot assist in the delivery unless the user is a veterinarian, etc. That is, for example, as described above, the determination unit 106 may determine whether delivery is possible based on the position of the fetus, and then determine whether a user (other than a veterinarian, etc.) can assist in the delivery of the fetus.

[0072] (output unit 103) The output unit 103 further outputs the determination result by the determination unit 106. The output unit 103 may output, for example, the estimation result by the estimation unit 102 and the determination result by the determination unit 106 together.

[0073] With the above-described configuration, the user can understand whether or not assistance is needed when the fetus is delivered from the captured image showing the limbs of the fetus at the beginning of delivery.

[0074] (Range estimation unit 107) The range estimation unit 107 inputs the image acquired by the acquisition unit 101 into a range estimation learning model that has been trained using training limb end images, which are images showing the end of a limb in at least one individual of a fetus after the start of calving and a newborn calf within five days after calving, and estimates the range showing the end of a limb in the acquired image. The training limb end images are image data that include one or more pairs of "images showing the end of a limb in at least one individual of a fetus after the start of calving and a newborn calf within five days after calving, and labels indicating that the images are of the end of a limb." For example, the fetus after the start of calving or the newborn calf within five days after calving shown in the training limb end image may be a different individual from the fetus at the start of calving shown in the image acquired by the acquisition unit 101.

[0075] Here, the individual shown in the learning extremity image may be, for example, as described above, at least one of a fetus after the start of parturition and a newborn calf within five days after delivery. That is, as a specific example, the individual shown in the learning extremity image may be: It may be a fetus at the onset of labor, or It may be a fetus at a stage after the onset of labor, or - It may be a newborn calf within 5 days of calving, or The animal may be a fetus after the start of parturition and / or a newborn calf within 5 days of parturition, or - It may be any combination of the above types of individuals.

[0076] The newborn calf shown in the learning limb image may be, for example, a newborn calf born within five days after delivery that has been stillborn or has died.

[0077] The range estimation learning model is a machine learning model for estimating the range indicating the end of a limb in an image captured including the end of a fetus' limb at the beginning of labor. Here, the range estimation learning model is a machine learning model that has been trained in advance using training images of the limb. That is, the training images of the limb may include, for example, training data, validation data, and test data for the range estimation learning model. A specific example of a range estimation learning model is YOLO.

[0078] Here, as an example, the range estimation learning model may be learned in advance by the range learning unit 110 described later. Also, as another example, the range estimation learning model may be learned in advance by a device external to the information processing device 1.

[0079] Furthermore, the range indicating the end of the limb estimated by range estimation section 107 may be, for example, the range of the image indicating the end of the limb.

[0080] (Range output unit 108) Range output unit 108 outputs an image corresponding to the range indicating the end of the limb based on the estimation result by range estimation unit 107. Range output unit 108 may, for example, trim the image acquired by acquisition unit 101 so that the area proportion of the range indicating the end of the limb estimated by range estimation unit 107, i.e., the range estimated by range estimation unit 107, becomes larger, and output the trimmed image.

[0081] As one example, when the range estimated by the range estimation unit 107 is a range that includes the tip of the leg to the fetlock, the range output unit 108 may trim the range that includes the tip of the leg to the fetlock, as shown in image TR1 in Fig. 5. Also, as another example, when the range estimated by the range estimation unit 107 is a region that includes the tip of the leg to the hoof, the range output unit 108 may trim the range that includes the tip of the leg to the hoof, as shown in image TR2 in Fig. 6.

[0082] Furthermore, as described above, when the range output unit 108 outputs an image corresponding to a range indicating the end of a limb in the image acquired by the acquisition unit 101, the estimation unit 102 may, for example, input the image output by the range output unit 108 into a learning model and estimate whether the end of the captured limb is a forelimb or a hind limb.

[0083] The above configuration can improve the accuracy of the estimation process for the end of the limb that has been imaged.

[0084] (Limb end image acquisition unit 109) The extremity image acquisition unit 109 acquires learning extremity images.

[0085] (Range learning unit 110) Range learning unit 110 trains a range estimation learning model using the training extremity images acquired by extremity image acquisition unit 109. For example, each process in extremity image acquisition unit 109 and range learning unit 110 may be executed in advance when processing in range estimation unit 107 is performed. That is, range estimation unit 107 may input the image acquired by acquisition unit 101 to the range estimation learning model trained by range learning unit 110, and estimate the range indicating the extremity of the extremity in the image acquired by acquisition unit 101.

[0086] With the above-described configuration, it is possible to train a machine learning model for estimating the area showing the end of a limb in an image that includes an image showing the end of a limb.

[0087] (Storage unit 20A) The storage unit 20A stores various data referenced by the control unit 10A and various data generated by the control unit 10A. Specific examples of data stored in the storage unit 20A include: ·Acquired image data ORI ·Learning image data TD Learning model LM Estimation result ER Hoof direction information DI ·Judgment result RE -Teaching limb image data TDT Range estimation learning model LMT ·Trimmed image data TR ·Extremity site information DE Examples include:

[0088] The acquired image data ORI, learning limb image data TD, learning model LM, and limb extremity position information DE have the same configuration as the configurations of the same names in the memory unit 20 provided in the information processing device 1, so explanations will be omitted here.

[0089] The estimation result ER is a result of estimation by the estimation unit 102 as to whether the end of the limb of the imaged fetus is a forelimb or a hindlimb. The estimation result ER may include, for example, one or more pairs of an image showing the end of the limb that was the subject of estimation processing by the estimation unit 102 and information indicating whether the end of the limb corresponds to a "forelimb" or a "hindlimb."

[0090] The hoof orientation information DI indicates whether the hoof orientation of the fetus's legs is upward or downward. The hoof orientation information DI may be estimated by the estimation unit 102, or may be obtained by the information processing device 1 based on a user's visual or other judgment.

[0091] The determination result RE is a result of the determination unit 106 determining whether assistance is available when the fetus is delivered based on the estimation result RE and the hoof direction information DI. The determination result RE may indicate, for example, whether assistance is available from a user (other than a veterinarian or the like) when the photographed fetus is delivered.

[0092] The learning limb end image data TDT is image data that includes one or more pairs of "images showing the end of a limb of at least one individual of a fetus after the start of calving and a newborn calf within five days after calving, and a label indicating that the image is of the end of a limb." For example, the fetus after the start of calving or the newborn calf within five days after calving shown in the learning limb end image data TDT may be a different individual from the fetus at the start of calving shown in the acquired image data ORI.

[0093] Here, the individual shown in the learning extremity image data TDT may be, for example, as described above, at least one of a fetus after the start of parturition and a newborn calf within five days after delivery. That is, as a specific example, the individual shown in the learning extremity image data TDT is It may be a fetus at the onset of labor, or It may be a fetus at a stage after the onset of labor, or - It may be a newborn calf within 5 days of calving, or The animal may be a fetus after the start of parturition and / or a newborn calf within 5 days of parturition, or - It may be any combination of the above types of individuals.

[0094] The newborn calf shown in the training extremity image data TDT may be, for example, a stillborn or dead calf born within five days after delivery.

[0095] The range estimation learning model LMT is a machine learning model for estimating the range indicating the end of a limb in an image captured including the end of a fetus' limb at the beginning of labor. Here, the range estimation learning model LMT is a machine learning model that has been trained in advance using training limb image data TDT. That is, the training limb image data TDT may include, for example, training data, validation data, and test data for the range estimation learning model LMT. A specific example of the range estimation learning model LMT is YOLO.

[0096] The trimmed image data TR is an image obtained by trimming a range showing the end of the limb in the acquired image data ORI based on the estimation result by the range estimation unit 107. Here, the range showing the end of the limb may be, for example, a range of an image showing the end of the limb, or may be a range including the image showing the end of the limb and its vicinity. Image TR1 in Figure 4 and image TR2 in Figure 5 are each examples of trimmed image data TR.

[0097] (Information processing method executed by information processing device 1A) The information processing device 1A executes information processing methods S1 and S2 in the same manner as the information processing device 1. Furthermore, the information processing device 1A may execute, for example, information processing methods S3 and S4, which will be described later.

[0098] (Flow of information processing method S3) The flow of information processing method S3 executed by information processing device 1A will be described with reference to Fig. 10. Fig. 10 is a flow diagram showing an example of the flow of information processing method S3. For example, as shown in Fig. 10, information processing method S3 includes an acquisition process (step) S31, a range estimation process (step) S32, a range output process (step) S33, an estimation process (step) S34, and an output process (step) S35.

[0099] (Step S31) In step S31, the acquisition unit 101 acquires an image of the end of a limb of a fetus at the beginning of delivery.

[0100] (Step S32) In step S32, the range estimation unit 107 inputs the image acquired by the acquisition unit 101 into a range estimation learning model that has been trained using training limb end images, which are images showing the ends of the limbs of at least one of a fetus after the start of delivery and a newborn calf within five days of delivery, and estimates the range showing the ends of the limbs in the acquired image.

[0101] (Step S33) In step S33, the range output unit 108 outputs an image corresponding to the range indicating the end of the limb based on the estimation result by the range estimation unit 107.

[0102] (Step S34) In step S34, the estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model trained using training limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the start of delivery and a newborn calf within five days after delivery, and estimates whether the end of the imaged limb is a forelimb or a hindlimb. Furthermore, when the range output unit 108 outputs an image corresponding to a range showing the end of a limb, the estimation unit 102 may, for example, input the image output by the range output unit 108 into the learning model and estimate whether the end of the imaged limb is a forelimb or a hindlimb.

[0103] (Step S35) In step S35, the determining unit 106 determines whether or not assistance is available for delivery of the fetus based on the estimation result by the estimating unit 102 and the hoof orientation information indicating the orientation of the hoof of the imaged leg.

[0104] (Step S36) In step S 36 , the output unit 103 outputs the determination result from the determination unit 106 .

[0105] (Flow of information processing method S4) The flow of the information processing method S4 executed by the information processing device 1A will be described with reference to Fig. 11. Fig. 11 is a flow diagram showing an example of the flow of the information processing method S4. For example, as shown in Fig. 11, the information processing method S4 includes a learning extremity image acquisition process (step) S41 and a range learning process (step) S42.

[0106] (Step S41) In step S41, the limb image acquisition unit 109 acquires a learning limb image, which is an image showing the end of a limb of at least one of a fetus after the start of delivery and a newborn calf within five days after delivery.

[0107] (Step S42) In step S42, the range learning unit 110 uses the learning extremity images acquired by the extremity image acquisition unit 109 to train a range estimation learning model.

[0108] [Software implementation example] The functions of the information processing device 1, 1A (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 10, 10A).

[0109] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0110] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0111] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0112] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0113] 〔summary〕 An information processing device according to aspect 1 of the present invention comprises an acquisition unit that acquires an image of the end of a limb of a fetus at the beginning of labor; an estimation unit that inputs the image acquired by the acquisition unit into a learning model that has been trained using training limb images that are images showing the forelimbs and hind limbs of at least one of a fetus after the beginning of labor and a newborn calf within five days of birth, and estimates whether the imaged end of the limb is a forelimb or a hind limb; and an output unit that outputs the estimation result by the estimation unit.

[0114] The information processing device of aspect 2 of the present invention, in accordance with aspect 1 above, further comprises a limb image acquisition unit that acquires the training limb images, and a learning unit that trains the learning model using the training limb images acquired by the limb image acquisition unit.

[0115] The information processing device according to aspect 3 of the present invention, in the above aspect 1 or 2, further includes a judgment unit that judges whether or not assistance is available when the fetus is delivered based on the estimation result by the estimation unit and hoof orientation information indicating the orientation of the hoof of the imaged limb, and the output unit further outputs the judgment result by the judgment unit.

[0116] In an information processing device according to aspect 4 of the present invention, in the above-mentioned aspect 3, the training limb image further indicates the direction of the hoof of the limb, and the estimation unit inputs the image acquired by the acquisition unit into the learning model trained using the training limb image, thereby further estimating the direction of the hoof of the imaged limb.

[0117] An information processing device according to aspect 5 of the present invention, in any of aspects 1 to 4 above, further comprises a range estimation unit that inputs the image acquired by the acquisition unit into a range estimation learning model trained using learning limb end images, which are images showing the ends of the limbs of at least one individual, namely a fetus after the start of delivery and a newborn calf within 5 days after delivery, to estimate the range showing the end of the limb in the acquired image, and a range output unit that outputs an image corresponding to the range showing the end of the limb based on the estimation result by the range estimation unit, and the estimation unit inputs the image output by the range output unit into the learning model to estimate whether the imaged end of the limb is a forelimb or a hind limb.

[0118] The information processing device of aspect 6 of the present invention, in accordance with aspect 5 above, further includes an extremity image acquisition unit that acquires the learning extremity images, and a range learning unit that uses the learning extremity images acquired by the extremity image acquisition unit to train the range estimation learning model.

[0119] In the information processing device according to aspect 7 of the present invention, in any one of aspects 1 to 6, the end of the limb is a region including the tip of the limb to the fetlock.

[0120] The information processing method according to aspect 8 of the present invention includes an acquisition process for acquiring an image of the end of a fetal limb at the beginning of labor; an estimation process for inputting the image acquired in the acquisition process into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one individual fetus after the beginning of labor and a newborn calf within five days after labor, to estimate whether the imaged end of the limb is a forelimb or a hind limb; and an output process for outputting the estimation result obtained by the estimation process.

[0121] The program according to aspect 9 of the present invention causes a computer to perform an acquisition process of acquiring an image of the end of a fetus's limb at the beginning of labor; an estimation process of inputting the image acquired in the acquisition process into a learning model trained using training limb images, which are images showing the forelimbs and hind limbs of at least one of a fetus after the beginning of labor and a newborn calf within five days after labor, to estimate whether the imaged end of the limb is a forelimb or a hind limb; and an output process of outputting the estimation result obtained by the estimation process.

[0122] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Example]

[0123] An embodiment of the present invention will be described below.

[0124] In this example, the information processing device 1 was used to estimate whether an image showing the end of a fetal limb at the beginning of labor is a forelimb or a hindlimb.

[0125] FIG. 12 is a diagram showing the number of images showing the end of the limbs of a fetus and a newborn calf within 5 days after delivery used in this example. FIG. 12 shows the number of images used for learning and verification of the information processing device 1. In this example, as shown in FIG. 12, the images were classified into "learning data, verification data, and test data for the forelimbs" and "learning data, verification data, and test data for the hindlimbs." In this example, the above images were used to perform learning and verification processes in the information processing device 1.

[0126] FIG. 13 is a diagram showing the processing results of the information processing device 1 in this embodiment. An "original" image (corresponding to image ORI in Figure 4) of a newborn calf taken within 5 days of delivery, including both the extremities and non-extremities. - An image of "Trimming 1 (Leg)" of a newborn calf within 5 days after delivery, cropped from the tip of the leg to roughly the fetlock (corresponding to image TR1 in Figure 5), and "Trimming 2 (hoof)" image of a newborn calf within 5 days after delivery, trimmed from the tip of the leg to roughly the hoof (corresponding to image TR2 in Figure 6) For each of the above, it was estimated whether it was a forelimb or a hindlimb. In this example, estimation was performed for each of several types of learning models, as shown in FIG.

[0127] 13 shows the determination accuracy calculated from the estimation results by the information processing device 1. Here, the determination accuracy is an index showing how accurately it was determined whether the fetal limb corresponds to a forelimb or a hindlimb. In this example, two types of determination accuracy values ​​were calculated, namely, accuracy and precision, as shown in FIG.

[0128] As a result of processing by the information processing device 1 in this example, the accuracy rate, which indicates the proportion of all processing results that were correctly estimated as either a front leg or a hind leg, exceeded 0.9 for the "original" image, as shown in Fig. 13. Also, as shown in Fig. 13, the accuracy rate exceeded 0.8 for the "trimming 1 (leg)" image. [Explanation of symbols]

[0129] 1, 1A Information processing equipment 10, 10A control section 20, 20A storage section 30 Communications Department 40 Input / output section 101 Acquisition Department 102 Estimation part 103 Output section 104 Limb Image Acquisition Unit 105 Learning Department 106 Judgment section 107 Range Estimation Unit 108 Range output section 109 Limb end image acquisition unit 110 Range Learning Section

Claims

1. an acquisition unit that acquires images of the ends of the limbs of the fetus at the beginning of labor; an estimation unit that inputs the image acquired by the acquisition unit into a learning model that has been trained using training limb images that are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the start of delivery and a newborn calf within five days after delivery, and estimates whether the imaged end of the limb is a forelimb or a hindlimb; an output unit that outputs an estimation result by the estimation unit, The beginning of parturition is the period from when the tip of the fetus's limbs can be visually confirmed from the genitals of the mother cow during parturition to when the head or tail of the fetus can be visually confirmed, The end of the leg is a part including the tip of the leg to the fetlock, or a part including the tip of the leg to the hoof, Information processing device.

2. a limb image acquisition unit that acquires the learning limb image; a learning unit that learns the learning model using the learning limb images acquired by the limb image acquisition unit, The information processing device according to claim 1 .

3. a determination unit that determines whether or not assistance is required when the fetus is delivered based on the estimation result by the estimation unit and hoof orientation information indicating the orientation of the hoof of the captured image of the leg, The output unit further outputs the determination result by the determination unit. The information processing device according to claim 1 .

4. The training limb image further indicates the orientation of the hoof of the limb; the estimation unit inputs the image acquired by the acquisition unit into the learning model trained using the training limb image, and further estimates the orientation of the hoof of the captured limb. The information processing device according to claim 3 .

5. a range estimation unit that inputs the image acquired by the acquisition unit into a range estimation learning model that has been trained using training images of the extremities, which are images showing the extremities of the extremities of at least one individual of a fetus after the start of delivery and a newborn calf within five days after delivery, and estimates a range showing the extremities of the extremities in the acquired image; a range output unit that outputs an image corresponding to a range indicating an end of the limb based on an estimation result by the range estimation unit, the estimation unit inputs the image output by the range output unit into the learning model and estimates whether the captured end of the limb is a forelimb or a hindlimb. The information processing device according to claim 1 .

6. an extremity image acquisition unit that acquires the learning extremity image; a range learning unit that uses the learning extremity image acquired by the extremity image acquisition unit to learn the range estimation learning model, The information processing device according to claim 5 .

7. an acquisition process for acquiring images of the ends of the fetal limbs at the beginning of labor; an estimation process in which the image acquired in the acquisition process is input into a learning model trained using training limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the start of delivery and a newborn calf within five days after delivery, and the imaged end of the limb is estimated to be either a forelimb or a hindlimb; an output process for outputting an estimation result obtained by the estimation process; The beginning of parturition is the period from when the tip of the fetus's limbs can be visually confirmed from the genitals of the mother cow during parturition to when the head or tail of the fetus can be visually confirmed, The end of the leg is a part including the tip of the leg to the fetlock, or a part including the tip of the leg to the hoof, Information processing methods.

8. On the computer, an acquisition process for acquiring images of the ends of the fetal limbs at the beginning of labor; an estimation process in which the image acquired in the acquisition process is input into a learning model trained using training limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the start of delivery and a newborn calf within five days after delivery, and the imaged end of the limb is estimated to be either a forelimb or a hindlimb; an output process for outputting an estimation result obtained by the estimation process; The beginning of parturition is the period from when the tip of the fetus's limbs can be visually confirmed from the genitals of the mother cow during parturition to when the head or tail of the fetus can be visually confirmed, The end of the leg is a part including the tip of the leg to the fetlock, or a part including the tip of the leg to the hoof, program.

Images