Information processing device, information processing method, and program

Abstract

This makes it easier to determine the fetal position of a fetus within the mother cow's womb. [Solution] The information processing device (1) includes an acquisition unit (101) that acquires images of the ends of the limbs of a fetus at the beginning of delivery, an estimation unit (102) that inputs the images acquired by the acquisition unit (101) into a learning model that has been trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs, and an output unit (103) that outputs the estimation result from the estimation unit (102).

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In the delivery of cows, there are cases where natural delivery is difficult (dystocia), and it is required to perform midwifery as necessary. In determining the necessity of midwifery, it is important to accurately grasp the posture of the fetus (fetal position). The fetal position during cow delivery can be roughly predicted by whether the extremities of the fetus seen from the vulva of the mother cow after the second rupture of membranes are the front limbs or the hind limbs. However, it is difficult to distinguish between the front and hind limbs based only on the direction and shape of the hooves. Generally, the front and hind limbs are determined by inserting a hand into the vagina to confirm the tip of the fetus's nose or by searching for the joints of the limbs.

[0003] As an example, Non-Patent Document 1 discloses a technique for inserting a hand into the birth canal of a mother cow to confirm the arrangement of the head and limbs of the fetus.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, even for a skilled veterinarian, it is heavy labor and highly difficult to search for the joints of the fetus due to the intense uterine contractions in the vagina of the mother cow during labor. Also, at the breeding site, there are cases where the livestock owner starts pulling only by visually judging the direction of the fetus's hooves, but if the fetal position is such that delivery is impossible, the fetus will become stuck in the mother's body, which is a matter of life and death for both the fetus and the mother.

[0006] Figure 1 shows an example of fetal position in a singleton fetus at the time of delivery. Examples of fetal position at the time of delivery include, as shown in Figure 1, • Upper fetal presentation: The fetus is exposed from the mother's womb with its forelimbs facing downwards. • Upper fetal position with tail pronation: The fetus is exposed from the mother's body with its hind limbs facing upwards. • Lower fetal presentation: The fetus is exposed from the mother's womb with its forelimbs facing upwards. • Downward-facing caudal presentation: The fetus is exposed from the mother's body starting with its hind limbs, and the underside of the hooves of the exposed hind limbs is facing downwards. These are the four main types. Furthermore, as shown in Figure 1, the supraceous cephalic presentation and supraceous caudal presentation are fetal positions that allow for delivery as is, while the supracephalic presentation and supracephalic caudal presentation are fetal positions that do not allow for delivery as is.

[0007] Here, when a fetus's limb is exposed outside the mother's body during calving, the orientation of the hoof can be relatively easily determined by the user's visual inspection. However, as mentioned above, it is difficult for the user to determine whether the exposed fetal limb is a forelimb or a hindlimb.

[0008] Therefore, if it is possible to determine whether the limbs of the fetus are forelimbs or hindlimbs as soon as they can be visually observed, it is thought that the fetal position of a singleton fetus can be predicted, enabling appropriate midwifery.

[0009] One aspect of the present invention aims to realize a technology that makes it easier to determine the posture of a single fetus in the womb of a mother cow. [Means for solving the problem]

[0010] To solve the above problems, an information processing device according to one aspect of the present invention includes: an acquisition unit that acquires images of the ends of the limbs of a fetus at the beginning of delivery; an estimation unit that inputs the images acquired by the acquisition unit into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs; and an output unit that outputs the estimation result from the estimation unit.

[0011] To solve the above problems, an information processing method according to one aspect of the present invention includes: an acquisition process to acquire an image of the end of a fetus's limb at the beginning of delivery; an estimation process to input the image acquired in the acquisition process into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb end belongs to a forelimb or a hindlimb; and an output process to output the estimation result from the estimation process.

[0012] To solve the above problems, a program according to one aspect of the present invention causes a computer to perform an acquisition process to acquire images of the ends of a fetus's limbs at the beginning of delivery; an estimation process to input the images acquired in the acquisition process into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs; and an output process to output the estimation results from the estimation process. [Effects of the Invention]

[0013] According to one aspect of the present invention, the position of a fetus within the womb of a mother cow can be more easily determined. [Brief explanation of the drawing]

[0014] [Figure 1] It is a diagram showing an example related to the fetal position during childbirth according to the present invention. [Figure 2] It is a block diagram showing a configuration example of an information processing apparatus according to the present invention. [Figure 3] It is a diagram showing an example of the state of a mother and fetus during the delivery of a cow according to the present invention. [Figure 4] It is an example of an image related to the end of a limb of a neonatal calf according to the present invention. [Figure 5] It is an example of an image related to the end of a limb of a neonatal calf according to the present invention. [Figure 6] It is an example of an image related to the end of a limb of a neonatal calf according to the present invention. [Figure 7] It is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 8] It is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 9] It is a block diagram showing a configuration example of an information processing apparatus according to the present invention. [Figure 10] It is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 11] It is a flowchart showing an example of the flow of an information processing method according to the present invention. [Figure 12] It is a diagram showing the number of images showing the end of a limb used in an embodiment according to the present invention. [Figure 13] It is a diagram showing the processing result in an embodiment according to the present invention.

Mode for Carrying Out the Invention

[0015] 〔Embodiment 1〕 Hereinafter, an embodiment of the present invention will be described in detail.

[0016] (Outline of Information Processing Apparatus 1) The information processing device 1 is a device for estimating, using a learning model, whether the end of a fetal limb (limb end) captured during the initial stage of delivery is a forelimb or a hindlimb. In this embodiment, the orientation of the hoof of the fetal limb may be determined, for example, by the user visually.

[0017] An example of the information processing device 1 is a server. In this case, the information processing device 1 may be connected to a terminal owned by a user via a network N described later. Here, the terminal owned by the user may have, for example, an imaging function. Specific examples of a terminal owned by a user include a digital camera, a smartphone, and a tablet computer. Each process in the information processing device 1 may be executed, for example, from a browser running on the terminal owned by the user, or from an application on the server via an API (Application Programming Interface).

[0018] Furthermore, each component of the information processing device 1 may, for example, be provided entirely by the server as described above, or it may be partially provided between the server connected via a network N and the terminal owned by the user. Also, as an example, each component of the information processing device 1 may be provided entirely by the terminal owned by the user.

[0019] (Configuration of Information Processing Device 1) The configuration of the information processing device 1 will be explained with reference to Figure 2. Figure 2 is a block diagram showing an example configuration of the information processing device 1. As shown in Figure 2, the information processing device 1 comprises a control unit 10, a storage unit 20, a communication unit 30, and an input / output unit 40.

[0020] (Control Unit 10) The control unit 10 controls all parts of the information processing device 1. The control unit 10 includes, for example, an acquisition unit 101, an estimation unit 102, an output unit 103, a limb image acquisition unit 104, and a learning unit 105, as shown in Figure 2.

[0021] (Acquisition part 101) The acquisition unit 101 acquires images of the extremities of the fetus's limbs during the initial stage of delivery.

[0022] The information processing device 1 according to this embodiment may process, for example, all breeds of cattle. The breeds of cattle that can be processed by the information processing device 1 may include, for example, beef cattle, dairy cattle, draft cattle, or ornamental / pet cattle. Specific examples of such breeds include, but are not limited to, Holstein, Japanese Black, other regional breeds, and crossbreeds. Furthermore, the mother cow and fetus that can be processed by the information processing device 1 may be of the same breed or of different breeds.

[0023] Figure 3 shows an example of the mother-offspring condition at the time of calving. "Fetus at the beginning of calving" can refer to the period from when the tips of the fetus's limbs can be visually confirmed from the mother cow's genitals during calving, as shown in the image PREG in Figure 3, until the fetus's head or tail can be seen.

[0024] For example, the end of a fetal limb may include the area from the tip of the limb to the fetlock joint. In this case, the end of the fetal limb includes the hoof located at the tip of the limb. Also, for example, the end of a fetal limb may include the area from the tip of the limb to the hoof. Here, for example, if only a portion of the area included in the end of the fetal limb is exposed outside the mother's body, the user may use their hands or other means to pull the fetal limb out of the mother's body until the entire area is exposed, and then image the end of the fetal limb.

[0025] The acquisition unit 101 may, for example, acquire images of the extremities of the fetus's limbs at the beginning of delivery via a terminal with imaging capabilities. Specific examples of terminals with imaging capabilities include digital cameras, smartphones, and tablet computers. The acquisition unit 101 may, for example, acquire images captured via a terminal with imaging capabilities via a network N connected to the information processing device 1. Here, the terminal with imaging capabilities may, for example, be a terminal owned by the user mentioned above.

[0026] (Example image of the end of a calf's leg) Figures 4 to 6 are examples of images of the limb ends of newborn calves within 5 days of birth. Image ORI in Figure 4 is an example of an image taken of a newborn calf within 5 days of birth, including the limb end and other objects. Image TR1 in Figure 5 is an example of an image showing the area from the tip of the limb to roughly the fetlock joint of a newborn calf within 5 days of birth. Image TR2 in Figure 6 is an example of an image showing the area from the tip of the limb to roughly the hoof of a newborn calf within 5 days of birth. In each of the images in Figures 4 to 6, a transparent plate with a grid of the same size is placed over the hoof.

[0027] For example, the acquisition unit 101 may acquire an image that includes the ends of the fetal limbs and objects other than the ends of the limbs, as shown in the image ORI of Figure 4. Alternatively, for example, the acquisition unit 101 may acquire an image showing a range from the tip of the fetal limb to approximately the fetlock, as shown in the image TR1 of Figure 5. Alternatively, for example, the acquisition unit 101 may acquire an image showing a range from the tip of the fetal limb to approximately the hoof, as shown in the image TR2 of Figure 6.

[0028] As an example, the acquisition unit 101 may acquire an image that has been cropped by the user so that the area of ​​the image showing the fetal limb ends is larger, as shown in image TR1 in Figure 5 or image TR2 in Figure 6, in an image that includes the ends of the fetal limbs and objects other than the ends of the limbs, as shown in image ORI in Figure 4.

[0029] (Estimation section 102) The estimation unit 102 inputs the images acquired by the acquisition unit 101 into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus after the start of calving and a newborn calf within 5 days after calving, to estimate whether the captured limb end belongs to a forelimb or a hindlimb. The learning limb images are image data that each include one or more pairs of "an image showing a forelimb and a label indicating that the image is a forelimb" and "an image showing a hindlimb and a label indicating that the image is a hindlimb" of at least one individual of a fetus after the start of calving and a newborn calf within 5 days after calving. For example, the fetus after the start of calving and the newborn calf within 5 days after calving shown in the learning limb images may be different individuals from the fetus at the start of calving shown in the images acquired by the acquisition unit 101.

[0030] Here, the individual shown in the learning limb image may be, for example, at least one of the following: a fetus after the start of calving, and a newborn calf within 5 days after calving, as described above. That is, as a specific example, the individual shown in the learning limb image may be: • The fetus may be in the early stages of delivery, or The fetus may be in a later stage than the start of delivery, or • It may be a newborn calf within 5 days of giving birth, or It may be at least one individual of a fetus in the period after the start of calving, or a newborn calf within 5 days after calving, or • Any multiple species from each of the above types may be included.

[0031] Furthermore, the newborn calves shown in the learning limb images may, for example, be newborn calves that were stillborn or died within 5 days of birth.

[0032] The learning model is a machine learning model for estimating whether the end of a fetal limb imaged at the beginning of delivery is a forelimb or a hindlimb. Here, the learning model is a machine learning model that has been pre-trained using training limb images. That is, the training limb images may include, for example, training data, validation data, and test data for the learning model. In this case, the training limb images may include, for example, "training data, validation data, and test data for the forelimb" and "training data, validation data, and test data for the hindlimb."

[0033] Furthermore, the learning model may be, for example, a machine learning model that has been deeply trained using training limb images. Specific examples of deep learning frameworks include PyTorch. Specific examples of learning models include VGG (Visual Geometry Group)Net, ResNet, AlexNet, DenseNet, SqueezeNet, SwinTransformer, and VisionTransformer.

[0034] In this embodiment, the estimation unit 102 inputs the image acquired by the acquisition unit 101 into a pre-trained model using training limb images to estimate whether the end of the fetal limb captured at the beginning of delivery is a forelimb or a hindlimb.

[0035] Here, as an example, the learning model may be pre-trained by the learning unit 105 described later. Alternatively, as another example, the learning model may be pre-trained by an external device to the information processing device 1.

[0036] The estimation result by the estimation unit 102 may include, for example, one or more pairs of images showing the end of a fetal limb that was the subject of estimation processing by the estimation unit 102, and information indicating whether the end of the limb corresponds to a "forelimb" or a "hindlimb".

[0037] (Output section 103) The output unit 103 outputs the estimation result from the estimation unit 102.

[0038] With the above configuration, it is possible to more accurately determine whether the end of a fetal limb at the beginning of delivery is a forelimb or a hindlimb using a machine learning model.

[0039] (Limb image acquisition unit 104) The limb image acquisition unit 104 acquires limb images for training.

[0040] (Learning Section 105) The learning unit 105 trains a learning model using the learning limb images acquired by the limb image acquisition unit 104. For example, the processes in the limb image acquisition unit 104 and the learning unit 105 may be performed in advance of the processes in the estimation unit 102. That is, the estimation unit 102 may, for example, input the image acquired by the acquisition unit 101 into the learning model trained by the learning unit 105 to estimate whether the captured limb end belongs to a forelimb or a hindlimb.

[0041] The above configuration allows for training a machine learning model to estimate whether the end of a fetal limb at the beginning of delivery is a forelimb or a hindlimb.

[0042] (Storage unit 20) The storage unit 20 stores various data referenced by the control unit 10, as well as various data generated by the control unit 10. Specific examples of data stored in the storage unit 20 include: • Acquired image data ORI • Learning objective image data TD • Learning Model LM • Estimated result ER ·Extremity site information DE These are some examples.

[0043] The acquired image data ORI is image data acquired by the acquisition unit 101, and is an image capturing the ends of the fetal limbs at the beginning of delivery. The acquired image data ORI may be, for example, an image capturing only the ends of the fetal limbs at the beginning of delivery, or an image capturing at least the ends of the fetal limbs at the beginning of delivery. The image ORI in Figure 3 is an example of acquired image data ORI.

[0044] The training limb image data TD is image data that includes one or more pairs of "a pair of an image showing a forelimb and a label indicating that the image is a forelimb from at least one individual of a fetus after the start of calving and a newborn calf within 5 days after calving" and "a pair of an image showing a hindlimb and a label indicating that the image is a hindlimb from at least one individual of a fetus after the start of calving and a newborn calf within 5 days after calving." For example, the fetus after the start of calving and the newborn calf within 5 days after calving shown in the training limb image data TD may be different individuals from the fetus at the start of calving shown in the acquired image data ORI.

[0045] Here, the individual shown in the learning limb image data TD may be, for example, at least one of the following: a fetus from the beginning of calving, or a newborn calf within 5 days after calving, as described above. That is, as a specific example, the individual shown in the learning limb image data TD may be: • The fetus may be in the early stages of delivery, or The fetus may be in a later stage than the start of delivery, or • It may be a newborn calf within 5 days of giving birth, or It may be at least one individual of a fetus in the period after the start of calving, or a newborn calf within 5 days after calving, or • Any multiple species from each of the above types may be included.

[0046] Furthermore, the newborn calves shown in the learning limb image data TD may, for example, be newborn calves that were stillborn or died within 5 days of birth.

[0047] The learning model LM is a machine learning model for estimating whether the end of a fetal limb imaged at the beginning of delivery is a forelimb or a hindlimb. Here, the learning model LM is a machine learning model that has been pre-trained using training limb image data TD. That is, the training limb image data TD may include, for example, training data, validation data, and test data for the learning model LM. In this case, the training limb image data TD may include, for example, "training data, validation data, and test data for the forelimb" and "training data, validation data, and test data for the hindlimb".

[0048] Furthermore, the learning model LM may be a machine learning model that has been deeply trained using, for example, training limb image data TD. Specific examples of deep learning frameworks include PyTorch. Specific examples of learning models LM include VGG (Visual Geometry Group)Net, ResNet, AlexNet, DenseNet, SqueezeNet, SwinTransformer, and VisionTransformer.

[0049] The estimated result ER is the result of the estimation unit 102 estimating whether the end of the captured fetal limb is a forelimb or a hindlimb. The estimated result ER may include, for example, one or more pairs of images showing the end of the fetal limb that was the subject of estimation processing by the estimation unit 102, and information indicating whether the end of the limb corresponds to a "forelimb" or a "hindlimb".

[0050] The limb end position information DE indicates the part included in the end of the fetal limb. For example, the end of the fetal limb may include the area from the tip of the limb to the fetlock joint. In this case, the end of the fetal limb includes the hoof located at the tip of the limb. Alternatively, for example, the end of the fetal limb may include the area from the tip of the limb to the hoof.

[0051] (Communications Section 30) The communication unit 30 communicates with devices outside the information processing device 1. For example, the communication unit 30 communicates with various external devices connected to the information processing device 1 via a network N (not shown). The communication unit 30 transmits data supplied from the control unit 10 to the outside and supplies data received from the external devices to the control unit 10. The specific configuration of the network N is not limited to this exemplary embodiment, but as an example, a wireless LAN (Local Area Network), wired LAN, WAN (Wide Area Network), public telephone network, mobile data communication network, or a combination of these networks can be used.

[0052] (Input / output section 40) The input / output unit 40 is configured to include at least one of the following input / output devices: a keyboard, mouse, display, printer, touch panel, etc. Alternatively, the input / output unit 40 may be configured to have input / output devices such as a keyboard, mouse, display, printer, touch panel, etc. connected to it. In this configuration, the input / output unit 40 receives various types of information from the connected input device to the information processing device 1. The input / output unit 40 also outputs various types of information to the connected output device under the control of the control unit 10. An interface such as USB (Universal Serial Bus) can be used as the input / output unit 40.

[0053] (Information processing method S1 flow) The flow of the information processing method S1 executed by the information processing device 1 will be explained with reference to Figure 7. Figure 7 is a flowchart showing an example of the flow of the information processing method S1. The information processing method S1 includes, for example, an acquisition process (step) S11, an estimation process (step) S12, and an output process (step) S13, as shown in Figure 7.

[0054] (Step S11) In step S11, the acquisition unit 101 acquires images of the extremities of the fetus's limbs at the beginning of delivery.

[0055] (Step S12) In step S12, the estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model that has been trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, either a fetus from the beginning of delivery or a newborn calf within 5 days after delivery, to estimate whether the captured limb end belongs to a forelimb or a hindlimb.

[0056] (Step S13) In step S13, the output unit 103 outputs the estimation result from the estimation unit 102.

[0057] (Information processing method S2 flow) The flow of the information processing method S2 executed by the information processing device 1 will be explained with reference to Figure 8. Figure 8 is a flowchart showing an example of the flow of the information processing method S2. The information processing method S2 includes, for example, a learning limb image acquisition process (step) S21 and a learning process (step) S22, as shown in Figure 8.

[0058] (Step S21) In step S21, the limb image acquisition unit 104 acquires learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, either a fetus from the beginning of delivery period or a newborn calf within 5 days after delivery.

[0059] (Step S22) In step S22, the learning unit 105 trains a learning model using the learning limb images acquired by the limb image acquisition unit 104.

[0060] [Embodiment 2] Other embodiments of the present invention are described below. For the sake of clarity, components having the same function as those described in the above embodiments will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0061] (Overview of Information Processing Device 1A) The information processing device 1A has the same configuration as the information processing device 1 according to Embodiment 1, and further includes a configuration that crops an image captured including the ends of the fetus's limbs and objects other than the ends of the limbs, so that the proportion of the area of ​​the image showing the ends of the fetus's limbs in the image becomes larger.

[0062] (Configuration of Information Processing Device 1A) The configuration of the information processing device 1A will be described with reference to Figure 9. Figure 9 is a block diagram showing an example configuration of the information processing device 1A. As shown in Figure 9, the information processing device 1A comprises a control unit 10A, a storage unit 20A, a communication unit 30, and an input / output unit 40. The communication unit 30 and the input / output unit 40 have the same configuration as those of the same name in the information processing device 1, so their explanation will be omitted here.

[0063] (Control Unit 10A) The control unit 10A controls all parts of the information processing device 1A. For example, as shown in Figure 9, the control unit 10A includes an acquisition unit 101, an estimation unit 102, an output unit 103, a limb image acquisition unit 104, a learning unit 105, a determination unit 106, a range estimation unit 107, a range output unit 108, a limb end image acquisition unit 109, and a range learning unit 110. The acquisition unit 101, the limb image acquisition unit 104, and the learning unit 105 have the same configuration as the corresponding components in the information processing device 1, so their explanation is omitted here.

[0064] (Estimation section 102) The estimation unit 102 inputs the images acquired by the acquisition unit 101 into a learning model that has been trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, either a fetus from the beginning of delivery or a newborn calf within 5 days after delivery, to estimate whether the captured limb end belongs to a forelimb or a hindlimb.

[0065] The training limb images may further indicate, for example, the orientation of the hoof of the limb. In this case, the estimation unit 102 may, for example, input the image acquired by the acquisition unit 101 into a learning model trained using the training limb images to further estimate the orientation of the hoof of the captured limb. The orientation of the hoof of a fetal limb is indicated, for example, by whether the orientation of the sole of the fetal limb's hoof is generally upward or downward in the vertical direction, assuming the mother cow is upright, as shown in the image PREG in Figure 3. In this case, the training limb images may be image data that includes, for example, one or more pairs of "an image showing an upward-facing sole of a hoof on the limb of at least one individual fetus from the beginning of calving and a newborn calf within 5 days after calving, and a label indicating that the image is of an upward-facing sole" and "an image showing a downward-facing sole of a hoof on the limb of at least one individual fetus from the beginning of calving and a newborn calf within 5 days after calving, and a label indicating that the image is of a downward-facing sole."

[0066] In this embodiment, the estimation unit 102 may, for example, input the images acquired by the acquisition unit 101 into a learning model trained using learning limb images, which are images that further indicate the orientation of the hooves of the limbs of at least one of the fetuses from the beginning of calving onward and newborn calves within 5 days after calving, in order to further estimate the orientation of the hooves of the fetuses captured at the beginning of calving.

[0067] With the above configuration, the orientation of the hooves of the fetus's limbs in images taken during the initial stages of delivery can be determined using a machine learning model.

[0068] Regarding the other components of the estimation unit 102, they are the same as those of the estimation unit 102 in the information processing device 1, so their explanation is omitted here.

[0069] (Judgment unit 106) The determination unit 106 determines whether assistance is possible during fetal delivery based on the estimation results from the estimation unit 102 and hoof orientation information indicating the orientation of the hoofs of the imaged limbs. The hoof orientation information indicates whether the hoofs of the fetal limbs are pointing upward or downward. The hoof orientation information may be estimated by the estimation unit 102, for example, or it may be the result of a user visually inspecting the hoof and acquired by the information processing device 1.

[0070] The determination unit 106 may, for example, determine which of the four aforementioned fetal positions the fetus has, based on the estimation result of the estimation unit 102, which estimates whether the end of the limb belongs to a forelimb or a hindlimb, and on the hoof orientation information, which indicates whether the hoof of the imaged limb is facing upward or downward. Then, the determination unit 106 may, for example, determine whether the fetal position shown in the image acquired by the acquisition unit 101 is suitable for delivery. That is, the determination unit 106 may, for example, determine whether the fetal position is suitable for delivery as is, or whether it is not suitable for delivery as is.

[0071] Here, if the fetus is in a position that allows for delivery, the user can assist with the delivery. On the other hand, if the fetus is not in a position that allows for delivery, it is practically impossible for the user to assist with the delivery unless the user is a veterinarian or the like. In other words, the determination unit 106 may, for example, determine whether the fetus is in a position that allows for delivery, as described above, and then determine whether the user (who is not a veterinarian or the like) can assist when the fetus is delivered.

[0072] (Output section 103) The output unit 103 further outputs the determination result from the determination unit 106. The output unit 103 may, for example, output the estimation result from the estimation unit 102 and the determination result from the determination unit 106 together.

[0073] With the above configuration, the user can determine whether assistance is needed during the delivery of the fetus from the images showing the fetus's limbs at the beginning of delivery.

[0074] (Range estimation unit 107) The range estimation unit 107 inputs the images acquired by the acquisition unit 101 into a range estimation learning model trained using learning limb end images, which are images showing the limb ends of at least one of the fetuses after the start of calving and newborn calves within 5 days after calving, to estimate the range showing the limb ends in the acquired images. The learning limb end images are image data that include one or more pairs of "an image showing the limb end of at least one of the fetuses after the start of calving and newborn calves within 5 days after calving, and a label indicating that the image is of a limb end." For example, a fetus after the start of calving or a newborn calf within 5 days after calving shown in the learning limb end image may be a different individual from, for example, a fetus at the start of calving shown in the image acquired by the acquisition unit 101.

[0075] Here, the individual shown in the learning limb image may be, for example, at least one of the following: a fetus after the start of calving, or a newborn calf within 5 days after calving, as described above. That is, as a specific example, the individual shown in the learning limb image may be: • The fetus may be in the early stages of delivery, or The fetus may be in a later stage than the start of delivery, or • It may be a newborn calf within 5 days of giving birth, or It may be at least one individual of a fetus in the period after the start of calving, or a newborn calf within 5 days after calving, or • Any multiple species from each of the above types may be included.

[0076] Furthermore, the newborn calves shown in the learning limb images may be, for example, newborn calves that were stillborn or died within 5 days of birth.

[0077] A range estimation learning model is a machine learning model for estimating the range of limb extremities in images taken at the beginning of delivery, including the limb extremities. Here, the range estimation learning model is a machine learning model that has been pre-trained using training limb extremity images. That is, the training limb extremity images may include, for example, training data, validation data, and test data for the range estimation learning model. A specific example of a range estimation learning model is YOLO.

[0078] Here, as an example, the range estimation learning model may be pre-trained by the range learning unit 110 described later. Alternatively, as another example, the range estimation learning model may be pre-trained by an external device to the information processing device 1.

[0079] Furthermore, the range indicating the limb end estimated by the range estimation unit 107 may be, for example, the range of the image indicating the limb end.

[0080] (Range output section 108) The range output unit 108 outputs an image corresponding to the range indicating the limb end, based on the estimation result by the range estimation unit 107. The range output unit 108 may, for example, crop the image acquired by the acquisition unit 101 so that the area ratio of the range indicating the limb end estimated by the range estimation unit 107, i.e., the area estimated by the range estimation unit 107, becomes larger, and output the cropped image.

[0081] For example, if the range estimated by the range estimation unit 107 includes the area from the tip of the limb to the fetlock joint, the range output unit 108 may trim the area including the area from the tip of the limb to the fetlock joint, as shown in image TR1 of Figure 5. Also, for example, if the range estimated by the range estimation unit 107 includes the area from the tip of the limb to the hoof, the range output unit 108 may trim the area including the area from the tip of the limb to the hoof, as shown in image TR2 of Figure 6.

[0082] Furthermore, as mentioned above, when the range output unit 108 outputs an image corresponding to the range indicating the limb end in the image acquired by the acquisition unit 101, the estimation unit 102 may, for example, input the image output by the range output unit 108 into a learning model to estimate whether the captured limb end belongs to a forelimb or a hindlimb.

[0083] The above configuration makes it possible to improve the accuracy of estimation processing for the captured limb ends.

[0084] (Limb end image acquisition unit 109) The limb end image acquisition unit 109 acquires limb end images for training.

[0085] (Scope Learning Section 110) The range learning unit 110 trains a range estimation learning model using the limb end images acquired by the limb end image acquisition unit 109. For example, the processes in the limb end image acquisition unit 109 and the range learning unit 110 may be performed in advance of the processes in the range estimation unit 107. That is, the range estimation unit 107 may, for example, input the images acquired by the acquisition unit 101 into the range estimation learning model trained by the range learning unit 110 to estimate the range indicating the limb end in the images acquired by the acquisition unit 101.

[0086] The above configuration allows for training a machine learning model to estimate the range representing the limb end in an image that includes an image showing the limb end.

[0087] (Storage unit 20A) The storage unit 20A stores various data referenced by the control unit 10A, and various data generated by the control unit 10A. Specific examples of data stored in the storage unit 20A include: • Acquired image data ORI • Learning objective image data TD • Learning Model LM • Estimated result ER • Hoof orientation information DI ·Judgment result RE • Learning limb end image data TDT • Range Estimation Learning Model (LMT) • Trimmed image data TR ·Extremity site information DE These are some examples.

[0088] The acquired image data ORI, the learning limb image data TD, the learning model LM, and the limb end-position information DE have the same configuration as those with the same names in the storage unit 20 of the information processing device 1, so their explanation is omitted here.

[0089] The estimated result ER is the result of the estimation unit 102 estimating whether the end of the captured fetal limb is a forelimb or a hindlimb. The estimated result ER may include, for example, one or more pairs of images showing the end of the limb that was the subject of estimation processing by the estimation unit 102, and information indicating whether the end of the limb corresponds to a "forelimb" or a "hindlimb".

[0090] The hoof orientation information DI indicates whether the hoof of the fetus's limb is facing upward or downward. The hoof orientation information DI may be estimated by, for example, the estimation unit 102, or it may be the result of a user visually inspecting the hoof and acquired by the information processing device 1.

[0091] The determination result RE is the result of the determination unit 106 determining whether assistance is possible when the fetus is delivered, based on the estimated result RE and the hoof orientation information DI. The determination result RE may, for example, indicate whether assistance by a user (not a veterinarian, etc.) is possible when the imaged fetus is delivered.

[0092] The training limb end image data (TDT) is image data that includes one or more pairs of "an image showing the end of a limb in at least one individual of a fetus after the start of calving, or a newborn calf within 5 days after calving, and a label indicating that the image is of the end of a limb." For example, a fetus after the start of calving, or a newborn calf within 5 days after calving, shown in the training limb end image data (TDT) may be a different individual from, for example, a fetus at the start of calving shown in the acquired image data (ORI).

[0093] Here, the individual shown in the training limb image data TDT may be, for example, at least one of the following: a fetus after the start of calving, and a newborn calf within 5 days after calving, as described above. That is, as a specific example, the individual shown in the training limb image data TDT may be: • The fetus may be in the early stages of delivery, or The fetus may be in a later stage than the start of delivery, or • It may be a newborn calf within 5 days of giving birth, or It may be at least one individual of a fetus in the period after the start of calving, or a newborn calf within 5 days after calving, or • Any multiple species from each of the above types may be included.

[0094] Furthermore, the newborn calves shown in the training limb end image data (TDT) may be, for example, newborn calves that were stillborn or died within 5 days of birth.

[0095] The Range Estimation Learning Model (LMT) is a machine learning model for estimating the range of limb extremities in images taken at the beginning of delivery, including the extremities of the fetus. Here, the Range Estimation Learning Model (LMT) is a machine learning model that has been pre-trained using training limb extremity image data (TDT). That is, the training limb extremity image data (TDT) may include, for example, training data, validation data, and test data for the Range Estimation Learning Model (LMT). A specific example of the Range Estimation Learning Model (LMT) is YOLO.

[0096] The trimmed image data TR is an image obtained by trimming the area representing the limb end in the acquired image data ORI based on the estimation result by the range estimation unit 107. Here, the area representing the limb end may be, for example, the area of ​​the image representing the limb end, or it may be the area including the image representing the limb end and its vicinity. Image TR1 in Figure 4 and Image TR2 in Figure 5 are examples of trimmed image data TR, respectively.

[0097] (Information processing method executed by information processing device 1A) The information processing device 1A executes information processing methods S1 and S2, similar to the information processing device 1. The information processing device 1A may also execute information processing methods S3 and S4, which will be described later.

[0098] (Information processing method S3 flow) The flow of the information processing method S3 executed by the information processing device 1A will be explained with reference to Figure 10. Figure 10 is a flowchart showing an example of the flow of the information processing method S3. The information processing method S3 includes, for example, an acquisition process (step) S31, a range estimation process (step) S32, a range output process (step) S33, an estimation process (step) S34, and an output process (step) S35, as shown in Figure 10.

[0099] (Step S31) In step S31, the acquisition unit 101 acquires images of the extremities of the fetus's limbs at the beginning of delivery.

[0100] (Step S32) In step S32, the range estimation unit 107 inputs the images acquired by the acquisition unit 101 into a range estimation learning model that has been trained using learning limb end images, which are images showing the limb ends of at least one of the fetuses from the beginning of delivery onward and newborn calves within 5 days after delivery, and estimates the range showing the limb ends in the acquired images.

[0101] (Step S33) In step S33, the range output unit 108 outputs an image corresponding to the range indicating the limb end, based on the estimation result by the range estimation unit 107.

[0102] (Step S34) In step S34, the estimation unit 102 inputs the image acquired by the acquisition unit 101 into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual of a fetus from the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb end belongs to a forelimb or a hindlimb. Alternatively, when the range output unit 108 outputs an image corresponding to the range showing the limb end, the estimation unit 102 may, for example, input the image output by the range output unit 108 into the learning model to estimate whether the captured limb end belongs to a forelimb or a hindlimb.

[0103] (Step S35) In step S35, the determination unit 106 determines whether assistance is possible when the fetus is delivered, based on the estimation result from the estimation unit 102 and the hoof orientation information indicating the orientation of the hoof of the imaged limb.

[0104] (Step S36) In step S36, the output unit 103 outputs the determination result from the determination unit 106.

[0105] (Information processing method S4 flow) The flow of the information processing method S4 executed by the information processing device 1A will be explained with reference to Figure 11. Figure 11 is a flowchart showing an example of the flow of the information processing method S4. The information processing method S4 includes, for example, a learning limb end image acquisition process (step) S41 and a range learning process (step) S42, as shown in Figure 11.

[0106] (Step S41) In step S41, the limb end image acquisition unit 109 acquires learning limb end images, which are images showing the limb ends of at least one of the following individuals: a fetus from the beginning of delivery onward, and a newborn calf within 5 days after delivery.

[0107] (Step S42) In step S42, the range learning unit 110 trains a range estimation learning model using the limb end images acquired by the limb end image acquisition unit 109.

[0108] [Examples of implementation using software] The functions of the information processing devices 1 and 1A (hereinafter referred to as "devices") are programs that cause the devices to function as computers, and these programs can be realized by programs that cause each control block of the devices (especially each part included in the control units 10 and 10A) to function as a computer.

[0109] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0110] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0111] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.

[0112] Furthermore, each process described in the above embodiments may be performed by AI (Artificial Intelligence). In this case, the AI ​​may operate on the control device described above, or it may operate on other devices (for example, an edge computer or a cloud server).

[0113] 〔summary〕 An information processing device according to Embodiment 1 of the present invention comprises: an acquisition unit that acquires images of the ends of the limbs of a fetus at the beginning of delivery; an estimation unit that inputs the images acquired by the acquisition unit into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs; and an output unit that outputs the estimation results from the estimation unit.

[0114] An information processing device according to aspect 2 of the present invention further comprises, in aspect 1 above, a limb image acquisition unit that acquires the limb images for learning, and a learning unit that trains the learning model using the limb images acquired by the limb image acquisition unit.

[0115] The information processing device according to embodiment 3 of the present invention further comprises, in embodiment 1 or 2 above, a determination unit that determines whether assistance is possible when the fetus is delivered based on the estimation result by the estimation unit and hoof orientation information indicating the orientation of the hoof of the captured limb, and the output unit further outputs the determination result by the determination unit.

[0116] In the information processing device according to aspect 4 of the present invention, in aspect 3 above, the learning limb image further indicates the orientation of the hoof of the limb, and the estimation unit inputs the image acquired by the acquisition unit into the learning model learned using the learning limb image to further estimate the orientation of the hoof of the captured limb.

[0117] An information processing device according to aspect 5 of the present invention, in any of aspects 1 to 4 above, further comprises: a range estimation unit that inputs the image acquired by the acquisition unit into a range estimation learning model trained using learning limb end images, which are images showing the limb ends of at least one individual of a fetus from the beginning of delivery and a newborn calf within 5 days after delivery, to estimate the range showing the limb end in the acquired image; and a range output unit that outputs an image corresponding to the range showing the limb end based on the estimation result by the range estimation unit, wherein the estimation unit inputs the image output by the range output unit into the learning model to estimate whether the captured limb end is a forelimb or a hindlimb.

[0118] An information processing device according to aspect 6 of the present invention further comprises, in aspect 5 above, an end-of-limb image acquisition unit that acquires the learning end-of-limb images, and a range learning unit that trains the range estimation learning model using the learning end-of-limb images acquired by the end-of-limb image acquisition unit.

[0119] In the information processing device according to embodiment 7 of the present invention, in any of embodiments 1 to 6 above, the end of the limb is a portion that includes the tip of the limb to the fetlock joint.

[0120] An information processing method according to aspect 8 of the present invention includes: an acquisition process for acquiring images of the ends of fetal limbs at the beginning of delivery; an estimation process for inputting the images acquired in the acquisition process into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs; and an output process for outputting the estimation results from the estimation process.

[0121] A program according to aspect 9 of the present invention causes a computer to perform an acquisition process to acquire images of the ends of a fetus's limbs at the beginning of delivery; an estimation process to input the images acquired in the acquisition process into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual fetus after the beginning of delivery and a newborn calf within 5 days after delivery, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs; and an output process to output the estimation results from the estimation process.

[0122] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Examples]

[0123] Examples of the present invention are described below.

[0124] In this embodiment, the information processing device 1 was used to estimate whether an image showing the end of a fetal limb at the beginning of delivery was a forelimb or a hindlimb.

[0125] Figure 12 shows the number of images used in this embodiment, which show the limb ends of fetuses and newborn calves within 5 days of birth. In Figure 12, the number of images used for training and validation for the information processing device 1 are shown, respectively. In this embodiment, as shown in Figure 12, the images were classified into "training data, validation data, and test data for the forelimbs" and "training data, validation data, and test data for the hindlimbs." In this embodiment, the above images were used to perform training and validation processing in the information processing device 1.

[0126] Figure 13 shows the processing results of the information processing device 1 in this embodiment. In this embodiment, • The "original" image (corresponding to the ORI image in Figure 4) of a newborn calf taken within 5 days of birth, including the limb ends and other parts of the body. • Images of "Cropped 1 (Limb)" (corresponding to image TR1 in Figure 5), which are cropped to include the area from the tip of the leg to roughly the fetlock of a newborn calf within 5 days of birth, and • A "trimmed 2 (hoof)" image (corresponding to image TR2 in Figure 6) of a newborn calf within 5 days of birth, with the area including roughly the tip of the leg and the hoof trimmed. For each of these, we estimated whether it was a forelimb or a hindlimb. In this embodiment, as shown in Figure 13, we also performed estimations for each of several learning models.

[0127] Figure 13 shows the judgment accuracy calculated from the estimation results by the information processing device 1. Here, judgment accuracy is an indicator of how accurately the device determined whether a fetal limb is a forelimb or a hindlimb. In this embodiment, as shown in Figure 13, two types of judgment accuracy values ​​were calculated: accuracy and precision.

[0128] As a result of processing by the information processing device 1 in this embodiment, as shown in Figure 13, the accuracy rate for the "original" image, which indicates the percentage of all processing results that correctly estimated whether it was a forelimb or a hindlimb, exceeded 0.9. Also, as shown in Figure 13, the accuracy rate for the "trimmed 1 (limb)" image exceeded 0.8. [Explanation of symbols]

[0129] 1. 1A Information Processing Device 10, 10A Control Unit 20, 20A storage section 30 Communications Department 40 Input / output section 101 Acquisition Department 102 Estimation part 103 Output section 104 Limb image acquisition unit 105 Learning Department 106 Judgment section 107 Range Estimation Unit 108 Range Output Section 109 Limb end image acquisition unit 110 Scope Learning Section

Claims

1. An acquisition unit that acquires images of the extremities of the fetus's limbs at the beginning of delivery, An estimation unit inputs the image acquired by the acquisition unit into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, either a fetus from the beginning of delivery or a newborn calf within 5 days after delivery, to estimate whether the captured limb end belongs to a forelimb or a hindlimb. The system includes an output unit that outputs the estimation results from the estimation unit, Information processing device.

2. A limb image acquisition unit that acquires the aforementioned learning limb images, The system further comprises a learning unit that uses the learning limb images acquired by the limb image acquisition unit to train the learning model. The information processing apparatus according to claim 1.

3. The system further includes a determination unit that determines whether assistance is possible when the fetus is delivered, based on the estimation results from the estimation unit and the hoof orientation information indicating the orientation of the hoof of the captured limb. The output unit further outputs the determination result from the determination unit. The information processing apparatus according to claim 1.

4. The aforementioned learning limb image further indicates the orientation of the hoof of the limb, The estimation unit inputs the image acquired by the acquisition unit into the learning model, which has been trained using the learning limb image, to further estimate the orientation of the hoof of the captured limb. The information processing apparatus according to claim 3.

5. The acquisition unit inputs the image acquired by the acquisition unit into a range estimation learning model trained using learning limb end images, which are images showing the limb ends of at least one individual of a fetus from the beginning of delivery and a newborn calf within 5 days after delivery, to estimate the range showing the limb end in the acquired image. The system further comprises a range output unit that outputs an image corresponding to the range showing the end of the limb based on the estimation result by the range estimation unit, The estimation unit inputs the image output by the range output unit into the learning model to estimate whether the captured limb end belongs to a forelimb or a hindlimb. The information processing apparatus according to claim 1.

6. A limb end image acquisition unit that acquires the limb end images for learning, The system further comprises a range learning unit that uses the learning limb images acquired by the limb image acquisition unit to train the range estimation learning model. The information processing apparatus according to claim 5.

7. The end of the limb is the portion that includes the tip of the limb up to the fetlock joint. The information processing apparatus according to any one of claims 1 to 6.

8. An acquisition process to obtain images of the extremities of the fetus's limbs at the beginning of delivery, The images acquired in the acquisition process are input into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, such as a fetus from the beginning of calving or a newborn calf within 5 days after calving, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs. The process includes output processing to output the estimation results obtained from the estimation process, Information processing methods.

9. On the computer, An acquisition process to obtain images of the extremities of the fetus's limbs at the beginning of delivery, The images acquired in the acquisition process are input into a learning model trained using learning limb images, which are images showing the forelimbs and hindlimbs of at least one individual, such as a fetus from the beginning of calving or a newborn calf within 5 days after calving, to estimate whether the captured limb ends belong to the forelimbs or hindlimbs. An output process that outputs the estimation result obtained by the estimation process described above is executed. program.

Images