Information processing device, information processing method, and program

By combining the information processing device with the image and the position information of the terminal device, the problem of degradation of individual disease risk estimation and identification accuracy is solved, high-precision animal position determination and health status monitoring are achieved, and the efficiency of dairy industry management is improved.

CN118973387APending Publication Date: 2024-11-15FURUKAWA ELECTRIC CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202380032075.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-31
Filing Date
2023-03-31
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing livestock disease management system cannot conduct individual disease risk presumption and physical condition management, especially when the number of livestock subjects increases or the assembly cannot be confirmed on the image, the individual recognition accuracy decreases.

Method used

An information processing device is designed to calculate the position of the animal in the feeding area by obtaining the image of the animal and the position information sent by the terminal device. If the difference is below the specified threshold, an association is established and identification information is recorded. The position determination unit is used to determine the position based on Learning models determine the animal's appearance and posture, conduct individual identification and health status prediction.

Benefits of technology

It realizes high-precision determination of animal location and individual identification in a specific area, can automatically record changes in animal appearance, predict health events such as diseases, estrus, and childbirth, and improves the accuracy and efficiency of health management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118973387A_ABST
    Figure CN118973387A_ABST
Patent Text Reader

Abstract

The present invention enables individual recognition of an animal in relation to a captured image on the basis of the captured image of the animal in a specific region. This animal posture determination device (100) is provided with: an image data analysis unit (112) that acquires a captured image including a cattle raised in a cattle pen, and acquires, as first position information, the position of the cattle in the cattle pen calculated on the basis of the image of the cattle in the captured image; a position data analysis unit (114) that acquires identification information from a sensor device (200) provided to the cattle and second position information indicating the position of the cattle in the cattle pen; and an individual identification unit (115) that, when it is determined that the difference between the first position information and the second position information is equal to or less than a predetermined threshold value, associates and records the identification information relating to the first position information and the second position information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and a program. Background Art

[0002] In order to continuously and stably operate the dairy farming business, it is important to efficiently manage the health of the animals being raised. For example, Patent Document 1 describes a livestock disease management system, which comprises: a feeding status acquisition unit that acquires feeding status information including image information obtained by photographing livestock; and a disease risk estimation unit that extracts livestock reflected in the image information acquired by the feeding status acquisition unit, analyzes the status of the livestock, and estimates the disease risk corresponding to the status of the livestock. Patent Document 2 describes an action volume calculation program, which is characterized in that a computer performs the following processing: acquiring a plurality of photographed images including a plurality of animals, detecting changes in an assembly mounted on each of the plurality of animals based on the plurality of photographed images, and calculating the action volume of each of the plurality of animals based on the detected changes in the assembly.

[0003] [Prior Technical Literature]

[0004] (Patent Document)

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-135956

[0006] Patent Document 2: Japanese Patent Application Publication No. 2018-148843 Summary of the invention

[0007] [Problems to be solved by the invention]

[0008] However, the technology of Patent Document 1 has the following problems: although it is possible to estimate the disease risk of a group, it is not possible to estimate the disease risk of an individual and manage the physical condition. The technology of Patent Document 2 has the following problems: when the number of livestock to be observed increases or when the attached objects cannot be confirmed in the image, the accuracy of individual recognition may sometimes decrease.

[0009] One of the purposes of the present invention is to provide an information processing device, an information processing method and a program, which can determine the position of an animal in a specific area with high precision based on a captured image of the animal in the specific area, thereby enabling individual identification of the animal related to the captured image.

[0010] [Technical means to solve the problem]

[0011] One aspect of the present invention is an information processing device comprising: a first position information acquisition unit, which acquires a captured image of an animal raised in a predetermined breeding area, and acquires the position of the animal in the breeding area calculated based on the image of the animal in the captured image as first position information; a second position information acquisition unit, which acquires identification information of the animal and second position information indicating the position of the animal in the breeding area from a terminal device provided on the animal; and an individual identification unit, which associates and records the first position information with the identification information related to the second position information when it is determined that the difference between the first position information and the second position information is below a prescribed threshold.

[0012] Optionally, the information processing device includes a posture determination unit, which determines the posture determination items pre-set regarding the posture of the animal based on the image of the animal taken, and uses the determination result as the posture information of the animal, obtains the first position information calculated based on the image of the animal that is the object of the determination, and associates the identification information associated with the first position information with the posture information and records them.

[0013] Optionally, the appearance determination unit determines the appearance of the animal using a learned model that has learned a relationship between an image acquired of the animal and a determination result regarding a preset appearance determination item, ie, appearance information.

[0014] Optionally, the appearance determination unit determines the appearance of the animal based on an image acquired of the animal and a determination criterion of features in the image of the animal set in advance for the appearance determination item.

[0015] The second position information may be calculated based on the direction of arrival of radio waves transmitted from a terminal device installed at the animal.

[0016] Optionally, the appearance information of each individual animal recorded in time series by the appearance determination unit is used to predict a period of at least one of disease, estrus, delivery, growth, and weaning of the animal.

[0017] Another aspect of the present invention is an information processing method, which performs the following processing: obtaining a captured image of an animal raised in a predetermined breeding area, and obtaining the position of the animal in the breeding area calculated based on the image of the animal in the captured image as first position information; obtaining identification information of the animal and second position information indicating the position of the animal in the breeding area from a terminal device installed on the animal; and, when it is determined that the difference between the first position information and the second position information is below a specified threshold, associating the first position information with the identification information related to the second position information and recording them.

[0018] Another aspect of the present invention is a program that causes an information processing device to perform the following processing: obtaining a captured image of an animal raised in a predetermined breeding area, and obtaining the position of the animal in the breeding area calculated based on the image of the animal in the captured image as first position information; obtaining identification information of the animal and second position information indicating the position of the animal in the breeding area from a terminal device installed on the animal; and, when it is determined that the difference between the first position information and the second position information is below a specified threshold, associating the first position information with the identification information related to the second position information and recording them.

[0019] (Effects of the Invention)

[0020] According to the present invention, the position of the animal in the specific area can be determined based on a captured image of the animal in the specific area, and the individual identification of the animal related to the captured image can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram illustrating the overall configuration of an animal posture determination system according to one embodiment of the present invention.

[0022] Figure 2 This is a block diagram showing the functional overview of an animal posture determination system in one embodiment of the present invention.

[0023] Figure 3 This is a block diagram illustrating the hardware and functional block configuration of an animal posture determination device in one embodiment of the present invention.

[0024] Figure 4 This is a block diagram illustrating the configuration of hardware and functional blocks of a sensor device and a position detector in one embodiment of the present invention.

[0025] Figure 5 This is an explanatory diagram showing an example of processing an image obtained by photographing an animal.

[0026] Figure 6 This is a schematic diagram that compares the position of an animal determined based on an image with the position of an animal determined based on identification information from a sensor device.

[0027] Figure 7 This is an explanatory diagram showing an example of appearance judgment items.

[0028] Figure 8 This is a diagram showing a configuration example of a data storage unit for appearance determination.

[0029] Fig. 9 It is a diagram showing a configuration example of a position information storage unit.

[0030] Fig.10 This is a diagram showing a configuration example of an appearance information storage unit.

[0031] Fig.11 This is a diagram showing a configuration example of a processing target data storage unit.

[0032] Fig.12 This is a flowchart showing an example of position information calculation processing executed by the animal posture determination device in one embodiment of the present invention.

[0033] Fig.13 This is a flowchart showing an example of the posture determination process executed by the animal posture determination device in one embodiment of the present invention. DETAILED DESCRIPTION

[0034] Hereinafter, the present invention will be described in conjunction with its embodiments and with reference to the accompanying drawings. In addition, the present invention is not limited to the following embodiments. In addition, the figures referred to in the following description are merely schematically showing shapes, sizes, and positional relationships to the extent that the contents of the present disclosure can be understood. That is, the present invention is not limited to the shapes, sizes, and positional relationships illustrated in the figures.

[0035] (Implementation Method)

[0036] <Animal Appearance Judgment System 1>

[0037] Figure 1 Schematic diagram showing the overall structure of an animal posture determination system 1 according to one embodiment of the present invention. Figure 1As shown, the animal posture determination system 1 of this embodiment has the following functions: based on the image obtained by shooting the posture of a cow C raised by a dairy farmer, for example, the determination result of the predetermined posture determination items, i.e., posture information, is recorded in a time series for each individual animal. The animals to be recorded are not limited to the cow C, but may include all animals that are conducive to the acquisition and use of posture information, including animals such as pigs, sheep, horses, goats, etc. that fall into the category of so-called economic animals, animals raised in zoos, etc., and so-called pets. In this embodiment, four items, "ear position", "eye opening degree", "head position", and "back curvature" are set as posture determination items. The posture determination items are not limited to such items, and may be appropriately set according to the target animal, the purpose of using the posture information, etc.

[0038] Figure 1 FIG. 1 is a plan view schematically showing a rectangular cowshed CB arranged in a cowshed. Figure 1 , a situation in which two cows C1 and C2 are kept in a cowshed CB is shown.

[0039] In the cowshed CB, feeding places F, nursing places M, and drinking places W are provided at appropriate locations, so that the cows C1 and C2 can eat feed, drink milk, and drink water at each location. In addition, the arrangement of these feeding places F, nursing places M, and drinking places W in the cowshed CB is not limited to Figure 1 For example, it can be decided appropriately.

[0040] The cows C1 and C2 are equipped with sensor devices 200, which are equipped with sensor devices for monitoring biological information such as the behavior and heart rate of each cow. Each sensor device 200 is assigned with unique identification information, which is used as the identification information of each cow. The sensor device 200 is provided with a data transmission function, and the data acquired by the sensor device is transmitted together with the aforementioned identification information, for example, by near field communication (NFC).

[0041] An animal posture determination device 100 is installed near a cowshed including a cowshed CB, such as a management building of a dairy cattle owner. The animal posture determination device 100 is an information processing device equipped with the main functions required for animal posture determination in this embodiment. The animal posture determination device 100 receives various data and identification information from a sensor device 200 mounted on the cow C, and is used to determine the position of each cow C in the cowshed CB.

[0042] A position detector 300 is provided in the cattle pen CB. The position detector 300 receives the radio waves transmitted by the sensor device 200 and generates data required for the animal posture determination device 100 to analyze the position of the sensor device 200 as the transmission source, that is, the position of the cows C1 and C2 in the cattle pen CB. Figure 1 In the example of FIG. 1 , two position detectors 300 are installed at appropriate locations in the cattle pen CB to detect the angle of the radio wave from the sensor device 200 and send it to the animal posture determination device 100. The animal posture determination device 100 uses the angle data from each position detector 300 to analyze and obtain the position of the cow C in the cattle pen CB. Figure 1 As shown in the example, for convenience, an X-axis, a Y-axis and a Z-axis orthogonal to the plane coordinate system can be set in the cowshed CB. The animal posture determination device 100 can also be configured to be able to communicate with an external communication network N such as the Internet, a wide area network (WAN), and a local area network (LAN). The structure and function of the animal posture determination device 100, the sensor device 200, and the position detector 300 will be described below with reference to the related diagrams.

[0043] One or more cameras 400 for capturing images of the cows C are provided in the cowshed CB. It is ideal to capture images of the cows C raised in the cowshed CB on a regular basis. Therefore, it is preferred to set the camera 400 in such a manner that it can capture such places in the cowshed CB where the cows C appear regularly (for example, once a day), such as near the feeding place or the drinking place, so that images of the cows C in the cowshed CB can be captured regularly. The camera 400 is, for example, a digital camera having an imaging element such as a charge coupled device (CCD), which captures dynamic images in the cowshed CB or static images at specified time intervals, and sends the image data to the animal appearance determination device 100.

[0044] Figure 2 It is shown in Figure 1The general structure and functions of the animal posture determination system 1. The animal posture determination system 1 of the present embodiment includes an animal posture determination device 100. The animal posture determination device 100 inputs the radio wave reception angle data from the position detector 300 set in the cattle pen CB and the identification information of the sensor device 200. The identification information of the sensor device 200 is included in the data received by the position sensor 300 from the sensor device 200. The animal posture determination device 100 analyzes the input radio wave reception angle data and the setting position of the position detector 300 by the position data analysis function F1, and calculates the position information of the sensor device 200 in the cattle pen CB by the position information calculation function F2. Since the identification information of the sensor device 200 is associated with each cow C, the position information of the sensor device 200 indicates the position of the cow C in the cattle pen CB on which the sensor device 200 is installed.

[0045] On the other hand, the animal appearance determination device 100 obtains the image data of the cowshed CB taken by the camera 400. The obtained image data is analyzed by the image analysis function F3, and the image of the cow C contained in each image is extracted by the animal extraction function F4. At this time, the position of each cow C in the image is calculated. The position of the cow C in the image is converted to the actual position in the cowshed CB based on the installation position, shooting direction and screen angle of the camera 400 that took the image.

[0046] The position information (second position information) based on the signal received from the sensor device 200 is compared with the information (first position information) of the position in the cattle pen CB obtained by transforming the position of the cow C in the image. When it is determined that the position information based on the image and the position information based on the signal from the sensor device 200 have a corresponding relationship, the image is determined to be an image of the cow C equipped with the sensor device 200 that is considered to have a corresponding relationship. As mentioned above, the identification information of the sensor device 200 is corresponding to the individual information of each cow C, so the above-mentioned image is corresponding to the individual information of the cow C. The position information based on the image and the position information based on the signal from the sensor device 200 have a corresponding relationship, which means that the difference (distance) of the position in the cattle pen CB determined by each position information is less than a predetermined threshold value. In other words, when the position in the cattle pen CB represented by the position information based on the image and the position in the cattle pen CB represented by the position information based on the signal from the sensor device 200 are included in a certain range, it is determined that the above-mentioned corresponding relationship exists. The aforementioned threshold value may be appropriately determined according to the target animal, etc. In the case of cattle in the present embodiment, it may be set at, for example, the order of tens of centimeters.

[0047] For the image of the cow C extracted from each image by the animal extraction function F4, the posture determination function F6 performs determination processing from the perspective of its posture. As mentioned above, four items, "ear position", "eye opening degree", "head position" and "back curvature" are set as posture determination items. In the posture determination function F6, quantitative determination is performed on each posture determination item based on the image of the cow C. After the image of the cow C corresponding to the individual information established by the individual recognition function F5 and the determination result based on each cow C image obtained by the posture determination function F6 are associated, they are continuously recorded in a time series by the posture information recording function F7.

[0048] <Animal Appearance Determination Device 100, Sensor Device 200, Position Detector 300>

[0049] Next, the structures and functions of the animal posture determination device 100, the sensor device 200, and the position detector 300 in this embodiment will be described in detail. Figure 3 , Figure 4 These are block diagrams illustrating the hardware and functional block configurations of the animal posture determination device 100 , the sensor device 200 , and the position detector 300 in the present embodiment.

[0050] <<Sensor device 200, position detector 300>>

[0051] First, refer to Figure 4 , the sensor device 200 and the position detector 300 in this embodiment are described. The sensor device 200 is a device that is mounted on an appropriate part of the body of the cow C, converts the movement of the cow C included in the behavior of the cow C into a physical quantity and outputs it. In this embodiment, the sensor device 200 is fixed to a collar and mounted on the neck of the cow C, for example, but can also be configured to be mounted on other parts, such as the ear.

[0052] In this embodiment, the sensor device 200 functions as a terminal device for enabling the position of the sensor device 200 to be acquired by the animal posture determination device 100. Therefore, in this embodiment, the sensor device 200 only needs to have the function of a terminal device that transmits its own identification information. Here, the structure of the sensor device 200 as such a terminal device is described. Figure 4As shown in the example, the sensor device 200 includes a processing unit 210 composed of a processor such as a central processing unit (CPU), and a communication unit 220. The processing unit 210 includes an identification information storage unit 211 storing identification information, i.e., data of the sensor device 200, and an identification information sending unit 212 for sending the data of the identification information to the outside. As the identification information, a serial number inherent to the device constituting the sensor device 200 may be used.

[0053] The communication unit 220 has a function of receiving identification information data from the identification information sending unit 212 of the processing unit 210 and wirelessly transmitting the data by a predetermined communication method. In the present embodiment, the communication unit 220 is configured as a communication module having a communication function based on Bluetooth (registered trademark). The digital data from the identification information sending unit 212 sent by the communication unit 220 is sent as Bluetooth data and received by the animal posture determination device 100. In addition, the Bluetooth data is also received by the position detector 300 provided in the cowshed CB. In addition, the above-mentioned communication method is an example and is not limited thereto.

[0054] The position detector 300 functions as a locator for utilizing the direction detection function included in Bluetooth 5.1, for example. Figure 4As shown, the position detector 300 can be configured to include a processing unit 310 including a processor, an antenna array 320, and a communication unit 330. The processing unit 310 can include a radio wave receiving unit 311 and a receiving angle calculation unit 312 as functional blocks. In Bluetooth 5.1, the angle of arrival of the Bluetooth signal can be obtained by the angle of arrival (AoA) of the radio wave. In the digital signal of Bluetooth 5.1, a monotonic sine wave of a fixed period called constant tone extension (CTE) is included after conventional data communication. The position detector 300 has the following functions: the CTE is received by the radio wave receiving unit 311 via the antenna array 320 including multiple antennas, and the incident angle of the Bluetooth signal is calculated in the receiving angle calculation unit 312 based on the difference in the phase angle of the received wave at each antenna. As described later, the reception angle data calculated by the reception angle calculation unit 312 of the position detector 300 is transmitted to the animal posture determination device 100 via the communication unit 330. In addition, the position detection method for detecting the position of the sensor device 200 is not limited to the method using the Bluetooth function, and other available methods such as the position detection method using a satellite positioning system such as the Global Positioning System (GPS) and the position determination method using radio waves from a Wi-Fi base station can be used.

[0055] <<Animal Appearance Determination Device 100>>

[0056] Next, an embodiment of the information processing device of the present invention, namely, an animal appearance determination device 100, will be described. Figure 3 As shown, the animal appearance determination device 100 is an information processing device such as a server computer or a personal computer, and includes a processing unit 110, a storage unit 120, an input / output unit 130, a data interface (data IF (Interface)) unit 140, and a communication unit 150. The processing unit 110, the storage unit 120, the input / output unit 130, the data IF unit 140, and the communication unit 150 are connected by a bus which is an internal communication line not shown in the figure.

[0057] The processing unit 110 is a computing device composed of a processor such as a CPU, and reads various programs and data from a storage unit 120 described later to execute them, thereby realizing the functions of the animal appearance determination device 100. In this embodiment, the processing unit 110 performs data processing of each functional unit of the image data receiving unit 111, the image data analyzing unit 112, the position data receiving unit 113, the position data analyzing unit 114, the individual identification unit 115, the appearance determination unit 116, the appearance information recording unit 117, and the appearance information output unit 118. The operation of each functional unit will be described below.

[0058] The storage unit 120 is a storage area for various programs and various data for the hardware group to function as the animal posture determination device 100, and can be composed of a read-only memory (ROM), a random access memory (RAM), a flash memory, a semiconductor drive (SSD) or a hard disk (HDD), etc. Specifically, the storage unit 120 stores programs for the processing unit 110 to execute various functions of the present embodiment (control programs of the animal posture determination device 100), various parameters, data used in individual identification and posture determination, data related to the cattle C as the processing object, operation input data input from the outside, and generated posture information data.

[0059] The input / output unit 130 includes various input devices such as a keyboard, a mouse, a touch panel, and a microphone for inputting data from the outside to the animal posture determination device 100 , and output devices such as a monitoring display and a speaker for outputting and displaying posture information data.

[0060] The data IF unit 140 has a function of controlling data communication among the processing unit 110 , the storage unit 120 , the input / output unit 130 , and the communication unit 150 .

[0061] The communication unit 150 is a communication module for transmitting and receiving various data between the sensor device 200 and the position detector 300, and is constituted as hardware such as a network interface card (NIC). The communication unit 150 can also be used to communicate with the animal posture determination device 100. Figure 1 In the present embodiment, although omitted, various control functions of the camera 400 installed in the cattle pen CB may be provided in the animal appearance determination device 100 .

[0062] Next, the functions of the animal appearance determination device 100 realized by each program executed by the processing unit 110 will be described.

[0063] The image data receiving unit 111 has a function of receiving image data captured by a camera 400 installed in the cowshed CB. The camera 400 is typically a digital camera having an imaging element such as a CCD and a required resolution, but is not limited thereto and may capture still images at a predetermined time interval.

[0064] The image data analysis unit 112 has the following functions: using image analysis data, it analyzes the captured image data in the cowshed CB received by the image data receiving unit 111, determines whether the image contains the cow C, and calculates the position of the cow C in the cowshed CB based on the position in the image. Figure 5 An example of an image of the inside of the cattle pen CB taken by the camera 400 is shown in FIG. Figure 5 In the image shown in the example, three cows C1 to C3 are photographed. The image data analysis unit 112 uses a learning model that has been previously learned using image data of various cows stored in the image analysis data storage unit 121 of the storage unit 120 to investigate whether a cow is photographed in each image, and for example Figure 5 As illustrated in the figure, a rectangle (x1, y1), (x2, y2), (x3, y3), and (x4, y4) is used to add a border to the area in the image that is judged to be a cow. (x1, y1),... represent the coordinates of the vertices of the rectangle in the screen coordinate system in the image. The training data used in the learning is annotation data obtained by determining the area where the cow is photographed for image data containing a variety of images of cows. The image data judged to contain cow C is used as data required for the appearance judgment described later. On the other hand, with respect to the image judged to contain cow C, the image data analysis unit 112 calculates the position of cow C in the cowshed CB based on the area of ​​cow C in the image. The area occupied by the cow C in the image, i.e., the position coordinates of the aforementioned rectangular vertices in the screen coordinate system, are projected and transformed to the coordinate system of the real space in the cowshed CB (world coordinate system) based on the setting position coordinates of the camera 400, the angle representing the shooting direction, such as the angle (depression angle) formed by the shooting direction of the camera 400 and the horizontal plane, and the angle relative to the coordinate axis in the cowshed CB, and the screen angle of the camera 400, so that the position occupied by the cow C in the image in the cowshed CB as the real space can be calculated. Specifically, a transformation matrix is ​​prepared in advance for each camera 400 to parallel translate and rotate the rectangle representing the position of the cow C in the screen coordinate system and transform it into a position in the world coordinate system. The position of the cow C calculated based on the image data is used for the individual recognition processing of each cow C described later.

[0065] The position data receiving unit 113 has a function of receiving position data from the position detector 300. As described above, the position data is, for example, the position coordinates of each position detector 300 in the cattle pen CB and the radio wave reception angle data from the sensor device 200 calculated by the reception angle calculation unit 312 of each position detector 300. The position coordinate data of the position detector 300 may also be stored in the storage unit 120 as a parameter in advance.

[0066] The position data analysis unit 114 calculates the position of the sensor device 200 in the cattle pen CB using the position coordinate data and the radio wave reception angle of the position detector 300 received and acquired by the position data receiving unit 113. Figure 1 As shown, if two sets of position detectors 300 are used, the position of the sensor device 200 as the radio wave transmission source can be obtained as the intersection of straight lines drawn from the positions of each position detector 300 according to the radio wave reception angle. According to the need for the position measurement accuracy of the sensor device 200, additional position detectors 300 can also be provided. By matching the obtained position of the sensor device 200 with the plane coordinates pre-set in the cattle pen CB, it is possible to determine where the cow C equipped with the sensor device 200 is in the cattle pen CB. The position of each sensor device 200 in the cattle pen CB calculated by the position data analysis unit 114 is used for the individual identification processing of each cow C described later. In addition, the position detection method of the above-mentioned sensor device 200 is an example, and other methods can also be used, such as calculating the position coordinates based on the positioning satellite signal of GPS, and using the radio waves from the Wi-Fi base station.

[0067] The individual recognition unit 115 has the function of performing individual recognition of each cow C captured in the image by comparing the estimated position of the cow C in the cowshed CB calculated by the image data analysis unit 112 based on the position of the cow C in the image included in the image data with the position of each sensor device 200 in the cowshed CB calculated by the position data analysis unit 114 based on the data from the position detector 300. As described above, the sensor device 200 attached to each cow C is given unique identification information, namely, the sensor device ID, which is received by the appearance determination device 100 via the position detector 300. Since the sensor device ID is associated with the individual identifier of each cow C, i.e., the individual ID, by comparing the position of the cow C estimated based on the image data with the position of the sensor device 200, it is possible to determine whether the difference between the position of the cow C estimated based on the image data and the position of the sensor device 200 is below a specified threshold (whether the two positions are close enough to be contained within a certain range within the cowshed CB), and to associate the cow C in the image with the individual ID (this association will be described below with respect to the structure of the processing object data storage unit). Figure 6, the position of the cow C in the cowshed CB estimated from the image data and the position of the cow C calculated based on the received radio waves from the sensor device 200 are compared and schematically shown. Figure 6 In the figure, the star-shaped mark indicates the position of the cow C estimated from the image data, and the circle-shaped mark indicates the position of the cow C calculated based on the received radio waves from the sensor device 200. The number of cows C based on the image data is less than the number of the sensor device 200, which means that some of the cows C in the cowshed CB cannot be detected from the image taken by the camera 400. Figure 6 In the figure, three cows (individual ID = 001 to 003) with round marks with slashes are associated with the positions of cows C (star marks) estimated from the image data. In this way, the individual identification unit 115 can determine which individual cow C is captured in the image. The position information determined for each cow C is recorded as a record associated with the recording date and time, the individual ID of the cow C, and the position in the cowshed CB. Fig. 9 The position information storage unit 123 is illustrated in FIG.

[0068] The appearance determination unit 116 has a function of quantitatively determining the appearance of the cow C based on the image of the cow C included in the image data captured by the camera 400. Quantitative determination of the appearance of the cow C can be utilized in health management of each cow C, disease prevention, and the like. Figure 7 , the posture determination items determined in the posture determination in the present embodiment are illustrated. In the present embodiment, as an example, the posture determination items of a calf are shown, and the four items of "ear position", "eye opening degree", "head position" and "back curvature" are respectively determined with the three levels from the best judgment, i.e., posture score 0 to posture score 2. In each item, the higher the ear is, the better, the wider the eyes are opened, the better, the higher the head position is, the better, and the straighter the back is without curvature, the better. The determination items can also be appropriately changed according to the object of posture determination, such as a multiparous cow. The posture determination unit 116 performs determination processing on the image data of the cow C to which the image data analysis unit 112 adds a frame in the image, using the posture determination data stored in the posture determination data storage unit 122. Figure 8 2 shows a configuration example of the appearance determination data storage unit 122. Figure 8The posture determination data illustrated in the example stores image data containing the cow C, the posture category of the cow C contained in the image such as the face and the side of the body, determination items such as the degree of eye opening, and the posture score assigned to the determination items of the corresponding image data. For example, the posture determination unit 116 can compare the image of the cow C as the determination object with the posture determination data, and determine the posture score of the determination object image based on the similarity of the determination items in the image. The posture determination method is not limited to this, and the following methods can be used:

[0069] From the images accumulated for a large number of cows, quantitative indicators are derived in advance based on the judgment items such as "ear position" and "eye opening degree", and the appearance judgment is performed based on the quantitative indicators. For example, in the case of "ear position", the judgment criterion is quantified in such a way that an appearance score can be assigned based on the drooping state of the ear from the horizontal line (the drooping angle of the ear).

[0070] The posture determination is performed on a new image of the cow C using the learned model that has learned the relationship between the image of the cow C's posture and the posture score of each determination item using the posture determination data.

[0071] The appearance information recording unit 117 has a function of recording the appearance determination result of each cow C generated by the appearance determination unit 116 as appearance information in a time series. The appearance information is recorded in the appearance information storage unit 124 of the storage unit 120 . Fig.10 2 shows a configuration example of the appearance information storage unit 124. Fig.10 In the appearance information storage unit 124 illustrated in FIG. 1 , the year, month, day and time when the appearance information is recorded, the individual ID of the cow C, the appearance determination items and the appearance score are recorded in association with each other.

[0072] The appearance information output unit 118 has the following functions: extracting appearance information data from the appearance information storage unit 124 under a predetermined condition by input operation from the input / output unit 130, etc. and outputting the data. For example, by extracting the appearance score corresponding to a specific appearance judgment item of a cow C having a specific individual ID and outputting the data as time series data, it can be used to capture changes in the health status of the cow C, signs of diseases or specific events (weaning, birth, etc.).

[0073] The processing target data storage unit 125 stores individual information about each cow C that is a target for appearance determination. Fig.11 2 shows a configuration example of the processing target data storage unit 125. Fig.11In the processing object data storage unit 125 illustrated in the example, the inherent identification information of the sensor device 200, i.e., the sensor device ID, the type of cattle indicating the type of cattle, the gender, age, health status, and medical history items are recorded, corresponding to the individual ID of each cow C that is assigned as an identifier to the object of posture determination. The sensor device ID is attached to the sensor data sent from each sensor device 200 installed on the cow C to the animal posture determination device 100, so that it is possible to identify which sensor device 200 has obtained the received sensor data. Since the sensor device ID is also received in the position detector 300, the position of a specific sensor device 200 can be associated with the sensor device ID. In addition to recording the health status as a determination value as shown in the figure, it can also be recorded specifically in text. Medical records can also be recorded in an appropriate recording format. The record items of the processing object data storage unit 125 are not subject to Fig.11 The constraints of the example can be determined appropriately.

[0074] <Data processing by the animal appearance determination device 100>

[0075] Next, the data processing performed by the animal posture determination device 100 in this embodiment is described. The animal posture determination device 100 mainly performs the following processing: position information calculation processing for detecting the position of the animal (a cow in this embodiment) captured in the image; and posture determination processing for determining the posture of the animal based on the image.

[0076] <<Position information calculation processing>>

[0077] exist Fig.12 1 shows an example of the processing flow of the position information calculation process executed by the animal posture determination device 100 in this embodiment. The position information calculation process is started when the animal posture determination device 100 is powered on and started, and is executed once at fixed time intervals.

[0078] The image data receiving unit 111 of the animal appearance determination device 100 receives the captured image data captured by the camera 400 from the camera 400 (step S11 ).

[0079] The image data analysis unit 112 analyzes the image data received from the camera 400 using the image analysis data stored in the image analysis data storage unit 121 (step S12 ), and extracts the image region of the cow C in the image (step S13 ).

[0080] The image data analysis unit 112 transforms the area occupied by the cow C in the image into a position in the spatial coordinate system using the relationship between the screen coordinate system, the camera coordinate system based on the position and shooting direction of the camera 400, and the spatial coordinate system of the real world, and calculates the position of the cow C in the cowshed CB (step S14).

[0081] On the other hand, the position data analysis unit 114 calculates the position of each sensor device 200 as the position of the cow C based on the arrival angle of the radio wave received by the position data receiving unit 113 from the position detector 300, and compares it with the position of the cow C estimated based on the image data in step S14 (step S15). Thus, the position of the cow C estimated based on the image data is verified by the position of the sensor device 200 compared, that is, the position of the cow C equipped with the sensor device 200, so as to realize the individual identification of each cow C photographed in the image. In addition, the verification of the position of the cow C can be implemented as follows, that is, first, the first position information indicating the position of the cow C estimated based on the image data is assumed to be the position of the cow C in the cowshed CB, and is implemented based on whether there is a corresponding position of the sensor device 200, in other words, whether the position coordinates of any sensor device 200 are detected within a certain range from the position coordinates of the cow C estimated based on the image data. Alternatively, the position of the sensor device 200 may be first used as the position of the cow C in the cowshed CB, and the position of the cow C may be verified based on whether there is first position information indicating the position of the cow C estimated based on the image data corresponding to them, in other words, whether there are position coordinates of the cow C estimated based on any image data within a certain range from the position coordinates of the sensor device 200.

[0082] The position data analyzing unit 114 records the position of the cow C identified in step S15 in the position information storage unit 123 (step S16 ).

[0083] The processing unit 110 determines whether a data processing end command is received via the input / output unit 130 or the communication unit 150 (step S17). If it is determined that the end command is not received (step S17: NO), the processing unit 110 returns to step S11 and continues the execution of the position information calculation process. If it is determined that the end command is received (step S17: YES), the processing unit 110 ends the position information calculation process.

[0084] <<Appearance judgment processing>>

[0085] exist Fig.131 shows an example of the processing flow of the posture determination process executed by the animal posture determination device 100 in this embodiment as a flowchart. The posture determination process is started when the animal posture determination device 100 is powered on and started, for example, and is executed once at fixed time intervals.

[0086] The posture determination process performed by the animal posture determination device 100 is repeatedly performed for each sensor device 200 mounted on the cow C during the operation of the animal posture determination device 100 (a loop of step S21). The posture determination unit 116 obtains the image of the cow C extracted from the image data by the image data analysis unit 112 (step S22). The posture determination unit 116 extracts images related to the posture determination items from the acquired image of the cow C (step S23). Specifically, the posture determination unit 116 investigates whether the acquired image contains the head, face, and body side of the cow C, and whether the "head", "ears", "eyes", and "back" related to the posture determination items can be recognized. This determination can use, for example, a learning model that has been learned by a large amount of training data including the "head", "ears", "eyes", and "back" of the cow C. In addition, with regard to the acquisition of images of the cow C, it is preferred to uniformly capture the body parts related to the posture determination items for each cow C. From this point of view, for example, it is preferable to set a plurality of cameras 400 in the cowshed CB where the cows C are raised, and shoot the cowshed CB from a plurality of different directions. Alternatively, it is also conceivable to configure, for example, the camera 400 to emit a sound, light, etc. that causes the cows C to turn their attention to the camera 400 during shooting.

[0087] The posture determination unit 116 compares the image related to the extracted posture determination item with the posture determination data read from the posture determination data storage unit 122 (step S24). Specifically, as described above, the posture determination unit 116 investigates the similarity with the posture determination data for the four items of "ear position", "eye opening degree", "head position" and "back curvature". Alternatively, a process of determining a new image of Cow C is performed by learning a completed model, wherein the completed model learns the relationship between the image of Cow C's posture and the posture score of each determination item by comparing the posture determination data with the quantitative indicators derived in advance based on each determination item.

[0088] The appearance determination unit 116 determines an appearance score for each determination item based on the comparison result of step S24 (step S25 ), and records it in the appearance information storage unit 124 as appearance information via the appearance information recording unit 117 (step S26 ).

[0089] By executing the above position information calculation processing and posture determination processing, the posture scores related to the posture determination items of each cow C, namely "ear position", "eye opening degree", "head position" and "back curvature" are accumulated in time series.

[0090] <Utilization of appearance determination results by the animal appearance determination device 100>

[0091] As described above, according to the animal posture determination device 100 in this embodiment, the determination index and posture score related to the "ear position", "eye opening degree", "head position" and "back bending" selected as the posture determination items are accumulated in a time series for each cow C. Therefore, by analyzing the temporal changes of the posture scores related to, for example, the "ear position", "eye opening degree", "head position" and "back bending" and the correlation between the health status, illness, estrus, birth, weaning and other events of the cow C and making the animal posture determination device 100 learn, the animal posture determination device 100 in this embodiment can be used for the purpose of detecting the signs of changes in the health status of the cow C, illness, estrus, birth, weaning and other events. In addition, in this embodiment, the cows C in the cowshed CB are photographed regularly, and each cow C is basically in a natural state. When the images are continuously taken in this state, for example, when a certain cow C is in poor physical condition, moves little, or has almost no posture change, it can be inferred that a small number of images that can be used as posture determination items or a small number of images related to a specific posture determination item are found for the specific cow C. That is, it can be considered that the change in health status of the cow C can be predicted by the temporal change in the number of images related to the posture determination item, rather than just the temporal change in the posture score.

[0092] According to the above-described embodiment, the following effects are achieved.

[0093] An animal posture determination device 100 in one embodiment of the present invention comprises: an image data analysis unit 112, which acquires a captured image of a cow C raised in a cowshed CB, and acquires the position of the cow C in the aforementioned breeding area calculated based on the image of the cow C in the aforementioned captured image as first position information; a position data analysis unit 114, which acquires identification information and second position information indicating the position of the cow C in the cowshed CB from a sensor device 200 provided on the cow C; and an individual identification unit 115, which associates the aforementioned first position information with the aforementioned identification information related to the aforementioned second position information and records the same when it is determined that the difference between the first position information and the second position information is less than a prescribed threshold value.

[0094] In this way, based on the captured image of the cow C in the cowshed CB, the position of the cow C in the cowshed CB can be determined with high accuracy, thereby enabling individual recognition of the cow C related to the captured image.

[0095] Optionally, the animal posture determination device 100 has a posture determination unit 116, which determines the posture determination items pre-set regarding the posture of the animal based on the image of the aforementioned animal taken, and uses its determination result as the posture information of the animal, obtains the aforementioned first position information calculated based on the image of the aforementioned animal that is the object of the aforementioned determination, and associates the aforementioned identification information corresponding to the aforementioned second position information associated with the first position information with the aforementioned posture information and records it.

[0096] In this way, the changes in the appearance and posture of each cow C identified by the individual can be recorded in a time series.

[0097] Alternatively, the appearance determination unit 116 determines the appearance of the cow C using a learned model that has learned the relationship between the image data acquired for the cow C and the determination results of the aforementioned appearance determination items that are set in advance, that is, the appearance information.

[0098] In this way, the appearance of each cow C can be automatically determined.

[0099] Alternatively, the appearance determination unit 116 determines the appearance of the cow C based on the acquired image of the cow C and a determination criterion of features in the image of the cow C set in advance for the appearance determination item.

[0100] In this way, the appearance of each cow C can be automatically determined.

[0101] The position information to be compared with the position estimated based on the image of the cow C can be calculated based on the arrival direction of the radio wave transmitted from the sensor device 200 installed on the cow C.

[0102] In this way, the position of the sensor device 200, that is, the cow C, can be detected by using the radio waves used in the data transmission from the sensor device 200.

[0103] Optionally, the appearance information of each individual animal recorded in time series by the appearance determination unit is used to predict a period of at least one of disease, estrus, delivery, growth, and weaning of the animal.

[0104] In this way, it is possible to timely and without omission know things such as the animal's illness, estrus, birth, growth, and weaning based on the animal's appearance.

[0105] The above series of processing can be performed by hardware or software. In other words, Figure 2 , Figure 3 The functional structure is merely an example and is not particularly limited. That is, as long as the animal appearance determination device 100 has a function that can execute the above series of processing as a whole, the function blocks used to implement the function are not particularly limited. Figure 2 , Figure 3 In addition, a functional block can be composed of a hardware unit, a software unit, or a combination of them. The functional structure in the present embodiment is realized by a processor that performs arithmetic processing. The processor that can be used in the present embodiment includes, in addition to processors composed of various processing devices such as single processors, multi-processors, and multi-core processors, processors composed of these various processing devices and processing circuits such as application specific integrated circuits (ASIC) or field programmable gate arrays (FPGA).

[0106] When a series of processes are performed by software, the program constituting the software is installed from a network or a recording medium to a computer, etc. The computer may also be a computer embedded in dedicated hardware. In addition, the computer may also be a computer capable of performing various functions by installing various programs, such as a general-purpose personal computer.

[0107] The recording medium containing such a program is composed not only of removable media such as Universal Serial Bus (USB) memory distributed separately from the device body in order to provide the program to the user, but also of recording media provided to the user in a state pre-embedded in the device body. The removable medium is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is composed of, for example, a compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a Blu-ray (registered trademark) Disc, etc. The magneto-optical disk is composed of a mini-disk (MD), etc. In addition, the recording medium provided to the user in a state pre-embedded in the device body is composed of, for example, a ROM on which the program is recorded or a hard disk contained in the storage unit 120.

[0108] In the present specification, the steps describing the program recorded in the recording medium naturally include processing performed in time series according to the order, and also include processing that is not necessarily processed in time series but is executed in parallel or independently.

[0109] Several embodiments of the present invention have been described above, but these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention may adopt various other embodiments, and the above-mentioned embodiments and the various structures of the modified examples may be combined. Furthermore, various changes such as omission or substitution may be made without departing from the scope of the present invention. These embodiments and their modifications are included in the scope or gist of the invention described in this specification, etc., and are included in the invention described in the claims and the scope of their equivalents.

[0110] Reference numerals

[0111] 1Animal Appearance Determination System

[0112] 100 Animal Appearance Determination Device

[0113] 110 Processing Department

[0114] 115 Individual Identification Department

[0115] 116 Appearance Judgment Department

[0116] 120 Storage Department

[0117] 122 Appearance judgment data storage unit

[0118] 124 Appearance Information Storage Department

[0119] 200 sensor devices

[0120] 300 position detector

[0121] 400 Camera

Claims

1. An information processing device comprising: A first position information acquisition unit, which acquires a photographed image including an animal kept in a predetermined breeding area, and acquires a position of the animal in the breeding area calculated based on the image of the animal in the photographed image as first position information; a second position information acquisition unit that acquires identification information of the animal and second position information indicating the position of the animal in the breeding area from a terminal device installed in the animal; and The individual identification unit associates the first location information with the identification information related to the second location information and records the associated information when determining that the difference between the first location information and the second location information is less than a predetermined threshold.

2. The information processing device according to claim 1 comprises a posture determination unit, which determines the posture determination items pre-set regarding the posture of the animal based on the image of the animal taken, and uses the determination result as the posture information of the animal, obtains the first position information calculated based on the image of the animal as the object of the determination, and associates the identification information associated with the first position information with the posture information and records them.

3. The information processing device according to claim 2, wherein: The appearance determination unit determines the appearance of the animal using a learned model that has learned a relationship between an image acquired of the animal and appearance information that is a determination result regarding the appearance determination item set in advance.

4. The information processing device according to claim 2, wherein: The appearance determination unit determines the appearance of the animal based on the image acquired for the animal and a determination criterion of features in the image of the animal set in advance for the appearance determination item.

5. The information processing device according to any one of claims 1 to 4, wherein: The second position information is calculated based on the arrival direction of radio waves transmitted from a terminal device installed on the animal.

6. The information processing device according to any one of claims 2 to 5, wherein: The appearance information of each individual of the animal recorded in time series by the appearance determination unit is used to predict a period of at least one of disease, estrus, delivery, growth, and weaning of the animal.

7. An information processing method, wherein an information processing device performs the following processing: Acquire a captured image of an animal kept in a predetermined feeding area, and acquire a position of the animal in the feeding area calculated based on the image of the animal in the captured image as first position information; acquiring identification information of the animal and second location information indicating the location of the animal in the breeding area from a terminal device installed at the animal; and When it is determined that the difference between the first location information and the second location information is equal to or smaller than a predetermined threshold, the first location information and the identification information associated with the second location information are associated with each other and recorded.

8. A program that causes an information processing device to execute the following processing: Acquire a captured image of an animal kept in a predetermined feeding area, and acquire a position of the animal in the feeding area calculated based on the image of the animal in the captured image as first position information; acquiring identification information of the animal and second location information indicating the location of the animal in the breeding area from a terminal device installed at the animal; and When it is determined that the difference between the first location information and the second location information is equal to or smaller than a predetermined threshold, the first location information and the identification information associated with the second location information are associated with each other and recorded.

Citation Information

Patent Citations

  • Behavior amount calculation program, behavior amount calculation method, behavior amount calculation device and animal monitoring system

    JP2018148843A

  • Livestock disease management system, livestock disease management server, livestock disease management method, and livestock disease management program

    JP2021135956A

  • Livestock information management system, livestock barn, livestock information management program, and livestock information management method

    CN111107740A

  • Livestock registration system and livestock registration method

    EP3342280A1

  • Estimation system for estimating health condition of animal, estimation method, and learning model generation method

    JP2021136868A