Feeding management system and feeding management method

Abstract

To provide a feeding management system contributing to overall five freedoms of animal welfare by contributing to other freedoms while feeding livestock in a manner of contributing to freedom of expressing normal behavior of the livestock.SOLUTION: A feeding management system 1 for a plurality of livestock movably bred in a feeding area, comprises an imaging device 30, a position measuring device 40 for identifying for every individual of the plurality of livestock and measuring position information, a storage part 110 for storing an area map of the feeding area, and an activity state estimation device 10 for estimating an activity state of the livestock. The activity state estimation device 10 includes an in-area position calculation part 123 capable of calculating an in-area position of an individual in the feeding area by using the position information and the area map, a livestock identification part 124 for identifying an individual captured in an image, an attitude estimation part 125 for estimating an attitude of the individual using the image, and an activity state estimation part 126 for estimating the activity state of the individual using the attitude and the in-area position of the individual.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present disclosure relates to a livestock breeding management system and a breeding management method. [Background technology]

[0002] In recent years, digital transformation has been promoted through the introduction of computers and surveillance cameras to address the labor shortage in the livestock industry, increase the efficiency of livestock farming operations, and monitor and control the livestock rearing environment.

[0003] Patent Document 1 describes a determination device that measures the frequency with which sows kept in stalls lie down per unit time and determines whether the sows are in heat based on the frequency within a set period of time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 045033 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in recent years, there has been a growing demand for animal welfare (AW) for livestock. For example, when the livestock is pigs, sows are often kept in fenced areas called gestation stalls where they are barely able to move, as described in the above-mentioned Patent Document 1. There are movements to abolish this type of housing style as it is seen as restricting the freedom of livestock.

[0006] On the other hand, once sows that were previously kept in gestation stalls are allowed to move freely within the pig pen, tracking and managing them becomes difficult.

[0007] Therefore, one objective of the present disclosure is to contribute to all five freedoms of animal welfare by raising livestock in a manner that contributes to the freedom of livestock to express normal behavioral patterns while also contributing to other freedoms. Note that problems that are obvious to a person skilled in the art and can be read from the embodiments and their explanations that are characteristic of the present disclosure and are described in the specification, drawings, etc. of the present disclosure may also be problems to be solved by a divided invention when a divisional application based on the present disclosure is filed. [Means for solving the problem]

[0008] In order to achieve the above-mentioned objective, the husbandry management system disclosed herein is a husbandry management system for multiple livestock kept in a specific keeping area in a movable state, and includes an imaging device that takes images within the keeping area, a position measurement device that identifies each of the multiple livestock kept in the keeping area and measures position information, a memory unit that stores an area map of the keeping area, and an activity state estimation device that estimates the activity state of the livestock.The activity state estimation device includes an area position calculation unit that can calculate the intra-area position of an individual within the keeping area using the position information and area map information, a livestock identification unit that identifies an individual captured in the image, a posture estimation unit that estimates the posture of the individual using the image, and an activity state estimation unit that estimates the activity state of the individual using the individual's posture and intra-area position information.

[0009] In addition, the husbandry management method disclosed herein is a husbandry management method for multiple livestock kept in a specific husbandry area in a movable state using a husbandry management system, and includes the following steps: an imaging step in which an imaging device captures an image within the husbandry area; a position measurement step in which a position measurement device identifies each of the multiple livestock kept in the husbandry area and measures position information; a sensing data acquisition step in which a sensing data acquisition unit of an activity state estimation device acquires an image and position information; a livestock identification step in which a livestock identification unit identifies an individual captured in the image; a posture estimation step in which a posture estimation unit estimates the posture of the individual using the image; a possible intra-area position calculation step in which a position calculation unit calculates the intra-area position of the individual within the husbandry area using the position information and area map information stored in a memory unit; and an activity state estimation step in which an activity state estimation unit estimates the activity state of the individual using the individual's posture and intra-area position information. Effect of the Invention

[0010] According to the present disclosure, raising farm animals in a manner that contributes to the freedom of farm animals to express their normal behavioural patterns whilst also contributing to other freedoms, can contribute to all five freedoms of animal welfare. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a network configuration and an overview of a breeding management system according to the present disclosure. [Diagram 2] 1 is a block diagram showing a configuration of an activity status estimation device; [Diagram 3] FIG. 1 is a diagram illustrating a livestock image and a livestock detection / identification method using the image. [Figure 4] FIG. 1 is a diagram showing an example of a livestock image; [Diagram 5] A diagram illustrating instance segmentation. [Figure 6] FIG. 1 is a diagram showing an example of livestock position measurement; [Figure 7] FIG. 1 is a diagram illustrating an example of livestock activity state estimation; [Figure 8]FIG. 1 is a diagram illustrating an example of livestock activity state estimation; [Figure 9] FIG. 1 is a sequence chart showing a method of using the present disclosure. [Figure 10] FIG. 1 is a diagram illustrating an example of livestock activity state estimation; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An overview will be provided in advance to facilitate understanding of the invention according to the present disclosure and to facilitate a smooth examination. In the conventional rearing method using stalls, particularly in the single-household method using "gestation stalls," sows were kept in small areas separated by individual sows, where they could barely move. In this case, the position of the sows did not change, and the use of gestation stalls made it easy to identify the sows reared there, and it was easy to manage each sow according to its condition, for example, to grasp the amount of feed given via an automatic feeder, to grasp the physical condition, and to take images using a fixed-point camera.

[0013] However, with regard to animal welfare, the introduction to the OIE (World Organization for Animal Health) guidelines lists five freedoms as guidelines: (1) freedom from hunger, thirst and malnutrition, (2) freedom from fear and distress, (3) freedom from physical and thermal discomfort, (4) freedom from pain, injury and disease, and (5) freedom to express normal behavioral patterns.

[0014] Group rearing of sows in free stalls has been proposed as a way to guarantee part of the "freedom to express normal behavioral patterns," one of these five freedoms. The free stall method is a method of rearing multiple pigs in a relatively large area. It has been found that group rearing in free stalls reduces stress on sows and strengthens their legs, which has a positive effect on birth. On the other hand, it increases the burden of monitoring on the breeder. With multiple pigs moving around freely in a jumble, it becomes difficult to track and identify each individual, and it is not easy to obtain and manage detailed data on each pig as when they are kept alone.

[0015] This makes it difficult to grasp the feeding, water supply, and physical condition of each pig, and makes it difficult to grasp whether the livestock are obtaining sufficient nutrition and whether they are experiencing problems such as injuries and illnesses. This makes it impossible to guarantee the other five freedoms, making it more difficult to achieve animal welfare as a whole. Therefore, the purpose of this disclosure is to contribute to all five freedoms of animal welfare by raising livestock in a manner that contributes to the freedom to express normal behavioral patterns of livestock while also contributing to the other freedoms.

[0016] Specifically, the feeding mode to which the feeding management system of the present disclosure can be applied is, for example, the group feeding method such as the above-mentioned free stall. The group feeding method such as the free stall may include a free access stall method in which an open individual stall is installed in the free stall area where livestock can freely enter and exit one by one and secure a place to stay, and an ESF (electric sow feeding) method in which only one livestock can access and is fed. Generally, piglets are often kept in groups, but sows have often been kept individually in the above-mentioned gestation stall. However, since pigs are social animals and prefer to live in groups, allowing even sows to move freely within a certain size of feeding area contributes to the "freedom to express normal behavior patterns."

[0017] Then, how the sow that moves freely in a specific rearing area eats and drinks water is tracked by a position measuring device described later, and the image is analyzed by an imaging device, and the activity state of the individual is estimated and understood from the position and posture obtained from them and the relationship with the facilities in the area map of the rearing area. The "activity state" here refers to a qualitative parameter in the present disclosure that represents the behavior of livestock, such as drinking water and eating, and for them, the frequency, interval or number of occurrences, type time, or a combination of these statistics can be calculated. If the number of times each individual drinks water or eats can be understood, the breeder can improve the rearing environment based on this. In addition, the weight and body condition score can also be understood by analyzing the captured livestock images. Note that the weight can be estimated using the technology disclosed in Japanese Patent Publication No. 2021-132634 and the like, and will not be described in detail here. As a result, the activity state obtained by the rearing management system of the present disclosure has an aspect that contributes to "freedom from hunger, thirst and malnutrition".

[0018] Furthermore, the activity state of sows moving freely within a specific rearing area is analyzed from their positions and postures obtained by an imaging device and a position measuring device, which will be described later. Examples of the activity state mentioned here include a state of dozing and a state of loitering. These are relaxed states, which are perceived as positive, and are data that indicate that the pigs are not feeling fear or pain. On the other hand, negative states include a state of fighting, a state of making high-pitched noises, and a state of abnormal posture, and pigs in these activity states may be feeling fear, stress, or pain. Sows are reorganized into groups according to the time of birth, etc., and fights may occur between livestock during the reorganization. Such states need to be detected as abnormalities or signs of abnormalities, but human caretakers have had to monitor them. According to the rearing management system, these states can be identified by the position and posture estimated based on the data automatically acquired by the device, or by acquiring additional data as necessary and further estimating them, and individual pigs can be identified and pigs that are feeling fear or pain, or pigs that are injured or sick can be isolated or protected. As a result, the activity state obtained by the husbandry management system disclosed herein has aspects that contribute to "freedom to express normal behavioral patterns," "freedom from fear and distress," and "freedom from pain, injury and disease."

[0019] In addition, the above-mentioned activity state estimation can be used to identify pigs that are shivering from cold or pigs that are breathing abnormally. For example, by adding temperature data measured by a thermometer as additional data, the reason why each livestock is in such an activity state can be estimated by explaining it from the temperature. Furthermore, by using an image as additional data, abnormal hair loss and skin discoloration can be detected, and the estimation accuracy can be improved. By adding gas concentration data detected by ammonia or other gas sensors, it is possible to more accurately detect an unpleasant environment caused by a foul odor generated from feces and urine. By improving such an environment or identifying and isolating pigs in an abnormal state, the activity state obtained by the feeding management system of the present disclosure has an aspect of contributing to "freedom from physical and thermal discomfort."

[0020] The above is an overview of the present disclosure, and specific configurations and embodiments will be described below with reference to the drawings. Note that the embodiments of the present disclosure described below will be mainly described using sows (female pigs), which have often been kept individually, as an example, but they can also be applied to boars and piglets (meat pigs and fattening pigs), which have traditionally been kept in groups, and can also be applied to animals that may be kept in groups, such as cows, sheep, goats, and some horses, by those skilled in the art using their normal creative abilities to the extent of design changes and making modifications to the required extent.

[0021] The feeding management system 1 in the present disclosure is an information processing system that provides a livestock feeding management service that supports the management of livestock feeding information in the livestock feeding business carried out by a user.

[0022] The users are mainly livestock breeders. A breeder is an individual or corporation that manages a farm that raises livestock, an employee of the farm, etc., and is also called a livestock farmer, livestock breeder, or livestock breeder. If the livestock is a pig, the user is someone who operates a pig farming business, and is also called a pig farmer. A user may also be a third party such as a company or organization that provides services to breeders.

[0023] In the embodiments, terms for livestock in general and terms for livestock specifically being pigs can be interpreted as follows: A barn, which is a facility where livestock products are raised on a farm, is also called a pigpen when the livestock are pigs. A livestock pen, which is a unit of partitioned sections within a barn, is also called a pigpen when the livestock are pigs. Note that the term sow in this disclosure is merely used for convenience in accordance with the habits of those skilled in the art for ease of understanding of the embodiments. Depending on the time period, a female pig in the period from birth to weaning the piglets is called a sow in contrast to piglets, and a female pig in the non-parturient period and the period from weaning the piglets to the next birth is also called a breeding sow.

[0024] 1 is a system configuration diagram of a livestock breeding management system according to an embodiment of the present disclosure. The breeding management system 1 includes an activity status estimation device 10, a user terminal 20, an imaging device 30, a position measurement device 40, and various other additional sensors 50, all of which are connected via a network NW. Here, the activity status estimation device 10 and the user terminal 20 do not necessarily have to be installed at a site such as a farm / pigpen / pig house. On the other hand, the imaging device 30, the position measurement device 40, and the additional sensor 50 are installed at a breeding site such as a farm / pigpen / pig house, and are intended to identify livestock within a specific breeding area and acquire data.

[0025] The activity status estimation device 10 may be realized by a management server or the like. The management server and the user terminal 20 are information processing devices, and are configured by a computer equipped with an arithmetic unit and a storage device. The imaging device 30 and the position measurement device 40 may also have an information processing device such as a microcomputer inside. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration may include a processor such as a CPU, GPU, or TPU, a main storage device such as a main memory, and an auxiliary storage device such as an SSD or HDD. The computational resources are not limited to being local or on-premise, and may utilize a cloud environment provided by another company.

[0026] The network NW is, for example, a network such as the Internet, a VPN (Virtual Private Network), an intranet, or a short-distance wireless communication. For simplicity of explanation, FIG. 1 shows one management server, user terminal 20, imaging device 30, position measuring device 40, and various other additional sensors 50 that realize the activity status estimation device 10, but each can be connected to two or more devices. Also, one user terminal 20 may be used by multiple users. A livestock husbandry management service operator can use the husbandry management system 1 to provide users with services to support the management of livestock husbandry.

[0027] The management server or the like in which the activity state estimation device 10 is implemented may be provided in a different country from the farm having the rearing area, the imaging device 30, and the position measurement device 40. The country in which the server is located does not pose an obstacle to the use of the network type system. Those who can benefit from the rearing management system 1 are solely those places where the farm having the rearing area, the imaging device 30, and the position measurement device 40 are located, and those who operate livestock farming in those places.

[0028] The imaging device 30 that captures livestock images is, for example, a visible light camera, which is a general term for devices that have the function of detecting light reflected from a subject and generating image (still or video) information. The camera may be a combination of an infrared camera and an infrared light in addition to the above-mentioned visible light camera so that nighttime imaging is also possible. Also, a depth camera or ToF camera may be used to make it possible to obtain distance information from the target livestock being imaged. Also, as will be described later, multiple cameras may be installed on one farm and multiple cameras may be installed in one breeding area.

[0029] The position measuring device 40 has a function of identifying each of the livestock kept in the keeping area and measuring the position information. As an example, the position measuring device 40 may be realized by an RFID tag and an RFID reader, but the specific realization means of the position measuring device 40 is not limited thereto. For example, as described above, by using a depth camera or a ToF camera as the image capturing device 30, the image capturing function may be provided with the function of the position measuring device 40, and the image capturing device 30 may serve as both the image capturing device 30 and the position measuring device 40.

[0030] Depth cameras use stereo vision and can measure the distance to an object by using the parallax between two cameras. ToF cameras can measure distance by using the speed of light and the time it takes for light to reflect off an object and return. However, livestock must be identified by a separate method, such as ear tags, which will be described later.

[0031] The specific operation of position measurement using RFID tags and RFID readers will be described later, but the RFID reader can measure distance by measuring the strength of the signal received from the RFID tag (RSSI). RFID tags are relatively inexpensive, and by selecting a specific frequency, it is possible to identify individual livestock that are communicating.

[0032] As other position measurement means, the use of GPS, BLE beacons, etc. is not excluded, but these may be inferior in operation compared to the above means. For example, GPS may be suitable for cows and horses that graze outdoors on a vast ranch, but for pigs that are kept in a limited area indoors, there are accuracy issues and a risk of contamination or destruction. Therefore, there may be an option to exclude the use of GPS, BLE beacons, etc. as a position measurement device of the present disclosure.

[0033] 2 is a block diagram showing a functional configuration of the activity status estimating device 10. The activity status estimating device 10 is an information processing device that provides a livestock breeding management service. The activity status estimating device 10 may include a memory unit 110 and a processing unit 120.

[0034] 2, the storage unit 110 is illustrated as being integrated with the activity status estimation device 10, but it may be an auxiliary storage device that is physically independent from the activity status estimation device 10. Being physically independent means that there is no effect on implementation even if it is in a different location, and it may be configured so that only the storage unit 110 exists in a server in a different location.

[0035] The storage unit 110 may include an application program 111, livestock basic data 112, livestock image data 113, area map data 114, activity state data 115, livestock detection data 116, livestock identification data 117, posture estimation data 118, and additional sensing data 119. The storage unit 110 may also store other information. Each piece of data may be organized in correspondence with each other according to a predetermined format or standard, such as a data table format, and stored as a database.

[0036] The above information stored in the storage unit 110 will be described. The livestock basic data 112 is basic information about livestock that are the subject of management in the livestock breeding management service, and may have data items such as the farm, pigpen, free stall, pig house, individual image, ear tag code, breed, lineage, sex, age, alarm conditions, and alarm history, with an individual identification code, which is a livestock ID, as a main key. The livestock basic information is generated for each livestock, and may be input in advance and stored in the storage unit 110.

[0037] For example, the livestock ID is a code for identifying the livestock being raised, and a unique value can be set for each individual. The farm, piggery, and free stall are IDs for managing the rearing area to which the livestock belongs.

[0038] Here, the definitions of farms, pig sties, pig pens, and free stalls as rearing areas vary depending on the region and the design of each farm, but we will try to explain them as generally as possible. Farm refers to the broadest area and can refer to the entire land or the name of the livestock farming business. Pigsty / Pig House These are sometimes used interchangeably and can refer to a specific building or area where pigs are raised. For example, one farm can have multiple pig sties and pig pens. Free Stall refers to an individual area where livestock can move around freely. Basically, a fence or other structure is installed inside a pig sty or pig pen to structurally secure space for livestock to move around freely. Its size is usually smaller than the entire pig sty or pig pen.

[0039] As the livestock basic data, a specific code for matching and identifying an individual livestock imaged by the imaging device 30 with a livestock individually identified by the position measuring device 40 and whose position information is measured may be stored in correspondence with the livestock ID. Here, the specific code is, for example, an ear tag code that differs for each combination of ear tag colors, or an ear-notched code, etc.

[0040] An example of a method for identifying livestock using ear tags will be described later, but in order to identify individuals more simply, inexpensively, and practically, a physical identifier attached to the body of the individual is used to identify the individual. Specifically, the physical identifier may be an ear tag.

[0041] The ear tag may have, for example, one or more color regions, for example, two color regions, extracted from an image captured by light including visible light or infrared light. Furthermore, the ear tag may have a radio wave reflector such as an RFID tag that reflects radio waves of a specific frequency to grasp the relative position of the livestock. Here, reflection includes not only direct reflection, but also re-transmission due to electromagnetic induction by radio waves emitted from the RFID reader that is the source of the reflection. Note that this does not exclude the identification of individuals by physical identifiers other than ear tags, such as ear markings, other IC tags, tattoos, etc.

[0042] The livestock basic data may include sex data of the livestock. Sex includes female, male, castrated male, etc. Livestock type data may be stored as a classification of pigs in pig breeding management. Classifications include sow, boar, etc. Lineage indicates which of the broad lineages, such as Landrace, Large Yorkshire, Duroc, etc., the pig belongs to when performing so-called three-way mating. The sex data, livestock type data, and lineage data can also be used as auxiliary explanatory variables when estimating the activity state described below.

[0043] The age data in days is an item in which information on the age in days or weeks of livestock is stored. The age data in days includes the birth date of the livestock and the age in days, which is the number of days elapsed since the birth date of the livestock is set as day 0. The age data in weeks similarly includes the number of weeks elapsed since the birth date of the meat pig is set as the reference date. Note that the age in days or weeks may be determined using the weaning date as the reference date instead of the birth date. These ages, such as the age in days, of an individual can be used as auxiliary explanatory variables when estimating the activity state, which will be described later.

[0044] The alarm condition is an item in which conditions for outputting various alarms when the abnormality detection unit 127 of the feeding management system 1 detects an abnormality are stored. Details of the alarm condition will be described later. The alarm history may store an alarm history regarding an alarm output based on the alarm condition.

[0045] In addition, the livestock basic data may include activity history for each individual animal, as described below.

[0046] The livestock image data is an image of the rearing area captured by the imaging device 30, and is an image of livestock. The livestock image may be a still image or a video. The pig captured in the livestock image may be one or more livestock. The livestock image may be used for various purposes. For example, the livestock image may include an image for identifying an individual livestock, an image for estimating the activity state of the livestock, and an image for estimating injuries, illnesses, abnormalities, etc.

[0047] The livestock images may be accompanied by information for identifying the rearing area captured by the imaging device 30. If the farm is large, it is normal that one camera or the like constituting the imaging device 30 is not enough to capture images of all rearing areas. Therefore, it is quite conceivable that the imaging device 30 is a plurality of cameras or a mobile camera. In this case, the accompanying information may include, for example, a camera ID, a pigpen ID, a pigpen ID, a free stall ID, an imaging position ID, and the like.

[0048] When the imaging device 30 is a fixed camera, the imaging position does not move, so it is possible to identify which rearing area is to be imaged using the camera ID, pigpen ID, pig pen ID, free stall ID, etc. On the other hand, when the imaging device 30 is a mobile camera, it is possible to specify which rearing area is to be imaged by separately providing a camera position measuring device such as an encoder or beacon and specifying the imaging position.

[0049] Although the location where the imaging device 30 is installed is not particularly shown, it is preferable to install it at a height between the ceiling and the rearing area, such as on the beams of the building that constitutes the pigpen / pigpen. For example, it may be installed with an angle of view that looks directly down on the rearing area from the beams, or it may be placed from a position higher than the fence so that the entire rearing area can be observed while looking down on the livestock somewhat horizontally.

[0050] In the case of a mobile type, the imaging device 30 may be configured to be movable along rails or the like provided at the upper part of the pigpen / pig house indoors, and may capture images with an angle of view looking straight down from there. For example, the operation of the driving device of the imaging device 30 may be controlled based on a control signal generated by a control program preset in an application program, or based on a control signal from the activity state estimation device 10. In addition, when a mechanism for changing the orientation of the camera of the imaging device 30 is provided, the moving direction, amount of movement, moving speed, lens orientation, and the like of the camera may be controlled.

[0051] In addition, the livestock image data includes the image capture date and time, and in the case of a video, may include the image capture start date and time and the image capture end date and time. The image capture date and time can be used when calculating the trajectory of the livestock's position within the area, which will be described later.

[0052] The information accompanying the image data may be either included as meta-information in the image data or stored in association with each other in a database.

[0053] When a plurality of livestock are captured in a livestock image, the livestock image data 113 may include individual livestock images, which are images of individual livestock extracted individually using a livestock detection unit 122 described later. A method for extracting individual livestock images will be described later.

[0054] The area map data 114 is data showing a map of a specific breeding area. It includes electronic data such as the area of ​​the breeding area, layout, type of breeding equipment installed, attributes, and location information thereof. The breeding equipment installed within the area map includes fences that define the breeding area, waterers that supply water to livestock, feeders that supply food, and enrichment materials such as play equipment.

[0055] The map information of the rearing area can be expressed by various data in two-dimensional or three-dimensional format. For example, it is not limited to a two-dimensional map with coordinates added to it when looking down on the pigpen from above, but may be a three-dimensional spatial map. For example, the map information may include the three-dimensional coordinates of the positions of the cameras serving as the imaging device 30 and various devices serving as the position measuring device 40 when they are installed above the heads of the livestock within the rearing area. In addition, the map information may include the position information of various additional sensors 50 and rearing equipment such as waterers, feeders, toys, and fans.

[0056] As will be described later, specific rearing equipment within the rearing area is used to estimate the activity state of an individual. For example, rearing equipment such as waterers and feeders are related to the eating behavior of livestock. Rearing equipment such as fences are related to abnormal behavior of livestock, such as crashing into fences or self-injury when livestock behave abnormally. Furthermore, enrichment materials such as play equipment are related to activities of livestock, such as playing and relaxing.

[0057] The activity state data 115 mainly defines and stores activity states estimated from the posture of the livestock. For example, the activity state is estimated from postures such as lying on one's back, lying on one's side, standing, sitting, etc., and the position of the livestock within the area. These include normal activity states such as resting, drinking water, dozing, standing, walking, etc., as well as abnormal activity states including stereotypic behaviors such as fighting, riding, and dog-sitting. The estimation of the activity state by the activity state estimation unit 126 will be described later.

[0058] Furthermore, the activity state data 115 may also store activity patterns that are further estimated from the status of a plurality of activity states. At least a transition from a first activity state to a second activity state different from the first activity state may be defined and stored as an occurrence of one activity pattern. As activity patterns, a water supply pattern, a rest pattern, a fighting pattern, etc. may be appropriately set and stored. The estimation of activity patterns by the activity state estimation unit 126 will be described later.

[0059] The livestock detection data 116 is data for the livestock detection unit 122 described later to detect livestock based on livestock images. For example, it includes a trained model for object detection. A method for the livestock detection unit 122 to detect livestock from an image will be described later.

[0060] The livestock identification data 117 is data for the livestock identification unit 124, which will be described later, to detect livestock based on an image. For example, the data includes a livestock ID, an ear tag code, and the like.

[0061] The posture estimation data 118 is data for the posture estimation unit 125 described later to estimate the posture of the livestock based on the image. For example, it includes a trained model for estimating the posture of the animal. A method for the posture estimation unit 125 to estimate the posture of the livestock from the image will be described later.

[0062] The additional sensing data 119 indicates data measured and acquired by additional sensors 50 such as a thermometer, a hygrometer, a barometer, a gas sensor, a feed amount meter, a water amount meter, a sound sensor, an air flow meter such as an anemometer, and a dust amount meter such as a particle counter or a dust monitor, other than the imaging device 30 and the position measurement device 40 in the breeding management system 1. For example, the additional sensing data 119 includes the temperature, humidity, air pressure, air flow, and concentration of a specific gas in the breeding area, the amount of feed supplied by the feed amount meter, the amount of water supplied by the water amount meter, sound data in the breeding area, and the amount of dust in the breeding area. The supplementary use of these additional data will be described later.

[0063] The processing unit 120 of the activity status estimation device 10 includes a sensing data acquisition unit 121, a livestock detection unit 122, an in-area position calculation unit 123, a livestock identification unit 124, a posture estimation unit 125, an activity status estimation unit 126, and an abnormality detection unit 127. In the processing by the processing unit 120, each function is realized by executing an application program 111 stored in the memory unit 110.

[0064] The sensing data acquisition unit 121 acquires livestock images from the imaging device 30 connected to the activity state estimation device 10 via the network NW, and stores the images in the storage unit 110. The sensing data acquisition unit 121 also acquires sensing data from the position measuring device 40 and the additional sensor 50 connected to the management server via the network NW, and stores the data in the storage unit 110.

[0065] The frequency and duration of acquiring livestock images and sensing data are matters that the livestock keeper should appropriately set. The sampling rate, image resolution, imaging (acquisition) frequency, and number of imaging (acquisition) times can be set according to the purpose of managing livestock, the number and performance of the imaging devices 30, position measuring devices 40, and additional sensors 50 that can be installed, and the amount of data that can be handled by the computational resources of the management server, etc.

[0066] The livestock detection unit 122 has a function of detecting individual livestock from the livestock image captured by the imaging device 30. For example, the livestock detection unit 122 may use object detection by machine learning using a trained model such as a trained deep learning model that detects objects from images. An example is Faster R-CNN.

[0067] In Faster R-CNN, there is no need to input the region of interest when using a trained model; simply inputting an image will give the position and class of the object. For example, as shown in Figure 3, the position of the detected object is the region enclosed by the bounding box BB in the x, y coordinates of the livestock image LP1. For example, for a pig, the coordinates of the frame that contains the entire pig's body are shown. And as class C, the classification of the object shown in the frame, such as pig, is shown. This figure is shown only for understanding, and since internal processing is actually performed, it is not necessary to display the detected image. Other known methods include YOLO, and the latest YOLOv7 can be used at this time. Various other object detection models are known, and an appropriate one can be selected from these, taking into account the required computational resources and dataset.

[0068] In addition, when multiple livestock are captured in an image, it is possible to divide the image into multiple segments using instance segmentation and classify each into a class. For example, Figure 4 is an imitation of an image of pigs captured from above to below in a free stall, and at this point, the individual pigs cannot be distinguished.

[0069] Therefore, by performing instance segmentation, even if pigs are in the same class, they can be individually segmented and identified as shown in Figure 5. One such method is known to be Mask R-CNN, which simultaneously performs general object detection and instance segmentation. Each instance shown with different hatching in this figure is assigned the class of pig, and each instance is assigned a temporary ID. However, even at this point, multiple pigs have merely been detected from the image, and processing by the livestock identification unit 124, described later, is required to identify each individual.

[0070] In addition to object detection by machine learning, the following methods may be used. For example, livestock may be detected by a background subtraction method. A reference background image may be captured when livestock are absent, such as when all livestock are all in and all out, and a pixel area where the difference in luminance between the background image and the captured image is greater than or equal to a predetermined value may be detected as livestock. Edge detection by the Canny method is also widely used. Livestock may be detected by matching edges detected by these methods with the similarity of a template livestock image, or by a feature extraction algorithm.

[0071] The area position calculation unit 123 has the function of calculating the area position indicating where each individual is located within the breeding area, using the position information identified for each individual by the position measurement device 40 and the area map information.

[0072] First, the acquisition of location information of an individual by the location measurement device 40 will be described using the above-mentioned RFID tag and RFID reader as an example. In Fig. 6, a rearing area FS1 is shown, and FS1 is, for example, a free stall defined by being surrounded by a fence on all sides. The livestock P1 to P5 reared in FS1 are each fitted with an ear tag, and the ear tag has an RFID tag built in that responds to a specific frequency.

[0073] 6 shows an example in which the position measuring device 40 is configured with RFID readers 41 and 42. The RFID readers 41 and 42 are installed, for example, on beams located below the ceiling above a rearing area such as a free stall, and can radiate radio waves of a predetermined frequency downward. However, this figure shows a plan view seen from above.

[0074] Here, if the positions of the RFID readers 41 and 42 are known and the distance L1 from the RFID reader 41 to the livestock P2 and the distance L2 from the RFID reader 42 to the livestock P2 can be obtained, the position of the livestock P2 can be measured from L1 and L2. This is the relative position from the RFID readers 41 and 42 to the livestock P2.

[0075] Here, if the RFID reader has the function of measuring the direction of radio wave emission by itself, the number of readers may be one. For example, if the reader has a rotation drive mechanism and can scan radio waves in the rotation direction and obtain the direction of radio wave emission by measuring the angle with an encoder, it is possible to obtain the relative position of the livestock.

[0076] A brief description will also be given of a case where a depth camera or a ToF camera is used as both the imaging device 30 and the position measuring device 40.

[0077] Depth cameras and stereo cameras capture multiple images, and generate a depth map based on those images, where each pixel represents the distance to an object. Objects are then detected from the depth map. Objects can be detected using a method based on the depth gradient. Then, a center or other position that serves as a reference for the position of the detected object is determined. Here, the average position of all pixels of the object is assumed to be the center of the object. The relative position of the object center and the camera center is then calculated using the object center, the camera center, and the camera's viewing angle. This makes it possible to detect objects, i.e. livestock, and measure the relative position of the livestock.

[0078] The ToF camera may be a range image sensor. As is well known, it is possible to measure the distance to an object by measuring the difference in the flight time to the object and back, and a range image sensor such as a 3D-ToF camera can measure the relative positional relationship between the camera and the object for all captured pixels. Images captured by the range image sensor may be used for both position measurement and object (livestock) detection.

[0079] In reality, the distance between the imaging device 30 installed in the rearing area and the livestock is considered to be about several meters, and the performance of measuring the relative positions of the livestock using a depth camera or ToF camera is well within an acceptable range.

[0080] The area position calculation unit 123 calculates the area position using the area map data and the relative position of the individual livestock in the rearing area. The area position may be expressed by coordinates, which may be two-dimensional coordinates on the XY plane. In FIG. 6, the area map is shown on a grid, and the minimum area is made up of 1 to 9 on the horizontal axis and A to H on the vertical axis, and for example, the area position of P2 is expressed as coordinates "F7". Here, the coordinates indicate that the livestock are located within one section. The resolution of the area position should be set appropriately from fine to coarse depending on the purpose. Depending on the size of the free stall, it may be sufficient to measure the area position with a section of 50 cm square units.

[0081] When measuring distance using RFID-RSSI, the accuracy of distance measurement is not high when using passive RFID tags. Instead, the tag has the advantage of not needing a power source. On the other hand, the accuracy of distance measurement is improved when using active RFID tags, but a power source such as a battery is required, and the batteries installed in the ear tags must be replaced. Therefore, the resolution of the area and position measurement should be set appropriately according to these operational aspects.

[0082] Using such an area map and the relative positions of the livestock, the in-area position calculation unit 123 can calculate the in-area position of each individual livestock. For example, in Fig. 6, the position of livestock P1 can be calculated as A9, the position of livestock P3 as C3, the position of livestock P4 as G4, and the position of livestock P5 as F4.

[0083] The livestock identification unit 124 has the function of identifying the individual captured in the image, measuring its position using the position measurement device 40, and identifying the individual by comparing it with the individual whose intra-area position has been calculated by the intra-area position calculation unit 123.

[0084] Returning to Fig. 3, the explanation will be given with reference to the livestock image LP1. The livestock P1 has two ear tags, an ear tag t11 and an ear tag t12, attached to its ears. Either of these ear tags may incorporate an RFID tag that responds to an RFID reader. The ear tags t11 and t12 have basic geometric shapes and are colored in highly saturated colors.

[0085] Basic geometric shapes are basic geometric shapes such as a perfect circle, equilateral triangle, square, regular pentagon, regular hexagon, etc. These shapes have geometric features that are easy to recognize in images, such as being equidistant from the center and each side being the same, and can be easily used as feature points when conducting machine learning.

[0086] Also, highly saturated colors are basically primary colors such as red, blue, green, and yellow, plus purple, orange, indigo, and black and white, which are counted as the seven colors of the rainbow, and about nine other colors that are easy to distinguish when processing images. Therefore, it is clear what highly saturated colors are. What is considered to be highly saturated colors varies depending on the color space model. For example, the most common ones are RGB (red, green, blue), CMYK (cyan, magenta, yellow, black), and HSB or HSV (hue, saturation, brightness or value). Of these, the HSB / HSV model directly expresses saturation. Such highly saturated colors are easier to detect than pale colors, and colors that are different from the color of pig skin are particularly advantageous for detection.

[0087] The color and shape of these ear tags are coded as ear tag codes and stored in association with the livestock ID of each individual. By combining not only the ear tag code but also the pigpen ID, pig house ID, and free stall ID, it becomes possible to manage a considerable number of livestock. In other words, the ear tag code may be a combination of the color of an ear tag having a predetermined basic geometric shape, or a code that combines multiple colors, and a code that can identify the breeding area, such as the pigpen ID, pig house ID, and free stall ID.

[0088] For example, in Figure 3, ear tags t11 and t12 are both circular, and since ear tag t11 is blue and ear tag t12 is red, the pig can be identified as having a livestock ID that corresponds to the ear tag code corresponding to the combination of blue and red.

[0089] In addition to ear tags, there is ear notching as an identifier that can identify an individual from an image. Ear notching is a method of identifying an individual by making a specific pattern of notches or holes in the ears of livestock. It is also possible to identify an individual by coding the ear notching pattern and detecting the ear notching pattern from a captured image of the ear. However, there is an aspect that goes against freedom from pain, injury, and disease, as the livestock may feel pain and there is a risk of infection. Therefore, in this disclosure, ear notching is not excluded, on the premise that it is performed under anesthesia or painkiller treatment by a trained veterinarian only when necessary in consideration of the overall five freedoms of animal welfare.

[0090] The livestock identification unit 124 recognizes the color combination of the ear tag by pattern recognition of color and shape obtained by analyzing the livestock image LP1. Then, the ear tag code can be identified from the color combination of the ear tag, and the livestock ID can be specified.

[0091] If the livestock ID can be identified, it is possible to match the tag ID of the RFID tag with the associated livestock ID and identify the individual livestock whose location within the area has been calculated.

[0092] The posture estimation unit 125 has a function of estimating the posture of an individual animal using a livestock image. An example will be described with reference to Fig. 3. The posture of an individual animal is estimated by at least estimating the posture of the livestock image.

[0093] For example, DeepLabCut can be used as an animal posture estimation model, and learning is possible with a small amount of labeling using images as a dataset. Other well-known methods include LEAP, SLEAP, which estimates the postures of multiple animals, and DPK, which can also be used. When using these models after obtaining a trained model, an estimated posture (data) can be obtained as output by inputting an image. More specifically, a posture is made up of multiple feature points extracted from an image and their coordinates.

[0094] The posture estimation shown in Figure 3 is merely an example, and is the extraction of feature points on the pig's body. There is no need to connect the feature points skeletal to each other. The posture can be estimated from the positions of the feature points. The image can be a video, and it is also possible to confirm the gait by tracking the movement of the feature points in the video.

[0095] Note that the posture estimation of an individual by the posture estimation unit 125 does not exclude the possibility of improving the estimation accuracy by acquiring learning data that includes a method of directly estimating posture by attaching a sensor such as a motion sensor to the body of the livestock.

[0096] The activity state estimation unit 126 has a function of estimating the activity state of an individual using the posture of the individual and the in-area position information. As described above, the livestock identification unit 124 can match and link an individual whose in-area position has been calculated with an individual whose posture has been estimated via the livestock ID. This makes it possible to estimate the activity state of each individual from the in-area position and posture.

[0097] Various methods can be used for estimating the activity state by the activity state estimation unit 126. For example, a classical rule-based method may be used to estimate the activity state from the position and posture within the area. For example, a rule that a specific position within the area and a specific posture correspond to a specific activity state may be created and applied. For example, in FIG. 6, livestock P3 is located in sections C2 to E4, which are one section away from the water supply equipment, which is a water supply device located at coordinate D3, which is an intra-area position. Then, it is assumed that the posture of P3 is estimated to be "standing head down" from multiple characteristic points. This posture is a state in which the head is lowered while standing upright on all four limbs. Then, it can be estimated that the activity state in which the posture is taken at that intra-area position is "watering". It is not necessary to explain here all the postures and intra-area positions that can be estimated as what activity state, and these are matters that the user of the feeding management system 1 should appropriately set according to the designed feeding area, the type of livestock, sex, age, feeding equipment, number of livestock, etc. For example, if the water dispenser is of the head-lifting type, the corresponding posture should be changed accordingly.

[0098] As described above, the activity state may be estimated using the relationship with the rearing equipment in addition to the position and posture within the area. For example, in FIG. 6, the sections C2 to E4, which are adjacent to the section where the water dispenser is arranged, are in a relationship of being in the vicinity of the water dispenser, and a specific posture in this section is related to a specific activity state. This relationship may be used to assist the estimation of the activity state. Regarding the relationship between a specific activity state and the rearing equipment, the storage unit 110 may have relationship data that associates the rearing equipment, the distance from the rearing equipment, the posture, and the activity state in order to derive what activity state should be estimated when the animal is close to what rearing equipment and what posture is taken.

[0099] To estimate the activity state, a trained model trained using a classical machine learning method such as SVM or random forest may be used. A trained model trained using deep learning with a neural network may also be used. Furthermore, the results of an ensemble of weak learners may be used. These machine learning methods may be used in the learning stage, and rule-based classification may be performed based on the output.

[0100] The activity state to be estimated may be only that which corresponds to the situation to be detected. For example, while "drinking water" is estimated based on a characteristic position or posture, if the animal is in a normal posture in an unspecified place in the breeding area, and this is not the situation to be detected, there is no need to estimate the activity state in particular. However, this does not exclude the estimation of the activity state at all times.

[0101] In addition, activity patterns may be estimated as a concept obtained by observing the activity states over time. An activity pattern is a behavioral pattern of livestock that is expressed as a minimum unit of a transition from a first activity state to a second activity state different from the first activity state.

[0102] The activity states and activity patterns will be described with reference to Figures 7 and 8. Figure 7 shows the position and posture of livestock P1 within a rearing area within a given time period. For convenience, only four examples of combination data of position and posture are shown in Figures 7 and 8, but in actual operation, more detailed data may be obtained to estimate more activity states.

[0103] FIG. 7 shows the time series changes of the intra-area positions from positions s1 to s4. Here, the absolute positions of the intra-area positions are conveniently expressed as "coordinates". The time series changes of the intra-area positions are shown by a trajectory R1. This trajectory is shown as a continuous line for understanding, but the actual trajectory of the intra-area positions may be intermittent. In this trajectory R1, the coordinates, trajectory, image, posture estimated from the image, activity state estimated from the coordinates and posture, attribute of the activity state, estimated activity pattern estimated by transition of the activity state, and attribute of the activity pattern are shown at each of the time series positions s1 to s4. Here, the meaning of the time series positions and the coordinates is somewhat overlapping, but the time series positions s1 to s4 represent the positions that change over time in the trajectory R1, and the coordinates represent the absolute positions on the area map regardless of time. The estimation of the activity state and the activity pattern in this example will be described below.

[0104] For example, the activity state estimation unit 126 can estimate the following using the acquired data. At the start position s1 of the trajectory R1, the livestock P1 is in a recumbent posture, and since there is no rearing facility around the coordinates and the coordinates are not characteristic, it is estimated that the livestock P1 is resting as an activity state. At position s2, the livestock P1 is located at coordinates near the water supply facility and is in a standing posture with its head down, so it is estimated that the livestock P1 is drinking water. Next, at position s3, there is no special note in the coordinates as with s1, and the livestock P1 is in a stepping posture in which it is stepping forward from a standing posture, so it is estimated that the livestock P1 is walking. Finally, at the end position s4 of the trajectory R1, there is no special note in the coordinates and the livestock P1 is standing without stepping forward, so it is estimated that the livestock P1 is standing still. The above is just one example, and the relationship between the position within the area, posture, and activity state can be appropriately modified by incorporating the knowledge of pig farming experts and machine learning.

[0105] Then, such transitions of activity states such as resting, drinking water, walking, and standing still may be classified as a watering and feeding activity pattern. If such transitions of activity states can be expressed as a single continuous sequence, the probability of the transition may also be defined and modeled as an HMM (Hidden Markov Model). The above series of actions such as resting, drinking water, walking, and standing still can be modeled with a single HMM. Conversely, it is also possible to model the activity patterns of livestock with an HMM based on the transition history of activity states.

[0106] It is useful to be able to obtain information on the activity state as a qualitative parameter representing the behavior of livestock using posture and position within the area, as in the present disclosure. For example, the ESF method has been known as a means capable of measuring the amount of feed and timing of feeding for each individual, but the ESF method cannot grasp the activity of livestock before and after feeding. In other words, it does not detect the position of each livestock, but only detects livestock that approach the feeding device. In addition, there is a problem that multiple livestock cannot eat at the same time, and multiple feeding devices compatible with ESF are required to feed them at the same time.

[0107] Figure 8 shows another example of the estimation of activity state and activity pattern. In this example, only the trajectory R2 of livestock P2 is shown on the area map. The typical activity pattern of livestock shown in this example is self-injurious behavior due to stress.

[0108] At positions s5 to s8, livestock P2 is circling only around the vicinity of one or two sections of the fence equipment that defines the rearing area. At position s5, livestock P2 is near one section of the fence equipment and is standing on its hind legs, i.e., with its front legs raised, so its activity state is presumed to be riding. Riding can be done to other livestock, but it can also be done to the fence. And since its subsequent activity state also includes intimidation and riding, its activity pattern is presumed to be a self-harming / aggression pattern. Livestock in such a state may be suffering from some kind of stress or injury, and the need for immediate measures such as isolation or protection should be considered.

[0109] The abnormality detection unit 127 may have a function of calculating the frequency, interval, number of times, or cumulative time of an individual's activity state, or statistics thereof, and comparing it with the frequency, interval, number of times, or cumulative time of an individual's activity state, or statistics thereof, of multiple livestock kept in the keeping area, to determine whether an individual has an abnormality or a sign of an abnormality.

[0110] As mentioned above, when an individual exhibits abnormal behavior, whether all livestock kept in the same rearing area tend to exhibit the same behavior or whether only that individual is abnormal is often determined by the experience of the human rearer. For example, when the air conditioning equipment breaks down and the temperature rises excessively, it is natural that all livestock kept in that rearing area will breathe abnormally. On the other hand, if only a specific individual is shivering or breathing abnormally even though the air conditioning equipment is not broken and the indoor temperature is stable, it is highly likely that that individual is in an abnormal condition such as an injury or illness.

[0111] Therefore, the abnormality detection unit 127 calculates the frequency, interval, number of times, or cumulative time of the activity state of a specific individual, or statistics thereof, and compares it with the frequency, interval, number of times, or cumulative time of the activity states of multiple livestock kept in the rearing area, or statistics thereof, thereby offsetting other environmental factors and accurately identifying an abnormal individual. By using livestock to be compared with livestock on the same farm, pig house / pig pen, or rearing area on the same date, time, etc., abnormalities can be detected more accurately.

[0112] For example, the abnormality detection unit 127 can detect abnormalities in a livestock by meeting various conditions such as the cumulative time or frequency that a livestock is in an active walking state being significantly higher or lower than statistical values ​​such as the average or median of the cumulative time or frequency that a livestock is in an active walking state, compared to multiple other livestock.

[0113] The abnormality detection unit 127 may also detect at least a transition from a first activity state of an individual to a second activity state different from the first activity state as an activity pattern, calculate the frequency, interval, number of times, or cumulative time of occurrence of the activity pattern, or statistics thereof, and compare the frequency, interval, number of times, or cumulative time of occurrence of the livestock activity patterns of multiple livestock kept in the rearing area, or statistics thereof, to determine that the specified individual has an abnormality or a sign thereof. As above, by using livestock to be compared with livestock on the same farm, piggery / pigpen, rearing area, on the same date, time, etc., abnormalities can be detected more accurately, and comparison can be made in a similar manner.

[0114] For example, the abnormality detection unit 127 can detect abnormalities in a livestock by determining whether the cumulative time or frequency at which a livestock exhibits a specified abnormal behavior pattern is significantly higher or lower than a statistical value, such as the average or median, of the cumulative time or frequency at which a specified abnormal behavior pattern is exhibited by multiple other livestock.

[0115] The form creation unit 128 has a function of outputting a form that can be used to check whether animal welfare is being properly implemented, using the activity state estimated by the activity state estimation unit 126. This will be described later.

[0116] In the present disclosure, an example of auxiliary use of additional sensing data in the estimation of the activity state will be described. Depending on the activity state to be estimated, there are cases in which the above-mentioned in-area position and posture alone are insufficient as information.

[0117] For example, when the temperature in the rearing area is high and livestock are exposed to heat stress, causing breathing to become rough, or when stress causes abnormal moans, it is difficult to estimate the activity state from images and postures alone. In such cases, the accuracy of estimating the activity state may be improved by adding sound data collected by a sound sensor, which is one of the additional sensors 50, to the position and posture within the area. Specifically, the occurrence of coughing or fighting, or an activity state that can be considered to be a stress state, may be estimated using a specific frequency, the volume of a specific sound, the timing of the sound, etc.

[0118] Next, FIG. 9 is a sequence chart showing the steps of the husbandry management method of the present disclosure, and shows the basic processing flow of the imaging device 30, activity status estimation device 10, position measurement device 40, memory unit 110, and additional sensor 50.

[0119] In step S301, the imaging device 30 captures an image of the inside of the breeding area (imaging step).

[0120] In step S401, the position measurement device 40 identifies each of a plurality of livestock kept in a keeping area, and acquires position information (position information acquisition step).

[0121] Also, if necessary, in step S501, the additional sensor 50 acquires additional sensing data.

[0122] These steps S301, S401, and S501 are executed each time according to the set frequency, and the order of the steps does not matter.

[0123] Next, in step S101, the sensing data acquisition section 121 of the activity status estimating device 10 acquires images and position information (sensing data acquisition step).

[0124] Furthermore, in step S111, the in-area position calculation section 123 of the activity status estimating device 10 may acquire area map information from the storage section 110 before calculating the in-area position in step S102.

[0125] In step S102, the livestock identification unit 124 identifies the individual captured in the image (livestock identification step).

[0126] In step S103, the posture estimation unit 125 estimates the posture of the individual using the image (posture estimation step).

[0127] In step S104, the intra-area position calculation unit 123 calculates the intra-area position of the individual within the breeding area, using the position information and the area map information stored in the storage unit 110 (internal area position calculation step).

[0128] It should be noted that steps S102, S103, and S104 may be executed in any order as long as they are executed before step S105.

[0129] In step S105, the activity state estimation unit 126 estimates the activity state of the individual using the posture of the individual and the in-area position information (activity state estimation step).

[0130] Furthermore, in step S106, the abnormality detection unit 127 determines whether an individual has an abnormality or a sign of an abnormality by using the frequency or cumulative time of occurrence, or the ratio of these, of the activity state and activity pattern of the individual or multiple livestock estimated by the activity state estimation unit 126 (abnormality detection step).

[0131] If, after going through these steps, it is determined that there is an abnormality or a sign of an abnormality, the husbandry management system 1 will notify the owner of the information as an alert via the user terminal 20, so that the owner can take appropriate action.

[0132] Although the embodiments of the present invention have been described above, these embodiments can be embodied in various other forms, examples of which are shown below.

[0133] 10 is a diagram showing an example of activity state estimation that can be used as another embodiment. In this diagram, livestock P1 to P6 are all located within the vicinity of one section of a water supply device installed at coordinate D3.

[0134] In this case, if some decrease in the accuracy of estimating the activity state is acceptable, the activity state estimation unit 126 may estimate the activity state of the livestock only from the relationship between the water supply device, which is a breeding facility, and the position within the area, without using the posture estimated from the image. For example, in this situation, the activity state of the livestock P1 to P6 may be estimated to be drinking water, based on the positional relationship with the water supply device.

[0135] Furthermore, the abnormality detection unit 127 can detect an abnormality or a sign of an abnormality by using the activity state of an individual. In Fig. 10, livestock are gathered near the water supply device in the upper left of the drawing, and when a predetermined number of livestock, for example, five or more, stay in a predetermined section, for example, within the vicinity of one section, with the water supply device as the center, for a predetermined time or more, for example, one hour or more, and the staying livestock are in a predetermined activity state, it can be determined that there is an abnormality or a sign of an abnormality.

[0136] For example, if the maximum number of animals that can use a waterer at the same time is four, and six or more animals gather nearby and stay there for a specified period of time or longer, there is a possibility that there is a fight between livestock or that the waterer is broken. Using such livestock activity status, the abnormality detection unit 127 can also determine that there is an abnormality or a sign of an abnormality. By dealing with this, the keeper can contribute to freedom from thirst and malnutrition, and to the welfare of the animals.

[0137] Also, in the lower right of Fig. 10, a predetermined number or more of livestock P7 to P12 have gathered in a section consisting of coordinates F6 to G8 where no feeding facilities are installed, and have been staying there for a predetermined time or more. In this case, it is possible that a fight has broken out between livestock, or that a livestock has suddenly died and other livestock have gathered there. Therefore, by setting the range of a predetermined section, the number of livestock, and the duration of stay, etc. as thresholds, it is possible to determine whether an abnormality such as a fight or an injury or illness has occurred or a sign of such an abnormality has occurred.

[0138] Also, in the upper right of FIG. 10, for example, a blower is installed at coordinate B9. However, in this case, the blower is installed above the livestock, not at a height where the livestock can stay. If the frequency or probability of the livestock stopping around B9, for example, in the areas A8 to C9 as shown in this figure, is low, it is presumed that the livestock avoid entering this area for some reason. An experienced breeder can also deduce that this is because the livestock dislike the wind and noise caused by the blower. The abnormality detection unit 127 can automatically detect this by using the time course and trajectory of the livestock's position within the area. The occurrence of such an area may be observed as a "positional abnormality."

[0139] A specific example of a method for observing positional abnormalities will be considered. The abnormality detection unit 127 acquires the time course and trajectories of the in-area positions of all or some of the livestock kept in the rearing area from the in-area position calculation unit 123. Then, the movement trajectories of the livestock are mapped on an area map to identify which sections the livestock have passed through or stayed in. Then, sections in which the livestock have not passed through or stayed, or sections in which the livestock pass through or stay less frequently than other sections, can be identified as non-stay sections. Non-stay sections identified in this way can also be detected as sections with positional abnormalities.

[0140] By visualizing the areas where locational abnormalities are detected, keepers can make improvements to areas where livestock cannot stay and the area of ​​the pig house / pig pen is not being used effectively, etc. In this way, locational abnormalities can be used by keepers to improve their rearing, as a qualitative parameter independent of the activity state of livestock, also to contribute to animal welfare.

[0141] In the above-described embodiment, the accuracy of the estimation of abnormalities can be improved by adding the posture of the livestock or sensing data from additional sensors.

[0142] Several embodiments of the present disclosure have been described above. We will now explain the relationship of these embodiments to the contributions of the five freedoms of animal welfare, and the function of the form creation unit 128 that creates forms that can confirm this.

[0143] The report creation unit 128 has a function of creating a report that uses the activity state estimated by the activity state estimation unit 126 to check whether each caregiver is implementing appropriate animal welfare on the farm.

[0144] First, "freedom from hunger, thirst and malnutrition." For this, by setting an activity state that affects the nutritional state of livestock, such as "feeding and watering," the frequency, interval, number of times, or cumulative time of occurrence of the activity state for each individual, or statistics thereof (including comparison with the previous month and previous year), or the frequency, interval, number of times, or cumulative time of occurrence of the activity state in multiple livestock groups for each rearing area, can be calculated and displayed or printed as a report on the user terminal 20. In addition, if conditions for normal and abnormal ranges of the frequency of occurrence of the activity state, etc., are set, it is also possible to output phenomena that meet the conditions as a report. This allows a breeder who recognizes a problem to provide fresh feed and water to livestock.

[0145] Next, for "freedom from fear and distress," by setting an activity state related to abnormal behavior that occurs when livestock feel fear or distress, such as "aggressive behavior" or "stereotypic behavior," the frequency, interval, number of times, or cumulative time of the occurrence of the activity state for each individual, or statistics thereof, or the frequency, interval, number of times, or cumulative time of the occurrence of the activity state in multiple livestock groups for each rearing area, can be calculated, and displayed or printed on the user terminal 20 as a report. In addition, if conditions for normal and abnormal ranges of the frequency of occurrence of the activity state, etc., are set, it is also possible to output phenomena that correspond to the conditions as a report. This makes it possible for a breeder who recognizes the problem to ensure a situation in which psychological distress of livestock is avoided.

[0146] Next, regarding "freedom from physical and thermal discomfort," for example, abnormal breathing due to heat, shivering due to cold, gathering in one place, skin discoloration, etc., are estimated by estimating additional data such as voice data, temperature data, image data, etc. in addition to the position and posture in the area, and the frequency, interval, number of times, or cumulative time of the occurrence of the activity state for each individual, or statistics of these, and the frequency, interval, number of times, or cumulative time of the occurrence of the activity state in multiple livestock groups in each rearing area can be calculated, and displayed or printed on the user terminal 20 as a report. In addition, if normal and abnormal range conditions such as the frequency of occurrence of the activity state are set, it is also possible to output the phenomenon that corresponds to the condition as a report. This allows the breeder who recognizes the problem to provide an appropriate rearing environment.

[0147] Next, regarding "freedom from pain, injury and disease," for example, abnormal postures caused by disease or injury of livestock, such as "dog sitting" or "lameness," or activity states that prevent normal walking can be set, and the frequency, interval, number of times, or cumulative time of occurrence of the activity state for each individual, or statistics thereof, or the frequency, interval, number of times, or cumulative time of occurrence of the activity state in multiple livestock groups in each rearing area can be calculated, and displayed or printed on the user terminal 20 as a report. In addition, if conditions for normal and abnormal ranges of the frequency of occurrence of the activity state are set, it is also possible to output phenomena that meet the conditions as a report. This allows a breeder who recognizes a problem to prevent disease, etc., and perform accurate diagnosis and prompt treatment.

[0148] Finally, regarding the "freedom to express normal behavioral patterns," group rearing in free stalls or other arrangements can provide livestock with the opportunity to behave in a natural way.

[0149] Note that this disclosure does not have to guarantee all five freedoms simultaneously and completely, but rather contributes to at least one of the other freedoms that would be difficult to realize when raising livestock in a manner that contributes to the "freedom to express normal behavioral patterns."

[0150] The embodiments described in the present disclosure are not limited to the above examples, and the configurations described above may be used in combination with each other.

[0151] In order to estimate the activity state, it is sufficient that at least the individual's position within the area and its posture are known, and in some cases, the activity state may be estimated from the relationship between the position within the area and the rearing equipment placed in the rearing area, and further, in some cases, the activity state may be estimated using only the posture.

[0152] The position and attitude within the area may be obtained by the imaging device 30 and the position measurement device 40, and the imaging device 30 and the position measurement device 40 may be devices that combine the functions of each. The position measurement device 40 may be capable of identifying the position of livestock, and is not limited to the means described in this disclosure.

[0153] The activity state, which is a qualitative parameter estimated by the activity state estimation unit 126, is not limited to those listed in this disclosure, and can be set appropriately taking into account the position and posture within the area, etc., depending on the state that the caregiver wants to recognize.

[0154] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0155] (Appendix 1) A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: An imaging device that captures images within the breeding area; a position measuring device for identifying each of the livestock kept in the keeping area and measuring its position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map information; A livestock identification unit for identifying an individual captured in the image; a posture estimation unit that estimates a posture of the individual using the image; and an activity state estimation unit that estimates the activity state of the individual using the posture of the individual and the area position information. This allows farm animals to be kept in a manner that contributes to the freedom of farm animals to express their normal behavioural patterns, such as in free stalls, whilst also contributing to other freedoms and thereby contributing to all five freedoms of animal welfare. (Appendix 2) The area map further includes the type of rearing equipment installed in the rearing area and its location within the area, A breeding management system as described in Appendix 1, wherein the activity state of the individual is estimated from the relationship between the posture and the breeding equipment using the posture of the individual, its position within the area, and the position of the breeding equipment within the area. This allows the activity status of livestock to be estimated in relation to the feeding facilities within the feeding area and contributes in particular to the five freedoms of freedom from hunger, thirst and malnutrition. (Appendix 3) The activity status estimation device comprises: 3. The husbandry management system according to claim 1 or 2, further comprising an anomaly detection unit that calculates the frequency, interval, number of times or cumulative time of the occurrence of the individual's activity state, or statistics thereof, and compares it with the frequency, interval, number of times or cumulative time of the occurrence of the activity state of a plurality of livestock kept in the keeping area, to determine whether there is an abnormality or a sign of an abnormality in the individual. This makes it possible to compare the activity status of an individual with other individuals kept in the same breeding area, offsetting other environmental factors and accurately identifying abnormal individuals. (Appendix 4) The activity status estimation device comprises: The husbandry management system described in Appendix 3, further comprising an anomaly detection unit that detects at least a transition from a first activity state of the individual to a second activity state different from the first activity state as an activity pattern, calculates the frequency, interval, number of times, or cumulative time of occurrence of the activity pattern, or statistics thereof, and compares it with the frequency, interval, number of times, or cumulative time of occurrence of the activity pattern of a plurality of livestock kept in the keeping area, or statistics thereof, to determine that the identified individual has an abnormality or a sign of an abnormality. This allows the activity patterns of an individual to be compared with other individuals kept in the same area, offsetting other environmental factors and allowing for accurate identification of abnormal individuals. (Appendix 5) The activity status estimation device comprises: 2. The farming management system described in Appendix 1, comprising an anomaly detection unit that determines whether an abnormality or a sign of an abnormality exists when a predetermined number or more of livestock stay in a predetermined section within the farming area for a predetermined period of time or more, based on the positions within the area of ​​multiple livestock raised within the farming area. This can contribute in particular to freedom from pain, injury and disease. (Appendix 6) The area map further includes feeding or watering equipment installed in the breeding area and its location within the area, using the postures and positions within the area of ​​the plurality of livestock and position information of the rearing facility, estimating activity states of the plurality of livestock from a relationship between the postures and the rearing facility; The husbandry management system described in Appendix 5, further comprising a section anomaly detection unit that determines whether or not there is an abnormality or a sign of an abnormality when a predetermined number or more of livestock are staying in a predetermined section centered on the husbandry facility for a predetermined period of time or more and the staying livestock are in a predetermined activity state. This can contribute, among other things, to freedom from hunger, thirst and malnutrition. (Appendix 7) A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: a position measuring device capable of identifying each of the livestock kept in the keeping area and acquiring position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The area map further includes the type of rearing equipment installed in the rearing area and its location within the area, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map information; The rearing management system is provided with an anomaly detection unit that determines whether or not there is an abnormality or a sign of an abnormality when a predetermined number or more of livestock stay in a predetermined section within the rearing area for a predetermined period of time or more, based on the positions within the area of ​​multiple livestock reared within the rearing area. This can contribute, among other things, to freedom from hunger, thirst and malnutrition. (Appendix 8) A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: a position measuring device for identifying each of the livestock kept in the keeping area and measuring its position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map information; A husbandry management system comprising an individual abnormality detection unit that uses the position of the livestock within the area to determine whether there is an abnormality or a sign of an abnormality based on the relationship between the posture and the husbandry equipment. This can contribute, among other things, to freedom from hunger, thirst and malnutrition. (Appendix 9) The livestock management system according to any one of claims 1, 2, 5 and 6, wherein the livestock identification unit identifies the individual captured in the image by an identification code coded according to a combination of highly saturated colors of a plurality of ear tags having a basic geometric shape worn by the individual. This makes it possible to identify individual livestock with high accuracy using simple means. (Appendix 10) A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: An imaging device that captures images within the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: A livestock identification unit that identifies an individual captured in the image by an identification code that is coded according to a combination of highly saturated colors of a plurality of ear tags having a basic geometric shape attached to the individual; a posture estimation unit that estimates a posture of the individual using the image; and an activity state estimation unit that estimates the activity state of the individual using the posture of the individual. This allows farm animals to be kept in a manner that contributes to the freedom of farm animals to express their normal behavioural patterns, such as in free stalls, whilst also contributing to other freedoms and thereby contributing to all five freedoms of animal welfare. (Appendix 11) A method for managing a plurality of livestock kept in a specific rearing area in a movable state using a rearing management system, comprising: An imaging step in which an imaging device captures an image of the rearing area; a position measuring step in which a position measuring device identifies each of the livestock kept in the keeping area and measures position information of the livestock; a sensing data acquisition step in which a sensing data acquisition unit of the activity status estimation device acquires the image and the position information; a livestock identification step in which a livestock identification unit identifies the individual captured in the image; a posture estimation step in which a posture estimation unit estimates a posture of the individual using the image; a possible in-area position calculation step in which a position calculation unit calculates an in-area position of the individual within the rearing area by using the position information and area map information stored in a storage unit; An activity state estimation step in which an activity state estimation unit estimates the activity state of the individual using the posture of the individual and the in-area position information. This allows livestock to be raised in a way that realizes the freedom to express normal behavioral patterns of livestock, such as in the free stall system, while also realizing other freedoms, thereby realizing all five freedoms of animal welfare. [Explanation of symbols]

[0156] 1. Animal husbandry management system 10 Activity status estimation device 20 User terminal 30 Imaging device 40 Position measuring device 50 Additional Sensors 110 Storage section 111 Application Program 112 Livestock Basic Data 113 Livestock image data 114 Area Map Data 115 Activity Status Data 116 Livestock Detection Data 117 Livestock Identification Data 118 Data for posture estimation 119 Additional Sensing Data 120 Processing section 121 Sensing data acquisition unit 122 Livestock detection unit 123 Area location detection unit 124 Livestock Specification Department 125 Posture estimation section 126 Activity State Estimation Unit 127 Anomaly detection unit

Claims

1. A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: An imaging device that captures images within the breeding area; a position measuring device for identifying each of the livestock kept in the keeping area and measuring its position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map; A livestock identification unit for identifying an individual captured in the image; a posture estimation unit that estimates a posture of the individual using the image; and an activity state estimation unit that estimates the activity state of the individual using the posture of the individual and its position within the area.

2. The area map further includes the type of rearing equipment installed in the rearing area and its location within the area, 2. The breeding management system according to claim 1, wherein the activity state of the individual is estimated from the relationship between the posture and the breeding equipment using the posture and position within the area of ​​the individual, and the position within the area of ​​the breeding equipment.

3. The activity status estimation device comprises:

3. The husbandry management system according to claim 1 or 2, further comprising an anomaly detection unit that calculates the frequency, interval, number of times, or cumulative time of the individual's activity state, or statistics thereof, and compares it with the frequency, interval, number of times, or cumulative time of the activity state of a plurality of livestock kept in the keeping area, or statistics thereof, to determine whether the individual has an abnormality or a sign of an abnormality.

4. The activity status estimation device comprises:

4. The husbandry management system according to claim 3, further comprising an anomaly detection unit that detects at least a transition from a first activity state of the individual to a second activity state different from the first activity state as an activity pattern, calculates the frequency, interval, number of times, or cumulative time of occurrence of the activity pattern, or statistics thereof, and compares it with the frequency, interval, number of times, or cumulative time of occurrence of the activity pattern of a plurality of livestock kept in the keeping area, or statistics thereof, to determine that the identified individual has an abnormality or a sign of an abnormality.

5. The activity status estimation device comprises:

2. The farming management system according to claim 1, further comprising an anomaly detection unit that determines, based on the positions within the area of ​​a plurality of livestock kept within the farming area, that there is an abnormality or a sign of an abnormality when a predetermined number or more of livestock stay within a predetermined section within the farming area for a predetermined period of time or more.

6. The area map further includes feeding or watering equipment installed in the breeding area and its location within the area, using the postures and positions within the area of ​​the plurality of livestock and the position within the area of ​​the rearing facility, estimating activity states of the plurality of livestock from a relationship between the postures and the rearing facility; 6. The animal husbandry management system according to claim 5, further comprising an anomaly detection unit that determines whether or not an abnormality or a sign of an abnormality exists when a predetermined number or more of livestock are staying within a predetermined area centered on the animal husbandry facility for a predetermined period of time or more and the staying livestock are in a predetermined activity state.

7. A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: a position measuring device capable of identifying each of the livestock kept in the keeping area and acquiring position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The area map further includes the type of rearing equipment installed in the rearing area and its location within the area, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map; The rearing management system is provided with an anomaly detection unit that determines whether or not there is an abnormality or a sign of an abnormality when a predetermined number or more of livestock stay in a predetermined section within the rearing area for a predetermined period of time or more, based on the positions within the area of ​​multiple livestock reared within the rearing area.

8. A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: a position measuring device for identifying each of the livestock kept in the keeping area and measuring its position information; A memory unit that stores an area map of the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: an area position calculation unit capable of calculating an area position of the individual within the rearing area by using the location information and the area map; A livestock husbandry management system comprising an individual abnormality detection unit that determines whether there is an abnormality or a sign of an abnormality based on the relationship between the position of the livestock within the area and the livestock husbandry equipment.

9. The livestock management system according to any one of claims 1, 2, 5 and 6, wherein the livestock identification unit identifies the individual captured in the image by an identification code coded according to a combination of highly saturated colors of a plurality of ear tags having a basic geometric shape worn by the individual.

10. A breeding management system for a plurality of livestock kept in a specific breeding area in a movable state, comprising: An imaging device that captures images within the breeding area; An activity state estimation device for estimating the activity state of the livestock, The activity status estimation device comprises: A livestock identification unit that identifies an individual captured in the image by an identification code that is coded according to a combination of highly saturated colors of a plurality of ear tags having a basic geometric shape attached to the individual; a posture estimation unit that estimates a posture of the individual using the image; and an activity state estimation unit that estimates the activity state of the individual using the posture of the individual.

11. A method for managing a plurality of livestock kept in a specific rearing area in a movable state using a rearing management system, comprising: An imaging step in which an imaging device captures an image of the rearing area; a position measuring step in which a position measuring device identifies each of the livestock kept in the keeping area and measures position information of the livestock; a sensing data acquisition step in which a sensing data acquisition unit of the activity status estimation device acquires the image and the position information; a livestock identification step in which a livestock identification unit identifies the individual captured in the image; a posture estimation step in which a posture estimation unit estimates a posture of the individual using the image; a possible in-area position calculation step in which a position calculation unit calculates an in-area position of the individual within the rearing area by using the position information and area map information stored in a storage unit; An activity state estimation step in which an activity state estimation unit estimates the activity state of the individual using the posture of the individual and the in-area position information.

Images