Image-based device and method for weighing animals.

Abstract

An embodiment of the present invention of a video-based animal weight measurement device includes a communication module that receives video from at least one camera that photographs a target body, a memory in which an animal weight measurement program is recorded, and a processor that executes the program recorded in the memory. The animal weight measurement program inputs the received video into an animal detection model to extract feature point information of the animal, connects the extracted feature point information to calculate the lateral and vertical lengths of the animal, and inputs the calculated lateral and vertical lengths into a weight estimation model to calculate the weight of the animal.

Description

[Technical Field]

[0001] The present invention relates to an animal weight measurement device and method based on video. [Background technology]

[0002] In the livestock industry, the ability to accurately measure animal weight is a core part of farm management. Weight is used as a key indicator to assess health, growth, and productivity, influencing important decisions regarding nutrition, disease prevention, and breeding, as well as economic value.

[0003] However, conventional methods of weighing livestock are limited by their high cost and potential to stress the animals. In recent years, advances in computer vision and machine learning technologies have led to the proposal of new methods to address these problems.

[0004] Such technologies reduce costs and animal stress through automated data collection and are used to identify livestock and estimate their weight by leveraging 2D and 3D imaging. This allows for real-time monitoring of animals and optimization of farm management, and can be integrated with existing farm management systems to support real-time decision-making.

[0005] In this regard, systems or methods have been developed to photograph animals and predict their volume, weight, etc., through the images. However, these systems suffer from the problem of reduced accuracy due to interference factors (sunlight, animal movement, etc.) and errors caused by other objects (ground, structures, etc.). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Registered Patent No. 10-2389931 of the Republic of Korea (Title of Invention: Mobile Device with Animal Weight Estimation Application Installed) [Overview of the project] [Problems that the invention aims to solve]

[0007] The present invention aims to solve the above-mentioned problems and to provide an image-based animal weight measurement device and method that estimates the weight of an animal from the horizontal and vertical lengths of the animal calculated using characteristic point information of the animal.

[0008] However, the technical problems that this embodiment aims to solve are not limited to those described above, and other technical problems may exist. [Means for solving the problem]

[0009] As a technical means to solve the above-mentioned technical problems, an animal weight measurement device based on video according to one embodiment of the present invention includes a communication module that receives video from at least one camera that photographs an object, a memory in which an animal weight measurement program is stored, and a processor that executes the program stored in the memory. The animal weight measurement program inputs the received video into an animal detection model to extract feature point information of the individual animal, connects the extracted feature point information to calculate the horizontal and vertical dimensions of the individual animal, and inputs the calculated horizontal and vertical dimensions into a weight estimation model to calculate the weight of the individual animal.

[0010] A method for measuring the weight of an animal based on video using a weight measuring device according to another embodiment of the present invention includes the steps of: (a) inputting video received from at least one camera that photographs an object into an animal detection model to extract feature point information of the animal; (b) connecting the extracted feature point information to calculate the width and height of the animal; and (c) inputting the calculated width and height into a weight estimation model to calculate the weight of the animal.

[0011] An image-based animal weight measurement device according to yet another embodiment of the present invention includes a communication module that receives images from at least one camera that photographs an object, a memory that stores an animal weight measurement program, and a processor that executes the program stored in the memory. The animal weight measurement program is configured to input the received images into an animal detection model to extract information on multiple feature points of the animal, the horizontal and vertical dimensions of the animal, and a bounding box formed to fit the animal. The horizontal and vertical dimensions of the animal, the multiple feature point information, the bounding box information, and the installation height of the camera that captured the images are input into a weight estimation model to output the weight of the animal. The feature point information includes the position of the animal's head (nose), neck, the first dorsal position on the head side (back1) when the back is divided into three parts, the second dorsal position in the center (back2), the third dorsal position on the tail side (back3), the right shoulder, the left shoulder, and the front flank position. The bounding box information includes the position of the armpit or the end of the torso (hip), and includes the center coordinates (Xc, Yc), width (W), length (H), and angle (θ) of the bounding box rotated relative to the reference axis. The weight estimation model is trained to estimate the weight of an animal based on the animal's horizontal and vertical dimensions, information on multiple feature points, information on the bounding box, and the installation height of the camera that captured the video. [Effects of the Invention]

[0012] According to any of the problem-solving methods of the present invention described above, the weight can be efficiently measured by processing the received video to detect and measure the animal.

[0013] Furthermore, by using a weight estimation model based on multiple regression analysis, the actual size and shape of the animal are taken into consideration when estimating weight, providing more accurate results compared to conventional weight measurement methods.

[0014] Furthermore, by providing a non-invasive measurement method for animals, the weight can be measured without stressing the animals.

Brief Description of the Drawings

[0015] [Figure 1] It is a block diagram showing the configuration of an animal weight measurement device based on video according to an embodiment of the present invention. [Figure 2] It is a flowchart showing a method for measuring the weight of an animal based on video according to an embodiment of the present invention. [Figure 3a] It is a diagram for explaining a method of calculating the horizontal length and vertical length of an animal individual by connecting the feature point information of the animal individual extracted by the animal detection model according to an embodiment of the present invention. [Figure 3b] It is a diagram for explaining a method of calculating the horizontal length and vertical length of an animal individual by connecting the feature point information of the animal individual extracted by the animal detection model according to an embodiment of the present invention. [Figure 4] It is a diagram for explaining the detailed components of an animal weight measurement device based on video according to an embodiment of the present invention. [Figure 5] It is a diagram for explaining the animal detection model of an animal weight measurement device based on video according to an embodiment of the present invention. [Figure 6a] It is a diagram for explaining the observation data collection of a weight estimation model according to an embodiment of the present invention. [Figure 6b] It is a diagram for explaining the observation data collection of a weight estimation model according to an embodiment of the present invention. [Figure 7] It is a diagram showing the comparative performance indicators of a weight estimation model according to an embodiment of the present invention. [Figure 8] It is a diagram for comparing and explaining the weight estimated value and the correct weight value of the OLS model according to an embodiment of the present invention. [Figure 9] It is a diagram for explaining the accuracy of the weight estimated value of an animal weight measurement device based on video according to an embodiment of the present invention. [Figure 10] It is a diagram for explaining a weight estimation model according to another embodiment of the present invention. [Figure 11] This figure illustrates the learning process of a weight estimation model according to another embodiment of the present invention. [Figure 12] This figure illustrates the performance of a weight estimation model according to another embodiment of the present invention. [Modes for carrying out the invention]

[0016] The present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in various forms and is not limited to the embodiments described herein. Furthermore, the accompanying drawings are provided to facilitate understanding of the embodiments disclosed herein and do not limit the technical ideas disclosed herein. In order to clearly illustrate the present invention in the drawings, parts that are not relevant to the description have been omitted, and the size, shape and form of each component shown in the drawings can be varied in various ways. Throughout this specification, identical or similar parts are denoted by the same or similar reference numerals.

[0017] The suffixes "module" and "part" used in the following description for the components are added solely for the convenience of drafting this specification and do not have any distinguishing meaning or role. Furthermore, in describing the embodiments disclosed herein, if it is determined that a specific description of the relevant prior art would obscure the gist of the embodiments disclosed herein, such detailed description will be omitted.

[0018] Throughout this specification, when a part is described as being “connected (in contact, joined or linked)” to another part, this includes not only “directly connected (in contact, joined or linked)” but also “indirectly connected (in contact, joined or linked)” through other members. Furthermore, when a part is described as “including (providing, equipping)” a component, unless otherwise stated, this does not exclude other components, but rather means that other components may be further “included (equipped or equipped)” them.

[0019] The ordinal terms such as "first" and "second" used herein are used solely for the purpose of distinguishing between components and do not limit the order or relationship of the components. For example, the first component of the present invention may be referred to as the second component, and similarly, the second component may be referred to as the first component.

[0020] One embodiment of the present invention will be described in detail below with reference to the attached drawings.

[0021] Figure 1 is a block diagram showing the configuration of an image-based animal weight measurement device according to one embodiment of the present invention, and Figure 2 is a flowchart showing an image-based animal weight measurement method according to one embodiment of the present invention.

[0022] Referring to Figure 1, the video-based animal weight measurement device (100) includes a communication module (110), memory (120), and a processor (130), and may also include a database (140). The video-based animal weight measurement device (100) receives video footage captured in real time through multiple CCTV or other cameras (10) located in the barn and uses this to perform the operation of measuring the weight of individual animals.

[0023] The video-based animal weight measurement device (100) can be implemented as a network-connected computer or mobile terminal. Here, the computer includes, for example, a notebook computer, a desktop computer, or a laptop computer, and the mobile terminal can include, for example, all types of handheld wireless communication devices such as smartphones, tablet PCs, and smartwatches, as wireless communication devices that guarantee portability and mobility. The video-based animal weight measurement device (100) can also function as a server that provides weight measurement results for individual animals based on video footage of animals received from an external source. In this case, the server may operate on a cloud computing service model such as SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service), or it may be built in the form of a private cloud, public cloud, or hybrid cloud.

[0024] A network refers to a connection structure that enables information exchange between nodes such as terminals and devices, and includes local area networks (LANs), wide area networks (WANs), the Internet (WWW), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WiMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth® communication, infrared communication, ultrasonic communication, visible light communication (VLC), and LiFi.

[0025] The communication module (110) can receive video footage of objects captured by cameras (10) installed within a predetermined height range of the animal farm and transmit it to the processor (130). The objects include various types of animals such as cows, pigs, and dogs. A detailed explanation of the camera (10) placement will follow later.

[0026] The communication module (110) may be a device that includes hardware and software necessary for connecting with other network devices by wire or wireless means and for sending and receiving signals such as control signals or data signals.

[0027] Memory (120) may contain an animal weight measurement program. The animal weight measurement program inputs the received video into an animal detection model to extract feature point information of the individual animal, connects the feature point information to calculate the width and height of the individual animal, and inputs the width and height of the individual animal into a weight estimation model to calculate the weight of the individual animal.

[0028] In this context, memory (120) is interpreted as a collective term for non-volatile memory devices that retain stored information even without power supply and volatile memory devices that require power to retain stored information. Memory (120) can perform the function of temporarily or permanently storing data processed by the processor (130). In addition to volatile memory devices that require power to retain stored information, memory (120) may also include magnetic storage media or flash storage media, but the scope of the present invention is not limited thereto.

[0029] The processor (130) executes an animal weight measurement program (hereinafter referred to as "the program") stored in memory (120), and provides the function of controlling the hardware of the image-based animal weight measurement device (100) by executing the program. In other words, by executing the program, the processor (130) can perform necessary hardware control functions such as file system, memory allocation, network, basic libraries, timer, device control (display, media, input device, 3D, etc.), and other utilities.

[0030] Referring to Figure 2, the processor (130) inputs the received video into an animal detection model to extract feature point information of individual animals (S110), connects the feature point information to calculate the width and height of the individual animal (S120), and inputs the width and height of the individual animal into a weight estimation model to calculate the weight of the individual animal (S130). The specific steps of the animal weight measurement process by program execution will be described later with reference to Figures 3 to 9.

[0031] In another embodiment, the processor (130) is configured to input the received video to an animal detection model, extract information on multiple feature points of the animal, the width and height of the animal, and information on a bounding box formed to fit the animal, and input the width, height, multiple feature points, bounding box information, and the installation height of the camera that captured the video to a weight estimation model to output the weight of the animal. A detailed explanation of the animal weight estimation process performed by program execution will be described later with reference to Figures 10 to 12.

[0032] The processor (130) can include all types of devices capable of processing data. For example, it can mean a data processing device embedded in hardware that has physically structured circuitry to perform a function represented by code or instructions contained in a program. Examples of such data processing devices embedded in hardware include, but are not limited to, microprocessors, central processing units (CPUs), processor cores, multiprocessors, ASICs (application-specific integrated circuits), and FPGAs (field programmable gate arrays).

[0033] The database (140), under the control of the processor (130), stores or provides data necessary for the image-based animal weight measurement device (100). For example, the database (140) cumulatively stores weights extracted by an animal weight estimation program and makes them available for various applications to monitor the animal's condition based on these weights. Such a database (140) may be included as a separate component from the memory (120) or may be built in a portion of the memory (120).

[0034] Figures 3a and 3b illustrate a method for calculating the horizontal and vertical dimensions of an animal by connecting characteristic point information of an animal according to one embodiment of the present invention, and Figure 4 illustrates the detailed components of an image-based animal weight measurement device according to one embodiment of the present invention.

[0035] First, referring to Figure 4, the processor (130) can implement detailed modules that perform various functions by executing an animal weight measurement program. For example, the animal weight measurement program is executed by the processor (130) and can implement an animal detection model (200) and a weight estimation model (300).

[0036] Referring to Figure 3a, the program inputs the video feed into the animal detection model (200) and extracts individual detection information. The individual detection information includes information about the bounding box formed to fit the animal individual detected from the input video, and information about the animal individual's feature points. For example, information about the animal individual's bounding box may include the coordinates of the center point of the bounding box (Xc, Yc), the width of the bounding box (W), the length of the bounding box (H), and the angle (θ) at which the bounding box is rotated with respect to the reference axis. In addition, information about the animal individual's feature points may include the position of the animal's head end (nose), neck, the first back position (back1) closest to the head, the second back position (back2) in the middle, the third back position (back3) closest to the tail, the right shoulder position, the left shoulder position, the front armpit position, and the hip position.

[0037] Furthermore, individual detection information can include additional information about the type (class) of the animal detected from the video. This information can be used to classify animals not only by species but also by growth stage within the same species. For example, in pig farming, animals can be classified into types such as suckling piglets, weaned piglets, growing pigs, fattening pigs, candidate pigs, pregnant pigs, and farrowing pigs.

[0038] On the other hand, training data for each video can be generated through a process of labeling each video with ground truth data, including the center point coordinates of the bounding box, the width of the bounding box, the length of the bounding box, the rotation angle of the bounding box, and the characteristic point information of the individual animal. To achieve this, labeling can be carried out by having experts manually identify the bounding boxes and determine the center point coordinates, width, and length of the bounding boxes. Alternatively, training data can be prepared by displaying the characteristic point information of the individual animal for each video.

[0039] Next, the program connects the feature point information of individual animals extracted from the animal detection model to calculate the horizontal and vertical dimensions of the individual animals.

[0040] For example, the program can calculate the width (SWM) of an animal by determining the length of the line segment connecting the right shoulder and left shoulder. It can also calculate the length (SLM) of an animal by determining the length of the line segment connecting the neck to the hip.

[0041] To achieve this, the program can use calibration pattern video captured by the camera (10) to convert the pixel distance between feature point information of individual animals extracted from the input video into a distance in centimeters.

[0042] For example, the program can perform distortion correction by using a calibration pattern image positioned so that the center of the chessboard is in the center of the camera's (10) field of view, so that the pixel distances of all the edges of the chessboard are uniform. For reference, as shown in Figure 6a later, the chessboard is made up of multiple squares arranged in a grid, and is mainly used to correct distortion in camera-captured images. In this case, the distortion correction method can utilize VPI (Vision Programming Interface) and a polynomial distortion model to uniformly represent the rectangles of the chessboard in the image. Since such a distortion correction method is a prior art, a detailed explanation of it will be omitted.

[0043] Next, the program can convert the pixel distances between feature points of individual animals extracted from the input video into centimeter distances. This is calculated using the following formula 1.

[0044]

number

[0045] In this case, chessboard pixels represent the pixel distance between the edges of the chessboard, and chessboard centimeters represent the actual length of the edges of the chessboard.

[0046] For example, the program can determine the horizontal length based on the distance between the shoulders from the feature point information, and the vertical length based on the distance between the neck and the end of the torso from the feature point information. As an example, the pixel-per-centimeter ratio value (p2c) can be used to determine the pixel distance between the right and left shoulders (dist right shoulder, left shoulder The actual width (SWM) of an individual animal can be obtained by multiplying it by (). The program also calculates the sum of the distances between the neck and the first dorsal point, the first and second dorsal points, the second and third dorsal points, and the third dorsal point and the end of the torso (dist neck, back1 +dipback1, back2 +dip back2, back3 +dip back3, hip By multiplying by , the actual vertical length (SLM) of an animal can be determined. Furthermore, the horizontal length can be determined along the horizontal direction of a livestock individual based on the distance between feature points at one end and feature points at the other end, and the vertical length can be determined along the vertical direction of a livestock individual based on the distance between feature points at one end and feature points at the other end. For example, feature points arranged along the horizontal direction of a livestock individual may include the left shoulder, right shoulder, etc. Also, feature points arranged along the vertical direction may include the nose, eyes, head, neck, back, tail, hips, etc.

[0047] Next, the program inputs the calculated horizontal (SWM) and vertical (SLM) values ​​into the weight estimation model (300) to calculate the weight of the individual animal.

[0048] The following describes the weight estimation model (300).

[0049] On the other hand, Figure 3b shows the upper triangular correlation matrix, representing SLM (body_cm) and SWM (shoulder_cm), which show the highest correlation with the correct weight (weight_answer) among the variables connected by feature points. Thus, in this invention, to construct the weight estimation model (300), horizontal (SWM) and vertical (SLM) variables were used from among the variables connected by feature points.

[0050] The weight estimation model (300) is a multiple regression analysis model, and the weight of individual animals can be calculated using Equation 2.

[0051]

number

[0052] Here, W is body weight, SLM (Spine Length Measure) is the vertical length of the animal, SWM (Shoulder Width Measure) is the horizontal length of the animal, and β0, β1, and β2 are regression coefficients adapted to the growth trajectory of the animal.

[0053] The program can collect observational data matching the width, height, and ground truth weight values ​​of individual animals to construct a weight estimation model (300), generate an OLS (Ordinary Least Squares) regression model based on the observational data, and determine the regression coefficients in Equation 1. The process of collecting observational data and generating the OLS regression model will be described later with reference to Figures 6a to 9.

[0054] The regression coefficients (β0, β1, and β2) can be optimized to minimize the error (e) in the multiple regression equation.

[0055] The program transforms equation 2 into a multiple regression matrix form as shown in equation 3, calculates the β matrix that minimizes the SSE function of the multiple regression according to equation 4, and can find the optimal β value.

[0056]

number

[0057]

number

[0058] The following describes the animal detection model that extracts the characteristic point information of the individual animals mentioned above.

[0059] Figure 5 is a diagram illustrating an animal detection model for a video-based animal weight measurement device according to one embodiment of the present invention.

[0060] The animal detection model (200) used in the present invention is constructed based on training data obtained by matching multiple videos containing at least one individual animal with individual detection information (feature point information) related to the individual animal contained in each video. After being trained through the learning process, the animal detection model (200) can automatically output individual detection information for actual input videos through an inference process.

[0061] The training data used in the training process of the animal detection model (200) includes multiple videos and individual detection information matched to each video. In this case, the individual detection information is manually extracted for each video. That is, experts can either manually view each video and directly input the individual detection information using appropriate software tools, or they can use an existing developed animal detector to automatically input the information, after which experts can correct / complete it. For example, for each individual animal in the video, the expert displays a bounding box considering the direction of rotation relative to the reference axis of the individual animal, and generates information such as the center coordinates of each bounding box, the width of the bounding box, the length of the bounding box, and the rotation angle of the bounding box relative to the reference axis. In addition, the worker extracts information about the type of animal individual and uses it as training data.

[0062] As shown in Figure 4, the animal detection model (200) may include a backbone (210), a neck (220), and a head (230).

[0063] The backbone (210) extracts features from the input video and is a component commonly used in video analysis and processing methods based on deep neural networks. The backbone (210) mainly takes the form of stacked 2D convolutional layers, and various neural network structures have been devised to improve its efficiency. All backbones of various structures receive video as input and play the role of extracting intermediate information, which is then transmitted to the neck (220).

[0064] The neck unit (220) integrates intermediate information from each layer of the backbone unit (210) based on the features extracted by the backbone unit (210). As a lower-level neural network constituting a general object detector, the neck unit (220) plays the role of integrating and analyzing the layer-specific intermediate information of the backbone unit (210). Because the resolution of the images analyzed in each layer differs, the neck unit (220) extracts layer-specific intermediate information and provides it to the head unit (230) in order to effectively detect animals of various sizes depending on whether the target is far away or close, or depending on the animal's body shape. The specific configuration of the neck unit (220) differs depending on the form of the backbone unit (210) used before it, and the number of layers and layer-specific hyperparameters of the specific neural network constituting the neck unit (220) may change depending on the form of the backbone unit (210).

[0065] The head unit (230) outputs individual detection information based on intermediate information collected by the neck unit (220). The head unit (230) receives intermediate information obtained from the neck unit (220) as input and outputs individual detection information. The head unit (230) receives intermediate information from each layer of the neck unit (220) as input and outputs individual detection information recognized for each layer. In particular, the head unit (230) of the present invention includes a plurality of animal detection subnets, and each animal detection subnet includes a subnet for extracting bounding boxes and feature point information, and a subnet for extracting animal species, as shown in Figure 5.

[0066] On the other hand, an NMS (Non-maximum Suppression) module may be further connected to the output terminal of the head unit (230). This is an algorithm that selects the boundary box with the highest similarity when multiple boundary boxes are generated for the same object, and since this is a well-known technique, a detailed explanation will be omitted.

[0067] The subnets from which bounding boxes and feature point information are extracted are composed of cascaded multi-lane deep convolutional networks. These cascaded multi-lane deep convolutional networks are structured according to a causal sequence for detecting bounding boxes and feature point information for a given animal image. The following causal sequence is followed to define one individual detection piece within each image.

[0068] Specifically, as shown in Figure 3a, first, the center point (Xc, Yc) and the feature point information (nose, ..., hip) described above are noted. Next, tangent lines are drawn that intersect the center point and one or more of the feature point information (excluding the right and left shoulder positions). Finally, the region (surface) through which the tangent line passes is specified.

[0069] In a hierarchical multi-lane deep convolutional network with this structure, information is transmitted according to the causal order described above, and each piece of information is output. Specifically, the first path outputs the center point and feature point information, the second path outputs the direction (θ) of the tangent, and the third path outputs the width (W) and height (H) of the region (bounding box) containing the tangent and the center point.

[0070] On the other hand, the subnets used to extract animal species are obtained using a general structure, namely a single-path deep convolutional network.

[0071] Such an animal detection model (200) can also be expressed by the following mathematical formula.

[0072]

number

[0073] In this case, A={R,K,c,p} represents vectorized individual detection information, M(x) represents the animal detection model (200), I represents the input image matrix (image width × image height × image channel dimension), and E(A) represents encoded individual detection information. Also, B(x), N(x), and H(x) represent the backbone (210), neck (220), and head (230), respectively.

[0074] The animal detection model (200) is constructed through a process in which the weights of the animal detection model (200) are repeatedly updated by backpropagation learning, after which the output of the input image matrix is ​​learned to match the encoded individual detection information E(A).

[0075] The training data used in the training process of the animal detection model (200) includes multiple images and individual detection information matched to each image. In this case, some of the individual detection information may be manually extracted for each image. That is, the operator can examine each image, identify the bounding box using an appropriate software tool, and directly input information such as the center point coordinates, width, length, and feature points of the animal individual in the bounding box, or they can use an existing animal detector to automatically input the information, after which the operator can correct and complete it. For example, the operator can display a bounding box for each animal individual in the image, considering the direction of rotation with respect to the reference axis of the animal individual, and generate the center point coordinates, width, and length of each bounding box. Then, for each animal individual contained within each bounding box, the operator generates nine feature point pieces of information, including the nose position, neck position, first back position (back1), second back position (back2), third back position (back3), right shoulder position, left shoulder position, front armpit position, and torso end position (hip), and include them in the training data. Furthermore, the operators extract additional information about the species of individual animals and utilize it as training data. The individual detection information included in the training data undergoes an encoding process before being used in the training process. The individual detection information is encoded through the following process:

[0076] First, for each head unit (230), the region of interest (R a Specify the region of interest (R). a The number of ) is defined by the processing area × size type × angle type × box ratio for each head unit (230).

[0077] Furthermore, the degree of overlap between the animal region and the region of interest recorded in the individual detection information (A) (O a,k ) is calculated using the following formula.

[0078]

number

[0079] Here, IoU(x, y) calculates the degree of overlap between two bounding boxes.

[0080] Next, for each region of interest, only the animal region (R k’ ) with the highest degree of overlap is selected. Here,

Number

[0081] Next, encoding between the region of interest and the corresponding animal region is performed.

[0082]

Number

[0083] At this time,

Number

[0084]

Number

Number

[0085] As a result, the output is as follows.

Number

[0086] Similarly,

Number

[0087]

Number

number

[0088] As a result, the output will be as follows:

number

[0089] The detection information encoded through the process described above is used in the construction of the animal detection model (200).

[0090] On the other hand, backpropagation learning may be used in the training process of the animal detection model (200). That is, the loss value between the encoded individual detection information E(A) and its estimate is calculated, and the neural network parameters that make up the animal detection model (200) are updated in a process that is repeated so as to minimize this loss value. For example, L1 or L2 loss can be used to calculate the loss values ​​for the bounding box (rbox) and the animal individual's feature points (keypoints), while a discrimination loss such as binary cross entropy loss or focal loss can be used for the loss values ​​for the animal individual's species (c) and animal individual's posture (p).

[0091] Using this loss function, we construct an animal detection model (200) by repeatedly training until the sum of the losses falls below the target value.

[0092] The process of inferring individual detection information (A) from an input image using the animal detection model (200)(M(x)) constructed in this manner is described below. This can be expressed mathematically as follows.

[0093]

number

[0094] In other words, when an input image is input to the animal detection model (200), encoded detection information (E(A)) can be obtained. Then, a process is executed to decode the encoded detection information.

[0095]

number

[0096] and,

number

number

[0097]

number

number

[0098] As a result, the output will be as follows:

number

[0099] Similarly,

number

[0100]

number

number

[0101] As a result, the output will be as follows:

number

[0102] The individual detection information (R, K, c, p) output by such a decoding process may contain a large amount of redundant information for a single animal individual. To address this, an algorithm can be applied to remove the redundant individual detection information.

[0103] To visually confirm this individual detection information, it is possible to overlay the individual detection information onto the image.

[0104] Figures 6a and 6b are diagrams illustrating the collection of observational data for a weight estimation model according to one embodiment of the present invention, and Figure 7 is a table summarizing the movement sessions of pigs at multiple farms, illustrating the observational data for a weight estimation model according to one embodiment of the present invention.

[0105] Referring to Figure 6a, the camera (10) installation height was set to 1.8 meters to 2.5 meters to adapt to various farm infrastructures, with a desirable installation height of 2.2 meters for optimal coverage. The camera (10) field of view was also set to a width of at least 0.9 meters to enable complete animal imaging. Meanwhile, an automatic adjustment device was used to precisely align the camera's field of view with the aisle width, ensuring uniform image quality and data accuracy.

[0106] Figure 6b is a table summarizing pig transfer sessions at various farms, categorized by group composition, and showing the distribution of O80 and U40 pigs.

[0107] As shown in Figure 6b, in one example, data was collected over a total of 290 pig relocation sessions at five different farms. This dataset includes the number of pig relocations, total weight, and video footage of the pigs relocating. In this invention, the data collection strategy was structured according to the growth stage of the pigs, classifying them into O80 (80 kg or more) and U40 (less than 40 kg) to ensure age-appropriate accuracy.

[0108] For the O80 group of adult pigs, individual weight measurement was impractical, requiring 10-20 minutes for each measurement. Therefore, transport weight certificates were utilized. These documents recorded the vehicle weight before and after loading the pigs. The average weight per pig was calculated from the net weight of the pigs to collect the correct gross weight (GT) values ​​for adult pigs.

[0109] For the younger U40 group of pigs, weight was collected using multi-pig scales that can efficiently weigh multiple pigs. Accurate average weight was calculated before farm relocation, and the correct weight values ​​for the young pigs were collected.

[0110] This dual data collection method simplifies the observational data collection stage and improves the reliability of weight estimation models for adult and young pigs.

[0111] Using an animal detection model (200), individual pig detection information was extracted, and the pixel-based feature point distances were converted to centimeter distances to obtain the pig's length and width (SLM / SWM). An OLS model was generated to accurately estimate the weight from these individual animal lengths and widths. The OLS model undergoes an iterative correction process to minimize the residual, which is the difference between the ground truth weight and the predicted weight. The orientation of the pig as it passes through the imaging region is also considered to prevent measurement distortion due to viewpoint.

[0112] This consideration of direction can be expressed by the following weight estimation formula.

number

[0113] Here, w * This shows an improved weight estimate, N l and N r The numbers indicate the number of pigs moving to the left and to the right, respectively, w l,i and w r,j These are the estimated weights for each individual. In this way, accurate weight estimation is achieved through data integration that takes directionality into account.

[0114] The prediction accuracy of the OLS model according to the present invention was improved by adjusting it to harmonize with the known average weight of the target group. This was achieved by fine-tuning the prediction accuracy by gradually reducing the range of prediction errors.

[0115] Figure 7 shows comparative performance indicators of a weight estimation model according to one embodiment of the present invention.

[0116] Referring to Figure 7, when an OLS model is constructed using the entire dataset, the average relative error is 0.0499. On the other hand, when the U40 and O80 data are separated and separate OLS models are constructed for each, the O80 OLS model shows a lower error rate and improved accuracy. In particular, the OLS model integrating each OLS (U40 and O80) shows an average relative error almost equivalent to that of existing DNN models. Thus, in this invention, by applying OLS to a weight estimation model, it is possible to achieve accuracy without loss of precision compared to DNNs, while also being simpler and reducing computational costs.

[0117] Figure 8 illustrates a comparison of the estimated weight and the true weight of an OLS model according to one embodiment of the present invention, and Figure 9 illustrates the accuracy of the weight estimate of an image-based animal weight measurement device according to one embodiment of the present invention.

[0118] As shown in Figure 8, the range of maximum error rates in the test sets for each farm was 5.8% to 17.4%, indicating an area where improvement is possible.

[0119] Figure 9 shows the prediction results of the OLS model for two segments, U40 and O80. In other words, the weight estimation results by the OLS model for the U40 and O80 groups demonstrate the accuracy of the model across various weight categories.

[0120] Figure 10 illustrates a weight estimation model according to another embodiment of the present invention, Figure 11 illustrates the learning process of a weight estimation model according to another embodiment of the present invention, and Figure 12 illustrates the performance of a weight estimation model according to another embodiment of the present invention.

[0121] A weight estimation model (310) according to another embodiment of the present invention will be described below.

[0122] The animal weight measurement program inputs the received video into an animal detection model (200) to extract information on multiple feature points of the animal, the horizontal and vertical dimensions of the animal, and information on a bounding box formed to fit the animal. It then inputs the horizontal and vertical dimensions of the animal, the information on multiple feature points, the bounding box, and the installation height of the camera that captured the video into a weight estimation model (310) to output the weight of the animal.

[0123] Here, feature point information includes the position of the animal's head (nose), neck, the first back position (back1) near the head (dividing the back into three equal parts), the second back position (back2) in the center, the third back position (back3) near the tail, the right shoulder, the left shoulder, the front armpit, or the hip. In addition, information about the bounding box includes the coordinates of the center point of the bounding box (Xc, Yc), width (W), length (H), and rotation angle (theta) relative to the reference axis.

[0124] For example, the horizontal length of an animal individual is the sum of the pixel distances between the neck and the first dorsal position, the pixel distance between the first and second dorsal positions, the pixel distance between the second and third dorsal positions, and the pixel distance between the third dorsal position and the end of the torso, among the feature point information of the animal individual. The vertical length of an animal individual may be the pixel distance between the right shoulder and the left shoulder, among the extracted feature point information of the animal individual.

[0125] The weight estimation model (310) was trained to estimate the weight of an individual animal based on the animal's horizontal and vertical dimensions, information on multiple feature points, information on the bounding box, and the height at which the camera capturing the image was installed.

[0126] Referring to Figure 10, the program generates sequence data based on the horizontal and vertical dimensions, multiple feature point information, bounding box information, and the installation height of the camera that captured the video of the animal individual extracted from the aforementioned animal detection model (200). The generated sequence data for each animal individual is then input into the weight estimation model (310) to output a predicted weight for each animal individual.

[0127] Referring to Figure 11, the sequence data is generated by grouping data obtained from multiple frames, including the body, shoulder, keypoints, bounding box information, and camera height, into a predetermined sequence length (number of frames). For example, if the sequence data has a sequence length (S) of 10 and 24 features per frame (X), it is structured in the format (B, 240), where B represents the batch size.

[0128] As an example, the weight estimation model (310) can take sequence data as input and output a predicted weight of an individual animal by performing sequential processing based on such sequence data as follows.

[0129] First, the weight estimation model (310) can perform a multihead attention operation on the input sequence data. Such a multihead attention operation is designed to learn by simultaneously considering the interrelationships between each frame that makes up the sequence from multiple perspectives, thereby integrating time-series motion features and important frame information to generate a high-dimensional embedding vector (B,E) that represents the sequence.

[0130] Next, the weight estimation model (310) can apply a scaled sigmoid activation function to the embedding vector to normalize the output value to a range between 0 and 1. This normalized value enhances the stability of model learning and adjusts the value to a limited range so that it can be converted to actual weight units in subsequent stages. For example, the scaled sigmoid activation function compensates for the resolution limitations of a general sigmoid function and includes alpha and beta parameters to improve prediction sensitivity, especially in the low weight range (e.g., 3-20 kg). The weight estimation model (310) is designed to maintain uniform prediction sensitivity across the entire weight range by searching for optimal alpha and beta values ​​during the learning process.

[0131] Next, the weight estimation model (310) can convert the normalized values ​​into values ​​within the actual weight range through weight denormalization. For example, the weight estimation model (310) can correct the normalized values ​​(e.g., values ​​between 0 and 1) based on predefined minimum weight (e.g., 3 kg) and maximum weight (e.g., 200 kg) to finally calculate the predicted weight (B, W) in kilograms.

[0132] Next, the learning process of a weight estimation model (310) according to another embodiment of the present invention will be described.

[0133] The weight estimation model (310) is constructed using training data that includes sequence data grouped by a predetermined sequence length based on multiple frames, each containing the horizontal and vertical dimensions of the animal, information on multiple feature points, information on the bounding box, and camera installation height, as well as average weight information obtained from movement record reports as the ground truth weight for each animal corresponding to the sequence data.

[0134] Here, the movement record report is defined based on the shipment performance report, and each report includes information on the total number of animals shipped and the total weight. This allows the average weight based on the report to be used as the ground truth data, and in most cases, actual weight information for individual animals does not exist separately.

[0135] The sequence data used to train the weight estimation model (310) is constructed based on movement information (sequence) tracked individually through an animal detection model (200) and an existing tracking model. Each frame is sorted and stored in chronological order, and only frames that satisfy a predefined pose for weight measurement are filtered and used in sequence construction.

[0136] Referring to Figure 11, the process of constructing the training data for the weight estimation model (310) will be explained.

[0137] As an example, track IDs that meet the minimum sequence length requirement are selected, and sampling is performed for each individual based on a set sequence length (e.g., 10 frames) from frames arranged in chronological order. This sampling is repeated multiple times (M times) for each track ID, generating multiple sequence samples for the same individual.

[0138] As an example, each sequence data includes information about the animal's body length, shoulder width, bounding box (rbbox(5)), multiple feature point information (keypoint(16)), and camera installation height (cam_height(1)), and is composed of a 24-dimensional feature vector, and is composed of an input tensor in the form (S×24) when the sequence length S is 10. Sequence sampling is repeated M times for one track ID, generating a total of M sequence data from trk1_s(1) to trk1_s(M). Each sequence data is input to the weight estimation model (310), and sample predicted weights (sample_est_weight) from W(1,1) to W(1,M) are calculated, and M average individual unit weight predictions (object_est_weight) are calculated for the same individual. In other words, individual unit weight predictions are calculated for each individual corresponding to track ID1 to track IDN, and these are averaged to calculate one report unit average weight prediction (report_est_avg_weight). At this time, the average forecast value for the overall report is calculated using the following formula.

[0139]

number

[0140] Here, N represents the total number of track IDs included in the report, M represents the number of sequence samples generated for each track ID, and w ij is the predicted weight value for the j-th sequence sample of the i-th individual, and W is the average of the sequence-predicted weights for all individuals, representing the report estimated average weight for that report unit.

[0141] Based on this training data, the weight estimation model (310) is trained using a combination of the five loss functions described below. Each loss function is applied with the same weights, and the overall loss is calculated as their average.

[0142] For example, Embedding Loss is guided to minimize the difference in embedding distance between sequence samples extracted from the same individual ID and maximize the embedding distance between different IDs, and is implemented using Triplet Loss. In this case, the margin is set to 0.1.

[0143] Consistency Loss is a loss function that minimizes the deviation between predicted weight values ​​for multiple sequence samples generated from the same ID, and is implemented using Smooth L1 Loss.

[0144] The Target Loss is a loss function based on the Huber Loss that guides the average of the N track IDs and the M sequence prediction values ​​generated for each ID in each report to resemble the average weight value of that report.

[0145] The Stddev Max Loss is a loss function used to prevent the overall prediction results from being excessively biased towards a specific report mean. It can be used by modifying the Smooth L1 Loss to maintain the variance between prediction values.

[0146] Rank Loss is a loss mechanism that guides the relative ranking of weight predictions within each report to match the relative ranking of the area calculated for each sequence, which is the product of the body length and shoulder width of the individual animal. It is implemented using ListNet Loss.

[0147] On the other hand, in training the weight estimation model (310), AdamW can be applied as the optimization function, and CosineAnnealingWarmRestarts can be applied as the learning rate scheduler. These are widely used components to improve the learning stability and generalization performance of the model.

[0148] Referring to Figure 12, the prediction accuracy of the weight estimation model (310) is described. Validation was performed based on evaluation reports collected separately and not used to train the weight estimation model of the present invention. The evaluation included 28 shipment reports collected from a total of five different farms, each containing movement information for a total of 861 individual pigs.

[0149] Each farm has an average weight range from a minimum of approximately 5 kg to a maximum of approximately 130 kg, demonstrating that the weight estimation model of the present invention has predictive performance that can be generalized to various weight ranges.

[0150] The model's performance was evaluated based on the Mean Absolute Error (MAE) and Mean Relative Error (MRE). The prediction results were compared separately for the basic DNN output and the DNN+B output with bias correction applied.

[0151] Specifically, Figure 12 is a table showing the MAE and MRE results, comparing the number of reports, number of individuals, mean bias value (DNN bias), and DNN-based prediction error with the corrected error (DNN+B) for each farm.

[0152] DNN bias represents the average difference between the predicted average weight and the actual average weight for each reporting criterion, and DNN+B shows the MAE and MRE performance calculated after applying this bias value to the predicted values.

[0153] Based on the average of the entire evaluation target, MAE improved from 2.22 kg before correction to 1.77 kg after correction, and MRE decreased from 3.7% before correction to 3.0% after correction, confirming that the weight estimation model (310) of the present invention can provide high prediction accuracy even in actual application environments.

[0154] In particular, in the case of j_farm, the MAE before correction was 3.22 kg and the MRE was 2.8%, but when the DNN+B correction was applied, the MAE decreased significantly to 0.73 kg and the MRE to 0.6%, demonstrating that post-correction of model prediction results is effective in substantially improving accuracy.

[0155] Furthermore, in the case of hs_farm and s2_farm, there was almost no difference before and after correction, which may mean that the model's predictions for the relevant data were already made without bias.

[0156] Therefore, the weight estimation model (310) according to another embodiment of the present invention can provide stable and reliable predictive performance in a variety of farm environments and weight ranges.

[0157] In the following sections, explanations of configurations identical to those described above will be omitted.

[0158] Referring again to Figure 2, the video-based animal weight measurement method using a weight measuring device includes the steps of: inputting video received from at least one camera (10) that photographs the target object into an animal detection model (200) to extract characteristic point information of the animal individual (S110); connecting the extracted characteristic point information to calculate the horizontal and vertical dimensions of the animal individual (S120); and inputting the calculated horizontal and vertical dimensions into a weight estimation model (300) to calculate the weight of the animal individual (S130).

[0159] The video-based method for measuring animal weight may further include a step of constructing a weight estimation model. In this case, the weight estimation model can calculate the weight of an individual animal using Equation 2 described above.

[0160] The stage of constructing a weight estimation model may include collecting observational data where the width, length, and ground truth weight values ​​of individual animals are matched, and generating an OLS (Ordinary Least Squares) regression model based on the observational data to determine the regression coefficients in Equation 2 above.

[0161] The S120 step may include a step to determine the horizontal length based on the distance between shoulders from the feature point information, and a step to determine the vertical length based on the distance between the neck and the end of the torso from the feature point information. It may also include a step to convert the pixel distance between feature point information into centimeter distance using calibration pattern video captured by the camera (10).

[0162] The animal detection model (200) is constructed based on training data obtained by matching multiple videos containing at least one animal individual with feature point information for the animal individual contained in each video, and includes a backbone unit (210) that extracts features from the input video, a neck unit (220) that aggregates intermediate information from each layer of the backbone unit (210) based on the features extracted by the backbone unit (210), and a head unit (230) that outputs feature point information based on the intermediate information collected by the neck unit (220).

[0163] One embodiment of the present invention can also be realized in the form of a recording medium containing computer-executable instructions, such as a program module executed by a computer. A computer-readable medium is any available medium accessible by a computer, and includes all volatile and non-volatile media, removable and non-removable media. A computer-readable medium may also include a computer storage medium. A computer storage medium includes all volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing computer-readable instructions, data structures, program modules, or other data.

[0164] Although the methods and systems of the present invention have been described in relation to specific embodiments, some or all of their components or operations can be implemented using a computer system having a general-purpose hardware architecture.

[0165] The above description of the present application is illustrative, and a person skilled in the art will understand that it can be readily modified into other specific forms without altering the technical idea or essential features of the present application. Therefore, the embodiments described above should be understood in all respects as illustrative and not limiting. For example, each component described in a single form may be implemented in a dispersed manner, and similarly, components described as dispersed may be implemented in a combined manner.

[0166] The scope of this application is defined more by the claims described below than by the detailed description above, and all modified or altered forms derived from the meaning and scope of the claims, as well as the concept of equivalents thereof, should be interpreted as being included within the scope of this application.

Claims

1. In a video-based animal weight measurement device, A communication module that receives video from at least one camera that photographs the target, The memory containing the animal weight measurement program, The system includes a processor that executes a program stored in the memory, The animal weight measurement program is characterized by inputting the received video into an animal detection model to extract feature point information of the animal, connecting the extracted feature point information to calculate the lateral and vertical lengths of the animal, and inputting the calculated lateral and vertical lengths into a weight estimation model to calculate the weight of the animal, and is an image-based animal weight measurement device.

2. In the video-based animal weight measurement device according to claim 1, The aforementioned animal weight measurement program is A video-based animal weight measurement device characterized by determining the lateral length based on the distance between the shoulders from the aforementioned feature point information, and determining the vertical length based on the distance between the neck and the end of the torso from the aforementioned feature point information.

3. In the video-based animal weight measurement device according to claim 1, The aforementioned weight estimation model is characterized by calculating the weight of the individual animal using the following formula 1, and is a video-based animal weight measurement device. [Math 1] Here, W is body weight, SLM (Spine Length Measure) is the longitudinal length of the animal, SWM (Shoulder Width Measure) is the transverse length of the animal, and β ０ , β １ and β ２ This is a regression coefficient that adapts to the growth trajectory of individual animals.

4. In the video-based animal weight measurement device according to claim 3, The aforementioned animal weight measurement program is A video-based animal weight measurement device characterized by collecting observational data in which the lateral length, vertical length, and correct weight values ​​of individual animals are matched, generating an OLS (least squares) regression model based on the observational data, and determining the regression coefficients of the formula 1.

5. In the video-based animal weight measurement device according to claim 1, The aforementioned animal weight measurement program is A video-based animal weight measurement device characterized by converting the pixel distance between characteristic point information of an animal individual into a centimeter distance using calibration pattern video captured by the aforementioned camera.

6. In the video-based animal weight measurement device according to claim 1, The animal detection model is constructed based on training data obtained by matching multiple videos containing at least one animal individual with the feature point information for each animal individual contained in each video. A video-based animal weight measurement device, comprising: a backbone unit that extracts features from input video; a neck unit that aggregates intermediate information from each layer of the backbone unit based on the features extracted from the backbone unit; and a head unit that outputs the feature point information based on the intermediate information collected from the neck unit.

7. In a video-based method for measuring animal weight using a weight measurement device, (a) A step of inputting video footage received from at least one camera that photographs the target object into an animal detection model to extract characteristic point information of the individual animal; (b) A step of connecting the extracted feature point information to calculate the lateral and longitudinal lengths of an animal; and (c) A step of inputting the calculated horizontal and vertical lengths into a weight estimation model to calculate the weight of the animal; A video-based method for measuring animal weight, characterized by including the following:

8. In the video-based animal weight measurement method according to claim 7, The above step (b) is, A video-based method for measuring animal weight, characterized by including a step of determining the lateral length based on the shoulder-to-shoulder distance from the aforementioned feature point information, and a step of determining the longitudinal length based on the distance between the neck and the end of the torso.

9. In the video-based animal weight measurement method according to claim 7, The process further includes the step of constructing the aforementioned weight estimation model, The weight estimation model is characterized by calculating the weight of the individual animal using the following formula 1, and is a video-based method for measuring animal weight. [Math 2] Here, W is body weight, SLM (Spine Length Measure) is the longitudinal length of the animal, SWM (Shoulder Width Measure) is the transverse length of the animal, and β ０ , β １ and β ２ This is a regression coefficient that adapts to the growth trajectory of individual animals.

10. In the video-based animal weight measurement method described in claim 9, The process of constructing the aforementioned weight estimation model is as follows: A video-based method for measuring animal weight, comprising the steps of: collecting observational data in which the lateral length, vertical length, and correct weight values ​​of an individual animal are matched; and generating an OLS (least squares) regression model based on the observational data to determine the regression coefficients of the formula 1.

11. In the video-based animal weight measurement method according to claim 7, The above step (b) is, A video-based method for measuring animal weight, characterized by including a step of converting the pixel distance between characteristic point information of an animal individual into a centimeter distance using calibration pattern video captured by the aforementioned camera.

12. In the video-based animal weight measurement method according to claim 7, The animal detection model is constructed based on training data obtained by matching multiple videos containing at least one individual animal with the feature point information corresponding to the individual animal contained in each video. A video-based method for measuring animal weight, comprising: a backbone unit that extracts features from input video; a neck unit that aggregates intermediate information from each layer of the backbone unit based on the features extracted by the backbone unit; and a head unit that outputs the feature point information based on the intermediate information collected by the neck unit.

13. A non-temporary computer-readable recording medium on which a computer program for performing the video-based animal weight measurement method according to any one of claims 7 to 12 is recorded.

14. In a video-based animal weight measurement device, A communication module that receives video from at least one camera that photographs an object; The memory in which the animal weight measurement program is recorded; and A processor that executes the program stored in the aforementioned memory; Includes, The animal weight measurement program is configured to input the received video into an animal detection model to extract information on multiple feature points of the animal, the lateral and vertical lengths of the animal, and information on a bounding box formed in a shape suitable for the animal. It then inputs the lateral and vertical lengths of the animal, the multiple feature point information, the information on the bounding box, and the installation height of the camera that captured the video into a weight estimation model to output the weight of the animal. The aforementioned feature point information includes the position of the animal's nose, neck, the first back position (back1) near the head when the back is divided into three equal parts, the second back position (back2) in the middle, the third back position (back3) near the tail, the right shoulder position, the left shoulder position, the front armpit position, or the end of the torso (hip). The information relating to the bounding box includes the coordinates of the center point of the bounding box (Xc, Yc), width (W), length (H), and the angle (theta) of rotation of the bounding box with respect to the reference axis. A video-based animal weight measurement device, wherein the weight estimation model is trained to estimate the weight of the animal based on the lateral length, vertical length, information on the multiple feature points, information on the bounding box, and the installation height of the camera that captured the video.

15. In the video-based animal weight measurement device according to claim 14, The aforementioned weight estimation model is For each of the multiple animal individuals, sequence data grouped into a predetermined sequence length based on multiple frames including the lateral length, vertical length, multiple feature point information, bounding box information, and camera installation height of the animal individual, and A video-based animal weight measurement device constructed using training data that includes average weight information obtained from movement record reports as the correct values ​​for the weight of each individual animal corresponding to the aforementioned sequence data.

Images