Animal body parameter estimation based on image processing

Abstract

A computer-implemented image processing method for estimating body parameters of a particular species or breed of an animal. The image acquisition device acquires an image data set including the depth map image to obtain an image of the animal. A plane estimation algorithm analyzes the depth map image to determine a background plane behind the animal. A segmentation algorithm identifies an animal mask. Depth values outside the animal mask are reset as the distance between the viewpoint of the image acquisition device and the estimated background plane. The depth map image is normalized, limiting the data within a fixed range of numerical values, thereby generating a processed depth map image of triplex depth data (T3D) containing distance. The processed depth map image and its reset depth value for the background plane are input to a pre-trained neural network that outputs an estimate of the animal body parameter.

Description

Background of the Invention This invention relates to a method for determining animal body parameter values ​​through computer-implemented image processing utilizing neural networks.

[0002] In animal husbandry, it is usually necessary to measure one or more physical parameters of animals regularly, especially weight and body size, in order to monitor the animals' growth.

[0003] The standard practice involves having the animals pass one by one through a narrow passage from their enclosure into a gated compartment equipped with a weighing device. Once the animals have passed through the passage and are confined in their compartments, the weighing device is used to determine each animal's weight. This process is not only labor-intensive but can also cause stress to the animals.

[0004] To avoid using this standard weighing method, various computer-implemented image processing methods for estimating animal weight have been developed over the past few decades, especially for pig weight estimation. In recent years, many of these methods have employed deep learning neural networks (DLNNs).

[0005] Zhang, J., Zhuang, Y., Ji, H., Teng, G. in […] Pig Weight and Body Size Estimation Using a Multiple Output Regression Convolutional Neural Network: A Fast and Fully Automatic Method 》(Sensors, 2021, 21, 3218, https: / / doi.org / 10.3390 / s21093218 The introduction of [1] summarizes known machine vision methods for pig weight estimation and categorizes them into four types: stereoscopic projection of the pig's shadow onto the ground; image processing of two-dimensional (2D) images of the pig's back; processing of depth map images of the pig's back; and ellipse fitting based on the correlation between geometric parameters identifiable from pig images and pig weight. The research topic described by Zhang et al. is to use a deep learning neural network model that takes two-dimensional images and depth map images as input and determines the weight of the pig by processing these images.

[0006] Suwannakhun, S. and Daungmala, P. in […] Estimating Pig Weight with Digital Image Processing using Deep Learning In the paper “Proceedings of the 14th International Conference on Signal and Image Technology and Internet Information Systems (SITIS), Beijing, China, November 26-29, 2018 [2]”, an example of the above ellipse fitting method based on a neural network is described.

[0007] US2022221325A1 (Viking Genetics FMBA) [3] discloses a computer-implemented image processing method for estimating animal weight, particularly cattle weight. An image of the animal's back is acquired by acquiring a depth map image from above. The animal's weight is then calculated by correlating the depth map image with a reference model containing information on the relationship between the back morphology and weight of the cattle breed being evaluated.

[0008] CN113240574A (Shenzhen Xiwei Smart Technology Co., Ltd.) [4] discloses a computer-implemented image processing method for estimating animal weight. A two-dimensional image of the back of an animal is acquired from above using a fisheye camera, and the height of the animal is estimated based on the two-dimensional image. Optionally, a distance sensor is also set to obtain a more accurate animal height value than using only a fisheye lens. In this case, the first height obtained from the two-dimensional image acquired by the fisheye camera is scaled proportionally to the second height obtained by the distance sensor. The two-dimensional image region related to the animal is segmented by a segmentation algorithm, which can be based on traditional image processing techniques (such as thresholding, morphological transformation, blob analysis, etc.) or on neural networks (i.e., artificial intelligence). Then, the animal height of the sub-regions around each of multiple specific target points on the back of the animal is determined. These height values ​​can be the first height value obtained only from the two-dimensional image, or a combination of the first and second height values ​​obtained by combining the two-dimensional image with the distance sensor data. Then, the corresponding weight value is determined for the sub-regions around each target point, and these weight values ​​are combined to obtain the total weight of the animal. Summary of the Invention

[0009] According to one aspect of this disclosure, a computer-implemented image processing method is provided for determining body parameter values ​​of a specified species or breed of animal, the method comprising: An image dataset is provided for an image region containing depth map images acquired by a camera from a viewpoint, the image region including animals imaged relative to a background plane. The depth map image is analyzed using a plane estimation algorithm to identify the background plane, and the distance between the viewpoint and the background plane is determined accordingly. The segmentation algorithm is applied to the image dataset to identify animals within image regions and define corresponding animal masks. Modify the depth map image by resetting the depth values ​​located outside the animal mask to the distance from the viewpoint to the background plane. The depth map image, along with its reset depth values, is fed into a neural network to determine the animal's body parameters, and The system receives animal body parameter estimates as the output of a neural network, which has been pre-trained by analyzing a dataset of similar images of animals of a specified species and / or breed (whose body parameter values ​​are known).

[0010] By performing comparative performance tests, the performance of the specific implementation of the method of the present invention was compared with that of other methods. For this purpose, the same overall dataset was used, i.e., the same training dataset was used for neural network training, and the same input image dataset (or a subset thereof) was used. The performance test results show that the method of the present invention significantly outperforms the following: Use only RGB images (i.e., do not use depth map images); Use only depth map images (i.e., do not use RGB images); Replace one channel of the RGB image with a depth map image; and The method uses RGB images and depth map images in the same way as the method of the present invention, but does not perform background plane estimation, so that the depth map image values ​​outside the animal mask are not reset to background values, but retain their original acquisition values.

[0011] A significant advantage of this invention is that the background plane distance obtained by analyzing depth map images provides absolute scale for segmented animals. This absolute scale, obtained by identifying the background plane, replaces existing methods that include reference objects (e.g., checkerboard patterns) in the image to achieve absolute scale, particularly for imaging dead pigs. The absolute scale provided by identifying the background plane also means that the viewpoint can vary between different images without affecting the method, thus allowing the use of handheld imaging devices for image acquisition. For example, in a specific embodiment of this invention, the parameter prediction neural network is trained using an image data training set containing images acquired at camera viewpoints ranging from 1.4 to 1.9 meters above the ground and at different camera tilt angles. Therefore, the trained neural network can adapt to situations where the operator holds a handheld imaging device at different heights and tilt angles. Variations in tilt angle between images do not hinder plane estimation, as the tilt angle is automatically determined once the background plane is identified from the depth map image. The free use of handheld imaging devices in this method contrasts with many existing methods that, if not using reference objects to achieve absolute scale, rely on the fixed position of the imager (i.e., the camera). The fixed position sets the camera viewpoint at a known fixed distance (and a fixed tilt angle), using this fixed distance to achieve absolute scale. In such prior art methods, it is the known fixed camera position that allows for the differentiation between small animals closer to the viewpoint and large animals farther away. Similarly, the method of the present invention is applicable not only to processing image datasets acquired by handheld imaging devices but also to processing image datasets acquired by image acquisition devices at fixed positions, in the latter case where position is not a critical factor. For example, since the training dataset used to train the parameter prediction neural network is known in advance, the fixed mounting specifications of the camera can be set accordingly. In an exemplary embodiment of the invention, the camera can be fixed at a height of 1.4 to 1.9 meters above the ground and mounted at an arbitrary tilt angle generally downwards to ensure that the image area includes a sufficiently large ground area considering the field of view of the image acquisition device.

[0012] Another significant advantage of this invention is that its implementation can handle multiple animals contained within a single image region. This is feasible because the segmentation algorithm can be configured to segment any number of animals within the image region, and the planar estimation algorithm can also operate when any number of animals are present in the image region. Furthermore, there is no need to physically constrain the animals within their enclosures to control their posture (e.g., ensuring they remain standing rather than lying down when the image dataset is acquired). Specifically, a neural network can be used as the segmentation algorithm and trained to distinguish between a desired posture (for which the parameter prediction neural network has been trained, such as standing) and other postures (for which the parameter prediction neural network has not been trained) using a binary classifier. Therefore, it is not necessary to ensure that animals are individually confined to their enclosures to ensure they are standing or that only a single animal appears within the image region. Instead, animals can move freely within their regular enclosures, and image datasets can be acquired continuously over a period of time to ensure parameter prediction for all animals. For example, even if some animals are lying down for part of the time or partially occluded due to contact with other animals, they will not be excluded from the prediction. Attached Figure Description

[0013] The invention will now be further described by way of example only and with reference to the accompanying drawings.

[0014] Figure 1 This is a perspective view of a handheld device, which includes an imaging device for acquiring color (RGB) images and depth map images within image area A, wherein... Figure 1A This is a bottom view of the imager.

[0015] Figure 2 It is by Figure 1 An exemplary RGB image captured by the handheld imaging device shown, the image containing at least one animal.

[0016] Figure 3 It is collecting Figure 2 The RGB image shown is simultaneously generated by Figure 1 An exemplary depth map image acquired by the handheld imaging device shown.

[0017] Figure 4 This is a flowchart of an image processing method according to an embodiment of the present invention.

[0018] Figure 5 Show Figure 2 The image shown is a processed version of the RGB image, in which animal pixels have been segmented and identified.

[0019] Figure 6 Show Figure 3 The image shown is an intermediate version of the depth map, in which non-animal pixels identified based on the RGB image segmentation results have been set to null values.

[0020] Figure 7 Show Figure 6 The image shown is a further processed version of the processed depth map image, in which non-animal pixels have been set to values ​​representing the estimated distance to the ground in the depth map image, which is calculated using a plane estimation algorithm. Figure 3 The depth map image shown is obtained by processing the non-animal pixels.

[0021] Figure 8 This is a block diagram of a Tensor Processing Unit (TPU), which can be used to perform the computations involved in implementing a neural network architecture, including the training phase or the real-time system runtime phase.

[0022] Figure 9 It is a block diagram of a computing device, which can be used as, for example, as Figure 8 The host computer uses the TPU shown. Detailed Implementation

[0023] In the following detailed description, specific details are set forth for illustrative purposes and not for limitation, in order to provide a better understanding of this disclosure. It will be apparent to those skilled in the art that this disclosure may be implemented in other embodiments that depart from these specific details.

[0024] The following describes one embodiment of the invention, which predicts animal body parameters, specifically weight, by image processing an image dataset, wherein the animal species is a pig. The image dataset input to the method comprises two-dimensional images, specifically RGB (color) images, and a depth map. A depth map, sometimes referred to as a point cloud, has each point resembling a pixel in a monochrome two-dimensional image, but carrying a distance value instead of a grayscale value. Therefore, the depth map will be referred to as a depth map image below, and its points as pixels. The image dataset is acquired from a viewpoint located above the animal standing on the ground. The RGB image and the depth map image thus form a pair of images acquired from substantially the same viewpoint, with a close spatial correspondence, which can be compared, for example, on a pixel-by-pixel basis. The two-dimensional image is a three-channel image, while the depth map image is a single-channel image.

[0025] The formal framework for training deep learning-based neural networks is as follows: Task (T): Predict animal weight.

[0026] Experience (E): a set of RGB (R) and corresponding depth (D) images [X -> (R+D)], along with the animal’s target weight (Y).

[0027] Performance (P): The mean absolute error between target weight (Y) and predicted weight (y).

[0028] In this method, it is axiomatically assumed that the projected surface area (A) in an image from a top view of a standing animal is proportional to the animal's volume (V), and the animal's volume is in turn proportional to its weight (Y): A ∝ V ∝ Y To determine an animal's weight, a processing pipeline needs to be established because the method considers not only the animal's local context but also its global context.

[0029] Local context is used to identify animals in an image to avoid noise from neighboring objects and / or to remove other instances of the same animal. Specifically, the method of this invention employs a segmentation operation to isolate and select one animal in the image for further processing. This can be achieved using a neural network-based instance segmentation model, such as MaskRCNN, MaskFormer, or Yolact. RGB images are used as the basis for segmentation to extract masks of individual animals from the RGB images.

[0030] The segmentation process used to isolate animals results in the loss of the animal's global context. Specifically, in the segmented region of an RGB image, the distance between the viewpoint and the animal is unknown. Consequently, the absolute area of ​​the animal is also unknown, and therefore the axiom A ∝ Y cannot be applied solely based on the segmented region of the RGB image containing the animal. For example, the closer an animal is to the viewpoint, the larger its projected area on a handheld imaging device, and thus the larger it appears in the RGB image. Therefore, a small animal located closer to the viewpoint will appear to be similar in size to a larger animal farther away in the segmented region of the RGB image.

[0031] To address this issue and provide global context, a plane estimation algorithm is applied to the depth map image pixels outside the animal mask to determine the background plane. In this embodiment, the background plane is the plane of the ground on which the animal is standing. The algorithm generates a distance value d, representing the distance between the camera viewpoint and the background plane.

[0032] The depth map image is then modified and synthesized: the original depth map values ​​are preserved within the animal subregion (i.e., within the mask) to provide local context; and the depth map values ​​are replaced with unidirectional distance values ​​(e.g., vertical distance values) from the estimated background plane outside the animal subregion (i.e., outside the mask) to provide global context.

[0033] Since most existing open-source neural network models require three-channel data input (e.g., red, green, and blue frames of an RGB image) when processing images, to use such models for processing depth map images, the modified depth map image is copied three times to provide three input arrays for the neural network. In the specific example, the model used is based on the model published by François Chollet in 2017. Xception: Deep Learning with Depthwise Separable Convolutions ( https: / / arxiv.org / abs / 1610.02357 / DOI: https: / / doi.org / 10.48550 / arXiv.1610.02357), and was redesigned by replacing the classification layer with a regression layer.

[0034] Figure 1 This is a perspective view of a handheld imaging device 10 used for acquiring image datasets. The handheld imaging device 10 includes an imager 12. Figure 1A As shown in the bottom view, the imager 12 includes a first camera 16 (e.g., a conventional RGB camera) for acquiring two-dimensional images and a second camera 15 (e.g., a stereo camera with two adjacent lenses) for acquiring depth map images. The two-dimensional camera 16 is operable to acquire three image frames—red, green, and blue—and combine them to form an RGB image. The depth map camera 15 is operable to acquire images representing distance from the camera viewpoint. O The depth or distance of the point cloud. In the context of the following main embodiments of the invention, the handheld imaging device 10 is held by the operator via a handle 14, and the imager 12 is supported at a height in a downward orientation by an elongated frame 13. The frame 13 also includes a stand for a smartphone 18, which can be used to store the image dataset acquired by the imager 12 and can wirelessly transmit it to the cloud or local area network for further processing. The imager 12 is operable to simultaneously acquire pairs of RGB images and depth map images. Cameras 15 and 16 preferably have the same or similar field of view, such that the RGB images and depth map images they acquire respectively cover the same or approximately the same image area A. Cameras 15 and 16 are also preferably from the same viewpoint. O Alternatively, each RGB and depth map image can be acquired from its own nearly overlapping viewpoints so that when jointly processing simultaneously acquired pairs of RGB and depth map images, they can be considered to originate from the same viewpoint. Image region A, as shown in the figure, is located at a distance from the viewpoint. O vertical distance d The plane at that location.

[0035] Figure 2 It is by Figure 1 An exemplary RGB image captured by the handheld imaging device shown is depicting a fully standing pig.

[0036] Figure 3It is collecting Figure 2 The RGB image shown is simultaneously generated by Figure 1 An exemplary depth map image acquired by the handheld imaging device shown.

[0037] Figure 4 This is a flowchart of a body parameter estimation method according to an embodiment of the present invention. The method includes the following steps: data acquisition (step S1), instance segmentation (step S2, for segmenting a complete standing animal from an image), background plane estimation (step S3, for estimating the distance between the image and the background), generation of a modified depth map image (step S4), and body parameter prediction (step S5, using a neural network based on a convolutional neural network (CNN) architecture for classification to make predictions).

[0038] Step S1 is the data acquisition stage, using... Figure 1 The handheld imaging device shown acquires a pair of matching images from a camera viewpoint positioned above an animal standing on the ground. The matching image pair consists of a two-dimensional image and a corresponding depth map image, each image being composed of an array of pixels.

[0039] Step S2 is the segmentation stage, in which the 2D image is segmented using a segmentation algorithm to identify one or more adjacent pixel sub-regions. Each sub-region is defined by its own binary mask and represents a complete and standing animal. In this implementation, a neural network-based segmentation algorithm is used, such as publicly available Visual Transformer (ViT) models like Maskformer and SegViT, to segment different animal instances in the RGB image. Next, a classification model is used to determine whether the animal is standing and complete (i.e., fully visible). Imposing the "standing and complete" condition has been shown to significantly improve prediction accuracy. For example, a pig that is fully visible but lying down will be excluded, as will a pig that is standing but partially occluded by other objects or partially outside the image area. When a pig that meets these conditions is identified, its relevant pixels are segmented according to the segmentation mask generated by the ViT model (…). segMask () is filtered out from RGB images. Figure 5 It shows how to process Figure 2 The RGB image defined by the RGB image shown is obtained using a segmentation mask. segMask A significant advantage of this segmentation algorithm is that if an image region contains multiple animals, the algorithm will segment them all if they all meet the "standing and fully visible" condition. This capability... Figure 4 The diagram illustrates multiple nested boxes in step S2.

[0040] Step S3 is the background plane estimation stage, in which the depth map image is processed by a plane estimation algorithm to estimate the plane of the ground on which the animal is standing. The vertical distance (i.e., height) from the camera viewpoint to the ground provides global context, which is used to provide absolute scale for the depth image data within the segmented animal subregion that only has local context.

[0041] like Figure 4 As illustrated, steps S2 and S3 are performed independently. In this embodiment, this independence is reflected in the fact that step S2 operates on the RGB image, while step S3 operates on the depth map image. However, in other embodiments, segmentation can also be performed on the depth map image (i.e., step S2). Even so, steps S2 and S3 remain independent because the determination of the background plane does not depend on the segmentation mask, and the segmentation process does not require background plane information.

[0042] Step S4 modifies the depth map image based on the results of steps S2 and S3, namely, the segmentation mask and the determined background plane. Specifically, the depth values ​​of depth map image pixels located outside the animal sub-region (i.e., outside the mask) are reset to values ​​corresponding to the distance between the viewpoint of the matched image pair and the estimated background plane, to form a uniform background without clutter. For example, see... Figure 2 and Figure 3 It is known that the depth information of the walls and gates does not contribute to weight estimation. Therefore, excluding them from the input neural network dataset can improve the accuracy of weight estimation. Figure 6 This demonstrates how to utilize the segmentation mask generated in step S2. segMask The pig is segmented from the depth map image. This process requires the RGB image to be aligned with the depth map image, i.e., to have a one-to-one spatial correspondence. After step S4, each point in the depth map image represents the measured distance from the viewpoint to the animal within the mask, and is a null value outside the mask.

[0043] In the implementation of this invention, it is assumed that there is a one-to-one spatial correspondence between the pixels of the RGB image and the points of the depth map image. This is because the lenses and sensors of the handheld imaging device are extremely close to each other and can be considered as image pairs acquired from the same viewpoint. If this assumption is not made, a distortion transformation can be applied to one image to spatially map it to the other. Subsequently, the distance from the estimated background plane (i.e., the ground) to the camera viewpoint is filled into the depth data value background. In other words, all point cloud distance values ​​are set to the vertical distance between the viewpoint and the estimated background plane, such as... Figure 7 As shown. Therefore, Figure 7 An example of a modified depth map image is shown, where the depth map image points inside the mask are the measured distance values ​​from the viewpoint to the animal, while the image points outside the mask are a single value, namely, the vertical distance value from the viewpoint to the background plane determined by a plane estimation algorithm.

[0044] Step S5 inputs the modified depth map image into the neural network. Since the modified depth map image is single-channel, it needs to be copied three times to become three-channel, matching the three-channel input required by standard open-source CNN architectures. Before being input into the CNN, the three-channel depth image is normalized to keep the data within a fixed range, such as [0, 255]. The resulting processed depth map image input into the CNN is called "distance-tripled / three-channel depth data (T3D)". The CNN then outputs a predicted value for the animal's weight. The CNN has been pre-trained by analyzing similar data from animals with known weights. A CNN classification model can be used as the weight estimator, such as the modified François CholletXceptionNet model described above, i.e., the Chollet Xception model whose classification layers are replaced with regression layers.

[0045] In step S2, the segmentation algorithm is used to identify... n In the case of one valid animal instance, the execution method of steps S4 and S5 is adjusted as follows: Step S4 is executed... n Each execution corresponds to one identified animal instance. Each execution of step S4 is labeled as step S4-. n The masked area of ​​non-selected animal instances is considered a non-animal sub-region in the image dataset. In other words, within the mask of selected animal instances, each point in the depth map image represents the measured distance from the viewpoint to the animal; outside the selected mask, it represents null values. Step S5 is also performed. n Each time, it corresponds to one identified animal instance, corresponding to step S4- n The result. Step S5- n The output is a depth map image, which is located under the selected mask. n The inner value represents the measured distance from the viewpoint to the animal, within the selected mask. n External (including non-selected masks) n (The values ​​within) are all single distance values ​​from the viewpoint to the background plane determined by a plane estimation algorithm. Therefore, in step S5, the neural network runs... n Next, to determine the identified features in the image area. n Head animals n Individual weight values. This multi-instance processing capability is in... Figure 4 The steps S2, S4, and S5 are illustrated using multiple nested boxes.

[0046] In summary, the main embodiments describe a computer-implemented image processing method for estimating the weight of an animal species or breed, the method comprising: Provides a pair of matching images acquired by a camera from a viewpoint located above an animal standing on the ground, the pair of matching images including a two-dimensional image and a depth map image; The depth map image is analyzed using a ground plane estimation algorithm to determine the estimated ground plane on which the animal is standing, with reference to the depth map image. By applying segmentation algorithms to process two-dimensional images, adjacent sub-regions representing animals can be identified in the two-dimensional image and the corresponding depth map image; Reset the depth values ​​outside the animal subregion to the values ​​corresponding to the distance between the viewpoint of the matched image pair and the estimated ground plane; The depth map image is normalized to limit the data to a fixed numerical range in order to generate a processed depth map image of "tripled / three-channel depth data (T3D) with distance"; The processed depth map image containing reset depth values ​​outside the animal subregion is input into a neural network, which has been pre-trained by analyzing similar depth map images of animals with known weights; and Receive the predicted animal weight as the output of the neural network.

[0047] Variant Although the example of acquiring images from above is used in the main implementation described above, images can also be acquired from the side, such as from the side or front of the animal, in which case the vertical wall behind the animal can serve as the background plane. It should be understood that when images from different directions (e.g., above, side, or front) are used, separate training is required for each.

[0048] Although the exemplary animal species in the above embodiments is a pig, it should be understood that the method of the present invention is also applicable to other animal species, including but not limited to livestock such as cattle, sheep, horses, ostriches, turkeys, chickens, and ducks. Furthermore, for animal species with multiple breeds, the method can also be applied to one or more specific breeds, such as Berkshire, Chesterfield, Duroc, or Hampshire pigs. Therefore, the animal species can be bipedal or quadrupedal.

[0049] Animals can also be humans. For example, this method can be applied to lateral image acquisition of people walking along a corridor, with one wall of the corridor serving as the background plane. This application scenario could be used, for example, at airport gates to determine the total weight of passengers boarding a commercial flight. The total passenger weight can then be added to the cargo weight to estimate the flight's total payload. This can serve as an additional check that the payload does not exceed certain limits. Simultaneously, the estimated total payload can also be used to cross-validate whether the aircraft's refueling amount matches the planned flight range. Too much refueling leads to excessive fuel consumption due to carrying excess fuel, while too little fuel poses significant safety risks.

[0050] As mentioned incidentally in the main implementation above, an embodiment of the invention can also be implemented that relies solely on depth map images (i.e., without requiring two-dimensional images). In this alternative embodiment, the instance segmentation step S2 is performed directly on the depth map image. A practical advantage of the main implementation compared to the alternative implementation is that most open-source neural network models are already extensively trained on RGB images; therefore, only relatively little additional training is required to adapt these neural network models to the present invention, thus achieving good results with low training costs. However, if segmentation is performed on depth map images, there are currently no known pre-trained open-source neural network models available for direct use; therefore, the neural network model may need to be trained from scratch, which would require a large custom training dataset and significant training effort. Nevertheless, from a technical standpoint, segmentation using depth map images remains feasible.

[0051] In the main implementation described above, the segmentation algorithm is based on a neural network. In alternative implementations, the segmentation algorithm may also be based on traditional image processing techniques, such as one or more of the following methods: Analysis of variance was used to identify seed regions. Adaptive thresholding Morphological operations (e.g., spot analysis). Contour recognition, Contour merging based on proximity heuristics Calculation of invariant image moments, Edge extraction (e.g., Sobel edge detection) Curvature flow filtering, Histogram matching is used to eliminate intensity variations between consecutive slices. Multi-resolution rigid / affine image registration (gradient descent optimizer). Non-rigid deformation / transformation Superpixel clustering.

[0052] In a further embodiment, the segmentation algorithm may preprocess the image data using certain traditional image processing techniques before inputting the image data into the neural network.

[0053] Figure 8The TPU disclosed by Jouppi et al. in 2017 is shown [5], which is Jouppi Figure 1 The TPU100 features a simplified copy. It has a shrinking matrix multiplication unit (MMU) 102 containing 256 × 256 MACs capable of performing 8-bit multiply-accumulate operations on signed or unsigned integers. The weights of the MMU are provided via a weight FIFO buffer 104, which reads the weights from memory 106 (e.g., off-chip 8GB DRAM) through a suitable memory interface 108. A uniform buffer (UB) 110 is provided to store intermediate results. The connected MMU 102 receives input from the weight FIFO interface 104 and UB 110 (via the shrinking data setting unit 112) and outputs the 16-bit MMU processing result to the accumulator unit 114. An activation unit 116 performs nonlinear function processing on the data in the accumulator unit 114. After further processing by a normalization unit 118 and a pooling unit 120, the intermediate results are sent back to UB 110 and then re-provided to the MMU 102 via the data setting unit 112. Pooling unit 120 can perform max pooling (i.e., maximum pooling) or average pooling as needed. Programmable DMA controller 122 is used to transfer data between the TPU host computer and UB 110. TPU instructions are sent from the host computer to controller 122 via host interface 124 and instruction buffer 126.

[0054] It should be understood that computing resources used to run neural networks, whether based on CPUs, GPUs, or TPUs, can be hosted on a local computer network (as described below) or remotely hosted in a data center acting as a network node. Furthermore, an AI processing node can correspond to a single physical processing device deployed on one or more server racks, or it can be distributed across more than two or more physical processing devices.

[0055] Figure 9 A block diagram of an exemplary computing device 500 that can be used in conjunction with various embodiments of this disclosure is shown. For example, the computing device 500 can serve as a computing node for use with a suitable GPU or [other processing unit] on the aforementioned computer network system (e.g., a host computer). Figure 8 The TPU shown performs CNN processing.

[0056] The computing device 500 may be a server, a general-purpose personal computer, or any other processor-enabled device capable of wired or wireless data communication. Those skilled in the art will understand that other computing devices, systems, and / or architectures may also be employed, including those that do not support wired or wireless data communication.

[0057] Computing device 500 preferably includes one or more processors, such as processor 510. Processor 510 may be, for example, a CPU, GPU, TPU, or an array or combination of these processors, such as a combination of CPU and TPU or a combination of CPU and GPU. Additional processors may also be provided, such as auxiliary processors for managing input / output, auxiliary processors for performing floating-point mathematical operations (e.g., TPU), dedicated microprocessors whose architecture is suitable for fast execution of signal processing algorithms (e.g., digital signal processors or image processors), slave processors (e.g., back-end processors) belonging to the main processing system, additional microprocessors, or controllers for dual-processor or multi-processor systems, or coprocessors. These auxiliary processors may be standalone processors or integrated with processor 510. Examples of CPUs that can be used with computing device 500 include Intel Pentium processors, Core i7 processors, and Xeon processors manufactured by Intel Corporation (Santa Clara, California). Examples of GPUs that can be used with computing device 500 include the Tesla K80 GPU manufactured by Nvidia Corporation (Santa Clara, California).

[0058] Processor 510 is connected to communication bus 505. Communication bus 505 may include data channels for transferring information between memory and other peripheral components of computing device 500. Communication bus 505 may also provide a set of signals required for communication with processor 510, including a data bus, an address bus, and a control bus (not shown). Communication bus 505 may employ any standard or non-standard bus architecture, such as conforming to Industry Standard Architecture (ISA), Extended Industry Standard Architecture (EISA), Micro Channel Architecture (MCA), Peripheral Component Interconnect (PCI) local bus, or standards such as IEEE 488 Universal Interface Bus (GPIB) and IEEE 696 / S-100 published by the Institute of Electrical and Electronics Engineers (IEEE).

[0059] The computing device 500 preferably includes a main memory 515, and may also include a secondary memory 520. The main memory 515 provides instruction and data storage for programs (such as one or more of the functions and / or modules described above) running on the processor 510. It should be understood that computer-readable program instructions stored in memory and executed by the processor 510 may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written and / or compiled in one or more programming languages, including but not limited to Smalltalk, C / C++, Java, JavaScript, Perl, Visual Basic, .NET, etc. The main memory 515 is generally a semiconductor-based memory, such as dynamic random access memory (DRAM) and / or stateful random access memory (SRAM). Other semiconductor-based memory types include synchronous dynamic random access memory (SDRAM), Rambus dynamic random access memory (RDRAM), ferroelectric random access memory (FRAM), and read-only memory (ROM).

[0060] Computer-readable program instructions may execute entirely on the user's computer, or partially on the user's computer as a standalone software package; they may also execute partially on the user's computer, partially on the user's computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or, for example, via the Internet through an Internet service provider.

[0061] Secondary storage 520 may optionally include internal storage 525 and / or removable medium 530. Removable medium 530 can be read and / or written in known ways. For example, removable storage medium 530 may include a magnetic tape drive, an optical disc (CD) drive, a digital versatile optical disc (DVD) drive, other optical drives, flash memory drives, etc.

[0062] Removable storage medium 530 is a non-transitory computer-readable medium that stores computer-executable code (i.e., software) and / or data. The computer software or data stored on removable storage medium 530 can be read into computing device 500 for execution by processor 510.

[0063] Secondary memory 520 may include other similar elements for enabling computer programs or other data or instructions to be loaded into computing device 500. For example, such a device may include external storage medium 545 and communication interface 540, which allows software and data to be transferred from external storage medium 545 to computing device 500. Examples of external storage medium 545 may include external hard disk drives, external optical drives, external magneto-optical drives, etc. Other examples of secondary memory 520 may include semiconductor-based memories such as programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), or flash memory (i.e., block-operating memory similar to EEPROM).

[0064] As described above, computing device 500 may include communication interface 540. Communication interface 540 allows software and data to be transferred between computing device 500 and external devices (such as printers), networks, or other information sources. For example, computer software or executable code can be transferred from a network server to computing device 500 via communication interface 540. Examples of communication interface 540 include built-in network adapters, network interface cards (NICs), PCMCIA network cards, card bus network adapters, wireless network adapters, Universal Serial Bus (USB) network adapters, modems, network interface cards (NICs), wireless data cards, communication ports, infrared interfaces, IEEE 1394 FireWire, or any means that enables interface system 550 to connect to a network or other computing device. The communication interface 540 preferably implements industry-released protocol standards, such as Ethernet IEEE 802 standard, Fibre Channel, Digital Subscriber Line (DSL), Asymmetric Digital Subscriber Line (ADSL), Frame Relay, Asynchronous Transfer Mode (ATM), Integrated Services Digital Network (ISDN), Personal Communication Services (PCS), Transmission Control Protocol / Internet Protocol (TCP / IP), Serial Line Internet Protocol / Point-to-Point Protocol (SLIP / PPP), etc., but can also implement customized or non-standard interface protocols.

[0065] Software and data transmitted through communication interface 540 typically appear in the form of electrical communication signals 555. These signals 555 can be transmitted to communication interface 540 via communication channel 550. In one embodiment, communication channel 550 can be a wired network or a wireless network, or any other form of communication link. Communication channel 550 carries signals 555 and can be implemented using various wired or wireless communication methods, such as wires or cables, optical fibers, traditional telephone lines, cellular telephone links, wireless data communication links, radio frequency (RF) links, or infrared links.

[0066] Computer-executable code (i.e., computer programs or software) is stored in main memory 515 and / or secondary memory 520. Computer programs may also be received and stored in main memory 515 and / or secondary memory 520 via communication interface 540. When executed, such computer programs enable computing device 500 to perform the functions of embodiments described in other parts of this disclosure.

[0067] In this disclosure, the term "computer-readable medium" refers to any non-transitory computer-readable storage medium used to provide computer-executable code (e.g., software and computer programs) to computing device 500. Examples of such media include main memory 515, secondary memory 520 (including internal memory 525, removable medium 530, and external storage medium 545), and any peripheral device (including network information server or other network device) communicatively connected to communication interface 540. These non-transitory computer-readable media are carriers for providing executable code, program instructions, and software to computing device 500. In implementations utilizing software, the software may be stored on a computer-readable medium and loaded into computing device 500 via removable medium 530, I / O interface 535, or communication interface 540. In such implementations, the software is loaded into computing device 500 in the form of an electrical communication signal 555. When executed by processor 510, the software preferably causes processor 510 to perform the features and functions described in other parts of this disclosure.

[0068] I / O interface 535 provides an interface between one or more components of computing device 500 and one or more input and / or output devices. Examples of input devices include, but are not limited to, keyboards, touchscreens or other touch-sensitive devices, biometric sensing devices, computer mice, trackballs, pen pointing devices, etc. Examples of output devices include, but are not limited to, cathode ray tube (CRT), plasma displays, light-emitting diode (LED) displays, liquid crystal displays (LCDs), printers, vacuum fluorescent displays (VFDs), surface conduction electron emission displays (SEDs), field emission displays (FEDs), etc.

[0069] The computing device 500 may also include optional wireless communication components for facilitating wireless communication over voice and / or data networks. These wireless communication components include an antenna system 570, a radio system 565, and a baseband system 560. In the computing device 500, radio frequency (RF) signals are transmitted and received over the air via the antenna system 570 under the control of the radio system 565.

[0070] Antenna system 570 may include one or more antennas and one or more multiplexers (not shown) that perform switching functions to provide transmit and receive signal paths to antenna system 570. In the receive path, the received RF signal may be coupled from the multiplexer to a low-noise amplifier (not shown), which amplifies the received RF signal and transmits the amplified signal to radio system 565.

[0071] Radio system 565 may include one or more radios configured to communicate on different frequencies. In one embodiment, radio system 565 may integrate a demodulator (not shown) and a modulator (not shown) in the same integrated circuit (IC). The demodulator and modulator may also be separate components. In the input path, the demodulator removes the RF carrier signal, leaving a baseband received audio signal, which is transmitted from radio system 565 to baseband system 560.

[0072] If the received signal contains audio information, the baseband system 560 decodes the signal and converts it into an analog signal. The signal is then amplified and output to a speaker. The baseband system 560 also receives analog audio signals from a microphone. These analog audio signals are converted into digital signals and encoded by the baseband system 560. The baseband system 560 further encodes the digital signals for transmission and generates a baseband transmit audio signal, which is routed to the modulator section of the radio system 565. The modulator mixes the baseband transmit audio signal with an RF carrier signal to generate an RF transmit signal, which is routed to the antenna system 570 and can pass through a power amplifier (not shown). The power amplifier amplifies the RF transmit signal and routes it to the antenna system 570, whereby the signal is relayed to the antenna port for transmission.

[0073] The baseband system 560 is also communicatively connected to the processor 510, which may be a central processing unit (CPU). The processor 510 has access to data storage areas 515 and 520. The processor 510 is preferably configured to execute instructions (i.e., computer programs or software) stored in the main memory 515 or secondary memory 520. The computer program may also be received by the baseband processor 560 and stored in the main memory 510 or secondary memory 520, or executed directly after being received. These computer programs, when running, enable the computing device 500 to perform the functions of the disclosed embodiments. For example, the data storage area 515 or 520 may include multiple software modules.

[0074] The computing device also includes a display 575 directly connected to the communication bus 505, which may be provided as an alternative to or supplement to any of the displays connected to the I / O interface 535 described above.

[0075] In various embodiments, the present invention can also be implemented primarily in hardware, for example using components such as application-specific integrated circuits (ASICs), programmable logic arrays (PLAs), or field-programmable gate arrays (FPGAs). It will also be apparent to those skilled in the art that a hardware state machine capable of performing the functions described herein can be implemented. Various embodiments can also be implemented using a combination of hardware and software.

[0076] Furthermore, those skilled in the art will understand that the various exemplary logic blocks, modules, circuits, and method steps described in relation to the foregoing figures and embodiments of this disclosure can generally be implemented using electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the foregoing exemplary components, functional blocks, modules, circuits, and steps have been summarized according to their functions. Whether these functions are implemented in hardware or software depends on the specific application and overall system design constraints. Those skilled in the art can adopt different functional implementation methods for different applications, but such selection of implementation schemes should not be considered as departing from the scope of the invention. Additionally, the division of functions into different modules, functional blocks, circuits, or steps is merely for illustrative purposes; specific functions or steps can be moved from one module, functional block, or circuit to another without departing from the spirit and scope of the invention.

[0077] Furthermore, the various exemplary logic blocks, modules, functions, and methods related to embodiments of this disclosure can be implemented or executed by a general-purpose processor, digital signal processor (DSP), ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic, stand-alone hardware component, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any type of processor, controller, microcontroller, or state machine. The processor may also be implemented by a combination of multiple computing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors and a DSP core, or any other similar configuration.

[0078] Furthermore, the steps of the methods or algorithms described in the embodiments of this disclosure can be implemented directly in hardware, or through software modules executed by a processor, or a combination of both. The software modules can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium, including network storage media. An exemplary storage medium can be coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be integrated with the processor. The processor and storage medium can also jointly constitute an ASIC.

[0079] The computer-readable storage medium described in this disclosure should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating via waveguides or other transmission media (e.g., optical pulses transmitted via optical fibers), or electrical signals transmitted via wires.

[0080] Any software component described in this disclosure can exist in various forms. For example, a component can be a standalone software package or a software package integrated as a tool into a larger software product. The component can be provided as a standalone product or plug-in package, downloaded via a network (e.g., a website) for installation in existing software applications; it can also be provided as a client-server software application, a network-enabled software application, and / or a mobile application.

[0081] The embodiments described herein are illustrated in the form of flowcharts and / or block diagrams, illustrating methods, apparatus (systems), and computer program products according to embodiments of the present invention. Those skilled in the art will understand that each functional block in the flowcharts and / or block diagrams, and combinations thereof, can be implemented by computer-readable program instructions.

[0082] Computer-readable program instructions may be provided to the processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to manufacture a machine that, when executed by the processor of the computer or other programmable data processing apparatus, performs the instructions to implement the functions / actions defined in the functional blocks of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium to instruct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Therefore, a computer-readable storage medium storing the instructions can be considered an article of manufacture containing instructions to implement the functions / actions defined in the functional blocks of the flowchart and / or block diagram.

[0083] Computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus or other device to cause the computer, other programmable apparatus or other device to perform a series of operational steps, thereby creating a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus or other device implement the functions / actions defined by the functional blocks in the flowchart and / or block diagram.

[0084] The flowcharts and block diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each functional block in the flowchart or block diagram may represent an instruction module, segment, or portion containing one or more sets of executable instructions for implementing a specific logical function. In some alternative implementations, the execution order of the functions shown in the blocks may differ from the order in the diagram. For example, two consecutive blocks may be executed almost simultaneously, or in reverse order, depending on the functional requirements. It can also be seen that each functional block and its combination in the flowchart and / or block diagram may also be implemented by a dedicated hardware system that performs the function or action, or a combination of dedicated hardware and computer instructions.

[0085] The devices and methods for implementing this invention can be hosted and delivered through a cloud computing environment. Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services) that can be rapidly configured and released with minimal management effort or service provider interaction. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0086] In summary, the above detailed description outlines a computer-implemented image processing method and apparatus for estimating body parameters of a specific species or breed of animal. An image acquisition device acquires an image dataset containing depth map images to obtain images of the animal. A plane estimation algorithm analyzes the depth map images to determine the background plane behind the animal. A segmentation algorithm identifies the animal mask. The depth values ​​outside the animal mask are reset to the distance between the viewpoint of the image acquisition device and the estimated background plane. The depth map images are normalized to restrict the data to a fixed numerical range, thereby generating a processed depth map image of triplet / three-channel depth data (T3D) including distance. The processed depth map image and the reset depth values ​​of its background plane are input into a pre-trained neural network, which outputs estimates of the animal's body parameters.

[0087] Those skilled in the art should understand that various improvements and modifications can be made to the above embodiments without departing from the scope of this disclosure.

[0088] References [1]Zhang, J.; Zhuang, Y.; Ji, H.; Teng, G. Pig Weight and Body SizeEstimation Using a Multiple Output Regression Convolutional Neural Network: AFast and Fully Automatic Method. Sensors 2021, 21, 3218. https: / / doi.org / 10.3390 / s21093218 [2]Suwannakhun, S.; Daungmala, P. Estimating Pig Weight with DigitalImage Processing using Deep Learning. The 14th Session of the Department of Signal and Image Technology and Internet Information in 2018 Proceedings of the International Conference on Systematics and Interdisciplinary Information Sciences (SITIS) Beijing, China, November 26-29, 2018 [3]US2022221325A1 (Viking Genetics FMBA) [4]CN113240574A (Shenzhen Xiwei Smart Technology Co., Ltd.) [5]Jouppi, Young, Patil et al., In-Datacenter Performance Analysis of aTensor Processing Unit, The 44th International Society of Computer Architecture (ISCA) Toronto, Canada, June 24–28, 2017 (submitted April 16, 2017), arXiv:1704.04760 [cs.AR] The entire contents of the above references are incorporated herein by reference.

Claims

1. A computer-implemented image processing method for determining body parameter values ​​of a specific species or breed of animal, the method comprising: Provides an image dataset of an image region, the image dataset comprising depth map images acquired by a camera from a viewpoint, the image region comprising animals imaged relative to a background plane. The depth map image is analyzed using a plane estimation algorithm to identify the background plane and determine the distance between the viewpoint and the background plane accordingly. The segmentation algorithm is applied to the image dataset to identify animals in the image regions and define corresponding animal masks. Modify the depth map image by resetting the depth values ​​located outside the animal mask to the distance values ​​from the viewpoint to the background plane. The depth map image is normalized to limit the data to a fixed numerical range, thereby generating a processed depth map image of triplet depth data (T3D) including distance. The processed depth map image and its reset depth value are input into a neural network to determine the animal's body parameters. This neural network has been pre-trained by analyzing a dataset of similar images of the specific species or breed of animal. It receives animal body parameter estimates as "neural network output".

2. The method according to claim 1, wherein, The image dataset of the image region further includes two-dimensional images acquired from the viewpoint, wherein the segmentation algorithm is applied to the two-dimensional images to identify animals within the image region and define corresponding animal masks.

3. The method according to claim 1, wherein, The segmentation algorithm is applied to the depth map image to identify animals within the image region and define corresponding animal masks.

4. The method according to any one of claims 1 to 3, wherein, The viewpoint is located above the animal, and the background plane is the ground on which the animal is situated.

5. The method according to any one of claims 1 to 3, wherein, The viewpoint is located on one side of the animal, and the background plane is the wall adjacent to the animal.

6. The method according to any one of the preceding claims, wherein, The animal's physical parameters are one of the following two: weight and distance dimension.

7. The method according to any one of the preceding claims, wherein, The segmentation algorithm identifies animals based on the condition that "the animal mask matches the complete animal of the species or breed that the method is configured to process".

8. The method according to claim 7, wherein, The segmentation algorithm further identifies animals based on the condition that "the animal mask matches an animal in a specific pose".

9. The method according to claim 8, wherein, The posture is standing, and the animal species is a bipedal or quadrupedal animal.

10. The method according to any one of claims 1 to 9, wherein, The segmentation algorithm is based on image processing using another neural network that has been pre-trained by analyzing a dataset of similar images of the specific species or breed of animal.

11. The method according to any one of claims 1 to 9, wherein, The segmentation algorithm is based on traditional image processing techniques.

12. The method according to any one of the preceding claims, wherein, The image region contains multiple animal instances imaged relative to the background plane, and when the segmentation algorithm identifies multiple animal instances within the image region, one of the animal instances is selected for further processing, wherein the other animal instances are treated as background and their depth values ​​are reset to the distance from the viewpoint to the background plane.

13. The method according to claim 12, wherein, The further processing is performed individually for each identified animal instance to determine the body parameter values ​​of each animal instance within the image region.

14. A computing device having machine-readable instructions loaded thereon for performing the method according to any one of the preceding claims.

Images