Method and system for estimating weight of meat poultry based on 2D image

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to a meat poultry weight estimation method and system based on a 2D image, and the method comprises the steps: responding to a sensor, sensing that meat poultry enters a shooting place, and collecting a three-view image set of the meat poultry through shooting equipment; an image classification network is constructed, the three-view image set is screened, and three-view images meeting shooting conditions are obtained; constructing an image segmentation network, and carrying out semantic segmentation processing on the three-view image meeting the shooting condition to obtain a segmented three-view image; and based on the segmented three-view image, inputting a convolutional neural network to estimate the weight of the meat poultry. The system comprises a meat poultry image acquisition device and an operation module. The consumption-level camera is used for collecting 2D images, efficient and stable meat poultry weight estimation is achieved, breeding work is assisted, and great significance is achieved for the meat poultry breeding industry.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a method and system for estimating the weight of meat poultry based on 2D images. Background Art

[0002] Growth traits are one of the important breeding directions for meat poultry, and the accuracy of weight trait measurement directly affects the accuracy of offspring selection. In the prior art, based on the traditional manual use of instruments such as tape measures, measuring rods, and electronic scales to measure duck body size and weight, the entire measurement process is time-consuming and labor-intensive, and the measured duck body size index data is easily affected by the subjective influence of the measurement personnel. At the same time, the sensors in precision instruments for weighing duck weights such as intelligent measuring equipment weighing modules are greatly affected by animal behavior in an exposed state, resulting in inaccurate measurement results. Manual measurement will also cause serious stress reactions to ducks, affecting the normal behavior of ducks, and not meeting animal welfare requirements. In the prior art, with the rapid development of machine vision technology, non-contact weight measurement methods for pigs, cattle, sheep, etc. based on machine vision have changed the traditional contact weight measurement method, and can better solve the problems in the traditional process of measuring animal weight data, such as: relying on measurement personnel to measure animal weight data continuously and contactlessly. Related machine vision technologies mainly include:

[0003] Image classification is an important task in computer vision, which refers to the classification of input images into different predefined categories. This process involves analyzing and extracting the content and features in the image and comparing them with a pre-trained model to identify the category to which the image belongs. Image classification has a wide range of applications in many fields, such as object detection, face recognition, medical image analysis, etc.

[0004] Semantic segmentation is a part of neural network application in computer vision, which involves taking some simple filtered and processed raw data as input and highlighting the parts of interest. Image semantic segmentation takes an image as input and highlights the areas of interest.

[0005] Convolutional neural network is a deep neural network with convolutional structure. The convolutional structure can reduce the amount of memory occupied by the deep network. Its three key operations are local receptive field, weight sharing and pooling layer, which effectively reduce the number of network parameters and alleviate the overfitting problem of the model. It is often used in the field of computer vision.

[0006] In the prior art, there are great difficulties in estimating the weight of meat poultry. Considering that meat poultry is relatively active and has a relatively light weight, the scheme of measuring with a weight scale is very unstable. In the prior art, the use of computer vision technology to estimate the weight of animals requires relying on high-definition 2D or 3D cameras to capture animal images, and extracting key parameters such as animal body shape through image processing technology. According to the extracted features, machine learning or deep learning regression methods are used to predict weight. It is mainly used in species such as pigs and cattle, and is not suitable for meat poultry that are naturally hyperactive. For example, meat poultry is constantly moving its body while its wings are moving, causing the wings to cover the body of the meat poultry, significantly increasing the difficulty of capturing meat poultry images. Therefore, there is an urgent need for a meat poultry weight estimation method and system based on 2D images, which uses consumer-grade cameras to collect 2D images, and realizes efficient and stable meat poultry weight estimation to assist breeding work, which is of great significance to the meat poultry breeding industry. Summary of the invention

[0007] The object of the present invention is to provide a method for estimating the weight of meat poultry based on 2D images, characterized in that it comprises the following steps:

[0008] Step S1: In response to the sensor sensing that the poultry enters the shooting location, the shooting device collects a set of three-view images of the poultry;

[0009] Step S2: construct an image classification network, screen the three-view image set, and obtain three-view images that meet the conditions for being photographed;

[0010] Step S3: construct an image segmentation network, perform semantic segmentation processing on the three-view images that meet the conditions for being photographed, and obtain segmented three-view images;

[0011] Step S4: Based on the segmented three-view images, the convolutional neural network is input to estimate the weight of the poultry.

[0012] The three-view images in step S1 include: a front view, a top view and a side view; the shooting equipment includes: a front view shooting equipment, a top view shooting equipment and a side view shooting equipment.

[0013] The sensor in step S1 is a radio frequency identification sensor, and the poultry wears a radio frequency identification tag to sense the poultry entering and leaving the shooting location and sense the poultry identity information.

[0014] The image classification network in step S2 is a DeformAttn-ShuffleNetV2 network, and the DeformAttn-ShuffleNetV2 network builds a deformable attention module on the basis of ShuffleNetV2 to introduce the SKnet attention mechanism and deformable convolution; the input of the DeformAttn-ShuffleNetV2 network is the side view in the three-view image; the output of the DeformAttn-ShuffleNetV2 network is the classification result of the meat and poultry status; the classification result includes: zhanli and qita; when the classification result is zhanli, it is determined that the three-view image corresponding to the input side view meets the shooting condition.

[0015] The network structure of the deformable attention module is as follows: the input feature channel is divided into a first branch and a second branch; the first branch includes: 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, and linear rectification function in sequence; the second branch includes: 1×1 convolution, batch normalization, linear rectification function, 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function, 1×1 convolution, batch normalization, linear rectification function, DA convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function; the first branch performs channel shuffling after merging with the second branch; the DA convolution includes: depthwise separable convolution with input and output channels both c, batch normalization, linear rectification function, deformable convolution with input and output channels both c, batch normalization, linear rectification function, the first fully connected layer, the second fully connected layer and the Softmax function.

[0016] The conditions for shooting in step S2 are: the poultry enters the cage, puts its head through the partition and starts eating; the poultry is in a standing position; and the wings of the poultry are in a tightened state.

[0017] The image segmentation network in step S3 is a Segment Anything segmentation model.

[0018] The convolutional neural network in step S4 receives a composite image composed of segmented three-view images and outputs an estimated weight of meat poultry; the composite image sequentially passes through a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a first fully connected layer, a second fully connected layer and an output layer to obtain an estimated weight of meat poultry; the kernel size of the first convolutional layer is 11×11, the convolution step is 1, and there is no padding; the kernel size of the second convolutional layer is 12×12, the convolution step is 1, and there is no padding; the first downsampling layer includes: a first ReLu activation function and an initial pooling layer, and the initial pooling layer has a 6×6 pooling window; the second downsampling layer includes: a second ReLu activation function and a second pooling layer, and the second pooling layer has a 5×5 pooling window; the first fully connected layer contains 80 nodes; the second fully connected layer contains 16 nodes; the output layer has no activation function.

[0019] The parameter adjustment process of the convolutional neural network is as follows:

[0020] First, adjust the learning rate from large to small, from 0.1 to 0.0001;

[0021] Secondly, adjust the batch size according to the size of the data set and the computing resources;

[0022] Adjust the optimizer again, the optimizer is SGD or Adam;

[0023] Finally, adjust the regularization parameters, including: L1 regularization, L2 regularization, and Dropout.

[0024] Another object of the present invention is to disclose a meat poultry weight estimation system according to the meat poultry weight estimation method based on 2D images of the present invention, characterized in that it comprises: a meat poultry image acquisition device and a computing module;

[0025] The meat and poultry image acquisition device comprises: a shell, a front view camera, a top view camera and a side view camera; a partition is provided in the shell, and the partition divides the inside of the shell into a feed storage area and a meat and poultry standing area; a feeding port feeding area is provided in the feed storage area; the meat and poultry standing area can accommodate a single meat and poultry standing; a feeding hole is provided on the partition, and the feeding hole restricts the meat and poultry to only pass the head through the feeding hole to complete feeding; the front view camera is located on the inner side of the shell side wall near the feeding port along the axial direction of the feeding hole, and is used to collect meat and poultry images facing the head of the meat and poultry; the top view camera is located on the inner side of the shell top wall, and is used to collect meat and poultry images from the top; the side view camera is located on the inner side of the shell side wall along the axial direction perpendicular to the feeding hole, and is used to collect meat and poultry images facing the trunk of the meat and poultry;

[0026] The operation module includes: an image classification network module, an image segmentation network module and a convolutional neural network module; the image classification network module is used to screen the three-view image set, input the side view in the three-view image into the DeformAttn-ShuffleNetV2 network, and output the classification result of the meat poultry state to obtain the screened three-view image; the image segmentation network module is used to perform semantic segmentation processing on the three-view images that meet the shooting conditions to obtain the segmented three-view images; the convolutional neural network module is used to use the segmented three-view images to estimate the weight of meat poultry.

[0027] The beneficial effects of the present invention are:

[0028] The present invention discloses a method for estimating the weight of meat poultry based on 2D images. Based on the existing intelligent weighing equipment or intelligent poultry breeding equipment, three consumer-grade infrared cameras are installed in front, on the side and on the top of the breeding cage, respectively, to collect three-view images of a single meat poultry in a natural eating state from three perspectives: front view, side view and top view. On the basis of reusing the existing breeding equipment, it is possible to collect complete body images of meat poultry at different ages throughout the growth cycle, and by integrating the image information from three perspectives, it is possible to better adapt to the active characteristics of meat poultry. Taking advantage of the eating nature of meat poultry, the meat poultry is likely to actively eat after entering the shooting area. After sensing that the meat poultry has entered the collection area, multiple groups of three-view images are collected within 1-3 seconds, and the camera is automatically set to enter the standby state, which greatly reduces the power consumption of the camera.

[0029] In the prior art, when weight estimation is performed based on computer vision technology, it is necessary to perform fine recognition of the main body features, such as chest width and body length, which requires high computing power and power consumption. This defect is particularly obvious for meat and poultry animals that are naturally active, whose wings have a great influence on the recognition of body contours, and whose number is quite large, such as ducks and chickens. The present invention has found through a large number of observations and experiments that meat and poultry animals are likely to eat immediately after entering the collection area, and in conjunction with the position of the partition and the feeding bowl, meat and poultry are likely to maintain a standing posture with wings tightened for a short time during the eating process, and extend their necks to pass their heads through the openings of the partitions to eat. In this state, when the meat and poultry are sensed to enter the collection area, multiple groups of three-view images are collected within 1-3 seconds, and there is no need to perform fine recognition of specific body parts of the meat and poultry, especially for the side view, while avoiding the difference in the degree of neck extension of different meat and poultry individuals when eating; the present invention automatically identifies the three-view images that meet the conditions for shooting by constructing an image classification network and defining the conditions for shooting three-view images, effectively reducing the computing power requirements in the recognition process.

[0030] The image classification network is a DeformAttn-ShuffleNetV2 network. A deformable attention module (DeformAttn Blocks) is constructed on the basis of ShuffleNetV2 to introduce the SKnet attention mechanism and deformable convolution. By constructing a deformable attention module, dynamic adjustment can be better performed to adapt to various environmental changes in the duck cage, such as light brightness, random debris changes, etc. Therefore, the DeformAttn-ShuffleNetV2 network disclosed in the present invention has achieved good performance in accuracy, parameter quantity and FLOPs.

[0031] Based on the accurate identification of the three-view images that meet the conditions for shooting by the DeformAttn-ShuffleNetV2 network, the Segment Anything segmentation model is used as the image segmentation network to obtain images with high segmentation accuracy in complex environments, and no data annotation is required. Only the pixel values ​​of the area to be segmented and several label points need to be pre-set. Since the feeding area of ​​meat poultry is relatively fixed, several fixed label points are selected for the three-view images to obtain segmented three-view images with high segmentation accuracy.

[0032] Based on the segmented three-view images, a convolutional neural network estimates the weight of meat poultry. A relatively accurate estimation result is obtained by adjusting the parameters of the convolutional neural network. Experimental verification shows that the predicted value obtained by applying the meat poultry weight estimation method based on 2D images disclosed in the present invention has a high degree of fit with the actual value, and a better prediction effect can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a flow chart of a method for estimating the weight of meat poultry based on 2D images according to the present invention;

[0034] Figure 2 : are three-view segmentation images in an embodiment of the present invention, wherein (a) is a front view-RGB image; (b) is a front view-segmentation image; (c) is a side view-RGB image; (d) is a side view-segmentation image; (e) is a top view-RGB image; (f) is a top view-segmentation image;

[0035] Figure 3 Schematic diagram of the camera installation position in an embodiment of the present invention, wherein (a) is a front view; (b) is a side view; (c) is a top view;

[0036] Figure 4 A schematic diagram of a group of RGB images of an object to be estimated in a natural eating state according to an embodiment of the present invention;

[0037] Figure 5A schematic diagram of the structure of the DeformAttn-ShuffleNetV2 network provided by the present invention;

[0038] Figure 6 for Figure 5 A magnified view of the network structure of the deformable attention module;

[0039] Figure 7 for Figure 6 A magnified image of the network structure of the DA convolution part;

[0040] Figure 8 A schematic diagram of the structure of the CNN network provided by the present invention. DETAILED DESCRIPTION

[0041] The present invention provides a method and system for estimating the weight of meat poultry based on 2D images, and the present invention is further described in detail below in conjunction with the accompanying drawings.

[0042] like Figure 1 The embodiment of the present invention shown in the figure discloses a method for estimating the weight of meat poultry based on 2D images, comprising the following steps:

[0043] Step S1: In response to the sensor sensing that the poultry enters the shooting location, the shooting device collects a set of three-view images of the poultry;

[0044] Step S2: construct an image classification network, screen the three-view image set, and obtain three-view images that meet the conditions for being photographed;

[0045] Step S3: construct an image segmentation network, perform semantic segmentation processing on the three-view images that meet the conditions for being photographed, and obtain segmented three-view images;

[0046] Step S4: Based on the segmented three-view images, the convolutional neural network is input to estimate the weight of the poultry.

[0047] In this embodiment, a 2D image-based poultry weight estimation method disclosed in the present invention is applicable to the weight estimation scenario of the object to be estimated, such as weight estimation in poultry breeding. In this embodiment, the weight estimation of ducks is taken as an example. The specific implementation methods of each step are introduced in combination with specific embodiments.

[0048] Step S1: In response to the sensor sensing that the poultry enters the shooting location, the shooting device collects a set of three-view images of the poultry;

[0049] The three-view images in step S1 include: a front view, a top view and a side view; the shooting equipment includes: a front view shooting equipment, a top view shooting equipment and a side view shooting equipment.

[0050] In this embodiment, the shooting equipment is three consumer-grade infrared cameras, which are respectively installed in front, on the side and above the weight estimation device to collect three-view images of a single meat poultry in a natural eating state from three perspectives: front view, top view and side view. Figure 4 The shooting time is from the time when the poultry stands up on the weight estimation device to the time when the poultry leaves the device after eating, and multiple groups of three-view images in the natural eating state are obtained to form a set of three-view images of the poultry.

[0051] Those skilled in the art should know that in response to the sensor sensing that the meat poultry meets the conditions for being able to be photographed, the camera is triggered to start photographing. The conditions for being able to be photographed are:

[0052] The poultry enters the cage and puts its head through the partition to start eating; the poultry is in a standing position; and the wings of the poultry are in a tightened state;

[0053] The sensor includes but is not limited to an infrared sensor, an optical sensor, a bioelectric sensor, a weight scale, etc., which is not specifically limited in this embodiment.

[0054] According to a large number of experimental observations by the inventor, in most cases, driven by the nature of meat poultry, when meat poultry enters the cage, it will immediately rush to the food to eat, prompting the meat poultry to stretch its neck so that its head passes through the partition in the cage, maintain a standing posture and keep its wings in a tightened state; therefore, in most cases, when the sensor senses that the meat poultry is in the photographable condition within 1 to 3 seconds of entering the cage, the meat poultry is likely to be in the photographable condition, and at this time, the shooting timing is triggered to call the camera to shoot 3 to 5 groups of three-view images. In this embodiment, even if the meat poultry does not enter the photographable condition within 1 to 3 seconds, unqualified pictures can be manually excluded in the model training stage. In the actual production environment, unqualified pictures are excluded by using the image classification network for judgment in the subsequent steps. Therefore, in this embodiment, the shooting timing of multiple groups of three-view images is within 1-3 seconds when the sensor senses that the meat poultry enters the cage. Therefore, the method of sensing the shooting timing used in this embodiment has the advantages of simple structure, quick judgment and low cost.

[0055] The sensor in step S1 is a radio frequency identification sensor, and the poultry wears a radio frequency identification tag to sense the poultry entering and leaving the shooting location and sense the poultry identity information.

[0056] In a preferred embodiment, the poultry is equipped with an electronic tag, and the sensor is an electronic tag sensor. The electronic tag sensor cooperates with the electronic tag to not only sense the poultry entering a designated area in the cage, but also obtain the identity information of the poultry, so as to facilitate weight tracking throughout the growth cycle of the poultry. The electronic tag includes but is not limited to an RFID tag, an NFC tag, a Bluetooth tag, a ZigBee tag, etc., which are not specifically limited in this embodiment.

[0057] In this embodiment, multiple groups of three-view images of meat and poultry in a natural eating state are captured, and then the multiple groups of images in a natural eating state obtained are selected to obtain multiple groups of natural eating state images that meet the shooting conditions, and the camera is controlled to enter a standby state.

[0058] In an optional embodiment, in response to the sensor sensing that the poultry has left the photographing location, the photographing device enters a standby state.

[0059] In this embodiment, the camera is installed at a position such as Figure 3 As shown in the figure, (a) is a front view; (b) is a side view; (c) is a top view. Consumer-grade 2D cameras were installed in front, on the side, and above the weight estimation device to capture images of meat ducks eating in their natural state. A total of 275 ducks were collected. Figure 1 A total of about 15,000 images are used as the data set for subsequent training models.

[0060] In an optional embodiment, multiple groups of three-view images of meat poultry in a natural eating state are captured, and then the multiple groups of images in a natural eating state that are obtained are selected to obtain multiple groups of images in a natural eating state that meet the conditions for being photographed. When the sensor senses that the meat poultry has left the breeding cage, a weighing scale is used to measure the actual weight of the poultry, and the actual weight value and the multiple groups of images in a natural eating state that meet the conditions for being photographed are simultaneously added to the data set for model training and testing.

[0061] Those skilled in the art should know that, considering that the weight of meat poultry changes greatly during the entire growth process, the installation position of the camera should ensure that the camera can capture a complete image of the body of the older meat poultry. Those skilled in the art will make corresponding adjustments based on the performance parameters of the selected camera, the breeding cage ruler and other factors, which are not specifically limited in this embodiment.

[0062] Step S2: construct an image classification network, screen the three-view image set, and obtain three-view images that meet the conditions for being photographed;

[0063] The image classification network in step S2 is a DeformAttn-ShuffleNetV2 network, and the DeformAttn-ShuffleNetV2 network builds a deformable attention module on the basis of ShuffleNetV2 to introduce the SKnet attention mechanism and deformable convolution; the input of the DeformAttn-ShuffleNetV2 network is the side view in the three-view image; the output of the DeformAttn-ShuffleNetV2 network is the classification result of the meat and poultry state; the classification result includes: zhanli and qita; when the classification result is zhanli, it is determined that the three-view image corresponding to the input side view meets the shooting condition;

[0064] like Figure 6 As shown, the network structure of the deformable attention module is as follows: the input feature channel is divided into a first branch and a second branch; the first branch includes: 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function; the second branch includes: 1×1 convolution, batch normalization, linear rectification function, 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function, 1×1 convolution, batch normalization, linear rectification function, DA convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function; the first branch performs channel shuffling after the second branch is merged. Figure 7 As shown, the DA convolution includes: depthwise separable convolution with input and output channels both being c, batch normalization, linear rectification function, deformable convolution with input and output channels both being c, batch normalization, linear rectification function, the first fully connected layer, the second fully connected layer, and the Softmax function.

[0065] In this embodiment, replacing the second depthwise separable convolution in the DA convolution with a deformable convolution improves the adaptability to geometric deformation. The deformable convolution can dynamically adjust the position of the convolution kernel according to the content of the input feature map, so as to better capture the changes in the geometric shape and posture of the object. In contrast, the traditional depthwise separable convolution has a fixed sampling position and cannot handle spatial deformation well; the feature expression ability is enhanced. The deformable convolution can adaptively adjust the sampling point position by introducing additional offset learning to better capture the detailed information and complex background of the target object; the detection and segmentation performance of the model is improved. The deformable convolution can enhance the model's ability to detect edges and shapes, thereby improving the overall detection and segmentation performance; at the same time, the advantages of multi-scale features and spatial deformation can be combined, so that the model can not only better utilize multi-scale information, but also better adapt to the deformation characteristics of the target object.

[0066] The conditions for shooting in step S2 are: the poultry enters the cage, puts its head through the partition and starts eating; the poultry is in a standing position; and the wings of the poultry are in a tightened state.

[0067] Considering that meat poultry have different feeding postures in their natural state, such as fully extended legs, half-squatted legs, and reclining meat poultry, etc. By observing the changes in posture of meat poultry during feeding, it is found that there is a big difference in the neck extension length of meat poultry when eating compared with the natural standing posture; when eating naturally, the naturally spread wings of meat poultry cover about half of the body outline; in addition, it is found that meat poultry will most likely change from a standing posture to a prone posture after eating for a few seconds. Therefore, the posture of meat poultry has an important influence on the accuracy of the subsequent weight estimation model. Image classification technology is used to identify and extract the posture images most suitable for weight estimation, and an image classification network model is introduced to automatically screen images to select meat poultry images that meet the said conditions for shooting.

[0068] In this embodiment, the obtained multiple groups of three-view images in the natural eating state are selected and classified to obtain a set of three-view images that meet the photographable conditions. The selected multiple groups of natural eating state images are classified. From the side view perspective, the natural eating state images with legs completely upright, wings completely tightened, and head lowered for eating are classified into the "zhanli" category; other eating postures are classified into the "qita" category, and other eating postures include but are not limited to lying down for eating, half squatting for eating, etc.

[0069] In this embodiment, the DeformAttn-ShuffleNetV2 network performs image classification based on the input of the side view in the three-view image, and divides the side view into two categories: standing posture and other postures, where other postures include half-squatting and lying down to eat. Based on the ShuffleNetV2 network, the SKnet attention mechanism is introduced, named SK-ShuffleNetV2, and further on the basis of the SK-ShuffleNetV2 network, deformable convolution is introduced, named DeformAttn-ShuffleNetV2, as the classification network actually used in this embodiment, as shown in Table 1, the DeformAttn-ShuffleNetV2 network has achieved good performance in accuracy, parameter quantity and FLOPs.

[0070] Table 1 Image classification results

[0071] Model Top-1(%) Params(M) FLOPs(G) ShuffleNetV2 97.20 1.26 1.45 SK-ShuffleNetV2 97.66 1.6 1.46 DeformAttn-ShuffleNetV2 98.13 6.15 6.87

[0072] ShuffleNetV2 is a novel neural network architecture in the prior art, which aims to effectively reduce parameters and computational complexity by implementing channel-by-channel group convolution and channel shuffle operations, thereby improving computational efficiency. The model introduces a channel-by-channel group convolution method, which divides the input channels into multiple groups and applies convolution operations independently to each group. Subsequently, the results of these operations are merged in the channel dimension, which can effectively improve the network capacity and information fluidity, thereby optimizing performance. By effectively reducing the number of parameters and computational load, ShuffleNetV2 achieves high classification accuracy while minimizing resource requirements. To further improve performance, ShuffleNetV2 uses repeated modules. Each module consists of three operations: channel-by-channel group convolution, channel mixing, and channel-by-channel group convolution again. By stacking multiple modules, the overall depth and width of the network are increased, thereby improving the expressiveness of the model.

[0073] In this embodiment, in order to improve the accuracy of detecting meat and poultry images in complex environments and select the image data with the best posture, two technologies are introduced on the basis of ShuffleNetV2: SKnet attention mechanism (Selective Kernel Networks) and deformable convolution.

[0074] SKnet's attention mechanism enables each neuron to adaptively adjust the size of its receptive field (i.e., convolution kernel) according to the different scales of the input information, effectively capturing multi-scale features in complex image spaces without consuming too much computing resources like traditional CNNs. In addition, SKnet's attention mechanism can also integrate deep features, thereby improving the interpretability and understanding of the captured features.

[0075] Deformable convolution can perform non-uniform and adaptive sampling and perception of input features. Deformable convolution can adaptively adjust the size and shape of the receptive field according to contextual information by learning offsets, thereby more effectively capturing the spatial structure and details of the target. Adding learnable offsets to deformable convolution enhances the adaptability of traditional convolution operations, thereby better adjusting the spatial relationship and deformation between features and improving the expressiveness of the model.

[0076] The structure of the DeformAttn-ShuffleNetV2 network is as follows Figure 5 shown.

[0077] The input of the DeformAttn-ShuffleNetV2 network is the side view in the three-view image of the meat poultry, and the output is the classification result of the meat poultry state; on the basis of the standard ShuffleNetV2 network structure, the deformable attention module is used to form Stage2, Stage3 and Stage4, realizing the fusion of SKnet attention mechanism (Selective KernelNetworks) and deformable convolution (Deformable Convolution); the network structure of the deformable attention module part is as follows Figure 6 shown.

[0078] In this embodiment, for the 13,320 RGB three-view images of 275 poultry obtained, each poultry corresponds to multiple groups of natural eating state images, and the multiple groups of natural eating state images corresponding to each poultry are screened, and then the screened multiple groups of natural eating state images are classified, and 423 side view RGB images of the "zhanli" category and 652 side view RGB images of the "qita" category are classified respectively, and 1,075 sample natural eating state images are obtained.

[0079] In this embodiment, after using 275 side-view images of meat ducks, the images were divided into two categories: standing posture and other postures, and the image classification model DeformAttn-ShuffleNetV2 was trained.

[0080] The model classification results and related data are as follows, where ShuffleNetV2 is the basic model, SK-ShuffleNetV2 and DeformAttn-ShuffleNetV2 are improved models, and the DeformAttn-ShuffleNetV2 model is finally selected for actual use:

[0081] Step S3: construct an image segmentation network, perform semantic segmentation processing on the three-view images that meet the conditions for being photographed, and obtain segmented three-view images;

[0082] The image segmentation network in step S3 is a Segment Anything segmentation model.

[0083] In this embodiment, the feeding area of ​​meat and poultry is relatively fixed, that is, they feed at the feeding port in the weight estimation device. The SegmentAnything segmentation model can obtain images with higher segmentation accuracy in complex environments, and does not require data annotation. It only needs to pre-set the pixel values ​​of the area to be segmented and several label points. Since the feeding area of ​​meat and poultry is relatively fixed, several fixed label points are selected for the three-view images to obtain images with higher segmentation accuracy.

[0084] In this embodiment, based on the Segment Anything segmentation model, there is no need to perform image annotation, which greatly reduces the workload while ensuring the segmentation accuracy. By setting different segmentation rectangular areas, labels of target objects and backgrounds for the three-view images respectively, the final segmented image is obtained. The mIoU indicators obtained for the front view, top view and side view in the dataset after image classification are shown in Table 3, which are: 0.9697 for the front view, 0.9871 for the top view, and 0.9769 for the side view. According to the multiple groups of sample images of natural eating states obtained, after obtaining the image classification results, they are input into the SegmentAnything segmentation model in turn, and rectangular areas and labels are set for the three view images respectively, to obtain the recognition results output by the initial image segmentation model as shown in Table 3. Figure 2 shown.

[0085] Table 3 Segment Anything segmentation model mIoU results statistics

[0086]

[0087] Step S4: Based on the segmented three-view images, the convolutional neural network is input to estimate the weight of the poultry.

[0088] The convolutional neural network in step S4 receives a composite image composed of segmented three-view images and outputs an estimated weight of meat poultry; the composite image sequentially passes through a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a first fully connected layer, a second fully connected layer and an output layer to obtain an estimated weight of meat poultry; the kernel size of the first convolutional layer is 11×11, the convolution step is 1, and there is no padding; the kernel size of the second convolutional layer is 12×12, the convolution step is 1, and there is no padding; the first downsampling layer includes: a first ReLu activation function and an initial pooling layer, and the initial pooling layer has a 6×6 pooling window; the second downsampling layer includes: a second ReLu activation function and a second pooling layer, and the second pooling layer has a 5×5 pooling window; the first fully connected layer contains 80 nodes; the second fully connected layer contains 16 nodes; the output layer has no activation function.

[0089] In this embodiment, a convolutional neural network is used to estimate the weight of a meat duck. The segmented three-view images obtained by the image segmentation model are unified as the input of the convolutional neural network model through the channel stitching technology, and the weight value corresponding to the individual is used as the true value to train a meat duck weight estimation model. For specific data, refer to Table 2.

[0090] The convolutional neural network receives a composite image composed of segmented three-view images and outputs an estimated weight of a meat poultry. Figure 8As shown, the synthetic image sequentially passes through the first convolution layer, the first downsampling layer, the second convolution layer, the second downsampling layer, the first fully connected layer, the second fully connected layer and the output layer to obtain the estimated weight of meat poultry; the kernel size of the first convolution layer is 11×11, the convolution step is 1, and there is no padding; the kernel size of the second convolution layer is 12×12, the convolution step is 1, and there is no padding; the first downsampling layer includes: a first ReLu activation function and an initial pooling layer, and the initial pooling layer has a 6×6 pooling window; the second downsampling layer includes: a second ReLu activation function and a second pooling layer, and the second pooling layer has a 5×5 pooling window; the first fully connected layer contains 80 nodes; the second fully connected layer contains 16 nodes; the output layer has no activation function.

[0091] In this embodiment, the process of obtaining the data set required for training the convolutional neural network is as follows:

[0092] Three cameras are installed on the weight estimation equipment in the meat poultry feeding area. By capturing three-view images of meat poultry eating in a natural state, a comprehensive data set containing three perspectives of the eating process is obtained. In order to accurately estimate the weight, the three-view images will be passed through the Segment Anything segmentation model to obtain segmented images that meet the unified input requirements of the weight estimation model. After taking the image of the meat duck, the staff will weigh the duck immediately and record the weight value as the true value. It is worth noting that the input of the model is a synthetic image spliced ​​from the segmented three-view images obtained from the above three perspectives.

[0093] In this embodiment, a total of 275 ducks were collected, and three Figure 1 A total of about 15,000 images are used as the data set for subsequent training models, with the ratio of training set to test set being 8:2.

[0094] The segmented three-view images in the training set are input into the convolutional neural network to obtain an estimated weight of the poultry, the estimated weight of the poultry is compared with the actual weight value, and the parameters in the convolutional neural network are continuously changed until the training stop condition is met to obtain a trained convolutional neural network.

[0095] In this embodiment, the specific process of changing the parameters in the convolutional neural network is as follows:

[0096] First, adjust the learning rate from large to small, from 0.1 to 0.0001;

[0097] In an optional embodiment, the learning rate is adjusted using cosine annealing and an adaptive learning rate;

[0098] Secondly, adjust the batch size according to the size of the data set and the computing resources;

[0099] Adjust the optimizer again, the optimizer is SGD or Adam;

[0100] Finally, adjust the regularization parameters, including: L1 regularization, L2 regularization, and Dropout;

[0101] In this embodiment, when the training data is small, L1 and L2 regularization helps prevent overfitting. Dropout reduces the complexity of the model by randomly discarding a part of neurons.

[0102] In this embodiment, after obtaining the image segmentation results, the corresponding three-view images are unified as the input of the convolutional neural network to train a weight estimation model for live meat ducks. The weight estimation results and related data are shown in Table 2:

[0103] Table 2 Weight estimation results

[0104] Metric MAE <![CDATA[R 2 ]]> MRE MAD RMSE CNN 40g 0.9631 1.64％ 208.6g 48.8g

[0105] The relevant values ​​in Table 2 are defined as follows:

[0106] MAE (Mean Absolute Error)

[0107] Definition: MAE is the average of the absolute differences between the predicted values ​​and the actual values. The formula is:

[0108]

[0109] Among them, y i is the actual value, is the predicted value and n is the number of samples.

[0110] Function: MAE is used to measure the average absolute difference between the model's predicted value and the actual value.

[0111] R 2

[0112] Definition: R 2 Measures the model's ability to explain data variance. The formula is:

[0113]

[0114] in, is the average of the actual values.

[0115] Role: R 2 Used in regression tasks to indicate the fit between the predicted value and the actual value. 2 The closer the value is to 1, the better the model explains the variance of the data.

[0116] MRE (Mean Relative Error)

[0117] Definition: MRE is the average relative error between the actual value and the predicted value, usually expressed as a percentage. The formula is:

[0118]

[0119] Function: MRE is used to evaluate the relative difference between the predicted value and the actual value, and is suitable for situations where the actual value is large and has obvious changes.

[0120] MAD (Median Absolute Deviation)

[0121] Definition: MAD is the median of the absolute differences between the actual and predicted values, and the formula is:

[0122]

[0123] Role: MAD is used as a robust alternative to MAE in regression tasks because the median is more robust to outliers than the mean.

[0124] RMSE (Root Mean Squared Error)

[0125] Definition: RMSE is the square root of the mean of the squared errors between the predicted values ​​and the actual values. The formula is:

[0126]

[0127] Role: RMSE is used in regression tasks to emphasize larger errors because the square term amplifies larger errors.

[0128] The convolutional neural network disclosed in this embodiment is different from the prior art that relies on 2D or 3D cameras to capture animal images. It uses image processing technology to extract key parameters such as animal body shape, and uses machine learning or deep learning regression methods to predict the weight of meat poultry based on the extracted features. The present invention discloses a method for estimating the weight of meat poultry based on 2D images, which does not need to rely on high-definition 2D or 3D cameras to capture animal images, and uses consumer-grade cameras to collect 2D images to achieve efficient and stable meat poultry weight estimation.

[0129] In a specific embodiment, the method for estimating the weight of meat poultry based on 2D images disclosed in the present invention is applied to automatically estimate the weight of meat ducks based on the trained convolutional neural network. The specific process is as follows:

[0130] The 2D three-view images of several meat ducks are collected to form a set of three-view images of meat poultry; the three-view image set of meat poultry is screened by using an image classification network to obtain screened three-view images that meet the preset meat duck posture requirements; the three-view images that meet the shooting conditions are semantically segmented by using an image segmentation network to obtain segmented three-view images; the segmented three-view images are uniformly used as the input of a convolutional neural network to estimate the weight of meat ducks.

[0131] In this embodiment, consumer-grade 2D cameras are installed in the front, side and top of the weight estimation device. The image classification model DeformAttn-ShuffleNetV2 is used to automatically collect images of the duck in a natural eating state. The image segmentation model Segment Anything Model segments the classified three-view images respectively. Then, the set of images with the best segmentation effect of the duck is used as the input of the weight estimation model to automatically estimate the weight of the duck.

[0132] Another embodiment of the present invention discloses a meat poultry weight estimation system based on 2D images, comprising: a meat poultry image acquisition device and a computing module;

[0133] The meat and poultry image acquisition device comprises: a shell, a front view camera, a top view camera and a side view camera; a partition is provided in the shell, and the partition divides the inside of the shell into a feed storage area and a meat and poultry standing area; a feeding port feeding area is provided in the feed storage area; the meat and poultry standing area can accommodate a single meat and poultry standing; a feeding hole is provided on the partition, and the feeding hole restricts the meat and poultry to only pass the head through the feeding hole to complete feeding; the front view camera is located on the inner side of the shell side wall near the feeding port along the axial direction of the feeding hole, and is used to collect meat and poultry images facing the head of the meat and poultry; the top view camera is located on the inner side of the shell top wall, and is used to collect meat and poultry images from the top; the side view camera is located on the inner side of the shell side wall along the axial direction perpendicular to the feeding hole, and is used to collect meat and poultry images facing the trunk of the meat and poultry;

[0134] The input of the DeformAttn-ShuffleNetV2 network is the side view in the three-view image; the output of the DeformAttn-ShuffleNetV2 network is the classification result of the meat poultry state; the classification result includes: zhanli and qita; when the classification result is zhanli, it is determined that the three-view image corresponding to the input side view meets the shooting condition.

[0135] The operation module includes: an image classification network module, an image segmentation network module and a convolutional neural network module; the image classification network module is used to screen the three-view image set, input the side view in the three-view image into the DeformAttn-ShuffleNetV2 network, and output the classification result of the meat poultry state to obtain the screened three-view image; the image segmentation network module is used to perform semantic segmentation processing on the three-view images that meet the shooting conditions to obtain the segmented three-view images; the convolutional neural network module is used to use the segmented three-view images to estimate the weight of meat poultry.

[0136] Considering that the postures of the top view and the front view are relatively stable under the shooting standard, in this embodiment, the image classification network module screens the side view, and the top view and the front view take corresponding results.

[0137] For example, if the object to be estimated is poultry, three consumer-grade 2D infrared cameras are installed in front, on the side, and above the weight estimation device to capture the three-view images of the poultry standing on the weight estimation device in a natural eating state, and the three-view images of the poultry are obtained by parsing the RTSP protocol stream using a Python program, and at least one first image of the object to be estimated can be obtained. The 2D infrared camera follows the RTSP protocol.

[0138] In this embodiment, a spatial rectangular coordinate system is established to accurately obtain the natural eating state image of meat poultry, such as Figure 2 shown. Figure 3 : is a diagram of the installation positions of three cameras in the weight estimation device provided by the present invention, wherein (a) is a front view; (b) is a side view; (c) is a top view; Figure 3 The lower left corner of the front view shown in (a) is the origin of the coordinate system. Figure 3 The right side of (a) is the positive direction of the x-axis, with a length of 46 cm; Figure 3 (a) The direction inside the breeding cage is the positive direction of the y axis, and the length is 58 cm; Figure 3 In (a), the top direction is the positive direction of the z-axis, and the length is 53 cm. Use a ruler to measure and obtain the coordinates of the three cameras in centimeters. The coordinates of the front view camera are x: 19 cm, y: 0 cm, z: 43.5 cm. The coordinates of the side view camera are x: 0 cm, y: 41 cm, z: 27.5 cm. The coordinates of the top view camera are x: 20 cm, y: 45.5 cm, z: 53 cm.

[0139] In this embodiment, those skilled in the art should know that, considering that the weight of meat poultry varies greatly during the entire growth process, the installation position of the camera should ensure that the camera can capture a complete image of the body of the older meat poultry. Those skilled in the art will make corresponding adjustments based on the performance parameters of the selected camera, the size of the breeding cage and other factors, which are not specifically limited in this embodiment. Compared with the prior art that mostly only captures 2D images from one or two perspectives for weight estimation, the present invention uses front view, top view and side view to obtain a complete image of the body of the older meat poultry. Figure 3 The images were collected from three directions, taking into account the fact that meat poultry is more active than pigs, cattle, sheep and other animals, and their body shape changes greatly during the breeding cycle. The standing posture and wing state have a great impact on weight estimation. The model was trained comprehensively using the image data collected from three perspectives to obtain more accurate estimation results. The weight estimation results and related data are shown in Table 2. For example, R 2 It is 0.9631, which proves that the fit between the predicted value and the actual value obtained by applying the meat poultry weight estimation method based on 2D images disclosed in the present invention is high, and a better prediction effect can be obtained.

Claims

1. A method for estimating the weight of meat poultry based on 2D images, characterized in that: The steps include: Step S1: In response to the sensor sensing that the poultry enters the shooting location, the shooting device collects a set of three-view images of the poultry; Step S2: construct an image classification network, screen the three-view image set, and obtain three-view images that meet the conditions for being photographed; Step S3: construct an image segmentation network, perform semantic segmentation processing on the three-view images that meet the conditions for being photographed, and obtain segmented three-view images; Step S4: Based on the segmented three-view images, the convolutional neural network is input to estimate the weight of the poultry.

2. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The three-view images in step S1 include: a front view, a top view and a side view; the shooting equipment includes: a front view shooting equipment, a top view shooting equipment and a side view shooting equipment.

3. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The sensor in step S1 is a radio frequency identification sensor, and the poultry wears a radio frequency identification tag to sense the poultry entering and leaving the shooting location and sense the poultry identity information.

4. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The image classification network in step S2 is a DeformAttn-ShuffleNetV2 network, and the DeformAttn-ShuffleNetV2 network builds a deformable attention module on the basis of ShuffleNetV2 to introduce the SKnet attention mechanism and deformable convolution; the input of the DeformAttn-ShuffleNetV2 network is the side view in the three-view image; the output of the DeformAttn-ShuffleNetV2 network is the classification result of the meat and poultry status; the classification result includes: zhanli and qita; when the classification result is zhanli, it is determined that the three-view image corresponding to the input side view meets the shooting condition.

5. The method for estimating the weight of meat poultry based on 2D images according to claim 4, characterized in that: The network structure of the deformable attention module is as follows: the input feature channel is divided into a first branch and a second branch; The first branch includes: 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, and linear rectification function in sequence; the second branch includes: 1×1 convolution, batch normalization, linear rectification function, 3×3 convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function, 1×1 convolution, batch normalization, linear rectification function, DA convolution, batch normalization, 1×1 convolution, batch normalization, linear rectification function; the first branch performs channel shuffling after merging with the second branch; the DA convolution includes: depthwise separable convolution with input and output channels both c, batch normalization, linear rectification function, deformable convolution with input and output channels both c, batch normalization, linear rectification function, the first fully connected layer, the second fully connected layer and the Softmax function.

6. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The conditions for shooting in step S2 are: the poultry enters the cage, puts its head through the partition and starts eating; the poultry is in a standing position; and the wings of the poultry are in a tightened state.

7. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The image segmentation network in step S3 is a Segment Anything segmentation model.

8. The method for estimating the weight of meat poultry based on 2D images according to claim 1, characterized in that: The convolutional neural network in step S4 receives a composite image composed of segmented three-view images and outputs an estimated weight of meat poultry; the composite image sequentially passes through a first convolutional layer, a first downsampling layer, a second convolutional layer, a second downsampling layer, a first fully connected layer, a second fully connected layer and an output layer to obtain an estimated weight of meat poultry; the kernel size of the first convolutional layer is 11×11, the convolution step is 1, and there is no padding; the kernel size of the second convolutional layer is 12×12, the convolution step is 1, and there is no padding; the first downsampling layer includes: a first ReLu activation function and an initial pooling layer, and the initial pooling layer has a 6×6 pooling window; the second downsampling layer includes: a second ReLu activation function and a second pooling layer, and the second pooling layer has a 5×5 pooling window; the first fully connected layer contains 80 nodes; the second fully connected layer contains 16 nodes; the output layer has no activation function.

9. The method for estimating the weight of meat poultry based on 2D images according to claim 8, characterized in that: The parameter adjustment process of the convolutional neural network is as follows: First, adjust the learning rate from large to small, from 0.1 to 0.0001; Secondly, adjust the batch size according to the size of the data set and the computing resources; Adjust the optimizer again, the optimizer is SGD or Adam; Finally, adjust the regularization parameters, including: L1 regularization, L2 regularization, and Dropout.

10. A poultry weight estimation system according to the poultry weight estimation method based on 2D images according to any one of claims 1 to 9, characterized in that: include: Meat and poultry image acquisition device and computing module; The meat and poultry image acquisition device comprises: a shell, a front view camera, a top view camera and a side view camera; a partition is provided in the shell, and the partition divides the inside of the shell into a feed storage area and a meat and poultry standing area; a feeding port feeding area is provided in the feed storage area; the meat and poultry standing area can accommodate a single meat and poultry standing; a feeding hole is provided on the partition, and the feeding hole restricts the meat and poultry to only pass the head through the feeding hole to complete feeding; the front view camera is located on the inner side of the shell side wall near the feeding port along the axial direction of the feeding hole, and is used to collect meat and poultry images facing the head of the meat and poultry; the top view camera is located on the inner side of the shell top wall, and is used to collect meat and poultry images from the top; the side view camera is located on the inner side of the shell side wall along the axial direction perpendicular to the feeding hole, and is used to collect meat and poultry images facing the trunk of the meat and poultry; The operation module includes: an image classification network module, an image segmentation network module and a convolutional neural network module; the image classification network module is used to screen the three-view image set, input the side view in the three-view image into the DeformAttn-ShuffleNetV2 network, and output the classification result of the meat poultry state to obtain the screened three-view image; the image segmentation network module is used to perform semantic segmentation processing on the three-view images that meet the shooting conditions to obtain the segmented three-view images; the convolutional neural network module is used to use the segmented three-view images to estimate the weight of meat poultry.

Images