A posture keyframe detection method for poultry weight estimation

Abstract

The present invention discloses a posture key frame detection method for poultry weight estimation. First, a chicken semantic segmentation model is obtained through a chicken color image and a chicken depth image; the chicken color image is segmented to obtain a segmentation mask image, and the chicken color image and the segmentation mask image are image-and-operated to obtain a chicken color segmentation map; the contour circumscribed rectangle of the segmentation mask image is calculated, and the direction of the chicken color segmentation map is adjusted by comparing the two sides of the circumscribed rectangle, and a transformed color map is obtained through an affine transformation operation; the direction of the chicken head in the transformed color map is adjusted to the same direction; the transformed color map is input into a chicken key posture recognition model, and the detection result of the chicken posture category to be tested is obtained, and it is judged whether the current frame is a posture key frame. The present invention improves the estimation accuracy of poultry weight and overcomes the problems of time-consuming and labor-intensive manual selection and poor real-time performance.

Description

Technical Field

[0001] The invention relates to a detection method for poultry weight estimation in the field of poultry detection, and in particular to a posture key frame detection method for poultry weight estimation. Background Art

[0002] Weight is one of the important growth indicators of poultry farming and an important basis for breeding. The weight of poultry is directly related to economic benefits. Weight information can be used to determine the best time to market poultry and guide daily feeding. Therefore, monitoring poultry weight is particularly important for breeders to timely understand the health status of poultry and adjust feeding plans in a timely manner.

[0003] At present, there are three methods for measuring poultry weight: manual measurement, weighing device method and machine vision monitoring method. Traditional manual sampling and weighing is time-consuming and laborious, and it is easy to cause stress reactions in chickens. People cannot weigh and observe their weight changes regularly. The weighing device method is less affected by the breeding environment and will not cause stress reactions in chickens, but this type of sensor has problems such as power consumption, installation, maintenance and recycling. In comparison, the machine vision monitoring method has the advantages of real-time objectivity, non-intrusiveness and strong maintainability. More and more scholars are using machine vision technology to conduct research on animal weight estimation.

[0004] Depth images can reflect the distance from the object in the scene to the camera. By obtaining the three-dimensional features of poultry such as back height, volume, and surface area, and adding them to the weight prediction model, the accuracy of poultry weight estimation can be effectively improved. Mortensen et al. used the Kinect depth camera to collect depth images of broilers aged 14-33 days, extracted 12 features including age, body length, body width, projection area, and volume to predict weight, with an average relative error of 7.8%. Based on Mortensen, Wang Lin et al. selected 9 features and measured an average relative error of 3.3%. However, the above methods generally have the following problems: Since poultry is a non-cooperative object with an uncertain body shape, changes in posture affect the accuracy of weight estimation. Currently, it mainly relies on manual selection of suitable images, which is highly subjective and time-consuming and labor-intensive, making it difficult to realize the automation of poultry weight estimation. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a method for determining the ideal posture of poultry based on SE-ResNet neural network to improve the accuracy of poultry weight estimation. The technical solution adopted by the present invention to solve its technical problem is as follows:

[0006] The posture key frame detection method includes the following steps:

[0007] 1) Acquire multiple color images and multiple depth images of a single chicken at different time periods, read the color image and depth image of the chicken in the current frame in sequence, obtain the chicken situation in the color image of the current frame and the chicken situation in the depth image of the current frame respectively, and then perform model training on the color image of the chicken in the current frame, the depth image of the chicken in the current frame, the chicken situation in the color image of the chicken in the current frame, and the chicken situation in the depth image of the chicken in the current frame to obtain a chicken semantic segmentation model;

[0008] 2) Use the chicken semantic segmentation model to analyze the chicken color image I containing the chicken to be tested L Perform segmentation to obtain the segmentation mask image I S , segment the mask image I S and a color image of chickens containing the chickens to be tested I L Perform image and operation to obtain the chicken color segmentation map I L ′;

[0009] 3) The segmentation mask image I in step 2) S Contains the chicken outline C, determine the segmentation mask image I S The minimum circumscribed rectangle of the chicken outline C is then combined with the minimum circumscribed rectangle of the chicken outline C to obtain the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformation T lateral direction;

[0010] 4) The transformed color image I T After cropping, the image is transformed to Lab color space to obtain the chicken color image I. Lab , according to the chicken color map I Lab After transformation, the color image I T Rotate and adjust all transformed color images I T The heads of the tested chickens are facing the same direction;

[0011] 5) The transformed color image I processed in step 4) T Input into the pre-trained chicken key posture recognition model to obtain the detection result of the chicken posture category to be tested. If the detection result is a standing posture, the current frame is the posture key frame; otherwise, read the next frame of chicken color image and repeat steps 2)-5).

[0012] The step 1) is specifically as follows:

[0013] Firstly, multiple color images and depth images of a single chicken are obtained at different time periods, and the color image and depth image of the chicken in the current frame are read in turn to obtain the chicken conditions in the color image of the current frame and the chicken conditions in the depth image of the current frame, respectively. Then, the RedNet neural network is used to train the model of multiple color images of a single chicken at different time periods, multiple depth images of chickens, the chicken conditions in the color image of the current frame, and the chicken conditions in the depth image of the current frame to obtain the chicken semantic segmentation results, and the trained RedNet neural network is used as the chicken semantic segmentation model.

[0014] The step 3) is specifically as follows:

[0015] First determine the segmentation mask image I S The minimum circumscribed rectangle of the chicken outline C is obtained, and then a new coordinate system is established with the vertex with the largest vertical coordinate in the pixel coordinate system as the coordinate origin. The straight line of the rectangular side in the minimum circumscribed rectangle passing through the coordinate origin and away from the vertical axis of the pixel coordinate system is set as the x-axis of the new coordinate system, and the straight line of the other rectangular side in the minimum circumscribed rectangle passing through the coordinate origin is set as the y-axis of the new coordinate system. The coordinates of the center point of the minimum circumscribed rectangle in the new coordinate system are P(x0, y0), and the length of the rectangular side on the x-axis is l1. The length of the side of the rectangle on the y-axis is l2, and the angle between the x-axis of the new coordinate system and the horizontal axis of the pixel coordinate system is θ. Then, the following judgment is made based on the size of the side length l1 of the rectangle on the x-axis and the side length l2 of the rectangle on the y-axis: if the side length l1 of the rectangle on the x-axis is less than the side length l2 of the rectangle on the y-axis, then the angle θ is subtracted by 270° to obtain a new angle θ; otherwise, the angle θ is kept unchanged; the rotation matrix M is determined based on the center coordinates P (x0, y0) and the angle θ, and the rotation matrix M is used to transform the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformation T Horizontal direction.

[0016] The step 4) is specifically as follows:

[0017] The transformed color image I T Color image after transformation I T The minimum circumscribed rectangle of the inner chicken outline C is clipped to obtain the clipped chicken color image I crop , the cut chicken color image I crop Transform to Lab color space and get the chicken color image I Lab , in the chicken color map I Lab Extract the a component to get the a component image I a ; The a component image I aSegment along its longitudinal central axis to obtain two equally divided parts, one of which is the left sub-image I a1 , the other part is the right sub-image I a2 , respectively calculate the left sub-image I a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I is obtained according to the calculation a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I a1 The average pixel value of the right sub-image I a2 The average pixel value of the transformed color image I T unchanged; otherwise, the transformed color image I T Perform a 180° rotation, that is, adjust all the transformed color images I T The heads of the tested chickens were facing in the same direction.

[0018] The chicken depth image is obtained directly through a depth camera or indirectly through a binocular color camera.

[0019] The rotation matrix M is mainly determined by the following formula:

[0020]

[0021] Among them, x0 and y0 are the horizontal and vertical coordinates of the center point of the minimum circumscribed rectangle respectively.

[0022] The affine transformation is mainly performed through the following formula:

[0023]

[0024] Among them, (x, y) is the chicken color segmentation map I L ′ is the pixel coordinates, (u, v) is the transformed color image I T The pixel coordinates of .

[0025] The chicken key posture recognition model is obtained by using the SE-ResNet neural network to perform model training on a posture atlas, and the posture atlas is obtained by repeating steps 2)-4) for a plurality of chicken color images of different chickens with known posture categories to be tested.

[0026] The chicken posture categories are divided into five categories: standing posture, head lowering posture, feather combing posture, wings spreading posture and squatting posture.

[0027] The present invention processes a single chicken color image through an SE-ResNet neural network to obtain a chicken semantic segmentation model, and obtains a chicken key posture recognition model by processing multiple chicken color images of known chicken posture categories to be tested, thereby providing a method for determining posture key frames for accurate estimation of poultry weight.

[0028] The beneficial effects of the present invention are as follows: a posture key frame detection method for poultry weight estimation is proposed, which can effectively improve the accuracy of poultry weight estimation and overcome the time-consuming, labor-intensive and poor real-time problems caused by relying on manual selection of suitable images for weight estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is the overall flow chart of the present invention;

[0030] Figure 2 is a color image of a chicken according to an embodiment of the present invention. L Schematic diagram;

[0031] Figure 3 is a schematic diagram of a chicken depth image according to an embodiment of the present invention;

[0032] Figure 4 is the segmentation mask image I of the embodiment of the present invention S Schematic diagram;

[0033] Figure 5 is the transformed color image I of the embodiment of the present invention T Schematic diagram;

[0034] Figure 6 It is a chicken color picture 1 of the embodiment of the present invention Lab The a component image I a Schematic diagram;

[0035] Figure 7 is the transformed color image I after processing in the embodiment of the present invention T Schematic diagram;

[0036] Figure 8 Schematic diagram of chicken posture categories according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] like Figure 1 As shown, the posture key frame detection method includes the following steps:

[0039] 1) First, 600 color images and 600 depth images of a single chicken are obtained at different time periods. The color images of the chickens in this embodiment are all obtained by a binocular color camera with an image resolution of 960pix*540pix, and the depth images of the chickens are all obtained by stereo matching the color images of the chickens. Then, the color image of the chicken and the depth image of the chicken in the current frame are read in sequence to obtain the chicken conditions in the color image of the current frame and the chicken conditions in the depth image of the current frame, respectively. Then, the color image of the chicken in the current frame, the depth image of the chicken in the current frame, the chicken conditions in the color image of the chicken in the current frame, and the chicken conditions in the depth image of the chicken in the current frame are trained to obtain a chicken semantic segmentation model. Among them, the depth image of the chicken in the current frame is as follows: Figure 3 shown.

[0040] 2) Use the chicken semantic segmentation model to analyze the chicken color image I containing the chicken to be tested L To split, such as Figure 2 As shown, a segmentation mask image I with foreground and background is obtained. S ,like Figure 4 As shown. The segmentation mask image I S and a color image of chickens containing the chickens to be tested I L Perform image and operation to obtain the chicken color segmentation map I L ′.

[0041] 3) Segmentation mask image I in step 2) S The minAreaRect function in the OpenCV library is used to determine the segmentation mask image I. S The minimum circumscribed rectangle of the chicken outline C is then combined with the minimum circumscribed rectangle of the chicken outline C to obtain the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T ,like Figure 5 As shown. The chicken head of the tested chicken is oriented in the direction of the color image I after the transformation T Horizontal direction.

[0042] 4) The transformed color image I T After cropping, the image is transformed to Lab color space to obtain the chicken color image I. Lab , according to the chicken color map I Lab After transformation, the color image I T Rotate and adjust all transformed color images I T The heads of the tested chickens were facing in the same direction.

[0043] 5) The transformed color image I processed in step 4) TInput into the pre-trained chicken key posture recognition model to obtain the detection result of the chicken posture category to be tested. If the detection result is a standing posture, the current frame is the posture key frame; otherwise, read the next frame of chicken color image and repeat steps 2)-5).

[0044] Step 1) is specifically as follows:

[0045] First, multiple color images and depth images of a single chicken are obtained at different time periods, and the color image and depth image of the chicken in the current frame are read in turn to obtain the chicken situation in the color image of the current frame and the chicken situation in the depth image of the current frame, respectively. Then, the RedNet neural network is used to train the model for multiple color images of a single chicken at different time periods, multiple depth images of chickens, the chicken situation in the color image of the current frame, and the chicken situation in the depth image of the current frame to obtain the chicken semantic segmentation results, and the trained RedNet neural network is used as the chicken semantic segmentation model. Specifically, during the training process, the number of iterations is set to 200 times, the minimum batch size is 4 pictures, BCEDiceLoss is used as the training loss function, and random gradient descent is used as the optimizer. The learning rate is initialized to 0.002, and the learning rate is adjusted using the learning rate decay training strategy.

[0046] Step 3) is specifically:

[0047] First, the minAreaRect function in the OpenCV library is used to determine the segmentation mask image I S The minimum circumscribed rectangle of the chicken outline C in the middle. Then, a new coordinate system is established with the vertex with the largest vertical coordinate in the minimum circumscribed rectangle in the pixel coordinate system as the coordinate origin, and the straight line where the rectangular side of the minimum circumscribed rectangle passes through the coordinate origin and is far away from the vertical axis of the pixel coordinate system is set as the x-axis of the new coordinate system, and the straight line where the other rectangular side of the minimum circumscribed rectangle passes through the coordinate origin is set as the y-axis of the new coordinate system. The rectangular side of the minimum circumscribed rectangle that is the x-axis and the rectangular side that is the y-axis in the minimum circumscribed rectangle are two adjacent sides, and the coordinates of the center point of the minimum circumscribed rectangle in the new coordinate system are obtained as P(x0, y0), and the coordinates on the x-axis are P(x0, y0). The length of the rectangle side is l1, the length of the rectangle side on the y-axis is l2, the angle between the x-axis of the new coordinate system and the horizontal axis of the pixel coordinate system is θ, and then the following judgment is made according to the size of the length of the rectangle side l1 on the x-axis and the length of the rectangle side l2 on the y-axis: if the length of the rectangle side l1 on the x-axis is less than the length of the rectangle side l2 on the y-axis, then the angle θ is subtracted by 270° to obtain a new angle θ; otherwise, the angle θ is kept unchanged; the rotation matrix M is determined according to the center coordinates P (x0, y0) and the angle θ, and the rotation matrix M is used to perform the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformationT Horizontal direction.

[0048] Determine the rotation matrix M according to the center coordinates P(x0, y0) and the angle θ, and use the rotation matrix M to segment the chicken color image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformation T Horizontal direction.

[0049] Step 4) is specifically as follows:

[0050] The transformed color image I T Color image after transformation I T The minimum circumscribed rectangle of the inner chicken outline C is clipped to obtain the clipped chicken color image I crop , the cut chicken color image I crop Transform to Lab color space and get the chicken color image I Lab , in the chicken color map I Lab Extract the a component to get the a component image I a ,like Figure 6 shown.

[0051] The a component image I a Segment along its longitudinal central axis to obtain two equally divided parts, one of which is the left sub-image I a1 , the other part is the right sub-image I a2 , respectively calculate the left sub-image I a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I is obtained according to the calculation a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I a1 The average pixel value of the right sub-image I a2 The average pixel value of the transformed color image I T constant.

[0052] Otherwise, the transformed color image I T Perform a 180° rotation, that is, adjust all the transformed color images I T The heads of the tested chickens are facing the same direction. Figure 7 shown.

[0053] In this embodiment, the left sub-image I is calculated a1 , right sub-image I a2 The average grayscale values ​​of the non-zero pixel areas are 131.0 and 133.6 respectively. T Perform a 180° rotation, such as Figure 7 As shown. The rotation matrix M is mainly determined by the following formula:

[0054]

[0055] Among them, x0 and y0 are the horizontal and vertical coordinates of the center point of the minimum circumscribed rectangle respectively.

[0056] Affine transformation is mainly performed through the following formula:

[0057]

[0058] Among them, (x, y) is the chicken color segmentation map I L ′ is the pixel coordinates, (u, v) is the transformed color image I T The pixel coordinates of .

[0059] The chicken key posture recognition model is obtained by training the posture atlas using the SE-ResNet neural network. The posture atlas is obtained by repeating steps 2)-4) for multiple color images of different chickens with known chicken posture categories. Specifically, in the process of training the posture atlas using the SE-ResNet neural network: the number of iterations is set to 200 times, the minimum batch size is 16 images, and in order to solve the problem of unbalanced data distribution, the FocalLoss loss function is used to improve the classification accuracy, the learning rate is initialized to 0.001, and the learning rate decay training strategy is used to adjust the learning rate.

[0060] Among them, Figure 8 As shown in the figure, the chicken posture categories are divided into five categories: standing posture, head down posture, feather combing posture, wings spread posture and squatting posture.

[0061] The embodiment of the present invention selects 480 chicken color images with known chicken postures from the posture atlas to train the chicken key posture recognition model, and another 120 chicken color images with known chicken postures are used as test objects. After being processed by the present invention, the detection accuracy of the chicken posture key frame is 97.9%.

[0062] Table 1 lists the average absolute error and average relative error of poultry weight estimation under different postures. It is found that the weight estimation method based on posture keyframes has the smallest error. The results show that the posture keyframe detection method for poultry weight estimation can effectively improve the accuracy of poultry weight estimation.

[0063] Table 1. Estimation error of poultry weight under different postures

[0064]

[0065] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any modification, equivalent substitution, equivalent change, etc. made according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall be included in the protection scope of the present invention.

Claims

1. A posture key frame detection method for poultry weight estimation, characterized in that: The following steps are involved: 1) Acquire multiple color images and multiple depth images of a single chicken at different time periods, read the color image and depth image of the chicken in the current frame in sequence, obtain the chicken situation in the color image of the current frame and the chicken situation in the depth image of the current frame respectively, and then perform model training on the color image of the chicken in the current frame, the depth image of the chicken in the current frame, the chicken situation in the color image of the chicken in the current frame, and the chicken situation in the depth image of the chicken in the current frame to obtain a chicken semantic segmentation model; 2) Use the chicken semantic segmentation model to analyze the chicken color image I containing the chicken to be tested L Perform segmentation to obtain the segmentation mask image I S , segment the mask image I S and a color image of chickens containing the chickens to be tested I L Perform image and operation to obtain the chicken color segmentation map I L ′; 3) The segmentation mask image I in step 2) S Contains the chicken outline C, determine the segmentation mask image I S The minimum circumscribed rectangle of the chicken outline C is then combined with the minimum circumscribed rectangle of the chicken outline C to obtain the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformation T lateral direction; 4) The transformed color image I T After cropping, the image is transformed to Lab color space to obtain the chicken color image I. Lab , according to the chicken color map I Lab After transformation, the color image I T Rotate and adjust all transformed color images I T The heads of the tested chickens are facing the same direction; 5) The transformed color image I processed in step 4) T Input into the pre-trained chicken key posture recognition model to obtain the detection result of the chicken posture category to be tested. If the detection result is a standing posture, the current frame is the posture key frame; otherwise, read the next frame of chicken color image and repeat steps 2)-5).

2. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The step 1) is specifically as follows: Firstly, multiple color images and depth images of a single chicken are obtained at different time periods, and the color image and depth image of the chicken in the current frame are read in turn to obtain the chicken conditions in the color image of the current frame and the chicken conditions in the depth image of the current frame, respectively. Then, the RedNet neural network is used to train the model of multiple color images of a single chicken at different time periods, multiple depth images of chickens, the chicken conditions in the color image of the current frame, and the chicken conditions in the depth image of the current frame to obtain the chicken semantic segmentation results, and the trained RedNet neural network is used as the chicken semantic segmentation model.

3. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The step 3) is specifically as follows: First determine the segmentation mask image I S The minimum circumscribed rectangle of the chicken outline C is obtained, and then a new coordinate system is established with the vertex with the largest vertical coordinate in the pixel coordinate system as the coordinate origin. The straight line of the rectangular side in the minimum circumscribed rectangle passing through the coordinate origin and away from the vertical axis of the pixel coordinate system is set as the x-axis of the new coordinate system, and the straight line of the other rectangular side in the minimum circumscribed rectangle passing through the coordinate origin is set as the y-axis of the new coordinate system. The coordinates of the center point of the minimum circumscribed rectangle in the new coordinate system are P(x0, y0), and the length of the rectangular side on the x-axis is l1. The length of the side of the rectangle on the y-axis is l2, and the angle between the x-axis of the new coordinate system and the horizontal axis of the pixel coordinate system is θ. Then, the following judgment is made based on the size of the side length l1 of the rectangle on the x-axis and the side length l2 of the rectangle on the y-axis: if the side length l1 of the rectangle on the x-axis is less than the side length l2 of the rectangle on the y-axis, then the angle θ is subtracted by 270° to obtain a new angle θ; otherwise, the angle θ is kept unchanged; the rotation matrix M is determined based on the center coordinates P (x0, y0) and the angle θ, and the rotation matrix M is used to transform the chicken color segmentation image I L ′ Perform affine transformation to obtain the transformed color image I T , so that the chicken head of the tested chicken faces the color image I after the transformation T Horizontal direction.

4. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The step 4) is specifically as follows: The transformed color image I T Color image after transformation I T The minimum circumscribed rectangle of the inner chicken outline C is clipped to obtain the clipped chicken color image I crop , the cropped chicken color image I crop Transform to Lab color space and get the chicken color image I Lab , in the chicken color map I Lab Extract the a component to get the a component image I a ; The a component image I a Segment along its longitudinal central axis to obtain two equally divided parts, one of which is the left sub-image I a1 , the other part is the right sub-image I a2 , respectively calculate the left sub-image I a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I is obtained according to the calculation a1 , right sub-image I a2 The average gray value of the non-zero pixel area in the left sub-image I a1 The average pixel value of the right sub-image I a2 The average pixel value of the transformed color image I T unchanged; otherwise, the transformed color image I T Perform a 180° rotation, that is, adjust all the transformed color images I T The heads of the tested chickens were facing in the same direction.

5. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The chicken depth image is obtained directly through a depth camera or indirectly through a binocular color camera.

6. The method for detecting posture key frames for poultry weight estimation according to claim 3, characterized in that: The rotation matrix M is determined by the following formula: Among them, x0 and y0 are the horizontal and vertical coordinates of the center point of the minimum circumscribed rectangle respectively.

7. A posture key frame detection method for poultry weight estimation according to claim 6, characterized in that: The affine transformation is performed by the following formula: Among them, (x, y) is the chicken color segmentation map I L ′ is the pixel coordinates, (u,v) is the transformed color image I T The pixel coordinates, M 11 、M 12 、M 13 、M 21 、M 22 and M 23 is the element in the rotation matrix M.

8. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The chicken key posture recognition model is obtained by using the SE-ResNet neural network to perform model training on a posture atlas, and the posture atlas is obtained by repeating steps 2)-4) for a plurality of chicken color images of different chickens with known posture categories to be tested.

9. The method for detecting posture key frames for poultry weight estimation according to claim 1, characterized in that: The chicken posture categories are divided into five categories: standing posture, head lowering posture, feather combing posture, wings spreading posture and squatting posture.

Images