Chicken flock uniformity estimation method, device, system and medium based on depth image
Automatically extract features through deep learning network to classify broiler weight, solving the stress response and high breeding costs caused by manual operation of traditional broiler weight measurement methods, and achieving automated chicken flock uniformity monitoring and efficient weight estimation.
Patent Information
- Application Number
- CN202111062601.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-09-10
AI Technical Summary
Traditional broiler weight measurement methods require manual operation, resulting in broiler stress response, high breeding costs and difficult timely monitoring of chicken flock uniformity, affecting the health and production efficiency of chicken flocks.
Deep learning network based on depth images is used to automatically extract features, weight classification and flock uniformity estimation are performed, reducing manpower investment and improving measurement accuracy.
Automatic flock uniformity monitoring is achieved, reducing the risk and cost of manual operation, and improving the accuracy of broiler weight estimation and the efficiency of flock management.
Smart Images

Figure CN113989538B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method, device, system and medium for estimating uniformity of a flock of chickens based on depth images, and belongs to the fields of computer vision, machine learning, deep learning and software development. Background Art
[0002] Traditionally, the calculation of the uniformity of a flock requires weighing the broilers. The traditional method of measuring the weight of broilers is mainly to use electronic weighing and measuring equipment. Although the measuring instrument can measure the weight of the chicken more accurately, the broilers often struggle during the weighing process, the weighing platform shakes significantly, and the accumulation of dirt will cause the result to deviate from the actual weight. At the same time, when measuring the weight of broilers, it is often necessary to contact the broilers, which will increase the stress response of the broilers, making the broilers in a poor state in the subsequent breeding environment. In severe cases, the broilers will be in a bad mood and die, which is not conducive to the welfare breeding of broilers. [1-2] Moreover, when measuring broilers, it is necessary to manually take the broilers out of the chicken coop or cage one by one, which consumes a lot of manpower and material resources, resulting in a sharp increase in breeding costs. In addition, it is impossible to frequently calculate the uniformity of the flock, so that farmers cannot obtain the uniformity of the flock in time, which is an important indicator for judging whether the flock is growing normally. As a result, there is a high probability that the flock will be in a sub-healthy or unhealthy state, and diseases will occur frequently, resulting in a decrease in the feed-to-meat ratio.
[0003] The uniformity of a flock specifically includes three aspects: uniformity of body shape, uniformity of weight, and uniformity of sexual maturity. The uniformity of body shape is basically finalized in the early stage of brooding, and it determines the uniformity of weight. The uniformity of weight can be improved by grouping, group adjustment, and feed regulation, which in turn determines the consistency of sexual maturity, that is, the uniformity of sexual maturity. These three uniformities are closely related and together constitute a complete concept of uniformity. In chicken production, it is very important to improve the uniformity of the flock. Grouping management is an important measure to improve the uniformity of the flock. When strengthening daily feeding management, the flocks with poor uniformity need to be reasonably grouped. Untimely grouping and adjustment are one of the main factors for the reduction of uniformity. If the uniformity of the flock is not good, the grouping operation or the daily inspection of the flock is found, it is inevitable to weigh the flock. Although sampling weighing (10% to 20%) is used, it still requires a lot of manpower and material resources, which increases the breeding cost.
[0004] In addition, due to the problem that traditional two-dimensional digital image processing technology can only obtain a small amount of information about objects, and the camera is sensitive to information about the surrounding environment, resulting in inaccurate monitoring accuracy. Therefore, in the field of computer vision, three-dimensional model acquisition technology has been favored by domestic and foreign researchers. Compared with traditional two-dimensional color images, depth images are not only unaffected by environmental factors such as external lighting, but also simple in post-image processing, so they are widely used in behavioral monitoring and weight prediction of large animals. [3-5] .
[0005] Since the 21st century, electronic information technology has developed rapidly, and its application in multiple industries has also changed the development model of these industries. At present, the research direction of many livestock and poultry experts is how to combine information technology with poultry farming to realize the automation and intelligence of poultry farming, so as to greatly improve the quality and output of poultry products. In terms of poultry behavior analysis and weight estimation, due to the characteristics of poultry such as small size, multiple joints, varied movement forms, and wide space range, many subtle movements of individuals are difficult to be captured and analyzed by cameras. Therefore, the current analysis and research on poultry is mainly focused on poultry group behavior. Weight weighing methods are often used in poultry breeding. The traditional weighing method uses manual weighing, but it is not only labor-intensive but also has high labor costs. In addition, due to various external reasons, the accuracy is not high. Therefore, with the development of computer vision technology, more and more studies are being conducted on its application in animal behavior and weight monitoring. Computer vision technology has many advantages such as low cost, no direct contact with animals, and continuous shooting, and is becoming a very important research method.
[0006] At present, computer vision technology is mainly used in the following two aspects in the research of poultry: 1) Monitoring of poultry body changes: When analyzing the behavior of poultry, body change characteristics are very important. Body changes include various behavioral information such as the position, posture, speed, and outline of the poultry. When poultry exhibit abnormal behavior, it indicates that there is an abnormality in its living environment or that it has health problems. Therefore, many scholars at home and abroad have begun to use digital image processing technology to automatically identify individual and group behaviors of poultry. Scholars such as Lao Fengdan [6] proposed using machine vision technology to analyze and identify images of laying hens. The method they proposed can very accurately identify nine behavioral characteristics of a single chicken, such as drinking water, eating, shaking, resting, flapping wings, and raising wings. Lao Fengdan et al. [7] used a 3D camera to synchronously collect 2D and depth image data of laying hens in an experimental environment, and developed intelligent recognition software for laying hen behavior. The automatic recognition software has an average recognition accuracy of over 80% for the four behaviors of laying hens: feeding, lying, standing and sitting. 2) Poultry weight estimation: The weight measurement method is mainly used in the growth and development of broilers. Because weight can reflect the growth status, feed conversion rate and flock uniformity of broilers, it is one of the important indicators for breeders to evaluate the growth status of broilers. Common broiler weight weighing methods are mainly carried out manually. The most serious problem of this method is that it requires direct contact with broilers, which causes stress reactions in broilers and even causes death in severe cases. With the rapid development of computer vision technology, many scholars are gradually taking 2D and depth images of animals, and calculating the volume and other weight-related data of poultry through recognition, analysis and feature extraction, and building corresponding mathematical models to estimate the weight of poultry. [8-9] Mollah et al.
[10] Using a camera to photograph broilers, a new image processing technology was developed based on the two-dimensional color images obtained. The surface area of the broiler was extracted from the image, and then a linear equation was established between the surface area of the broiler in the image and the actual weight. The experimental results showed that the average relative error of using this technology to estimate the weight of broilers was only 0.04% to 16.47%. Wet et al.
[11] The correlation between the surface area and peripheral features of broiler chickens and their weight was studied and analyzed using image processing technology. The results showed that the average relative error of the estimated weight was only 11%. The above experiments fully demonstrate that machine vision technology can be used to estimate the weight of poultry.
[0007] Traditionally, when estimating the uniformity of a flock of chickens and measuring the weight of broilers, it is necessary to catch the chickens and then use hanging scales, electronic balances, etc. to perform manual measurement and calculation, and perform statistical data to obtain the uniformity information of the flock of chickens. The present invention aims to use new computer vision and machine learning technologies to predict the average weight of the flock of chickens, automatically obtain the uniformity information of the flock of chickens, simplify the operation, and facilitate the flock managers to manage the flock of chickens, improve the overall quality of broilers, reduce manpower input and reduce breeding costs.
[0008] Foreign scholar Anders Krogh Mortensen
[12] et al. have realized the use of depth images to estimate the weight of broiler chickens. After extracting 12 features from the depth map, they used multiple linear regression, feedforward neural network and Bayesian artificial neural network to conduct modeling experiments. The results showed that the average relative error between the predicted weight and the reference weight was 7.8%. The model diagram of their data acquisition system is shown in the figure below. Figure 1 As shown in the flowchart of estimating broiler weight from depth images Figure 2 shown.
[0009] The broiler image segmentation in the image processing process uses the watershed segmentation algorithm. The one-dimensional features of age, projection area, broiler width, circumference, maximum inscribed circle radius and eccentricity are used as two-dimensional features in the model. Volume, convex volume, surface area, convex surface area, back width and back height are used as three-dimensional features. Multiple linear regression, feedforward neural network and Bayesian artificial neural network are used for modeling and estimation, and the best average relative average error is 7.8%.
[0010] Domestic scholar Wang Lin
[13] et al. have also used depth images taken by Kinect cameras and BP neural networks to build a model to estimate the body weight of broilers and used it to fit the growth model of broilers. They chose image selection and cropping, median filtering, Otsu threshold segmentation and binarization, object identification method to obtain the maximum target, and morphological opening and closing reconstruction steps to process the broiler depth images. Then 9 features were also extracted from the depth map. However, unlike Anders Krogh Mortensen abroad, Wang Lin et al. used BP neural network (a typical supervised neural network classifier BP neural network) for weight modeling. The results showed that the accuracy rate was as high as 99.43%. Their image processing process and target extraction result diagram are shown below. Figure 3 and Figure 4 shown.
[0011] Nine features, including age, length and width of the target minimum rectangle, projection area, contour perimeter, maximum inscribed circle radius and eccentricity, volume and back width, were used in combination with BP neural network to estimate the weight of broiler chickens in a group. The estimated results were compared with the actual measurement results. The results showed that the root mean square error of the two was 0.048, the average relative error was 3.3%, the absolute error was in the range of 0.0010-0.0682kg, and the optimal fit was 0.9943. In the broiler weight grading experiment, the support vector machine (SVM) algorithm and RBF neural network algorithm were used to identify and verify the broiler weight grading (three levels). The results showed that based on the nine extracted features, the grading accuracy of the support vector machine and RBF neural network algorithms for the test set reached 94.17% and 86.39% respectively. The weight grading effect is good and can meet the requirements of automatic weight grading of broiler pictures.
[0012] In summary, the shortcomings of the existing technology are: 1) The image information acquisition device is fixed in the laboratory and cannot be put into the real production environment; 2) Image processing does not make full use of depth information, and target recognition and segmentation are incomplete and inefficient; 3) Effective feature extraction is insufficient, and the model effect can be improved; 4) A better model can be used for fitting and estimating weight; 5) There is no client system provided for chicken managers to track information. The reasons are: 1) A complete set of equipment and feasible solutions have not been designed to enable it to be truly applied in the production environment; 2) Broilers are active and changeable, and their behavior under the camera is uncontrolled, and they often shake, spread their wings, and other behaviors, resulting in inaccurate classification or estimation results.
[0013] References are as follows:
[0014] [1] Zhang Jie, Zhang Denghui. Comparison and reflection on the current status of animal welfare at home and abroad [J]. Journal of Animal Husbandry and Veterinary Medicine, 2013, 32(1): 36-38
[0015] [2] Zhao Ziguang. Effects of feeding methods on broiler welfare[D]. Harbin: Northeast Agricultural University, 2011.
[0016] [3]J Kongsro. Estimation of pig weight using a Microsoft Kinectprototype imaging system[J]. Computers&Electronics in Agriculture, 2014, 109:32-35.
[0017] [4] Guo Hao, Wang Peng, Ma Qin, et al. Cow body shape assessment index acquisition technology based on depth image [J]. Transactions of the Chinese Society of Agricultural Machinery, 2013, 44(S1): 273-276.
[0018] [5] Liu Bo, Zhu Weixing, Yang Jianjun, et al. Pig cadence feature extraction based on depth image and pig skeleton endpoint analysis [J]. Transactions of the Chinese Society of Agricultural Engineering, 2014, 30(10): 131-137.
[0019] [6] Lao Fengdan, Teng Guanghui, Li Jun, et al. Method of machine vision recognition of single laying hen behavior[J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(24): 157-163.
[0020] [7] Lao Fengdan, Du Xiaodong, Teng Guanghui. Layer behavior recognition method based on depth image[J]. Transactions of the Chinese Society of Agricultural Machinery, 2017, 48(1): 156-162.
[0021] [8] Shen Mingxia, Liu Longshen, Yan Li, et al. Research progress in individual information monitoring technology for livestock and poultry breeding [J]. Transactions of the Chinese Society of Agricultural Machinery, 2014, 45(10): 245-251.
[0022] [9] Xiong Benhai, Luo Qingyao, Yang Liang. Research on key technologies of Internet of Things for livestock precision feeding[J]. China Agricultural Science and Technology Herald, 2011, 13(5):19-25.
[0023]
[10] M Mollah, M Masan, M Salam.Digital image analysis to estimate the live weigt of broiler[J].Computers&Electronics in Agriculture, 2012, 72(1):48-52.
[0024]
[11] W De, E Vranken, A Chedad, et al. Computer-assisted image analysis to quantify daily growth rates of broiler chickens[J]. British Poultry Science, 2003, 44(4):524-32.
[0025]
[12] Mortensen A, Lisouski P, Ahrendt P.Weight prediction of broilerchickens using 3D computer vision[J].Computers&Electronics in Agriculture, 2016,123(C):319-326
[0026]
[13] Wang Lin, Sun Chuanheng, Li Wenyong, Ji Zengtao, Zhang Xiang, Wang Yizhong, Lei Peng, Yang Xinting. Broiler body mass estimation model based on deep image and BP neural network[J]. Transactions of the Chinese Society of Agricultural Engineering, 2017, 33(13): 199-205. Summary of the invention
[0027] In view of this, the present invention provides a method, device, system, computer equipment and storage medium for estimating the uniformity of a flock of chickens based on depth images, which utilizes a deep learning network to automatically extract image features and perform weight classification, increasing the number of classifications to achieve a weight estimation within an error (50g), and finally calculating the uniformity of the flock of chickens in combination with the weight.
[0028] The first object of the present invention is to provide a method for estimating uniformity of a flock of chickens based on depth images.
[0029] The second object of the present invention is to provide a chicken flock uniformity estimation device based on depth images.
[0030] The third object of the present invention is to provide a chicken flock uniformity estimation system based on depth images.
[0031] A fourth object of the present invention is to provide a computer device.
[0032] A fifth object of the present invention is to provide a storage medium.
[0033] The first object of the present invention can be achieved by adopting the following technical solutions:
[0034] A method for estimating uniformity of a flock of chickens based on a depth image, the method comprising:
[0035] Get the broiler chicken depth image library;
[0036] According to the broiler chicken deep image library, construct the target segmentation dataset and weight classification dataset;
[0037] Build target segmentation network and weight classification network;
[0038] The target segmentation network is trained using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset;
[0039] The weight classification network is trained using the target mask image dataset, the rectangular cropped image dataset, and the weight classification dataset;
[0040] The trained weight classification network is used to estimate the test image to obtain the weight category and the corresponding weight range, and the flock uniformity is calculated based on the flock data and the standard broiler weight at day age.
[0041] Furthermore, the target segmentation network is trained using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset, specifically including:
[0042] Process the images in the target segmentation dataset;
[0043] Input the processed image into the target segmentation network, use the Dice loss function to train the target segmentation network, and generate a weight file;
[0044] According to the weight file, generate the target mask image dataset and rectangular cropped image dataset.
[0045] Furthermore, the Dice loss function is as follows:
[0046] d=1-2|X∩Y||X|+|Y|
[0047] Where |X∩Y| refers to the intersection between X and Y; |X| and |Y| represent the number of elements in X and Y respectively, X represents the GT segmentation image, and Y represents the Pred segmentation image.
[0048] Furthermore, the training of the weight classification network using the target mask image dataset, the rectangular cropped image dataset and the weight classification dataset specifically includes:
[0049] The target mask image dataset, rectangular cropped image dataset and weight classification dataset are input into the weight classification network, and the weight classification network is trained using the CrossEntropy loss function.
[0050] Furthermore, the CrossEntropy loss function specifically includes:
[0051]
[0052] Among them, p(x) is the true probability distribution and q(x) is the predicted probability distribution.
[0053] Furthermore, the target segmentation network uses a Unet neural network for pre-segmentation, and a watershed segmentation algorithm is used for re-segmentation;
[0054] The weight classification network is a modified basic image classification network, in which the last classifier network layer of the basic image classification network is replaced by a fully connected layer to output quality category scores; the basic image classification network weights are initialized by training on the ImageNet dataset, and the fully connected weights added are randomly initialized.
[0055] Furthermore, the flock uniformity is calculated based on the flock data and the standard broiler weight at day age, as shown in the following formula:
[0056] B=[0.1-0.15]
[0057] H1=WW*B
[0058] H2=W+W*B
[0059] A=P / X*100%
[0060] Among them, B ranges from 0.1 to 0.15, W is the standard weight of the chicken breed or the average weight of the sample, P is the number of chickens between the upper and lower limits of weight, and A is the uniformity of weight.
[0061] The second object of the present invention can be achieved by adopting the following technical solutions:
[0062] A chicken flock uniformity estimation device based on depth images, the device comprising:
[0063] The acquisition module is used to obtain the broiler chicken depth image library;
[0064] A construction module is used to construct a target segmentation dataset and a weight classification dataset based on a broiler chicken depth image library;
[0065] Building modules for building object segmentation networks and weight classification networks;
[0066] A first training module is used to train a target segmentation network using a target segmentation data set to generate a target mask image data set and a rectangular cropped image data set;
[0067] A second training module is used to train a weight classification network using a target mask image dataset, a rectangular cropped image dataset, and a weight classification dataset;
[0068] The estimation module is used to estimate the image to be tested using the trained weight classification network to obtain the weight category and the corresponding weight range, and calculate the uniformity of the flock based on the flock data and the standard broiler weight at day age.
[0069] The third object of the present invention can be achieved by adopting the following technical solutions:
[0070] A chicken flock uniformity estimation system based on depth images, the system comprising an image acquisition device, a client and a server, the image acquisition device and the client are connected to the server respectively;
[0071] An image acquisition device, used for acquiring depth images of broiler chickens;
[0072] The client is used to collect the images to be tested, upload the images to be tested and the necessary information of the chicken farm to the server, and receive the uniformity of the chicken flock returned by the server;
[0073] The server is used to execute the above-mentioned chicken flock uniformity estimation method.
[0074] The fourth object of the present invention can be achieved by adopting the following technical solutions:
[0075] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned chicken flock uniformity estimation method is implemented.
[0076] The fifth object of the present invention can be achieved by adopting the following technical solutions:
[0077] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned chicken flock uniformity estimation method is implemented.
[0078] The present invention has the following beneficial effects compared with the prior art:
[0079] 1. The present invention does not manually extract features from the broiler depth image, but uses a deep neural network to automatically extract features. The present invention proposes a new solution for animal weight prediction. Compared with the prior art that uses the extracted feature data to predict weight using regression prediction or Bayesian network or BP neural network, the present invention converts the task of predicting weight into an image classification problem, and uses an image classification network to predict weight, and then obtains the uniformity of the chicken flock, that is, the neural network is used to automatically extract features and use image classification methods to estimate weight. This is different from the previous manual extraction of image features and the use of machine learning models to estimate weight. It is more operator-friendly and has little impact on the chicken flock. The use of deep images can better reflect target features than RGB images, and no 3D point cloud modeling is used on the model, thereby speeding up the model.
[0080] 2. Compared with the prior art which constructs a broiler data set of only a top view at a fixed height, the present invention also constructs data sets of top view and side view, and then creates a binary group of the data set for a weight classification network.
[0081] 3. For the weight estimation of a single chicken from a depth image, the present invention uses the original image after center cropping at each angle for weight classification, as well as the original image after fixed-size rectangular cropping for weight classification. For the weight estimation of multiple chickens from depth images, the segmented mask image is used for weight classification.
[0082] 4. The target segmentation network of the present invention uses the Unet segmentation network for pre-segmentation. For abnormal segmentation, such as the situation where multiple chickens are stuck together, the watershed segmentation algorithm is used for re-segmentation to obtain each chicken instance.
[0083] 5. The weight classification network of the present invention is a modified basic image classification network, in which the last layer of the classifier network of the basic image classification network is replaced by a fully connected layer to output a quality category score;
[0084] 6. The present invention provides a computer-side management system and a WeChat applet side for chicken flock managers to submit data acquisition results and track the results to make decisions on chicken flock management. BRIEF DESCRIPTION OF THE DRAWINGS
[0085] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.
[0086] Figure 1 This is a model diagram of the data acquisition system in the prior art.
[0087] Figure 2 Flowchart for estimating broiler weight using depth images from the prior art.
[0088] Figure 3 The figure is a flowchart of image processing in the prior art.
[0089] Figure 4 Result graph of target extraction for prior art.
[0090] Figure 5 This is a design diagram of a supporting folding camera according to embodiment 1 of the present invention.
[0091] Figure 6 This is a design diagram of a handheld camera according to Embodiment 1 of the present invention.
[0092] Figure 7 A schematic diagram of a depth image collected according to Example 1 of the present invention.
[0093] Figure 8Schematic diagram of another depth image collected in Example 1 of the present invention
[0094] Fig. 9 This is a flow chart of a method for estimating uniformity of a flock of chickens based on depth images according to Example 1 of the present invention.
[0095] Fig.10 This is a schematic diagram of the construction principle of the broiler depth image dataset of Example 1 of the present invention.
[0096] Fig.11 This is a structural diagram of the target segmentation network of Example 1 of the present invention.
[0097] Fig.12 This is a structural diagram of the weight classification network of Example 1 of the present invention.
[0098] Fig.13 This is a diagram of the image classification network structure based on Example 1 of the present invention.
[0099] Fig.14 This is a diagram of the interaction between the client and the server in Example 1 of the present invention.
[0100] Fig.15 This is a structural block diagram of a chicken flock uniformity estimation device based on depth images according to Embodiment 2 of the present invention.
[0101] Fig.16 This is a structural block diagram of a computer device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION
[0102] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0103] Embodiment 1:
[0104] The poultry farming industry, like other industries, needs to seek new ways of transformation. This embodiment uses computer vision to predict the weight of broiler chickens and then calculate the uniformity of the flock, achieving zero-contact prediction of the uniformity of the flock, thereby significantly reducing the manpower and material resources consumed by traditional weighing of broiler chickens and manual calculation of the uniformity of the flock, and most importantly, without causing any irritation to the flock.
[0105] This embodiment provides a chicken flock uniformity estimation system based on depth images. The system includes an image acquisition device, a client and a server. The image acquisition device and the client are connected to the server respectively.
[0106] The image acquisition device is used to acquire the depth image of the broiler. The image acquisition device can be an automatic support folding camera, and its design diagram is as follows: Figure 5 As shown, the specific operation is: set up a supporting folding camera in the chicken house in advance (opened horizontally at 90 degrees), pull out the data transmission line and connect it to the remote computer, and the computer program controls the camera to take a photo every 1 minute (to avoid the problem of uneven sampling after continuous shooting), shoot at one sampling point for 15 to 20 minutes, and then move the supporting folding camera to the next sampling location; the image acquisition device can also be a handheld camera, and its design diagram is shown in Figure 6 As shown, the top is a depth camera, which is fixed to the handle using a 1 / 4-20UNC threaded hole. The data cable is wrapped around the handle, and the end of the data cable is connected to a laptop computer. The handle is 60-100cm. The specific operation is: hold the camera and hold the pole to randomly walk around the chicken house (shoot or record), connect the data transmission line to the portable laptop or tablet, start recording video or taking images, and use the program to control the camera to shoot and save images at intervals of 1 second, or manually control the computer to shoot. A chicken house with about 1,000 chickens takes 10 to 15 minutes to shoot. The images taken may have poor or invalid images, which are manually removed. If you shoot videos, use the program to automatically track the chickens and remove the image data with poor filtering effects or invalid images during data training, and only track and count the chickens in the valid images.
[0107] The collected depth image information will be automatically saved to the computer disk for local processing or uploaded to the server (cloud server) for further processing. The depth images of broilers are collected from multiple angles:
[0108] 1) Top view, the distance between the camera plane and the broiler is about 50 cm (the same below).
[0109] 2) Side view (including left and right sides).
[0110] 3) Front view (including front and back).
[0111] If you use a handheld depth camera to record a video, use a lower frame rate, such as 6 frames, a resolution of 1280*720, a shooting height of about 1m (waist position), and a shooting time of 10 minutes. Save the depth video in .bag format; then parse each frame of the video, capture the depth matrix at 1 frame per second or 1 frame every two seconds, convert it into a grayscale image, and then convert it into a pseudo-color image and save it for making a data set.
[0112] A "depth camera" is a camera that can capture the real world as humans perceive it, both in color and in "distance". A depth camera provides an RGB image and a depth image. The depth image is a distance image with added colors to depict the depth range, displayed as a pseudo-color image, such as Figure 7 and Figure 8 shown.
[0113] The client can provide a management system based on the computer, allowing the chicken farm manager to upload image data and necessary information about the chicken farm. After processing by the background system, the chicken group weight information and the uniformity of the chicken group can be obtained. It can also provide a small program client or build a mobile phone APP based on the WeChat platform to query and track the results of previous sampling.
[0114] like Fig. 9 As shown, this embodiment provides a method for estimating uniformity of a flock of chickens based on a depth image, the method comprising the following steps:
[0115] S901, obtaining a broiler chicken depth image library.
[0116] The broiler depth images are collected by an image acquisition device and preprocessed (abnormal images are removed, such as images without targets in the field of view) to obtain a broiler depth image library.
[0117] S902: construct a target segmentation dataset and a weight classification dataset based on the broiler chicken depth image library.
[0118] According to the broiler chicken depth images in the broiler chicken depth image library, a training set and a test set for target segmentation are constructed as the target segmentation dataset, and a training set and a test set for weight classification are constructed as the weight classification dataset. The specific principle is as follows Fig.10 shown.
[0119] S903, building an object segmentation network (OSN) and a weight classification network (WSN).
[0120] The structure of the target segmentation network is as follows Fig.11 As shown, the pre-segmentation is Unet neural network, and the re-segmentation is the watershed segmentation algorithm; the structure of the weight classification network is shown in 12, which is a modified basic image classification network. The structure of the basic image classification network is as follows Fig.12 As shown in Figure 1, the last layer of the classifier network of the basic image classification network is replaced by a fully connected layer to output the quality category score; the weights of the basic image classification network are initialized by training on the ImageNet dataset, and the fully connected weights added are randomly initialized.
[0121] S904: Train the target segmentation network using the target segmentation data set to generate a target mask image data set and a rectangular cropped image data set.
[0122] The step S904 specifically includes:
[0123] S9041. Process the images in the target segmentation data set.
[0124] The processing performed in this embodiment specifically includes: converting to a grayscale image; reducing to 256*256; and normalizing the image with a mean of 0.5 and a standard deviation of 0.5.
[0125] S9042. Input the processed image into the target segmentation network, use the Dice loss function to train the target segmentation network, and generate a weight file.
[0126] Specifically, the processed training set images and test set images are respectively input into the target segmentation network, and the target segmentation network is trained using the Dice loss function.
[0127] The Dice loss function is as follows:
[0128] d=1-2|X∩Y||X|+|Y|
[0129] Among them, |X∩Y| refers to the intersection between X and Y; |X| and |Y| represent the number of elements in X and Y respectively. The coefficient 2 in the numerator is because the common elements between X and Y are repeatedly counted in the denominator. X represents the GT segmentation image, and Y represents the Pred segmentation image.
[0130] S9043. Generate a target mask image dataset and a rectangular cropped image dataset according to the weight file.
[0131] S905 , training a weight classification network using the target mask image dataset, the rectangular cropped image dataset, and the weight classification dataset.
[0132] Specifically, the training set and test set of the target mask image dataset, rectangular cropped image dataset, and weight classification dataset are respectively input into the weight classification network, and the weight classification network is trained using the CrossEntropy loss function.
[0133] CrossEntropy loss function, specifically includes:
[0134]
[0135] Among them, p(x) is the true probability distribution and q(x) is the predicted probability distribution.
[0136] S906. Use the trained weight classification network to estimate the image to be tested, obtain the weight category and the corresponding weight range, and calculate the uniformity of the flock according to the flock data and the standard broiler weight at day age.
[0137] Save the trained weight classification network weights. In actual use, input the image to be tested sent by the client into the trained weight classification network to obtain the weight category and the corresponding weight range, and calculate the uniformity of the flock based on the flock data and the standard broiler day-age weight, and return it to the client. The client and server interact as follows Fig.14 shown.
[0138] The flock uniformity is calculated based on the flock data and the standard broiler weight at day age, as follows:
[0139] B=[0.1-0.15]
[0140] H1=WW*B
[0141] H2=W+W*B
[0142] A=P / X*100%
[0143] Among them, B ranges from 0.1 to 0.15, W is the standard weight of the chicken breed or the average weight of the sample, P is the number of chickens between the upper and lower limits of weight, and A is the uniformity of weight.
[0144] Compared with the prior art, the present embodiment uses the method of estimating the uniformity of a flock of 200 yellow dwarf chickens before they are put on the market. The results show that the relative error of the uniformity is very small (1.53%), the accuracy of the weight estimation is relatively high, and the average relative error of the weight estimation is 4.27%, which is better than the existing prediction methods.
[0145] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.
[0146] Embodiment 2:
[0147] like Fig.15 As shown, this embodiment provides a chicken flock uniformity estimation device based on depth images, which includes an acquisition module 1501, a construction module 1502, a building module 1503, a first training module 1504, a second training module 1505 and an estimation module 1506. The specific functions of each module are as follows:
[0148] The acquisition module 1501 is used to acquire a broiler chicken depth image library.
[0149] The construction module 1502 is used to construct a target segmentation dataset and a weight classification dataset based on the broiler chicken depth image library.
[0150] The building module 1503 is used to build the target segmentation network and the weight classification network.
[0151] The first training module 1504 is used to train the target segmentation network using the target segmentation data set to generate a target mask image data set and a rectangular cropped image data set.
[0152] The second training module 1505 is used to train the weight classification network using the target mask image dataset, the rectangular cropped image dataset and the weight classification dataset.
[0153] The estimation module 1506 is used to estimate the image to be tested using the trained weight classification network to obtain the weight category and the corresponding weight range, and calculate the uniformity of the chicken flock according to the chicken flock data and the standard broiler chicken day-age weight.
[0154] The specific implementation of each module in this embodiment can refer to the above-mentioned embodiment 1, which will not be described one by one here; it should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.
[0155] It is understood that the terms "first", "second", etc. used in the above apparatus may be used to describe various modules, but these modules are not limited by these terms. These terms are only used to distinguish the first module from another module. For example, without departing from the scope of the present invention, the first training module may be referred to as the second training module, and similarly, the second training module may be referred to as the first training module, and both the first training module and the second training module are acquisition modules, but not the same acquisition module.
[0156] Embodiment 3:
[0157] This embodiment provides a computer device, which may be a computer, a server, etc. Fig.16As shown, it includes a processor 1602, a memory, an input device 1603, a display device 1604 and a network interface 1605 connected through a system bus 1601. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 1606 and an internal memory 1607. The non-volatile storage medium 1606 stores an operating system, a computer program and a database. The internal memory 1607 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 1602 executes the computer program stored in the memory, the chicken flock uniformity estimation method of the above-mentioned embodiment 1 is implemented as follows:
[0158] Get the broiler chicken depth image library;
[0159] According to the broiler chicken deep image library, construct the target segmentation dataset and weight classification dataset;
[0160] Build target segmentation network and weight classification network;
[0161] The target segmentation network is trained using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset;
[0162] The weight classification network is trained using the target mask image dataset, the rectangular cropped image dataset, and the weight classification dataset;
[0163] The trained weight classification network is used to estimate the test image to obtain the weight category and the corresponding weight range, and the flock uniformity is calculated based on the flock data and the standard broiler weight at day age.
[0164] Embodiment 4:
[0165] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, the chicken flock uniformity estimation method of the above embodiment 1 is implemented as follows:
[0166] Get the broiler chicken depth image library;
[0167] According to the broiler chicken deep image library, construct the target segmentation dataset and weight classification dataset;
[0168] Build target segmentation network and weight classification network;
[0169] The target segmentation network is trained using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset;
[0170] The weight classification network is trained using the target mask image dataset, the rectangular cropped image dataset, and the weight classification dataset;
[0171] The trained weight classification network is used to estimate the test image to obtain the weight category and the corresponding weight range, and the flock uniformity is calculated based on the flock data and the standard broiler weight at day age.
[0172] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0173] In summary, the present invention uses a deep learning network to automatically extract image features and perform weight classification, increases the number of classifications to achieve weight estimation within an error (50g), and finally calculates the uniformity of the flock in combination with the weight. A client is provided to chicken farm managers for use in submitting image data, obtaining results and management suggestions, and tracking results to make management decisions.
[0174] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and inventive concept of the present invention within the scope disclosed by the present invention, which shall fall within the protection scope of the present invention.
Claims
1. A method for estimating uniformity of a flock of chickens based on depth images, characterized in that: The method comprises: Get the broiler chicken depth image library; According to the broiler chicken deep image library, construct the target segmentation dataset and weight classification dataset; Build a target segmentation network and a weight classification network. The target segmentation network uses a Unet neural network for pre-segmentation and a watershed segmentation algorithm for re-segmentation. The weight classification network is a modified basic image classification network. The last classifier network layer of the basic image classification network is replaced by a fully connected layer to output quality category scores. The weights of the basic image classification network are initialized by training on the ImageNet dataset, and the fully connected weights added are randomly initialized. The target segmentation network is trained using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset; The weight classification network is trained using the target mask image dataset, the rectangular cropped image dataset, and the weight classification dataset; The trained weight classification network is used to estimate the image to be tested, obtain the weight category and the corresponding weight range, and calculate the uniformity of the flock based on the flock data and the standard broiler weight at day age; The method of training the weight classification network by using the target mask image dataset, the rectangular cropped image dataset and the weight classification dataset specifically includes: Input the target mask image dataset, rectangular cropped image dataset and weight classification dataset into the weight classification network, and use the CrossEntropy loss function to train the weight classification network; The CrossEntropy loss function specifically includes: Among them, p(x) is the true probability distribution, and q(x) is the predicted probability distribution; The flock uniformity is calculated based on the flock data and the standard broiler weight at day age, as shown in the following formula: B=[0.1-0.15] H1=WW*B H2=W+W*B A=P / Q*100% Among them, B ranges from 0.1 to 0.15, W is the standard weight of the chicken breed or the average weight of the sample, P is the number of chickens between the upper and lower limits of weight, and A is the uniformity of weight.
2. The method for estimating uniformity of a flock of chickens according to claim 1, characterized in that: The target segmentation network is trained by using the target segmentation dataset to generate a target mask image dataset and a rectangular cropped image dataset, specifically including: Process the images in the target segmentation dataset; Input the processed image into the target segmentation network, use the Dice loss function to train the target segmentation network, and generate a weight file; According to the weight file, generate the target mask image dataset and rectangular cropped image dataset.
3. The method for estimating uniformity of a flock of chickens according to claim 2, characterized in that: The Dice loss function is as follows: d=1-2|X∩Y||X|+|Y| Where |X∩Y| refers to the intersection between X and Y; |X| and |Y| represent the number of elements in X and Y respectively, X represents the GT segmentation image, and Y represents the Pred segmentation image.
4. A chicken flock uniformity estimation device based on depth image, characterized in that: The device comprises: The acquisition module is used to obtain the broiler chicken depth image library; A construction module is used to construct a target segmentation dataset and a weight classification dataset based on a broiler chicken depth image library; A building module is used to build a target segmentation network and a weight classification network. The target segmentation network uses a Unet neural network for pre-segmentation and a watershed segmentation algorithm for re-segmentation. The weight classification network is a modified basic image classification network. The last layer of the classifier network of the basic image classification network is replaced by a fully connected layer to output quality category scores. The weights of the basic image classification network are initialized by training on the ImageNet dataset, and the fully connected weights added are randomly initialized. A first training module is used to train a target segmentation network using a target segmentation data set to generate a target mask image data set and a rectangular cropped image data set; A second training module is used to train a weight classification network using a target mask image dataset, a rectangular cropped image dataset, and a weight classification dataset; The estimation module is used to estimate the image to be tested using the trained weight classification network to obtain the weight category and the corresponding weight range, and calculate the uniformity of the flock based on the flock data and the standard broiler weight at day age; The method of training the weight classification network by using the target mask image dataset, the rectangular cropped image dataset and the weight classification dataset specifically includes: Input the target mask image dataset, rectangular cropped image dataset and weight classification dataset into the weight classification network, and use the CrossEntropy loss function to train the weight classification network; The CrossEntropy loss function specifically includes: Among them, p(x) is the true probability distribution, and q(x) is the predicted probability distribution; The flock uniformity is calculated based on the flock data and the standard broiler weight at day age, as shown in the following formula: B=[0.1-0.15] H1=WW*B H2=W+W*B A=P / Q*100% Among them, B ranges from 0.1 to 0.15, W is the standard weight of the chicken breed or the average weight of the sample, P is the number of chickens between the upper and lower limits of weight, and A is the uniformity of weight.
5. A chicken flock uniformity estimation system based on depth image, characterized in that: The system comprises an image acquisition device, a client and a server, wherein the image acquisition device and the client are connected to the server respectively; An image acquisition device, used for acquiring depth images of broiler chickens; The client is used to collect the images to be tested, upload the images to be tested and the necessary information of the chicken farm to the server, and receive the uniformity of the chicken flock returned by the server; A server end, used to execute the chicken flock uniformity estimation method described in any one of claims 1-3.
6. A storage medium storing a program, characterized in that: When the program is executed by a processor, the method for estimating uniformity of a flock of chickens as described in any one of claims 1 to 3 is implemented.
