Live pig weight estimation system and method

Abstract

The invention provides a live pig weight estimation system and method, and relates to the technical field of smart agriculture, the system comprises an image acquisition device and a first electronic device; the image acquisition device comprises a moving mechanism, a supporting mechanism and an image sensor; the image sensor is mounted on the supporting mechanism and is used for acquiring an RGB image and a depth image of a target live pig below the image sensor; the first electronic equipment is used for performing data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig, inputting the depth image after the data preprocessing of the target live pig into a live pig weight estimation model, and obtaining an estimated value of the weight of the target live pig output by the live pig weight estimation model. According to the live pig weight estimation system and method provided by the invention, the accuracy, speed and efficiency of live pig weight estimation can be remarkably improved, manual intervention is reduced during live pig weight estimation, and the practicability of live pig weight estimation is improved.

Description

Technical Field

[0001] The present invention relates to the field of smart agricultural technology, and in particular to a pig weight estimation system and method. Background Art

[0002] The weight of pigs is one of the important indicators for monitoring pig growth during pig breeding. The weight of pigs can directly reflect the health status and development progress of pigs. Quickly and accurately estimating the weight of pigs can help producers promptly identify pigs with abnormal growth or growth that deviates from the expected target, thereby reducing management labor costs and feed costs by adjusting feeding strategies.

[0003] In the related art, the traditional pig weight estimation method can use computer vision technology, machine learning technology or three-dimensional sensor technology to estimate the weight of the pig. However, the traditional pig weight estimation method relies on the image of the pig, and the traditional pig image acquisition method has the defects of easily causing stress reaction in the pig, low quality of the collected pig image and poor directionality.

[0004] In addition, the traditional pig weight estimation method in the related art has the technical defects of low pig weight estimation accuracy, large amount of calculation, high computational complexity, and low pig weight estimation efficiency. Therefore, how to improve the practicality, estimation accuracy and estimation efficiency of pig weight estimation is a technical problem to be solved in this field. Summary of the invention

[0005] The present invention provides a pig weight estimation system and method, which are used to solve the defects of traditional pig weight estimation methods in the prior art, such as low estimation accuracy or large calculation amount resulting in low calculation efficiency, so as to achieve more accurate and efficient estimation of pig weight.

[0006] The present invention provides a live pig weight estimation system, comprising: an image acquisition device and a first electronic device; the image acquisition device is electrically connected to the first electronic device; the image acquisition device includes a moving mechanism, a support mechanism and an image sensor; the moving mechanism is connected to the support mechanism, and the moving mechanism is used to drive the support mechanism to move; the image sensor is installed on the support mechanism, and the shooting direction of the image sensor is vertically downward perpendicular to the horizontal plane. The image sensor is used to acquire the RGB image and depth image of the target live pig located below the image sensor, and send the acquired RGB image and depth image of the target live pig to the first electronic device; the first electronic device is configured to, when receiving the RGB image and depth image of the target live pig, perform data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig, and input the depth image of the target live pig after data preprocessing into the live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model; wherein, the live pig weight estimation model is constructed based on an improved EfficientNetV2 network and is trained based on the depth image of the preprocessed sample live pig data and the actual value of the weight of the sample live pig; the depth image of the preprocessed sample live pig data is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the method of performing data preprocessing on the depth image of the sample live pig is the same as the method of performing data preprocessing on the depth image of the target live pig.

[0007] According to a live pig weight estimation system provided by the present invention, the support mechanism includes a support base and a multi-axis robotic arm. One end of the multi-axis robotic arm is fixed on the support base, and the other end of the multi-axis robotic arm is connected to the image sensor; the multi-axis robotic arm includes a plurality of connecting rods, and the connecting rods are sequentially rotationally connected.

[0008] According to a live pig weight estimation system provided by the present invention, the first electronic device is specifically configured to perform image recognition and instance segmentation on the RGB image of the target live pig, and after obtaining the mask of the target live pig, when it is determined that the mask of the target live pig is complete, map the mask of the target live pig to the depth image of the target live pig to obtain the segmentation result of the target live pig in the depth image of the target live pig, as the depth image of the target live pig after data preprocessing.

[0009] According to a live pig weight estimation system provided by the present invention, the live pig weight estimation model includes: a backbone network and a weight estimation head module connected in sequence; the backbone network is constructed based on the improved EfficientNetV2 network, and the improved EfficientNetV2 network is an EfficientNetV2 network in which the SE module is replaced by the CBAM module.

[0010] According to a live pig weight estimation system provided by the present invention, the weight estimation head module includes: a global average pooling unit, a first fully connected layer unit, a ReLU activation function unit, a regularization calculation unit, and a second fully connected layer unit connected in sequence.

[0011] According to a live pig weight estimation system provided by the present invention, the first electronic device is specifically configured to input the RGB image of the target live pig into the live pig recognition model to obtain the mask of the target live pig output by the live pig recognition model. The first electronic device is further configured to map the mask of the target live pig into a blank image having the same size as the RGB image of the target live pig. When it is determined that the mask of the target live pig does not intersect the boundary of the blank image, or although the mask of the target live pig intersects the boundary, the distance at which the mask of the target live pig intercepts the boundary is less than the distance threshold, it is determined that the mask of the target live pig is complete. The live pig recognition model is obtained by performing transfer learning on the training model based on the mask of the sample live pig, and the mask of the sample live pig is obtained by performing image recognition and instance segmentation on the RGB image of the sample live pig.

[0012] According to a live pig weight estimation system provided by the present invention, the support base has a telescopic function and can drive the multi-axis robotic arm to move in a direction perpendicular to the horizontal plane.

[0013] According to a live pig weight estimation system provided by the present invention, the first electronic device is electrically connected to at least one of the support base, the multi-axis robotic arm, and the moving mechanism; when the first electronic device is electrically connected to the moving mechanism, it is configured to control the moving direction and / or the moving distance of the moving mechanism; when the first electronic device is electrically connected to the multi-axis robotic arm, it is configured to control the angle between at least two adjacent connecting rods in the multi-axis robotic arm; when the first electronic device is electrically connected to the support base, it is configured to control the height of the support base.

[0014] A live pig weight estimation system provided according to the present invention, wherein the image acquisition device further includes a second electronic device; the second electronic device is electrically connected to at least one of the support base, the multi-axis robotic arm, and the moving mechanism; when the second electronic device is electrically connected to the moving mechanism, it is configured to control the moving direction and / or moving distance of the moving mechanism; when the second electronic device is electrically connected to the multi-axis robotic arm, it is configured to control the angle between at least two adjacent connecting rods in the multi-axis robotic arm; when the second electronic device is electrically connected to the support base, it is configured to control the height of the support base.

[0015] The present invention also provides a live pig weight estimation method implemented based on the live pig weight estimation system described in any one of the above, including: obtaining the RGB image and depth image of a target live pig, where the RGB image and depth image of the target live pig are collected by an image sensor located above the target live pig; performing data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig to obtain the depth image of the target live pig after data preprocessing; inputting the depth image of the target live pig after data preprocessing into a live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model; wherein, the live pig weight estimation model is constructed based on an improved EfficientNetV2 network and trained based on the depth image of the sample live pig after data preprocessing and the actual value of the weight of the sample live pig; the depth image of the sample live pig after data preprocessing is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the way of performing data preprocessing on the depth image of the sample live pig is the same as the way of performing data preprocessing on the depth image of the target live pig.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, it implements the live pig weight estimation method described in any one of the above.

[0017] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the live pig weight estimation method described in any one of the above.

[0018] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the live pig weight estimation method described in any one of the above.

[0019] The pig weight estimation system and method provided by the present invention. The pig weight estimation system includes an image acquisition device and a first electronic device. The image acquisition device includes a moving mechanism, a support mechanism, and an image sensor, and can use the image sensor to vertically capture the RGB image and depth image of the pig, and flexibly adjust the image acquisition range in combination with the moving mechanism, and can more accurately, efficiently, and flexibly acquire the image of the pig without causing stress reaction of the pig. After the first electronic device preprocesses the depth image using the RGB image of the pig, it uses the improved EfficientNetV2 model to estimate the pig weight and obtains the estimated value of the pig weight. Through the optimized training of the model, it can effectively fuse spatial and three-dimensional information during pig weight estimation, can significantly improve the accuracy of pig weight estimation, and the lightweight design of the model reduces the amount of calculation and computational complexity during pig weight estimation, can significantly improve the speed and efficiency of pig weight estimation, reduces manual intervention during pig weight estimation, can better avoid stress reaction of the pig, improves the practicability of pig weight estimation, can provide more reliable and efficient technical support for pig breeding management, and is applicable to the rapid and automatic monitoring of pig weight in large-scale breeding scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic structural diagram of the pig weight estimation system provided by the present invention.

[0022] Figure 2 It is a schematic structural diagram of the image acquisition device in the pig weight estimation system provided by the present invention.

[0023] Figure 3 It is a weight distribution diagram of the sample pig in the pig weight estimation system provided by the present invention.

[0024] Figure 4 It is a schematic flow diagram of the first electronic device in the pig weight estimation system for preprocessing the depth image of the target pig.

[0025] Figure 5 It is a schematic structural diagram of the pig weight estimation model in the pig weight estimation system provided by the present invention.

[0026] Figure 6 It is a comparison schematic diagram of the SE module and the CBAM module.

[0027] Figure 7 It is a schematic structural diagram of the weight estimation head module in the live pig weight estimation model of the live pig weight estimation system provided by the present invention.

[0028] Figure 8 It is a schematic diagram for comparing the structures of the weight estimation head module in the live pig weight estimation model provided by the present invention with two other weight estimation variants.

[0029] Figure 9 It is a schematic flowchart of the live pig weight estimation method provided by the present invention.

[0030] Figure 10 It is a schematic structural diagram of the first electronic device provided by the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0032] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the internal connection of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0033] In the description of the present application, the terms "first", "second", etc. are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. usually belong to the same category, and the number of objects is not limited. For example, the first object may be one or more. In addition, in the description of the present application, " / and" means at least one of the connected objects, and the character " / " generally means an "or" relationship between the associated objects before and after.

[0034] It should be noted that the weight of live pigs is one of the important indicators for monitoring the growth of live pigs during the breeding process. The weight of live pigs can directly reflect the health status and development progress of live pigs. Quickly and accurately estimating the weight of live pigs can help producers timely detect live pigs with abnormal growth or growth deviating from the expected target, thereby reducing the input of management labor costs and feed costs by adjusting the breeding strategy.

[0035] In related technologies, producers usually measure the weight of live pigs by driving the live pigs onto a weighing scale. The accuracy of the weight of live pigs obtained by the above method is the highest. However, the above method requires a large amount of labor costs, and at the same time, it is easy to cause stress reactions in live pigs. The live pigs weighed by the above method may reduce their feed intake and feeding frequency. In addition, excessive manual contact may accelerate the spread of swine fever among live pigs.

[0036] With the development of computer vision technology, image-based non-contact live pig weight estimation has attracted great attention in the agricultural field. In related technologies, computer vision technology can be used to achieve automatic non-contact estimation of live pig weight, which not only reduces the manual intervention during live pig weight estimation, avoids stress reactions in live pigs, but also reduces the input of labor costs.

[0037] For example, traditional live pig weight estimation methods based on computer vision technology and machine learning technology in related technologies can use feature engineering to extract weight-related features in live pig images, such as body shape parameters, etc., and then use a regression model to estimate the weight of live pigs based on the above body shape parameters. However, the above traditional live pig weight estimation methods based on computer vision technology and machine learning technology have certain limitations in feature selection and are difficult to capture complex non-linear features in live pig images, resulting in low accuracy of the estimated live pig weight.

[0038] In related technologies, the traditional live pig weight estimation methods based on computer vision technology and machine learning technology can also utilize models such as convolutional neural networks (CNNs) to automatically learn efficient feature representations, thus significantly improving the accuracy of live pig weight estimation. For example, the traditional live pig weight estimation methods based on computer vision technology and machine learning technology in related technologies can combine a live pig instance segmentation model based on Mask R-CNN (Mask Region-based Convolutional Neural Network) and an improved ResNet (Residual Network) weight estimation algorithm to achieve real-time estimation of live pig weights. Another example is that the traditional live pig weight estimation methods based on computer vision technology and machine learning technology in related technologies can, after extracting the contour of the live pig based on the Mask R-CNN live pig instance segmentation model, convert the mask image into a binary image to optimize the contour accuracy; combine XGBoost to correct the body length, hip width, and the distance from the camera to the pig's back, and adopt three feature combination strategies to predict the weight of the live pig.

[0039] However, although the above traditional live pig weight estimation methods based on computer vision technology and machine learning technology can achieve good results on specific data sets, their generalization ability is limited. Moreover, live pigs of different breeds and / or at different growth stages may have significant differences in morphology and weight, resulting in a decrease in the estimation accuracy when the above traditional live pig weight estimation methods based on computer vision technology and machine learning technology are applied to new live pig populations. Additionally, the above traditional live pig weight estimation methods based on computer vision technology and machine learning technology have poor robustness under different lighting conditions.

[0040] With the development of three-dimensional sensor technology, depth images and point cloud data have gradually been applied to live pig weight estimation. The above three-dimensional data can not only accurately capture the volume and shape information of live pigs but also avoid the influence of occlusion or light changes on the results of live pig weight estimation. For example, the traditional live pig weight estimation methods based on three-dimensional sensor technology in related technologies can extract the body size parameters of live pigs based on the surface point cloud of the live pigs and respectively construct stepwise regression, ridge regression, and partial least squares weight regression models for the estimation of live pig weights. Another example is that the traditional live pig weight estimation methods based on three-dimensional sensor technology in related technologies can also integrate the body size parameters of the pig's back with a convolutional neural network to estimate the weight of the live pig. However, the volume of three-dimensional data is large, and the resource requirements for processing and storage are higher, which will lead to a significant increase in computational costs.

[0041] Therefore, the traditional live hog weight estimation method in the related art proposes a live weight measurement method based on ConvNet, which only uses the depth image of the pig to predict the live hog weight without feature extraction. The traditional live hog weight estimation method in the related art can also use a regression network similar to BotNet for live hog weight estimation. The traditional live hog weight estimation method in the related art can also use an RGB-D two-stream network to simultaneously fuse the texture information of the RGB image and the geometric information of the depth image, further improving the accuracy of live hog weight estimation in some models.

[0042] However, when the above traditional live hog weight estimation method estimates the live hog weight, the number of parameters and the computational complexity increase significantly, resulting in a large computational amount for live hog weight estimation and a significant decrease in the estimation efficiency of live hog weight.

[0043] In live hog weight estimation, the selection of the image acquisition method has an important impact on practical applications. The traditional live hog image acquisition methods in the related art mainly include restricted space acquisition methods, channel acquisition methods, and free environment acquisition methods.

[0044] The restricted space acquisition method means that the live hog is separately restricted in a narrow area by a fence or a pig barrier. Since the acquisition range of the restricted space acquisition method is small and fixed, the live hog image obtained by the acquisition can be more easily segmented into the pig body. However, the restricted space acquisition method is likely to cause stress reactions in live hogs, which is not conducive to the healthy growth of live hogs.

[0045] The channel acquisition method is to install a camera above the channel. By driving, the live hog passes through the above channel, and the video stream is collected and the image frames of the live hog passing through the channel are extracted by using a target detection algorithm, so as to obtain the image of the live hog. However, the channel acquisition method is likely to result in a low accuracy of the estimated live hog weight because the live hog is located at the boundary of the image.

[0046] The free environment acquisition method selects to take pictures of live hogs at the drinking point or the feed trough in the pigsty. The free environment acquisition method meets the requirements of non-invasive acquisition. However, due to the limited camera field of view, it is usually impossible to collect the images of each live hog, and the appearance of live hogs in the images is random, making it difficult to estimate the weight of a certain live hog directionally.

[0047] Therefore, the application effects of the above traditional image acquisition methods in actual breeding scenarios still have limitations, and it is necessary to further improve the practicality of live hog image acquisition.

[0048] In summary, the traditional pig weight estimation methods in the relevant technologies have the following technical defects: First, some traditional pig weight estimation methods rely on fixed-position image acquisition devices or complex restriction structures, which are difficult to adapt to the dynamic and complex actual environment of the farm, affecting the practicality and promotion of traditional pig weight estimation methods. Secondly, although the traditional pig weight estimation method based on body size is simple and intuitive, it relies on accurate body size parameter extraction. In the actual environment, the surface contour of the pig is complex, and there are problems such as motion and occlusion, and the accuracy of body size measurement is difficult to guarantee. Finally, the traditional pig weight estimation method lacks targeted optimization of the model design, especially in terms of feature extraction and weight estimation performance improvement. There is still room for improvement. Although some deep learning models have improved the estimation accuracy, the complex model structure limits their application in resource-constrained environments.

[0049] Therefore, how to develop a more flexible and efficient pig image acquisition device, improve the accuracy of pig weight estimation, reduce the complexity and calculation amount of pig weight estimation, and thus improve the efficiency of pig weight estimation is a technical problem that needs to be urgently solved in this field.

[0050] In view of the above technical problems, the present invention provides a pig weight estimation system. The pig weight estimation system provided by the present invention includes a movable image acquisition device and an electronic device, and proposes a lightweight network based on an improved EfficientNetV2, which uses the pig depth image after instance segmentation to estimate the pig weight more conveniently, efficiently and contactlessly in actual pig farm management.

[0051] The pig weight estimation system provided by the present invention includes a mobile image acquisition device. Farmers do not need to drive pigs through the passage, and can operate flexibly in the farm environment to achieve efficient acquisition of pig back images. The present invention proposes a lightweight pig weight estimation model based on an improved EfficientNetV2 network that uses the spatial structure information of depth images. The CBAM attention mechanism is introduced in the EfficientNetV2 network to enhance the ability to capture spatial features.

[0052] Combine the following Figures 1-8 The present invention describes a pig weight estimation system.

[0053] Figure 1 This is a schematic diagram of the structure of the pig weight estimation system provided by the present invention. Figure 1 The pig weight estimation system provided by the present invention is described. Figure 1 As shown, the pig weight estimation system 101 includes: an image acquisition device 102 and a first electronic device 103; the image acquisition device 102 is electrically connected to the first electronic device 103.

[0054] The image acquisition device 102 includes a moving mechanism, a supporting mechanism, and an image sensor; the moving mechanism is connected to the supporting mechanism, and the moving mechanism is used to drive the supporting mechanism to move.

[0055] The image sensor is installed on the supporting mechanism. The shooting direction of the image sensor is vertically downward perpendicular to the horizontal plane. The image sensor is used to acquire the RGB image and the depth image of the target live pig located below the image sensor, and send the acquired RGB image and depth image of the target live pig to the first electronic device 103.

[0056] Specifically, the target live pig is the estimation object of the live pig weight estimation system 101 provided by the present invention. Based on the live pig weight estimation system 101 provided by the present invention, the weight of the target live pig can be estimated to obtain the weight estimation value of the target live pig.

[0057] It should be noted that the target live pig in the embodiments of the present invention can be determined based on actual needs. For example, in the embodiments of the present invention, each live pig in the pigsty can be sequentially determined as the target live pig; or, in the embodiments of the present invention, any live pig in the pigsty can also be determined as the target live pig according to actual needs.

[0058] In the embodiments of the present invention, the moving mechanism in the image acquisition device 102 can be used to move the image sensor above the target live pig. The supporting mechanism in the embodiments of the present invention can have degrees of freedom in both the horizontal and vertical directions. Therefore, the moving mechanism and the supporting mechanism can be used to move the image sensor above the target live pig.

[0059] It should be noted that the shooting direction of the image sensor in the embodiments of the present invention is the direction that passes through the lens center of the image sensor and is perpendicular to the plane where the lens of the image sensor is located, and extends in the direction away from the image sensor.

[0060] Figure 2 It is a schematic structural diagram of the image acquisition device in the live pig weight estimation system provided by the present invention. As an optional embodiment, as Figure 2 shown, the supporting mechanism includes a supporting base 201 and a multi-axis robotic arm 202. One end of the multi-axis robotic arm 202 is fixed on the supporting base 201, and the other end of the multi-axis robotic arm 202 is connected to the image sensor 203 continuously; the multi-axis robotic arm 202 includes a plurality of connecting rods, and each connecting rod is rotationally connected in sequence.

[0061] Specifically, the supporting mechanism in the embodiments of the present invention includes a supporting base 201 and a multi-axis robotic arm 202. The multi-axis robotic arm 202 is composed of a plurality of connecting rods, and each connecting rod is sequentially connected through a rotating joint. The above rotating joint can allow relative rotation between adjacent connecting rods, so as to realize the bending and extension of the above multi-axis robotic arm 202 in three-dimensional space.

[0062] By adjusting the included angle between the connecting rods in the above multi-axis robotic arm 202, the image sensor 203 can be driven to move above the target live pig.

[0063] Optionally, the material of the support mechanism in the embodiments of the present invention can be aluminum profiles, which are characterized by being lightweight, having relatively high strength, and being relatively inexpensive.

[0064] Optionally, the moving mechanism 204 can include four pulleys and a brake to ensure the stability of the moving mechanism 204 during the moving process.

[0065] As an optional embodiment, the support base 201 has a telescopic function and can drive the multi-axis robotic arm 202 to move in a direction perpendicular to the horizontal plane.

[0066] As an optional embodiment, the first electronic device 103 is electrically connected to at least one of the support base 201, the multi-axis robotic arm 202, and the moving mechanism 204; when the first electronic device 103 is electrically connected to the moving mechanism 204, it is used to control the moving direction and / or moving distance of the moving mechanism 204; when the first electronic device 103 is electrically connected to the multi-axis robotic arm 202, it is used to control the included angle between at least two adjacent connecting rods in the multi-axis robotic arm 202; when the first electronic device 103 is electrically connected to the support base 201, it is used to control the height of the support base 201.

[0067] As an optional embodiment, the image acquisition device 102 further includes a second electronic device; the second electronic device is electrically connected to at least one of the support base 201, the multi-axis robotic arm 202, and the moving mechanism 204; when the second electronic device is electrically connected to the moving mechanism 204, it is used to control the moving direction and / or moving distance of the moving mechanism 204; when the second electronic device is electrically connected to the multi-axis robotic arm 202, it is used to control the included angle between at least two adjacent connecting rods in the multi-axis robotic arm 202; when the second electronic device is electrically connected to the support base 201, it is used to control the height of the support base 201.

[0068] Optionally, the second electronic device in the embodiments of the present invention can be a notebook terminal with a quad-core and eight-thread i7-8650 CPU and 8GB of memory, the operating system is Windows, and it is powered by an external mobile power supply.

[0069] As an optional embodiment, the image sensor 203 includes an RGB image sensor 203 and a depth image sensor 203.

[0070] Optionally, the depth image sensor 203 in the embodiments of the present invention may be the Orbbec Femto Bolt depth camera. The working mode of the above depth camera is set to WFOV 2X2BINNED. The field of view of the acquired depth image is 120°×120°, the effective measurement range is 0.25 to 2.88 meters, and the frame rate is 30 frames per second.

[0071] It should be noted that after the RGB image and the depth image acquired by the RGB image sensor 203 and the depth image sensor 203 in the embodiments of the present invention are subjected to D2C alignment, the RGB image and the depth image are consistent in resolution and pixel points, and the depth map can be saved in a 16-bit depth format.

[0072] Optionally, the resolution of the RBG image sensor 203 is set to 1920×1080.

[0073] It can be understood that since the shooting direction of the image sensor 203 is perpendicular to the horizontal plane downward, the RGB image and the depth image of the target live pig are the RGB image and the depth image of the back of the target live pig.

[0074] In the embodiments of the present invention, the moving mechanism is connected to the supporting mechanism and can drive the supporting mechanism to move, so that the image acquisition device can adjust its position within a large space range, providing a greater shooting freedom for users. It can collect the RGB image and the depth image of any live pig as needed. The supporting mechanism includes a supporting base and a multi-axis robotic arm. The multi-axis robotic arm is formed by sequentially rotating and connecting a plurality of connecting rods, enabling the image sensor to achieve precise positioning and shooting in three-dimensional space. By adjusting the rotation angles of the respective connecting rods, the image sensor can be flexibly moved above the target live pig. By integrating the moving mechanism, the supporting mechanism, and the image sensor, the image acquisition device realizes the automation and high efficiency of the shooting process. Users only need to perform simple operations to quickly adjust the shooting position of the image sensor, which can significantly improve the shooting efficiency, better adapt to the dynamic and complex actual environment of the farm, reduce the manual intervention when collecting live pig images, better avoid the stress reaction of live pigs, improve the practicality of live pig weight estimation, provide a more accurate and efficient data basis for live pig weight estimation, and improve the practicality and popularization of live pig weight estimation.

[0075] The first electronic device 103 is configured to, when receiving the RGB image and the depth image of the target live pig, perform data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig, and input the depth image of the target live pig after data preprocessing into the live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model.

[0076] Among them, the live pig weight estimation model is constructed based on the improved EfficientNetV2 network and trained based on the depth images after preprocessing of the sample live pig data and the actual values of the weights of the sample live pigs; the depth images after preprocessing of the sample live pig data are obtained by performing data preprocessing on the depth images of the sample live pigs based on the RGB images of the sample live pigs; the method of performing data preprocessing on the depth images of the sample live pigs is the same as the method of performing data preprocessing on the depth images of the target live pigs.

[0077] Specifically, the first electronic device 103 in the embodiments of the present invention may be an electronic device such as a computer or a server.

[0078] Optionally, the first electronic device 103 may be configured to operate on the Ubuntu 18.04 system. The hardware configuration of the first electronic device 103 includes an Intel Xeon Platinum 8375C CPU and an NVIDIA RTX A6000 GPU, and the video memory capacity is 24GB.

[0079] When the first electronic device 103 receives the RGB image and the depth image of the target live pig, it may perform data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig through numerical calculation, mathematical statistics, deep learning, etc., to obtain the depth image after preprocessing of the target live pig data.

[0080] After obtaining the depth image after preprocessing of the target live pig data, the depth image after preprocessing of the target live pig data may be input into the live pig weight estimation model to obtain the estimated value of the weight of the target live pig output by the above live pig weight estimation model.

[0081] It should be noted that the actual values of the weights of the sample live pigs in the embodiments of the present invention may be obtained based on the following method: First, each sample live pig is weighed using an electronic weighing scale. After recording the weight value of the first measurement, the instrument is reset to zero and the second weighing is performed. Any sample live pig is weighed three times, and the average value of the three weights is taken as the actual value of the weight of the sample live pig, and the corresponding number is recorded.

[0082] The collection time is arranged between 7:00 and 8:00 every day. This time period is before the live pigs eat. The pig herd is active, almost all in a standing state, and usually gathers around the feed trough before the feed is supplied, which is convenient for data collection.

[0083] It can be understood that the sample live pigs can be determined based on the actual situation. In the embodiments of the present invention, live pigs with different weights in a certain pig farm may be determined as sample live pigs. Figure 3 It is the weight distribution diagram of the sample live pigs in the live pig weight estimation system provided by the present invention.

[0084] In the embodiments of the present invention, an image acquisition device 102 can be used to obtain RGB images and depth images of sample live pigs.

[0085] Specifically, an experimenter can control a moving mechanism 204 and / or a support mechanism through a first electronic device 103 or a second electronic device to drive an image sensor 203 to move above each sample live pig, and keep the shooting direction of the image acquisition device 102 perpendicular to the horizontal plane and downward.

[0086] To reduce duplicate frames, the image sensor 203 in the embodiments of the present invention can collect and save one frame every 3 seconds, and about 150 frames of RGB images and depth images are collected for each sample live pig. RGB images and depth images of about 10 sample live pigs are collected every day, and RGB images and depth images of 145 sample live pigs with different weights are collected in total, which can cover the weight changes during the whole life cycle of live pigs.

[0087] After obtaining the RGB images and depth images of the sample live pigs, images that do not contain sample live pigs or lack sample live pigs in the above images can be excluded.

[0088] It can be understood that there is a corresponding relationship between the actual value of the weight of the sample live pig and the RGB image and depth image of the sample live pig.

[0089] It should be noted that the training of the live pig weight estimation model in the embodiments of the present invention adopts the PyTorch deep learning framework, and the relevant environment includes CUDA 11.3, PyTorch 1.9.0 and Python 3.8. The depth images are all scaled to a resolution of 224×224. During the training process, random horizontal flipping, vertical flipping and rotation are also applied for data augmentation. Among them, in order to prevent excessive loss of the pig's back due to rotation, the random rotation angle is limited within 20° in the embodiments of the present invention. The parameter settings are as follows: the optimizer selects AdamW, the iteration period is set to 100 epochs, the learning rate is set to , and the weight decay is set to , and the cosine annealing (CosineAnnealingLR) learning rate adjustment strategy is used, and the batch size is 32.

[0090] During the training process of the live pig weight estimation model, the mean squared error (MSE) is used as the loss function. The MSE loss function guides the learning process of the model by measuring the squared error between the predicted value and the true value of the weight, and its formula is defined as: Among them, represents the weight value predicted by the model, represents the true weight value, is the number of samples. The main advantage of using MSE as the loss function is that it is more sensitive to larger errors, thus effectively punishing significant deviations of the model in predicting weight. This characteristic is particularly important for achieving accurate weight estimation because small errors in live pig weight prediction can accumulate into larger actual deviations.

[0091] In the embodiments of the present invention, four evaluation indicators are used to evaluate the trained live pig weight estimation model. The above four evaluation indicators are respectively: Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), and Coefficient of Determination ( ). MAE is used to measure the regression error and can reflect the actual error of the predicted value. The calculation formula of MAE is as follows: where, is the true value of the test image , and is the predicted value.

[0092] MAPE is used to measure the fitting degree of the model. The smaller the MAPE value, the better the fitting effect and the higher the accuracy of the prediction model. The calculation formula of MAPE is as follows: RMSE represents the sample standard deviation of the difference between the predicted value and the true value, and can be used to reflect the fluctuation degree of the weight measurement error. The calculation formula of RMSE is as follows: Coefficient of Determination ( ) is used to measure the fitting effect of the model. The closer the value is to 1, the better the fitting effect. The calculation formula of is as follows: is the average value of the true values of the test images.

[0093] In the pig weight estimation system according to the embodiments of the present invention, it includes an image acquisition device and a first electronic device. The image acquisition device includes a moving mechanism, a supporting mechanism, and an image sensor, which can use the image sensor to vertically capture the RGB image and depth image of the pig, and flexibly adjust the image acquisition range in combination with the moving mechanism, and can more accurately, efficiently, and flexibly acquire the image of the pig without causing stress response in the pig. After the first electronic device preprocesses the data of the depth image using the RGB image of the pig, it uses the improved EfficientNetV2 model to estimate the weight of the pig and obtains the estimated value of the pig weight. Through the optimized training of the model, it can effectively fuse spatial and three-dimensional information during pig weight estimation, significantly improve the accuracy of pig weight estimation, and the lightweight design of the model reduces the calculation amount and calculation complexity during pig weight estimation, can significantly improve the speed and efficiency of pig weight estimation, reduces manual intervention during pig weight estimation, can better avoid stress response in pigs, improves the practicability of pig weight estimation, can provide more reliable and efficient technical support for pig breeding management, and is applicable to the rapid automatic monitoring of pig weight in large-scale breeding scenarios.

[0094] Figure 4 It is a schematic flowchart of the first electronic device in the pig weight estimation system provided by the present invention for preprocessing the data of the depth image of the target pig. As an optional embodiment, the first electronic device 103 is specifically configured to perform image recognition and instance segmentation on the RGB image of the target pig, and after obtaining the mask of the target pig, when it is determined that the mask of the target pig is complete, map the mask of the target pig to the depth image of the target pig, and obtain the segmentation result of the target pig in the depth image of the target pig as the depth image after data preprocessing of the target pig.

[0095] Specifically, in the embodiments of the present invention, deep learning technology can be used to identify the target pig in the RGB image of the target pig and perform instance segmentation on the identified target pig to obtain the mask of the target pig.

[0096] After obtaining the mask of the target pig, the integrity of the mask of the target pig can be determined by means of conditional judgment. When it is determined that the mask of the target pig is complete, the mask of the target pig can be mapped to the depth image of the target pig to obtain the segmentation result of the target pig in the depth image of the target pig as the depth image after data preprocessing of the target pig.

[0097] As an optional embodiment, the first electronic device 103 is specifically configured to input the RGB image of the target live pig into the live pig recognition model to obtain the mask of the target live pig output by the live pig recognition model. The first electronic device 103 is further configured to map the mask of the target live pig into a blank image having the same size as the RGB image of the target live pig. If it is determined that the mask of the target live pig does not intersect the boundary of the blank image, or although the mask of the target live pig intersects the boundary, the distance between the mask of the target live pig and the boundary is less than the distance threshold, then it is determined that the mask of the target live pig is complete. The live pig recognition model is obtained by performing transfer learning on the training model based on the masks of the sample live pigs. The masks of the sample live pigs are obtained by performing image recognition and instance segmentation on the RGB images of the sample live pigs.

[0098] Specifically, since the RGB image has texture and color information and the accuracy of instance segmentation is higher than that of the depth image, in the embodiments of the present invention, actual segmentation is performed based on the RGB image.

[0099] In the embodiments of the present invention, the EISeg software can be used to label the sample live pigs in the RGB images of some sample live pigs as training samples. Furthermore, transfer learning can be performed on the general pre-trained model SAM2 based on the above training samples to obtain the live pig recognition model.

[0100] After inputting the RGB image of the target live pig into the above live pig recognition model, the mask of the target live pig in the RGB image of the target live pig output by the above live pig recognition model can be obtained.

[0101] After obtaining the mask of the target live pig in the RGB image of the target live pig, the mask of the target live pig can be mapped into a blank image having the same size as the RGB image of the target live pig.

[0102] If it is determined that the mask of the target live pig does not intersect the boundary of the above blank image, or although the mask of the target live pig intersects the boundary of the above blank image, the distance between the mask of the target live pig and the boundary of the above blank image is less than the distance threshold, then it can be determined that the mask of the target live pig is complete. Further, based on the correspondence between the RGB image of the target live pig and the depth image, the mask of the target live pig can be mapped into the depth image of the target live pig to obtain the segmentation result of the target live pig in the depth image of the target live pig, which is used as the depth image of the target live pig after data processing and used as the input of the live pig weight estimation model.

[0103] It should be noted that the distance threshold in the embodiments of the present invention can be determined based on prior knowledge and / or actual situations. For example, the value range of the distance threshold can be 13 to 17 pixels. The specific value of the distance threshold in the embodiments of the present invention is not limited.

[0104] It should be noted that after obtaining the above-mentioned live pig recognition model, based on the above-mentioned live pig recognition model, image recognition and instance segmentation can also be performed on the RGB images of the remaining unlabeled sample live pigs to obtain the masks of the sample live pigs in the RGB images of the remaining unlabeled sample live pigs.

[0105] After obtaining the masks of the sample live pigs in the RGB images of the sample live pigs, the masks of the sample live pigs can be mapped to a blank image with the same size as the RGB images of the sample live pigs.

[0106] If it is determined that the mask of the sample live pig does not intersect the boundary of the above blank image, or although the mask of the sample live pig intersects the boundary of the above blank image, the distance at which the mask of the sample live pig intercepts the boundary of the above blank image is less than the distance threshold, it can be determined that the mask of the sample live pig is complete. Further, based on the correspondence between the RGB image and the depth image of the sample live pig, the mask of the sample live pig can be mapped to the depth image of the sample live pig to obtain the segmentation result of the sample live pig in the depth image of the sample live pig, which is used as the depth image after processing the sample live pig data.

[0107] After obtaining the depth image after processing the sample live pig data, which has a relationship with the actual value of the weight of the sample live pig, the processed live pig images of the sample live pig data can be divided into a training set and a test set, which are respectively used for training and testing the live pig weight estimation model, so as to obtain a trained live pig weight estimation model. Among them, the sample live pigs in the test set are randomly selected and have a uniform weight distribution, and the sample live pigs in the training set are other live pigs except the sample live pigs in the test set.

[0108] When training the initial model constructed based on the improved EfficientNetV2 network, the depth image after processing the sample live pig data in the training set can be used as the training sample, and the actual value of the weight of the sample live pig corresponding to the depth image after processing the sample live pig data in the training set can be used as the sample label to train the above initial model, so as to obtain a trained live pig weight estimation model.

[0109] It should be noted that after obtaining the masks of the sample live pigs in the RGB images of the sample live pigs in the embodiments of the present invention, a comparative experiment can be performed on the trained live pig weight estimation model based on the masks of the sample live pigs in the RGB images of the sample live pigs and the segmentation results of the sample live pigs in the depth images of the sample live pigs, and further, the performance of the trained live pig weight estimation model can be evaluated.

[0110] In the embodiment of the present invention, the first electronic device performs instance segmentation on the RGB image of the target live pig by using the live pig recognition model obtained based on transfer learning. After generating the mask of the target live pig, by performing mask integrity judgment to ensure that the mask of the target live pig is truncated or occluded, and then mapping the mask of the target live pig to the depth image, the segmentation result of the target live pig in the depth image of the target live pig can be accurately obtained. As the depth image after preprocessing the target live pig data, it can effectively remove the background noise interference in the depth image of the target live pig, can realize the accurate alignment of the RGB image and the depth image of the target live pig, can significantly improve the reliability of preprocessing the data of the depth image of the target live pig, can reduce the error caused by incomplete image acquisition through mask integrity verification, and can enhance the data consistency through the spatial alignment of the RGB image and the depth image, providing high-precision input for the live pig weight estimation model, thereby improving the robustness and accuracy of the live pig weight estimation system, and being applicable to the automated monitoring requirements of large-scale breeding scenarios.

[0111] Figure 5 It is a schematic structural diagram of the live pig weight estimation model in the live pig weight estimation system provided by the present invention. As Figure 5 shown, as an optional embodiment, the live pig weight estimation model includes: a backbone network and a weight estimation head module connected in sequence; the backbone network is constructed based on the improved EfficientNetV2 network, and the improved EfficientNetV2 network is the EfficientNetV2 network in which the SE module is replaced by the CBAM module.

[0112] Specifically, after obtaining the depth image after preprocessing the target live pig data, each pixel point in the depth image after preprocessing the target live pig data represents the actual distance between the image sensor 203 and a certain point on the back of the target live pig, and can reflect the spatial structure information of the back of the target live pig. The trained live pig weight estimation model can extract the morphological and volume features of the target live pig from the depth image after preprocessing the target live pig data, and then establish a regression relationship between the above features and the weight of the target live pig to estimate the weight of the target live pig.

[0113] In order to meet the requirements of most deep learning networks for three-channel input, in the embodiment of the present invention, the single channel number of the depth image is copied into three copies. Considering the real-time and efficient requirements for live pig weight estimation in practical applications, the EfficientNetV2 network is selected to construct the backbone network of the live pig weight estimation model.

[0114] On this basis, in order to enhance the spatial information capture ability of the live pig weight estimation model, in the embodiments of the present invention, the original SE (Squeeze-and-Excitation) module in the EfficientNetV2 network is replaced with a CBAM (Convolutional Block Attention Module) module. For the morphological and volume features of the target live pig extracted by the backbone network, a weight estimation head module is designed to process the above features and estimate the weight of the target live pig.

[0115] During the training process, the live pig weight estimation model inputs the depth image preprocessed from the sample live pig data with corresponding relationships and the actual value of the sample live pig weight. The optimizer continuously adjusts the model parameters of the live pig weight estimation model to minimize the loss function, and finally obtains a trained live pig weight estimation model.

[0116] It should be noted that the EfficientNetV2 network is an efficient convolutional neural network. The EfficientNetV2 network has achieved excellent performance on multiple benchmark datasets by combining training-aware neural architecture search (NAS) and progressive learning strategies.

[0117] The SE module is a channel attention mechanism that dynamically adjusts the weights of each channel to make the network focus on more important feature channels. The CBAM module combines channel attention and spatial attention to enhance the feature expressions of important channels and spatial regions through a two-step attention mechanism.

[0118] It should be noted that the architecture of the EfficientNetV2 network includes multiple stages, and different modules are used in each stage. As Figure 5 shown, the backbone network in the live pig weight estimation model includes 8 stages, and different modules are used in each stage.

[0119] In the zero-th stage (Stage0), a convolutional calculation module (Conv 3×3) is set to perform convolutional calculation on the input image (Input) with a convolution kernel of 3×3 and a stride of 2 (Stride=2). The number of layers in the zero-th stage is 1 (Layer*1).

[0120] In the first stage (Stage1), a fused mobile inverted bottleneck convolution module (CBAM Fused-MBConv1) with a stride of 1 (Stride=1), an expansion rate of 1, and the SE module replaced by the CBAM module is set. The number of layers in the second stage is 2 (Layer*2).

[0121] The second stage (Stage2) is set with a stride of 2 (Stride = 2), an expansion rate of 4, and a fused mobile inverted bottleneck convolution module (CBAM Fused-MBConv4) that uses the CBAM module to replace the SE module. The number of layers in the third stage is 3 (Layer * 3).

[0122] The third stage (Stage3) is set with a stride of 2 (Stride = 2), an expansion rate of 4, and a fused mobile inverted bottleneck convolution module (CBAM Fused-MBConv4) that uses the CBAM module to replace the SE module. The number of layers in the fourth stage is 3 (Layer * 3).

[0123] The fourth stage (Stage4) is set with a stride of 2 (Stride = 2), an expansion rate of 4, and a mobile inverted bottleneck convolution module (CBAM MBConv4) that uses the CBAM module to replace the SE module. The number of layers in the fifth stage is 4 (Layer * 4).

[0124] The fifth stage (Stage5) is set with a stride of 1 (Stride = 1), an expansion rate of 6, and a mobile inverted bottleneck convolution module (CBAM MBConv6) that uses the CBAM module to replace the SE module. The number of layers in the fifth stage is 6 (Layer * 6).

[0125] The sixth stage (Stage6) is set with a stride of 2 (Stride = 2), an expansion rate of 6, and a mobile inverted bottleneck convolution module (CBAM MBConv6) that uses the CBAM module to replace the SE module. The number of layers in the sixth stage is 12 (Layer * 12).

[0126] The seventh stage (Stage7) is set with a convolution pooling fully connected calculation module (Conv 1×1, Pooling, and FC) for performing convolution calculation with a convolution kernel of 1×1, pooling calculation, and fully connected calculation on the input feature image in sequence. The number of layers in the seventh stage is 1 (Layer * 1).

[0127] It should be noted that MBConv represents Mobile Inverted Bottleneck Convolution, which is an efficient convolution module.

[0128] As Figure 5 shown, the fused MBConv module that uses the CBAM module to replace the SE module in the embodiments of the present invention may include a sequentially connected convolution calculation module (Conv 3×3), a CBAM module, a convolution calculation module (Conv 1×1), and a feature fusion module ( ). Among them, the above-mentioned feature fusion module is used to fuse the input of the convolutional calculation module (Conv 3×3) and the output of the convolutional calculation module (Conv 1×1).

[0129] As Figure 5 shown, the MBConv module that uses the CBAM module to replace the SE module in the embodiment of the present invention may include a sequentially connected convolutional calculation module (Conv 3×3), a depthwise separable convolutional calculation module (Depthwise Conv 1×1), a CBAM module, a convolutional calculation module (Conv 1×1), and a feature fusion module ( ). Among them, the above-mentioned feature fusion module is used to fuse the input of the convolutional calculation module (Conv 3×3) and the output of the convolutional calculation module (Conv 1×1).

[0130] In the embodiment of the present invention, the backbone network of the live pig weight estimation model is selected to be constructed based on the advanced EfficientNetV2 network, mainly because of the excellent performance of the EfficientNetV2 network in convolutional neural networks (CNNs) and its strong representation ability for fine-grained features. Compared with the EfficientNetV1 network, the EfficientNetV2 network performs better in resource-constrained environments and is very suitable as the backbone network of the live pig weight estimation model. Its design combines the mobile inverted bottleneck convolution (MBConv) module and the fused-mobile inverted bottleneck convolution module (Fused-MBConv), giving full play to the advantages of both in the network structure. The MBConv module expands and compresses channels through an inverted bottleneck design, balancing the expressive power and computational cost of the network; while the Fused-MBConv module further simplifies this process and improves performance through a fusion layer.

[0131] In order to further improve the ability of the trained live pig weight estimation model to capture spatial and channel features, in the embodiment of the present invention, the EfficientNetV2 network is improved by replacing the SE module in the EfficientNetV2 network with a CBAM module to enhance the attention of the live pig weight estimation model to the features of key regions and optimize the spatial information processing ability.

[0132] The SE module mainly models the importance of each channel through the channel attention mechanism. However, in depth images, spatial information is particularly important for pig weight estimation, and using only channel attention may not be sufficient to capture the key geometric features in depth images. Therefore, in the embodiments of the present invention, the CBAM mechanism is introduced into the EfficientNetV2 network. The CBAM module can, while retaining the channel attention mechanism, further incorporate a spatial attention mechanism, combine the global and local information of the input features, and model the spatial distribution of the feature map. This design enables the network to more accurately focus on important regions in depth images, such as the key geometric structures on the back of the pig.

[0133] Figure 6 is a comparison schematic diagram of the SE module and the CBAM module. As Figure 6 shown, the CBAM module takes the intermediate feature map as the input, and successively infers a one-dimensional channel attention map and a two-dimensional spatial attention map , as Figure 6 (b) shown. The entire attention processing process can be summarized as: Among them, represents the element-wise multiplication operation. During the multiplication process, the attention values will be broadcast (copied) as needed: the channel attention value is broadcast along the spatial dimension, while the spatial attention value is broadcast along the channel dimension. The finally obtained is the output feature map.

[0134] In the channel attention module, two different spatial context descriptors are generated: and , representing the average-pooled feature and the max-pooled feature respectively. These two descriptors are then input into a shared network to generate the channel attention map . After the shared network acts on each descriptor respectively, the output feature vectors are combined by element-wise summation. The calculation formula for channel attention is: Among them, represents the sigmoid function, , , where r is the scaling ratio. It should be noted that the weights and of the MLP are shared on the two inputs, and, Followed by the ReLU activation function.

[0135] In the spatial attention module, the spatial attention map is generated by leveraging the spatial relationships of features. Different from channel attention, the focus of spatial attention is on determining "where" the information-rich regions are, which complements channel attention. First, average pooling and max pooling operations are applied along the channel axis and concatenated to generate an efficient feature descriptor. On the concatenated feature descriptor, a standard convolutional layer is applied to generate the spatial attention map , which encodes the positions that need to be emphasized or suppressed. By aggregating the channel information of the feature map through the two pooling operations, two two-dimensional maps are generated: and , representing the average pooled feature and max pooled feature across channels, respectively. Then these features are concatenated and passed through a standard convolutional layer to generate a two-dimensional spatial attention map. The formula for spatial attention is: where, represents the sigmoid function, represents the convolutional operation with a convolutional kernel size of .

[0136] Figure 7 is a schematic structural diagram of the weight estimation head module in the pig weight estimation model of the pig weight estimation system provided by the present invention. As Figure 7 shown, as an optional embodiment, the weight estimation head module includes: a global average pooling unit, a first fully connected layer unit, a ReLU activation function unit, a regularization calculation unit, and a second fully connected layer unit connected in sequence.

[0137] Specifically, in order to further improve the calculation accuracy of the pig weight estimation model, a lightweight weight estimation head module (Weight estimation head) is designed in the embodiments of the present invention. The design purpose of the weight estimation head module is to further process the high-level features extracted by the backbone network in the pig weight estimation model and map them to the target space to complete the estimation of the pig weight.

[0138] The global average pooling unit (GAP) is used to perform an aggregation operation on the global spatial information of the feature map output by the backbone network. This operation effectively reduces the dimension of the feature map by calculating the average value of each channel in the spatial dimension, while retaining the global context information of the input depth image.

[0139] The first fully connected layer unit (FC) is used to map the global feature vector generated by the global average pooling unit into a high-dimensional latent space, further exploring the complex relationship between the input features and the target body weight. To introduce non-linearity, in the embodiment of the present invention, a ReLU activation function unit (ReLU) is arranged after the global average pooling unit, which is used to calculate the ReLU activation function for the output of the global average pooling unit, thereby enhancing the expression ability of the live pig body weight estimation model for non-linear features.

[0140] In addition, to enhance the generalization ability of the live pig body weight estimation model and alleviate the overfitting problem, a regularization calculation unit (Dropout) is arranged after the ReLU activation function unit in the body weight estimation head module, and the regularization effect is achieved by randomly discarding some neurons.

[0141] The second fully connected layer unit (FC) further maps the features after non-linear transformation and regularization into the final target space, and outputs the predicted value of the target live pig body weight.

[0142] Since the pooling layer plays an important role in feature learning and model generalization ability, in the embodiment of the present invention, two other weight estimation head variables are designed to explore the influence of the above factors. Figure 8 It is a schematic diagram of the structural comparison between the body weight estimation head module and two other weight estimation variants in the live pig body weight estimation model provided by the present invention. Figure 8 (a) is the body weight estimation head module in the live pig body weight estimation model provided by the present invention.

[0143] As Figure 8 As shown in (b), the first weight estimation variant is to replace the global average pooling unit with a global max pooling unit (GMP). The global max pooling unit emphasizes the local strongest response in the image by selecting the most significant features in each channel. The global max pooling unit focuses on capturing the most important regional information in feature extraction, which helps to enhance the model's attention to key pig back features.

[0144] As Figure 8 As shown in (c), the second weight estimation variant is to replace the global average pooling unit with a generalized mean pooling unit (GeM). The generalized mean pooling unit balances the advantages and disadvantages of average pooling and max pooling by adjusting the pooling exponent. The advantage is that it can automatically select the appropriate pooling method according to the task requirements, thereby effectively extracting features at different levels. In the task of body weight estimation, the generalized mean pooling unit can better adapt to the features of different image contents and enhance the model's learning ability for fine-grained features.

[0145] In the pig weight estimation model in the embodiments of the present invention, by replacing the SE module in the EfficientNetV2 network with the CBAM module, and then constructing a pig weight estimation model based on the improved EfficientNetV2 network, the ability of the pig weight estimation model to capture spatial and channel features can be enhanced, especially the attention to key geometric features in depth images, so as to achieve accurate estimation of pig weights in resource-constrained environments. By constructing a lightweight weight estimation head module, the computational amount and computational complexity of the model can be further reduced without reducing the computational accuracy of the model.

[0146] The pig weight estimation system 101 provided by the present invention can be used for pig weight estimation in the relatively limited space of a single pigsty. The image acquisition is mainly selected during the active period before the pigs eat, and at this time the pigs are usually located in the edge area of the pigsty. Under such conditions, the image acquisition device 102 in the pig weight estimation system 101 provided by the present invention can effectively locate directly above the pigs and complete data acquisition. The above-mentioned image acquisition device shows good feasibility in the pig breeding scenario, providing a reference and exploration direction for the future design of more flexible and wider coverage data acquisition devices.

[0147] Although the pig weight estimation model provided by the present invention is based on depth images, the present invention performs data preprocessing on the depth images based on the RGB images of the pigs. The results show that the effect of RGB image segmentation is significantly better than that of depth image segmentation, because the RGB images contain rich texture features, which helps to improve the accuracy of segmentation.

[0148] The pig weight estimation system 101 provided by the present invention includes a movable image acquisition device 102 and a first electronic device 103, which can be used for non-contact estimation of pig weights, avoiding the operation of driving pigs through traditional channels, significantly improving the acquisition efficiency and reducing the interference to the pigs. The proposed improved EfficientNetV2 lightweight model enhances the ability to capture spatial features of depth images by introducing the CBAM attention mechanism. The experimental results show that the pig weight estimation model provided by the present invention performs better than other methods in the weight estimation task, with the mean absolute error (MAE) being only 3.263 kg, and at the same time having low computational complexity and resource consumption. In addition, the performance of different weight estimation heads is also compared in detail, further verifying the superiority of the proposed method. The pig weight estimation system 101 provided by the present invention provides technical support and theoretical basis for non-contact weight estimation in the intelligent breeding scenario, has important practical significance, and helps to improve breeding efficiency, improve animal welfare and achieve refined management.

[0149] Based on the content of the above embodiments, the present invention also provides a live pig weight estimation method implemented based on the live pig weight estimation system 101 described above. Figure 9 It is a schematic flowchart of the live pig weight estimation method provided by the present invention. As Figure 9 shown, the method includes the following: Step 901, obtain the RGB image and depth image of the target live pig, where the RGB image and depth image of the target live pig are collected by an image sensor located above the target live pig; Step 902, perform data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig to obtain the depth image of the target live pig after data preprocessing; Step 903, input the depth image of the target live pig after data preprocessing into the live pig weight estimation model to obtain the estimated value of the weight of the target live pig output by the live pig weight estimation model; Among them, the live pig weight estimation model is constructed based on the improved EfficientNetV2 network and is trained based on the depth image of the sample live pig after data preprocessing and the actual value of the weight of the sample live pig; the depth image of the sample live pig after data preprocessing is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig.

[0150] It should be noted that the live pig weight estimation method provided by the present invention is implemented based on the above live pig weight estimation system 101. For the specific execution steps of the live pig weight estimation method provided by the present invention, reference may be made to the content of the above embodiments, and details will not be described in the embodiments of the present invention.

[0151] In the embodiments of the present invention, after performing data preprocessing on the depth image by using the RGB image of the live pig, the improved EfficientNetV2 model is used to estimate the weight of the live pig to obtain the estimated value of the weight of the live pig. Through the optimized training of the model, spatial and three-dimensional information can be effectively fused during the live pig weight estimation, which can significantly improve the accuracy of the live pig weight estimation. Moreover, the lightweight design of the model reduces the calculation amount and computational complexity during the live pig weight estimation, which can significantly improve the speed and efficiency of the live pig weight estimation. During the live pig weight estimation, manual intervention is reduced, which can provide more reliable and efficient technical support for the live pig breeding management and is applicable to the rapid and automated monitoring of the live pig weight in large-scale breeding scenarios.

[0152] Figure 10 It is a schematic structural diagram of the first electronic device provided by the present invention. As Figure 10As shown in the figure, the electronic device may include: a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040. Among them, the first electronic device 1031010, the communication interface 1020, and the memory 1030 complete communication with each other through the communication bus 1040. The processor 1010 may call the logical instructions in the memory 1030 to execute the live pig weight estimation method, which includes: obtaining the RGB image and depth image of the target live pig, where the RGB image and depth image of the target live pig are collected by an image sensor located above the target live pig; performing data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig to obtain the depth image of the target live pig after data preprocessing; inputting the depth image of the target live pig after data preprocessing into the live pig weight estimation model to obtain the estimated value of the weight of the target live pig output by the live pig weight estimation model; where the live pig weight estimation model is constructed based on the improved EfficientNetV2 network and is trained based on the depth image of the sample live pig after data preprocessing and the actual value of the weight of the sample live pig; the depth image of the sample live pig after data preprocessing is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the method of performing data preprocessing on the depth image of the sample live pig is the same as the method of performing data preprocessing on the depth image of the target live pig.

[0153] In addition, when the logical instructions in the above-mentioned memory 1030 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, may be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0154] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor 1010, the computer can execute the live pig weight estimation method provided by each of the above methods. The method includes: obtaining an RGB image and a depth image of a target live pig, where the RGB image and the depth image of the target live pig are collected by an image sensor located above the target live pig; performing data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig to obtain the depth image of the target live pig after data preprocessing; inputting the depth image of the target live pig after data preprocessing into a live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model; wherein, the live pig weight estimation model is constructed based on an improved EfficientNetV2 network and is trained based on the depth image of the sample live pig after data preprocessing and the actual value of the weight of the sample live pig; the depth image of the sample live pig after data preprocessing is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the way of performing data preprocessing on the depth image of the sample live pig is the same as the way of performing data preprocessing on the depth image of the target live pig.

[0155] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor 1010, it is implemented to execute the live pig weight estimation method provided by each of the above methods. The method includes: obtaining an RGB image and a depth image of a target live pig, where the RGB image and the depth image of the target live pig are collected by an image sensor located above the target live pig; performing data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig to obtain the depth image of the target live pig after data preprocessing; inputting the depth image of the target live pig after data preprocessing into a live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model; wherein, the live pig weight estimation model is constructed based on an improved EfficientNetV2 network and is trained based on the depth image of the sample live pig after data preprocessing and the actual value of the weight of the sample live pig; the depth image of the sample live pig after data preprocessing is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the way of performing data preprocessing on the depth image of the sample live pig is the same as the way of performing data preprocessing on the depth image of the target live pig.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0157] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A live pig weight estimation system, characterized in that, Including: An image acquisition device and a first electronic device; the image acquisition device is electrically connected to the first electronic device; The image acquisition device includes a moving mechanism, a support mechanism, and an image sensor; the moving mechanism is connected to the support mechanism, and the moving mechanism is used to drive the support mechanism to move; The image sensor is installed on the support mechanism, the shooting direction of the image sensor is vertically downward perpendicular to the horizontal plane, and the image sensor is used to acquire the RGB image and depth image of the target live pig located below the image sensor, and send the acquired RGB image and depth image of the target live pig to the first electronic device; The first electronic device is used to, when receiving the RGB image and depth image of the target live pig, perform data preprocessing on the depth image of the target live pig based on the RGB image of the target live pig, and input the preprocessed depth image of the target live pig into the live pig weight estimation model to obtain the estimated value of the weight of the target live pig output by the live pig weight estimation model; Wherein, the live pig weight estimation model is constructed based on the improved EfficientNetV2 network and is trained based on the preprocessed depth image of the sample live pig data and the actual value of the weight of the sample live pig; the preprocessed depth image of the sample live pig data is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the method of performing data preprocessing on the depth image of the sample live pig is the same as the method of performing data preprocessing on the depth image of the target live pig.

2. The live pig weight estimation system according to claim 1, wherein The support mechanism includes a support base and a multi-axis robotic arm. One end of the multi-axis robotic arm is fixed to the support base, and the other end of the multi-axis robotic arm is connected to the image sensor continuously; the multi-axis robotic arm includes a plurality of connecting rods, and each of the connecting rods is rotationally connected in sequence.

3. The live pig weight estimation system according to claim 1, characterized in that, The first electronic device is specifically used to perform image recognition and instance segmentation on the RGB image of the target live pig. After obtaining the mask of the target live pig, when it is determined that the mask of the target live pig is complete, map the mask of the target live pig to the depth image of the target live pig to obtain the segmentation result of the target live pig in the depth image of the target live pig as the preprocessed depth image of the target live pig data.

4. The live pig weight estimation system according to claim 1, characterized in that, The live pig weight estimation model includes: a backbone network and a weight estimation head module connected in sequence; the backbone network is constructed based on the improved EfficientNetV2 network, and the improved EfficientNetV2 network is the EfficientNetV2 network with the SE module replaced by the CBAM module.

5. The live pig weight estimation system according to claim 4, wherein, The weight estimation head module includes: a global average pooling unit, a first fully connected layer unit, a ReLU activation function unit, a regularization calculation unit, and a second fully connected layer unit connected in sequence.

6. The live pig weight estimation system according to claim 3, wherein The first electronic device is specifically configured to input the RGB image of the target live pig into a live pig recognition model to obtain the mask of the target live pig output by the live pig recognition model. The first electronic device is further configured to map the mask of the target live pig into a blank image having the same size as the RGB image of the target live pig. When it is determined that the mask of the target live pig does not intersect the boundary of the blank image, or although the mask of the target live pig intersects the boundary, the distance at which the mask of the target live pig intercepts the boundary is less than a distance threshold, it is determined that the mask of the target live pig is complete. The live pig recognition model is obtained by performing transfer learning on a training model based on the mask of the sample live pig, and the mask of the sample live pig is obtained by performing image recognition and instance segmentation on the RGB image of the sample live pig.

7. The live pig weight estimation system according to claim 2, characterized in that, The support base has a telescoping function and is capable of driving the multi-axis robotic arm to move in a direction perpendicular to the horizontal plane.

8. The live hog weight estimation system according to claim 7, wherein, The first electronic device is electrically connected to at least one of the support base, the multi-axis robotic arm, and the moving mechanism; when the first electronic device is electrically connected to the moving mechanism, it is configured to control the moving direction and / or the moving distance of the moving mechanism; when the first electronic device is electrically connected to the multi-axis robotic arm, it is configured to control the angle between at least two adjacent connecting rods in the multi-axis robotic arm; when the first electronic device is electrically connected to the support base, it is configured to control the height of the support base.

9. The live hog weight estimation system according to claim 7, wherein The image acquisition device further includes a second electronic device; the second electronic device is electrically connected to at least one of the support base, the multi-axis robotic arm, and the moving mechanism; when the second electronic device is electrically connected to the moving mechanism, it is configured to control the moving direction and / or the moving distance of the moving mechanism; when the second electronic device is electrically connected to the multi-axis robotic arm, it is configured to control the angle between at least two adjacent connecting rods in the multi-axis robotic arm; when the second electronic device is electrically connected to the support base, it is configured to control the height of the support base.

10. A pig weight estimation method implemented based on the pig weight estimation system according to any one of claims 1 to 9, characterized in that, Including: Obtain the RGB image and the depth image of the target live pig, where the RGB image and the depth image of the target live pig are collected by an image sensor located above the target live pig; Based on the RGB image of the target live pig, perform data preprocessing on the depth image of the target live pig to obtain the depth image of the target live pig after data preprocessing; Input the depth image of the target live pig after data preprocessing into a live pig weight estimation model to obtain an estimated value of the weight of the target live pig output by the live pig weight estimation model; Among them, the live pig weight estimation model is constructed based on the improved EfficientNetV2 network and is trained based on the depth image after preprocessing of the sample live pig data and the actual value of the weight of the sample live pig; the depth image after preprocessing of the sample live pig data is obtained by performing data preprocessing on the depth image of the sample live pig based on the RGB image of the sample live pig; the method of performing data preprocessing on the depth image of the sample live pig is the same as the method of performing data preprocessing on the depth image of the target live pig.

Images