High-density cluster feeding live pig body temperature monitoring method and system based on dual-light fusion

Abstract

The invention provides a body temperature monitoring method and system for high-density cluster-fed live pigs based on dual-light fusion, and belongs to the field of artificial feeding in animal husbandry. The method comprises the following steps: firstly, acquiring a thermal infrared image and a visible light image of high-density cluster-fed live pigs, and adjusting the resolution of the thermal infrared image in a dual-light image to obtain an extended thermal infrared image, so that temperature matrix data of the thermal infrared image and the visible light image have matched resolution; temperature calibration is carried out on the expanded thermal infrared image through black-body furnace image data, image registration is carried out on the calibrated thermal infrared image and the visible light image, instance segmentation is carried out on the registered image, the head of the live pig in the registered visible light image is marked, and the marking range comprises areas in front of and behind the ear roots of the live pig; and the body temperature of the group of health-preserving pigs is extracted by matching and registering the temperature of the region corresponding to the thermal infrared image. According to the invention, the body temperature of pigs is efficiently and continuously monitored, and early warning is provided for the outbreak of fever diseases of pigs.

Description

Technical Field

[0001] The present invention belongs to the field of animal husbandry artificial breeding, and in particular relates to a method and system for monitoring the body temperature of high-density cluster-raised pigs based on dual-light fusion. Background Art

[0002] As the population grows, the demand for food also grows dramatically. In order to meet people's material needs, large-scale farming is becoming increasingly important, and pork is a food that is more suitable for people's taste. In large-scale farming, pigs live and are raised in high-density clusters, which poses a greater risk of disease. When pigs are infected with diseases such as African swine fever, they will have typical symptoms of intermittent fever. Since the virus usually spreads quickly in the pig pen, certain measures need to be taken to monitor the body temperature of pigs in real time and detect abnormalities in time, so as to take countermeasures as soon as possible to reduce the economic losses of breeding companies.

[0003] In the prior art, a non-contact temperature measurement method using infrared thermal imaging is usually used to monitor the body temperature of group-raised pigs. The principle is to use a non-cooled infrared detector to capture the long-wave infrared radiation radiated by the animal's body surface, and then use thermal imaging to achieve non-contact temperature measurement of the animal's surface. Using infrared technology to monitor the body temperature of group-raised animals has the advantages of no stress, sustainability, and simple data acquisition. For example, some scholars use the OpenCV-python tool to monitor the body temperature of pigs in thermal imaging images, and some scholars extract thermal infrared images of specific parts of pigs based on YOLO v4 to achieve pig body temperature monitoring. However, the use of tools such as OpenCV-python to process thermal infrared images requires manual processing, and the real-time temperature measurement is poor; the YOLO algorithm is used to process thermal infrared images of specific parts of pigs, the amount of information obtained is low, the ability to extract local temperature of pigs is limited, and there is a lack of continuous monitoring capabilities; in addition, the pig recognition algorithm has poor robustness for high-density cluster-raised pigs, and the recognition accuracy is poor when monitoring the body temperature of group-raised pigs in an adhesion state in thermal infrared images. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, the present invention aims to provide a method and system for monitoring the body temperature of live pigs in high-density clusters based on dual-light fusion, which utilizes multi-source images to identify live pigs, combines thermal infrared images with visible light images, enhances the thermal infrared data through linear interpolation algorithms and instrument calibration, and introduces image registration and instance segmentation algorithms to improve the universality and accuracy of temperature measurement of different dual-light devices, thereby realizing automatic and continuous monitoring of the body temperature of groups of pigs using dual-light images, and improving the accuracy and real-time performance of pig temperature monitoring.

[0005] In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solution:

[0006] In a first aspect, an embodiment of the present invention provides a method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion, comprising the following steps:

[0007] Step S1, collecting bi-optical images of pigs raised in high-density groups, and using a temperature and humidity recorder to obtain indoor temperature and humidity; the bi-optical images include thermal infrared images and visible light images;

[0008] Step S2, adjusting the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image;

[0009] Step S3, collecting the black body furnace image data set by the camera in an open room of the same size, and performing temperature calibration on the extended thermal infrared image by using the black body furnace image data to obtain a calibrated thermal infrared image;

[0010] Step S4, performing image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image;

[0011] Step S5, performing instance segmentation on the registered image, marking the pig head in the registered visible light image, and the marked range includes the area before and after the pig's ear root; then, by matching the temperature of the corresponding area of ​​the registered thermal infrared image, the body temperature of the group-raised pigs is extracted.

[0012] As a preferred embodiment of the present invention, a dual-spectrum hemispherical camera is used as an image acquisition device; the thermal imaging sensor of the camera has a resolution of 160×120 pixels, a temperature measurement range of 30°C to 45°C, and a temperature measurement accuracy of ±0.5°C; the maximum resolution of the visible light sensor is 4 million pixels.

[0013] As a preferred embodiment of the present invention, step S2 uses a non-uniform interpolation algorithm to expand the acquired temperature matrix to match the visible light image, including the following steps:

[0014] The two-dimensional temperature matrix corresponding to the thermal infrared image is {z|z=(x,y)}. Linear interpolation is performed on each point z in the two-dimensional temperature matrix. During linear interpolation, an empty temperature matrix with a size of 640×480 is generated, and each point is (x′,y′). The resolution of the empty temperature matrix is ​​four times that of the original temperature matrix. (x′,y′) is regarded as a 1×2 array, where x′ is the first dimension and y′ is the second dimension. For each interpolation point (x,y) on the two-dimensional grid, the interpolation operation is performed point by point starting from the last point. In each dimension, the interpolation interval of (x′,y′) in each dimension is first determined, and its relative position t in the interval is calculated, and finally linear interpolation calculation is performed in the dimension. Assuming that z1 and z2 are two endpoints in the interpolation interval, the interpolation result z′ is calculated by formula (1):

[0015] z′= (1 – t)* z1 + t * z2 (1)

[0016] In formula (1), for the last dimension of the two-dimensional grid, z′ is the interpolation result at the interpolation point (x′, y′).

[0017] As a preferred embodiment of the present invention, when temperature calibration is performed in step S3, the calibration model of the thermal infrared image temperature data is as shown in formula (2):

[0018] T B =β0+β1D+β2H+β3T C (2)

[0019] In formula (2), β0 is the intercept of the calibration model; β1, β2 and β3 are the slope coefficients of the model respectively; D is the horizontal distance, H is the camera height, T C is the camera temperature reading, T B is the blackbody furnace temperature.

[0020] As a preferred embodiment of the present invention, the registration in step S4 specifically includes:

[0021] Step S41, using an arbitrary segmented model SAM to perform homogenization processing on the bi-optical image to obtain mask images respectively, and the masks of the thermal infrared image and the visible light image generated by SAM both contain the contour information of the live pig;

[0022] Step S42, performing feature point matching on the mask of the calibrated thermal infrared image and the mask of the visible light image.

[0023] As a preferred embodiment of the present invention, step S41 specifically includes:

[0024] The model ViT-H with the largest backbone size in SAM was used, and the operating parameters of SAM were fine-tuned. The parameter value for controlling the number of generated grid points was set to 1 to reduce the generation of tiny objects not related to pigs and reduce the processing complexity and time. The parameter for controlling the minimum area size of the generated mask was set to 4 to make the model pay more attention to the pig objects in the image, thereby improving the accuracy and stability of the segmentation results and obtaining a fine mask image.

[0025] As a preferred embodiment of the present invention, in step S42, the scale-invariant feature transform (SIFT) algorithm is used to perform feature point matching on the mask of the bi-optical image; specifically, the following steps are included:

[0026] When performing feature point matching, all mask images are first subjected to image perspective transformation, and then bi-light matching is performed; the SIFT algorithm includes scale space extreme value detection, key point positioning, direction assignment, feature description, and feature matching. After the homography calibration matrix is ​​calculated using the mask image feature matching points, the matrix is ​​applied to the visible light image to complete the calibration of the visible light image;

[0027] Among them, at least four matching points are required to calculate the homography calibration matrix. Wrong matching points will lead to unreliable calibration data. The mask of the bi-optical image is grayed out, and the parameters of the ratio test in the SIFT algorithm feature matching process are fine-tuned to achieve the appropriate number of matching points. According to the given matching points (x, y) in the visible light image mask and the points (x i ′,y i ′) The formula for calculating the homography matrix G is shown in formula (6):

[0028]

[0029] In formula (6), G is a 3×3 matrix, which is mapped to a plane in another image through perspective transformation, as shown in formula (7):

[0030]

[0031] For each pixel point (x, y) of the visible light image, the homography matrix G obtained by matching is used to calculate the pixel point (x i ″,y i ″), the calculation method is shown in formula (8):

[0032]

[0033] If there is no corresponding position, fill the position with black;

[0034] Then, based on each pixel point of the visible light image, the homography matrix G obtained by matching is used to calculate the pixel point of the corresponding thermal infrared image.

[0035] As a preferred embodiment of the present invention, in step S5, the registered image is instance segmented, and the instance segmentation model of YOLOv8-seg is adopted; when the pig head in the registered visible light image is annotated, a universal pixel point is adopted for annotation; the image annotation area is a mask of the pig head, and the pig head appearing in each image is annotated, and the pixel points of the annotated area are saved in a txt file, and the content is in YOLO format; a training data set is constructed using the annotated images, and model training is performed. After the training is completed, the model recognizes the pixel position of the corresponding area in the registered visible light image to obtain the pixel position of the pig head, and then matches this pixel area with the pixel area of ​​the thermal infrared image after "linear interpolation", and finds the highest temperature of the area as the temperature of the pig.

[0036] As a preferred embodiment of the present invention, the method further includes:

[0037] Step S6, evaluating the calibration result of the pig body temperature;

[0038] Step S7, evaluating the performance of the pig body temperature extraction model;

[0039] Step S8, evaluation of the results of all-weather extraction of the pig group's body temperature.

[0040] In a second aspect, an embodiment of the present invention further provides a high-density cluster-feeding pig temperature monitoring system based on dual-light fusion, the system comprising: a dual-light camera, a data acquisition module, a resolution adjustment module, a blackbody furnace, a temperature calibration module, an image registration module, an image segmentation module and a pig temperature extraction module; wherein,

[0041] The camera is used to capture thermal infrared images and visible light images;

[0042] The data acquisition module is used to compile a camera control program and collect bi-optical images of high-density cluster-raised pigs based on the camera's shooting, while using a temperature and humidity recorder to obtain indoor temperature and humidity;

[0043] The resolution adjustment module is used to adjust the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image;

[0044] The black body furnace is used as a temperature calibration element;

[0045] The temperature calibration module is used to collect black body furnace image data set by the camera in an open room of the same size, and perform temperature calibration on the extended thermal infrared image through the black body furnace image data to obtain a calibrated thermal infrared image;

[0046] The image registration module is used to perform image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image;

[0047] The image segmentation module is used to perform instance segmentation on the registered image and mark the head of the pig in the registered visible light image, and the marked range includes the front and rear areas of the pig's ear roots;

[0048] The pig temperature extraction module is used to match the segmented visible light image with the temperature of the corresponding area of ​​the thermal infrared image to extract the body temperature of the group-raised pigs.

[0049] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0050] The high-density cluster-raised pig temperature monitoring method and system based on dual-light fusion provided by the embodiment of the present invention uses a blackbody furnace to calibrate the thermal infrared image temperature measurement data, then performs dual-light image matching based on the SIFT algorithm, and then uses the YOLO v8-seg instance segmentation model to accurately identify the heads of group-raised pigs, and finally achieves the temperature measurement of group-raised pigs by matching the thermal infrared temperature of the corresponding area. The average accuracy of the model instance segmentation used for group-raised pig head monitoring by this method reached 98.06%, and the average absolute error of the temperature after calibration was ±0.35℃; the results of all-weather pig temperature monitoring show that the use of this method can not only efficiently and continuously monitor the body temperature of group pigs, but also provide early warning for the outbreak of febrile diseases in pigs.

[0051] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 This is a flow chart of the method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to an embodiment of the present invention;

[0054] Figure 2 It is a thermal infrared image and a selected area temperature matrix imaging effect diagram and an image generated by a temperature matrix in a specific application example of the present invention;

[0055] Figure 3 It is a mask image of a visible light image obtained by using SAM in a specific application example of the present invention;

[0056] Figure 4 is a mask image of a calibrated thermal infrared image obtained by using SAM in a specific application example of the present invention;

[0057] Figure 5 It is a SIFT matching and image conversion process and effect diagram in a specific application example of the present invention;

[0058] Figure 6 is the loss change in the model training process in a specific application example of the present invention;

[0059] Figure 7 This is the monitoring effect of the pig body temperature detection model in a specific application example of the present invention;

[0060] Figure 8 It is a curve of the maximum body temperature change of group-raised pigs in a single day in a specific application example of the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. It should be noted that the embodiments of the present invention and the features in the embodiments can also be combined with each other without conflict.

[0062] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present invention, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0063] In high-density cluster-raised pigs, the change in pig body temperature is usually closely related to their health status. Traditional manual observation cannot continuously and in real time monitor the body temperature of group-raised pigs, and the existing pig body temperature monitoring has many shortcomings. Based on this, the embodiment of the present invention provides a method and system for monitoring the body temperature of high-density cluster-raised pigs based on dual-light fusion. First, a dual-light camera is used to collect thermal infrared images and visible light images, and the expansion of thermal infrared temperature data is realized by spline linear interpolation algorithm. Then, a black body furnace is used to calibrate the thermal infrared data in combination with the equipment installation distance; secondly, the dual-light images are matched by image processing technology and scale-invariant feature conversion algorithm; finally, the YOLO v8-seg instance segmentation model is used to locate the pig head area in the visible light image, and then the highest temperature of the corresponding area is extracted to obtain the pig body temperature. The experimental results show that the determination coefficient of the temperature calibration model is 0.957, the average absolute error is ±0.35℃, and it can adapt to different camera installation positions. The mean average precision (MAP) of the instance segmentation of the YOLO v8-seg model is 98.6%, and the recall rate is 0.98. It achieves accurate positioning of the head area of ​​group-raised pigs, improves the robustness of the pig temperature monitoring results, and realizes real-time, automatic and accurate monitoring of the body temperature of group pigs.

[0064] like Figure 1 As shown, the method for monitoring body temperature of pigs raised in high-density clusters based on dual-light fusion described in this embodiment includes the following steps:

[0065] Step S1, collecting bi-optical images of pigs raised in high-density clusters, and using a temperature and humidity recorder to obtain indoor temperature and humidity; the bi-optical images include thermal infrared images and visible light images.

[0066] In this step, the high-density group feeding generally refers to feeding a number of pigs at the same time in a fixed space, for example, a feeding area or pig pen of 3.5 meters long, 2.7 meters wide and 2 meters high, feeding 7 male large white pigs of about 100 days old at the same time.

[0067] In order to collect the dual-light image of the pig, a dual-light camera is used for image acquisition, and thermal infrared images and visible light images are collected at the same time. Preferably, a consumer-grade dual-spectrum hemispherical camera is used as an image acquisition device. The thermal imaging sensor resolution of the camera is 160×120 pixels, the temperature measurement range is 30℃~45℃, and the temperature measurement accuracy is ±0.5℃; the maximum resolution of the visible light sensor is 4 million pixels. When performing image acquisition, the camera is fixed to the center of the top of the site, and its viewing angle is vertically downward, the field of view covers the entire ground, and the desktop computer is connected to the camera via the network to capture the dual-light image. Because the infrared detector of the camera is actually affected by environmental factors and produces large errors, which may cause misjudgment or omission of febrile diseases in pigs, this step selects a black body furnace to calibrate the thermal imaging temperature measurement data of the camera in an open room of the same size. The opening size of the black body furnace is 80mm×80mm, the effective emissivity is 0.97±0.02, the black body temperature range is 5℃~50℃, and the temperature accuracy is ±0.1℃. At the same time, in order to verify the monitoring effect of the present embodiment on the body temperature of live pigs, the temperature and humidity changes of the pig pen are observed synchronously, and the environmental factor data are recorded by an epitaxial probe type temperature and humidity recorder. The temperature and humidity recorder is placed at the edge of the site, and its epitaxial probe is placed above the activity position of the live pigs.

[0068] When a dual-light camera is used for data collection, what is collected is a time-synchronized thermal imaging temperature matrix and a visible light image. In order to visualize the collected thermal imaging temperature matrix as a thermal infrared image, a program is developed in the Python3.12 software environment based on the network SDK, and the collected temperature matrix data is visualized as a thermal infrared image using the Plotly package. In order to make the collected thermal imaging temperature closest to the surface temperature of the pig, the camera thermal imaging emissivity parameter is set to 0.97. According to the built-in field of view area matching matrix parameters of the dual-light camera, the dual-light data is cropped to be consistent with the field of view. Usually, the core temperature of normal pigs changes little in a short period of time, while the body temperature of pigs infected with febrile diseases will gradually increase within a day. Therefore, a set of thermal imaging temperature matrices and visible light images are captured at intervals of 2 minutes, and a current temperature and humidity information is recorded at the same time, and the collection continues for 24 hours.

[0069] Step S2, adjusting the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image.

[0070] In this step, the imaging resolution of the thermal infrared image is low, which makes it difficult to distinguish the pig features in the thermal infrared image. Therefore, the temperature matrix data is expanded to a resolution similar to that of the visible light image so that the acquired temperature matrix matches the visible light image. When expanding the matrix data, the non-uniform interpolation algorithm is used to expand the acquired temperature matrix to adapt to the irregularity of the temperature data, while making the temperature data transition smoother after linear interpolation.

[0071] Specifically, the obtained temperature matrix is ​​expanded using a non-uniform interpolation algorithm to match the visible light image, including the following steps:

[0072] The two-dimensional temperature matrix corresponding to the thermal infrared image is {z|z=(x,y)}. Linear interpolation is performed on each point z in the two-dimensional temperature matrix. The linear interpolation steps are as follows: Generate an empty temperature matrix of size 640×480, each point is (x′,y′), and the resolution of the empty temperature matrix is ​​four times that of the original temperature matrix. Treat (x′,y′) as a 1×2 array, where x′ is the first dimension and y′ is the second dimension. For each interpolation point (x,y) on the two-dimensional grid, start from the last point and perform interpolation operations point by point. In each dimension, first determine the interpolation interval of (x′,y′) in each dimension, and calculate its relative position t in the interval, and finally perform linear interpolation calculation in this dimension. Assuming that z1 and z2 are two endpoints in the interpolation interval, the interpolation result z′ is calculated by formula (1):

[0073] z′=(1–t)*z1+t*z2 (1)

[0074] In formula (1), for the last dimension of the two-dimensional grid, z′ is the interpolation result at the interpolation point (x′, y′). Figure 2 As shown, in a specific embodiment, the selected area temperature matrix imaging effect of the thermal infrared image A is as follows Figure 2 As shown in image B, after interpolation, the image generated by the temperature matrix is ​​as follows Figure 2 As shown in image C.

[0075] Step S3, collecting the image data of a black body furnace set by the camera in an open room of the same size, and performing temperature calibration on the extended thermal infrared image by using the image data of the black body furnace to obtain a calibrated thermal infrared image.

[0076] In this step, the infrared data obtained by the camera needs to be calibrated because the distance between the thermal infrared camera and the object to be measured will affect the temperature measurement results. A black body furnace is used indoors to simulate pigs, and a temperature and humidity recorder is used to obtain the indoor temperature and humidity. The thermal infrared temperature readings (T C). D is adjusted to 1m, 2m and 3m respectively; H is adjusted to 0m, 1m and 1.5m respectively; T C The center temperature of the blackbody furnace is selected with an accuracy of 0.01°C. The skin surface temperature of a normal pig is ~35°C. When a pig is infected with African swine fever or other diseases, the skin surface temperature will rise to ~41.1°C. To simulate the normal and abnormal states of a pig, the blackbody furnace temperature (T B ) from 35℃ to 41℃, with an adjustment range of 1℃. The data obtained by the thermal infrared camera was calibrated using the temperature-expanded blackbody temperature matrix data. A total of 63 sets of D, H, T C and T B Calibration data. Calculate T C and T B The difference between D, H and T is taken as the error value, and the calibration data with the largest and smallest errors are eliminated respectively. C is a dense matrix X, T B As the actual value y. The calibration data is randomly divided into training data (X train and train ) and test data (X test and test ), X train and train For training the calibration model, X test and test To test the model fitting performance, the calibration model of thermal infrared image temperature data is shown in formula (2):

[0077] T B =β0+β1D+β2H+β3T C (2)

[0078] In formula (2), β0 is the intercept of the calibration model; β1, β2 and β3 are the slope coefficients of the model respectively; D is the horizontal distance, H is the camera height, T C Taking temperature readings for the camera, small changes in these parameters will affect the calibration temperature.

[0079] Using X test and test When testing the model fitting performance, the mean square error (MSE), mean absolute error (MAE) and coefficient of determination (R 2 ) three evaluation indicators. MSE is used to describe the average deviation between the actual value ytest and the model predicted value y; MAE is used to measure the actual value y testThe average error between the predicted value y and the model prediction value y. This indicator is insensitive to outliers. Both MSE and MAE are used to measure the error between the predicted value and the actual value. The smaller the value, the better. The calculation methods of MSE and MAE are shown in formula (3) and formula (4) respectively:

[0080]

[0081]

[0082] In formulas (3) to (4), n is the number of test data sets, y test,i is the actual value corresponding to the i-th group of data, y i is the corresponding predicted result value.

[0083] R 2 It is used to measure the degree of fit of the model to the test data. The closer it is to 1, the better. Its calculation method is shown in formula (5):

[0084]

[0085] In formula (5), n is the number of test data sets, y test,i is the actual value corresponding to the i-th group of data, is the average of the actual values ​​corresponding to the data, y i is the corresponding predicted result value.

[0086] Step S4, performing image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image.

[0087] In this step, in order to extract the body temperature of the pig, it is first necessary to register and fuse the bi-optical images to determine the body part of the pig to extract the body temperature. Since the visible light image and the thermal infrared image provide different types of information, they need to be registered. The registration specifically includes:

[0088] In step S41, two different types of images are homogenized using a Segment Anything Model (SAM) to obtain mask images respectively, and the masks of the thermal infrared image and the visible light image generated using SAM both contain the contour information of the live pig.

[0089] In this step, SAM is a deep learning image segmentation model based on a large model. It can adapt to images of multiple imaging modalities and can be used to generate masks for all objects in the image without training before use. In order to obtain a fine mask image, the model ViT-H with the largest backbone size is used, and the operating parameters of SAM are fine-tuned. The parameter value for controlling the number of generated grid points (points_per_side) is set to 1 to reduce the generation of tiny objects not related to pigs and reduce the complexity and time of processing; the parameter for controlling the minimum area size of the generated mask (min_mask_region_area) is set to 4 to make the model pay more attention to the pig objects in the image, thereby improving the accuracy and stability of the segmentation results. The masks generated by SAM for thermal infrared images and visible light images contain the contour information of pigs. As Figure 3 FIG. 4 is a mask image of a visible light image obtained by using SAM in a specific application example of the present invention. Figure 4 is a mask image of a calibrated thermal infrared image obtained using SAM.

[0090] Step S42, performing feature point matching on the mask of the calibrated thermal infrared image and the mask of the visible light image.

[0091] In this step, due to the limitation of the physical characteristics of the camera sensor, the bi-optical data after field of view cropping still has the problem of image mismatch. In order to make the visible light area correspond to the thermal imaging area, the scale-invariant feature transform (SIFT) algorithm is used to match the feature points of the bi-optical image mask.

[0092] like Figure 5 As shown in the figure, when performing feature point matching, all mask images are first subjected to image perspective transformation, and then bi-light matching is performed. The SIFT algorithm includes scale space extreme value detection, key point positioning, direction assignment, feature description and feature matching, and is robust to images with scale, rotation, affine transformation and illumination changes. After calculating the homography calibration matrix using the mask image feature matching points, the matrix is ​​applied to the visible light image to complete the calibration of the visible light image. At least 4 matching points are required to calculate the homography calibration matrix, and incorrect matching points will result in unreliable calibration data. The mask of the bi-light image is gray-scaled, and the appropriate number of matching points is achieved by fine-tuning the ratio test parameter in the SIFT algorithm feature matching process to 0.4. According to the given matching point (x, y) in the visible light image mask and the point (x i ′,y i ′) The formula for calculating the homography matrix G is shown in formula (6):

[0093]

[0094] In formula (6), G is a 3×3 matrix that can be mapped to a plane in another image through perspective transformation, as shown in formula (7):

[0095]

[0096] For each pixel point (x, y) of the visible light image, the homography matrix G obtained by matching is used to calculate the pixel point (x i ″,y i ″), the calculation method is shown in formula (8):

[0097]

[0098] If there is no corresponding position, fill the position with black.

[0099] like Figure 5 As shown in Figure C, it is an effect diagram after feature point matching in a specific application example of the present invention.

[0100] Step S5, performing instance segmentation on the registered image, marking the pig head in the registered visible light image, and the marked range includes the area before and after the pig's ear root; then, by matching the temperature of the corresponding area of ​​the registered thermal infrared image, the body temperature of the group-raised pigs is extracted.

[0101] In this step, the registered image is instance segmented, and the instance segmentation model of YOLOv8-seg is used. When annotating the pig head in the registered visible light image, a universal pixel point is used for annotation, that is, mask-type annotation. The image annotation area is the mask of the pig's head. The pig's head that appears in each image is annotated, and the pixel points of the annotated area are saved in a txt file, and the content is in YOLO format. Use the annotated images to build a training data set and perform model training. After the training is completed, the model recognizes the pixel position of the corresponding area in the pig's head obtained by the registered visible light image, and then matches this pixel area with the pixel area of ​​the thermal infrared image after "linear interpolation" to find the highest temperature in the area as the temperature of the pig.

[0102] For example, in a specific temperature measurement application example, the developed program was used to randomly capture thermal imaging temperature matrices and visible light data in the test site, and 1,389 sets of data were obtained to construct a data set for the pig body temperature extraction model. In order to achieve the matching of the head area and the temperature matrix, 201 converted visible light images were manually annotated, and after augmenting the annotated images, a data set containing 483 images was constructed; during the model training process, the model parameters were set as follows: using the pre-trained weights of YOLOv8 "YOLOv8m-seg.pt", Batch size = 16, Epoch = 300, lr0 = 0.001, lrf = 0.001; the graphics card uses RTX 4090, and the graphics card memory is 24G. Using the trained YOLO v8-seg model, the head area of ​​group-raised pigs in the visible light image is first automatically located and registered, and then the body temperature of group-raised pigs is measured by matching the thermal infrared temperature of the corresponding area.

[0103] After completing the pig body temperature measurement, it is necessary to evaluate the measurement results accordingly. Therefore, the pig body temperature measurement method of this embodiment also includes:

[0104] Step S6, evaluating the calibration result of the pig's body temperature.

[0105] This step is described in detail through a specific application example. In a specific example of measuring the body temperature of a pig, 31 sets of data (X test and test ) is used to test the fitted temperature calibration model. The comparison results of the predicted temperatures at different actual temperatures (blackbody furnace) are shown in Table 1. The R2, MSE, MAE and minimum absolute error of the predicted value of the temperature calibration model in Table 1 are 0.957, 0.167 and 0.352, respectively. The minimum absolute error of the predicted value is 0.03°C. These evaluation indicators show that the model has a good fitting effect and can further reduce the temperature measurement error of the dual-light camera. It can be used to monitor the body temperature of pigs. In addition, since these 31 sets of data are obtained under the conditions of changes in the camera installation height and the horizontal distance between the camera and the object being measured, the model has a certain robustness in dealing with changes in the camera installation position in actual scenes.

[0106] Table 1 Comparison between the predicted temperature of the model and the actual temperature (blackbody furnace)

[0107]

[0108]

[0109] Step S7, evaluating the performance of the pig body temperature extraction model.

[0110] The YOLO v8m-seg algorithm was trained and tested on the NVIDIV GeForce RXT 4090 GPU. The key parameters of the model training were adjusted, and the batch size (Batchsize) was set to 32, the number of iterations (Epoch) was set to 300, and the learning rate (Learning Rate) was set to 0.0001. The segmentation loss (Loss) of the model training process is as follows Figure 6 As shown in the figure, the curve shows a downward trend, indicating that the model fits the data set well. The mean average precision (mAP) of the model instance segmentation is 98.06%, the recall rate (Recall) is 0.98, and the mAP50-95 performance of target detection reaches 82.9%, indicating that the model can be used to effectively perform instance segmentation on the pig head area.

[0111] The application effect of the trained YOLO v8m-seg model in group-rearing pig detection is as follows: Figure 7 As shown in the figure, it shows that it can accurately segment the head area of ​​each pig. According to the segmented area, the maximum value in the thermal infrared temperature matrix corresponding to each pig is extracted, and after combining the camera installation position parameters, it is sent to the calibration model to obtain the calibrated temperature as the monitored pig body temperature. Figure 7 It can be seen that when there are multiple pigs in the image, the model can also accurately extract the pig ear root area.

[0112] Step S8, evaluation of the results of all-weather extraction of the pig group's body temperature.

[0113] In order to verify the effect of the method in this paper on extracting body temperature over a long period of time, the collected all-weather dual-light data were tested to count the body temperature of group-raised pigs. Due to the large number of group-raised pigs and the relatively stable ambient temperature and humidity observed through the temperature and humidity recorder, the highest body temperature value among the group-raised pigs was selected as the statistical result to facilitate early warning in practical applications. The model missed the detection of pigs, resulting in the missing of a single time point in the data, and the temperature measurement data was supplemented manually based on thermal infrared images. The all-weather group-raised pig data collection time started at 16:43 on May 11, 2024, and the data lasted for about 24 hours. The body temperature measurement results are statistically shown as follows: Figure 8 As shown. Figure 8 It can be seen that the body temperature of live pigs is lower at night and rises in the afternoon.

[0114] Based on the same idea, the embodiment of the present invention also provides a high-density cluster feeding pig body temperature monitoring system based on dual-light fusion, the system includes: a dual-light camera, a data acquisition module, a resolution adjustment module, a black body furnace, a temperature calibration module, an image registration module, an image segmentation module and a pig temperature extraction module; wherein,

[0115] The camera is used to capture thermal infrared images and visible light images;

[0116] The data acquisition module is used to compile a camera control program and collect bi-optical images of high-density cluster-raised pigs based on the camera's shooting, while using a temperature and humidity recorder to obtain indoor temperature and humidity;

[0117] The resolution adjustment module is used to adjust the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image;

[0118] The black body furnace is used as a temperature calibration element;

[0119] The temperature calibration module is used to collect black body furnace image data set by the camera in an open room of the same size, and perform temperature calibration on the extended thermal infrared image through the black body furnace image data to obtain a calibrated thermal infrared image;

[0120] The image registration module is used to perform image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image;

[0121] The image segmentation module is used to perform instance segmentation on the registered image and mark the head of the pig in the registered visible light image, and the marked range includes the front and rear areas of the pig's ear roots;

[0122] The pig temperature extraction module is used to match the segmented visible light image with the temperature of the corresponding area of ​​the thermal infrared image to extract the body temperature of the group-raised pigs.

[0123] In this embodiment, each module is implemented by a processor, and a memory is appropriately added when storage is required. Among them, the processor can be but is not limited to a microprocessor MPU, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0124] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0125] It should also be noted that the high-density cluster-raised pig temperature monitoring system based on dual-light fusion described in this embodiment corresponds to the high-density cluster-raised pig temperature monitoring method based on dual-light fusion, and the description and limitation of the method are also applicable to the system, which will not be repeated here.

[0126] It can be seen from the above technical solutions that the high-density cluster-raised pig body temperature monitoring method and system based on dual-light fusion provided by the embodiment of the present invention uses a blackbody furnace to calibrate the thermal infrared image temperature measurement data, and then uses the SIFT algorithm to match the dual-light images, and then uses the YOLO v8-seg instance segmentation model to accurately identify the heads of group-raised pigs, and finally achieves the temperature measurement of group-raised pigs by matching the thermal infrared temperature of the corresponding area. The accuracy of the model instance segmentation used for group-raised pig head monitoring by this method reaches an average of 98.06%, and the average absolute error of the temperature after calibration is ±0.35℃; the results of all-weather pig body temperature monitoring show that the use of this method can not only efficiently and continuously monitor the body temperature of group pigs, but also provide early warning for the outbreak of febrile diseases in pigs.

[0127] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. It is not intended to limit the scope of the invention claimed for protection, but only represents the preferred embodiment of the present invention. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

Claims

1. A method for monitoring body temperature of high-density clustered pigs based on dual-light fusion, characterized in that: The steps include: Step S1, collecting bi-optical images of pigs raised in high-density groups, and using a temperature and humidity recorder to obtain indoor temperature and humidity; the bi-optical images include thermal infrared images and visible light images; Step S2, adjusting the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image; Step S3, collecting the black body furnace image data set by the camera in an open room of the same size, and performing temperature calibration on the extended thermal infrared image by using the black body furnace image data to obtain a calibrated thermal infrared image; Step S4, performing image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image; Step S5, performing instance segmentation on the registered image, marking the pig head in the registered visible light image, and the marked range includes the area before and after the pig's ear root; then, by matching the temperature of the corresponding area of ​​the registered thermal infrared image, the body temperature of the group-raised pigs is extracted.

2. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 1 is characterized in that: A dual-spectrum hemispherical camera is used as the image acquisition device; the thermal imaging sensor of the camera has a resolution of 160×120 pixels, a temperature measurement range of 30℃~45℃, and a temperature measurement accuracy of ±0.5℃; the maximum resolution of the visible light sensor is 4 million pixels.

3. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 1 is characterized in that: Step S2 uses a non-uniform interpolation algorithm to expand the acquired temperature matrix to match the visible light image, including the following steps: The two-dimensional temperature matrix corresponding to the thermal infrared image is {z|z=(x,y)}. Linear interpolation is performed on each point z in the two-dimensional temperature matrix. During linear interpolation, an empty temperature matrix with a size of 640×480 is generated. ′ ,y ′ ), the resolution of the empty temperature matrix is ​​four times that of the original temperature matrix; ′ ,y ′ ) is considered as a 1×2 array, where x ′ is the first dimension, y ′ is the second dimension; for each interpolation point (x, y) on the two-dimensional grid, interpolation is performed point by point starting from the last point; in each dimension, first determine (x ′ ,y ′ ) in each dimension, and calculate its relative position t in the interval, and finally perform linear interpolation calculation in the dimension; assuming that z1 and z2 are the two endpoints in the interpolation interval, the interpolation result z ′ Calculated by formula (1): z ′ = (1 – t)* z1 + t * z2 (1) In formula (1), for the last dimension of the two-dimensional grid, z ′ At the interpolation point (x ′ ,y ′ ) is the interpolation result at .

4. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 1 is characterized in that: When temperature calibration is performed in step S3, the calibration model of the thermal infrared image temperature data is shown in formula (2): T B =β0+β1D+β2H+β3T C (2) In formula (2), β0 is the intercept of the calibration model; β1, β2 and β3 are the slope coefficients of the model respectively; D is the horizontal distance, H is the camera height, T C is the camera temperature reading, T B is the blackbody furnace temperature.

5. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 1, characterized in that: The registration in step S4 specifically includes: Step S41, using an arbitrary segmented model SAM to perform homogenization processing on the bi-optical image to obtain mask images respectively, and the masks of the thermal infrared image and the visible light image generated by SAM both contain the contour information of the live pig; Step S42, performing feature point matching on the mask of the calibrated thermal infrared image and the mask of the visible light image.

6. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 5 is characterized in that: Step S41 specifically includes: The model ViT-H with the largest backbone size in SAM was used, and the operating parameters of SAM were fine-tuned. The parameter value for controlling the number of generated grid points was set to 1 to reduce the generation of tiny objects not related to pigs and reduce the processing complexity and time. The parameter for controlling the minimum area size of the generated mask was set to 4 to make the model pay more attention to the pig objects in the image, thereby improving the accuracy and stability of the segmentation results and obtaining a fine mask image.

7. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 5, characterized in that: In step S42, the scale-invariant feature transform (SIFT) algorithm is used to match feature points of the mask of the bi-optical image; specifically, the following steps are performed: When performing feature point matching, all mask images are first subjected to image perspective transformation, and then bi-light matching is performed; the SIFT algorithm includes scale space extreme value detection, key point positioning, direction assignment, feature description, and feature matching. After the homography calibration matrix is ​​calculated using the mask image feature matching points, the matrix is ​​applied to the visible light image to complete the calibration of the visible light image; Among them, at least four matching points are required to calculate the homography calibration matrix. Wrong matching points will lead to unreliable calibration data. The mask of the bi-optical image is grayed out, and the parameters of the ratio test in the feature matching process of the SIFT algorithm are fine-tuned to achieve the appropriate number of matching points. According to the given matching points (x, y) in the visible light image mask and the points (x′ i ,y′ i ) The formula for calculating the homography matrix G is shown in formula (6): In formula (6), G is a 3×3 matrix, which is mapped to a plane in another image through perspective transformation, as shown in formula (7): For each pixel point (x, y) of the visible light image, the homography matrix G obtained by matching is used to calculate the corresponding pixel point (x″) of the thermal infrared image i ,y″ i ), the calculation method is shown in formula (8): If there is no corresponding position, fill the position with black; Then, based on each pixel point of the visible light image, the homography matrix G obtained by matching is used to calculate the pixel point of the corresponding thermal infrared image.

8. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to claim 1, characterized in that: In step S5, the registered image is instance segmented using the instance segmentation model of YOLOv8-seg; when the pig head in the registered visible light image is annotated, universal pixel points are used for annotation; the image annotation area is the mask of the pig head, and the pig head appearing in each image is annotated, and the pixel points of the annotated area are saved in a txt file, and the content is in YOLO format; the annotated images are used to construct a training data set, and model training is performed. After the training is completed, the model recognizes the pixel position of the corresponding area in the pig head obtained by the registered visible light image, and then matches this pixel area with the pixel area of ​​the thermal infrared image after "linear interpolation" to find the highest temperature in the area as the temperature of the pig.

9. The method for monitoring body temperature of high-density cluster-raised pigs based on dual-light fusion according to any one of claims 1 to 8, characterized in that: The method further comprises: Step S6, evaluating the calibration result of the pig body temperature; Step S7, evaluating the performance of the pig body temperature extraction model; Step S8, evaluation of the results of all-weather extraction of the pig group's body temperature.

10. A high-density cluster-feeding pig temperature monitoring system based on dual-light fusion, characterized in that: The system includes: a dual-light camera, a data acquisition module, a resolution adjustment module, a black body furnace, a temperature calibration module, an image registration module, an image segmentation module and a pig temperature extraction module; wherein, The camera is used to capture thermal infrared images and visible light images; The data acquisition module is used to compile a camera control program, and to collect bi-optical images of high-density cluster-raised pigs based on the camera's shooting, while using a temperature and humidity recorder to obtain indoor temperature and humidity; The resolution adjustment module is used to adjust the resolution of the thermal infrared image in the dual light image to obtain an extended thermal infrared image so that the temperature matrix data of the thermal infrared image has a matching resolution with the visible light image; The black body furnace is used as a temperature calibration element; The temperature calibration module is used to collect black body furnace image data set by the camera in an open room of the same size, and perform temperature calibration on the extended thermal infrared image through the black body furnace image data to obtain a calibrated thermal infrared image; The image registration module is used to perform image registration on the calibrated thermal infrared image and the visible light image to obtain a registered visible light image and a registered thermal infrared image; The image segmentation module is used to perform instance segmentation on the registered image and mark the head of the pig in the registered visible light image, and the marked range includes the front and rear areas of the pig's ear roots; The pig temperature extraction module is used to match the segmented visible light image with the temperature of the corresponding area of ​​the thermal infrared image to extract the body temperature of the group-raised pigs.

Images