Live pig abnormity monitoring and early warning method, device and equipment and storage medium

Abstract

The invention provides a live pig abnormity monitoring and early warning method, device and equipment and a storage medium, and the method comprises the steps: obtaining infrared image data, air temperature data, air humidity data and visible light image data of a live pig breeding site; obtaining live pig individuals and behavior types corresponding to the live pig individuals in each frame of visible light image data based on the visible light image data; target tracking is carried out on the visible light image data, and the duration proportion of different behavior types of each live pig individual is obtained in combination with the behavior types corresponding to the live pig individuals; obtaining the highest temperature in the infrared matrix corresponding to the live pig individuals as a body temperature measurement value corresponding to each live pig individual; correcting the body temperature measured value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each live pig individual; and performing abnormity monitoring based on the duration proportion and the actual individual body temperature. According to the scheme, more accurate abnormity monitoring can be carried out on the live pigs.

Description

Technical Field

[0001] The present application relates to the field of animal husbandry phenotypic identification and detection technology, and in particular to a method, device, equipment and storage medium for abnormal monitoring and early warning of pigs. Background Art

[0002] In the related technologies, the methods for abnormal pig monitoring based on artificial intelligence are mostly single behavior monitoring, which is affected by equipment performance and ambient light, and has low monitoring accuracy. Moreover, the single behavior monitoring method cannot adapt well to various pig breeding environments and has weak generalization ability. Summary of the invention

[0003] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0004] In a first aspect, the present application proposes a method for monitoring and warning of abnormal pigs, the method comprising: obtaining multiple sets of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period; performing target detection and behavior recognition based on the visible light image data to obtain individual pigs in each frame of the visible light image data and the behavior type corresponding to the individual pigs; based on the Euclidean distance of the detection frames in the visible light image data of adjacent frames, the visible light image data is tracked by combining the Kalman filter algorithm, and combined with the behavior type corresponding to the individual pigs, the duration ratio of different behavior types of each individual pig is obtained; performing image segmentation on the infrared image data to obtain the infrared matrix corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig; correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig; and performing abnormal monitoring on the individual pigs based on the duration ratio and the actual individual body temperature of each individual pig.

[0005] In one implementation, the step of performing target detection and behavior recognition based on the visible light image data to obtain individual pigs in each frame of the visible light image data and the behavior types corresponding to the individual pigs is performed by calling a pre-trained YOLO model; wherein the YOLO model is a Bi-PAN-FPN structure, and uses an Effiententv2 network as a backbone network, the MBConv module of the Effiententv2 network contains a Resnet link, the neck network in the YOLO model includes a convolutional attention CBAM module, and the YOLO model is trained using a WIOU loss function.

[0006] In one implementation, the target tracking of the visible light image data is performed based on the Euclidean distance of the detection frames in the visible light image data of adjacent frames in combination with the Kalman filter algorithm, and the duration ratio of different behavior types of each individual pig is obtained in combination with the behavior type corresponding to the individual pig, including: obtaining the detection frame corresponding to the individual pig in each frame of the visible light image data; performing target tracking based on the Euclidean distance between the center positions of the detection frames to obtain an initial target tracking result; correcting the initial target tracking result using the Kalman filter algorithm to obtain a target tracking result; determining the position information of the same individual pig in each frame of the visible light image data based on the target tracking result; obtaining the behavior type of each individual pig in each frame of the visible light image data based on the position information of the individual pig in each frame of the visible light image data and the behavior type corresponding to the individual pig in each frame of the visible light image data; performing statistical analysis based on the behavior type of each individual pig in each frame of the visible light image data to obtain the duration ratio of different behavior types of each individual pig.

[0007] In one implementation, the step of performing image segmentation on the infrared image data, obtaining the infrared matrix corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig is performed by calling a pre-trained OneFormer model; wherein the OneFormer model is trained using a weighted sum as a loss function, and the loss function can be expressed as follows:

[0008]

[0009]

[0010]

[0011] in, is the loss function, is the object-to-text contrast loss, is the contrastive loss from text to object, is the binary cross entropy on category prediction, is the standard classification loss for class prediction, is the binary cross entropy on the mask predictions, is the dice loss on mask prediction, is the weight of the dice loss, is the dice loss, is the number of target-text query objects, Indicates For target queries, Indicates For text queries, is the learnable temperature parameter, Represents the exclusive OR operator.

[0012] In one implementation, the step of correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each of the individual pigs is performed by calling a pre-trained long short-term memory LSTM network, the input data of the LSTM network is the air temperature data, the air humidity data and the infrared temperature measurement value, the output data of the LSTM network is the actual individual body temperature, and the LSTM network is trained using the mean square error as the loss function.

[0013] In one implementation, the individual pigs are monitored for abnormalities based on the time ratio and the actual individual body temperature of each individual pig, including: obtaining the growth period of each individual pig; obtaining the activity type proportion threshold of each individual pig based on the growth period of each individual pig; determining whether the behavior of the individual pig is abnormal based on the time ratio and the activity type proportion threshold range of each individual pig; determining whether the body temperature of the individual pig is abnormal based on the actual individual body temperature corresponding to the individual pig and a preset body temperature threshold range.

[0014] In the second aspect, the present application proposes a pig abnormality monitoring and early warning device, which includes: an acquisition module for acquiring multiple groups of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period; a first processing module for performing target detection and behavior recognition based on the visible light image data, and obtaining individual pigs in each frame of the visible light image data and the behavior types corresponding to the individual pigs; a second processing module for tracking targets in the visible light image data based on the Euclidean distance of detection boxes in the visible light image data of adjacent frames in combination with a Kalman filtering algorithm , and combined with the behavior type corresponding to the individual pig, obtain the duration ratio of different behavior types of each individual pig; a third processing module, used to perform image segmentation on the infrared image data, obtain the infrared matrix corresponding to the individual pig, and obtain the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig; a fourth processing module, used to correct the body temperature measurement value based on the air temperature data and the air humidity data, and obtain the actual individual body temperature of each individual pig; a monitoring module, used to perform abnormal monitoring on the individual pig based on the duration ratio and the actual individual body temperature of each individual pig.

[0015] In one implementation, the first processing module can perform the step of performing target detection and behavior recognition based on the visible light image data to obtain the individual pigs in each frame of the visible light image data and the behavior types corresponding to the individual pigs by calling a pre-trained YOLO model; wherein the YOLO model is a Bi-PAN-FPN structure, and uses the Effiententv2 network as the backbone network, the MBConv module of the Effiententv2 network contains a Resnet link, the neck network in the YOLO model includes a convolutional attention CBAM module, and the YOLO model is trained using the WIOU loss function.

[0016] In one implementation, the second processing module can be used to: obtain a detection frame corresponding to the individual pig in each frame of the visible light image data; perform target tracking based on the Euclidean distance between the center positions of the detection frames to obtain an initial target tracking result; use a Kalman filter algorithm to correct the initial target tracking result to obtain a target tracking result; determine the position information of the same individual pig in each frame of the visible light image data based on the target tracking result; obtain the behavior type of each individual pig in each frame of the visible light image data based on the position information of the individual pig in each frame of the visible light image data and the behavior type corresponding to the individual pig in each frame of the visible light image data; perform statistical analysis based on the behavior type of each individual pig in each frame of the visible light image data to obtain the duration ratio of different behavior types of each individual pig.

[0017] In one implementation, the third processing module can perform the steps of performing image segmentation on the infrared image data, obtaining the infrared matrix corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig by calling the pre-trained OneFormer model; wherein the model is trained using the weighted sum as the loss function, and the loss function can be expressed as follows:

[0018]

[0019]

[0020]

[0021] in, is the loss function, is the object-to-text contrast loss, is the contrastive loss from text to object, is the binary cross entropy on category prediction, is the standard classification loss for class prediction, is the binary cross entropy on the mask predictions, is the dice loss on mask prediction, is the weight of the dice loss, is the dice loss, is the number of target-text query objects, Indicates For target queries, Indicates For text queries, is the learnable temperature parameter, Represents the exclusive OR operator.

[0022] In one implementation, the fourth processing module calls a pre-trained long short-term memory (LSTM) network to execute the step of correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each of the live pigs. The input data of the LSTM network are the air temperature data, the air humidity data and the infrared temperature measurement value, and the output data of the LSTM network is the actual individual body temperature. The LSTM network is trained using the mean square error as the loss function.

[0023] In one implementation, the monitoring module can be used to obtain the growth period of each of the individual pigs; based on the growth period of each of the individual pigs, obtain the activity type ratio threshold of each of the individual pigs; based on the duration ratio of each of the individual pigs and the activity type ratio threshold range, determine whether the behavior of the individual pigs is abnormal; based on the actual individual body temperature corresponding to the individual pigs and the preset body temperature threshold range, determine whether the body temperature of the individual pigs is abnormal.

[0024] In a third aspect, the present application proposes an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the pig abnormality monitoring and early warning method as described in the first aspect.

[0025] In a fourth aspect, the present application proposes a computer-readable storage medium for storing instructions, which, when executed, enables the method described in the first aspect to be implemented.

[0026] In a fifth aspect, the present application proposes a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the pig abnormality monitoring and early warning method as described in the first aspect.

[0027] The pig abnormality monitoring and early warning method, device, equipment and storage medium provided in the present application can perform target detection and behavior recognition based on multiple frames of visible light image data of the pig breeding site within the second preset time period, and based on the Euclidean distance between individual pigs, the visible light image data is combined with the Kalman filter algorithm to track the target, so as to obtain the duration ratio of different behavior types of each individual pig, and at the same time obtain the corresponding body temperature measurement value of the individual pig based on the infrared image data, and correct the body temperature measurement value in combination with the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig, so as to perform abnormal monitoring on the individual pig based on the duration ratio and the actual individual body temperature of each individual pig. More accurate abnormal monitoring of individual pigs in the breeding site can be performed.

[0028] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which: Figure 1 It is a flow chart of a method for abnormal monitoring and early warning of pigs provided in an embodiment of the present application; Figure 2 It is a flow chart of another method for abnormal monitoring and early warning of pigs provided in an embodiment of the present application; Figure 3 This is a flow chart of another method for abnormal pig monitoring and early warning provided by an embodiment of the present application; Figure 4 It is a schematic diagram of a pig behavior monitoring process provided in an embodiment of the present application; Figure 5 It is a structural diagram of a YOLO model provided in an embodiment of the present application; Figure 6 It is a flow chart of a pig abnormal behavior identification solution provided in an embodiment of the present application; Figure 7 It is a structural schematic diagram of a pig abnormality monitoring and early warning device provided in an embodiment of the present application; Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0031] The following describes the abnormal pig monitoring and early warning method and device of the embodiment of the present application with reference to the accompanying drawings.

[0032] Figure 1 Schematic diagram of a method for monitoring and warning abnormal pigs provided in an embodiment of the present application. Figure 1 As shown, the method may include but is not limited to the following steps: Step S101: Acquire multiple sets of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period.

[0033] Exemplarily, taking the current moment as the end time, within a first preset time period (for example, 30 minutes), one frame of infrared image data and corresponding air temperature data and air humidity data are acquired every first preset time period, thereby acquiring multiple sets of infrared image data, air temperature data, and air humidity data within the first preset time period, and taking the current moment as the end time, within a second preset time period (for example, 24 hours), one frame of visible light data of the pig breeding farm is acquired every second preset time period, thereby acquiring multiple frames of visible light image data within the second preset time period.

[0034] In the embodiment of the present application, the second preset period may be longer than the first preset period, that is, the second preset period includes the first preset period.

[0035] It should be noted that, in the embodiment of the present application, the acquisition time of part of the visible light image data may be the same as the acquisition time of the infrared image data, the air temperature data, and the air humidity data, that is, at the same time within the first preset time period, a group of infrared image data, air temperature data, air humidity data, and visible light image data are acquired simultaneously.

[0036] Step S102: performing target detection and behavior recognition based on the visible light image data, and obtaining individual pigs and behavior types corresponding to the individual pigs in each frame of visible light image data.

[0037] Exemplarily, target detection and behavior recognition are performed on each frame of visible light image using a pre-trained neural network model to obtain individual pigs in each frame of visible light image data and the behavior type corresponding to each individual pig in each frame of visible light image data.

[0038] Step S103: Based on the Euclidean distance of the detection frame in the visible light image data of adjacent frames, the visible light image data is tracked in combination with the Kalman filter algorithm, and the duration ratio of different behavior types of each individual pig is obtained in combination with the behavior types corresponding to the individual pigs.

[0039] Exemplarily, the pig individuals corresponding to the two detection frames with the smallest Euclidean distance between the detection frames in adjacent frames of visible light image data are taken as the same pig individual, so as to perform target tracking on the pig individuals and obtain initial target tracking results. The initial target tracking results are corrected by using the Kalman filtering algorithm to obtain target tracking results, so as to determine the position of the same pig individual in different frames of visible light image data, and combine the behavior type corresponding to each pig individual in each frame of visible light image data to obtain the behavior type of the same pig individual in different frames of visible light image data.

[0040] Among them, in the embodiments of the present application, the above-mentioned behavior types may include but are not limited to: lying down behavior and standing up behavior.

[0041] It should be noted that, in the embodiments of the present application, the pig individual may be lying on the ground or with the forelimbs perpendicular to the ground and the tail touching the ground as the prone behavior state.

[0042] It should be noted that, in the embodiments of the present application, the pig individual can be regarded as being in a standing behavior state in which the four legs are vertical, the body is vertical to the four legs, and the pig is in a stationary or walking state.

[0043] In one implementation, in order to avoid deformation of individual pigs caused by operations such as rotation, scaling, and translation of the visible light image, which may result in loss of some reference information in the background, the visible light image may be enhanced by using methods such as image noise addition and blurring before executing step S103.

[0044] Step S104: performing image segmentation on the infrared image data to obtain infrared matrices corresponding to individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig.

[0045] Exemplarily, based on the target detection results obtained by performing target detection on the visible light image data in the aforementioned steps, image segmentation is performed on the infrared image data acquired at the same time, an infrared matrix corresponding to each individual pig in each frame of infrared image data is obtained, and the highest temperature in each infrared matrix is ​​obtained as the body temperature measurement value corresponding to each individual pig.

[0046] For example, the following process can be used to obtain the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig: A1: Traverse each pixel point (x, y) in the binary image of the infrared temperature measurement data corresponding to each temperature matrix.

[0047] A2: When the value of the pixel point (x, y) is 1, the temperature of the point is added to the temperature set body T corresponding to the individual pig; or, when the value of the pixel point (x, y) is 0, the temperature corresponding to the pixel point is discarded.

[0048] A3: Obtain the highest temperature in the temperature set body T corresponding to the individual pig as the body temperature measurement value corresponding to each individual pig.

[0049] Step S105: Correct the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig.

[0050] Exemplarily, by statistically analyzing multiple sets of infrared temperature measurement data, air temperature data, and air humidity, the impact of environmental changes in pig breeding sites on infrared temperature measurement is determined, and infrared temperature measurement data, air temperature data, and air humidity data at multiple consecutive moments are used. These data can be used to analyze the impact of environmental changes on infrared temperature measurement. A time series prediction model is used to perform statistical analysis on multiple sets of infrared temperature measurement data, air temperature data, and air humidity to determine the impact of environmental changes in pig breeding sites on infrared temperature measurement, thereby correcting the infrared body temperature measurement value of individual pigs at the current moment, and obtaining the actual individual body temperature of the individual pigs at the current moment.

[0051] Step S106: Based on the duration ratio and actual individual body temperature of each individual pig, perform abnormal monitoring on the individual pigs.

[0052] Exemplarily, if the duration ratio corresponding to any behavior type of a certain individual pig is greater than or equal to the corresponding proportion threshold, it is determined that the individual pig is abnormal.

[0053] Exemplarily, if the actual individual body temperature of a certain individual pig is not within the preset normal body temperature range, it is determined that the individual pig is abnormal.

[0054] By implementing the embodiment of the present application, target detection and behavior recognition can be performed based on multiple frames of visible light image data of the pig breeding site within the second preset time period, and target tracking can be performed on the visible light image data based on the Euclidean distance between individual pigs in combination with the Kalman filter algorithm, thereby obtaining the duration ratio of different behavior types of each individual pig, and at the same time obtaining the corresponding body temperature measurement value of the individual pig based on the infrared image data, and correcting the body temperature measurement value in combination with the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig, thereby performing abnormal monitoring on the individual pig based on the duration ratio and the actual individual body temperature of each individual pig. More accurate abnormal monitoring of pigs can be performed.

[0055] In some embodiments, a Kalman filter algorithm may be used to modify the initial target final result obtained based on the Euclidean distance between individual pigs. As an example, see Figure 2 , Figure 2 FIG. 1 is a flow chart of another method for abnormal pig monitoring and early warning provided by the embodiment of the present application. Figure 2 As shown, the method may include but is not limited to the following steps: Step S201: Acquire multiple sets of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period.

[0056] In the embodiments of the present application, step S201 can be implemented by any of the methods in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0057] Step S202: performing target detection and behavior recognition based on the visible light image data, and obtaining individual pigs and behavior types corresponding to the individual pigs in each frame of visible light image data.

[0058] In the embodiments of the present application, step S202 can be implemented by any of the methods in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0059] Step S203: Obtain the detection frame corresponding to the individual pig in each frame of visible light image data.

[0060] Exemplarily, a detection frame corresponding to an individual pig in each frame of visible light image obtained by performing target detection on each frame of visible light image is obtained.

[0061] Step S204: performing target tracking based on the Euclidean distance between the center positions of the detection frames to obtain an initial target tracking result.

[0062] Exemplarily, non-large extreme value suppression is used to control the number of detection frames, the non-large extreme value suppression list is traversed, and the center position of each detection frame is calculated based on the coordinates of the four corners of the detection frame using the average value of the coordinates. The Euclidean distance between the center positions of the two detection frames corresponding to two adjacent frames of visible light images is obtained, and the minimum Euclidean distance is used as the judgment basis to obtain the same pig individual in two adjacent frames of visible light images, thereby obtaining the initial target tracking result.

[0063] Step S205: Using the Kalman filter algorithm to correct the initial target tracking result to obtain the target tracking result.

[0064] Exemplarily, the linear state equations of the Kalman filter algorithm can be expressed as follows:

[0065] in, express The estimated value of the state at the moment (i.e., corresponding to the Nth frame), express The measured value at the moment, is the state transfer matrix, is a noise sequence that conforms to the Gaussian distribution, is the measurement matrix, is a Gaussian measurement noise sequence. First, based on the equation Perform state prediction, where the covariance matrix of state prediction can be expressed as follows:

[0066] Based on the equation To predict the measured value, the Kalman gain can be expressed as follows:

[0067] Update the target state based on the prediction results: , the state covariance matrix is .

[0068] Step S206: Determine the position information of the same individual pig in each frame of visible light image data based on the target tracking result.

[0069] Step S207: based on the position information of the individual pigs in each frame of visible light image data and the behavior type corresponding to the individual pigs in each frame of visible light image data, the behavior type of each individual pig in each frame of visible light image data is obtained.

[0070] Exemplarily, the behavior type of each individual pig in each frame of visible light image is determined based on the position information of each individual pig in each frame of visible light image and the behavior types of individual pigs at different positions in each frame of visible light image.

[0071] Step S208: Perform statistical analysis based on the behavior type of each individual pig in each frame of visible light image data to obtain the duration ratio of different behavior types of each individual pig.

[0072] Exemplarily, taking the example that the behavior types include behavior type A and behavior type B, if the behavior type of pig individual A in M ​​frames of visible light images is behavior type A, and the behavior type of pig individual A in N frames of visible light images is behavior type B, and there are M+N frames of visible light images in total, then the duration ratio of pig individual A corresponding to behavior type A can be M / (M+N).

[0073] Step S209: performing image segmentation on the infrared image data to obtain the infrared matrix corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig.

[0074] In the embodiments of the present application, step S209 can be implemented in any of the ways in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0075] Step S210: Correct the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig.

[0076] In the embodiments of the present application, step S210 can be implemented in any of the ways in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0077] Step S211: Based on the duration ratio and actual individual body temperature of each individual pig, perform abnormal monitoring on the individual pigs.

[0078] In the embodiments of the present application, step S211 can be implemented in any of the ways in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0079] By implementing the embodiments of the present application, target tracking can be performed based on the Euclidean distance between detection frames of different frames of visible light image data to obtain initial target tracking results, and the initial target tracking results can be corrected using a Kalman filtering algorithm to obtain more accurate target tracking results, thereby determining the behavior type of individual pigs in each frame of visible light image data based on the target tracking results to improve the accuracy of monitoring abnormal behavior of individual pigs.

[0080] In some embodiments of the present application, the individual characteristics of each individual pig can be combined to monitor the individual pigs for abnormalities based on the time ratio and actual individual body temperature of each individual pig. Figure 3 , Figure 3 2 is a flow chart of another method for monitoring and warning abnormal pigs provided in the embodiment of the present application. Figure 3 As shown, the method may include but is not limited to the following steps: Step S301: Acquire multiple sets of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period.

[0081] In the embodiments of the present application, step S301 can be implemented in any of the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0082] Step S302: Perform target detection and behavior recognition based on the visible light image data to obtain individual pigs and behavior types corresponding to the individual pigs in each frame of visible light image data.

[0083] In the embodiments of the present application, step S302 can be implemented by any of the methods in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0084] Step S303: Based on the Euclidean distance of the detection frame in the visible light image data of adjacent frames, the visible light image data is tracked in combination with the Kalman filter algorithm, and the duration ratio of different behavior types of each individual pig is obtained in combination with the behavior types corresponding to the individual pigs.

[0085] In the embodiments of the present application, step S303 can be implemented by any of the methods in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0086] Step S304: performing image segmentation on the infrared image data to obtain infrared matrices corresponding to individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig.

[0087] In the embodiments of the present application, step S304 can be implemented by any of the methods in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0088] Step S305: Correct the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig.

[0089] In the embodiments of the present application, step S305 can be implemented in any of the ways in the embodiments of the present application, and the embodiments of the present application do not limit this and will not be described in detail.

[0090] Step S306: Obtain the growth period of each individual pig.

[0091] Exemplarily, image recognition is performed on visible light images based on a pre-trained neural network model to obtain the growth period of each individual pig according to the image recognition result.

[0092] Exemplarily, the growth period of each individual pig is manually labeled.

[0093] Step S307: Based on the growth period of each individual pig, the activity type ratio threshold of each individual pig is obtained.

[0094] As an example, please refer to Table 1, which is a comparison table of growth periods and behavior type durations provided by the present application.

[0095] Table 1 Comparison of growth period and duration of behavior type

[0096] Step S308: Based on the duration ratio and activity type ratio threshold range of each individual pig, determine whether the behavior of the individual pig is abnormal.

[0097] Exemplarily, based on the prone / standing time ratio ranges corresponding to pigs in different growth periods in Table 1, the prone / standing time ratio ranges corresponding to each individual pig in the visible light image are determined respectively, and the time ratio of each individual pig is compared with the corresponding lying / standing time ratio range. If the time ratio is within the corresponding lying / standing time ratio range, it is determined that there is no abnormality in the individual pig; if the time ratio is not within the corresponding lying / standing time ratio range, it is determined that there is an abnormality in the individual pig.

[0098] Step S309: Based on the actual individual body temperature corresponding to the individual pig and the preset body temperature threshold range, determine whether the body temperature of the individual pig is abnormal.

[0099] Exemplarily, if the actual individual body temperature of a certain individual pig is not within a preset body temperature threshold range, it is determined that the body temperature of the individual pig is abnormal.

[0100] Among them, the above-mentioned preset body temperature threshold range can be 38.5℃~39.5℃.

[0101] By implementing the embodiments of the present application, the growth period of each individual pig can be obtained, and the corresponding behavior type duration ratio range can be determined based on the growth period of each individual pig, so as to determine whether the behavior of the individual pig has abnormal behavior based on the duration ratio and activity type proportion threshold range of each individual pig, and determine whether the body temperature of the individual pig has abnormal body temperature based on the actual individual body temperature corresponding to the individual pig and the preset body temperature threshold range. More accurate abnormality detection can be performed on individual pigs.

[0102] In some embodiments of the present application, the step of performing target detection and behavior recognition based on visible light image data in any of the aforementioned embodiments to obtain individual pigs in each frame of visible light image data and the behavior types corresponding to the individual pigs is executed by calling a pre-trained YOLO model, wherein the YOLO model is a Bi-PAN-FPN structure, and uses the Effiententv2 network as the backbone network, the MBConv module of the Effiententv2 network contains a Resnet link, the neck network in the YOLO model includes a convolutional attention CBAM (Convolutional Block Attention Module) module, and the YOLO model is trained using the WIOU (Wise Intersection over Union) loss function.

[0103] As an example, see Figure 4 , Figure 4 Schematic diagram of a pig behavior monitoring process provided by the embodiment of the present application. Figure 4 As shown, the behavior of individual pigs can be identified based on the improved YOLO model, and based on the DKF (Distributed Kalman filtering) Mot (Multiple Object Tracking) tracking method, the minimum distance and Kalman filtering tracking methods are comprehensively applied to track the behavior of individual pigs, so as to obtain the time proportion of lying time and standing time of each individual pig in the second preset time period, so as to monitor the abnormality of pigs based on the time proportion.

[0104] It should be noted that during the training of the YOLO model, the multi-target tracking and annotation tool DarkLabel can be used to manually annotate the sample visible light image data. If there are pigs blocking each other or the pigs have insufficient field of vision in the visible light image, only the visible part is retained. Samples are screened based on the annotation results to obtain sample data of individual pigs lying down and standing, so as to train the YOLO model based on the sample data.

[0105] As an example, see Figure 5 , Figure 5 Schematic diagram of the structure of a YOLO model provided in an embodiment of the present application. Figure 5As shown, in order to improve the recognition ability of the behavior model for pig individuals that are farther away and smaller, Bi (Bidirectional)-PAN (Path Aggregation Network)-FPN (Feature Pyramid Network) is used as the feature fusion network in the neck network of the YOLIO model, and dynamic sampling (DySample) is added to FPN as the upsampling structure of the neck network, and the CBAM module is embedded after upsampling. The optimized Effiententv2 network is selected as the backbone network of the YOLO model. The Efficientnetv2 network used in this application consists of fused mobile inverted bottleneck convolution (Fused-MBConv) and mobile inverted bottleneck convolution (MBConv) modules. The first three layers use the Fused-MBConv module, the last three layers use the MBConv module, and the CBAM attention module is embedded in the MBConv module. At the same time, in order to retain more details and semantic information of the data, Resnet jump connections are added to the backbone network to avoid information feature loss.

[0106] The CBAM attention module includes a channel attention module (CAM) and a spatial attention module (SAM). The CAM module calculates the maximum and average features on each channel in parallel through the global maximum and average pooling branches to extract feature maps; the feature map is input to the shared fully connected layer to compress and expand the number of channels to generate the final attention vector; after adding the attention vectors of the two channels, they are normalized by the sigmoid activation function to generate the attention weight coefficient. It can be expressed as follows:

[0107] in, is the sigmoid activation function, is a shared fully connected layer, is the channel description feature after average pooling, Describes the features of the channel after maximum pooling.

[0108] SAM module for feature map Perform channel average pooling and maximum pooling, concatenate the two pooling generated images to obtain a feature map with two channels, and then perform convolution and activation operations to obtain the feature map adjustment weights. , the final weighted feature map is as follows:

[0109] in, is the sigmoid activation function, The convolution kernel size is The convolution operation.

[0110] In one implementation, WIOU can be used as a loss function in the process of training the YOLO model, which can be specifically expressed as follows:

[0111]

[0112]

[0113] in, is the outlier factor of the anchor box, and is a hyperparameter, and is the coordinate of the center point of the prediction box, and is the coordinate of the center point of the real frame, is the width of the minimum enclosing box formed by the predicted box and the true box, It is the height of the minimum enclosing box formed by the predicted box and the true box.

[0114] In some embodiments of the present application, the steps of performing image segmentation on the infrared image data in any of the aforementioned embodiments to obtain an infrared matrix corresponding to an individual pig, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig can be performed by calling a pre-trained OneFormer model, which is trained using a weighted sum as a loss function. The loss function can be expressed as follows:

[0115]

[0116]

[0117]

[0118] in, is the loss function, is the object-to-text contrast loss, is the contrastive loss from text to object, is the binary cross entropy on category prediction, is the standard classification loss for class prediction, is the binary cross entropy on the mask predictions, is the dice loss on mask prediction, is the weight of the dice loss, is the dice loss, is the number of target-text query objects, Indicates For target queries, Indicates For text queries, is the learnable temperature parameter, Represents the exclusive OR operator.

[0119] For example, you can set is 0.5, is 2, is 5, is 5.

[0120] It should be noted that in the process of training the OneFormer model, you can use text ( ) and target ( ) two groups of queries, representing the representation of the target to be segmented in the image based on text and image respectively. For text query ( ), first through Tokenize and pass the tokenized representation through a text encoder, which consists of six layers of Transformers. Text embeddings represent the binary masks of the input images and the number of their corresponding classes. Furthermore, a set of learnable text context embeddings ( ) is concatenated to the encoded text embedding to obtain the final Text query ( ).

[0121] It is understandable that since the maximum value in the temperature matrix of individual pigs is taken as the body temperature measurement value in this application, higher accuracy is required when segmenting infrared images. OneFormer adds a text encoding to extract the binary mask of each category of the image from the specific task label. In OneFormer, the instance segmentation task extracts non-overlapping binary masks of the object class and ignores other areas. The mask set is then iterated to create a text list ( ), which stores the binary mask class name. To maintain the uniformity of the list length, when the number of binary masks in the dataset is insufficient, the text list is filled with text entries to obtain a constant length. Fill the list ( ),use Compute the contrastive loss and use the task input with the template ( ) adjusts the task architecture, the templates are tokenized and mapped to task tokens ( ), used to control the algorithm. In the process of training the OneFormer model, the text ( ) represents the text-based representation of the target to be segmented in the image, using the target ( ) represents the image-based representation of the target to be segmented in the image.

[0122] For text queries ( ), first through Tokenize and pass the tokenized representation through a text encoder, where the text encoder consists of six layers of Transformer. Encoded Text embeddings represent the binary masks of the input images and the number of their corresponding classes. Furthermore, a set of learnable text context embeddings ( ) is concatenated to the encoded text embedding to obtain the final Text query ( ).

[0123] For target queries ( ), initialize the target query to be repeated Task token ). Then use the internal structure of the double-layer Transformer The updated target query contains the image context information, which is consistent with Connect, get The query task conditions are expressed.

[0124] In some embodiments of the present application, the steps of performing target detection and behavior recognition based on visible light image data in any of the aforementioned embodiments to obtain individual pigs in each frame of visible light image data and the behavior types corresponding to the individual pigs can be executed by calling a pre-trained YOLO model; wherein the YOLO model is a Bi-PAN-FPN structure, and uses the Effiententv2 network as the backbone network, the MBConv module of the Effiententv2 network contains a Resnet link, the neck network in the YOLO model includes a convolutional attention CBAM module, and the YOLO model is trained using the WIOU loss function.

[0125] Exemplarily, the number of input layer neurons of the LSTM model is set to 3, and the number of output layer neurons is set to 1. The air temperature, air humidity and body temperature measurement values ​​corresponding to each individual pig are used as input data of the model to obtain the actual individual body temperature of each individual pig output by the model.

[0126] In the embodiment of the present application, the number of neurons in the hidden layer of the above LSTM model can be obtained by the following formula:

[0127] in, is the number of neurons, is the number of LSTM output factors, is the number of input factors, is a constant, .

[0128] Among them, in the embodiments of the present application, the input data of the LSTM network are air temperature data, air humidity data and infrared temperature measurement values, the output data of the LSTM network is the actual individual body temperature, and the LSTM network is trained using mean square error as the loss function.

[0129] It should be noted that each unit in the LSTM network consists of an input gate, an output gate, and a forget gate. The algorithms of each gate are as follows:

[0130]

[0131]

[0132]

[0133]

[0134]

[0135] in, express Input at the moment, express The output at the moment (i.e. the infrared image data, air temperature and air humidity corresponding to the moment t), express The state at the previous moment. The output at the previous moment and the input at the current moment will be combined with the weight and the bias function Through the activation function and The function gets the output Then based on Calculate the current state variable by comparing it with the previous state variable , and finally through The function gets the output of the output gate.

[0136] See also Figure 6 , Figure 6 Schematic diagram of a pig abnormal behavior identification solution provided in the embodiment of the present application. Figure 6As shown, by installing fixed shooting equipment on the wall of the pen of the pig breeding base in advance, the individual pigs in the field can be photographed from a side view to obtain visible light images and infrared images. The pre-trained YOLO model is used to perform target detection and behavior recognition on the visible light image to obtain the individual pigs and corresponding behavior types in each frame of the visible light image; and the individual pigs in the visible light image are tracked to obtain the same individual pig in each frame of the visible light image, thereby obtaining the behavior type of each individual pig in each frame of the visible light image, and finally obtaining the time proportion of the behavior type of each individual pig.

[0137] At the same time, the infrared image is segmented based on the pre-trained OneFormer model, and the body temperature measurement value of each individual pig is obtained based on the image segmentation result; then, based on the pre-established LSTM regression model, the body temperature measurement value is corrected in combination with the air temperature and air humidity to obtain the actual body temperature value of each individual pig. Finally, based on the time proportion of the individual pig's behavior type and the actual body temperature value, abnormal pig monitoring is carried out.

[0138] See also Figure 7 , Figure 7 Schematic diagram of a pig abnormality monitoring and early warning device provided in the embodiment of the present application. Figure 7 As shown, the device 700 includes: an acquisition module 701, which is used to acquire multiple sets of infrared image data, air temperature data, air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period; a first processing module 702, which is used to perform target detection and behavior recognition based on the visible light image data, and obtain individual pigs in each frame of visible light image data and the behavior type corresponding to the individual pigs; a second processing module 703, which is used to track targets in the visible light image data based on the Euclidean distance of the detection frames in the visible light image data of adjacent frames in combination with the Kalman filter algorithm , and combined with the corresponding behavior type of the individual pig, the duration ratio of different behavior types of each individual pig is obtained; the third processing module 704 is used to perform image segmentation on the infrared image data, obtain the infrared matrix corresponding to the individual pig, and obtain the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig; the fourth processing module 705 is used to correct the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig; the monitoring module 706 is used to perform abnormal monitoring on the individual pig based on the duration ratio and the actual individual body temperature of each individual pig.

[0139] In one implementation, the first processing module 702 can perform target detection and behavior recognition based on visible light image data by calling a pre-trained YOLO model to obtain the steps of individual pigs in each frame of visible light image data and the behavior types corresponding to the individual pigs; wherein the YOLO model is a Bi-PAN-FPN structure, and uses the Effiententv2 network as the backbone network, the MBConv module of the Effiententv2 network contains a Resnet link, the neck network in the YOLO model includes a convolutional attention CBAM module, and the YOLO model is trained using the WIOU loss function.

[0140] In one implementation, the second processing module 703 can be used to: obtain a detection frame corresponding to an individual pig in each frame of visible light image data; perform target tracking based on the Euclidean distance between the center positions of the detection frames to obtain an initial target tracking result; use a Kalman filter algorithm to correct the initial target tracking result to obtain a target tracking result; determine the position information of the same individual pig in each frame of visible light image data based on the target tracking result; obtain the behavior type of each individual pig in each frame of visible light image data based on the position information of the individual pig in each frame of visible light image data and the behavior type corresponding to the individual pig in each frame of visible light image data; perform statistical analysis based on the behavior type of each individual pig in each frame of visible light image data to obtain the duration ratio of different behavior types of each individual pig.

[0141] In one implementation, the third processing module 704 can perform the steps of performing image segmentation on the infrared image data, obtaining the infrared matrix corresponding to the individual pig, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig by calling the pre-trained OneFormer model; wherein the OneFormer model is trained using the weighted sum as the loss function, and the loss function can be expressed as follows:

[0142]

[0143]

[0144]

[0145] in, is the loss function, is the object-to-text contrast loss, is the contrastive loss from text to object, is the binary cross entropy on category prediction, is the standard classification loss for class prediction, is the binary cross entropy on the mask predictions, is the dice loss on mask prediction, is the weight of the dice loss, is the dice loss, is the number of target-text query objects, Indicates For target queries, Indicates For text queries, is the learnable temperature parameter, Represents the exclusive OR operator.

[0146] In one implementation, the fourth processing module 705 calls a pre-trained long short-term memory (LSTM) network to perform a step of correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each pig. The input data of the LSTM network are the air temperature data, the air humidity data and the infrared temperature measurement value, and the output data of the LSTM network is the actual individual body temperature. The LSTM network is trained using the mean square error as the loss function.

[0147] In one implementation, the monitoring module 706 can be used to: obtain the growth period of each individual pig; obtain the activity type ratio threshold of each individual pig based on the growth period of each individual pig; determine whether the behavior of the individual pig is abnormal based on the duration ratio and activity type ratio threshold range of each individual pig; determine whether the body temperature of the individual pig is abnormal based on the actual individual body temperature corresponding to the individual pig and the preset body temperature threshold range.

[0148] Through the device of the embodiment of the present application, target detection and behavior recognition can be performed based on multiple frames of visible light image data of the pig breeding site within the second preset time period, and target tracking can be performed on the visible light image data based on the Euclidean distance between individual pigs in combination with the Kalman filter algorithm, so as to obtain the duration ratio of different behavior types of each individual pig, and at the same time, the body temperature measurement value corresponding to the individual pig is obtained based on the infrared image data, and the body temperature measurement value is corrected in combination with the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig, so as to monitor the individual pig for abnormalities based on the duration ratio and the actual individual body temperature of each individual pig. More accurate abnormal monitoring of pigs can be performed.

[0149] It should be noted that the above explanation of the embodiment of the abnormal pig monitoring and early warning method is also applicable to the abnormal pig monitoring and early warning device of this embodiment, and will not be repeated here.

[0150] In order to implement the above embodiment, the present application also proposes an electronic device. Figure 8 , Figure 8 Schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 8 As shown, the electronic device 800 includes: a processor 801, and a memory 802 communicatively connected to the processor 801; the memory 802 stores computer-executable instructions; the processor 801 executes the computer-executable instructions stored in the memory to implement the method provided in the aforementioned embodiment.

[0151] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0152] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0153] In the description of this application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0154] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0155] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0156] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0157] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0158] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0159] A person skilled in the art may understand that all or part of the steps in the above-mentioned embodiment method may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0161] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for monitoring and early warning of abnormal pigs, characterized in that: include: Acquire multiple sets of infrared image data, air temperature data, and air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period; Performing target detection and behavior recognition based on the visible light image data to obtain individual pigs in each frame of the visible light image data and the behavior type corresponding to the individual pigs; Based on the Euclidean distance of the detection frame in the visible light image data of adjacent frames, the visible light image data is tracked in combination with the Kalman filter algorithm, and combined with the behavior type corresponding to the individual pigs, the duration ratio of the different behavior types of each individual pig is obtained; Performing image segmentation on the infrared image data to obtain infrared matrices corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as a body temperature measurement value corresponding to each individual pig; Correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each of the individual pigs; Based on the time ratio and the actual individual body temperature of each individual pig, the individual pigs are monitored for abnormalities.

2. The method according to claim 1, characterized in that The step of performing target detection and behavior recognition based on the visible light image data to obtain individual pigs in each frame of the visible light image data and the behavior type corresponding to the individual pigs is performed by calling a pre-trained YOLO model; wherein, The YOLO model has a Bi-PAN-FPN structure and uses the Effiententv2 network as the backbone network. The MBConv module of the Effiententv2 network contains a Resnet link. The neck network in the YOLO model includes a convolutional attention CBAM module. The YOLO model is trained using the WIOU loss function.

3. The method according to claim 1, characterized in that The target tracking of the visible light image data is performed based on the Euclidean distance of the detection frame in the visible light image data of adjacent frames in combination with the Kalman filter algorithm, and the duration ratio of different behavior types of each individual pig is obtained in combination with the behavior type corresponding to the individual pig, including: Obtain the detection frame corresponding to the individual pig in each frame of the visible light image data; Performing target tracking based on the Euclidean distance between the center positions of the detection frames to obtain an initial target tracking result; The initial target tracking result is corrected by using a Kalman filter algorithm to obtain a target tracking result; Determine the position information of the same individual pig in each frame of the visible light image data based on the target tracking result; Based on the position information of the individual pig in each frame of the visible light image data and the behavior type corresponding to the individual pig in each frame of the visible light image data, the behavior type of each individual pig in each frame of the visible light image data is obtained; Statistical analysis is performed based on the behavior type of each individual pig in each frame of the visible light image data to obtain the duration ratio of different behavior types of each individual pig.

4. The method according to claim 1, characterized in that The step of performing image segmentation on the infrared image data, obtaining the infrared matrix corresponding to the individual pigs, and obtaining the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig is performed by calling the pre-trained OneFormer model; wherein, The OneFormer model is trained using the weighted sum as the loss function, which can be expressed as follows: in, is the loss function, is the object-to-text contrast loss, is the contrastive loss from text to object, is the binary cross entropy on category prediction, is the standard classification loss for class prediction, is the binary cross entropy on the mask predictions, is the dice loss on mask prediction, is the weight of the dice loss, is the dice loss, is the number of target-text query objects, Indicates For target queries, Indicates For text queries, is the learnable temperature parameter, Represents the exclusive OR operator.

5. The method according to claim 1, characterized in that The step of correcting the body temperature measurement value based on the air temperature data and the air humidity data to obtain the actual individual body temperature of each individual pig is performed by calling a pre-trained long short-term memory LSTM network; wherein, The input data of the LSTM network are the air temperature data, the air humidity data and the infrared temperature measurement value, the output data of the LSTM network is the actual individual body temperature, and the LSTM network is trained using the mean square error as the loss function.

6. The method according to claim 1, characterized in that The abnormal monitoring of the individual pigs based on the time ratio and the actual individual body temperature of each individual pig includes: Obtaining the growth period of each individual pig; Based on the growth period of each individual pig, obtain the activity type ratio threshold of each individual pig; Based on the duration ratio and the activity type ratio threshold range of each individual pig, determine whether the behavior of the individual pig is abnormal; Based on the actual individual body temperature corresponding to the individual pig and a preset body temperature threshold range, it is determined whether the body temperature of the individual pig is abnormal.

7. A pig abnormality monitoring and early warning device, characterized in that: include: An acquisition module, used to acquire multiple sets of infrared image data, air temperature data, and air humidity data of a pig breeding site in a first preset time period and multiple frames of visible light image data of the pig breeding site in a second preset time period; A first processing module is used to perform target detection and behavior recognition based on the visible light image data, and obtain individual pigs in each frame of the visible light image data and the behavior type corresponding to the individual pigs; A second processing module is used to track the target of the visible light image data based on the Euclidean distance of the detection frame in the visible light image data of adjacent frames in combination with the Kalman filter algorithm, and to obtain the duration ratio of different behavior types of each individual pig in combination with the behavior type corresponding to the individual pig; A third processing module is used to perform image segmentation on the infrared image data, obtain the infrared matrix corresponding to the individual pig, and obtain the highest temperature in each infrared matrix as the body temperature measurement value corresponding to each individual pig; A fourth processing module, configured to correct the body temperature measurement value based on the air temperature data and the air humidity data to obtain an actual individual body temperature of each of the individual pigs; A monitoring module is used to perform abnormal monitoring on the individual pigs based on the time ratio and the actual individual body temperature of each individual pig.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images