Pig body temperature detection system and detection method based on visible light and infrared thermal imaging video

Abstract

The invention provides a pig body temperature detection system and method based on visible light and infrared thermal imaging videos, and the system comprises a main control module, a pig identity recognition module, a pig temperature detection module, an early warning module, a processor, a system background, a wireless communication module, a display module, and an auxiliary recognition device. The main control module is used for acquiring information of the pig identity recognition module and the pig temperature detection module and transmitting the information to a system background through the wireless communication module; the processor calls pig house images in a system background, identifies pigs by adopting a YOLOV7-tiny algorithm, analyzes pig codes by adopting an end-to-end multi-direction character detection and identification algorithm FOTS, calls pig temperature videos, intercepts corresponding temperature maps according to pig house image acquisition time, analyzes and encodes pig temperatures, and sends the pig temperature videos to the pig house. Meanwhile, the detected high-temperature pigs are tracked and detected, early warning is conducted through the early warning module, rapid and accurate body temperature monitoring is conducted on a large number of pigs, the detection efficiency is improved, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a pig body temperature detection system and method based on visible light and infrared thermal imaging video. Background Art

[0002] Pig temperature detection is a very important part of pig farm management. It can effectively monitor the health of pigs, detect diseases and illnesses in time, and take targeted prevention and treatment measures. In addition, pig temperature detection can also improve breeding efficiency. In production practice, it is found that slight changes in pig body temperature can lead to problems such as decreased appetite and decreased production capacity, which in turn have an adverse effect on breeding profits. Through regular temperature detection, these changes can be detected in time and corresponding adjustment measures can be taken, thereby improving breeding efficiency and economic benefits.

[0003] Traditional temperature detection methods require manual measurement of the ears or anus, which is not only time-consuming and labor-intensive, but also easily causes stress and shock to the pigs. The existing handheld temperature monitoring system and the automatic monitoring method of waiting for the pigs to approach with a fixed temperature measuring device cannot implement all-round and fully automatic monitoring of the pig house and analyze and process the monitoring results.

[0004] Therefore, the pig temperature detection system using visible light and infrared thermal imaging video can achieve large-scale, continuous, and automated monitoring. Compared with the traditional manual detection method, this technology can quickly and accurately monitor the body temperature of a large number of pigs, improve the detection efficiency, and reduce labor costs. Compared with the existing monitoring system, it can store and analyze the test results and respond to abnormal conditions detected in a timely manner. When the pig's body temperature is abnormal, this technology can detect the problem in time through monitoring data, and take measures based on the analysis results to avoid the rapid spread of the disease in the farm. Compared with the use of thermal imaging equipment that can directly perform target detection, the present invention uses a combination of visible light cameras and ordinary thermal imaging cameras to reduce the cost of use. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a pig temperature detection system and method based on visible light and infrared thermal imaging video, so as to realize rapid and accurate temperature monitoring of a large number of pigs, improve detection efficiency, reduce labor costs, and take corresponding measures according to the results.

[0006] Technical solution: To achieve the above purpose, the present invention provides a method for detecting pig body temperature based on visible light and infrared thermal imaging video, comprising the following steps:

[0007] Step 1: Use infrared thermal imaging camera video to dynamically record the temperature of pigs in the pig house to obtain a pig temperature video;

[0008] Step 2: Use a visible light camera to shoot the pig house at intervals of Δt to obtain the image of the pig house and the shooting time t;

[0009] Step 3: For the pig temperature video in step 1, capture the pig temperature graph at time t;

[0010] Step 4: Obtain the pig bounding box: Preset the calibration size, intercept the pig house image according to the preset calibration size, and use the YOLO V7-tiny algorithm to identify the pigs in the image to obtain the pig bounding box containing the pigs at time t;

[0011] Step 5: Use the end-to-end multi-directional text detection and recognition algorithm FOTS to obtain the pig code: crop according to the pig bounding box marked in step 4, determine the image with the center coordinate (i, j) that can be used for text recognition, and locate the text line area features according to the center coordinate (i, j), convert the text line area into the corresponding character sequence, and decode the character sequence to obtain the final pig code;

[0012] Step 6: Retrieve the pig temperature video, intercept the corresponding temperature information within the coordinates of the identified and coded pig boundary box according to the pig house image acquisition time t, and calculate the average temperature as the pig temperature of this coded pig at time t.

[0013] Furthermore, the ears of the pigs in the pig house are provided with RFID ear tags with printed codes indicating the pigs' identities.

[0014] Furthermore, the specific operation of using the YOLO V7-tiny algorithm to identify the pigs in the image in step 4 is as follows:

[0015] First, input the image into the neural network and use the activation function sigmoid;

[0016] Then extract the features and input the feature map into the detection head network to generate candidate boxes and corresponding prediction boxes;

[0017] Then the non-maximum suppression (NMS) algorithm is used to screen the candidate boxes, and the prediction box with the largest confidence is retained to obtain the pig bounding box containing the pig image.

[0018] Furthermore, the specific operation of obtaining the pig code in step 5 is:

[0019] Using the end-to-end multi-directional text detection and recognition algorithm FOTS, the cropped image is input into the convolutional neural network CNN to extract features. ResNet50 is used as the feature extractor. On the feature map, the Anchor-Free framework is used to detect text lines. For each detected text line, RoIAlign is used to crop the features of the text line area to obtain an image that can be used for text recognition with the center coordinates (i, j) for subsequent recognition tasks, where (w, h) is the cropped width and height, G is the weight matrix, and u, v are the mappings;

[0020]

[0021] The text line region features are located by the center coordinates (i, j), and the text line region is converted into a corresponding character sequence using the recognition module Char-Net based on the recurrent neural network RNN;

[0022] Finally, the CTC text recognition algorithm is used to decode the character sequence to obtain the final text recognition result LER and mark it on the bounding box:

[0023]

[0024] Among them, X is the input space, Z is the target space, h is the function from the input space to the target space, ED is the edit distance of the sequence, and the minimum number of insertions, replacements, and deletions required to transform the input space into the target space.

[0025] Furthermore, in step 6, the temperature of the pig detected in the pig temperature map at time t is compared with the temperature threshold. If there is a temperature higher than the temperature threshold, the following steps are performed:

[0026] Obtain the coordinates of the high temperature area, compare the coordinates of the pig bounding box containing the pig image, and obtain the high temperature pig bounding box;

[0027] If the high-temperature pig boundary box has identified the pig code, the temperature of the coded pig is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times;

[0028] If the high-temperature pig boundary box cannot identify the pig code, the temperature of the high-temperature area is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times;

[0029] If the high-temperature pig boundary box identifies multiple pig codes, the temperature of multiple coded pigs is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times;

[0030] Pigs with temperatures above the temperature threshold are coded or high temperature areas are visualized.

[0031] The present invention also discloses a detection system based on the above-mentioned pig body temperature detection method based on visible light and infrared thermal imaging video, comprising:

[0032] Main control module, pig identification module, pig temperature detection module, processor, system background, wireless communication module, auxiliary identification device;

[0033] The main control module is used to obtain information from the pig identification module and the pig temperature detection module, and transmit it to the system background through the wireless communication module;

[0034] The processor is used to retrieve the pig house image in the system background, use the YOLO V7-tiny algorithm to identify the pigs in the image, and use the end-to-end multi-directional text detection and recognition algorithm FOTS to parse the pig code, and at the same time retrieve the pig temperature video to intercept the corresponding temperature map according to the pig house image acquisition time, parse the coded pig temperature and store it in the system background, and track and detect the detected high-temperature pigs or high-temperature areas;

[0035] The auxiliary identification device uses an RFID ear tag with a printed code that can identify the identity of the pig;

[0036] The pig identification module is used to identify the pigs in the pig house image using the YOLO V7-tiny algorithm and parse the pig code using the end-to-end multi-directional text detection and recognition algorithm FOTS;

[0037] The pig temperature detection module is used to dynamically record the temperature of pigs in the pig house using infrared thermal imaging camera video, and intercept the corresponding temperature information within the coordinates of the identified and coded pig boundary box according to the pig house image acquisition time t, and calculate the average temperature as the pig temperature of this coded pig at time t.

[0038] Preferably, an early warning module is also included, and the processor compares the temperature of the pig detected in the pig temperature map at time t with the temperature threshold. If there is a temperature higher than the temperature threshold, the high temperature area coordinates are obtained, and the coordinates of the pig boundary box containing the pig image are compared to obtain the high temperature pig boundary box:

[0039] If the high-temperature pig boundary box has identified the pig code, the temperature of the coded pig is tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm;

[0040] If the high-temperature pig boundary box cannot identify the pig code, the temperature of the high-temperature area is tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm;

[0041] If the high-temperature pig boundary box identifies multiple pig codes, the temperatures of the multiple coded pigs are tracked. When the temperature is higher than the temperature threshold multiple times, the main control module controls the early warning module to alarm.

[0042] Preferably, a display model is also included, and the main control module displays the codes of pigs with temperatures higher than the temperature threshold or the high temperature area on the display module.

[0043] Beneficial effects:

[0044] The present invention adopts a method combining visible light camera identification and infrared thermal imaging video identification of temperature to reduce the cost of use. By obtaining the information of the pig identification module and the pig temperature detection module, it is transmitted to the system background through the wireless communication module. The processor retrieves the pig house image in the system background and uses the target detection method based on lightweight deep learning to identify the pigs and parse the pig codes. At the same time, the pig temperature video is retrieved according to the pig house image acquisition time to intercept the corresponding temperature map, parse the coded pig temperature and store it in the system background, and the detected high-temperature pigs are tracked and detected. By presetting the temperature threshold, it can detect abnormal conditions and respond in time. When the pig's body temperature is abnormal, this technology can detect the problem in time through monitoring data and take measures according to the analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural diagram of the detection system of the present invention;

[0046] Figure 2 The present invention is a flow chart of the detection method. DETAILED DESCRIPTION

[0047] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

[0048] As introduced in the background technology, there are deficiencies in the prior art. In order to solve the above technical problems, the present invention proposes a pig body temperature detection system and detection method based on visible light and infrared thermal imaging video.

[0049] See also Figure 1 :Detection system structure diagram

[0050] The present invention discloses a pig temperature detection system based on visible light and infrared thermal imaging video, and its structure includes: a main control module, a pig identity recognition module, a pig temperature detection module, an early warning module, a processor, a system background, a wireless communication module, a display module and an auxiliary recognition device.

[0051] The main control module is used to obtain the information of the pig identification module and the pig temperature detection module, and transmit it to the system background through the wireless communication module.

[0052] The processor is used to retrieve the pig house image in the system background, and use the YOLO V7-tiny algorithm to identify the pigs in the image and parse the pig code. At the same time, it retrieves the pig temperature video and captures the corresponding temperature map according to the pig house image acquisition time. The pig body temperature detection method is used to parse the coded pig temperature and store it in the system background, and the detected high-temperature pigs or high-temperature areas are tracked and detected.

[0053] The auxiliary identification device uses an RFID ear tag with a printed code to identify the pig's identity.

[0054] The pig identification module uses a visible light camera to capture images in the pig house. The processor intercepts the pig house image with an identifiable temperature range according to the preset calibration size, uses the YOLO V7-tiny algorithm to identify the target image in real time, and performs convolution operations on the images in the pig house according to the targeted trained detection model, extracts and analyzes features frame by frame, and obtains multiple prediction frames. The prediction frames are then stacked, and then non-maximum suppression is performed. The prediction frame with the highest confidence is selected and retained to obtain the pig bounding box and position coordinates containing the pig image. The identity coding ear tag on the pig is identified, the coding information is obtained, the pig identity is bounded, and the pig identity information, body temperature information and detailed location are marked and displayed with bounding boxes of different colors in combination with the temperature measurement information of the pig temperature detection module.

[0055] The CTC text recognition algorithm is used for pig code recognition. The input image is scaled or cropped and resized to a fixed input size. ResNet50 is used as a feature extractor, and the output feature map is used to extract features from the input image using a pre-trained convolutional neural network to obtain the feature representation of each pixel. The extracted feature map is input into a bidirectional network, and a text box at each position is obtained by sliding a sliding window on the feature map. Two branches are used to detect text: one branch outputs a binary text image, and the other branch predicts the position and orientation angle of the text box, which is used to calculate the coordinates and direction of the text box, and then decoded by the CTC text recognition algorithm to obtain the final text recognition result. The text box is non-maximum suppressed to remove overlapping text boxes, and finally the final recognition result is marked on the pig's bounding box. At the same time, the encoding information of the system background is retrieved, the pig identity is bound, and the temperature information of the pig temperature detection module is combined to mark and display the pig's identity information, body temperature information and detailed location with bounding boxes of different colors.

[0056] The pig temperature monitoring module uses an infrared thermal imaging camera to dynamically record the temperature of pigs in the pig house in the form of video. The processor takes a screenshot of the pig temperature map of the corresponding time period according to the acquisition time of the pig house image.

[0057] See also Figure 2 :Detection method flow chart

[0058] Step 1: The pig temperature detection module uses an infrared thermal imaging camera to dynamically record the temperature of the pigs in the pig house in the form of video to obtain the pig temperature video. The main control module transmits the pig temperature video to the system background through the wireless communication module.

[0059] Step 2: The pig identification module uses a visible light camera to photograph the pig house at intervals of Δt, and the main control module transmits the images taken by the camera and the shooting time t to the system background through the wireless communication module.

[0060] Step 3: The processor retrieves the pig temperature video in the system background and captures the pig temperature graph at time t.

[0061] Step 4: The processor captures the pig house image according to the preset calibration size and uses YOLO V7-tiny to identify the pigs in the pig house image: First, the image is input into the neural network and the activation function sigmoid is used:

[0062]

[0063] The predicted values ​​are mapped to the range of 0 to 1 and then the softmax function is used to normalize the confidence of each category so that their sum is 1.

[0064]

[0065] Then, the features are extracted and the feature map is input into the detection head network to generate candidate boxes and corresponding prediction boxes (including category probability and bounding box position). The non-maximum suppression (NMS) algorithm is then used to screen the candidate boxes and calculate the IoU between the bounding boxes, where A overlap is the intersection area of ​​the predicted box and the true annotation box, A union is the union area of ​​the predicted box and the true annotation box;

[0066]

[0067] By calculating the IoU between bounding boxes, we can decide which bounding boxes need to be suppressed to remove duplicate boxes. Based on the predicted probability and the set threshold, we can filter out boxes with a confidence level higher than the threshold. According to the set rules, we can adjust and merge the boxes to obtain the pig bounding box F containing the pig at time t and the position coordinates (x, y) of the center point.

[0068] The preset calibration size obtains the position coordinates of the thermal imaging camera recognition area in the visible light camera shooting area according to the fixed positions of the visible light camera and the thermal imaging camera and the recognition accuracy of the thermal imaging camera. When the processor recognizes the identity, it only recognizes the range of the thermal imaging camera recognition area.

[0069] Step 5: The processor crops the pig bounding box marked in step 4, and then uses the end-to-end multi-directional text detection and recognition algorithm FOTS to input the cropped image into the convolutional neural network (CNN) to extract features. On the feature map, the Anchor-Free framework is used to detect text lines. For each detected text line, RoIAlign is used to crop the features of the text line area to obtain an image that can be used for text recognition with the center coordinates i, j) for subsequent recognition tasks, where (w, h) is the cropped width and height, G is the weight matrix, and u, v are the mappings:

[0070]

[0071] The text line region feature is located by the center coordinate (i, j), and the text line region is converted into a corresponding character sequence using the recurrent neural network (RNN)-based recognition module Char-Net. Finally, the character sequence is decoded using the CTC text recognition algorithm to obtain the final text recognition result LER and mark it on the bounding box;

[0072]

[0073] Among them, X is the input space, Z is the target space, h is the function from the input space to the target space, ED is the edit distance of the sequence, and the minimum number of insertions, replacements, and deletions required to transform the input space into the target space.

[0074] Step 6: The processor compares the pig temperature detected in the pig temperature map at time t with the temperature threshold. If all are lower than the temperature threshold, execute step 7; otherwise, execute step 8.

[0075] Step 7: The processor obtains the temperature information within the coordinates of the identified coded pig boundary box, calculates the average temperature as the temperature information of this coded pig at time t, and the main control module transmits it to the system background through the wireless communication module.

[0076] Step 8: The processor obtains the coordinates of the high temperature area, compares the coordinates of the pig bounding box containing the pig image, and obtains the high temperature pig bounding box:

[0077] If the high-temperature pig boundary box has identified the pig code, the temperature of the coded pig will be tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm.

[0078] If the high-temperature pig boundary box cannot identify the pig code, the temperature of the high-temperature area is tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm.

[0079] If the high-temperature pig boundary box identifies multiple pig codes, the temperatures of the multiple coded pigs are tracked. When the temperature is higher than the temperature threshold multiple times, the main control module controls the early warning module to alarm.

[0080] Step 9: The main control module codes the pigs whose temperature is higher than the temperature threshold or displays the high temperature area on the display module.

[0081] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting pig body temperature based on visible light and infrared thermal imaging video, characterized in that: The steps include: Step 1: Use infrared thermal imaging camera video to dynamically record the temperature of pigs in the pig house to obtain a pig temperature video; Step 2: Use a visible light camera to shoot the pig house at intervals of Δt to obtain the image of the pig house and the shooting time t; Step 3: For the pig temperature video in step 1, capture the pig temperature graph at time t; Step 4: Obtain the pig bounding box: Preset the calibration size, intercept the pig house image according to the preset calibration size, and use the YOLO V7-tiny algorithm to identify the pigs in the image to obtain the pig bounding box containing the pigs at time t; Step 5: Use the end-to-end multi-directional text detection and recognition algorithm FOTS to obtain the pig code: crop according to the pig bounding box marked in step 4, determine the image with the center coordinate (i, j) that can be used for text recognition, and locate the text line area features according to the center coordinate (i, j), convert the text line area into the corresponding character sequence, and decode the character sequence to obtain the final pig code; Step 6: Retrieve the pig temperature video, intercept the corresponding temperature information within the coordinates of the identified and coded pig boundary box according to the pig house image acquisition time t, and calculate the average temperature as the pig temperature of this coded pig at time t.

2. The method for detecting pig body temperature based on visible light and infrared thermal imaging video according to claim 1 is characterized in that: The ears of the pigs in the pig house are provided with RFID ear tags with printed codes indicating the pigs' identities.

3. The method for detecting pig body temperature based on visible light and infrared thermal imaging video according to claim 1 is characterized in that: In step 4, the specific operation of using the YOLO V7-tiny algorithm to identify the pigs in the image is as follows: First, input the image into the neural network and use the activation function sigmoid; Then extract the features and input the feature map into the detection head network to generate candidate boxes and corresponding prediction boxes; Then the non-maximum suppression (NMS) algorithm is used to screen the candidate boxes, and the prediction box with the largest confidence is retained to obtain the pig bounding box containing the pig image.

4. The method for detecting pig body temperature based on visible light and infrared thermal imaging video according to claim 1, characterized in that: The specific operation of obtaining the pig code in step 5 is: Using the end-to-end multi-directional text detection and recognition algorithm FOTS, the cropped image is input into the convolutional neural network CNN to extract features. ResNet50 is used as the feature extractor. On the feature map, the Anchor-Free framework is used to detect text lines. For each detected text line, RoIAlign is used to crop the features of the text line area to obtain an image that can be used for text recognition with the center coordinates (i, j) for subsequent recognition tasks, where (w, h) is the cropped width and height, G is the weight matrix, and u, v are the mappings; The text line region features are located by the center coordinates (i, j), and the text line region is converted into a corresponding character sequence using the recognition module Char-Net based on the recurrent neural network RNN; Finally, the CTC text recognition algorithm is used to decode the character sequence to obtain the final text recognition result LER and mark it on the bounding box: Among them, X is the input space, Z is the target space, h is the function from the input space to the target space, ED is the edit distance of the sequence, and the minimum number of insertions, replacements, and deletions required to transform the input space into the target space.

5. The method for detecting pig body temperature based on visible light and infrared thermal imaging video according to claim 1, characterized in that: In step 6, the temperature of the pig detected in the pig temperature map at time t is compared with the temperature threshold. If the temperature is higher than the temperature threshold, the following steps are performed: Obtain the coordinates of the high temperature area, compare the coordinates of the pig bounding box containing the pig image, and obtain the high temperature pig bounding box; If the high-temperature pig boundary box has identified the pig code, the temperature of the coded pig is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times; If the high-temperature pig boundary box cannot identify the pig code, the temperature of the high-temperature area is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times; If the high-temperature pig boundary box identifies multiple pig codes, the temperature of multiple coded pigs is tracked, and an early warning is issued when the temperature exceeds the temperature threshold multiple times; Pigs with temperatures above the temperature threshold are coded or high temperature areas are visualized.

6. A detection system based on the pig body temperature detection method based on visible light and infrared thermal imaging video according to claim 1, characterized in that: include: Main control module, pig identification module, pig temperature detection module, processor, system background, wireless communication module, auxiliary identification device; The main control module is used to obtain information from the pig identification module and the pig temperature detection module, and transmit it to the system background through the wireless communication module; The processor is used to retrieve the pig house image in the system background, use the YOLO V7-tiny algorithm to identify the pigs in the image, and use the end-to-end multi-directional text detection and recognition algorithm FOTS to parse the pig code, and at the same time retrieve the pig temperature video to intercept the corresponding temperature map according to the pig house image acquisition time, parse the coded pig temperature and store it in the system background, and track and detect the detected high-temperature pigs or high-temperature areas; The auxiliary identification device uses an RFID ear tag with a printed code that can identify the identity of the pig; The pig identification module is used to identify the pigs in the pig house image using the YOLO V7-tiny algorithm and parse the pig code using the end-to-end multi-directional text detection and recognition algorithm FOTS; The pig temperature detection module is used to dynamically record the temperature of pigs in the pig house using infrared thermal imaging camera video, and intercept the corresponding temperature information within the coordinates of the identified and coded pig boundary box according to the pig house image acquisition time t, and calculate the average temperature as the pig temperature of this coded pig at time t.

7. The detection system according to claim 6, characterized in that: It also includes an early warning module, wherein the processor compares the temperature of the pig detected in the pig temperature map at time t with the temperature threshold. If the temperature is higher than the temperature threshold, the high temperature area coordinates are obtained, and the coordinates of the pig boundary box containing the pig image are compared to obtain the high temperature pig boundary box: If the high-temperature pig boundary box has identified the pig code, the temperature of the coded pig is tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm; If the high-temperature pig boundary box cannot identify the pig code, the temperature of the high-temperature area is tracked. When the temperature exceeds the temperature threshold for multiple times, the main control module controls the early warning module to alarm; If the high-temperature pig boundary box identifies multiple pig codes, the temperatures of the multiple coded pigs are tracked. When the temperature is higher than the temperature threshold multiple times, the main control module controls the early warning module to alarm.

8. The detection system according to claim 7, characterized in that: It also includes a display model, and the main control module displays the pig codes with temperatures higher than the temperature threshold or the high temperature area on the display module.

Images