Pig farm mouse detection method, system, device and medium based on deep learning

Abstract

The application discloses a pig farm mouse detection method, system, device and medium based on deep learning, and the method comprises the steps of: frame extraction processing is performed on a video of a to-be-detected area; image preprocessing is performed on an image frame set; a current frame after preprocessing is input into a mouse detection model to learn the appearance of a mouse; the mouse detection model is obtained by using an improved RT-DETR detection model; a pixel with a value of 255 in a binary image obtained by using a frame difference method is taken as a motion pixel, and a corresponding detection result is reserved; a mouse detection result is recorded, and a mouse appearing time length is obtained by weighted summation of the number of mice recorded after each frame detection, so as to calculate a mouse activity index of the to-be-detected area. By adding a frame difference prior branch, the binary image obtained by using the frame difference method is added to the decoding part of the RT-DETR algorithm, and double thresholds are set when the detection result is reserved, so that the model focuses more on the motion area in detection, and the detection success rate of small and fast mice is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a pig farm mouse detection method, system, device and medium based on deep learning. BACKGROUND

[0002] The control of harmful organisms such as mice is an important problem in the fields of agriculture and food processing industry, and mice can gnaw feed and fertilizer, cause pollution, damage facilities, and spread diseases, which are very harmful and need to be controlled and managed by taking appropriate measures. Traditional control methods include chemical spraying and physical isolation, which are high in cost, easy to cause pollution, and difficult to solve the problem of long-term monitoring.

[0003] At present, target detection technology based on deep learning has made progress in the field of biological identification. However, mice are usually small in size and active in dark environments. When using monitoring video to detect mice, the data volume of the monitoring video is large, and the running speed is slow when batch detection is performed; the target is diversified, and in different environments, mice may exhibit different morphological and behavioral characteristics such as size and color; the light and background are complex, and low light, glare and other situations may occur, so the traditional target detection algorithm performs poorly in dark environments and detection of small targets.

[0004] Therefore, the present application uses target detection and image enhancement and denoising knowledge to design a control biological detection method that can be applied to various complex environments and conditions. Because the environment of the pig farm is complex and mice are common control organisms, the detection of mice in the pig farm will be taken as an example to specifically introduce the designed control biological detection method. SUMMARY

[0005] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a pig farm mouse detection method, system, device and medium based on deep learning, which improves the detection accuracy by frame difference method, image low light enhancement, and anti-glare, and calculates the mouse activity index of the region using the detection result.

[0006] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0007] In a first aspect, the present application provides a pig farm mouse detection method based on deep learning, comprising the following steps:

[0008] Frame extraction processing is performed on the video of the region to be detected to obtain a set of image frames;

[0009] Image preprocessing is performed on the set of image frames;

[0010] The preprocessed current frame is input into a mouse detection model to learn the appearance of mice; the mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; the frame difference prior branch uses a frame difference method to perform previous and next frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image, and convolution and normalization operations are performed on the binary image, which is then connected with the remaining features for the next step of calculation;

[0011] Set a low threshold L and a high threshold H, and use the pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Keep the detection results with a confidence level higher than the high threshold H. For detection results with a confidence level between the high threshold H and the low threshold L, only keep the detection results with 5 or more motion pixels in the bounding box.

[0012] The rat detection results are recorded. When the detection of the entire video of the detection area is completed, the number of rats recorded after each frame detection is weighted and summed to obtain the rat appearance duration to calculate the rat activity index of the detection area.

[0013] As a preferred technical solution, the video of the area to be detected is subjected to frame extraction processing to obtain an image frame set, specifically:

[0014] The video of the area to be detected is acquired in a 24-hour cycle, and the video is divided into frames. One frame is extracted every set number of image frames to obtain an image frame set.

[0015] As a preferred technical solution, the image preprocessing of the image frame set is specifically as follows:

[0016] The image frames in the image frame set are subjected to image diagnosis, low-light enhancement, glare repair and data expansion processing. Specifically, the current frame is judged. If the current frame is a low-light image, low-light enhancement is performed. If glare exists in the current frame, glare repair is performed on the glare area. Finally, data expansion is performed using the "copy and paste" method.

[0017] As a preferred technical solution, the image diagnosis, low-light enhancement, glare repair and data expansion processing are performed on the image frames in the image frame set, specifically:

[0018] First, the current frame f k Convert it to a grayscale image, traverse each pixel, count the frequency of grayscale values, and then calculate the average grayscale value of the image. Compare the average grayscale value with the set low-light threshold. If it is less than the low-light threshold, the current frame f needs to be k Perform low-light enhancement, input it into the low-light enhancement module, and perform low-light enhancement through the Retinexformer model, otherwise no low-light enhancement is performed;

[0019] The grayscale image is judged. If there is no pixel grayscale value higher than the set glare threshold, no glare repair is required; if there is a pixel grayscale value higher than the set glare threshold, the current frame is input into the glare repair module, and the Uformer model is used to repair the glare area therein to obtain the final preprocessed image μ.

[0020] The image μ is expanded using the “copy and paste” method, and the bounding box area of ​​the mouse in an original image in the dataset is copied to a random position in another random original image to generate a new sample.

[0021] As a preferred technical solution, the pre-processed current frame is input into the mouse detection model to learn the appearance of the mouse, specifically:

[0022] The current frame is passed into the frame difference prior branch and converted into a grayscale image. The specific formula is as follows:

[0023] gray=(299*r+587*g+114*b)÷1000

[0024] Among them, grey is the grayscale value of the image, r, g, and b represent the values ​​of the R, G, and B channels of each pixel in the image;

[0025] The inter-frame difference method is used to calculate the difference between the grayscale image of the current frame and the grayscale images of the previous frame and the next frame, and the absolute value of the grayscale difference is binarized by the set threshold T to obtain the binary image R k After calculation, the point with a grayscale value of 255 is the foreground point, and the point with a grayscale value of 0 is the background point. The specific formula is as follows:

[0026] D k (x,y)=|f k+1 (x,y)-f k (x,y)|∪|f k (x,y)-f k-1 (x,y)|

[0027]

[0028] Here, the k-1th frame, kth frame and k+1th frame images in the video sequence are respectively f k-1 、f k and f k+1 , the grayscale value of the pixel (x, y) of the three corresponding images is recorded as f k-1 (x,y) and f k (x,y) and f k+1 , perform differential calculation on it to obtain the differential image D of the kth frame k(x, y), and then perform binarization processing on it to obtain the binary image R k ;

[0029] Through the morphological corrosion operation, the binary image R is convolved with a 6×6 convolution kernel. k Erosion is performed to reduce the noise of the image and obtain image R k ′, and then use 25×25 convolution kernel to correct the image R k ′ is expanded to fill the outline of the object in the image and obtain the image R k ″, represents the area of ​​the moving object in the current frame;

[0030] The image R k After batch normalization, the specific formula is as follows:

[0031]

[0032] Among them, μ and σ represent the mean and standard deviation of all samples in the current batch respectively, ∈ is a small constant to avoid the denominator being zero; N is the number of pictures in the current batch, and the size of each picture is H×W. The image R is calculated k Results after batch normalization

[0033] Afterwards, Channel convolution is used to obtain adaptability of channel dimensions. The specific formula is as follows:

[0034]

[0035] Where C represents the number of input channels, w k represents the learnable weight corresponding to the k-th channel, Represents image R k The result after batch normalization is calculated to obtain the feature map F ij .

[0036] As a preferred technical solution, the pixels with a value of 255 in the binarized image obtained by the frame difference method are used as motion pixels, and the detection results with a confidence level higher than a high threshold H are retained. The following formula is used for calculation:

[0037]

[0038] Among them, L and H represent the set high threshold and low threshold respectively, c i Indicates each test result d i The confidence level, R k ″ represents the binary grayscale image after morphological processing by frame difference method. When the detection result d i The confidence level c ihigher than the high threshold H, or the confidence is between the high threshold H and the low threshold L and the bounding box corresponds to the binary gray image R k If the number of values equal to 255 in the range of the high threshold H and the low threshold L is 5 or more, the result is reserved.

[0039] As a preferred technical solution, the number of mice detected after each frame is weighted and summed to obtain the mouse appearance time, specifically:

[0040] The frames that need to be low-light enhanced are set to a weight of 1, and the remaining frames are set to a weight of 1.5, and the calculation formula is:

[0041]

[0042] Where t is the mouse appearance time of the entire area, n v represents the number of 24-hour monitoring videos in the detection area, n f represents the number of frames of each video, represents the number of mice detected in the jth frame of the ith video, f j i represents the weight corresponding to the jth frame of the ith video;

[0043] Since the number of monitoring cameras in each area is different, the mouse appearance time of the entire area is divided by the number of monitoring cameras in the area as an indicator of the mouse activity level in the area, and the calculation formula is as follows:

[0044] a=t / n c

[0045] Where a is the mouse activity level of the detection area, t is the mouse appearance time of the detection area, n c is the number of monitoring cameras in the detection area.

[0046] In a second aspect, the present application provides a pig farm mouse detection system based on deep learning, which is applied to the pig farm mouse detection method based on deep learning, and includes a frame extraction module, a preprocessing module, a mouse detection module, a mouse motion detection module, and a mouse activity calculation module.

[0047] The frame extraction module is used for frame extraction processing of the video of the detection area to obtain an image frame set.

[0048] The preprocessing module is used for image preprocessing of the image frame set.

[0049] The mouse detection module is used to input the preprocessed current frame into the mouse detection model to learn the appearance of the mouse; the mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; the frame difference prior branch uses the frame difference method to perform previous and next frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image, and performs convolution and normalization operations on the binary image, and then connects it with the remaining features for the next calculation step;

[0050] The mouse motion detection module is configured to set a low threshold L and a high threshold H, and to use pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Detection results with a confidence level higher than the high threshold H are retained. For detection results with a confidence level between the high threshold H and the low threshold L, only detection results with 5 or more motion pixels within the bounding box are retained.

[0051] The mouse activity calculation module is used to record the mouse detection results. When the detection of the video of the entire area to be detected is completed, the number of mice recorded after each frame detection is weighted and summed to obtain the mouse appearance duration to calculate the mouse activity index of the area to be detected.

[0052] In a third aspect, the present invention provides an electronic device, comprising:

[0053] at least one processor; and,

[0054] a memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the pig farm rat detection method based on deep learning.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the pig farm rat detection method based on deep learning.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. This invention uses an image preprocessing module and an image diagnosis module to determine the current frame. It performs low-light enhancement on frames requiring low-light enhancement and glare removal on frames requiring glare removal. After image enhancement and denoising, it is passed to the next module. Pig farm surveillance videos feature complex lighting and backgrounds, and diverse targets. This improves the accuracy of subsequent rat detection.

[0059] 2、The application adds a frame difference prior branch in the RT-DETR target detection model to help the model locate the position of the mouse faster and more accurately, and sets a low threshold L and a high threshold H for the detection result to retain, thereby reducing the false detection probability;

[0060] 3、The application proposes a regional mouse activity degree calculation index. The mouse appearance duration detected from the monitoring video of each camera in a region is weighted and accumulated in 24 hours as a cycle, and then divided by the number of cameras in the region to obtain the mouse activity degree index of the region. Mice like shade, and the mouse appearance duration is weighted and accumulated according to whether low-light enhancement is needed for the frame. The weight value of the frame needing low-light enhancement is 1, and the weight value of the frame not needing low-light enhancement is 1.5. Then, the weighted sum of the mouse appearance duration is divided by the number of cameras in the region, which can better reflect the severity of the mouse disaster in the region. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0062] Figure 1 is a flowchart of the pig farm mouse detection method based on deep learning of the embodiment of the present application;

[0063] Figure 2 is an algorithm flowchart of the image preprocessing module of the embodiment of the present application;

[0064] Figure 3 is a result image of the image preprocessing of the low-light frame of the embodiment of the present application;

[0065] Figure 4 is a result image of the image preprocessing of the frame with glare of the embodiment of the present application;

[0066] Figure 5 is an improved RT-DETR model diagram of the embodiment of the present application

[0067] Figure 6 is a result image of the mouse detection of the embodiment of the present application;

[0068] Figure 7 is a block diagram of the pig farm mouse detection system based on deep learning of the embodiment of the present application

[0069] Figure 8 is a structural diagram of the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0071] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0072] See also Figure 1 In one embodiment of the present application, a method for detecting rats in a pig farm based on deep learning is provided, comprising the following steps:

[0073] (1) Extract frames from the surveillance video of the area to be detected and input them into calculation frame by frame;

[0074] Furthermore, the surveillance videos of the detection area, that is, all surveillance videos of the area are collected in a 24-hour period, and the videos are frame-by-frame processed.

[0075] (2) Please refer to Figure 2 、 Figure 3 、 Figure 4 , the current frame is input into the image preprocessing module, which includes the image diagnosis module, low-light enhancement module, glare repair module and data expansion module; the image diagnosis module judges the current frame. If the current frame is a low-light image, low-light enhancement is performed. If there is glare in the current frame, glare repair is performed on the glare area to obtain the image preprocessing result; finally, the "copy and paste" method is used to perform data expansion;

[0076] Furthermore, the image preprocessing module includes an image diagnosis module, a low-light enhancement module, and a glare repair module, which improves the accuracy of detection by performing image enhancement and denoising on the current frame, specifically:

[0077] (2-1), the image diagnosis module firstly converts the current frame f k Convert it to a grayscale image, traverse each pixel, count the frequency of grayscale values, and then calculate the average grayscale value of the image and compare it with the set low-light threshold. Here, the low-light threshold is set to 75. If it is less than the low-light threshold, the current frame f needs to bek For low light enhancement, input it into the low light enhancement module and perform low light enhancement through the Retinexformer model. Otherwise, no low light enhancement is required. Figure 4 The left side is the original frame, and the right side is the image after low-light enhancement by Retinexformer. After low-light enhancement, the appearance of the mouse is clearer, which can reduce the difficulty of subsequent detection;

[0078] (2-2) The image diagnosis module judges the grayscale image. If there is no pixel grayscale value higher than the set glare threshold, no glare repair is required. If there is a pixel grayscale value higher than the set glare threshold, the current frame is input into the glare repair module and the Uformer model is used to repair the glare area. Finally, the final pre-processed image μ is obtained;

[0079] (2-3) Use the “copy and paste” method to expand the image μ, and copy the bounding box area where the mouse is located in a certain original image in the dataset to a random position in another random original image to generate a new sample.

[0080] (3) Please refer to Figure 5 The current frame is input into a mouse detection model to learn the appearance of mice. The mouse detection model uses an improved RT-DETR detection model. The improved RT-DETR detection model adds a frame difference prior branch before feature concatenation. The frame difference prior branch uses the frame difference method to perform frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image. This image is then convolved and normalized, and then concatenated with the remaining features for the next calculation.

[0081] Furthermore, the improved RT-DETR target detection model is used to detect mice because mice are small and move quickly. The improved RT-DETR target detection model is more likely to focus on the moving area in the current frame, thereby improving the detection success rate.

[0082] (3-1) Pass the current frame into the frame difference prior branch and convert the current frame into a grayscale image. The specific formula is as follows:

[0083] gray=(299*r+587*g+114*b)÷1000

[0084] Among them, grey is the grayscale value of the image, and r, g, and b represent the values ​​of the R, G, and B channels of each pixel in the image.

[0085] The inter-frame difference method is used to calculate the difference between the grayscale image of the current frame and the grayscale images of the previous frame and the next frame, and the absolute value of the grayscale difference is binarized by the set threshold T to obtain the binary image R k, the threshold is set to 10. After calculation, the point with a grayscale value of 255 is the foreground point, and the point with a grayscale value of 0 is the background point. The specific formula is as follows:

[0086] D k (x,y)=|f k+1 (x,y)-f k (x,y)|∪|f k (x,y)-f k-1 (x,y)|

[0087]

[0088] Here, the k-1th frame, kth frame and k+1th frame images in the video sequence are respectively f k-1 、f k and f k+1 , the grayscale value of the pixel (x, y) of the three corresponding images is recorded as f k-1 (x,y) and f k (x,y) and f k+1 , perform differential calculation on it to obtain the differential image D of the kth frame k (x, y), and then perform binarization processing on it to obtain the binary image R k ;

[0089] (3-3) Through the morphological corrosion operation, the binary image R is convolved with a 6×6 convolution kernel. k Erosion is performed to reduce the noise of the image and obtain image R k ′, and then use 25×25 convolution kernel to correct the image R k ′ is expanded to fill the outline of the object in the image and obtain the image R k ″, represents the area of ​​the moving object in the current frame;

[0090] (3-4), image R k After batch normalization, the specific formula is as follows:

[0091]

[0092]

[0093] Among them, μ and σ represent the mean and standard deviation of all samples in the current batch respectively, ∈ is a small constant to avoid the denominator being zero; N is the number of pictures in the current batch, and the size of each picture is H×W. The image R is calculated k Results after batch normalization

[0094] Afterwards, Channel convolution is used to obtain adaptability of channel dimensions. The specific formula is as follows:

[0095]

[0096] wherein C denotes the number of input channels, w k denotes the learnable weight corresponding to the kth channel, and F ij is obtained.

[0097] (4), please refer to Figure 6 , set low threshold L and high threshold H, the frame difference method obtained in the binary image value for 255 pixel as a motion pixel, keep the detection results of confidence higher than the high threshold H, for the confidence between the high threshold H and low threshold L, only keep the detection results of 5 or more motion pixels in the bounding box;

[0098] Further, the binary image obtained by the frame difference method is set to 255 pixel as a motion pixel, and the low threshold and high threshold are set to keep the detection results, because the pig farm environment is complex, the mouse volume is small, the movement speed is fast, and the motion blur may be generated when moving, the false detection probability is large, and the high threshold is set for the static mouse and the low threshold is set for the mouse in motion when detecting, which can reduce the possibility of false detection and missed detection, and the specific formula is:

[0099]

[0100] wherein L, H respectively represent the set high threshold and low threshold, c i represents the confidence of each detection result d i , R k " represents the binary gray image after morphological processing of the frame difference method. When the confidence c i of the detection result d i is higher than the high threshold H, or the confidence is between the high threshold H and the low threshold L, and the range of the binary gray image R k " corresponding to the bounding box has a value of 255, the number of which is 5 or more, the result is kept;

[0101] (5), check whether the detection of the entire area video has been completed, otherwise go back to (1) to continue calculation, yes, then use the number of mice recorded after each frame detection to get weighted sum to get the mouse appearance duration, to calculate the mouse activity index of the area;

[0102] Further, the mouse appearance duration refers to the weighted accumulation of the number of mice detected in each frame of the entire video in the entire area to obtain the mouse appearance duration, wherein the frames requiring low light enhancement are set to a weight value of 1, and the remaining frames are set to a weight value of 1.5, and the calculation formula is:

[0103]

[0104] Among them, t refers to the duration of mouse appearance in the entire area, n v Indicates the number of surveillance videos in the entire area for 24 hours, n f Refers to the number of frames in each video. Refers to the number of mice detected in the jth frame of the i-th video, f j i Refers to the weight corresponding to the jth frame of the i-th video.

[0105] Since the number of surveillance cameras in each area may be different, the duration of rat appearance in the entire area is divided by the number of surveillance cameras in the area as an indicator of rat activity. The calculation formula is as follows:

[0106] a=t / n c

[0107] Among them, a is the activity level of mice in the detection area, t is the duration of mice appearing in the detection area, and n c is the number of surveillance cameras in the detection area.

[0108] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0109] Based on the same concept as the deep learning-based pig farm rat detection method in the above-mentioned embodiment, the present invention also provides a deep learning-based pig farm rat detection system, which can be used to implement the above-mentioned deep learning-based pig farm rat detection method. For ease of explanation, the structural diagram of the embodiment of the deep learning-based pig farm rat detection method only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it can include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0110] See also Figure 7 In another embodiment of the present application, a pig farm rat detection system 100 based on deep learning is provided, comprising a frame extraction module 101, a pre-processing module 102, a rat detection module 103, a rat motion detection module 104, and a rat activity calculation module 105;

[0111] The frame extraction module 101 is used to extract frames from the video of the area to be detected to obtain a set of image frames;

[0112] The pre-processing module 102 is used to perform image pre-processing on the image frame set;

[0113] The mouse detection module 103 is used to input the preprocessed current frame into the mouse detection model to learn the appearance of the mouse; the mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; the frame difference prior branch uses a frame difference method to perform previous and next frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image, and performs convolution and normalization operations on the binary image, and then connects it with the remaining features for the next calculation step;

[0114] The mouse motion detection module 104 is configured to set a low threshold L and a high threshold H, and to use pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Detection results with a confidence level higher than the high threshold H are retained. For detection results with a confidence level between the high threshold H and the low threshold L, only detection results with 5 or more motion pixels within the bounding box are retained.

[0115] The mouse activity calculation module 105 is used to record the mouse detection results. When the detection of the entire video of the detection area is completed, the number of mice recorded after each frame detection is weighted and summed to obtain the mouse appearance duration to calculate the mouse activity index of the detection area.

[0116] To improve the efficiency and accuracy of mouse detection, the present invention improves the RT-DETR target detection model by adding a frame difference prior branch before feature concatenation. The frame difference prior branch uses a frame difference method to perform frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image. This image is then convolved and normalized, and then concatenated with the remaining features for the next calculation step. A low threshold L and a high threshold H are set for the calculation results. Pixels with a value of 255 in the binary image obtained by the frame difference method are considered moving pixels. Detection results with a confidence level above the high threshold H are retained. For detection results with a confidence level between the high threshold H and the low threshold L, only detection results with five or more moving pixels within the bounding box are retained.

[0117] It should be noted that the pig farm rat detection system based on deep learning of the present invention corresponds one to one with the pig farm rat detection method based on deep learning of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the pig farm rat detection method based on deep learning are applicable to the embodiment of the pig farm rat detection system based on deep learning. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0118] In addition, in the implementation of the pig farm rat detection system based on deep learning in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the pig farm rat detection system based on deep learning is divided into different program modules to complete all or part of the functions described above.

[0119] See also Figure 8 In one embodiment, an electronic device for implementing a pig farm rat detection method based on deep learning is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a pig farm rat detection program 203 based on a deep learning algorithm.

[0120] The first memory 202 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 200. Furthermore, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the pig farm rat detection program 203 based on the deep learning algorithm, but can also be used to temporarily store data that has been output or is about to be output.

[0121] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the first memory 202, as well as calling data stored in the first memory 202, to perform various functions of the electronic device 200 and process data.

[0122] Figure 8 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 7 The structure shown does not constitute a limitation on the electronic device 200 , and the electronic device 200 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0123] The pig farm rat detection program 203 based on a deep learning algorithm stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve the following:

[0124] Perform frame extraction on the video of the area to be detected to obtain a set of image frames;

[0125] performing image preprocessing on the set of image frames;

[0126] The preprocessed current frame is input into a mouse detection model to learn the appearance of mice; the mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; the frame difference prior branch uses a frame difference method to perform previous and next frame calculations on the current frame, the previous frame, and the next frame to obtain a binary image, and convolution and normalization operations are performed on the binary image, which is then connected with the remaining features for the next step of calculation;

[0127] Set a low threshold L and a high threshold H, and use the pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Keep the detection results with a confidence level higher than the high threshold H. For detection results with a confidence level between the high threshold H and the low threshold L, only keep the detection results with 5 or more motion pixels in the bounding box.

[0128] The mouse detection result is recorded, and when the detection of the video of the entire to-be-detected region is completed, the mouse appearance duration is obtained by weighted summation of the mouse number recorded after detection of each frame, so as to calculate the mouse activity index of the to-be-detected region.

[0129] Further, the modules / units of the electronic device 200, if realized in the form of software function units and sold or used as independent products, can be stored in a nonvolatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0131] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0132] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement modes and shall be included in the protection scope of the present application.

Claims

1. A pig farm rat detection method based on deep learning, characterized by: The steps include: Perform frame extraction on the video of the area to be detected to obtain a set of image frames; performing image preprocessing on the set of image frames; Input the preprocessed current frame into the mouse detection model to learn the appearance of the mouse; The mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; The frame difference prior branch uses the frame difference method to calculate the previous and next frames of the current frame, the previous frame, and the next frame to obtain a binary image, and performs convolution and normalization operations on the binary image, and then connects it with the remaining features for the next step of calculation; The pre-processed current frame is input into the mouse detection model to learn the appearance of the mouse, specifically: The current frame is passed into the frame difference prior branch and converted into a grayscale image. The specific formula is as follows: gray=(299*r+587*g+114*b)÷1000 Among them, grey is the grayscale value of the image, r, g, and b represent the values ​​of the R, G, and B channels of each pixel in the image; The inter-frame difference method is used to calculate the difference between the grayscale image of the current frame and the grayscale images of the previous frame and the next frame, and the absolute value of the grayscale difference is binarized by the set threshold T to obtain the binary image R k After calculation, the point with a grayscale value of 255 is the foreground point, and the point with a grayscale value of 0 is the background point. The specific formula is as follows: D k (x,y)=|f k+1 (x,y)-f k (x,y)|∪|f k (x,y)-f k-1 (x,y)| Here, the k-1th frame, kth frame and k+1th frame images in the video sequence are respectively f k-1 、f k and f k+1 , the grayscale value of the pixel (x, y) of the three corresponding images is recorded as f k-1 (x,y) and f k (x,y) and f k+1 , perform differential calculation on it to obtain the differential image D of the kth frame k (x, y), and then perform binarization processing on it to obtain the binary image R k ; Through the morphological corrosion operation, the binary image R is convolved with a 6×6 convolution kernel. k Erosion is performed to reduce the noise of the image and obtain image R k ′, and then use 25×25 convolution kernel to correct the image R k ′ is expanded to fill the outline of the object in the image and obtain the image R k ″, represents the area of ​​the moving object in the current frame; The image R k After batch normalization, the specific formula is as follows: Among them, μ and σ represent the mean and standard deviation of all samples in the current batch respectively, ∈ is a small constant to avoid the denominator being zero; N is the number of pictures in the current batch, and the size of each picture is H1×W. The image R is calculated k Results after batch normalization Afterwards, Channel convolution is used to obtain adaptability of channel dimensions. The specific formula is as follows: Where C represents the number of input channels, w k represents the learnable weight corresponding to the k-th channel, Represents image R k The result after batch normalization is calculated to obtain the feature map F ij ; Set a low threshold L and a high threshold H, and use the pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Keep the detection results with a confidence level higher than the high threshold H. For detection results with a confidence level between the high threshold H and the low threshold L, only keep the detection results with 5 or more motion pixels in the bounding box. The rat detection results are recorded. When the detection of the entire video of the detection area is completed, the number of rats recorded after each frame detection is weighted and summed to obtain the rat appearance duration to calculate the rat activity index of the detection area.

2. The pig farm rat detection method based on deep learning according to claim 1 is characterized in that: The video of the area to be detected is subjected to frame extraction processing to obtain an image frame set, specifically: The video of the area to be detected is acquired in a 24-hour cycle, and the video is divided into frames. One frame is extracted every set number of image frames to obtain an image frame set.

3. The pig farm rat detection method based on deep learning according to claim 1 is characterized in that: The image frame set is subjected to image preprocessing, specifically: Perform image diagnosis, low-light enhancement, glare repair, and data expansion on the image frames in the image frame set. Specifically, the current frame is judged. If the current frame is a low-light image, low-light enhancement is performed. If glare exists in the current frame, glare repair is performed on the glare area. Finally, data expansion is performed using the "copy and paste" method.

4. The pig farm rat detection method based on deep learning according to claim 3 is characterized in that: The image diagnosis, low-light enhancement, glare repair and data expansion processing are performed on the image frames in the image frame set, specifically: First, the current frame f k Convert it to a grayscale image, traverse each pixel, count the frequency of grayscale values, and then calculate the average grayscale value of the image. Compare the average grayscale value with the set low-light threshold. If it is less than the low-light threshold, the current frame f needs to be k Perform low-light enhancement, input it into the low-light enhancement module, and perform low-light enhancement through the Retinexformer model, otherwise no low-light enhancement is performed; The grayscale image is judged. If there is no pixel grayscale value higher than the set glare threshold, no glare repair is required. If there is a pixel grayscale value higher than the set glare threshold, the current frame is input into the glare repair module, and the Uformer model is used to repair the glare area therein to obtain the final preprocessed image μ; The image μ is expanded using the "copy and paste" method, and the bounding box area of ​​the mouse in an original image in the dataset is copied to a random position in another random original image to generate a new sample.

5. The pig farm rat detection method based on deep learning according to claim 1 is characterized in that: The pixels with a value of 255 in the binarized image obtained by the frame difference method are regarded as motion pixels, and the detection results with a confidence level higher than the high threshold H are retained. The following formula is used for calculation: Among them, L and H represent the set high threshold and low threshold respectively, c i Indicates each test result d i The confidence level, R k ″ represents the binary grayscale image after morphological processing by the frame difference method; when the detection result d i The confidence level c i Higher than the high threshold H, or the confidence is between the high threshold H and the low threshold L and the bounding box corresponds to the binary grayscale image R k If there are 5 or more pixels with a value of 255 in the range of ", the result is retained.

6. The pig farm rat detection method based on deep learning according to claim 1 is characterized in that: The duration of mouse appearance is obtained by weighted summing the number of mice recorded after each frame detection, specifically: The weight of the frame that needs low-light enhancement is set to 1, and the weight of the remaining frames is set to 1.

5. The calculation formula is: Among them, t refers to the duration of mouse appearance in the entire area, n v Indicates the number of 24-hour surveillance videos in the area to be detected, n f Refers to the number of frames in each video. Refers to the number of mice detected in the jth frame of the i-th video, f j i Refers to the weight corresponding to the jth frame of the i-th video; Since the number of surveillance cameras in each area is different, the duration of rat appearance in the entire area is divided by the number of surveillance cameras in the area to serve as an indicator of the rat activity level in the area. The calculation formula is as follows: a=t / n c Among them, a is the activity level of mice in the detection area, t is the duration of mice appearing in the detection area, and n c is the number of surveillance cameras in the detection area.

7. Pig farm rat detection system based on deep learning, characterized by: A pig farm rat detection method based on deep learning applied to any one of claims 1-6, comprising a frame extraction module, a preprocessing module, a rat detection module, a rat motion detection module, and a rat activity calculation module; The frame extraction module is used to extract frames from the video of the area to be detected to obtain a set of image frames; The preprocessing module is used to perform image preprocessing on the image frame set; The mouse detection module is used to input the pre-processed current frame into the mouse detection model to learn the appearance of the mouse; The mouse detection model is obtained using an improved RT-DETR detection model; the improved RT-DETR detection model adds a frame difference prior branch before feature connection; The frame difference prior branch uses the frame difference method to calculate the previous and next frames of the current frame, the previous frame, and the next frame to obtain a binary image, and performs convolution and normalization operations on the binary image, and then connects it with the remaining features for the next step of calculation; The mouse motion detection module is configured to set a low threshold L and a high threshold H, and to use pixels with a value of 255 in the binarized image obtained by the frame difference method as motion pixels. Detection results with a confidence level higher than the high threshold H are retained. For detection results with a confidence level between the high threshold H and the low threshold L, only detection results with 5 or more motion pixels within the bounding box are retained. The mouse activity calculation module is used to record the mouse detection results. When the detection of the video of the entire area to be detected is completed, the number of mice recorded after each frame detection is weighted and summed to obtain the mouse appearance duration to calculate the mouse activity index of the area to be detected.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the pig farm rat detection method based on deep learning as described in any one of claims 1-6.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by the processor, the pig farm rat detection method based on deep learning as described in any one of claims 1 to 6 is implemented.

Images