Image detection method, device and equipment for barn pests

By applying YOLOv7 model and image enhancement technology in the granary, combined with dynamic biological detection, real-time high-precision monitoring of pests in the granary is achieved, the problem of not being able to detect pests in the existing technology is solved, reducing food losses and improving management intelligence level of transformation.

CN118918527AInactive Publication Date: 2024-11-08JINLING INST OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410961864.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing technology cannot achieve real-time, comprehensive and accurate monitoring of pests in granaries, resulting in the inability to detect and deal with pests in the early stages of pest invasion, resulting in serious food losses and quality declines.

Method used

Using the granary pest detection method based on the YOLOv7 model, the sensitivity to small-scale targets and perception of target spatial position information is improved by integrating ACmix adaptive convolutional separation module and CBAM attention module, and combined with denoising algorithm and contrast Image enhancement technologies such as stretching and histogram equalization, combined with dynamic biological detection and adjacent frame image detection algorithms, realize 24-hour uninterrupted monitoring and high-precision recognition.

Benefits of technology

Real-time monitoring and high-precision identification of pests in the granary are realized, misjudgment is reduced, detection accuracy and system robustness of mobile pests are improved, the safety of grain storage is ensured, and the intelligent level of granary management is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918527A_ABST
    Figure CN118918527A_ABST
Patent Text Reader

Abstract

The invention provides an image detection method, device and equipment for granary pests, and relates to the technical field of image processing, computer vision, artificial intelligence and the Internet of Things. Pests possibly existing in a granary are recognized through an image detection algorithm based on a single-stage detector, the activity condition of the pests in the granary can be monitored in real time, and the detection accuracy of the pests in the granary is improved. Through image acquisition, processing and analysis, pests are effectively identified and positioned, so that the safety of grain storage is guaranteed, grain loss is reduced, the intelligent level of granary management is improved, and the method can be widely applied to granary storage management, agricultural production, food safety detection and other scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of granary pest detection, and in particular to an image detection method, device and equipment for granary pests. Background Art

[0002] Granaries are important facilities for national grain reserves. Ensuring the safe storage of grain in granaries is of great significance to national food security, social stability and economic development. As a basic livelihood security resource, the quality and safety of grain are directly related to the quality of life of the people and the national food security strategy. Ensuring that grain stored in granaries is not infested by pests can not only reduce food losses, but also maintain the quality of grain and prevent food security problems caused by pests.

[0003] Since the granary environment stores a large amount of grain all year round, the temperature and humidity are relatively suitable, which makes it easy for various pests to breed, such as rats, cockroaches, beetles, etc. These pests will not only eat the grain directly, but also reproduce and spread in the grain, produce pathogens and toxins, and further contaminate the grain. Especially in southern my country, the humid climate is more conducive to the reproduction and growth of pests. Therefore, how to effectively monitor and prevent pests in granaries has become a key issue in ensuring food security.

[0004] In my country, most granaries still rely on traditional manual inspection and physical capture methods, which are unable to achieve real-time, comprehensive and accurate pest monitoring. This results in the inability to detect and deal with pests in the early stages of invasion. By the time the problem is exposed, it has often caused serious food losses and quality degradation. The application of pest detection systems based on image processing and artificial intelligence technology in granaries is still in the exploratory stage, with low market penetration and a lack of mature solutions and successful cases.

[0005] After searching, the Chinese invention patent with application number CN2024103189752 discloses a stored-grain pest detection method based on the YOLOv7 model, which involves the field of image target detection and includes the following steps: S1: photograph and collect image information of stored-grain pests and establish a stored-grain pest dataset; S2: establish a pest detection model based on YOLOv7; S3: train the pest detection model according to the stored-grain pest dataset; S4: input the image to be tested into the trained pest detection model to obtain the detection result. The present invention improves the sensitivity to small-scale targets and reduces the influence of noise by integrating the ACmix adaptive convolution separation module and the CBAM attention module; improves the perception ability of the target spatial position information; and reduces the degree of freedom of the loss function by improving the loss function, thereby improving the robustness of the network. High-precision recognition and positioning can be achieved under local occlusion, motion blur and impurity adhesion conditions, and the performance is better than other existing models, which can provide a reference for intelligent monitoring of stored-grain pests.

[0006] The technical differences between the present invention and the above-mentioned reference documents are as follows:

[0007] 1. A method for detecting stored-grain pests based on the YOLOv7 model uses an improved YOLOv7 model for stored-grain pest detection. By integrating the ACmix adaptive convolution separation module and the CBAM attention module, the sensitivity to small-scale targets is improved and the impact of noise is reduced; the perception of the target's spatial position information is improved, and the degree of freedom of the loss function is reduced by improving the loss function to improve the robustness of the network. High-precision recognition and positioning can be achieved under local occlusion, motion blur, and impurity adhesion. The patent for granary pests uses 24-hour uninterrupted monitoring, real-time video data collection through the camera, and uses image enhancement technologies such as denoising algorithms, contrast stretching, and histogram equalization, combined with an image detection algorithm based on a single-stage detector for pest identification. The similarity is confirmed by dynamic biological detection and adjacent frame image detection (inter-frame gap algorithm) to improve the accuracy of mobile pest detection and the robustness of the system, and an alarm signal is generated and sent to the control center.

[0008] 2. A stored grain pest detection method based on the YOLOv7 model improves the detection capability of small-scale targets by improving the YOLOv7 model and feature processing module, and maintains high accuracy and robustness in complex environments. The granary pest patent improves the intelligence level and detection accuracy through all-weather monitoring and dynamic biological detection, combined with a variety of image processing technologies and inter-frame gap algorithms. In addition, the granary pest patent pays special attention to real-time and mobile target detection, and identifies and locates pests through the collection and processing of real-time video data.

[0009] After searching, the Chinese invention patent with application number CN2021109945213 involves a method and system for intelligent identification and positioning of agricultural pests. The method includes collecting agricultural pest images by data crawling and real-time image shooting; performing data normalization, data enhancement processing and data labeling on the collected agricultural pest images; establishing an orchard pest target detection model based on YOLOv4; training and accuracy evaluation of the orchard pest target detection model based on YOLOv4 to obtain an evaluated orchard pest target detection model based on YOLOv4; arbitrarily selecting an agricultural pest image and inputting it into the evaluated orchard pest target detection model based on YOLOv4 to identify and locate agricultural pests. The method has high detection accuracy and efficiency for agricultural pests. The method performs well in a variety of test scenarios and has good robustness.

[0010] The technical differences between the present invention and the above-mentioned reference documents are as follows:

[0011] 1. A method and system for intelligent identification and positioning of agricultural pests uses visual saliency algorithms and deep learning models, uses high-resolution cameras to regularly collect images, accurately locates pests through salient area analysis, and combines deep learning algorithms to improve detection accuracy and efficiency. The patent for granary pests uses 24-hour uninterrupted monitoring, collects video data in real time through cameras, uses image enhancement technologies such as denoising algorithms, contrast stretching and histogram equalization, and combines image detection algorithms based on single-stage detectors to identify pests. The similarity is confirmed through dynamic biological detection and adjacent frame image detection (inter-frame gap algorithm) to improve the accuracy of mobile pest detection and the robustness of the system, and an alarm signal is generated and sent to the control center.

[0012] 2. A method and system for intelligent identification and positioning of agricultural pests uses data crawlers and real-time image shooting to collect agricultural pest images; normalizes, enhances and annotates the collected agricultural pest images; establishes an orchard pest target detection model based on YOLOv4; trains and evaluates the accuracy of the model to obtain an evaluated orchard pest target detection model based on YOLOv4. Randomly select an agricultural pest image and input it into the evaluated model for agricultural pest identification and positioning. The patent for granary pests uses 24-hour uninterrupted monitoring, collects video data in real time through a camera, uses image enhancement technologies such as denoising algorithms, contrast stretching and histogram equalization, and combines an image detection algorithm based on a single-stage detector to identify pests. The similarity is confirmed by dynamic biological detection and adjacent frame image detection (inter-frame gap algorithm) to improve the accuracy of mobile pest detection and the robustness of the system, and generates an alarm signal to send to the control center.

[0013] 3. A method and system for intelligent identification and positioning of agricultural pests is to conduct regular detection and precise positioning through significant area analysis and deep learning algorithms, while the patent for granary pests is to improve the level of intelligence and detection accuracy through all-weather monitoring and dynamic biological detection, combined with a variety of image processing technologies and inter-frame gap algorithms. In addition, the patent for granary pests pays special attention to real-time and mobile target detection, and identifies and locates pests through the collection and processing of real-time video data.

[0014] 4. A method and system for intelligent identification and positioning of agricultural pests collects image data through data crawling and real-time image shooting, and improves the detection accuracy of the model through data normalization and enhancement processing. The patent for granary pests improves the level of intelligence and detection accuracy through all-weather monitoring and dynamic biological detection, combined with a variety of image processing technologies and inter-frame gap algorithms. In addition, the patent for granary pests pays special attention to real-time and mobile target detection, and identifies and locates pests through the collection and processing of real-time video data.

[0015] After searching, the Chinese invention patent with application number CN2020116185268 provides a method and device for detecting grain pests based on visual saliency, the method comprising: obtaining a current image to be detected; inputting the current image to be detected into a pre-trained grain pest detection model, and outputting a visual saliency map of grain pests, wherein the grain pest detection model is a reverse cascade feature aggregation neural network, including a backbone network, a cascaded hole convolution module, and a reverse feature aggregation module. The present invention uses visual saliency for the detection of small targets such as grain pests, increases the convolution receptive field through the cascaded hole convolution module, and enhances and enriches the feature representation of salient small targets through the reverse cascade feature aggregation module, so that the detection results of grain pests are more accurate.

[0016] The technical differences between the present invention and the above-mentioned reference documents are as follows:

[0017] 1. A grain pest detection method and device based on visual saliency uses the YOLOv5s algorithm for stored-grain pest detection. The small target detection capability is improved by optimizing the YOLOv5s model, and images are collected regularly using a high-resolution camera. The optimized model is obtained through data enhancement processing and backbone network lightweight processing. The patent for granary pests uses 24-hour uninterrupted monitoring, real-time video data collection through cameras, and image enhancement technologies such as denoising algorithms, contrast stretching and histogram equalization are used to identify pests in combination with an image detection algorithm based on a single-stage detector. The similarity is confirmed through dynamic biological detection and adjacent frame image detection (inter-frame gap algorithm) to improve the accuracy of mobile pest detection and the robustness of the system, and an alarm signal is generated and sent to the control center.

[0018] 2. A grain pest detection method and device based on visual saliency improves the accuracy and efficiency of regular detection by optimizing the YOLOv5s model and data enhancement processing, while the granary pest patent improves the intelligence level and detection accuracy by combining various image processing technologies and inter-frame gap algorithms through all-weather monitoring and dynamic biological detection. In addition, the granary pest patent pays special attention to real-time and mobile target detection, and identifies and locates pests through the collection and processing of real-time video data. Summary of the invention

[0019] In order to solve the above technical problems and improve the accuracy and efficiency of granary pest detection, the present invention proposes an image detection method, device and equipment for granary pests, which can monitor the activities of pests in the granary in real time, and effectively identify and locate pests through image acquisition, processing and analysis, thereby ensuring the safety of grain storage, reducing grain losses, and improving the intelligence level of granary management.

[0020] To achieve the above object, the technical solution adopted by the present invention is:

[0021] The image detection method of granary pests is characterized by comprising the following steps:

[0022] S101, obtaining an image to be processed:

[0023] The camera is used to capture the images in the granary in real time, and the captured images are enhanced. The enhanced images to be processed are stored and wait for the pest detection model to detect them.

[0024] S102: Put the image to be processed into the pest detection model for processing to obtain multiple candidate boxes and confidence levels:

[0025] Build a pest detection model, and input the enhanced image to be processed into the pest detection model for frame-by-frame detection, obtain multiple candidate detection frames and confidence information corresponding to the candidate detection frames, find the picture suspected of containing pests in the candidate detection frames and confidences, and mark the candidate frames and confidences of the suspected pests;

[0026] S103: If a possible harmful organism is identified in the image, the image of the adjacent frame of the image to be processed is obtained and put into the pest detection model for detection:

[0027] Obtain adjacent frame images of the image to be processed. If the pest detection model detects that there are suspected pests in the image to be processed, the adjacent frame images of the image to be processed will be sent to the pest detection model for detection, and the position candidate frame and confidence of the suspected pest will be marked;

[0028] S104, determining the location information of harmful organisms in adjacent frame images as the first image:

[0029] Marking the location information of the pests in the adjacent frame images as the first image, including the candidate frame of the pest location, the confidence, the type of the pest, and the time when the pest appeared;

[0030] S105, determining the location information of harmful organisms in the image to be processed as the second image:

[0031] Marking the location information of the pests in the image to be processed as a second image, including a candidate frame of the pest location, a confidence level, the type of the pest, and the time when the pest appeared;

[0032] S106: Compare the similarity between the first image and the second image. If the similarity is less than a threshold, it indicates that harmful organisms do exist. If the similarity is greater than the threshold, it indicates that harmful organisms do not exist.

[0033] By comparing the candidate frame positions of the organisms in the first image and the second image, the similarity is obtained. If the similarity is greater than a threshold, it is considered a misjudgment, that is, the identified organism is not a pest. If the similarity is less than the threshold, it is considered that the identified organism is indeed a pest.

[0034] As a preferred technical solution of the present invention: in step S101, the image to be processed is obtained including the following processing steps:

[0035] First, obtain the video stream information to be processed through the camera;

[0036] Then, the frame extraction frequency information is determined according to the moving speed of the organism to be monitored, and the organism to be monitored is the pest appearing in the monitoring area corresponding to the video stream information to be processed;

[0037] Finally, according to the frame extraction frequency information, the video stream information to be processed is subjected to frame extraction processing to obtain the image to be processed.

[0038] As the preferred technical solution of the present invention: the frame extraction frequency is determined as follows: when the camera captures a moving organism, it will not immediately send it to the pest detection model for detection, but will first detect its movement speed and determine the frame extraction frequency based on its movement speed. The α, β and γ parameters can be flexibly adjusted according to specific environmental conditions and detection requirements to achieve the best detection effect. Through appropriate parameter adjustment, it can be ensured that the system can maintain an efficient processing speed while ensuring the accuracy and reliability of detection when detecting pests. The specific formula is as follows:

[0039]

[0040] The speed of the creature v: detects the speed of the creature moving in the picture, in pixels / second.

[0041] Camera frame rate f: the number of frames captured by the camera per second, in frames per second.

[0042] Expected biological movement distance d between adjacent frames: the pixel distance that the biological is expected to move between adjacent frames. The larger the value, the more the biological movement between adjacent frames. Conversely, the smaller the value, the less the biological movement between adjacent frames.

[0043] α: Adjusts the effect of biological movement speed. By adjusting α, you can amplify or reduce the effect of biological movement speed on the frame rate.

[0044] β: Adjusts the effect of expected movement distance. By adjusting β, you can optimize the effect of creature movement distance on frame rate.

[0045] γ: Adjust the impact of the camera frame rate. By adjusting γ, you can change the impact of the camera frame rate on the frame extraction frequency.

[0046] As a preferred technical solution of the present invention: in step S103, the pest detection model is trained by optimizing the loss function, and the optimization loss function formula is as follows:

[0047]

[0048] in:

[0049] IoU: The intersection over union of the predicted box and the true box, used to measure the degree of overlap between the predicted box and the true box.

[0050] It is used to consider the distance between the center point of the predicted box and the real box. The larger the distance, the greater the loss.

[0051] c: The diagonal length of the minimum enclosing area containing the predicted box and the true box,

[0052] v: used to measure the consistency of the aspect ratio between the predicted box and the real box,

[0053]

[0054] This term is used to measure the difference in aspect ratio between the predicted box and the true box:

[0055] a: used to balance the contribution of IoU and aspect ratio consistency terms to the total loss,

[0056]

[0057] As a preferred technical solution of the present invention: in step S103, before the image to be processed is placed into the pest detection model, the image to be processed is processed, specifically as follows: first, the color space mode of the image to be processed is converted to obtain the converted image to be processed, and then, the image to be processed is format converted to obtain the image to be processed after format conversion, wherein the format conversion processing includes: normalization processing and image size adjustment processing.

[0058] As a preferred technical solution of the present invention: in step S106, first histogram information of the first image is extracted, second histogram information of the second image is extracted, and the similarity between the first image and the second image is determined by comparing the first histogram information and the second histogram information.

[0059] In the above structure: the image detection method of granary pests proposed in the present invention can monitor the activities of pests in the granary in real time, and effectively identify and locate pests through image acquisition, processing and analysis, thereby ensuring the safety of grain storage, reducing grain losses, and improving the intelligence level of granary management.

[0060] The image to be processed and the shooting time are obtained through the camera installed in the granary, and then the image to be processed is enhanced and input into the pest detection model for image processing to obtain the image processing result; the image processing result includes the location information and type of the pests. The pest detection model can identify the pests in the image to be monitored and clearly mark their location information and type, which is very convenient.

[0061] If the image processing result indicates the presence of pests, then adjacent frame images adjacent to the image to be processed are obtained, and the similarity between the adjacent frame images and the pests in the image to be processed is compared to further determine whether there are pests in the image to be processed, thereby improving the reliability of image recognition.

[0062] After obtaining the information of the pests through comparison, the image of the pests and the time and other information are uploaded to the server to remind the administrator to go and deal with it.

[0063] To enhance the clarity of images captured by the camera, make adjustments in the following ways:

[0064] According to the lighting conditions of the granary, histogram equalization is performed. By adjusting the distribution of the image's histogram and enhancing the contrast, the obtained image can be made clearer.

[0065] Apply a denoising algorithm, such as mean filtering, median filtering, or Gaussian filtering, to reduce noise in the image.

[0066] Adjust the camera exposure time and extend the camera exposure time to capture more light and enhance the clarity of the captured pictures.

[0067] The image detection method is combined with the camera, and the entire workflow of the camera from shooting to alarm in the present invention is described in detail:

[0068] S201, obtaining the image to be processed and the shooting time,

[0069] The camera continuously shoots the contents of the granary 24 hours a day. Since the granary is usually in dim light, the exposure time can be appropriately extended according to the light conditions to obtain a clearer picture. This is also the initial picture enhancement performed by the camera settings.

[0070] S202, determine the frame extraction frequency,

[0071] When the camera captures a moving creature, it will not immediately send it to the pest detection model for detection. Instead, it will first detect its movement speed and determine the frame extraction frequency based on its movement speed to facilitate the subsequent extraction of adjacent frame images of the image to be processed.

[0072]

[0073] 1. The speed of the organism v: The speed at which the organism is detected in the picture, in pixels per second.

[0074] 2. Camera frame rate f: the number of frames captured by the camera per second, in frames per second.

[0075] 3. Expected object movement distance d between adjacent frames: The pixel distance that the organism is expected to move between adjacent frames. The larger the value, the more the organism moves between adjacent frames, which is suitable for high-speed moving objects. On the contrary, the smaller the value, the less the organism moves between adjacent frames, which is suitable for low-speed moving objects.

[0076] 4. α: Adjust the effect of biological movement speed. By adjusting α, you can magnify or reduce the effect of biological movement speed on the frame rate.

[0077] 5. β: Adjust the effect of expected movement distance. By adjusting β, the effect of creature movement distance on frame rate can be optimized.

[0078] 6.γ: Adjust the impact of camera frame rate. By adjusting γ, you can change the impact of camera frame rate on frame extraction frequency.

[0079] S203, image preprocessing and enhancement,

[0080] After obtaining the images captured by the camera, we first preprocess the images and adjust the size and format of the images to make all the images uniform for subsequent processing. After the images are uniform, we normalize the images. The formula is as follows:

[0081]

[0082] Among them: I is the pixel value of the original image, Imin is the minimum pixel value of the original image, Imax is the maximum pixel value of the original image, I′ is the normalized pixel value, λ is the offset constant used to adjust the normalization process, and k is a small constant to prevent division by zero errors. Through this formula, the pixel value of the image can be normalized to the range of [0,1], thereby improving the performance of subsequent image processing and analysis algorithms. At the same time, λ and k are introduced to enhance the flexibility and robustness of the image normalization process, avoid division by zero errors, adjust the normalization range, and enhance contrast;

[0083] The technology of image smoothing by using Gaussian filtering through weighted average method can effectively retain the edge information of the image while removing image noise. The two-dimensional form of Gaussian function is as follows:

[0084]

[0085] in:

[0086] 1. The value of the Gaussian function at point (x, y),

[0087] 2. Standard deviation, which determines the width of the Gaussian function and the smoothness of the filter,

[0088] 3. Relative position to the filter center,

[0089] For each pixel in the image, take a (2k+1)×(2k+1) neighborhood with it as the center, use the Gaussian kernel G to perform weighted averaging on the neighborhood, and calculate the new pixel value. The specific operation formula is as follows:

[0090]

[0091] in:

[0092] 1. I(x, y) is the pixel value of the original image at position (x, y),

[0093] 2. I′(x, y) is the pixel value of the filtered image at position (x, y),

[0094] 3. G(i, j) is the value of the Gaussian kernel at position (i, j),

[0095] 4.k is the radius of the window, and the window size is (2k+1)×(2k+1),

[0096] Histogram equalization adjusts the contrast of the image to make the grayscale value distribution of the image more uniform, enhance the overall contrast of the image, highlight the details in the image, and make the visual effect more obvious. The steps are as follows:

[0097] 1. Calculate the original histogram:

[0098]

[0099] Let I be the original image, with L gray levels (usually 256, that is, the gray value range is 0 to 255), calculate the frequency n of each gray level k , that is, the number of pixels with gray value k. γ (k) is the probability of gray value k, N is the total number of pixels in the image,

[0100] 2. Calculate the cumulative distribution function (CDF):

[0101]

[0102] Cumulative distribution function C(r k ) is the cumulative probability of gray value from 0 to k,

[0103] 3. Calculate the gray value after equalization:

[0104] Linearly transform the cumulative distribution function to a new grayscale range (0 to L-1),

[0105] S k =(L-1)·C(r k ) (10)

[0106] Where S k is the gray value after equalization,

[0107] 4. Map to new image:

[0108] Map the grayscale value r of each pixel in the original image to the new grayscale value S to generate a balanced image.

[0109] S204: Detecting the enhanced image to be processed

[0110] The enhanced image to be processed is put into the pest detection model for detection. The specific detection process is as follows: Figure 1 As shown, by calling the detection model, the detection result is obtained;

[0111] S205, Alarm prompt

[0112] If the model detects that pests do exist in the image to be processed, an alarm message will be sent to the administrator, including the type of pests and the time when the pests appeared.

[0113] An image detection device for pests in a granary is characterized in that it includes a camera, wherein the camera is provided with an image acquisition module, an image enhancement module, an image detection module, a determination module and a prompt module, wherein the image acquisition module is used to photograph the granary in real time, the image acquisition mode is connected to the image enhancement module, the image enhancement module is used to enhance the video image photographed by the image acquisition module, the image enhancement module is connected to the image detection module, the image detection module is used to detect and process pre-selected boxes and confidences in the image to be processed and the adjacent frame images, the image detection module is connected to the determination module, the determination module is used for the similarity between the first image and the second image, the determination module is connected to the prompt module, the prompt module is used to prompt the administrator that pests appear in the granary, and when the image to be processed is processed and it is confirmed that there are pests in the image to be processed, an alarm message is sent to the administrator, and the alarm message includes the type of pests and the time of appearance.

[0114] As a preferred technical solution of the present invention: the image detection module includes a first processing unit, a first judgment unit and a second processing unit, wherein:

[0115] First processing unit: When the image to be processed is placed in the pest detection module, the first processing unit draws a pre-selected box and confidence level of the pest.

[0116] The first judgment unit: judges whether there are suspected pests in the picture according to the pre-selected box and confidence of the pests, and whether it is necessary to perform pest detection on the adjacent frames of the image to be processed.

[0117] The second processing unit: when the adjacent frame images of the image to be processed are placed into the pest detection module, the second processing unit draws the pre-selection box and confidence of the pest;

[0118] The determination module includes a first determination unit, a second determination unit, a third determination unit and a fourth determination unit, wherein:

[0119] The first determination unit: determines the candidate frames and confidences of the pests in the adjacent frame images of the image to be processed,

[0120] The second determination unit: determines the candidate frame and confidence of the pest in the image to be processed,

[0121] The third determination unit includes a first extraction unit, a second extraction unit and a first determination subunit, wherein:

[0122] The first extraction unit is used to extract the first histogram information of the first image and obtain the distribution of each grayscale pixel value in the first image.

[0123] The second extraction unit is used to extract the second histogram information of the second image and obtain the distribution of each grayscale pixel value in the second image.

[0124] The first determination subunit: confirms the similarity based on the information of the first histogram and the second histogram. The two histograms are compared using the Euclidean formula, which is as follows:

[0125]

[0126] Where h1 and h2 are the histograms of the two images respectively, i is the bin index of the histogram,

[0127] The similarity is confirmed by calculating the "straight-line" distance between the histogram vectors of the two images. The smaller the value, the more similar the histograms of the two images are;

[0128] The fourth determining unit includes a second determining subunit and a first deleting module, wherein:

[0129] The second determination subunit is used to determine whether there are pests according to the similarity between the two images; if there are pests, the subsequent first deletion unit and prompt module are executed, otherwise nothing is executed.

[0130] The first deletion module: Use non-maximum suppression to delete redundant detection results in the candidate detection frame and only retain the detection frame that is most likely to contain the target.

[0131] In the above structure: the image detection device for granary pests proposed by the present invention includes a camera, in which an image acquisition module, an image enhancement module, an image detection module, a determination module and a prompt module are arranged, wherein

[0132] Image acquisition module: This module is used to continuously shoot video surveillance inside the granary 24 hours a day, and input the obtained video stream into the image enhancement module for processing;

[0133] Image enhancement module: This module is used to enhance the video images directly captured by the camera, including camera enhancement and algorithm enhancement. Since granaries are usually in a dry, ventilated and low-temperature environment, the lighting conditions in granaries are usually poor. Depending on the situation, the camera exposure time can be extended to obtain a clearer picture. This is camera enhancement.

[0134] After obtaining the enhanced image by the camera, the algorithm will enhance it again. First, the image is preprocessed to adjust the size and format of the image. After unifying the image standards, the image is normalized, denoised, and histogram equalized to enhance the overall details and contrast of the image.

[0135] The image detection module includes a first processing unit, a first judgment unit and a second processing unit, wherein:

[0136] First processing unit: When the image to be processed is placed in the pest detection module, the first processing unit draws a pre-selected box and confidence level of the pest.

[0137] The first judgment unit: judges whether there are suspected pests in the picture according to the pre-selected box and confidence of the pests, and whether it is necessary to perform pest detection on the adjacent frames of the image to be processed.

[0138] Second processing unit: When the adjacent frame images of the image to be processed are placed into the pest detection module, the second processing unit draws the pre-selection box and confidence of the pest.

[0139] The determination module includes a first determination unit, a second determination unit, a third determination unit and a fourth determination unit, wherein:

[0140] The first determination unit: determines the candidate frames and confidences of the pests in the adjacent frame images of the image to be processed,

[0141] The second determination unit: determines the candidate frame and confidence of the pest in the image to be processed,

[0142] The third determination unit includes a first extraction unit, a second extraction unit and a first determination subunit, wherein:

[0143] The first extraction unit is used to extract the first histogram information of the first image and obtain the distribution of each grayscale pixel value in the first image.

[0144] The second extraction unit is used to extract the second histogram information of the second image and obtain the distribution of each grayscale pixel value in the second image.

[0145] The first determination subunit: confirms the similarity based on the information of the first histogram and the second histogram. The two histograms are compared using the Euclidean formula, which is as follows:

[0146]

[0147] Where h1 and h2 are the histograms of the two images respectively, i is the bin index of the histogram,

[0148] The similarity is confirmed by calculating the "straight-line" distance between the histogram vectors of the two images. The smaller the value, the more similar the histograms of the two images are;

[0149] The fourth determining unit includes a second determining subunit and a first deleting module, wherein:

[0150] The second determination subunit is used to determine whether there are pests according to the similarity between the two images; if there are pests, the subsequent first deletion unit and prompt module are executed, otherwise nothing is executed.

[0151] The first deletion module: Non-Maximum Suppression (NMS) is used to reduce redundant detection results in candidate detection frames, and only the detection frames that are most likely to contain the target are retained. NMS simplifies the final detection results by suppressing detection frames with high overlap and low confidence. In the present invention, the use of the NMS mechanism can effectively reduce redundant detection frames, ensure the accuracy of the detection results, and improve the performance and efficiency of the pest detection system. After the deletion of this module, the location, confidence, type and other information of the pests in the image are finally confirmed.

[0152] Prompt module: This module is used to prompt the administrator that pests have appeared in the granary. When the image to be processed has been processed by all the above modules and the presence of pests in the image is confirmed, an alarm message will be sent to the administrator. The alarm message includes the type of pests and the time of appearance.

[0153] The image detection device for granary pests is characterized by: comprising:

[0154] At least one camera: the camera is used to collect and store the images to be processed in the granary.

[0155] A development board, comprising a main board, a memory and a bus for connecting the main board and the memory,

[0156] The mainboard includes at least one central processor for performing pest detection on the image to be processed and the adjacent frame images. The pest detection model is obtained from the memory, and the memory is used to store computer instructions and the pest detection model.

[0157] The storage is installed on a mainboard, and the mainboard is connected to the camera for communication and is used for receiving and processing image data.

[0158] As a preferred technical solution of the present invention: the camera is provided with an input unit, a storage unit and an output unit, the input unit is used to obtain a picture taken by the camera, the storage unit is used to temporarily store the image to be processed obtained from the input unit, the output unit is used to send the image to be processed in the storage unit to the main board for pest detection, the memory includes a random access memory and a read-only memory, the random access memory stores computer instructions, the read-only memory stores a pest detection model, the development board is installed with a memory and a communication unit, the memory is used to obtain the computer instructions currently required to be executed by the central processing unit from the memory, and the communication unit is used to receive the information of the pests in the image to be processed detected by the central processing unit and send it to the administrator.

[0159] In the above structure: the camera in the image detection device for granary pests proposed by the present invention is used to collect and store the images to be processed in the granary, the development board includes a main board, a memory and a bus for connecting the main board and the memory, the main board contains at least one central processing unit for performing pest detection on the images to be processed and adjacent frame images, the pest detection model is obtained from the memory, and the memory is used to store computer instructions and pest detection models.

[0160] The input unit is a part of the camera, and is used to obtain the images taken by the camera.

[0161] The storage unit is a part of the camera, and is used to temporarily store the input image obtained from the input unit.

[0162] The output unit is a part of the camera, which is used to send the image to be processed in the camera storage unit to the development board for pest detection.

[0163] The bus is used to connect the camera and the development board to complete data transmission between the camera and the development board.

[0164] The memory is a part of the development board. The memory includes random access memory (RAM) and read-only memory (ROM). RAM is used to store the currently running machine instructions, while ROM stores the pest detection model, all computer instructions, etc. The above-mentioned computer programs, computer instructions, etc. can be partitioned and stored in one or more memories.

[0165] The memory is a part of the development board that is used to obtain the computer instructions that the central processor currently needs to execute from the storage.

[0166] The central processing unit is a part of the development board and is used for performing pest detection on the image to be processed obtained from the output unit, and the pest detection model is obtained from the memory.

[0167] The communication unit is a part of the development board, and is used to receive information about pests in the image to be processed detected by the central processor and send it to the administrator.

[0168] Compared with the prior art, the present invention has the following beneficial effects:

[0169] The present invention can monitor the activities of pests in the granary in real time, and effectively identify and locate pests through image acquisition, processing and analysis, thereby ensuring the safety of grain storage, reducing grain losses, and improving the intelligent level of granary management; it identifies pests that may exist in the granary through an image detection algorithm based on a single-stage detector, and usually introduces a dynamic biological detection algorithm, so that the camera's detection ability for dynamic organisms is further improved, effectively reducing the misjudgment of static objects in the granary. BRIEF DESCRIPTION OF THE DRAWINGS

[0170] Figure 1 The pest identification process of the present invention;

[0171] Figure 2 The overall workflow of the present invention is as follows;

[0172] Figure 3 It is a detailed schematic diagram of the functional modules of the present invention;

[0173] Figure 4 It is a physical structure schematic diagram of the present invention;

[0174] Figure 5 It is a schematic diagram of the logical structure of the present invention. DETAILED DESCRIPTION

[0175] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0176] At present, with the development of artificial intelligence technology, the application of artificial intelligence technology in the field of monitoring is becoming more and more extensive. Traditional monitoring systems mainly rely on manual viewing of video images, which is not only time-consuming and labor-intensive, but also prone to omissions and unable to detect abnormal situations in a timely and accurate manner. With the advancement of artificial intelligence technologies such as computer vision and deep learning, intelligent monitoring systems have emerged, greatly improving monitoring efficiency and accuracy.

[0177] Most granaries still rely on traditional manual inspection and physical capture methods, which are unable to achieve real-time, comprehensive and accurate pest monitoring. This results in the inability to detect and deal with pests in the early stages of invasion. By the time the problem is exposed, it has often caused serious food losses and quality degradation.

[0178] To this end, the present invention proposes a granary pest detection image acquisition device based on an image detection algorithm of a single-stage detector, specifically a granary pest image detection method, device and equipment, which aims to identify pests that may exist in the granary through an image detection algorithm based on a single-stage detector, and usually introduces a dynamic organism detection algorithm, so that the camera's detection ability for dynamic organisms is further enhanced, effectively reducing the misjudgment of static objects in the granary. The present invention is further described below in conjunction with the drawings in the specification.

[0179] like Figure 1 As shown, the image detection method for granary pests proposed by the present invention comprises the following steps:

[0180] S101, obtaining an image to be processed:

[0181] The camera is used to capture the images in the granary in real time, and the captured images are enhanced. The enhanced images to be processed are stored and wait for the pest detection model to detect them.

[0182] S102: Put the image to be processed into the pest detection model for processing to obtain multiple candidate boxes and confidence levels:

[0183] Build a pest detection model, and input the enhanced image to be processed into the pest detection model for frame-by-frame detection, obtain multiple candidate detection frames and confidence information corresponding to the candidate detection frames, find the picture suspected of containing pests in the candidate detection frames and confidences, and mark the candidate frames and confidences of the suspected pests;

[0184] S103: If a possible harmful organism is identified in the image, the image of the adjacent frame of the image to be processed is obtained and put into the pest detection model for detection:

[0185] Obtain adjacent frame images of the image to be processed. If the pest detection model detects that there are suspected pests in the image to be processed, the adjacent frame images of the image to be processed will be sent to the pest detection model for detection, and the position candidate frame and confidence of the suspected pest will be marked;

[0186] S104, determining the location information of harmful organisms in adjacent frame images as the first image:

[0187] Marking the location information of the pests in the adjacent frame images as the first image, including the candidate frame of the pest location, the confidence, the type of the pest, and the time when the pest appeared;

[0188] S105, determining the location information of harmful organisms in the image to be processed as the second image:

[0189] Marking the location information of the pests in the image to be processed as a second image, including a candidate frame of the pest location, a confidence level, the type of the pest, and the time when the pest appeared;

[0190] S106: Compare the similarity between the first image and the second image. If the similarity is less than a threshold, it indicates that harmful organisms do exist. If the similarity is greater than the threshold, it indicates that harmful organisms do not exist.

[0191] By comparing the candidate frame positions of the organisms in the first image and the second image, the similarity is obtained. If the similarity is greater than a threshold, it is considered a misjudgment, that is, the identified organism is not a pest. If the similarity is less than the threshold, it is considered that the identified organism is indeed a pest.

[0192] In step S101, the image to be processed is obtained, including the following processing steps:

[0193] First, obtain the video stream information to be processed through the camera;

[0194] Then, the frame extraction frequency information is determined according to the moving speed of the organism to be monitored, and the organism to be monitored is the pest appearing in the monitoring area corresponding to the video stream information to be processed;

[0195] Finally, according to the frame extraction frequency information, the video stream information to be processed is subjected to frame extraction processing to obtain the image to be processed.

[0196] The frame extraction frequency is determined as follows: When the camera captures a moving organism, it will not immediately send it to the pest detection model for detection, but will first detect its movement speed and determine the frame extraction frequency based on its movement speed. The specific formula is as follows:

[0197]

[0198] The speed of the creature v: detects the speed of the creature moving in the picture, in pixels / second.

[0199] Camera frame rate f: the number of frames captured by the camera per second, in frames per second.

[0200] Expected biological movement distance d between adjacent frames: the pixel distance that the biological is expected to move between adjacent frames. The larger the value, the more the biological movement between adjacent frames. Conversely, the smaller the value, the less the biological movement between adjacent frames.

[0201] α: Adjusts the effect of biological movement speed. By adjusting α, you can amplify or reduce the effect of biological movement speed on the frame rate.

[0202] β: Adjusts the effect of expected movement distance. By adjusting β, you can optimize the effect of creature movement distance on frame rate.

[0203] γ: Adjust the impact of the camera frame rate. By adjusting γ, you can change the impact of the camera frame rate on the frame extraction frequency.

[0204] In step S103, the pest detection model is trained by optimizing the loss function. The formula for optimizing the loss function is as follows:

[0205]

[0206] in:

[0207] IoU: The intersection over union of the predicted box and the true box, used to measure the degree of overlap between the predicted box and the true box.

[0208] It is used to consider the distance between the center point of the predicted box and the real box. The larger the distance, the greater the loss.

[0209] c: The diagonal length of the minimum enclosing area containing the predicted box and the true box,

[0210] v: used to measure the consistency of the aspect ratio between the predicted box and the real box,

[0211]

[0212] This term is used to measure the difference in aspect ratio between the predicted box and the true box:

[0213] a: used to balance the contribution of IoU and aspect ratio consistency terms to the total loss,

[0214]

[0215] In step S103, before the image to be processed is placed into the pest detection model, the image to be processed is processed, specifically as follows: first, the color space mode of the image to be processed is converted to obtain the converted image to be processed, and then, the image to be processed is format converted to obtain the image to be processed after format conversion, wherein the format conversion processing includes: normalization processing and image size adjustment processing.

[0216] In step S106, first histogram information of the first image is extracted, second histogram information of the second image is extracted, and the similarity between the first image and the second image is determined by comparing the first histogram information and the second histogram information.

[0217] The image detection method for granary pests proposed in the present invention can monitor the activities of pests in the granary in real time, and effectively identify and locate pests through image acquisition, processing and analysis, thereby ensuring the safety of grain storage, reducing grain losses and improving the intelligence level of granary management.

[0218] The image to be processed and the shooting time are obtained through the camera installed in the granary, and then the image to be processed is enhanced and input into the pest detection model for image processing to obtain the image processing result; the image processing result includes the location information and type of the pests. The pest detection model can identify the pests in the image to be monitored and clearly mark their location information and type, which is very convenient.

[0219] If the image processing result indicates the presence of pests, then adjacent frame images adjacent to the image to be processed are obtained, and the similarity between the adjacent frame images and the pests in the image to be processed is compared to further determine whether there are pests in the image to be processed, thereby improving the reliability of image recognition.

[0220] After obtaining the information of the pests through comparison, the image of the pests and the time and other information are uploaded to the server to remind the administrator to go and deal with it.

[0221] To enhance the clarity of images captured by the camera, make adjustments in the following ways:

[0222] According to the lighting conditions of the granary, histogram equalization is performed. By adjusting the distribution of the image's histogram and enhancing the contrast, the obtained image can be made clearer.

[0223] Apply a denoising algorithm, such as mean filtering, median filtering, or Gaussian filtering, to reduce noise in the image.

[0224] Adjust the camera exposure time and extend the camera exposure time to capture more light and enhance the clarity of the captured pictures.

[0225] Will Figure 1 The image detection method is combined with the camera, such as Figure 2 As shown, the entire workflow of the camera from shooting to alarm in the present invention is described in detail:

[0226] S201, obtaining the image to be processed and the shooting time,

[0227] The camera continuously shoots the contents of the granary 24 hours a day. Since the granary is usually in dim light, the exposure time can be appropriately extended according to the light conditions to obtain a clearer picture. This is also the initial picture enhancement performed by the camera settings.

[0228] S202, determine the frame extraction frequency,

[0229] When the camera captures a moving creature, it will not immediately send it to the pest detection model for detection. Instead, it will first detect its movement speed and determine the frame extraction frequency based on its movement speed to facilitate the subsequent extraction of adjacent frame images of the image to be processed.

[0230]

[0231] 1. The speed of the organism v: The speed at which the organism is detected in the picture, in pixels per second.

[0232] 2. Camera frame rate f: the number of frames captured by the camera per second, in frames per second.

[0233] 3. Expected object movement distance d between adjacent frames: The pixel distance that the organism is expected to move between adjacent frames. The larger the value, the more the organism moves between adjacent frames, which is suitable for high-speed moving objects. On the contrary, the smaller the value, the less the organism moves between adjacent frames, which is suitable for low-speed moving objects.

[0234] 4. α: Adjust the effect of biological movement speed. By adjusting α, you can magnify or reduce the effect of biological movement speed on the frame rate.

[0235] 5. β: Adjust the effect of expected movement distance. By adjusting β, the effect of creature movement distance on frame rate can be optimized.

[0236] 6.γ: Adjust the impact of camera frame rate. By adjusting γ, you can change the impact of camera frame rate on frame extraction frequency.

[0237] S203, image preprocessing and enhancement,

[0238] After obtaining the images captured by the camera, we first preprocess the images and adjust the size and format of the images to make all the images uniform for subsequent processing. After the images are uniform, we normalize the images. The formula is as follows:

[0239]

[0240] Among them: I is the pixel value of the original image, Imin is the minimum pixel value of the original image, Imax is the maximum pixel value of the original image, I′ is the normalized pixel value, λ is the offset constant used to adjust the normalization process, and k is a small constant to prevent division by zero errors. Through this formula, the pixel value of the image can be normalized to the range of [0,1], thereby improving the performance of subsequent image processing and analysis algorithms. At the same time, λ and k are introduced to enhance the flexibility and robustness of the image normalization process, avoid division by zero errors, adjust the normalization range, and enhance contrast;

[0241] The technology of image smoothing by using Gaussian filtering through weighted average method can effectively retain the edge information of the image while removing image noise. The two-dimensional form of Gaussian function is as follows:

[0242]

[0243] in:

[0244] 1. The value of the Gaussian function at point (x, y),

[0245] 2. Standard deviation, which determines the width of the Gaussian function and the smoothness of the filter,

[0246] 3. Relative position to the filter center,

[0247] For each pixel in the image, take a (2k+1)×(2k+1) neighborhood with it as the center, use the Gaussian kernel G to perform weighted averaging on the neighborhood, and calculate the new pixel value. The specific operation formula is as follows:

[0248]

[0249] in:

[0250] 1. I(x, y) is the pixel value of the original image at position (x, y),

[0251] 2. I′(x, y) is the pixel value of the filtered image at position (x, y),

[0252] 3. G(i, j) is the value of the Gaussian kernel at position (i, j),

[0253] 4.k is the radius of the window, and the window size is (2k+1)×(2k+1),

[0254] Histogram equalization adjusts the contrast of the image to make the grayscale value distribution of the image more uniform, enhance the overall contrast of the image, highlight the details in the image, and make the visual effect more obvious. The steps are as follows:

[0255] 1. Calculate the original histogram:

[0256]

[0257] Let I be the original image, with L gray levels (usually 256, i.e., the gray value range is 0 to 255),

[0258] Calculate the frequency n of each gray level k , that is, the number of pixels with gray value k. γ (k) is the probability of gray value k, N is the total number of pixels in the image,

[0259] 2. Calculate the cumulative distribution function (CDF):

[0260]

[0261] Cumulative distribution function C(r k ) is the cumulative probability of gray value from 0 to k,

[0262] 3. Calculate the gray value after equalization:

[0263] Linearly transform the cumulative distribution function to a new grayscale range (0 to L-1),

[0264] S k =(L-1)·C(r k ) (10)

[0265] Where S k is the gray value after equalization,

[0266] 4. Map to new image:

[0267] Map the grayscale value r of each pixel in the original image to the new grayscale value S to generate a balanced image.

[0268] S204: Detecting the enhanced image to be processed

[0269] The enhanced image to be processed is put into the pest detection model for detection. The specific detection process is as follows: Figure 1 As shown, by calling the detection model, the detection result is obtained;

[0270] S205, Alarm prompt

[0271] If the model detects that pests do exist in the image to be processed, an alarm message will be sent to the administrator, including the type of pests and the time when the pests appeared.

[0272] like Figure 3As shown, the image detection device for granary pests proposed in the present invention includes a camera, and the camera is provided with an image acquisition module, an image enhancement module, an image detection module, a determination module and a prompt module. The image acquisition module is used to shoot the granary in real time, the image acquisition mode is connected to the image enhancement module, the image enhancement module is used to enhance the video image shot by the image acquisition module, the image enhancement module is connected to the image detection module, the image detection module is used to detect and process the pre-selected boxes and confidences in the image to be processed and the adjacent frame images, the image detection module is connected to the determination module, the determination module is used for the similarity between the first image and the second image, the determination module is connected to the prompt module, the prompt module is used to prompt the administrator that pests appear in the granary, and when the image to be processed is processed and it is confirmed that there are pests in the image to be processed, an alarm message will be sent to the administrator, and the alarm message includes the type of pests and the time of appearance.

[0273] The image detection module includes a first processing unit, a first judgment unit and a second processing unit, wherein:

[0274] First processing unit: When the image to be processed is placed in the pest detection module, the first processing unit draws a pre-selected box and confidence level of the pest.

[0275] The first judgment unit: judges whether there are suspected pests in the picture according to the pre-selected box and confidence of the pests, and whether it is necessary to perform pest detection on the adjacent frames of the image to be processed.

[0276] The second processing unit: when the adjacent frame images of the image to be processed are placed into the pest detection module, the second processing unit draws the pre-selection box and confidence of the pest;

[0277] The determination module includes a first determination unit, a second determination unit, a third determination unit and a fourth determination unit, wherein:

[0278] The first determination unit: determines the candidate frames and confidences of the pests in the adjacent frame images of the image to be processed,

[0279] The second determination unit: determines the candidate frame and confidence of the pest in the image to be processed,

[0280] The third determination unit includes a first extraction unit, a second extraction unit and a first determination subunit, wherein:

[0281] The first extraction unit is used to extract the first histogram information of the first image and obtain the distribution of each grayscale pixel value in the first image.

[0282] The second extraction unit is used to extract the second histogram information of the second image and obtain the distribution of each grayscale pixel value in the second image.

[0283] The first determination subunit: confirms the similarity based on the information of the first histogram and the second histogram. The two histograms are compared using the Euclidean formula, which is as follows:

[0284]

[0285] Where h1 and h2 are the histograms of the two images respectively, i is the bin index of the histogram,

[0286] The similarity is confirmed by calculating the "straight-line" distance between the histogram vectors of the two images. The smaller the value, the more similar the histograms of the two images are;

[0287] The fourth determining unit includes a second determining subunit and a first deleting module, wherein:

[0288] The second determination subunit is used to determine whether there are pests according to the similarity between the two images; if there are pests, the subsequent first deletion unit and prompt module are executed, otherwise nothing is executed.

[0289] The first deletion module: Use non-maximum suppression to delete redundant detection results in the candidate detection frame and only retain the detection frame that is most likely to contain the target.

[0290] The image detection device for granary pests proposed by the present invention comprises a camera, in which an image acquisition module, an image enhancement module, an image detection module, a determination module and a prompt module are arranged, wherein

[0291] Image acquisition module: This module is used to continuously shoot video surveillance inside the granary 24 hours a day, and input the obtained video stream into the image enhancement module for processing;

[0292] Image enhancement module: This module is used to enhance the video images directly captured by the camera, including camera enhancement and algorithm enhancement. Since granaries are usually in a dry, ventilated and low-temperature environment, the lighting conditions in granaries are usually poor. Depending on the situation, the camera exposure time can be extended to obtain a clearer picture. This is camera enhancement.

[0293] After obtaining the enhanced image by the camera, the algorithm will enhance it again. First, the image is preprocessed to adjust the size and format of the image. After unifying the image standards, the image is normalized, denoised, and histogram equalized to enhance the overall details and contrast of the image.

[0294] The image detection module includes a first processing unit, a first judgment unit and a second processing unit, wherein:

[0295] First processing unit: When the image to be processed is placed in the pest detection module, the first processing unit draws a pre-selected box and confidence level of the pest.

[0296] The first judgment unit: judges whether there are suspected pests in the picture according to the pre-selected box and confidence of the pests, and whether it is necessary to perform pest detection on the adjacent frames of the image to be processed.

[0297] Second processing unit: When the adjacent frame images of the image to be processed are placed into the pest detection module, the second processing unit draws the pre-selection box and confidence of the pest.

[0298] The determination module includes a first determination unit, a second determination unit, a third determination unit and a fourth determination unit, wherein:

[0299] The first determination unit: determines the candidate frames and confidences of the pests in the adjacent frame images of the image to be processed,

[0300] The second determination unit: determines the candidate frame and confidence of the pest in the image to be processed,

[0301] The third determination unit includes a first extraction unit, a second extraction unit and a first determination subunit, wherein:

[0302] The first extraction unit is used to extract the first histogram information of the first image and obtain the distribution of each grayscale pixel value in the first image.

[0303] The second extraction unit is used to extract the second histogram information of the second image and obtain the distribution of each grayscale pixel value in the second image.

[0304] The first determination subunit: confirms the similarity based on the information of the first histogram and the second histogram. The two histograms are compared using the Euclidean formula, which is as follows:

[0305]

[0306] Where h1 and h2 are the histograms of the two images respectively, i is the bin index of the histogram,

[0307] The similarity is confirmed by calculating the "straight-line" distance between the histogram vectors of the two images. The smaller the value, the more similar the histograms of the two images are;

[0308] The fourth determining unit includes a second determining subunit and a first deleting module, wherein:

[0309] The second determination subunit is used to determine whether there are pests according to the similarity between the two images; if there are pests, the subsequent first deletion unit and prompt module are executed, otherwise nothing is executed.

[0310] The first deletion module: Non-Maximum Suppression (NMS) is used to reduce redundant detection results in candidate detection frames, and only the detection frames that are most likely to contain the target are retained. NMS simplifies the final detection results by suppressing detection frames with high overlap and low confidence. In the present invention, the use of the NMS mechanism can effectively reduce redundant detection frames, ensure the accuracy of the detection results, and improve the performance and efficiency of the pest detection system. After the deletion of this module, the location, confidence, type and other information of the pests in the image are finally confirmed.

[0311] Prompt module: This module is used to prompt the administrator that pests have appeared in the granary. When the image to be processed has been processed by all the above modules and the presence of pests in the image is confirmed, an alarm message will be sent to the administrator. The alarm message includes the type of pests and the time of appearance.

[0312] like Figure 4-5 As shown, the image detection device for granary pests proposed by the present invention comprises

[0313] At least one camera: the camera is used to collect and store the images to be processed in the granary.

[0314] A development board, comprising a main board, a memory and a bus for connecting the main board and the memory,

[0315] The mainboard includes at least one central processor for performing pest detection on the image to be processed and the adjacent frame images. The pest detection model is obtained from the memory, and the memory is used to store computer instructions and the pest detection model.

[0316] The storage is installed on a mainboard, and the mainboard is connected to the camera for communication and is used for receiving and processing image data.

[0317] The camera is provided with an input unit, a storage unit and an output unit, the input unit is used to obtain a picture taken by the camera, the storage unit is used to temporarily store the image to be processed obtained from the input unit, the output unit is used to send the image to be processed in the storage unit to the main board for pest detection, the memory includes a random access memory and a read-only memory, the random access memory stores computer instructions, the read-only memory stores a pest detection model, the development board is installed with a memory and a communication unit, the memory is used to obtain the computer instructions currently required to be executed by the central processing unit from the memory, and the communication unit is used to receive the information of the pests in the image to be processed detected by the central processing unit and send it to the administrator.

[0318] In the above structure: the camera in the image detection device for granary pests proposed by the present invention is used to collect and store the images to be processed in the granary, the development board includes a main board, a memory and a bus for connecting the main board and the memory, the main board contains at least one central processing unit for performing pest detection on the images to be processed and adjacent frame images, the pest detection model is obtained from the memory, and the memory is used to store computer instructions and pest detection models.

[0319] The input unit is a part of the camera, and is used to obtain the images taken by the camera.

[0320] The storage unit is a part of the camera, and is used to temporarily store the input image obtained from the input unit.

[0321] The output unit is a part of the camera, which is used to send the image to be processed in the camera storage unit to the development board for pest detection.

[0322] The bus is used to connect the camera and the development board to complete data transmission between the camera and the development board.

[0323] The memory is a part of the development board. The memory includes random access memory (RAM) and read-only memory (ROM). RAM is used to store the currently running machine instructions, while ROM stores the pest detection model, all computer instructions, etc. The above-mentioned computer programs, computer instructions, etc. can be partitioned and stored in one or more memories.

[0324] The memory is a part of the development board that is used to obtain the computer instructions that the central processor currently needs to execute from the storage.

[0325] The central processing unit is a part of the development board and is used for performing pest detection on the image to be processed obtained from the output unit, and the pest detection model is obtained from the memory.

[0326] The communication unit is a part of the development board, and is used to receive information about pests in the image to be processed detected by the central processor and send it to the administrator.

[0327] The present invention can monitor the activities of pests in the granary in real time, and effectively identify and locate pests through image acquisition, processing and analysis, thereby ensuring the safety of grain storage, reducing grain losses, and improving the intelligent level of granary management; it identifies pests that may exist in the granary through an image detection algorithm based on a single-stage detector, and usually introduces a dynamic biological detection algorithm, so that the camera's detection ability for dynamic organisms is further improved, effectively reducing the misjudgment of static objects in the granary.

[0328] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. Image detection method for granary pests, Features: The steps include: S101, obtaining an image to be processed: The camera is used to capture the images in the granary in real time, and the captured images are enhanced. The enhanced images to be processed are stored and wait for the pest detection model to detect them. S102: Put the image to be processed into the pest detection model for processing to obtain multiple candidate boxes and confidence levels: Build a pest detection model, and input the enhanced image to be processed into the pest detection model for frame-by-frame detection, obtain multiple candidate detection frames and confidence information corresponding to the candidate detection frames, find the picture suspected of containing pests in the candidate detection frames and confidences, and mark the candidate frames and confidences of the suspected pests; S103: If a possible harmful organism is identified in the image, the image of the adjacent frame of the image to be processed is obtained and put into the pest detection model for detection: Obtain adjacent frame images of the image to be processed. If the pest detection model detects that there are suspected pests in the image to be processed, the adjacent frame images of the image to be processed will be sent to the pest detection model for detection, and the position candidate frame and confidence of the suspected pest will be marked; S104, determining the location information of harmful organisms in adjacent frame images as the first image: Marking the location information of the pests in the adjacent frame images as the first image, including the candidate frame of the pest location, the confidence, the type of the pest, and the time when the pest appeared; S105, determining the location information of harmful organisms in the image to be processed as the second image: Marking the location information of the pests in the image to be processed as a second image, including a candidate frame of the pest location, a confidence level, the type of the pest, and the time when the pest appeared; S106: Compare the similarity between the first image and the second image. If the similarity is less than a threshold, it indicates that harmful organisms do exist. If the similarity is greater than the threshold, it indicates that harmful organisms do not exist. By comparing the candidate frame positions of the organisms in the first image and the second image, the similarity is obtained. If the similarity is greater than a threshold, it is considered a misjudgment, that is, the identified organism is not a pest. If the similarity is less than the threshold, it is considered that the identified organism is indeed a pest.

2. The image detection method for granary pests according to claim 1, Features: In step S101, the image to be processed is obtained, including the following processing steps: First, obtain the video stream information to be processed through the camera; Then, the frame extraction frequency information is determined according to the moving speed of the organism to be monitored, and the organism to be monitored is the pest appearing in the monitoring area corresponding to the video stream information to be processed; Finally, according to the frame extraction frequency information, the video stream information to be processed is subjected to frame extraction processing to obtain the image to be processed.

3. The image detection method for granary pests according to claim 2, Features: The frame extraction frequency is determined as follows: When the camera captures a moving organism, it will not immediately send it to the pest detection model for detection, but will first detect its movement speed and determine the frame extraction frequency based on its movement speed. The specific formula is as follows: The speed of the creature v: detects the speed of the creature moving in the picture, in pixels / second. Camera frame rate f: the number of frames captured by the camera per second, in frames per second. Expected distance the creature moves between adjacent frames d : The pixel distance that the organism is expected to move between adjacent frames. The larger the value, the more the organism moves between adjacent frames. Conversely, the smaller the value, the less the organism moves between adjacent frames. a: Adjust the effect of biological motion speed. By adjusting α, you can amplify or reduce the effect of biological motion speed on the frame rate. β: Adjusts the effect of expected movement distance. By adjusting β, you can optimize the effect of creature movement distance on frame rate. γ: Adjust the impact of the camera frame rate. By adjusting γ, you can change the impact of the camera frame rate on the frame extraction frequency.

4. The image detection method for granary pests according to claim 1, Features: In step S103, the pest detection model is trained by optimizing the loss function. The formula for optimizing the loss function is as follows: in: IoU: The intersection over union (IoU) of the predicted box and the true box, which is used to measure the degree of overlap between the predicted box and the true box. It is used to consider the distance between the center point of the predicted box and the real box. The larger the distance, the greater the loss. c: The diagonal length of the minimum enclosing area containing the predicted box and the true box, v: used to measure the consistency of the aspect ratio between the predicted box and the real box, This term is used to measure the difference in aspect ratio between the predicted box and the true box: a: used to balance the contribution of IoU and aspect ratio consistency terms to the total loss, 5. The image detection method for granary pests according to claim 1, Features: In step S103, before the image to be processed is placed into the pest detection model, the image to be processed is processed, specifically as follows: first, the color space mode of the image to be processed is converted to obtain the converted image to be processed, and then, the image to be processed is format converted to obtain the image to be processed after format conversion, wherein the format conversion processing includes: normalization processing and image size adjustment processing.

6. The image detection method for granary pests according to claim 1, Features: In step S106, first histogram information of the first image is extracted, second histogram information of the second image is extracted, and the similarity between the first image and the second image is determined by comparing the first histogram information and the second histogram information.

7. Image detection device for granary pests, Features: The invention comprises a camera, wherein the camera is provided with an image acquisition module, an image enhancement module, an image detection module, a determination module and a prompt module, wherein the image acquisition module is used to photograph the granary in real time, the image acquisition mode is connected to the image enhancement module, the image enhancement module is used to enhance the video image photographed by the image acquisition module, the image enhancement module is connected to the image detection module, the image detection module is used to detect and process the pre-selected boxes and confidences in the image to be processed and the adjacent frame images, the image detection module is connected to the determination module, the determination module is used for the similarity between the first image and the second image, the determination module is connected to the prompt module, the prompt module is used to prompt the administrator that pests appear in the granary, and when the image to be processed is processed and it is confirmed that pests exist in the image to be processed, an alarm message is sent to the administrator, and the alarm message includes the type of pests and the time of appearance.

8. The image detection device for granary pests according to claim 7, Features: The image detection module includes a first processing unit, a first judgment unit and a second processing unit, wherein: First processing unit: When the image to be processed is placed in the pest detection module, the first processing unit draws a pre-selected box and confidence level of the pest. The first judgment unit: judges whether there are suspected pests in the picture according to the pre-selected box and confidence of the pests, and whether it is necessary to perform pest detection on the adjacent frames of the image to be processed. The second processing unit: when the adjacent frame images of the image to be processed are placed into the pest detection module, the second processing unit draws the pre-selection box and confidence of the pest; The determination module includes a first determination unit, a second determination unit, a third determination unit and a fourth determination unit, wherein the first determination unit: determines a candidate frame and a confidence level of the pest in an adjacent frame image of the image to be processed, The second determination unit: determines the candidate frame and confidence of the pest in the image to be processed, The third determination unit includes a first extraction unit, a second extraction unit and a first determination subunit, wherein: The first extraction unit is used to extract the first histogram information of the first image and obtain the distribution of each grayscale pixel value in the first image. The second extraction unit is used to extract the second histogram information of the second image and obtain the distribution of each grayscale pixel value in the second image. The first determination subunit: confirms the similarity based on the information of the first histogram and the second histogram. The two histograms are compared using the Euclidean formula, which is as follows: Where h1 and h2 are the histograms of the two images respectively, i is the bin index of the histogram, The similarity is confirmed by calculating the "straight-line" distance between the histogram vectors of the two images. The smaller the value, the more similar the histograms of the two images are; The fourth determining unit includes a second determining subunit and a first deleting module, wherein: The second determination subunit is used to determine whether there are pests according to the similarity between the two images; if there are pests, the subsequent first deletion unit and prompt module are executed, otherwise nothing is executed. The first deletion module: Use non-maximum suppression to delete redundant detection results in the candidate detection frame and only retain the detection frame that is most likely to contain the target.

9. Image detection equipment for granary pests, Features: include At least one camera: the camera is used to collect and store the images to be processed in the granary. A development board, comprising a main board, a memory and a bus for connecting the main board and the memory, The mainboard includes at least one central processor for performing pest detection on the image to be processed and the adjacent frame images. The pest detection model is obtained from the memory, and the memory is used to store computer instructions and the pest detection model. The storage is installed on a mainboard, and the mainboard is connected to the camera for communication and is used for receiving and processing image data.

10. The image detection device for granary pests according to claim 9, Features: The camera is provided with an input unit, a storage unit and an output unit, the input unit is used to obtain a picture taken by the camera, the storage unit is used to temporarily store the image to be processed obtained from the input unit, the output unit is used to send the image to be processed in the storage unit to the main board for pest detection, the memory includes a random access memory and a read-only memory, the random access memory stores computer instructions, the read-only memory stores a pest detection model, the development board is installed with a memory and a communication unit, the memory is used to obtain the computer instructions currently required to be executed by the central processing unit from the memory, and the communication unit is used to receive the information of the pests in the image to be processed detected by the central processing unit and send it to the administrator.

Citation Information

Patent Citations

  • Image detection method, device and equipment for pests

    CN116343076A

  • Forestry pest detection method and system based on multi-scale feature enhancement and fusion

    CN117671655A

  • Method, system and equipment for identifying insect pests in granary based on image analysis and medium

    CN118314526A

  • Method for sex sorting of mosquitoes and apparatus therefor

    US20200154685A1