Mouse disease degree detection system based on YOLO and detection method thereof

Abstract

The invention belongs to the technical field of agricultural intellectualization, provides a YOLO-based mouse disease degree detection system and a YOLO-based mouse disease degree detection method, and aims to solve the problem of low accuracy of mouse disease detection in a farmland in the prior art, and the method comprises the following steps: determining a current monitoring image corresponding to a preset target farmland and a corresponding mouse monitoring audio; determining the individual morphological characteristics of the preset mouse; when it is judged that the mouse monitoring audio contains the mouse behavior sound corresponding to the mouse behavior, and according to preset mouse independent morphological characteristics, under the condition that it is preliminarily judged that the current monitoring image contains the mouse image, the current frame mouse independent morphological characteristics corresponding to the current monitoring image are determined; and finally determining that the current monitoring image contains the mouse image under the condition of determining that the current frame mouse independent morphological characteristics accord with the mouse action morphological trajectory, and determining the rodent damage degree corresponding to the preset target farmland according to a plurality of final judgments, so that the accuracy of farmland rodent damage detection can be improved.

Description

Technical Field

[0001] The present invention relates to the fields of agricultural intelligence and machine learning technology, and in particular to a YOLO-based mouse disease degree detection system and a detection method thereof. Background Art

[0002] In agriculture, strengthening the detection of rats and the damage they cause to farmland is a crucial factor in ensuring food production security. With the development of intelligent agriculture, artificial intelligence (AI) technology is increasingly being applied to the detection of rats and the damage they cause, enabling automated and efficient intelligent monitoring of farmland rats and the damage they cause.

[0003] In intelligent agricultural technology, to strengthen the detection of damage caused by rats to farmland, we must first identify rats in farmland. In traditional technology, images corresponding to farmland are generally collected, and target detection based on machine learning is performed to determine whether the collected images contain rats. When identifying rats, the overall shape outline of the rat is used as the basis for identifying the rat and corresponding detection is performed. Then, based on the detected rats, the degree of damage caused by the rats to the farmland is identified, thereby realizing the detection of the degree of damage caused by rats to the farmland.

[0004] However, the inventors realized that in traditional technologies, due to the complex environmental background of mice in farmland, unclear features and protective color of their fur, the target color corresponding to the target mouse is similar to its corresponding background color, which increases the difficulty of detecting mice as targets. In other words, it is difficult to accurately detect target mice in farmland, and it is also difficult to accurately and automatically detect the damage caused by mice.

[0005] Therefore, how to improve the accuracy of rodent pest detection in farmland has become an urgent problem that needs to be solved in the field of agricultural intelligence. Summary of the Invention

[0006] The technical problem solved by the present invention is to solve the problem of low accuracy in detecting rodent damage in farmland in traditional technologies.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: determining the current monitoring image and the corresponding rat monitoring audio corresponding to the preset target farmland, and determining the preset rat-specific morphological characteristics; when it is determined that the rat monitoring audio contains the rat behavior sound corresponding to the rat behavior, and based on the preset rat-specific morphological characteristics, and based on the preset rat target detection YOLO model, when it is preliminarily determined that the current monitoring image contains a rat image, determining the current frame rat-specific morphological characteristics corresponding to the current monitoring image; when it is determined that the current frame rat-specific morphological characteristics are consistent with the rat movement morphological trajectory, finally determining that the current monitoring image contains a rat image, and based on several of the above final judgments, determining the degree of rodent damage corresponding to the preset target farmland.

[0008] Preferably, when determining that the mouse-specific morphological features of the current frame are consistent with the mouse's motion morphological trajectory, the method includes: determining several frames of monitoring adjacent images corresponding to the current monitoring image based on a time series; determining the mouse-specific morphological features of adjacent frames corresponding to the monitoring adjacent images, and obtaining several adjacent frames of mouse-specific morphological features; based on the corresponding time series, forming a sequence of the mouse-specific morphological features of the current frame and all the mouse-specific morphological features of the adjacent frames to obtain a mouse-specific morphological feature sequence; determining whether the mouse-specific morphological feature sequence is consistent with the preset mouse motion morphological law corresponding to the mouse; if the above judgment is yes, determining that the mouse-specific morphological features of the current frame are consistent with the mouse motion morphological trajectory.

[0009] The present invention also provides a YOLO-based system for detecting the degree of rat disease, including: a first determination module, used to determine the current monitoring image and corresponding rat monitoring audio corresponding to the preset target farmland, and determine the preset unique morphological characteristics of the rat; a first judgment module, used to determine that the rat monitoring audio contains the rat behavior sound corresponding to the rat behavior, and based on the preset unique morphological characteristics of the rat, and based on the preset rat target detection YOLO model, when it is preliminarily determined that the current monitoring image contains a rat image, determine the current frame unique morphological characteristics of the rat corresponding to the current monitoring image; a second judgment module, used to finally determine that the current monitoring image contains a rat image when it is determined that the unique morphological characteristics of the rat in the current frame are consistent with the rat movement morphological trajectory, and determine the degree of rat damage corresponding to the preset target farmland based on several of the above final judgments.

[0010] The beneficial effects of the present invention are as follows: by determining the current monitoring image and the corresponding mouse monitoring audio corresponding to the preset target farmland, and determining the unique morphological characteristics of the preset mice, the accurate detection of farmland mice is achieved according to the unique morphological characteristics of the mice, and combined with the target detection based on image recognition and voiceprint recognition, and then the degree of rodent damage corresponding to the preset target farmland is accurately determined, thereby not only realizing the different-dimensional cross-modal parallel detection strategy combining voiceprint recognition with target detection based on the unique morphological characteristics of mice, but also realizing the progressive hierarchical detection strategy of "preliminary judgment-rejudgment and verification". Compared with the single-dimensional target detection in traditional technology, it can perform multiple composite verifications on mouse detection from different angles, effectively improving the accuracy of farmland mouse detection, so as to adapt to the environmental characteristics corresponding to the complex background, unclear features and protective color of mice in farmland, and then accurately assess the damage caused by mice to farmland, improve the accuracy of farmland mouse damage detection, and realize automatic and efficient intelligent monitoring of farmland mice and their damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A flowchart of a method for detecting the severity of mouse disease based on YOLO provided in an embodiment of the present invention.

[0012] Figure 2 Schematic diagram of the overall process of the YOLO-based method for detecting the degree of mouse disease provided in an embodiment of the present invention.

[0013] Figure 3 This is a schematic diagram of the first sub-flow of the YOLO-based method for detecting the degree of disease in mice provided in an embodiment of the present invention.

[0014] Figure 4 Schematic diagram of the second sub-flow of the method for detecting the degree of disease in mice based on YOLO provided in an embodiment of the present invention.

[0015] Figure 5 A schematic block diagram of a YOLO-based mouse disease severity detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0017] The embodiments of the present invention provide a YOLO-based system and method for detecting the degree of mouse disease. The detection system and method can be applied to devices such as edge devices, smart phones, tablet computers, computer devices, servers, etc., and can be used in, but not limited to, performing YOLO-based detection of the degree of mouse disease.

[0018] Example 1, please refer to Figure 1 and Figure 2 , Figure 1 This is a flow chart of a method for detecting the degree of mouse disease based on YOLO provided in an embodiment of the present invention. Figure 2 The overall flow chart of the method for detecting the degree of mouse disease based on YOLO provided in the embodiment of the present invention is as follows. Figure 1 As shown, in this embodiment, the method includes but is not limited to the following steps S101-S103:

[0019] S101. Determine the current monitoring image and corresponding mouse monitoring audio corresponding to the preset target farmland, and determine the unique morphological characteristics of the preset mice.

[0020] Image acquisition equipment and audio acquisition equipment are pre-deployed at the preset target farmland. The image acquisition equipment captures video or images of the preset target farmland. Based on the captured video or images, rat detection is performed through target detection to monitor rats in the preset target farmland. Audio acquisition equipment is used to capture audio from the preset target farmland and, through voiceprint recognition, also detect rats in the preset target farmland. The image acquisition equipment includes but is not limited to infrared cameras that support nighttime photography, starlight-level low-light cameras, and visible light cameras (RGB cameras). The audio acquisition equipment includes but is not limited to omnidirectional condenser microphones and directional microphone arrays with waterproof high-sensitivity microphone arrays. Thus, the video or images and audio corresponding to the preset target farmland are captured, and the current monitoring image and corresponding rat monitoring audio corresponding to the preset target farmland are obtained. Subsequently, rats in the preset target farmland are detected based on the current monitoring image and the corresponding rat monitoring audio. Among them, the current monitoring image represents the image of the rat monitoring of the preset target farmland corresponding to the current moment of collection, and the corresponding rat monitoring audio represents the audio of the rat monitoring of the preset target farmland belonging to the same monitoring space target and the same time as the current monitoring image. For example, when the current monitoring image corresponds to the monitoring space target and time B corresponding to the position A of the preset target farmland, the corresponding rat monitoring audio represents the audio corresponding to the monitoring space target and time B corresponding to the position A of the preset target farmland.

[0021] The unique morphological characteristics of a mouse are pre-set, that is, the unique morphological characteristics of a mouse are preset. The preset unique morphological characteristics of a mouse represent the unique morphological characteristics of a mouse, and the preset unique morphological characteristics of a mouse can be used to identify that the identification object is a unique morphological identifier of a mouse. The preset unique morphological characteristics of a mouse include but are not limited to the unique morphological characteristics of the mouse's head, the unique morphological characteristics of the mouse's trunk, the unique morphological characteristics of the mouse's limbs, the unique morphological characteristics of the mouse's tail, the unique morphological characteristics of the mouse's movement and stillness, and a combination of the above morphological characteristics at different angles. The unique morphological characteristics of the mouse's head include but are not limited to a pointed snout, small round ears (ear length ≈ distance from eyes to nose), prominent whiskers (antennae), and a combination of the above. The unique morphological characteristics of the mouse's trunk include but are not limited to being soft and without edges, a slightly swollen abdomen (more obvious after eating), an elongated cylindrical trunk, and a combination of the above. The unique morphological characteristics of the mouse's limbs include but are not limited to short forelimbs with sharp claws (suitable for digging holes), slender hind limbs (hind legs straightened when jumping), and a combination of the above. The unique morphological characteristics of the mouse's tail include but are not limited to a length ≈ body length (for house mice) or longer (for example, Rattus norvegicus), with ringed scales on its surface and a thicker base, and tail posture (raised / dragging) reflecting emotional state (such as erect when alert), as well as combinations of the aforementioned aspects. Unique morphological characteristics of mice in motion and stillness include, but are not limited to, a raised back when in motion, a curled-up body when at rest, a running posture (e.g., an elongated body with alternating front and rear legs extending, resulting in a "wavy" gait), a gnawing posture (e.g., holding food with the front paws and frequently moving the head from side to side), and combinations of the aforementioned aspects. Unique morphological characteristics of mice represent structures and states unique to mice. The difference between form and shape is that form (morphology) represents the structure and state exhibited by mice, integrating geometric shapes and dynamic changes, while shape (shape) represents the geometric characteristics of the mouse's outer contour, focusing on the static spatial properties of the mouse's outer contour boundary (such as length, width, angle, curvature, etc.). Therefore, the meaning and content of mouse form characteristics and mouse shape characteristics are different. In embodiments of the present invention, corresponding unique characteristics are determined from the perspective of mouse morphology to perform target detection and recognition on mice. Furthermore, for several preset mouse-specific morphological features, a preset mouse-specific morphological feature set can be used to represent them. Thus, determining the preset mouse-specific morphological features is expressed as determining the preset mouse-specific morphological feature set, wherein the preset mouse-specific morphological feature set includes several preset mouse-specific morphological features, thereby being able to more accurately cover several preset mouse-specific morphological features of the mouse.

[0022] Determine the current monitoring image and corresponding rat monitoring audio corresponding to the preset target farmland, including:

[0023] Determine the rat monitoring video and the corresponding rat monitoring audio corresponding to the preset target farmland;

[0024] Determining a plurality of single-frame monitoring video images corresponding to the mouse monitoring video;

[0025] Based on a preset image determination method, a current monitoring image corresponding to the preset target farmland is determined from a plurality of single-frame monitoring video images.

[0026] Specifically, a pre-set image determination method, namely, a preset image determination method, represents a method of determining a frame of monitoring video image from several single-frame monitoring video images corresponding to the rat monitoring video as the current monitoring image corresponding to the preset target farmland. The preset image determination method may be a random method, namely, randomly determining a frame of monitoring video image from several single-frame monitoring video images corresponding to the rat monitoring video as the current monitoring image corresponding to the preset target farmland. The preset image determination method may also be to use the first frame of monitoring video image from several single-frame monitoring video images corresponding to the rat monitoring video as the current monitoring image corresponding to the preset target farmland. The preset image determination method is not limited here.

[0027] According to the above settings, the rat monitoring video and its corresponding rat monitoring audio corresponding to the preset target farmland are determined. The rat monitoring video and its corresponding rat monitoring audio are the audio and video corresponding to the preset target farmland, that is, the audio and video of the same monitoring space target and the same time period. Then the rat monitoring video is converted into a corresponding single-frame image, so as to determine a number of single-frame monitoring video images corresponding to the rat monitoring video. The single-frame monitoring video image represents a single frame of an image based on the monitoring video. The several single-frame monitoring video images are generally single-frame images based on a time series, and correspond to the timing of the rat monitoring audio. Then, based on the preset image determination method, the current monitoring image corresponding to the preset target farmland is determined from the several single-frame monitoring video images. Therefore, the current monitoring image corresponding to the preset target farmland is determined by the video stream of the rat monitoring video corresponding to the preset target farmland and its corresponding rat monitoring audio. This ensures the temporal continuity of the several single-frame monitoring video images and corresponds to the rat monitoring audio, so that rats can be detected based on the corresponding several single-frame monitoring video images and rat monitoring audio, thereby improving the effect and accuracy of rat detection.

[0028] S102. When it is determined that the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior, and based on the preset mouse-specific morphological features and the preset mouse target detection YOLO model, when it is preliminarily determined that the current monitoring image contains a mouse image, determine the mouse-specific morphological features of the current frame corresponding to the current monitoring image.

[0029] A pre-set mouse target detection YOLO model, i.e., a preset mouse target detection YOLO model, refers to a model based on the YOLO model that uses mice as detection targets. The YOLO model, i.e., the YOLO (You Only Look Once) model, is a real-time target detection model. Pre-set mouse target detection YOLO models include but are not limited to YOLOv3 to YOLOv8.

[0030] A mouse behavior voiceprint recognition model is pre-set, that is, a preset mouse behavior voiceprint recognition model. The preset mouse behavior voiceprint recognition model represents a voiceprint recognition model used to identify the sounds corresponding to mouse behaviors. The preset mouse behavior voiceprint recognition model includes but is not limited to deep neural networks (DNN), convolutional neural networks (CNN), and recurrent neural networks (RNN). The sounds corresponding to mouse behaviors include but are not limited to the squeaking sounds of mice corresponding to mouse interactive behaviors, the screaming sounds of mice corresponding to mouse being invaded, the gnawing sounds of mice corresponding to mouse gnawing behaviors, and the friction sounds of mice corresponding to mouse movement behaviors.

[0031] Based on the above description, according to the preset mouse-specific morphological features, that is, using the preset mouse-specific morphological features as the detection target to identify mice, and based on the preset mouse target detection YOLO model, a preliminary judgment is made as to whether the current monitoring image contains a mouse image, that is, whether the current monitoring image contains the corresponding preset mouse-specific morphological features, so as to preliminarily judge whether the current monitoring image contains a mouse image, and based on the preset mouse behavior voiceprint recognition model, it is judged whether the mouse monitoring audio contains the mouse behavior sound corresponding to the mouse behavior, and the mouse behavior sound represents the sound corresponding to the mouse behavior, and the mouse behavior sound is as described above.

[0032] When it is determined that the mouse monitoring audio contains the mouse behavior sound corresponding to the mouse behavior, and in the above preliminary judgment, it is preliminarily determined that the current monitoring image contains the mouse image, since the mouse behavior is generally accompanied by sound and shape, this indicates that the current monitoring image is very likely to contain the mouse image, but further verification is needed. Therefore, the current frame mouse-specific morphological features corresponding to the current monitoring image are determined. The current frame mouse-specific morphological features represent the corresponding mouse-specific morphological features contained in the current monitoring image. The current frame mouse-specific morphological features can be a specific preset mouse-specific morphological feature mentioned above, or a collection of several preset mouse-specific morphological features. There is no limitation here. The current frame mouse-specific morphological features corresponding to the current monitoring image can be obtained by outputting the corresponding detected features when performing target detection based on the preset mouse target detection YOLO model.

[0033] Similarly, when it is preliminarily determined that the current single-frame monitoring video image does not contain a mouse image, or the mouse monitoring audio does not contain the sound of mouse behavior, it indicates that the current monitoring image most likely does not contain a mouse image. The default is that the current monitoring image does not contain a mouse image, and the unique morphological features of the current frame corresponding to the current single-frame monitoring video image are uncertain, so no further verification is required.

[0034] In summary, by combining voiceprint recognition with target detection based on the unique morphological characteristics of mice, a cross-modal parallel detection strategy in different dimensions can improve the accuracy of the above preliminary judgment, and thus effectively improve the accuracy of farmland mouse detection.

[0035] S103. When it is determined that the unique morphological features of the mouse in the current frame are consistent with the mouse movement morphological trajectory, it is finally determined that the current monitoring image contains a mouse image, and based on the above-mentioned final determinations, the degree of rodent damage corresponding to the preset target farmland is determined.

[0036] Since the behavior of mice is generally a continuous expression accompanied by sound and shape within a certain period of time, after preliminarily determining that the current monitoring image contains a mouse image, we then judge from the perspective of the continuous morphological expression law corresponding to the above continuous expression (i.e., the mouse movement morphological trajectory) whether the unique morphological features of the mouse in the current frame corresponding to the current monitoring image are consistent with the mouse movement morphological trajectory, so as to verify whether the current monitoring image does indeed contain a mouse image.

[0037] Based on the above description, it is determined whether the unique mouse morphological features of the current frame conform to the mouse motion morphological trajectory. The mouse motion morphological trajectory represents a set of expressions of the morphological manifestations of the corresponding mouse motion and the laws of manifestation based on time sequence or order. For example, in the case of a mouse running continuously, since the mouse's running posture is characterized by an elongated body and alternating extension of the front and back legs (a "wave-like" gait), the mouse's running motion morphological trajectory at least includes the alternating extension of the front and back legs (a "wave-like" gait) based on time sequence. In other words, the expression of the alternating extension of the front and back legs (a "wave-like" gait) based on time sequence is the mouse's running motion morphological trajectory. The unique mouse morphological features corresponding to a single frame of monitoring image can be: the unique mouse morphological feature corresponding to one frame of monitoring image is the extension of the front legs, the unique mouse morphological feature corresponding to the next frame of monitoring image is the extension of the hind legs, and the unique mouse morphological feature corresponding to the next frame of monitoring image is the extension of the front legs again, .... The alternating front and hind leg extensions represent the morphological manifestations of the mouse's running motion and the laws of manifestation based on time sequence, and so on.

[0038] Therefore, when it is determined that the unique morphological features of the mouse in the current frame are consistent with the mouse's motion morphological trajectory, it indicates that from the perspective of the continuous morphological expression law corresponding to the above-mentioned continuous expression (i.e., the mouse's motion morphological trajectory), the occurrence of the unique morphological features of the mouse in the current frame corresponding to the current monitoring image is consistent with the continuous morphological expression law corresponding to the corresponding behavior of the mouse, that is, from the perspective of the continuous morphological expression law corresponding to the above-mentioned continuous expression (i.e., the mouse's motion morphological trajectory), the current monitoring image should correspond to the unique morphological features of the mouse in the current frame. For example, in the case of a mouse running continuously, since the mouse's running posture law is that the body is elongated and The front and rear legs are extended alternately (the gait is "wavy"), that is, the running movement trajectory of the mouse at least includes the front and rear legs being extended alternately (the gait is "wavy"). Therefore, assuming that the running posture of the previous frame monitoring image corresponds to the previous frame mouse's unique morphological feature of the front legs being extended, according to the running posture law, the current frame mouse's unique morphological feature corresponding to the running posture of the current monitoring image should be the hind legs being extended. If this is indeed the case, that is, the current frame mouse's unique morphological feature is indeed the hind legs being extended, it indicates that the current frame mouse's unique morphological feature is consistent with the mouse's movement trajectory, and it can also further prove the preliminary judgment that the current monitoring image contains The mouse image has a greater probability of being the correct judgment, so it is assumed that the current monitoring image contains a mouse image, and based on the above final judgments, the degree of rodent damage corresponding to the preset target farmland is determined. The degree of rodent damage indicates the degree of damage caused by mice to the preset target farmland, that is, the degree of disease caused by mice to the preset target farmland. The degree of rodent damage can be measured according to the size of the mouse population (that is, the number of mice detected in the preset target farmland), the activity level of the mice, or a combination of the above indicators. Otherwise, it indicates that from the perspective of the continuous morphological expression law corresponding to the above continuous expression (that is, the mouse movement morphological trajectory), the current monitoring image is determined. The occurrence of the corresponding unique morphological features of the mouse in the current frame does not conform to the continuous morphological expression rules corresponding to the corresponding behaviors of the mouse. That is, from the perspective of the continuous morphological expression rules corresponding to the above-mentioned continuous expression (that is, the morphological trajectory of the mouse's movements), the current monitoring image should not correspond to the unique morphological features of the mouse in the current frame, indicating that the above-mentioned preliminary judgment may have produced a misjudgment. It is finally determined that the current monitoring image does not contain a mouse image, and the preliminary judgment is corrected. Thus, the progressive hierarchical detection strategy of "preliminary judgment-rejudgment and verification" is realized, which can perform compound verification of mouse detection from different angles and effectively improve the accuracy of farmland mouse detection.

[0039] In an embodiment of the present invention, the current monitoring image and the corresponding mouse monitoring audio corresponding to the preset target farmland are determined, and the preset mouse-specific morphological features are determined; when it is determined that the mouse monitoring audio contains the mouse behavior sound corresponding to the mouse behavior, and based on the preset mouse-specific morphological features, and based on the preset mouse target detection YOLO model, when it is preliminarily determined that the current monitoring image contains a mouse image, the current frame mouse-specific morphological features corresponding to the current monitoring image are determined; when it is determined that the current frame mouse-specific morphological features are consistent with the mouse action morphological trajectory, it is finally determined that the current monitoring image contains a mouse image, and based on several of the above final determinations, the preset target farmland is determined. The corresponding degree of rodent damage is calculated, thereby not only realizing a cross-modal parallel detection strategy of different dimensions that combines voiceprint recognition with target detection based on the unique morphological characteristics of mice, but also realizing a progressive hierarchical detection strategy of "preliminary judgment-rejudgment and verification". Compared with the single-dimensional target detection in traditional technologies, it can perform compound verification of mouse detection from different angles, and can effectively improve the accuracy of farmland mouse detection to adapt to the environmental characteristics of mice in farmland where the environmental background is complex, the features are not obvious, and the fur color presents a protective color, thereby accurately assessing the damage caused by mice to farmland, improving the accuracy of farmland mouse damage detection, and realizing automatic and efficient intelligent monitoring of farmland mice and their damage.

[0040] In one embodiment, see Figure 3 , Figure 3 This is a schematic diagram of the first sub-flow of the method for detecting the degree of mouse disease based on YOLO provided in an embodiment of the present invention. Figure 3 As shown, in this embodiment, when determining that the unique morphological features of the mouse in the current frame conform to the mouse motion morphological trajectory, the following steps are included:

[0041] S301, determining a number of time-series-based adjacent monitoring images corresponding to the current monitoring image;

[0042] S302, determining unique mouse morphological features in adjacent frames corresponding to the monitored adjacent images, and obtaining unique mouse morphological features in a plurality of adjacent frames;

[0043] S303, based on the corresponding time sequence, combining the mouse-specific morphological features of the current frame and the mouse-specific morphological features of all adjacent frames into a sequence to obtain a mouse-specific morphological feature sequence;

[0044] S304, determining whether the mouse-specific morphological feature sequence conforms to the preset mouse motion morphological rule corresponding to the mouse;

[0045] S305: If the above judgment is yes, it is determined that the unique morphological feature of the mouse in the current frame is consistent with the mouse motion trajectory;

[0046] S306: If the above judgment is no, it is determined that the unique morphological features of the mouse in the current frame do not conform to the mouse motion morphological trajectory.

[0047] The mouse movement morphological rules are set in advance, that is, the mouse movement morphological rules are preset. The preset mouse movement morphological rules indicate that the morphological manifestation rules of the mouse's corresponding movements are based on the expression of time sequence or order. For example, as mentioned above, when the mouse runs continuously, the front leg extension and the hind leg extension corresponding to the alternating extension of the mouse's front and rear legs when running alternate in sequence, and the unique morphological features of the mouse's front legs and the unique morphological features of the mouse's hind legs corresponding to several monitoring images based on time sequence appear alternately in sequence, which is the content of the preset mouse running morphological front and rear leg rules.

[0048] When judging whether the unique morphological features of the mouse in the current frame conform to the morphological trajectory of the mouse movement, that is, when judging whether the unique morphological features of the mouse in the current frame appear at the right time, from the perspective of sequential expression based on time sequence, a sequence of unique morphological features of the mouse corresponding to the unique morphological features of several adjacent frames corresponding to several monitoring images based on time sequence is constructed, and then it is judged whether the sequence of unique morphological features of the mouse conforms to the morphological manifestation law of the corresponding action based on the set expression of time sequence or order, thereby converting the relationship judgment between the unique morphological features of the mouse in the current frame (that is, the dimension of "point") and the morphological trajectory of the mouse movement (that is, the dimension of "set") into a set relationship judgment between the unique morphological features of the mouse (that is, the dimension of "set") and the preset morphological law of the mouse movement (that is, the dimension of "set"). Not only is the judgment dimension converted to facilitate judgment, but also a continuity judgment corresponding to the duration of the action is performed based on several continuous monitoring images. With the help of continuity judgment, an expanded judgment of the action logic from the dimension of the previous and subsequent continuous processes is realized, which can improve the accuracy of the corresponding judgment.

[0049] According to the above settings and concepts, a number of monitoring adjacent images based on a time series corresponding to the current monitoring image are determined. The monitoring adjacent images represent monitoring images that are in an adjacent frame relationship with the current monitoring image. The number of monitoring adjacent images can be monitoring images that are in front of the current monitoring image and are in a front adjacent frame relationship with the current monitoring image. The number of monitoring adjacent images can also be monitoring images that are behind the current monitoring image and are in a rear adjacent frame relationship with the current monitoring image. The number of monitoring adjacent images can also be monitoring images that are in the middle of the current monitoring image and are in a front adjacent frame relationship with the current monitoring image. And according to the current frame corresponding to the current monitoring image, the number of monitoring adjacent images can be determined. The unique morphological features of mice are determined in an identical or similar manner to the adjacent frames of each monitored adjacent image, and a number of unique morphological features of mice in adjacent frames are obtained. The unique morphological features of mice in adjacent frames represent the unique morphological features of mice in each monitored adjacent image. Then, based on the corresponding time series, that is, based on the time sequence corresponding to the current monitored image and all the monitored adjacent images, the unique morphological features of mice in the current frame and the unique morphological features of mice in all adjacent frames are combined into a sequence to obtain a unique morphological feature sequence of mice. The unique morphological feature sequence of mice represents a regular expression set of the corresponding unique morphological features of mice based on the time series. Determine whether the mouse-specific morphological feature sequence conforms to the preset mouse motion morphological rule corresponding to the mouse. For example, when the mouse-specific morphological feature sequence represents the running posture process, the preset mouse motion morphological rule is the above-mentioned regular expression set corresponding to the alternating extension of the front and rear legs. That is, determine whether the mouse-specific morphological feature sequence conforms to the regular expression corresponding to the alternating extension of the front and rear legs corresponding to the mouse, that is, determine whether the mouse-specific morphological feature sequence is expressed as follows: the mouse-specific morphological feature corresponding to one frame of monitoring image is the extension of the front legs, the mouse-specific morphological feature corresponding to the next frame of monitoring image is the extension of the hind legs, and the mouse-specific morphological feature corresponding to the next frame of monitoring image is the extension of the hind legs. The corresponding mouse-specific morphological feature is the extension of the front legs,..., the mouse-specific morphological features corresponding to the above-mentioned front leg extension and hind leg extension appear alternately in sequence. If the above judgment is yes, it is determined that the mouse-specific morphological feature of the current frame is consistent with the mouse's movement morphological trajectory. If the above judgment is no, it is determined that the mouse-specific morphological feature of the current frame is not consistent with the mouse's movement morphological trajectory. Not only is the judgment dimension converted to facilitate judgment, but also a continuity judgment corresponding to the duration of the action is performed based on several continuous monitoring images. With the help of continuity judgment, the expanded judgment of the action logic from the dimension of the previous and subsequent continuous processes is realized, which can improve the accuracy of the corresponding judgment.

[0050] In an embodiment of the present invention, whether the unique morphological features of the mouse in the current frame conform to the mouse motion morphological trajectory is determined by determining whether the unique morphological features of the mouse in the current frame conform to the mouse motion morphological trajectory. From the perspective of the mouse motion morphological trajectory, and with the help of the unique morphological features of the mouse in adjacent frames based on the time series, whether the current monitoring image contains a mouse image is re-verified to verify whether the initial determination that the current monitoring image contains a mouse image is a misjudgment. Since, when the current monitoring image contains a mouse image, the mouse will inevitably appear in the monitoring adjacent images of adjacent frames. At the same time, assuming that the appearance of a mouse in the current monitoring image and the monitoring adjacent images of a single frame is a misjudgment, it is impossible to make a misjudgment of whether the unique morphological features sequence based on the time series conforms to the preset mouse motion morphological trajectory. Therefore, with the help of the "preliminary determination-re-determination verification" composite determination detection strategy, compared with the single-dimensional target detection in traditional technologies, the accuracy of farmland mouse detection can be effectively improved to adapt to the environmental characteristics of mice in farmland, where the environmental background is complex, the features are not obvious, and the fur color presents a protective color.

[0051] In one embodiment, when it is determined that the mouse monitoring audio includes mouse behavior sounds corresponding to mouse behavior, and based on preset mouse-specific morphological features and a preset mouse target detection YOLO model, when it is preliminarily determined that the current monitoring image includes a mouse image, determining the mouse-specific morphological features of the current frame corresponding to the current monitoring image includes:

[0052] According to the preset mouse-specific morphological features and the current monitoring image, and based on the preset mouse target detection YOLO model, it is detected whether the current monitoring image contains a mouse image.

[0053] Based on the preset mouse behavior voiceprint recognition model, identify whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior.

[0054] When it is detected that the current monitoring image includes a mouse image and the mouse monitoring audio is identified to include mouse behavior sounds, a mouse-specific morphological feature of the current frame corresponding to the current monitoring image is determined.

[0055] When it is not detected that the current monitoring image contains a mouse image, or when it is not recognized that the mouse monitoring audio contains mouse behavior sounds, the mouse-specific morphological features of the current frame corresponding to the current monitoring image are determined.

[0056] As described above, a mouse target detection YOLO model is pre-set, that is, a preset mouse target detection YOLO model, and a mouse behavior soundprint recognition model is pre-set, that is, a preset mouse behavior soundprint recognition model. Then, based on the preset mouse target detection YOLO model, a preliminary judgment is made as to whether the current monitoring image contains a mouse image, so as to detect whether the current monitoring image contains a mouse image, and based on the preset mouse behavior soundprint recognition model, a judgment is made as to whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior, so as to identify whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior. When it is detected that the current monitoring image contains a mouse image and the mouse monitoring audio is identified as containing the mouse behavior sounds, that is, when a mouse is identified from both target detection and sound recognition, the unique morphological features of the mouse in the current frame corresponding to the current monitoring image are determined to further verify the target detection. When it is not detected that the current monitoring image contains a mouse image, or when it is not identified that the mouse monitoring audio contains mouse behavior sounds, that is, when a mouse is not identified from both target detection and sound recognition, the unique morphological features of the mouse in the current frame corresponding to the current monitoring image are not determined, and the target detection is not further verified.

[0057] In an embodiment of the present invention, a detection strategy that combines voiceprint recognition with target detection based on the unique morphological characteristics of mice is implemented to determine whether to perform a re-determination and verification. Since the form and sound corresponding to the behavior of mice are generally closely connected, that is, the sound and shape are dependent on each other, voiceprint recognition is combined with target detection based on image recognition. When detecting whether the current monitoring image contains a mouse image, a cross-modal parallel detection strategy of different dimensions that combines voiceprint recognition with target detection based on the unique morphological characteristics of mice is implemented. This can improve the accuracy of the above-mentioned preliminary judgment, and then perform a re-determination and verification, and perform multiple composite verifications on mouse detection from different angles, thereby effectively improving the accuracy of farmland mouse detection as a whole.

[0058] In one embodiment, detecting whether the current monitoring image contains a mouse image based on preset mouse-specific morphological features and the current monitoring image and based on a preset mouse target detection YOLO model includes:

[0059] A preset mouse-specific morphological feature set is determined, wherein the preset mouse-specific morphological feature set includes at least one of the following: a preset mouse-specific morphological feature of the head, a preset mouse-specific morphological feature of the trunk, and a preset mouse-specific morphological feature of the tail.

[0060] Based on the preset mouse target detection YOLO model, detect whether the current monitoring image contains the preset mouse head unique morphological features, the preset mouse torso unique morphological features, or the preset mouse tail unique morphological features.

[0061] If the above detection result is yes, it is determined that the current monitoring image is detected to include a mouse image.

[0062] If the above detection result is negative, it is determined that the current monitoring image is not detected to include a mouse image.

[0063] According to the morphological characteristics of mice, the unique morphological characteristics of mice are divided into morphological characteristics of the head, trunk and tail, i.e., preset unique morphological characteristics of the mouse head, preset unique morphological characteristics of the mouse trunk and preset unique morphological characteristics of the mouse tail, and a morphological feature set is formed, i.e., preset unique morphological characteristics of mice. The preset unique morphological characteristics set of mice includes at least one of the following: preset unique morphological characteristics of the mouse head, preset unique morphological characteristics of the mouse trunk and preset unique morphological characteristics of the mouse tail, wherein the preset unique morphological characteristics of the mouse head include but are not limited to a pointed snout, small round ears (ear length ≈ distance from eye to nose), protruding whiskers (antennae), gnawing posture (front paw holding food, head The preset unique morphological characteristics of a mouse's torso include but are not limited to being soft and without edges, a slightly swollen abdomen, short forelimbs with sharp claws, slender hind limbs, a running posture (the body is elongated, and the front and rear legs are alternately stretched (a "wavy gait")) and a combination of several of the above forms. The preset unique morphological characteristics of a mouse's tail include but are not limited to a length ≈ body length (house mouse) or longer (such as brown rat), ring-shaped scales on the surface, a thicker base, a tail posture (tilted / dragging on the ground) and a combination of several of the above forms. Thus, mice can be identified based on a single unique mouse morphological feature or a combination of several unique mouse morphological features contained in the preset set of unique mouse morphological characteristics.

[0064] According to the above description, a preset set of unique morphological features of a mouse is determined, wherein the preset set of unique morphological features of a mouse includes at least one of the following: a preset unique morphological feature of a mouse head, a preset unique morphological feature of a mouse torso, and a preset unique morphological feature of a mouse tail; and based on the preset mouse target detection YOLO model, it is detected whether the current monitoring image contains a single morphological feature or a combination of morphological features that can identify a mouse corresponding to the preset unique morphological feature of the mouse head, the preset unique morphological feature of the mouse torso, or the preset unique morphological feature of the mouse tail, and if the above detection is yes, it is determined that the current monitoring image is detected to contain a mouse image. Similarly, if the above detection is no, it is determined that the current monitoring image is not detected to contain a mouse image.

[0065] In order to verify each other during the target detection process and implement target detection more accurately, based on the preset mouse target detection YOLO model, when detecting whether the current monitoring image contains the preset mouse head unique morphological features, the preset mouse torso unique morphological features or the preset mouse tail unique morphological features, it is detected whether the current monitoring image contains the unique morphological features of at least two parts of the preset mouse head unique morphological features, the preset mouse torso unique morphological features and the preset mouse tail unique morphological features, and a comprehensive verification test is performed by repeatedly detecting the unique morphological features of different parts of the mouse, that is, the mouse is detected from the unique morphological features of one part, and at the same time, if the mouse is also detected from the unique morphological features of another part, the repeated superposition of the two will achieve successful detection of the mouse target, and the accuracy of mouse target detection will be further improved, thereby improving the accuracy of mouse target detection. However, compared with the traditional technology of overall detection of the mouse outline, mouse target detection is still relatively simple.

[0066] Therefore, when performing mouse target detection, the mouse target detection is transformed from the traditional mouse overall contour detection to the detection of local unique morphological features. The mouse is identified and detected by using points instead of surfaces and local instead of the whole. There is no need to make a complete judgment of the overall features. The flexibility and accuracy of mouse detection can be improved to adapt to the environmental characteristics of mice in farmland, where the environmental background is complex, the features are not obvious, and the fur color presents a protective color.

[0067] It should be noted that the difference between the preset mouse-specific morphological features and the key point features is that the single preset mouse-specific morphological features in the embodiment of the present invention can perform target recognition and detection of the mouse as a whole, while the key point features in traditional technology only represent the local key points of the mouse. A single key point feature cannot represent the mouse as a whole, and a single key point feature cannot be used to perform target recognition and detection of the mouse as a whole. A single key point feature must be combined with other key point features to perform target recognition and detection of the mouse as a whole.

[0068] In an embodiment of the present invention, the detection of whether the current monitoring image contains a mouse image is converted into "detecting whether the current monitoring image contains a preset mouse head unique morphological feature, a preset mouse trunk unique morphological feature or a preset mouse tail unique morphological feature". Since the preset mouse unique morphological feature represents a high degree of recognition for identifying mice by using the mouse's unique morphological feature, that is, a mouse can be identified as a mouse by seeing the corresponding preset mouse unique morphological feature. Thus, the mouse is detected by replacing the entire mouse with a local unique morphology that has mouse recognition, and the detection of mice is carried out from point to surface. Compared with the target detection of the overall shape of the mouse in traditional technology, the flexibility and robustness of mouse detection in farmland can be improved to adapt to the environmental characteristics of mice in farmland, where the environmental background is complex, the features are not obvious, and the fur color presents a protective color, and the accuracy of mouse detection in farmland can be effectively improved.

[0069] In one embodiment, based on a preset mouse behavior soundprint recognition model, identifying whether the mouse monitoring audio includes mouse behavior sounds corresponding to mouse behavior includes at least one of the following:

[0070] Detect whether the mouse monitoring audio contains mouse squeaking sounds corresponding to mouse interactive behavior;

[0071] Detect whether the rat monitoring audio contains rat screams corresponding to rat invasion behavior;

[0072] Detect whether the mouse monitoring audio contains the mouse gnawing sound corresponding to the mouse gnawing behavior;

[0073] Detect whether the mouse monitoring audio contains the mouse friction sound corresponding to the mouse's movement behavior;

[0074] Detects whether the mouse monitoring audio contains sick mouse sounds corresponding to sick mouse behaviors.

[0075] Based on a preset mouse behavior soundprint recognition model, identifying whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior includes at least one of the following:

[0076] Detect whether the mouse monitoring audio contains mouse squeaking sounds corresponding to mouse interactive behaviors, wherein mouse interactive behaviors include but are not limited to behaviors corresponding to young mice begging for food and social interactions, corresponding to squeaks, and the characteristics of mouse squeaking sounds include but are not limited to short, high-frequency, single or repeated.

[0077] Detect whether the mouse monitoring audio contains mouse screams corresponding to mouse aggression behaviors, where mouse aggression behaviors include but are not limited to pain and behaviors corresponding to being attacked by predators, which correspond to screams. Characteristics of mouse screams include but are not limited to being sharp, high-intensity, and lasting for 0.5 to 2 seconds.

[0078] Detect whether the rat monitoring audio contains rat gnawing sounds corresponding to rat gnawing behaviors, wherein rat gnawing behaviors include but are not limited to behaviors corresponding to gnawing on crops or building materials, and the characteristics of the rat gnawing sounds are but are not limited to intermittent, broadband, and regular rhythm.

[0079] Detect whether the rat monitoring audio contains rat friction sounds corresponding to rat movement behaviors, where rat movement behaviors include but are not limited to running, digging holes, or crossing vegetation, which correspond to friction sounds. Characteristics of rat friction sounds include but are not limited to low-frequency rustling sounds and irregular patterns.

[0080] Detect whether the mouse monitoring audio contains sick mouse sounds corresponding to sick mouse behaviors, where sick mouse behaviors include but are not limited to respiratory infections, corresponding to high-frequency screaming or coughing, and sick mouse sounds include but are not limited to coughing / sneezing.

[0081] Since the form and sound corresponding to the mouse's behavior are generally closely linked, combining the corresponding audio recognition of the mouse during mouse target detection can further improve the accuracy of mouse target detection.

[0082] The embodiments of the present invention combine voiceprint recognition with target detection based on image recognition by means of sound detection covering different life scenarios of mice. Since the form corresponding to the behavior of mice is generally closely linked to the sound, that is, the form produced by the corresponding behavior of mice must correspond to the corresponding sound, that is, the sound and form are interdependent. When detecting whether the current monitoring image contains a mouse image, a cross-modal parallel detection strategy of different dimensions combining voiceprint recognition with target detection based on the unique morphological characteristics of mice is implemented, which can improve the accuracy of the above-mentioned preliminary judgment, and then conduct re-judgment and verification, and perform multiple composite verifications on mouse detection from different angles, thereby effectively improving the accuracy of farmland mouse detection as a whole.

[0083] In one embodiment, determining the current monitoring image corresponding to the preset target farmland includes:

[0084] Determining several types of current initial monitoring images of different spectral bands corresponding to a preset target farmland, the several types of current initial monitoring images including at least two of the following: a visible light current initial monitoring image, a near infrared current initial monitoring image, and a thermal infrared current initial monitoring image;

[0085] All current initial monitoring images are aligned and spliced ​​to obtain the current monitoring image corresponding to the preset target farmland.

[0086] Determine several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland, and the several types of current initial monitoring images include at least two of the following: visible light current initial monitoring image, near infrared current initial monitoring image, and thermal infrared current initial monitoring image, wherein the visible light current initial monitoring image represents the current initial monitoring image based on visible light (RGB), and the visible light current initial monitoring image can be acquired based on an RGB image or an RGB video acquisition device, the near infrared current initial monitoring image represents the current initial monitoring image based on near infrared light (NIR), and the near infrared current initial monitoring image can be acquired based on a near infrared camera, and the thermal infrared current initial monitoring image represents the current initial monitoring image based on thermal infrared light (TIR), and the thermal infrared current initial monitoring image can be acquired based on a thermal infrared (TIR) ​​camera. Thus, determine several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland, align and splice all the current initial monitoring images, and obtain the current monitoring image corresponding to the preset target farmland.

[0087] Among them, aligning and stitching all current initial monitoring images means geometrically correcting and spatially matching images from different spectral bands (such as RGB, near-infrared, thermal infrared, etc.) to ensure that they overlap accurately at the pixel level and generate a fused image representation. Image alignment refers to the use of geometric transformations (such as translation, rotation, and scaling) to make the same scene points in different images completely coincide in pixel coordinates to eliminate offsets caused by differences in sensor position, perspective, or time. For example, the mouse hotspot in the thermal infrared image is aligned with the mouse outline in the visible light image; image stitching is to merge the aligned multi-band images into a single multi-channel image (such as an RGB + near-infrared 4-channel image) to generate a fused image representation that retains the information of each band for subsequent model processing.

[0088] The embodiment of the present invention realizes multi-spectral fusion image enhancement by aligning and splicing several current initial monitoring images of different spectral bands. Rodent pest monitoring can be upgraded from "single target detection" to "target-environment collaborative analysis" image enhancement strategy, which can improve the robustness of mouse target detection to adapt to the environmental characteristics of mice in farmland, such as complex environmental background, unclear features and protective color of fur. It can further improve the accuracy of farmland mouse detection, and then accurately assess the damage caused by mice to farmland, significantly improve the accuracy and efficiency of agricultural management, and thus realize automatic and efficient intelligent monitoring of farmland mice and their damage.

[0089] In one embodiment, see Figure 4 , Figure 4 This is a schematic diagram of the second sub-flow of the method for detecting the degree of mouse disease based on YOLO provided in an embodiment of the present invention. Figure 4 As shown, in this embodiment, several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland are determined, including:

[0090] S401, responding to different preset collection trigger events and using corresponding preset video collection equipment, collecting several types of rat monitoring videos in different spectral bands corresponding to preset target farmland;

[0091] S402, determining a number of single-frame monitoring video images corresponding to each type of the mouse monitoring video;

[0092] S403. According to the preset image determination method and based on the same time sequence, determine the current initial monitoring image of each type corresponding to the preset target farmland from several single-frame monitoring video images of each type, and obtain several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland.

[0093] The collection trigger event is set in advance, that is, the preset collection trigger event. The preset collection trigger event refers to the event that triggers the collection of the mouse monitoring video corresponding to the preset target farmland. The preset collection trigger event includes but is not limited to time trigger, event occurrence trigger and the trigger of the combination of the above-mentioned related factors. The time trigger means that the corresponding collection is triggered when the peak time of mouse activity corresponding to 2 hours after dusk (18:00-20:00 after sunset), 1-2 hours before dawn (03:00-05:00), and the new moon occurs. The event occurrence trigger means that the corresponding collection is triggered when the corresponding events corresponding to cloudy days, underground sensors detecting digging signals, and traps placed on "rat paths" (such as pedal-type mousetraps) detecting vibration signals occur.

[0094] In addition, for different preset acquisition trigger events, corresponding preset video acquisition equipment is deployed. For example, for the preset acquisition trigger event corresponding to a sunny day, a visible light video acquisition equipment (such as an RGB camera equipment) and a near-infrared video acquisition equipment are deployed. For the preset acquisition trigger event corresponding to a sunny night without moonlight, a thermal infrared video acquisition equipment and a preset short-wave infrared video acquisition equipment are deployed. And so on. Corresponding video acquisition equipment is deployed for different acquisition environments and acquisition conditions, so that the acquisition of mouse monitoring videos is adapted to the corresponding acquisition conditions, so as to adapt to the environmental characteristics of mice in farmland, where the environmental background is complex, the features are not obvious, and the fur color presents a protective color. This can further improve the accuracy of farmland mouse detection, and then accurately assess the damage caused by mice to farmland.

[0095] According to the above description, in response to different preset collection trigger events, the corresponding preset video collection device is enabled, and based on the corresponding preset video collection device, several types of rat monitoring videos of different spectral bands corresponding to the preset target farmland are collected. Several types of rat monitoring videos of different spectral bands can be collected based on the same video collection device that supports the collection of several types of rat monitoring videos of different spectral bands, or can be collected based on different video collection devices for the corresponding spectral band types. For example, visible light RGB rat monitoring videos and near infrared (NIR) rat monitoring videos can be collected based on a dual-spectral camera, or an RGB camera can be used to collect visible light RGB rat monitoring videos, and a near infrared camera can be used to collect near infrared (NIR) rat monitoring videos, and so on.

[0096] Based on several types of rat monitoring videos of different spectral bands corresponding to the preset target farmland, for each type of rat monitoring video corresponding to each spectral band, several single-frame monitoring video images corresponding to each type of rat monitoring video are determined, that is, the video is converted into several single-frame monitoring video images.

[0097] According to the preset image determination method and based on the same time sequence, the current initial monitoring image of each type corresponding to the preset target farmland is determined from several single-frame monitoring video images of each type, and several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland are obtained. That is, according to the same image determination method, several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland are determined from several single-frame monitoring video images of each type.

[0098] In response to different preset collection trigger events and based on corresponding preset video collection equipment, several types of rat monitoring videos in different spectral bands corresponding to preset target farmland are collected, including at least one of the following:

[0099] When a preset acquisition trigger event corresponds to a preset good light condition, a preset visible light video acquisition device and a preset near-infrared video acquisition device are used to acquire several types of mouse monitoring videos in different spectral bands corresponding to the preset target farmland;

[0100] When the preset acquisition trigger event corresponds to the preset low-light condition, the preset RGB-NIR multispectral camera video acquisition device and the preset thermal infrared video acquisition device are used to collect several types of rat monitoring videos in different spectral bands corresponding to the preset target farmland;

[0101] When the preset acquisition trigger event corresponds to the preset light-free condition, based on the preset thermal infrared video acquisition device and the preset short-wave infrared video acquisition device, several types of mouse monitoring videos of different spectral bands corresponding to the preset target farmland are collected.

[0102] Specifically, in response to different preset collection trigger events and based on corresponding preset video collection equipment, several types of rat monitoring videos in different spectral bands corresponding to preset target farmland are collected, including at least one of the following:

[0103] When the preset acquisition trigger event corresponds to a preset good light condition, for example, when the preset acquisition trigger event corresponds to a sunny day, based on the preset visible light video acquisition device (such as an RGB camera) and the preset near-infrared video acquisition device (such as an NIR camera), several types of mouse monitoring videos of different spectral bands corresponding to the preset target farmland are collected, thereby obtaining visible light images to detect mouse morphology, and capturing near-infrared bands (healthy vegetation has high reflectivity, and rodent-infested areas have low reflectivity), thereby using the sunlight reflection spectrum to detect mouse activity traces and crop damage, and can enhance the contrast between mice and crop background; similarly, when the acquisition trigger event corresponds to a preset weak light condition, for example, When the preset acquisition trigger event corresponds to dusk or cloudy day, based on the preset RGB-NIR multispectral camera video acquisition device and the preset thermal infrared video acquisition device, several types of rat monitoring videos of different spectral bands corresponding to the preset target farmland are collected under the video acquisition conditions corresponding to dusk or cloudy day; when the preset acquisition trigger event corresponds to the preset light-free light condition, for example, when the preset acquisition trigger event corresponds to the lightless night, based on the preset thermal infrared video acquisition device and the preset shortwave infrared video acquisition device, several types of rat monitoring videos of different spectral bands corresponding to the preset target farmland are collected under the video acquisition conditions corresponding to the lightless night.

[0104] The embodiment of the present invention is event-driven, and for different preset acquisition trigger events, corresponding preset video acquisition equipment is used to collect several types of rat monitoring videos of different spectral bands corresponding to the preset target farmland, and then several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland are determined. According to different acquisition trigger event conditions, the corresponding preset video acquisition equipment can be started, so that the acquisition of the rat monitoring video is adapted to the corresponding acquisition conditions, which can improve the acquisition quality of the rat monitoring video, so that the corresponding rat detection has a better detection effect, and can further improve the accuracy of farmland rat detection.

[0105] In one embodiment, the rodent damage level corresponding to the preset target farmland is determined based on the above-mentioned final determinations, including:

[0106] Based on the above final determinations, the rat activity intensity index corresponding to the preset target farmland is calculated, and includes at least one of the following:

[0107] Determining whether the rat activity intensity index is greater than or equal to a preset first rat activity intensity index threshold, and if so, determining that the rodent damage level corresponding to the preset target farmland is in a high-risk state;

[0108] Determine whether the rat activity intensity index is less than the preset second rat activity intensity index threshold, and if the above judgment is yes, determine that the rodent damage level corresponding to the preset target farmland is in a low-risk state, wherein the preset first rat activity intensity index threshold is greater than or equal to the preset second rat activity intensity index threshold.

[0109] A first rat activity intensity index threshold is set in advance, that is, the first rat activity intensity index threshold is preset. The preset first rat activity intensity index threshold represents the lowest critical value of a higher rat activity intensity. The higher the rat activity intensity, the greater the harm to the preset target farmland. The rodent damage level corresponding to the preset target farmland is at high risk. Similarly, the preset second rat activity intensity index threshold represents the highest critical value of a lower rat activity intensity. The lower the rat activity intensity, the smaller the harm to the preset target farmland. The rodent damage level corresponding to the preset target farmland is at low risk. When the preset first rat activity intensity index threshold and the preset second rat activity intensity index threshold are not the same value, there may still be medium risk between high risk and low risk.

[0110] According to the above description, since the above final determination is the target detection result corresponding to a single rat target detection, based on several of the above final determinations, the rat activity intensity index corresponding to the preset target farmland is calculated. The rat activity intensity index includes but is not limited to the size of the rat population and the frequency of rat activity. The size of the rat population is the number of rats detected. The frequency of rat activity can be measured by the number of rat appearances within a period of time, for example, the number of rat appearances within a day, and includes at least one of the following:

[0111] Determining whether the rat activity intensity index is greater than or equal to a preset first rat activity intensity index threshold, and if so, determining that the rat infestation level corresponding to the preset target farmland is in a high-risk state; similarly, if not, determining that the rat infestation level corresponding to the preset target farmland is not in a high-risk state;

[0112] Determine whether the rat activity intensity index is less than the preset second rat activity intensity index threshold value, and if the above judgment is yes, determine that the rodent infestation level corresponding to the preset target farmland is in a low-risk state. Similarly, if the above judgment is no, determine that the rodent infestation level corresponding to the preset target farmland is not in a low-risk state, wherein the preset first rat activity intensity index threshold value is greater than or equal to the preset second rat activity intensity index threshold value.

[0113] The embodiments of the present invention, through the accurate judgment of whether the current monitoring image contains a mouse image based on the above, can accurately assess the degree of damage caused by mice to farmland, realize automatic and efficient intelligent monitoring of farmland mice and their damage, and improve the level of agricultural intelligence.

[0114] It should be noted that the YOLO-based mouse disease severity detection method described in the above embodiments can recombine the technical features contained in different embodiments as needed to obtain a combined implementation plan, but all of them are within the scope of protection required by the present invention.

[0115] In one embodiment, a YOLO-based system for detecting the degree of disease in mice is provided. The YOLO-based system for detecting the degree of disease in mice corresponds to the YOLO-based method for detecting the degree of disease in mice in the above-mentioned embodiment. Figure 5 , Figure 5 This is a schematic block diagram of a system for detecting mouse disease severity based on YOLO provided by an embodiment of the present invention. Figure 5 As shown, the YOLO-based mouse disease degree detection system 500 includes a first determination module 501, a first judgment module 502, and a second judgment module 503. The above functional modules are described in detail as follows:

[0116] The first determination module 501 is used to determine the current monitoring image and the corresponding mouse monitoring audio corresponding to the preset target farmland, and determine the unique morphological characteristics of the preset mouse;

[0117] A first determination module 502 is configured to, upon determining that the mouse monitoring audio includes mouse behavior sounds corresponding to mouse behavior, determine mouse-specific morphological features of a current frame corresponding to the current monitoring image based on preset mouse-specific morphological features and a preset mouse object detection YOLO model, upon preliminarily determining that the current monitoring image includes a mouse image;

[0118] The second determination module 503 is used to finally determine that the current monitoring image contains a mouse image when it is determined that the unique morphological features of the mouse in the current frame are consistent with the mouse movement morphological trajectory, and determine the degree of rodent damage corresponding to the preset target farmland based on the above-mentioned final determinations.

[0119] In one embodiment, the second determining module 503 includes:

[0120] The first determination submodule is used to determine a number of frames of adjacent monitoring images corresponding to the current monitoring image based on a time sequence;

[0121] The second determining submodule is used to determine the unique morphological features of mice in adjacent frames corresponding to the monitored adjacent images, and obtain the unique morphological features of mice in a plurality of adjacent frames;

[0122] A sequence composition submodule is used to combine the unique morphological features of the mouse in the current frame with the unique morphological features of the mouse in all adjacent frames into a sequence based on the corresponding time sequence to obtain a unique morphological feature sequence of the mouse;

[0123] The first judgment submodule is used to judge whether the mouse-specific morphological feature sequence conforms to the preset mouse motion morphological rules corresponding to the mouse;

[0124] The first determination submodule is configured to determine, if the above determination is yes, whether the unique morphological features of the mouse in the current frame are consistent with the mouse motion morphological trajectory.

[0125] In one embodiment, the first determination module 502 includes:

[0126] A first detection submodule is configured to detect whether the current monitoring image contains a mouse image based on preset mouse-specific morphological features and the current monitoring image, and based on a preset mouse target detection YOLO model;

[0127] The first recognition submodule is configured to identify whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior based on a preset mouse behavior soundprint recognition model;

[0128] The third determination submodule is configured to determine the mouse-specific morphological features of the current frame corresponding to the current monitoring image when it is detected that the current monitoring image includes a mouse image and the mouse monitoring audio is identified to include mouse behavior sounds.

[0129] In one embodiment, the first detection submodule includes:

[0130] a fourth determining submodule, configured to determine a preset set of unique mouse morphological features, wherein the preset set of unique mouse morphological features includes at least one of the following: a preset unique mouse head morphological feature, a preset unique mouse torso morphological feature, and a preset unique mouse tail morphological feature;

[0131] The second detection submodule is used to detect whether the current monitoring image contains a preset mouse head unique morphological feature, a preset mouse torso unique morphological feature, or a preset mouse tail unique morphological feature based on a preset mouse target detection YOLO model;

[0132] A second determination submodule is configured to determine that the current monitoring image contains a mouse image if the above detection result is yes;

[0133] The third determination submodule is configured to determine that the current monitoring image does not contain a mouse image if the above detection result is negative.

[0134] In one embodiment, the first identification submodule includes at least one of the following:

[0135] a third detection submodule, configured to detect whether the mouse monitoring audio contains a mouse squeaking sound corresponding to mouse interactive behavior;

[0136] A fourth detection submodule is used to detect whether the mouse monitoring audio contains a mouse scream corresponding to a mouse being invaded;

[0137] a fifth detection submodule, configured to detect whether the mouse monitoring audio contains a mouse gnawing sound corresponding to a mouse gnawing behavior;

[0138] a sixth detection submodule, configured to detect whether the mouse monitoring audio includes a mouse friction sound corresponding to a mouse movement behavior;

[0139] The seventh detection submodule is used to detect whether the mouse monitoring audio contains the sound of a sick mouse corresponding to the mouse's sick behavior.

[0140] In one embodiment, the first determining module 501 includes:

[0141] a fifth determination submodule, configured to determine a plurality of types of current initial monitoring images of different spectral bands corresponding to a preset target farmland, the plurality of types of current initial monitoring images comprising at least two of the following: a visible light current initial monitoring image, a near infrared current initial monitoring image, and a thermal infrared current initial monitoring image;

[0142] The splicing submodule is used to align and splice all the current initial monitoring images to obtain the current monitoring image corresponding to the preset target farmland.

[0143] In one embodiment, the fifth determining submodule includes:

[0144] The first acquisition submodule is used to respond to different preset acquisition trigger events and, based on corresponding preset video acquisition devices, acquire several types of rat monitoring videos in different spectral bands corresponding to preset target farmland;

[0145] a sixth determining submodule, configured to determine a number of single-frame monitoring video images corresponding to each type of rat monitoring video;

[0146] The seventh determination submodule is used to determine the current initial monitoring image of each type corresponding to the preset target farmland from several single-frame monitoring video images of each type according to the preset image determination method and based on the same time sequence, and obtain several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland.

[0147] In one embodiment, the first acquisition submodule includes at least one of the following:

[0148] The second acquisition submodule is configured to acquire, when a preset acquisition trigger event corresponds to preset good light conditions, several types of rat monitoring videos in different spectral bands corresponding to a preset target farmland using a preset visible light video acquisition device and a preset near-infrared video acquisition device;

[0149] The third acquisition submodule is configured to acquire, when a preset acquisition trigger event corresponds to a preset low-light condition, several types of rat monitoring videos in different spectral bands corresponding to a preset target farmland based on a preset RGB-NIR multispectral camera video acquisition device and a preset thermal infrared video acquisition device;

[0150] The fourth acquisition submodule is used to collect several types of mouse monitoring videos in different spectral bands corresponding to the preset target farmland based on the preset thermal infrared video acquisition device and the preset short-wave infrared video acquisition device when the preset acquisition trigger event corresponds to the preset light-free condition.

[0151] In one embodiment, the second determination module 503 includes:

[0152] The statistical submodule is used to calculate the rat activity intensity index corresponding to the preset target farmland based on the above-mentioned final determinations, and includes at least one of the following:

[0153] The second judgment submodule is configured to judge whether the rat activity intensity index is greater than or equal to a preset first rat activity intensity index threshold, and if so, to determine that the rat infestation level corresponding to the preset target farmland is in a high-risk state;

[0154] The third judgment submodule is used to judge whether the rat activity intensity index is less than the preset second rat activity intensity index threshold, and if the above judgment is yes, it is determined that the rodent damage level corresponding to the preset target farmland is in a low-risk state, wherein the preset first rat activity intensity index threshold is greater than or equal to the preset second rat activity intensity index threshold.

[0155] The embodiment of the present invention provides a system for detecting the degree of mouse disease based on YOLO, which determines the current monitoring image and the corresponding mouse monitoring audio corresponding to the preset target farmland, and determines the preset mouse-specific morphological features; when it is determined that the mouse monitoring audio contains the mouse behavior sound corresponding to the mouse behavior, and based on the preset mouse-specific morphological features, and based on the preset mouse target detection YOLO model, when it is preliminarily determined that the current monitoring image contains a mouse image, the current frame mouse-specific morphological features corresponding to the current monitoring image are determined; when it is determined that the current frame mouse-specific morphological features are consistent with the mouse action morphological trajectory, it is finally determined that the current monitoring image contains a mouse image, and according to the above-mentioned final determinations, , determine the degree of rodent damage corresponding to the preset target farmland, thereby not only realizing a cross-modal parallel detection strategy of different dimensions that combines voiceprint recognition with target detection based on the unique morphological characteristics of mice, but also realizing a progressive hierarchical detection strategy of "preliminary judgment-rejudgment and verification". Compared with the single-dimensional target detection in traditional technologies, it can perform multiple composite verifications on mouse detection from different angles, effectively improving the accuracy of farmland mouse detection, so as to adapt to the environmental characteristics of mice in farmland, where the environmental background is complex, the features are not obvious, and the fur color presents a protective color, thereby accurately assessing the damage caused by mice to farmland, improving the accuracy of farmland rodent damage detection, and realizing automatic and efficient intelligent monitoring of farmland mice and their damage.

[0156] The specific limitations of the YOLO-based mouse disease detection system can be found in the limitations of the YOLO-based mouse disease detection method described above and will not be further elaborated here. Each module in the YOLO-based mouse disease detection system can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0157] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] The relevant data collection in the embodiments of the present invention complies with the requirements of relevant laws and regulations, such as China's Personal Information Protection Law, GDPR (EU General Data Protection Regulation) or information security standards of other countries and regions.

[0159] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for detecting the degree of mouse disease based on YOLO, characterized in that: include: Determine the current monitoring image and corresponding mouse monitoring audio corresponding to the preset target farmland, and determine the unique morphological characteristics of the preset mouse; Upon determining that the rat monitoring audio includes rat behavior sounds corresponding to rat behavior, and based on the preset rat-specific morphological features and a preset rat target detection YOLO model, upon preliminarily determining that the current monitoring image includes a rat image, determining the rat-specific morphological features of the current frame corresponding to the current monitoring image; When it is determined that the unique morphological features of the mouse in the current frame are consistent with the mouse movement morphological trajectory, it is finally determined that the current monitoring image contains a mouse image, and based on several of the above final determinations, the degree of rodent damage corresponding to the preset target farmland is determined.

2. The method for detecting the degree of mouse disease based on YOLO according to claim 1, characterized in that: When it is determined that the unique morphological feature of the mouse in the current frame is consistent with the mouse motion trajectory, the method includes: Determining a plurality of frames of adjacent monitoring images corresponding to the current monitoring image based on a time sequence; Determining unique mouse morphological features in adjacent frames corresponding to the monitored adjacent images to obtain unique mouse morphological features in a plurality of adjacent frames; Based on the corresponding time sequence, the mouse-specific morphological feature of the current frame and the mouse-specific morphological features of all adjacent frames are combined into a sequence to obtain a mouse-specific morphological feature sequence; Determining whether the mouse-specific morphological feature sequence conforms to the preset mouse motion morphological rule corresponding to the mouse; If the above judgment is yes, it is determined that the unique morphological features of the mouse in the current frame are consistent with the mouse action morphological trajectory.

3. The method for detecting the degree of disease in mice based on YOLO according to claim 1, wherein: The method further comprises: determining the mouse behavior sound corresponding to the mouse behavior in the mouse monitoring audio, and determining the mouse-specific morphological features of the current frame corresponding to the current monitoring image based on the preset mouse-specific morphological features and the preset mouse target detection YOLO model when the current monitoring image is preliminarily determined to include a mouse image. According to the preset mouse-specific morphological features and the current monitoring image, and based on a preset mouse target detection YOLO model, detecting whether the current monitoring image contains a mouse image; Based on a preset mouse behavior voiceprint recognition model, identifying whether the mouse monitoring audio contains mouse behavior sounds corresponding to mouse behavior; When it is detected that the current monitoring image includes a mouse image and it is recognized that the mouse monitoring audio includes the mouse behavior sound, the mouse-specific morphological features of the current frame corresponding to the current monitoring image are determined.

4. The method for detecting the degree of disease in mice based on YOLO as claimed in claim 3, wherein: The detecting whether the current monitoring image contains a mouse image based on the preset mouse-specific morphological features and the current monitoring image and based on a preset mouse target detection YOLO model includes: Determining a preset set of unique mouse morphological features, wherein the preset set of unique mouse morphological features includes at least one of the following: a preset unique mouse head morphological feature, a preset unique mouse torso morphological feature, and a preset unique mouse tail morphological feature; Based on a preset mouse target detection YOLO model, detecting whether the current monitoring image includes the preset mouse head-specific morphological features, the preset mouse torso-specific morphological features, or the preset mouse tail-specific morphological features; If the above detection result is yes, it is determined that the current monitoring image is detected to include a mouse image; If the above detection result is negative, it is determined that the current monitoring image is not detected to include a mouse image.

5. The method for detecting the degree of disease in mice based on YOLO as claimed in claim 3, characterized in that: Based on a preset mouse behavior voiceprint recognition model, identifying whether the mouse monitoring audio includes mouse behavior sounds corresponding to mouse behavior includes at least one of the following: detecting whether the mouse monitoring audio includes a mouse squeaking sound corresponding to a mouse interactive behavior; detecting whether the rat monitoring audio includes a rat screaming sound corresponding to a rat being invaded; detecting whether the mouse monitoring audio includes a mouse gnawing sound corresponding to a mouse gnawing behavior; detecting whether the mouse monitoring audio includes a mouse friction sound corresponding to a mouse action behavior; Detect whether the mouse monitoring audio includes a sick mouse sound corresponding to the sick mouse behavior.

6. The method for detecting the degree of disease in mice based on YOLO as claimed in claim 1, wherein: Determining the current monitoring image corresponding to the preset target farmland includes: Determining several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland, the several types of current initial monitoring images including at least two of the following: a visible light current initial monitoring image, a near infrared current initial monitoring image, and a thermal infrared current initial monitoring image; All the current initial monitoring images are aligned and spliced ​​to obtain the current monitoring image corresponding to the preset target farmland.

7. The method for detecting the degree of disease in mice based on YOLO according to claim 6, wherein: The determining of several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland includes: In response to different preset collection trigger events, and based on corresponding preset video collection equipment, collect several types of rat monitoring videos in different spectral bands corresponding to the preset target farmland; Determining a number of single-frame monitoring video images corresponding to each type of the mouse monitoring video; According to the preset image determination method and based on the same time sequence, the current initial monitoring image of each type corresponding to the preset target farmland is determined from several single-frame monitoring video images of each type, and several types of current initial monitoring images of different spectral bands corresponding to the preset target farmland are obtained.

8. The method for detecting the degree of disease in mice based on YOLO according to claim 7, wherein: The step of responding to different preset acquisition trigger events and acquiring several types of rat monitoring videos in different spectral bands corresponding to the preset target farmland based on corresponding preset video acquisition equipment includes at least one of the following: When the preset acquisition trigger event corresponds to a preset good light condition, a preset visible light video acquisition device and a preset near-infrared video acquisition device are used to acquire several types of mouse monitoring videos in different spectral bands corresponding to the preset target farmland; When the preset acquisition trigger event corresponds to a preset weak light condition, based on a preset RGB-NIR multispectral camera video acquisition device and a preset thermal infrared video acquisition device, several types of mouse monitoring videos in different spectral bands corresponding to the preset target farmland are collected; When the preset acquisition trigger event corresponds to a preset dark light condition, based on the preset thermal infrared video acquisition equipment and the preset short-wave infrared video acquisition equipment, several types of mouse monitoring videos of different spectral bands corresponding to the preset target farmland are collected.

9. The method for detecting the degree of disease in mice based on YOLO as claimed in claim 1, wherein: The determination of the rodent damage level corresponding to the preset target farmland based on the above-mentioned final determinations includes: Based on the above final determinations, the rat activity intensity index corresponding to the preset target farmland is calculated, and includes at least one of the following: determining whether the rat activity intensity index is greater than or equal to a preset first rat activity intensity index threshold, and if so, determining that the rodent damage level corresponding to the preset target farmland is in a high-risk state; Determine whether the rat activity intensity index is less than the preset second rat activity intensity index threshold, and if the above judgment is yes, determine that the rodent damage level corresponding to the preset target farmland is in a low-risk state, wherein the preset first rat activity intensity index threshold is greater than or equal to the preset second rat activity intensity index threshold.

10. A YOLO-based system for detecting the degree of mouse disease, characterized in that: include: The first determination module is used to determine the current monitoring image and the corresponding mouse monitoring audio corresponding to the preset target farmland, and to determine the unique morphological characteristics of the preset mouse; A first determination module is configured to, upon determining that the mouse monitoring audio includes mouse behavior sounds corresponding to mouse behavior, and based on the preset mouse-specific morphological features and a preset mouse target detection YOLO model, determine the mouse-specific morphological features of a current frame corresponding to the current monitoring image when it is preliminarily determined that the current monitoring image includes a mouse image; The second judgment module is used to finally judge that the current monitoring image contains a mouse image when it is judged that the unique morphological features of the mouse in the current frame are consistent with the mouse movement morphological trajectory, and determine the degree of rodent damage corresponding to the preset target farmland based on several of the above final judgments.

Images