A method, device and equipment for monitoring and analyzing biological behavior of farmland based on video stream

Abstract

Embodiments of the present application disclose a kind of based on video stream's agricultural land biological behavior monitoring and analysis method, device and equipment.The based on video stream's agricultural land biological behavior monitoring and analysis method includes: obtaining agricultural land monitoring video stream data;According to agricultural land monitoring video stream data, the category and behavior trajectory of each target organism in video stream are determined;Based on the category and behavior trajectory of each target organism, whether each target organism has harmfulness to agricultural production is determined;If target organism has harmfulness to agricultural production, then the corresponding alarm information is generated and sent to user confirmation and storage.Through the method, the following can be realized: using agricultural land monitoring video stream data as data source, the category and behavior trajectory of target organism in field of view range are identified and analyzed, to judge the harmfulness of target organism to agricultural production, if certain harm is generated, alarm information is generated to notify relevant user.To realize the intelligent supervision of agricultural land, reduce labor cost, improve the efficiency of supervision.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of agricultural and livestock production safety management technology, and in particular to a method, device and equipment for monitoring and analyzing the biological behavior of farmland based on video streams. Background Technology

[0002] Animals frequently roam farmland, which can damage agricultural production to some extent. For example, wild boars, bears, badgers, tigers, and other animals entering farmland can damage crops and affect agricultural production. Therefore, it is necessary to monitor the environmental safety of farmland.

[0003] The existing method mainly involves arranging workers to inspect farmland regularly, which requires high labor costs and has relatively low efficiency. Summary of the Invention

[0004] This invention provides a method, apparatus, and equipment for monitoring and analyzing the biological behavior of farmland based on video streams, so as to realize intelligent supervision of the farmland environment and improve the efficiency of supervision.

[0005] In a first aspect, embodiments of the present invention provide a method for monitoring and analyzing the biological behavior of farmland based on video streams, the method comprising:

[0006] Acquire farmland monitoring video stream data;

[0007] The categories and behavioral trajectories of each target organism in the video stream are determined based on the farmland monitoring video stream data;

[0008] Determine whether each target organism poses a threat to agricultural production based on its category and behavioral patterns.

[0009] If the target organism is harmful to agricultural production, a corresponding alarm message will be generated, sent to the user for confirmation, and stored.

[0010] Optionally, the farmland monitoring video stream data includes multiple video frames; determining the category and behavioral trajectory of each target organism in the video stream based on the farmland monitoring video stream data includes:

[0011] Target detection algorithms are used to determine the category and relationship of each target organism in consecutive video frames;

[0012] The behavioral trajectory of each target organism is determined based on the correlation between the target organisms in the preceding and following video frames.

[0013] Optionally, the step of using a target detection algorithm to determine the category and association of each target organism in the preceding and following video frames includes: using a target detection algorithm to determine the category, bounding box, and confidence level of each target organism in the preceding and following video frames;

[0014] The association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence of each target organism in the preceding and following video frames.

[0015] The association relationship between target organisms in consecutive video frames is obtained by solving the association weight of each target organism in consecutive video frames using the maximum weight matching algorithm.

[0016] Optionally, determining the association weights of each target organism in the preceding and following video frames based on the category, bounding box, and confidence level of each target organism in the preceding and following video frames includes:

[0017] The association weights of each target organism in the preceding and following video frames are determined by one or more combinations of the following: the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames; the difference in the bounding box area and aspect ratio; whether the organisms are of the same category; and whether the classification confidence reaches a preset confidence level.

[0018] Optionally, the preceding and following video frames are identified video frames with a preset video frame duration interval.

[0019] Optionally, the video stream-based method for monitoring and analyzing the behavior of organisms in farmland also includes: using a preset correction model to correct the association weights of the current video frame based on the calculation results of historical video frames, so as to correct the category of the target organism in the current video frame.

[0020] Optionally, the target detection algorithm is one of the R-CNN series algorithms, YOLO series algorithms, and SSD series algorithms.

[0021] Optionally, determining whether each target organism poses a threat to agricultural production based on its category and behavioral trajectory includes:

[0022] The categories and behavioral trajectories of each target organism are compared with the risk levels corresponding to the occurrence duration of preset organism categories and preset behavioral trajectories to determine whether each target organism is harmful to agricultural production.

[0023] Secondly, embodiments of the present invention also provide a video stream-based agricultural biological behavior monitoring and analysis device, which includes:

[0024] The video stream data acquisition module is used to acquire video stream data from farmland monitoring.

[0025] The category and behavior trajectory determination module is used to determine the category and behavior trajectory of each target organism in the video stream based on the farmland monitoring video stream data;

[0026] The hazard determination module is used to determine whether each target organism poses a hazard to agricultural production based on its category and behavioral trajectory.

[0027] The alarm information generation module is used to generate corresponding alarm information if the target organism is harmful to agricultural production.

[0028] The alarm information sending module is used to send alarm information to the user for confirmation;

[0029] The alarm information storage module is used to store alarm information.

[0030] Thirdly, embodiments of the present invention also provide a monitoring and analysis device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and further including: a camera for collecting farmland monitoring video stream data; wherein, when the processor executes the program, it implements the farmland biological behavior monitoring and analysis method based on video stream as described in the first aspect.

[0031] This invention provides a method, apparatus, and equipment for monitoring and analyzing the behavior of organisms in farmland based on video streams. The method includes: acquiring farmland monitoring video stream data; determining the category and behavioral trajectory of each target organism in the video stream based on the data; determining whether each target organism poses a threat to agricultural production based on its category and behavioral trajectory; and generating and storing corresponding alarm information if the target organism is harmful to agricultural production, using farmland monitoring video stream data as the data source to identify and analyze the category and behavioral trajectory of target organisms within the field of view to determine their potential harm to agricultural production. If harm is caused, an alarm is generated to notify relevant users. This achieves intelligent monitoring of farmland biological behavior, reduces human intervention, lowers labor costs, and improves monitoring efficiency. Attached Figure Description

[0032] Figure 1 This is a flowchart of a method for monitoring and analyzing the biological behavior of farmland based on video streams, as described in Embodiment 1 of the present invention.

[0033] Figure 2 This is a flowchart of a video stream-based method for monitoring and analyzing the biological behavior of farmland, as described in Embodiment 2 of the present invention.

[0034] Figure 3 This is a schematic diagram of the structure of a surrounding box for a target organism in Embodiment 2 of the present invention;

[0035] Figure 4 This is a schematic diagram of the structure of a target biological behavior trajectory in Embodiment 2 of the present invention;

[0036] Figure 5 This is a structural block diagram of a video stream-based monitoring and analysis device for agricultural biological behavior according to Embodiment 3 of the present invention;

[0037] Figure 6 This is a schematic diagram of the structure of a device in Embodiment 4 of the present invention. Detailed Implementation

[0038] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0039] Example 1

[0040] Figure 1 This is a flowchart of a video stream-based method for monitoring and analyzing farmland biological behavior, as provided in Embodiment 1 of the present invention. This embodiment is applicable to an agricultural safety processing platform, enabling intelligent and efficient monitoring of farmland biological behavior. The method can be executed by a video stream-based farmland biological behavior monitoring and analysis device, which can be implemented in software and / or hardware. This device can be configured in the server of the processing platform. (Refer to...) Figure 1 Specifically, it includes the following steps:

[0041] Step 110: Obtain farmland monitoring video stream data;

[0042] Specifically, agricultural monitoring video stream data can be obtained by receiving monitoring video streams from devices such as cameras installed at agricultural and livestock production sites. To protect user privacy, encrypted data transmission technology can be used when receiving video stream data. The video stream consists of multiple frames of still images that switch at a certain frequency; this switching frequency is called the frame rate. For example, if the video stream frame rate is 60 frames per second (60 FPS), then the video stream contains 60 frames per second, meaning the images switch 60 times per second.

[0043] Step 120: Determine the category and behavioral trajectory of each target organism in the video stream based on the farmland monitoring video stream data;

[0044] The target organisms refer to all living things appearing in the video stream data, specifically all living things appearing within the monitoring area of ​​the farmland monitoring equipment. The category of the target organism refers to its species, such as humans or animals, with animals further categorized into domesticated livestock and wild animals. The behavioral trajectory of the target organism refers to its activity status within the farmland monitoring area, such as its movement patterns, duration of movement, and dwell time.

[0045] The video stream data originates from monitoring equipment on farmland, such as cameras. This data monitors the environment of the monitored farmland area, reflecting the presence of organisms, their types, and behavioral patterns. Therefore, video stream data can be used to identify and analyze the types and behavioral patterns of various target organisms appearing in the video stream, facilitating subsequent assessments of whether agricultural production has been harmed.

[0046] Step 130: Determine whether each target organism poses a threat to agricultural production based on its category and behavioral trajectory.

[0047] Specifically, because certain humans or animals can harm agricultural production—for example, humans stealing crops, and wild boars, bears, badgers, etc., stealing or trampling crops—these actions can damage crops and affect agricultural production. Therefore, based on the type and behavior of the target organism, it can be determined whether the target organism has entered farmland and whether its behavior there poses a threat or potential threat to the farmland. For example, assuming the target organism in the video stream is a wild boar, and its behavior appears frequently in the video stream for tens of seconds, it can be determined that this wild boar poses a certain degree of harm to agricultural production.

[0048] Step 140: If the target organism is harmful to agricultural production, generate a corresponding alarm message, send it to the user for confirmation, and store it.

[0049] The alarm information can include details such as an animal being present in farmland for tens of seconds, an animal moving rapidly in farmland, or an animal appearing in a video stream for a period of time. For example, assuming the target animal is a tiger, and its behavior involves it appearing in the video stream for a period of time, then the tiger poses a certain threat to agricultural production, and a corresponding alarm message would be generated, such as "A tiger appeared in farmland for 10 seconds" or "It appeared in the video for 10 seconds." The alarm information can be sent to relevant users for confirmation via SMS, push notifications, etc. Furthermore, the information regarding the threat posed by the target animal to agricultural production and the corresponding alarm information will be stored for easy retrieval or problem identification later.

[0050] In the technical solution of this embodiment, the working principle of the video stream-based method for monitoring and analyzing the biological behavior of farmland is as follows: (Refer to...) Figure 1First, farmland monitoring video stream data is acquired. Then, based on this data, the category and behavioral trajectory of each target organism in the video stream are determined. Finally, based on the category and behavioral trajectory of each target organism, it is determined whether each target organism poses a threat to agricultural production. If a target organism is found to be harmful to agricultural production, a corresponding alarm message is generated, sent to the user for confirmation, and stored. Therefore, this method can monitor and analyze the biological behavior of farmland. Using farmland monitoring video stream data as the data source, it identifies and analyzes the category and behavioral trajectory of target organisms within the field of view to determine their potential harm to agricultural production. If harm is caused, an alarm message is generated to notify relevant users. This achieves intelligent supervision of farmland biological behavior, reducing manual intervention, lowering labor costs, and improving the efficiency of supervision.

[0051] The technical solution of this embodiment provides a video stream-based method for monitoring and analyzing the behavior of organisms in farmland. This method includes: acquiring farmland monitoring video stream data; determining the category and behavioral trajectory of each target organism in the video stream based on the data; determining whether each target organism poses a threat to agricultural production based on its category and behavioral trajectory; and generating and storing corresponding alarm information if the target organism poses a threat to agricultural production. This monitoring and analysis method achieves the following: using farmland monitoring video stream data as the data source, identifying and analyzing the category and behavioral trajectory of target organisms within the field of view to determine their potential harm to agricultural production; and generating alarm information to notify relevant users if harm is caused. This enables intelligent monitoring of farmland biological behavior, reduces manual intervention, lowers labor costs, and improves monitoring efficiency.

[0052] Based on the above technical solutions, optionally, the hazard to agricultural production of each target organism can be determined based on its category and behavioral trajectory, including:

[0053] The categories and behavioral trajectories of each target organism are compared with the risk levels corresponding to the occurrence duration of preset organism categories and preset behavioral trajectories to determine whether each target organism is harmful to agricultural production.

[0054] Because different organisms pose varying degrees of harm to farmland, the system pre-sets the categories of organisms, their corresponding pre-defined behavioral trajectories, and the associated levels of harm. For example, the pre-defined organism categories are bears, cattle, and badgers. The corresponding risk levels for the duration of their pre-defined behavioral trajectories are as follows: a bear appearing in the video stream for more than 3 seconds corresponds to a high risk level; a cattle appearing in the video stream moving rapidly for more than 10 seconds corresponds to a medium-high risk level; and a badger appearing in the video stream for a period of time corresponds to a medium-high risk level.

[0055] It should be noted that the specific preset biological categories, preset behavioral trajectories, and risk levels corresponding to the duration of the preset behavioral trajectories can be set according to the actual situation, and no specific limitations are made here.

[0056] For example, assuming the target creature in the video stream is a cow, and its behavior trajectory is a rapid movement in the video stream for 15 seconds, the risk level corresponding to the appearance duration of the target creature is compared with the preset creature category and preset behavior trajectory. It can be determined that the risk level of the target creature is medium to high risk and it is harmful to agricultural production. Then, the corresponding alarm information is generated and sent to the user for confirmation. The hazard information and alarm information are stored for subsequent traceability.

[0057] Example 2

[0058] Figure 2 This is a flowchart of a method for monitoring and analyzing the biological behavior of farmland based on video streams, provided in Embodiment 2 of the present invention. Figure 3 This is a schematic diagram of the structure of a surrounding box for a target organism provided in Embodiment 2 of the present invention. Figure 4 This is a schematic diagram of the structure of a target biological behavioral trajectory provided in Embodiment 2 of the present invention. Optionally, based on Embodiment 1 above, refer to... Figure 2 The method specifically includes the following steps:

[0059] Step 210: Obtain farmland monitoring video stream data;

[0060] Step 220: Use an object detection algorithm to determine the category and relationship of each target organism in the preceding and following video frames;

[0061] The agricultural land monitoring video stream data consists of multiple video frames, which are switched at a certain frequency, known as the frame rate. For example, if the video stream frame rate is 60 frames per second (60 FPS), then the video stream contains 60 frames per second, meaning the screen switches 60 times. Due to potentially high computational burden, the recognition frame rate (how many frames are recognized per second) is generally less than or equal to the video frame rate. For example, assuming the video stream frame rate is 60 FPS, the recognition frame rate might be 60 FPS, 30 FPS, 20 FPS, etc. When the recognition frame rate is less than the video frame rate, the server uses the time interval between several frames of the video stream to perform the recognition calculation for one video frame.

[0062] Optionally, the preceding and following video frames are identified video frames with a preset video frame duration interval.

[0063] For example, the time interval between the video frame currently being processed by the server and the video frame it previously processed is the time interval of the video stream's frames. Here, "before" and "after" video frames refer to: the currently processed video frame being the "after" video frame, and the previously processed video frame being the "before" video frame. Both "before" and "after" video frames are recognition video frames.

[0064] Specifically, target detection algorithms are used to determine the category and relationship of each target organism in each preceding and following video frame in all video frames identified according to the recognition frame rate, so as to identify the various organisms contained in each video frame, and to facilitate the subsequent determination of the behavioral trajectory of each target organism based on the relationship between each target organism in each preceding and following video frame.

[0065] Optionally, an object detection algorithm is used to determine the category and relationship of each target organism in consecutive video frames, including:

[0066] The object detection algorithm is used to determine the category, bounding box, and confidence of each target organism in the preceding and following video frames;

[0067] The association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence of each target organism in the preceding and following video frames.

[0068] The association relationship between target organisms in consecutive video frames is obtained by solving the association weight of each target organism in consecutive video frames using the maximum weight matching algorithm.

[0069] Object detection algorithms are a type of task in artificial intelligence image recognition and processing. They can recognize images with a resolution of 1080P and above in real time, identify the category and bounding box (i.e., the boundaries in four directions) of objects in the image, and calculate the confidence score of the object classification. Therefore, object detection algorithms can determine the category, bounding box, and confidence score of each target organism in consecutive video frames. The bounding box of the target organism can be used to characterize its positional state. An exemplary diagram of the bounding box of a target organism can be found here. Figure 3 In the diagram, the short dashed box represents the two bounding boxes of the first target creature, the solid box represents the two bounding boxes of the second target creature, and the long dashed box represents the two bounding boxes of the third target creature.

[0070] Among them, confidence level refers to the confidence level in the classification of the target organism. The higher the confidence level, the higher the reliability of the classification of the target organism, and vice versa.

[0071] The maximum weight matching algorithm mainly includes the Kuhn-Munkres algorithm and the Jonker-Volgenant algorithm. This algorithm primarily solves the problem of finding the maximum matching solution in a weighted bipartite graph. A weighted bipartite graph is a graph in which the vertices can be completely divided into two mutually exclusive sets, all edges have weights, and the two vertices connected by each edge are located in the two sets respectively. The maximum matching solution is to select a subset of the edge set in the weighted bipartite graph such that no edges share vertices, and the sum of the weights of the edges in this subset is maximized among all possible subsets of this type.

[0072] Specifically, object detection algorithms can obtain the recognition results for each video frame, namely the category, bounding box, and classification confidence of each target organism in each video frame. To accurately associate the same target organism in two adjacent video frames and prevent different organisms from being associated with the same trajectory in subsequent frames, it is necessary to calculate the probability of correlation between the recognition results of consecutive video frames, known as the association weight. Specifically, this can be calculated based on a combination of one or more elements from the recognition results of consecutive video frames, such as the target organism's category, bounding box, and confidence score.

[0073] Optionally, the association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence level of each target organism in the preceding and following video frames, including:

[0074] The association weights of each target organism in the preceding and following video frames are determined by one or more combinations of the following: the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames; the difference in the bounding box area and aspect ratio; whether the organisms are of the same category; and whether the classification confidence reaches a preset confidence level.

[0075] Specifically, parameterized formulas can be used to calculate the association weights of each target organism in consecutive video frames. For example, refer to... Figure 3 To calculate the association weight between the recognition results represented by the long dashed box and the recognition results represented by the solid box, it is necessary to calculate based on one or more combinations of the recognition results of the preceding and following video frames. These combinations include factors such as the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames, the difference in bounding box area and aspect ratio, whether the organism categories are the same, and whether the classification confidence reaches a preset confidence level. For example, assuming only the reciprocal of the distance between the center points of the bounding boxes is used as the element for calculating the association weight, the calculated association weights would resemble those in Table 1.

[0076] Table 1. Results of Association Weight Calculation

[0077] Solid line box | Long dashed line box Recognition Result 1 Recognition Result 2 Recognition Result 1 1 / 0.5cm = 2 1 / 1.0cm = 1 Recognition Result 2 1 / 3.0cm = 0.333 1 / 2.0cm = 0.5

[0078] As shown in the table, the corresponding association weight can be calculated for each pair of recognition results between consecutive video frames. Generally speaking, the association is weaker when the center points of the bounding boxes are far apart, and the association is weaker when the biological categories are different.

[0079] Based on the calculated association weights of each pair of recognition results in the preceding and following video frames, the maximum weight matching algorithm can find the matching scheme that maximizes the sum of association weights. This is the most likely connection between the recognized objects in the preceding and following video frames, thus determining the association relationship between the target organisms in the preceding and following video frames. For example, taking Table 1 as an example, there are two matching schemes: (long dashed box 1 matches solid box 1, long dashed box 2 matches solid box 2) and (long dashed box 1 matches solid box 2, long dashed box 2 matches solid box 1). The sum of association weights for the two matching schemes are 2 + 0.5 = 2.5 and 1 + 0.333 = 1.333, respectively. Therefore, the maximum weight matching algorithm will select the first matching scheme as the association relationship connecting the target organisms in the preceding and following video frames.

[0080] Step 230: Determine the behavioral trajectory of each target organism based on the correlation between each target organism in the preceding and following video frames.

[0081] In this context, a video stream is the sum of many still video frames in a time series, meaning it is discontinuous in time. To track the behavioral or activity trajectory of a target organism, we need to determine the bounding box (location), organism category, and classification confidence of each target organism in each preceding and following video frame. Then, based on these parameters, we determine the association weights between the identification results. Next, we determine the association relationships between the target organisms in each preceding and following video frame based on these association weights. Finally, we connect these relationships using line segments or interpolated curves to obtain the complete behavioral trajectory (shaped like a curve). For an example, refer to [reference needed]. Figure 4 The lines in the middle.

[0082] Step 240: Determine whether each target organism poses a threat to agricultural production based on its category and behavioral trajectory.

[0083] Step 250: If the target organism is harmful to agricultural production, generate a corresponding alarm message, send it to the user for confirmation, and store it.

[0084] Based on the above technical solution, optionally, the method for monitoring and analyzing the biological behavior of farmland based on video streams also includes: using a preset correction model to correct the association weight of the current video frame according to the calculation results of historical video frames, so as to correct the category of the target organism in the current video frame.

[0085] Occasionally, the identification results may misclassify the organism, such as misidentifying a cow as a sheep. The lower the confidence level of the identification result, the greater the possibility of classification error. Therefore, to avoid this situation, the association weights need to be corrected. Specifically, based on the calculation results of historical video frames, a preset correction model can be used, combined with the identification frame rate information, to correct the association weights of the current video frame, thereby correcting the category of the target organism in the current video frame. The purpose of this correction is to use the accumulated knowledge from the identification results of multiple video frames to eliminate situations such as "an organism continuously identified as a cow being identified as a sheep in some frames, causing a large error in the motion trajectory generated by the algorithm" and "a large-scale global error caused by a localized, occasional error." After correction, the association weights are adjusted, eliminating illogical matching schemes between consecutive video frames, thereby improving the calculation effect, reducing errors, and ensuring the accuracy of monitoring and analysis.

[0086] The historical video frames can be several or dozens of frames before the current video frame. The specific number of historical video frames selected can be set according to the actual situation, and no specific limit is made here.

[0087] The preset correction model can be a calculation model that utilizes object inertia, pixel comparison, etc.

[0088] Optionally, the object detection algorithm is one of the R-CNN series algorithms, YOLO series algorithms, and SSD series algorithms.

[0089] Furthermore, when new target categories are introduced, deep neural networks or transfer learning methods can be used to adjust model parameters to produce correct classification results. A deep neural network model consists of multiple neural computational units, each containing a certain number of parameters, and the connections between these units are adjusted during training. During computation (or prediction), the network structure and parameter set are used to derive results (e.g., bounding boxes, biological classification, confidence scores). Transfer learning methods involve adding a certain amount of newly labeled target category data to a pre-trained model parameter set, running a training program to obtain a new parameter set that differs from the original. Using the original network structure and the new parameter set, the newly introduced target category can be identified.

[0090] Example 3

[0091] Figure 5 This is a structural block diagram of a video stream-based agricultural biological behavior monitoring and analysis device provided in Embodiment 3 of the present invention. (Reference) Figure 5 The video stream-based farmland biological behavior monitoring and analysis device 100 includes:

[0092] Video stream data acquisition module 10 is used to acquire farmland monitoring video stream data;

[0093] The category and behavior trajectory determination module 20 is used to determine the category and behavior trajectory of each target organism in the video stream based on the farmland monitoring video stream data;

[0094] The hazard determination module 30 is used to determine whether each target organism is harmful to agricultural production based on its category and behavioral trajectory.

[0095] The alarm information generation module 40 is used to generate corresponding alarm information if the target organism is harmful to agricultural production.

[0096] The alarm information sending module 50 is used to send alarm information to the user for confirmation;

[0097] The alarm information storage module 60 is used to store alarm information.

[0098] This embodiment provides a video stream-based agricultural biological behavior monitoring and analysis device. This device includes: a video stream data acquisition module for acquiring agricultural monitoring video stream data; a category and behavior trajectory determination module for determining the category and behavior trajectory of each target organism in the video stream based on the agricultural monitoring video stream data; a hazard determination module for determining whether each target organism poses a hazard to agricultural production based on its category and behavior trajectory; an alarm information generation module for generating corresponding alarm information if the target organism poses a hazard to agricultural production; an alarm information sending module for sending the alarm information to the user for confirmation; and an alarm information storage module for storing the alarm information. This device can use agricultural monitoring video stream data as a data source to identify and analyze the category and behavior trajectory of target organisms within the field of view to determine the hazard posed by the target organisms to agricultural production. If a hazard is caused, an alarm information is generated to notify relevant users. This achieves intelligent monitoring of agricultural biological behavior, reduces manual intervention, lowers labor costs, and improves monitoring efficiency.

[0099] Optionally, the category and behavior trajectory determination module 20 includes:

[0100] The category and association determination unit is used to determine the category and association of each target organism in consecutive video frames using a target detection algorithm;

[0101] The behavior trajectory determination unit is used to determine the behavior trajectory of each target organism based on the correlation between each target organism in the preceding and following video frames.

[0102] Optionally, the category and association determination unit is used to: use an object detection algorithm to determine the category, bounding box, and confidence of each target organism in the preceding and following video frames;

[0103] The association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence of each target organism in the preceding and following video frames.

[0104] The association relationship between target organisms in consecutive video frames is obtained by solving the association weight of each target organism in consecutive video frames using the maximum weight matching algorithm.

[0105] Optionally, determining the association weights of each target organism in the preceding and following video frames based on the category, bounding box, and confidence level of each target organism in the preceding and following video frames includes:

[0106] The association weights of each target organism in the preceding and following video frames are determined by one or more combinations of the following: the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames; the difference in the bounding box area and aspect ratio; whether the organisms are of the same category; and whether the classification confidence reaches a preset confidence level.

[0107] Optionally, the preceding and following video frames are identified video frames with a preset video frame duration interval.

[0108] Optionally, the video stream-based farmland biological behavior monitoring and analysis device further includes: a correction module, used to correct the association weight of the current video frame based on the calculation results of historical video frames using a preset correction model, so as to correct the category of the target organism in the current video frame.

[0109] Optionally, the object detection algorithm is one of the R-CNN series algorithms, YOLO series algorithms, and SSD series algorithms.

[0110] Optionally, the hazard determination module 30 includes:

[0111] The hazard assessment unit is used to compare the category and behavioral trajectory of each target organism with the risk level corresponding to the occurrence duration of preset organism categories and preset behavioral trajectories to determine whether each target organism is harmful to agricultural production.

[0112] The video stream-based farmland biological behavior monitoring and analysis device provided in this embodiment of the invention can execute the video stream-based farmland biological behavior monitoring and analysis method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0113] Example 4

[0114] Figure 6 This is a schematic diagram of the structure of a device provided in Embodiment 4 of the present invention, as shown below. Figure 6 As shown, the device includes a processor 70, a memory 71, an input device 72, an output device 73, and a camera 74; the number of processors 70 in the device can be one or more. Figure 6 Taking a processor 70 as an example; the processor 70, memory 71, input device 72, and output device 73 in the device can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0115] The memory 71, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the video stream-based farmland biological behavior monitoring and analysis method in this embodiment of the invention (e.g., the video stream data acquisition module 10, the category and behavior trajectory determination module 20, the hazard determination module 30, the alarm information generation module 40, the alarm information sending module 50, and the alarm information storage module 60 in the video stream-based farmland biological behavior monitoring and analysis device). The processor 70 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 71, thereby realizing the above-mentioned video stream-based farmland biological behavior monitoring and analysis method.

[0116] The memory 71 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 71 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 71 may further include memory remotely located relative to the processor 70, which can be connected to the device / terminal / server via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] Input device 72 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 73 may include display devices such as a display screen. Camera 74 is used to collect farmland monitoring video stream data. The farmland biological behavior monitoring and analysis device 100 based on video stream can obtain farmland monitoring video stream data collected by monitoring camera 74 through encrypted transmission technology, thereby protecting user privacy.

[0118] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for monitoring and analyzing the biological behavior of farmland based on video streams, characterized in that, include: Acquire farmland monitoring video stream data; The categories and behavioral trajectories of each target organism in the video stream are determined based on the farmland monitoring video stream data; Determine whether each target organism poses a threat to agricultural production based on its category and behavioral patterns. If the target organism is harmful to agricultural production, a corresponding alarm message will be generated, sent to the user for confirmation, and stored. The farmland monitoring video stream data includes multiple video frames; determining the category and behavioral trajectory of each target organism in the video stream based on the farmland monitoring video stream data includes: Target detection algorithms are used to determine the category and relationship of each target organism in consecutive video frames; The behavioral trajectory of each target organism is determined based on the correlation between the target organisms in the preceding and following video frames; wherein, the behavioral trajectory of the target organism refers to the activity status of the target organism in the farmland monitoring area, and the activity status includes: movement status, movement duration, and dwell time; the category and behavioral trajectory of the target organism are used to determine whether the target organism has entered the farmland and whether its behavior in the farmland has caused or has the potential to cause harm to the farmland. The step of using a target detection algorithm to determine the category and relationship of each target organism in consecutive video frames includes: The object detection algorithm is used to determine the category, bounding box, and confidence of each target organism in the preceding and following video frames; The association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence of each target organism in the preceding and following video frames. The association relationship between target organisms in consecutive video frames is obtained by solving the association weight of each target organism in consecutive video frames using the maximum weight matching algorithm. The step of determining the association weights of each target organism in the preceding and following video frames based on the category, bounding box, and confidence level of each target organism in the preceding and following video frames includes: The association weights of each target organism in the preceding and following video frames are determined by one or more combinations of the following: the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames; the difference in the bounding box area and aspect ratio; whether the organisms are of the same category; and whether the classification confidence reaches a preset confidence level.

2. The method for monitoring and analyzing farmland biological behavior based on video streams according to claim 1, characterized in that, The preceding and following video frames are identified video frames with a preset video frame duration interval.

3. The method for monitoring and analyzing farmland biological behavior based on video streams according to claim 1, characterized in that, Also includes: Based on the calculation results of historical video frames, a preset correction model is used to correct the association weights of the current video frame in order to correct the category of the target organism in the current video frame.

4. The method for monitoring and analyzing farmland biological behavior based on video streams according to claim 1, characterized in that, The target detection algorithm is one of the R-CNN series algorithms, YOLO series algorithms, and SSD series algorithms.

5. The method for monitoring and analyzing farmland biological behavior based on video streams according to claim 1, characterized in that, The determination of whether each target organism poses a threat to agricultural production based on its category and behavioral patterns includes: The categories and behavioral trajectories of each target organism are compared with the risk levels corresponding to the occurrence duration of preset organism categories and preset behavioral trajectories to determine whether each target organism is harmful to agricultural production.

6. A device for monitoring and analyzing the biological behavior of farmland based on video streams, characterized in that, include: The video stream data acquisition module is used to acquire video stream data from farmland monitoring. The category and behavior trajectory determination module is used to determine the category and behavior trajectory of each target organism in the video stream based on the farmland monitoring video stream data; The hazard determination module is used to determine whether each target organism poses a hazard to agricultural production based on its category and behavioral trajectory. The alarm information generation module is used to generate corresponding alarm information if the target organism is harmful to agricultural production. The alarm information sending module is used to send alarm information to the user for confirmation; The alarm information storage module is used to store alarm information; The category and behavior trajectory determination module includes: The category and association determination unit is used to determine the category and association of each target organism in consecutive video frames using a target detection algorithm; The behavior trajectory determination unit is used to determine the behavior trajectory of each target organism based on the correlation between each target organism in the preceding and following video frames; wherein, the behavior trajectory of the target organism refers to the activity status of the target organism in the farmland monitoring area, and the activity status includes: movement status, movement duration, and dwell time; the category and behavior trajectory of the target organism are used to determine whether the target organism has entered the farmland and whether its behavior trajectory in the farmland has caused or has the potential to cause harm to the farmland; The category and association determination unit is used to: determine the category, bounding box, and confidence level of each target organism in the preceding and following video frames using a target detection algorithm; The association weights of each target organism in the preceding and following video frames are determined based on the category, bounding box, and confidence of each target organism in the preceding and following video frames. The association relationship between target organisms in consecutive video frames is obtained by solving the association weight of each target organism in consecutive video frames using the maximum weight matching algorithm. The step of determining the association weights of each target organism in the preceding and following video frames based on the category, bounding box, and confidence level of each target organism in the preceding and following video frames includes: The association weights of each target organism in the preceding and following video frames are determined by one or more combinations of the following: the distance between the center points of the bounding boxes of each target organism in the preceding and following video frames; the difference in the bounding box area and aspect ratio; whether the organisms are of the same category; and whether the classification confidence reaches a preset confidence level.

7. A monitoring and analysis device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, Also includes: A camera is used to collect video stream data for monitoring farmland; wherein, when the processor executes the program, it implements the video stream-based farmland biological behavior monitoring and analysis method as described in any one of claims 1-5.

Images