Distributed wild animal monitoring system

Abstract

The invention provides a distributed wild animal monitoring system, which comprises a plurality of unmanned aerial vehicles and a monitoring scheduling device arranged on each unmanned aerial vehicle, and is characterized in that the monitoring scheduling device comprises a data acquisition unit used for acquiring states and monitoring data of the unmanned aerial vehicles; the recognition unit is used for recognizing the environment state and the actual state and behavior mode of the target wild animal; the tracking control unit comprises a target tracking module and a motion prediction module, and the motion prediction module outputs the prediction state of the target wild animal when the behavior mode of the target wild animal is the high-speed motion mode; the target tracking module determines a tracking action for monitoring the target wild animal based on the state of the unmanned aerial vehicle, the monitoring environment data and the actual state or the predicted state of the target wild animal, and the data synchronization unit is used for performing data synchronization with monitoring scheduling devices installed on other unmanned aerial vehicles. The monitoring system provided by the invention can continuously and effectively monitor the target wild animals.

Description

Technical Field

[0001] The present application belongs to the field of unmanned aerial vehicle intelligent control technology, and specifically relates to a distributed wildlife monitoring system. Background Art

[0002] With the advancement of drone technology, the application areas of using drones for environmental monitoring are becoming more and more extensive. Since drones can provide real-time aerial perspectives and conduct long-term continuous monitoring, they effectively solve the limitations of manual monitoring and have good application prospects.

[0003] Existing technical solutions for using drones for environmental monitoring generally use drones to cruise along a preset path, or use manual control to adjust the path in real time. Drones collect environmental information through sensors (such as cameras, thermal imagers, LiDAR) and upload it to ground stations for processing and analysis.

[0004] Although the above scheme can effectively realize the environmental monitoring task, it cannot meet the requirements of continuous monitoring of specific wild animals. The reason is that the behavior patterns and activity ranges of specific wild animals are in a constant state of change. In the case of dynamic changes in complex wild environments, especially when wild animals escape, the response of drones is often delayed, and even cannot meet the needs of focusing on monitoring specific target animals in dynamic and complex wild environments. In addition, how to continuously monitor target wild animals and avoid target loss due to the constraints of drone endurance is an urgent problem to be solved in wildlife drone monitoring. Summary of the invention

[0005] The purpose of the present application is to provide a distributed wildlife monitoring system, which uses a distributed architecture to dispatch multiple drones to continuously monitor target wildlife.

[0006] The distributed wildlife monitoring system includes a plurality of drones, each of which is provided with a monitoring and dispatching device, and the monitoring and dispatching device includes:

[0007] Data acquisition unit, used to obtain the status and monitoring data of the drone;

[0008] an identification unit, used to identify the environmental state, the actual state and behavior pattern of the target wild animal from the monitoring data, wherein the behavior pattern of the target wild animal at least includes a habitat pattern, a migration pattern and a high-speed movement pattern;

[0009] A tracking control unit, comprising a target tracking module and a motion prediction module, wherein the motion prediction module outputs its predicted state based on the actual state of the target wild animal when the behavior mode of the target wild animal is a high-speed motion mode; the target tracking module determines the tracking action for monitoring the target wild animal based on the state of the drone, monitoring environment data, and the predicted state of the target wild animal when the behavior mode of the target wild animal is a high-speed motion mode, and determines the tracking action for monitoring the target wild animal based on the state of the drone, monitoring environment data, and the actual state of the target wild animal when the behavior mode of the target wild animal is a mode other than the high-speed motion mode;

[0010] The data synchronization unit is used to synchronize data with monitoring and scheduling devices installed on other drones.

[0011] An embodiment of the present application provides a distributed wildlife monitoring system, which continuously monitors target wildlife through the autonomous scheduling of multiple drones, wherein an identification unit further determines the behavior pattern after obtaining the actual motion state of the target wildlife, and a tracking control unit determines the tracking action of the drone to track and monitor the target wildlife based on the actual state or predicted state of the target wildlife based on the different behavior patterns of the target wildlife, and utilizes the different characteristics of the motion state and activity range of the target wildlife under different behavior patterns to determine different tracking strategies. When the target wildlife is moving at high speed, the drone can be driven along a relatively simple path according to the predicted results of its motion state to achieve effective tracking of the target wildlife, which can greatly reduce the energy consumption of the drone in monitoring the high-speed moving target wildlife and significantly extend the drone's flight time.

[0012] Furthermore, the status of the drone includes the drone's location information, speed information, remaining energy information and monitoring coverage information; the monitoring data includes visible light video and / or infrared video, three-dimensional ranging data and meteorological monitoring data of the monitoring area; the environmental status includes terrain information, climate condition information and obstacle location information of the monitoring area.

[0013] Furthermore, the identification unit identifies the actual state and behavior pattern of the target wild animal based on the following steps:

[0014] Step 510, extracting visible light video and / or infrared video and three-dimensional ranging data of the monitoring area;

[0015] Step 520, using a target detection algorithm to identify the target wildlife from the visible light video and / or infrared video information and determine the actual state of the target wildlife, wherein the actual state includes the actual position, actual speed and actual activity range of the target wildlife;

[0016] Step 530: determining the behavior pattern of the target wildlife based on the actual state of the target wildlife.

[0017] Furthermore, for any drone, when its identification unit identifies a target wild animal, its target tracking module determines a tracking action for monitoring the target wild animal through the following steps:

[0018] Step 610, obtaining the state of the drone, the state of the environment, the actual state and behavior pattern of the target wild animal;

[0019] Step 620, determining whether the behavior mode of the target wild animal is a high-speed movement mode;

[0020] Step 630: If the judgment result is no, then construct the state space S(t) for monitoring the target wild animal based on equation (1).

[0021]

[0022] If the judgment result is yes, the predicted state of the target wildlife is further obtained, and the state space S(t) of the target wildlife monitoring is constructed based on formula (2):

[0023]

[0024] Among them, S UAV (t) is the state of the drone, Env(t) is the state of the environment, The actual status of the target wildlife, The predicted state of the target wild animal output by the movement prediction module;

[0025] Step 640, using a tracking strategy network based on reinforcement learning to determine a tracking action for monitoring the target wildlife, and updating the tracking strategy network based on the result of the UAV performing the tracking action;

[0026] Step 650, determining whether the remaining energy of the drone is lower than the return energy lower limit threshold, if not, returning to step 610, if yes, executing step 660;

[0027] Step 660, dispatching other drones to take over the monitoring of the target wildlife through the data synchronization unit.

[0028] Preferably, the state space of the tracking strategy network is S(t); the action space of the tracking strategy network is the change in height, speed and direction of the UAV; the instant reward function of the tracking strategy network includes the following reward items: distance reward item, energy consumption reward item, monitoring coverage reward item and tracking stability reward item.

[0029] Preferably, when the target wild animal is in a mode other than the high-speed motion mode, the lower limit threshold of the return energy is not less than the minimum energy required for the return of the UAV; when the target wild animal is in the high-speed motion mode, the lower limit threshold of the return energy is not less than 2+k times the minimum energy required for the return of the UAV, wherein k is a high-speed motion compensation factor, and the positive or negative value of the k value is determined by the deviation between the high-speed motion direction of the target wild animal and the return direction of the UAV.

[0030] Preferably, step 660 further includes the following steps:

[0031] Step 661, sending the actual status, behavior pattern and movement prediction network of the target wild animal to other drones through the data synchronization unit;

[0032] Step 662, after determining that at least one other drone has taken over the monitoring of the target wildlife, control the drone to return.

[0033] Preferably, when the target wildlife is in a mode other than the high-speed movement mode, the drone starts to return after determining that at least one other drone has taken over the monitoring of the target wildlife and taken off; when the target wildlife is in a high-speed movement mode, the drone starts to return after the identification unit of at least one other drone has identified the target wildlife.

[0034] Preferably, the motion prediction module uses a motion prediction network based on reinforcement learning to predict and output the predicted state of the target wild animal. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of an application scenario of a distributed wildlife monitoring system provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of the architecture of a monitoring and scheduling device provided according to an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the workflow of a monitoring and scheduling device provided according to an embodiment of the present application;

[0038] Figure 4 A schematic diagram of the workflow of the identification unit provided according to an embodiment of the present application;

[0039] Figure 5 A schematic diagram of the working process of a target tracking module provided according to an embodiment of the present application;

[0040] Figure 6 A schematic diagram of the architecture of a target tracking network based on reinforcement learning provided according to an embodiment of the present application;

[0041] Figure 7 A schematic diagram of a process for dispatching other drones to take over monitoring of target wildlife according to an embodiment of the present application;

[0042] Figure 8 A schematic diagram of a drone taking over scheduling when a target wild animal is in a habitat mode according to an embodiment of the present application;

[0043] Fig. 9 This is a schematic diagram of drone takeover scheduling when the target wild animal is in high-speed movement mode according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] Hereinafter, the present application will be further described based on preferred embodiments with reference to the accompanying drawings.

[0045] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the products of the embodiments of the present application are usually placed when in use, it is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, in the description of the present application, in order to distinguish different units, the words first, second, etc. are used in this specification, but these are not limited by the order of manufacture, nor can they be understood as indicating or implying relative importance, and their names may be different in the detailed description and claims of the present application.

[0046] The vocabulary in this specification is used to illustrate the embodiments of the present application, but is not intended to limit the present application. It should also be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, an indirect connection through an intermediate medium, or a connection between the two elements. For those skilled in the art, the specific meanings of the above terms in this application can be specifically understood.

[0047] The present application provides a distributed wildlife monitoring system through an embodiment, which is composed of multiple drones and continuously monitors target wildlife in a distributed monitoring manner.

[0048] Figure 1 The embodiment shown shows a specific application scenario of the distributed wildlife monitoring system, such as Figure 1As shown, the distributed wildlife monitoring system includes four drones, among which the drone 210 is in the monitoring area 310 and changes the flight altitude, flight direction and flight speed through its flight control system to track and monitor the target wildlife 100.

[0049] Although conventional drones that cruise along fixed routes also have the ability to identify and track wild animals, they generally do not track a specific target continuously (for example, for several days or even dozens of days). However, such continuous tracking and monitoring is necessary in some cases. For example, when it is necessary to understand the status of wild animals in special periods (such as recovery from illness or lactation), it is necessary to keep track of their habitats, foraging, migration, and escape in dangerous situations for a considerable period of time.

[0050] In the process of monitoring the target wild animals, a single UAV is limited by its endurance and is generally unable to continuously track and monitor the target wild animals 100. Therefore, in addition to the UAV 210 in the monitoring state, such as Figure 1 As shown, the monitoring system also includes three drones (i.e., drones 220, 230, and 240) located in a base 320. The three drones are in a standby state after being charged, refueled, and undergoing equipment maintenance in the base 320. The four drones can communicate and synchronize data through a wireless network 330 or other remote communication means. When drone 210 needs to return to the base because its remaining energy is reduced to the lower limit threshold of the endurance energy described later, one of the three drones originally located in the base 320 will fly to the monitoring area 310 and take over the monitoring of the target wild animal 100. During the continuous monitoring of the target wild animal 100, the four drones will repeatedly switch between the monitoring state and the standby state, thereby continuously monitoring the target wild animal 100.

[0051] Obviously, the number of drones can be determined based on the performance of the drones (such as maximum flight distance, endurance), the characteristics of the target wild animals that need to be monitored (such as activity range, movement ability), etc. For example, in some embodiments, when the drone has a longer flight time, two drones can be used to form the monitoring system to achieve continuous monitoring of the target wild animals; for example, in other embodiments, the target wild animals have a larger activity range, in this case, the number of drones can be increased to avoid losing track of the target wild animals.

[0052] Figure 2A framework schematic diagram of the monitoring and scheduling device set on each drone is shown. In an embodiment of the present application, the monitoring and scheduling device is used to control the drone in which it is located to identify the target wild animals in the monitoring area and plan a reasonable flight path according to the various activities of the target wild animals (it should be noted that after determining the flight path, the drone can adjust the flight altitude, speed and direction through the existing flight control system). At the same time, different drones switch monitoring tasks through communication with other drones, thereby realizing distributed scheduling among multiple drones.

[0053] like Figure 2 As shown, the monitoring and scheduling device further includes a data acquisition unit, an identification unit, a tracking control unit and a data synchronization unit, specifically:

[0054] The data acquisition unit is used to obtain the status and monitoring data of the drone.

[0055] The identification unit is used to identify the environmental status, the actual status and behavior pattern of the target wild animal 100 from the monitoring data.

[0056] The tracking control unit further includes a target tracking module and a motion prediction module, wherein the motion prediction module outputs its predicted state based on the actual state of the target wild animal 100 when the behavior mode of the target wild animal 100 is a high-speed motion mode; the target tracking module determines the tracking action for monitoring the target wild animal based on the state of the drone, the monitoring environment data, and the predicted state of the target wild animal 100 when the behavior mode of the target wild animal 100 is a high-speed motion mode; when the behavior mode of the target wild animal 100 is a mode other than the high-speed motion mode, the tracking action for monitoring the target wild animal 100 is determined based on the state of the drone, the monitoring environment data, and the actual state of the target wild animal 100.

[0057] The data synchronization unit is used to synchronize data with monitoring and scheduling devices installed on other drones.

[0058] Figure 3 In one specific embodiment, a drone (e.g. Figure 1 The following is a flowchart of the drone 210 in the figure tracking and monitoring the target wild animal 100 through the monitoring and scheduling device set thereon. Figure 2 , Figure 3 , the working process of the monitoring and scheduling device is explained in detail.

[0059] 1. Data collection:

[0060] like Figure 2 , Figure 3As shown, after the UAV takes over the monitoring of the target wild animal 100, it first executes step 400 through the data acquisition unit to obtain the state and monitoring data of the UAV.

[0061] In the embodiments of the present application, the data acquisition unit may include various sensors and data acquisition devices known to those skilled in the art, such as a GPS receiver for positioning, an inertial measurement unit (IMU) for measuring acceleration and angular velocity, a high-definition camera for acquiring images and videos, an infrared camera for acquiring heat distribution, a laser radar (LiDAR) for acquiring three-dimensional terrain information, and a sensor for measuring weather conditions such as temperature, humidity, air pressure, wind speed and direction, and precipitation, etc. The above sensors and data acquisition devices are installed at corresponding positions of the drone to obtain the status S of the drone in real time. UAV (t) and monitoring data, where the state of the drone S UAV (t) is used to characterize the state information of the drone at each time t, and the monitoring data includes the environmental and biological data within the monitoring area collected by the drone.

[0062] In some optional embodiments, the longitude, latitude and altitude information of the drone can be obtained by using a GPS device installed on the drone, so as to obtain the location information P of the drone. UAV (t); The acceleration and angular velocity of the drone are obtained through the inertial measurement unit including the accelerometer and gyroscope, and then the speed information V of the drone is obtained. UAV (t); Obtain the remaining energy information of the drone through the battery management system or the sensor that monitors the fuel level. UAV (t). The above-mentioned devices and methods for obtaining the status of the drone are well known to those skilled in the art and will not be described in detail here.

[0063] In some optional embodiments, the status of the drone also includes monitoring coverage information C UAV (t), the monitoring coverage characterizes the detection capability of the UAV, and its expression is shown in the following formula:

[0064]

[0065] Among them, z UAV (t),θ UAV They are respectively the flight altitude of the UAV and the field of view of the image sensor and / or infrared sensor used for monitoring.

[0066] Optionally, the various status information of the drone acquired by the data acquisition unit can be constructed in vector form, such as: S UAV (t) = {P UAV (t),V UAV(t),e UAV (t),C UAV (t)}, which is further input into the tracking control unit as part of the state space of the tracking strategy network based on the reinforcement learning algorithm described later.

[0067] In some optional embodiments, the monitoring data includes visible light video of the monitoring area 310 recorded by a high-definition camera, and / or infrared video recorded by an infrared camera, as well as three-dimensional ranging data collected by LiDAR and meteorological monitoring data collected by various meteorological sensors.

[0068] 2. Environmental status and target wildlife identification:

[0069] like Figure 2 , Figure 3 As shown, after collecting various monitoring data in step 400, the identification unit processes the data by executing step 500 to identify the environmental state Env(t) and the actual state of the target wild animal 100. and behavioral patterns.

[0070] 2.1 Identification of environmental status:

[0071] In some specific embodiments, the environmental state Env(t) may include the obstacle position O(t), terrain information T(t), and climate condition information W(t) in the monitoring area 310 .

[0072] In an embodiment of the present application, a preset three-dimensional terrain imaging algorithm can be used to construct the three-dimensional terrain information T(t) of the monitoring area 310 and identify the obstacle position O(t) contained therein, using visible light video, three-dimensional ranging data, etc., and obtain the climate condition information W(t) of the monitoring area 310 from meteorological monitoring data. The above information constitutes the environmental state Env(t), which also constitutes a part of the state space of the tracking strategy network based on the reinforcement learning algorithm described later.

[0073] 2.2 Identification of target wild animals:

[0074] By using the visible light video captured by the high-definition camera of the drone and / or the infrared video recorded by the infrared camera, the target wild animal 100 can be identified, and its actual status can be obtained. And further determine the behavior pattern of the target wild animal 100.

[0075] In some embodiments, the actual state of the target wildlife 100 Includes actual location of 100 targeted wildlife Actual speed and the actual scope of activities The actual activity range indicates the maximum distance that the target wild animal can reach from its current position within a fixed period of time (such as one minute).

[0076] In some embodiments, the behavior mode of the target wild animal 100 includes at least a habitat mode, a migration mode and a high-speed movement mode. When the wild animal is in different behavior modes, its movement speed, activity range, etc. are significantly different. For example, when the target wild animal 100 is in the habitat mode, it generally remains stationary for a period of time, at which time its speed can be reduced to 0, and its activity range is limited to a very small area around its location; when the target wild animal 100 is in the migration mode, it is continuously in a relatively uniform low-speed movement state for a long period of time, and the movement direction remains roughly the same; when the target wild animal 100 is hunting other animals or avoiding its natural enemies, it maintains high-speed movement for a period of time (for example, large wild animals such as tigers may reach or exceed 50 kilometers per hour when hunting), and the movement direction also changes continuously as the prey escapes or avoids natural enemies, and is affected by the terrain at the same time. Under the joint action of multiple factors, its position, speed, and direction are extremely irregular. In the embodiments of the present application, this state is referred to as a high-speed movement state. Under different behavior patterns, the difficulty of the drone tracking and monitoring the target wild animal 100 varies greatly. Therefore, it is necessary to first accurately identify the target wild animal 100 and accurately determine its actual state and behavior pattern.

[0077] Figure 4 A flow chart showing, in some preferred embodiments, a recognition unit identifies a target wild animal 100 and its behavior pattern from monitoring data, specifically comprising the following steps:

[0078] Step 510: extracting visible light video and / or infrared video and three-dimensional ranging data of the monitoring area.

[0079] Specifically, when the monitoring period is daytime, visible light video alone can be extracted, or visible light video and infrared video can be extracted simultaneously to identify the target wild animal 100. When the monitoring period is nighttime, infrared video alone can be extracted to identify the target wild animal 100. The three-dimensional ranging data is used to determine the location of the identified target wild animal 100.

[0080] Step 520, using a target detection algorithm to identify the target wildlife from the visible light video and / or infrared video information and determine the actual state of the target wildlife, wherein the actual state includes the actual position, actual speed and actual activity range of the target wildlife.

[0081] Specifically, a dynamic target detection or recognition algorithm known to those skilled in the art, such as the YOLO algorithm, can be used to detect the target wild animal 100 in motion in real time from the video data. At the same time, combined with the terrain information obtained from the three-dimensional ranging data, the actual location of the target wild animal 100 can be determined. Actual speed And count its position changes over a period of time to get the actual activity range Similarly, the above information can be constructed as the actual state of the target wild animal 100 As part of the state space of the tracking strategy network based on the reinforcement learning algorithm described later.

[0082] Step 530: determining the behavior pattern of the target wildlife based on the actual state of the target wildlife.

[0083] In some embodiments, the actual state of the target wild animal 100 obtained in step 520 can be used as input data, and the classification model 810 can be used to identify its behavior pattern. The classification model 810 can use traditional feature extraction and classification algorithms, such as K-means clustering, hierarchical clustering, etc., or it can use a pre-trained deep learning network, such as a convolutional neural network (CNN), etc. The above-mentioned pattern recognition methods are all known to those skilled in the art and will not be repeated here.

[0084] 3. Tracking of target wild animals:

[0085] In the embodiment of the present application, the tracking of the target wild animal 100 is implemented by the tracking control unit through step 600, such as Figure 2 and Figure 3 As shown, the tracking control unit includes two core modules, a target tracking module and a motion prediction module. The target tracking module is used to implement the main tracking process 600, and its output result is the action of the drone tracking and monitoring the target wild animal. The motion prediction module executes the motion prediction process 700 when the target wild animal 100 is in a high-speed motion mode, and its output result is the predicted state of the target wild animal 100. Reference target wildlife 100 actual status Its predicted status includes the predicted location of the target wildlife 100 Prediction speed and predicted activity range ), the above prediction state It can also be constructed in vector form to input into the target tracking module, which selects the actual state according to the behavior mode of the target wild animal 100. Or predicted status And combined with the state S of the droneUAV (t) and the environmental state Env(t) to construct the corresponding state space to achieve accurate and efficient tracking of target wild animals in different behavior modes.

[0086] Predicting the state of the target wild animal 100 in the high-speed motion mode separately and tracking the target wild animal 100 in high-speed motion according to the prediction results can significantly improve the stability and efficiency of the drone tracking and monitoring. The reason is that the high-speed motion of the target wild animal 100 due to hunting or avoiding natural enemies involves a large number of random changes in direction and turns, that is, it may perform multiple high-speed motions in a smaller range than the migration mode. If the drone is driven to track its actual motion direction and position in real time, it is bound to perform extremely energy-intensive acceleration, braking, changing direction and other maneuvers, which will obviously greatly shorten the monitoring time of a single drone. If the position, speed and activity range of the target wild animal 100 in the high-speed motion mode can be predicted more accurately, it is only necessary to drive the drone along a relatively simple path to achieve effective tracking of it, without having to aim to ensure the distance between the two in real time. This can greatly reduce the energy consumption of the drone in monitoring the high-speed moving target wild animal 100 and significantly extend the drone's flight time.

[0087] 3.1 Tracking the main process:

[0088] Figure 5 In some preferred embodiments, the target tracking module implements specific steps of the tracking main process 600, such as Figure 5 As shown, for any UAV, when its recognition unit identifies a target wild animal, its target tracking module determines the tracking action for monitoring the target wild animal through the following steps:

[0089] 610, obtaining the state of the drone, the state of the environment, the actual state and behavior pattern of the target wildlife.

[0090] Specifically, as mentioned above, S UAV (t), Env(t), The behavior pattern of the target wild animal 100 can be obtained through the data acquisition unit and the recognition unit respectively, which will not be repeated here.

[0091] 620, determining whether the behavior pattern of the target wild animal is a high-speed movement pattern;

[0092] 630, if the judgment result is no, then the state space S(t) of the target wildlife monitoring is constructed based on equation (1),

[0093]

[0094] If the judgment result is yes, the predicted state of the target wildlife is further obtained, and the state space S(t) of the target wildlife monitoring is constructed based on formula (2):

[0095]

[0096] 640, using a tracking strategy network based on reinforcement learning to determine a tracking action for monitoring the target wildlife, and updating the tracking strategy network based on the result of the UAV performing the tracking action.

[0097] like Figure 3 As shown, in the embodiment of the present application, the target tracking module uses a tracking strategy network 820 based on reinforcement learning to generate the tracking action of the drone. As known to those skilled in the art, reinforcement learning (RL) is a machine learning model in which an agent learns strategies to maximize rewards or achieve specific goals during the interaction with the environment. In the process of interacting with the environment, the agent learns the best strategy through trial and error and feedback to achieve a certain goal. The basic components include the agent, the environment, the state, the action, the reward / penalty and the strategy, such as Figure 6 As shown in the figure, a typical reinforcement learning-based policy network usually constructs a state space S by acquiring the state of the environment and the agent, and outputs its action A in a given state through the policy network. The action A is executed by the agent and interacts with the environment, and rewards or penalties are generated based on the feedback signals obtained from the environment (the rewards or penalties are generally determined based on the tasks or goals that the agent needs to perform). The above rewards or penalties are used to train and update the policy network to optimize future decision-making processes.

[0098] Specifically, in the embodiment of the present application, the construction of the state space S(t) changes according to the different behavior modes of the target wild animal 100. When the target wild animal 100 is in a habitat mode or migration mode with a higher certainty, its actual state is adopted. As a component of the state space, the state space S(t) shown in formula (1) is constructed to accurately track the target wild animal 100; when the target wild animal 100 is in a high-speed motion mode with low certainty, its predicted state is generated according to the motion prediction network described later, and the state space S(t) shown in formula (2) is constructed. Since it includes the predicted state of the target wild animal 100, especially the predicted activity range prediction By predicting its activity range, the drone does not need to frequently accelerate, decelerate or change direction to track the actual position and speed of the target wild animal 100. It only needs to maneuver within its predicted activity range to achieve effective tracking, which can effectively reduce the energy consumption of the drone and significantly enhance the effective monitoring time of a single drone.

[0099] After the state space is constructed, the tracking action of the drone can be generated through the tracking strategy network 820. The tracking strategy network 820 can be implemented by various reinforcement learning algorithms known to those skilled in the art. For example, the Q-learing algorithm can be used to enable the drone to evaluate the expected benefits of taking each action in each state by updating the Q value (i.e., state-action value) at each moment. Specifically, the Q value update formula is shown as follows:

[0100] Q(s,a)←Q(s,a)+α(R(t)+γmaxQ(s',a')-Q(s,a)) ，

[0101] Among them, s is the state space S(t), a is the action selected by the drone, including the changes in its height, speed and direction. The above variables constitute the action space of the drone, α is the learning rate, R(t) is the immediate reward, γ is the discount factor, Q(s,a) represents the current Q value of action a selected in state s, maxQ(s',a') is the maximum value of the Q values ​​of all possible actions a' in the new state s', which represents the best expected return that the agent can obtain in the new state.

[0102] In some specific embodiments, R(t) may include the tracking distance d UAV-animal (t), energy consumption ΔE UAV (t) / Δt (i.e., the rate of change of residual energy), monitoring coverage C UAV (t) and tracking stability (such as ΔV UAV According to the different effects of each reward item on tracking and monitoring the target wild animal 100, different weights and symbols can be applied to them to obtain a specific expression of R(t). For example, when tracking the target wild animal 100 in high-speed motion mode, more emphasis should be placed on ensuring the stability of the drone to avoid increased energy consumption and flight risks caused by frequent changes in direction. The specific implementation methods of constructing instant rewards are already known to those skilled in the art and will not be repeated here.

[0103] After completing the output of the drone tracking action, the target tracking module continues to execute step 650, as shown in the figure, to determine whether the remaining energy of the drone is lower than the lower limit threshold of the return energy. If not, it returns to step 610. If so, it executes step 660 to schedule other drones to take over the monitoring of the target wildlife through the data synchronization unit.

[0104] 4. Distributed scheduling of continuous monitoring by multiple drones:

[0105] As described above, steps 650 and 660 are used to implement the monitoring of the drone (e.g. Figure 1 The endurance of the drone 210 in the monitoring system is monitored, and other drones (such as one of the drones 220, 230, and 240) are dispatched to take over the monitoring of the target wildlife 100 at appropriate times to ensure continuous monitoring of the target wildlife 100.

[0106] For the distributed monitoring system architecture provided in this application, there is no central control unit for unified management of each drone. Therefore, it is necessary to achieve reasonable scheduling between drones through communication between drones in monitoring state and drones in standby state, so as to increase the monitoring time of a single drone as much as possible while avoiding the problem of losing target wild animals in the process of replacing monitoring drones.

[0107] To this end, in some preferred embodiments of the present application, Figure 7 As shown, during the execution of step 660, the target tracking module first executes step 661 to send the actual state, predicted state and motion prediction network of the target wildlife to other drones through the data synchronization unit; then executes step 662 to control the drone to return after determining that at least one other drone has taken over the monitoring of the target wildlife.

[0108] Obviously, since the drone in monitoring state will lose monitoring of the target wildlife when it returns to the destination when it exceeds its monitoring coverage, and the drone taking over cannot effectively monitor the target wildlife when it does not enter its monitoring coverage, the timing of triggering the scheduling between different drones will significantly affect the tracking and monitoring effect and the efficiency of the drone. To this end, in the embodiments of the present application, different return energy lower limit thresholds are set according to the different behavior modes of the target wildlife 100, so as to achieve a balance between the tracking and monitoring effect and the efficiency of the drone. Figure 8 , Fig. 9 The setting method of the lower limit threshold of the return energy is introduced in detail.

[0109] like Figure 8As shown, when the target wildlife 100 is in the habitat mode or the migration mode, its activity range is small, and it only maintains low-speed movement in the same direction, or even does not move for a long period of time. At this time, the lower limit threshold of the return energy of the drone 210 that triggers the scheduling operation can be appropriately lowered to keep it at a level that can ensure the return of the drone 210, thereby extending its monitoring time. When it is determined through step 650 that its energy is only enough for return, the scheduling operation is triggered to send the actual location and behavior pattern information of the target wildlife 100 to the drone in the standby state, and when any other drone (such as Figure 8 The drone 240 in the figure takes over the monitoring of the target wild animal 100 and starts to return after taking off. During this process, since the target wild animal 100 is at a low speed or stationary state, although Figure 8 As shown in the figure, during the takeover process, the monitoring coverage of the two drones may be different within a period of time ( Figure 8 The target wild animal 100 is not covered by the drone 240 (shown by the middle dotted line), that is, there may be a "monitoring blank area" for a period of time, but the drone 240 can still successfully track and monitor the target wild animal 100.

[0110] like Fig. 9 As shown, when the target wildlife is in high-speed motion mode, its position, speed, direction, and range of activity are constantly changing. At this time, it is necessary to avoid the appearance of any monitoring blank area. Therefore, it is necessary to set the lower limit threshold of the return energy of the drone 210 to be no less than 2+k times of the minimum energy required for its return. Among them, setting 2 times is used to reserve the time for the drone 240 that takes over to fly to the target wildlife 100, to ensure that the drone 210 can continue to track until the drone 240 that takes over the monitoring arrives near the target wildlife 100 and then returns. k is a high-speed motion compensation factor, and the positive or negative value of the k value is determined by the deviation between the high-speed motion direction of the target wildlife 100 and the return direction of the drone 210. If the high-speed motion direction of the target wildlife 100 is opposite to the return direction of the drone 210, k takes a positive value to further increase the lower limit threshold. Otherwise, it can take a negative value to appropriately reduce the lower limit threshold and extend the monitoring time.

[0111] Accordingly, when the target wild animal 100 is in a high-speed movement mode, such as Fig. 9 As shown, the drone 210 starts to return only after the identification unit of at least one other drone (such as the drone 240) has identified the target wild animal 100, that is, after the monitoring coverage of the drone 240 has covered the target wild animal 100.

[0112] 5. Determine the predicted status of the target wildlife:

[0113] like Figure 2 , Figure 5 As shown, when the target wild animal is in a high-speed motion mode, the prediction of its state is obtained by executing the motion prediction process 700 by the motion prediction module. In some optional embodiments, similar to the target tracking process, the prediction process can be implemented using a motion prediction network 830 based on reinforcement learning. For example, the target wild animal 100 can be used as an intelligent agent, with the goal of hunting prey or escaping natural enemies, and the actions taken by it, as well as the corresponding position, speed and range of activity are continuously obtained. The specific implementation methods of constructing the motion prediction network based on reinforcement learning are all known to those skilled in the art and will not be repeated here.

[0114] The reason for setting the motion prediction network 830 for the high-speed motion state of the target wild animal 100 and the target tracking network 820 of the drone independently is that the motion prediction network 830 only reflects the behavior pattern of the target wild animal 100 itself in hunting or escaping. When different drones are dispatched to take over the tracking and monitoring of the target wild animal 100, by synchronizing the motion prediction network 830, the drone that takes over can directly have a good prediction ability for the high-speed motion of the target wild animal 100 without retraining; and the target tracking network 830 of the drone optimizes its tracking ability based on each unmanned state-related parameter. Since different drones have differences in performance, structure, specifications, etc., synchronizing the target tracking network of a drone to another drone with different specifications and performance is not significant. Therefore, separating the two networks based on reinforcement learning and only synchronizing the network for predicting the high-speed motion of the target wild animal not only reduces the network scale and model complexity, but also ensures the synchronization of the trained effective model between multiple drones.

[0115] The above is a detailed introduction to the specific implementation methods of the present application. For those skilled in the art, several improvements and modifications may be made to the present application without departing from the principles of the present application. These improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A distributed wildlife monitoring system, comprising a plurality of drones, each of which is provided with a monitoring and dispatching device, characterized in that: The monitoring and scheduling device comprises: Data acquisition unit, used to obtain the status and monitoring data of the drone; an identification unit, used to identify the environmental state, the actual state and behavior pattern of the target wild animal from the monitoring data, wherein the behavior pattern of the target wild animal at least includes a habitat pattern, a migration pattern and a high-speed movement pattern; A tracking control unit, comprising a target tracking module and a motion prediction module, wherein the motion prediction module outputs its predicted state based on the actual state of the target wild animal when the behavior mode of the target wild animal is a high-speed motion mode; the target tracking module determines the tracking action for monitoring the target wild animal based on the state of the drone, monitoring environment data, and the predicted state of the target wild animal when the behavior mode of the target wild animal is a high-speed motion mode, and determines the tracking action for monitoring the target wild animal based on the state of the drone, monitoring environment data, and the actual state of the target wild animal when the behavior mode of the target wild animal is a mode other than the high-speed motion mode; The data synchronization unit is used to synchronize data with monitoring and scheduling devices installed on other drones.

2. The distributed wildlife monitoring system according to claim 1, characterized in that: The state of the drone includes the drone's location information, speed information, remaining energy information, and monitoring coverage information; The monitoring data includes visible light video and / or infrared video, three-dimensional ranging data and meteorological monitoring data of the monitoring area; The environmental status includes terrain information, climate condition information and obstacle location information of the monitoring area.

3. The distributed wildlife monitoring system according to claim 2, characterized in that: The recognition unit recognizes the actual state and behavior pattern of the target wild animal based on the following steps: Step 510, extracting visible light video and / or infrared video and three-dimensional ranging data of the monitoring area; Step 520, using a target detection algorithm to identify the target wildlife from the visible light video and / or infrared video information and determine the actual state of the target wildlife, wherein the actual state includes the actual position, actual speed and actual activity range of the target wildlife; Step 530: determining the behavior pattern of the target wild animal based on the actual state of the target wild animal.

4. The distributed wildlife monitoring system according to claim 1, characterized in that: For any drone, when its recognition unit identifies a target wildlife, its target tracking module determines the tracking action for monitoring the target wildlife through the following steps: Step 610, obtaining the state of the drone, the state of the environment, the actual state and behavior pattern of the target wild animal; Step 620, determining whether the behavior mode of the target wild animal is a high-speed movement mode; Step 630: If the judgment result is no, then construct the state space S(t) for monitoring the target wildlife based on equation (1). If the judgment result is yes, the predicted state of the target wildlife is further obtained, and the state space S(t) of the target wildlife monitoring is constructed based on formula (2): Among them, S UAV (t) is the state of the drone, Env(t) is the state of the environment, The actual status of the target wildlife, The predicted state of the target wild animal output by the movement prediction module; Step 640, using a tracking strategy network based on reinforcement learning to determine a tracking action for monitoring the target wildlife, and updating the tracking strategy network based on the result of the UAV performing the tracking action; Step 650, determining whether the remaining energy of the drone is lower than the return energy lower limit threshold, if not, returning to step 610, if yes, executing step 660; Step 660, dispatching other drones to take over the monitoring of the target wildlife through the data synchronization unit.

5. The distributed wildlife monitoring system according to claim 4, characterized in that: The state space of the tracking strategy network is S(t); The action space of the tracking strategy network is the change in altitude, speed and direction of the drone; The instant reward function of the tracking strategy network includes the following reward items: a distance reward item, an energy consumption reward item, a monitoring coverage reward item, and a tracking stability reward item.

6. The distributed wildlife monitoring system according to claim 4, characterized in that: When the target wildlife is in a mode other than the high-speed movement mode, the return energy lower limit threshold is not less than the minimum energy required for the drone to return; When the target wildlife is in high-speed motion mode, the return energy lower limit threshold is not less than 2+k times the minimum energy required for the drone to return, where k is the high-speed motion compensation factor, and the positive or negative value of k is determined by the deviation between the high-speed motion direction of the target wildlife and the return direction of the drone.

7. The distributed wildlife monitoring system according to claim 4, characterized in that: Step 660 further includes the following steps: Step 661, sending the actual status, behavior pattern and movement prediction network of the target wild animal to other drones through the data synchronization unit; Step 662, after determining that at least one other drone has taken over the monitoring of the target wildlife, control the drone to return.

8. The distributed wildlife monitoring system according to claim 7, characterized in that: When the target wildlife is in a mode other than high-speed movement mode, the drone begins to return after determining that at least one other drone has taken over monitoring of the target wildlife and taken off; When the target wildlife is in a high-speed motion mode, the drone begins to return after the identification unit of at least one other drone has identified the target wildlife.

9. The distributed wildlife monitoring system according to claim 1, characterized in that: The motion prediction module uses a motion prediction network based on reinforcement learning to predict and output the predicted state of the target wild animal.

Images