Drone Behavior Monitoring Method, Device, Electronic Device, and Computer Readable Medium

Abstract

Embodiments of the present disclosure disclose a method, apparatus, electronic device, and computer-readable medium for monitoring the behavior of drones. A specific implementation of the method includes: determining the monitoring range corresponding to each drone monitoring device included in the drone monitoring system; determining the total monitoring range corresponding to the drone monitoring device; performing regional division processing on the total monitoring range; real-time monitoring whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system; obtaining the drone status information and drone trajectory information of the drone; inputting the drone status information and drone trajectory information into a pre-trained drone behavior prediction model to obtain drone prediction behavior information; and in response to the drone prediction behavior information satisfying a preset behavior condition, performing a drone control intervention operation on the drone. This implementation avoids large deviations in the prediction results, thereby avoiding waste of drone resources.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and in particular, to methods, devices, electronic devices, and computer-readable media for monitoring the behavior of unmanned aerial vehicles (UAVs). Background Art

[0002] Currently, with the rapid development of UAV technology, UAVs have been widely used in various fields of life. How to monitor and control the behavior of UAVs has become an important research topic. Currently, when monitoring the behavior of UAVs, the commonly used method is to determine the position of the UAV through a single active detection device or passive detection device, predict the behavior of the UAV based on the position of the UAV, and control the UAV with dangerous behavior.

[0003] However, when monitoring the behavior of UAVs in the above manner, the following technical problems often exist:

[0004] Predicting the behavior of the UAV only based on the position of the UAV does not identify features such as the attitude of the UAV, resulting in incorrect identification of the behavior of the UAV, and thus wasting UAV resources.

[0005] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0006] This content part of the present disclosure is used to briefly introduce concepts that will be described in detail in the following detailed implementation part. This content part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure propose methods, devices, electronic devices, and computer-readable media for monitoring the behavior of UAVs to solve one or more of the technical problems mentioned in the above background art section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for monitoring the behavior of an unmanned aerial vehicle (UAV). The method includes: determining the monitoring range corresponding to each UAV monitoring device included in the UAV monitoring system to obtain a set of monitoring ranges; determining the total monitoring range corresponding to the UAV monitoring device according to the set of monitoring ranges; performing a regional division process on the total monitoring range according to preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range; real-time monitoring, by at least one UAV monitoring device included in the UAV monitoring system, whether a UAV appears within the total monitoring range; in response to detecting that a UAV appears within the total monitoring range, determining the UAV status information and the UAV trajectory information of the UAV; inputting the UAV status information and the UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information; and in response to the UAV prediction behavior information satisfying a preset behavior condition, performing a UAV control intervention operation on the UAV, where the preset behavior condition is that the UAV prediction behavior information indicates that the UAV flies towards the third monitoring range.

[0009] In a second aspect, some embodiments of the present disclosure provide a UAV behavior monitoring device. The device includes: a first determination unit configured to determine the monitoring range corresponding to each UAV monitoring device included in the UAV monitoring system to obtain a set of monitoring ranges; a second determination unit configured to determine the total monitoring range corresponding to the UAV monitoring device according to the set of monitoring ranges; a regional division unit configured to perform a regional division process on the total monitoring range according to preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range; a monitoring unit configured to real-time monitor, by at least one UAV monitoring device included in the UAV monitoring system, whether a UAV appears within the total monitoring range; a third determination unit configured to, in response to detecting that a UAV appears within the total monitoring range, determine the UAV status information and the UAV trajectory information of the UAV; an input unit configured to input the UAV status information and the UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information; and a UAV control intervention unit configured to, in response to the UAV prediction behavior information satisfying a preset behavior condition, perform a UAV control intervention operation on the UAV.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the method described in any implementation manner of the first aspect.

[0011] Fourthly, some embodiments of the present disclosure provide a computer-readable medium, on which a computer program is stored. When the program is executed by a processor, the method described in any implementation manner of the first aspect above is implemented.

[0012] The above various embodiments of the present disclosure have the following beneficial effects: Through the unmanned aerial vehicle (UAV) behavior monitoring method of some embodiments of the present disclosure, the waste of UAV resources is avoided. Specifically, the reason for the waste of UAV resources is that the behavior of the UAV is predicted only based on the position of the UAV, and features such as the attitude of the UAV are not identified, resulting in incorrect identification of the behavior of the UAV, and thus waste of UAV resources. Based on this, in the UAV behavior monitoring method of some embodiments of the present disclosure, first, the monitoring range corresponding to each UAV monitoring device included in the above UAV monitoring system is determined to obtain a set of monitoring ranges; according to the above set of monitoring ranges, the total monitoring range corresponding to the above UAV monitoring device is determined. Thus, the range within which the UAV monitoring system can monitor the UAV can be determined. Secondly, according to the preset area division information, the above total monitoring range is subjected to area division processing to generate a first monitoring range, a second monitoring range, and a third monitoring range. Thus, the monitoring range can be divided into three levels. Then, through at least one UAV monitoring device included in the above UAV monitoring system, it is monitored in real time whether a UAV appears within the above total monitoring range. Thus, it can be determined whether a UAV enters the monitoring range. After that, in response to monitoring that a UAV appears within the above total monitoring range, the UAV state information and the UAV trajectory information of the UAV are determined. Thus, the flight state and flight trajectory of the UAV can be determined simultaneously. Then, the above UAV state information and the above UAV trajectory information are input into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information. Thus, the behavior of the UAV can be predicted. Finally, in response to the above UAV prediction behavior information satisfying the preset behavior condition, a UAV control intervention operation is performed on the above UAV. Thus, a UAV with a dangerous predicted behavior can be controlled and intervened. Also, because the behavior of the UAV is predicted based on features such as the flight state and flight trajectory, large deviations in the prediction results are avoided, and thus waste of UAV resources is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the elements and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flowchart of some embodiments of the UAV behavior monitoring method according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of a drone behavior monitoring device according to the present disclosure;

[0016] Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed implementation manners

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0018] In addition, it should be noted that only parts related to the relevant invention are shown in the drawings for the convenience of description. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0019] It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence relationship of the functions executed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0023] Figure 1 Flow 100 of some embodiments of a drone behavior monitoring method according to the present disclosure is shown. The drone behavior monitoring method includes the following steps:

[0024] Step 101, determine the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a set of monitoring ranges.

[0025] In some embodiments, the execution subject of the UAV behavior monitoring method (such as a server) may determine the monitoring range corresponding to each UAV monitoring device included in the above UAV monitoring system, and obtain a set of monitoring ranges. Among them, the above UAV monitoring system includes at least one UAV monitoring device. The UAV monitoring devices in the above at least one set of UAV monitoring devices include monitoring cameras and monitoring sensors. The above monitoring sensors may be sensors for emitting monitoring signals.

[0026] In practice, the monitoring range corresponding to each UAV monitoring device included in the UAV monitoring system can be determined through the following steps:

[0027] First step, for each UAV monitoring device among the at least one UAV monitoring device included in the above UAV monitoring system, perform the following processing steps:

[0028] First processing step, determine the sensor radius of the monitoring sensors included in the above unmanned monitoring device. Among them, the above sensor radius may be the farthest distance at which the above monitoring sensors can emit signals.

[0029] Second processing step, obtain the coordinate information of the above UAV monitoring device. In practice, the coordinate information of the UAV monitoring device can be obtained from the target database through wired connection or wireless connection. The above coordinate information may be the location where the above unmanned monitoring device is installed and altitude information.

[0030] Third processing step, according to the above coordinate information and the above sensor radius, determine the sensor monitoring range corresponding to the above UAV monitoring device. In practice, a circular area with the coordinate represented by the above coordinate information as the center and the sensor radius as the radius can be determined as the sensor monitoring range corresponding to the above UAV monitoring device.

[0031] Fourth processing step, obtain the camera information of the monitoring camera included in the above UAV monitoring device. Among them, the above camera information includes the camera orientation, the camera installation height, and the camera pitch angle. The above camera orientation may be the direction in which the camera captures the picture.

[0032] Fifth processing step, according to the camera installation height and the camera pitch angle included in the above camera information, determine the camera monitoring range corresponding to the above UAV monitoring device.

[0033] Sixth processing step, perform a merging process on the above sensor monitoring range and the above camera monitoring range to generate a merged monitoring range as the monitoring range.

[0034] Second step, determine the generated respective monitoring ranges as a set of monitoring ranges.

[0035] Step 102: Determine the total monitoring range corresponding to the UAV monitoring device according to the monitoring range set.

[0036] In some embodiments, the above-mentioned execution entity may determine the total monitoring range corresponding to the UAV monitoring device according to the above-mentioned monitoring range set. In practice, each monitoring range included in the above-mentioned monitoring range set may be combined into the total monitoring range.

[0037] Step 103: Perform regional division processing on the total monitoring range according to the preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range.

[0038] In some embodiments, the above-mentioned execution entity may perform regional division processing on the above-mentioned total monitoring range according to the preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range. Among them, the above-mentioned first monitoring range may represent a safety range. The above-mentioned second monitoring range may represent a warning range. The above-mentioned third monitoring range may represent a protection range.

[0039] In practice, the following steps may be used to perform regional division processing on the above-mentioned total monitoring range:

[0040] The first step: Determine a first regional range, a second regional range, and a third regional range according to the first division region, the second division region, and the third division region included in the above-mentioned preset regional division information. Among them, the above-mentioned first division region is the UAV identification region, the above-mentioned second division region is the UAV warning region, and the above-mentioned third division region is the UAV control region.

[0041] The second step: Perform regional division processing on the above-mentioned total monitoring range according to the above-mentioned first regional range, second regional range, and third regional range to generate a first monitoring range, a second monitoring range, and a third monitoring range.

[0042] Step 104: Real-time monitor whether a UAV appears within the total monitoring range through at least one UAV monitoring device included in the UAV monitoring system.

[0043] In some embodiments, the above-mentioned execution entity may real-time monitor whether a UAV appears within the above-mentioned total monitoring range through at least one UAV monitoring device included in the above-mentioned UAV monitoring system.

[0044] In practice, the following steps may be used to real-time monitor whether a UAV appears within the above-mentioned total monitoring range:

[0045] The first step: In response to the monitoring sensor included in any one of the at least one UAV monitoring device detecting the existence of a target object, obtain the target object images captured by at least one monitoring camera to obtain a set of target object images.

[0046] In the second step, perform UAV recognition processing on each target object image in the above target object image set to generate recognition results, obtaining a recognition result set. In practice, each target object image in the above target object image set can be input into a pre-trained UAV recognition model to obtain recognition results. Among them, the above UAV recognition model can be a pre-trained convolutional neural network model.

[0047] In the third step, in response to the above recognition result set satisfying the first preset condition, determine the above target object as the target UAV. Among them, the above first preset condition can be that the number of recognition results indicating the recognition of the UAV in the above recognition result set is greater than or equal to the preset recognition number.

[0048] Step 105, in response to detecting a UAV in the total monitoring range, determine the UAV status information and UAV trajectory information of the UAV.

[0049] In some embodiments, the above execution entity can, in response to detecting a UAV in the above total monitoring range, determine the UAV status information and UAV trajectory information of the UAV.

[0050] In practice, when positioning a UAV with only a single sensor, there is a deviation in the signal transmission angle, resulting in the position of the recognized UAV not matching the actual position of the UAV, wrongly determining the status of the UAV, and further leading to incorrect control of the UAV, causing waste of UAV resources.

[0051] In some optional implementation manners of some embodiments, the above execution entity can determine the UAV status information and UAV trajectory information of the UAV through the following steps:

[0052] In the first step, in response to detecting a UAV in the above total monitoring range, determine at least one monitoring sensor corresponding to the UAV as the target sensor group. Among them, the target sensor in the above target sensor group can be a monitoring sensor capable of monitoring the above UAV.

[0053] In the second step, for each target sensor in the above target sensor group, perform the following determination steps:

[0054] The first determination step is to control the above target sensor to send a detection signal to the above UAV in real time and receive the reflected signal.

[0055] The second determination step is to generate a UAV monitoring position sequence according to the direction of the received reflected signal and the signal transmission time.

[0056] Step 3: Select, from each of the generated UAV monitoring position sequences, the UAV monitoring position sequence that meets the second preset condition as the target position sequence. Among them, the above-mentioned second preset condition may be to select the UAV monitoring position sequence corresponding to the target sensor closest to the UAV.

[0057] Step 4: Perform a first correction process on each target position in the target position sequence according to each UAV monitoring position sequence other than the above-mentioned target position sequence to generate a corrected position sequence. Among them, the above-mentioned first correction process may be, for each target position in the target position sequence, to correct the target position according to the center point of each UAV monitoring position corresponding to the target position.

[0058] Step 5: Obtain a set of UAV image groups corresponding to the UAV captured by at least one monitoring camera to obtain a UAV image group set.

[0059] Step 6: For each UAV image group in the UAV image group set, perform the following recognition steps:

[0060] First recognition step: Identify at least one environmental feature point of each UAV image included in the UAV image group to generate an environmental feature point group and obtain an environmental feature point group set.

[0061] Second recognition step: Determine the model point position corresponding to the UAV image group according to the environmental feature point group set and the preset three-dimensional environmental model.

[0062] Third recognition step: Determine the UAV position corresponding to the model point position.

[0063] Fourth recognition step: Perform a second correction process on the corrected position corresponding to the UAV position according to the UAV position to generate a corrected position.

[0064] Step 7: Combine the generated corrected positions into a corrected position sequence and determine the UAV trajectory information as the above-mentioned corrected position sequence.

[0065] Step 8: Determine the UAV state information according to the above-mentioned UAV trajectory information.

[0066] The above first step - eighth step, as an inventive point of the embodiment of the present disclosure, in combination with the following step "step 107", solves the technical problem that "when positioning a drone through a single sensor, there is a deviation in the signal transmission angle, resulting in the identified position of the drone not matching the actual position of the drone, and further leading to incorrect control of the drone, causing waste of drone resources". The reasons for the waste of drone resources are as follows: When positioning a drone through a single sensor, there is a deviation in the signal transmission angle, resulting in the identified position of the drone not matching the actual position of the drone, and further leading to incorrect control of the drone, causing waste of drone resources. If the above factors are solved, the effect of avoiding waste of drone resources can be achieved. To achieve this effect, the present disclosure first, in response to detecting a drone within the above-mentioned total monitoring range, determines at least one monitoring sensor corresponding to the drone as a target sensor group. Thus, the sensors that can monitor the drone can be determined. Second, for each target sensor in the above target sensor group, perform the following determination steps: Real-time control the target sensor to send a detection signal to the drone and receive the reflected signal; generate a drone monitoring position sequence based on the direction of the received reflected signal and the signal transmission time. Thus, the captured drone position can be determined. Third, select from the generated drone monitoring position sequences the drone monitoring position sequences that meet the second preset condition as the target position sequences. Thus, the drone monitoring position sequences that most conform to the actual position of the drone can be selected. Fourth, perform a first correction process on each target position in the above target position sequence according to the drone monitoring position sequences other than the above target position sequence to generate a corrected position sequence. Thus, the captured drone position can be corrected. Fifth, obtain a set of drone image groups corresponding to the drone captured by at least one monitoring camera to obtain a set of drone image group sets. Thus, images of the drone during flight can be collected. Sixth, for each drone image group in the above set of drone image group sets, perform the following identification steps: Identify at least one environmental feature point in each drone image included in the drone image group to generate a group of environmental feature points to obtain a set of environmental feature point groups; determine the model point corresponding to the drone image group according to the above set of environmental feature point groups and a preset three-dimensional environmental model; determine the drone position corresponding to the model point; perform a second correction process on the corrected position corresponding to the drone position according to the above drone position to generate a corrected position. Thus, the drone position can be corrected again through environmental comparison. Seventh, combine the generated corrected positions into a corrected position sequence, and determine the above corrected position sequence as the drone trajectory information; determine the drone state information according to the above drone trajectory information. Thus, the current state of the drone can be determined.In combination with step "step 107", in response to the above-mentioned UAV prediction behavior information satisfying the preset behavior conditions, perform a UAV control intervention operation on the above-mentioned UAV. Thereby, incorrect control intervention operations on the UAV are avoided, and further waste of UAV resources is avoided.

[0067] Step 106, input the UAV state information and the UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information.

[0068] In some embodiments, the above-mentioned execution subject may input the above-mentioned UAV state information and the above-mentioned UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information.

[0069] Optionally, the above-mentioned UAV behavior prediction model may be trained through the following steps:

[0070] The first step is to obtain a sample set.

[0071] In some embodiments, the above-mentioned execution subject may obtain a sample set. Among them, the samples in the above-mentioned sample set include sample UAV state information and sample UAV trajectory information, as well as sample UAV prediction behavior information corresponding to the above-mentioned sample UAV state information and sample UAV trajectory information.

[0072] The second step is to select samples from the above-mentioned sample set.

[0073] In some embodiments, the above-mentioned execution subject may select samples from the above-mentioned sample set. Here, the above-mentioned execution subject may randomly select samples from the above-mentioned sample set.

[0074] The third step is to input the above-mentioned samples into an initial network model to obtain UAV prediction behavior information corresponding to the above-mentioned samples.

[0075] In some embodiments, the above-mentioned execution subject may input the above-mentioned samples into an initial network model to obtain UAV prediction behavior information corresponding to the above-mentioned samples. Among them, the above-mentioned initial neural network may be a classification model capable of obtaining UAV prediction behavior information based on UAV state information and UAV trajectory information.

[0076] The fourth step is to determine the loss value between the above-mentioned UAV prediction behavior information and the sample UAV prediction behavior information included in the above-mentioned samples.

[0077] In some embodiments, the above-mentioned execution entity may determine a loss value between the above-mentioned predicted UAV behavior information and the sample UAV predicted behavior information included in the above-mentioned sample. In practice, based on a preset loss function, the loss value between the above-mentioned predicted UAV behavior information and the sample UAV predicted behavior information included in the above-mentioned sample may be determined. For example, the above-mentioned preset loss function may be a cross-entropy loss function.

[0078] In the fifth step, in response to the above-mentioned loss value being greater than or equal to a preset threshold, adjust the network parameters of the above-mentioned initial network model.

[0079] In some embodiments, the above-mentioned execution entity may, in response to the above-mentioned loss value being greater than or equal to a preset threshold, adjust the network parameters of the above-mentioned initial network model. Here, there is no limitation on the setting of the preset threshold. For example, the difference between the loss value and the preset threshold may be calculated to obtain a loss difference. On this basis, methods such as backpropagation and stochastic gradient descent are used to forward the error value from the last layer of the model to adjust the parameters of each layer. Of course, according to needs, the method of network freezing (dropout) may also be adopted to keep the network parameters of some layers unchanged without adjustment, and no limitation is imposed on this.

[0080] Optionally, in response to the above-mentioned loss value being less than the above-mentioned preset threshold, determine the above-mentioned initial network model as a UAV behavior prediction model.

[0081] In some embodiments, the above-mentioned execution entity may, in response to the above-mentioned loss value being less than the above-mentioned preset threshold, determine the above-mentioned initial network model as a UAV behavior prediction model.

[0082] Step 107, in response to the predicted UAV behavior information satisfying a preset behavior condition, perform a UAV control intervention operation on the UAV.

[0083] In some embodiments, the above-mentioned execution entity may, in response to the above-mentioned predicted UAV behavior information satisfying a preset behavior condition, perform a UAV control intervention operation on the above-mentioned UAV. Among them, the above-mentioned preset behavior condition may be that the predicted UAV behavior information indicates that the UAV flies towards the above-mentioned third monitoring range. The above-mentioned UAV control intervention operation may be to shoot down the UAV by physical means.

[0084] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the UAV behavior monitoring method of some embodiments of the present disclosure, the waste of UAV resources is avoided. Specifically, the reason for the waste of UAV resources is that only the position of the UAV is used to predict the UAV behavior, and features such as the attitude of the UAV are not recognized, resulting in incorrect identification of the UAV behavior, and thus waste of UAV resources. Based on this, in the UAV behavior monitoring method of some embodiments of the present disclosure, first, determine the monitoring range corresponding to each UAV monitoring device included in the above UAV monitoring system to obtain a set of monitoring ranges; according to the above set of monitoring ranges, determine the total monitoring range corresponding to the above UAV monitoring device. Thus, the range within which the UAV monitoring system can monitor the UAV can be determined. Secondly, according to the preset area division information, perform area division processing on the above total monitoring range to generate a first monitoring range, a second monitoring range, and a third monitoring range. Thus, the monitoring range can be divided into three levels. Then, through at least one UAV monitoring device included in the above UAV monitoring system, real-time monitor whether a UAV appears within the above total monitoring range. Thus, it can be determined whether a UAV enters the monitoring range. After that, in response to detecting that a UAV appears within the above total monitoring range, determine the UAV status information and UAV trajectory information of the UAV. Thus, the flight state and flight trajectory of the UAV can be determined simultaneously. Then, input the above UAV status information and the above UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information. Thus, the behavior of the UAV can be predicted. Finally, in response to the above UAV prediction behavior information meeting the preset behavior conditions, perform a UAV control intervention operation on the above UAV. Thus, control intervention can be performed on UAVs with predicted dangerous behaviors. Also, because the UAV behavior is predicted through features such as the flight state and flight trajectory, large deviations in the prediction results are avoided, and thus waste of UAV resources is avoided.

[0085] Further referring to Figure 2 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a UAV behavior monitoring device. These device embodiments correspond to Figure 1 the method embodiments shown, and the UAV behavior monitoring device can be specifically applied to various electronic devices.

[0086] As Figure 2As shown in the figure, the UAV behavior monitoring device 200 of some embodiments includes: a first determination unit 201, a second determination unit 202, a region division unit 203, a monitoring unit 204, a third determination unit 205, an input unit 206, and a UAV control intervention unit 207. Among them, the first determination unit 201 is configured to determine the monitoring range corresponding to each UAV monitoring device included in the above UAV monitoring system, and obtain a set of monitoring ranges; the second determination unit 202 is configured to determine the total monitoring range corresponding to the above UAV monitoring device according to the above set of monitoring ranges; the region division unit 203 is configured to perform region division processing on the above total monitoring range according to preset region division information to generate a first monitoring range, a second monitoring range, and a third monitoring range; the monitoring unit 204 is configured to monitor in real time whether a UAV appears within the above total monitoring range through at least one UAV monitoring device included in the above UAV monitoring system; the third determination unit 205 is configured to determine the UAV status information and UAV trajectory information of the UAV in response to monitoring that a UAV appears within the above total monitoring range; the input unit 206 is configured to input the above UAV status information and the above UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information; the UAV control intervention unit 207 is configured to perform a UAV control intervention operation on the above UAV in response to the above UAV prediction behavior information satisfying a preset behavior condition.

[0087] It can be understood that the various units described in the UAV behavior monitoring device 200 correspond to the respective steps in the method described in the reference Figure 1 Therefore, the operations, features, and beneficial effects described above for the method also apply to the UAV behavior monitoring device 200 and the units included therein, and will not be elaborated here.

[0088] Next, refer to Figure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not impose any limitations on the functions and usage ranges of the embodiments of the present disclosure.

[0089] As Figure 3As shown, the electronic device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0090] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 the electronic device 300 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 3 Each block shown in may represent one device or, as needed, multiple devices.

[0091] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above functions defined in the methods of some embodiments of the present disclosure are executed.

[0092] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0093] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0094] The above computer-readable medium may be included in the above electronic device; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: determine the monitoring range corresponding to each unmanned aerial vehicle (UAV) monitoring device included in the above UAV monitoring system to obtain a set of monitoring ranges. Determine the total monitoring range corresponding to the above UAV monitoring device according to the above set of monitoring ranges. Perform a regional division process on the above total monitoring range according to preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range. Real-time monitor whether a UAV appears within the above total monitoring range through at least one UAV monitoring device included in the above UAV monitoring system. In response to detecting a UAV within the above total monitoring range, determine the UAV state information and the UAV trajectory information of the UAV. Input the above UAV state information and the above UAV trajectory information into a pre-trained UAV behavior prediction model to obtain UAV prediction behavior information. In response to the above UAV prediction behavior information meeting a preset behavior condition, perform a UAV control intervention operation on the above UAV, where the above preset behavior condition is that the UAV prediction behavior information indicates that the UAV is flying towards the above third monitoring range.

[0095] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0097] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a first determination unit, a second determination unit, a region division unit, a monitoring unit, a third determination unit, an input unit, and a drone control intervention unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the first determination unit can also be described as "the unit that determines the monitoring range corresponding to each drone monitoring device included in the above drone monitoring system and obtains a set of monitoring ranges".

[0098] The functions described above can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0099] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, technical solutions formed by mutually replacing the above features with (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A drone behavior monitoring method, applied to a drone monitoring system, wherein: The drone monitoring system includes at least one drone monitoring device, including: Determine a monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; Determine the total monitoring range corresponding to the UAV monitoring device according to the monitoring range set; According to the preset area division information, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range; Real-time monitoring of whether a drone appears within the total monitoring range by at least one drone monitoring device included in the drone monitoring system; In response to detecting the presence of a drone within the total monitoring range, determining drone state information and drone trajectory information of the drone; Wherein, in response to monitoring the presence of a drone within the total monitoring range, determining drone status information and drone trajectory information of the drone includes: In response to detecting the presence of a drone within the total monitoring range, determining at least one monitoring sensor corresponding to the drone as a target sensor group; For each target sensor in the target sensor group, the following determination steps are performed: Controlling the target sensor in real time to send a detection signal to the UAV and receive a reflected signal; Generate a drone monitoring position sequence based on the direction of the received reflected signal and the signal transmission time; Selecting a drone monitoring position sequence that meets a second preset condition from the generated drone monitoring position sequences as a target position sequence; According to each drone monitoring position sequence excluding the target position sequence, performing a first correction process on each target position in the target position sequence to generate a corrected position sequence; Acquire a drone image group corresponding to the drone captured by at least one monitoring camera to obtain a drone image group set; For each drone image group in the drone image group set, the following identification steps are performed: Identify at least one environmental feature point of each drone image in each drone image included in the drone image group to generate an environmental feature point group, thereby obtaining an environmental feature point group set; Determining the model points corresponding to the drone image group according to the environmental feature point group set and the preset three-dimensional environmental model; Determine the position of the drone corresponding to the model point; According to the drone position, performing a second correction process on the corrected position corresponding to the drone position to generate a corrected position; Combining the generated correction positions into a correction position sequence, and determining the correction position sequence as UAV trajectory information; Determining drone status information according to the drone trajectory information; Inputting the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; In response to the drone predicted behavior information satisfying a preset behavior condition, a drone control intervention operation is performed on the drone, wherein the preset behavior condition is that the drone predicted behavior information indicates that the drone is flying toward the third monitoring range.

2. The method according to claim 1, wherein: The drone monitoring device in the at least one drone monitoring device comprises: a monitoring camera and a monitoring sensor; And, the determining of the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set includes: For each drone monitoring device in the at least one drone monitoring device included in the drone monitoring system, the following processing steps are performed: Determining a sensor radius of a monitoring sensor included in the drone monitoring device; Obtaining coordinate information of the drone monitoring device; Determine the sensor monitoring range corresponding to the drone monitoring device according to the coordinate information and the sensor radius; Obtaining camera information of a monitoring camera included in the drone monitoring device, wherein the camera information includes a camera installation height and a camera pitch angle; Determine the camera monitoring range corresponding to the drone monitoring device according to the camera installation height and the camera pitch angle included in the camera information; The sensor monitoring range and the camera monitoring range are combined to generate a combined monitoring range as the monitoring range; The generated monitoring ranges are determined as a monitoring range set.

3. The method according to claim 1, wherein: The performing of area division processing on the total monitoring range according to the preset area division information to generate a first monitoring range, a second monitoring range and a third monitoring range includes: Determine a first area range, a second area range, and a third area range according to the first area range, the second area range, and the third area range included in the preset area division information, wherein the first area range is a drone identification area, the second area range is a drone early warning area, and the third area range is a drone control area; According to the first area range, the second area range and the third area range, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range.

4. The method according to claim 1, wherein: The at least one drone monitoring device included in the drone monitoring system monitors in real time whether a drone appears within the total monitoring range, including: In response to a monitoring sensor included in any one of the at least one unmanned aerial vehicle monitoring devices detecting the presence of a target object, acquiring an image of the target object taken by at least one monitoring camera to obtain a set of target object images; Performing drone recognition processing on each target object image in the target object image set to generate a recognition result, thereby obtaining a recognition result set; In response to the recognition result set satisfying a first preset condition, the target object is determined to be a target drone.

5. The method according to claim 1, wherein: The drone behavior prediction model is trained by the following steps: Acquire a sample set, wherein the samples in the sample set include sample drone state information and sample drone trajectory information, and sample drone predicted behavior information corresponding to the sample drone state information and the sample drone trajectory information; Selecting a sample from the sample set; Inputting the sample into the initial network model to obtain the predicted behavior information of the drone corresponding to the sample; Determine a loss value between the drone prediction behavior information corresponding to the sample and the sample drone prediction behavior information included in the sample; In response to the loss value being greater than or equal to a preset threshold, adjusting the network parameters of the initial network model.

6. The method according to claim 5, wherein: The method further comprises: In response to the loss value being less than the preset threshold, the initial network model is determined as a drone behavior prediction model.

7. A drone behavior monitoring device, comprising: A first determining unit is configured to determine a monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; A second determination unit is configured to determine a total monitoring range corresponding to the drone monitoring device according to the monitoring range set; A region division unit is configured to perform region division processing on the total monitoring range according to preset region division information to generate a first monitoring range, a second monitoring range and a third monitoring range; A monitoring unit configured to monitor in real time whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system; a third determining unit, configured to determine drone state information and drone trajectory information of the drone in response to detecting the presence of a drone within the total monitoring range; The third determining unit is further configured to: In response to detecting the presence of a drone within the total monitoring range, determining at least one monitoring sensor corresponding to the drone as a target sensor group; For each target sensor in the target sensor group, the following determination steps are performed: Controlling the target sensor in real time to send a detection signal to the UAV and receive a reflected signal; Generate a drone monitoring position sequence based on the direction of the received reflected signal and the signal transmission time; Selecting a drone monitoring position sequence that meets a second preset condition from the generated drone monitoring position sequences as a target position sequence; According to each drone monitoring position sequence excluding the target position sequence, performing a first correction process on each target position in the target position sequence to generate a corrected position sequence; Acquire a drone image group corresponding to the drone captured by at least one monitoring camera to obtain a drone image group set; For each drone image group in the drone image group set, the following identification steps are performed: Identify at least one environmental feature point of each drone image in each drone image included in the drone image group to generate an environmental feature point group, thereby obtaining an environmental feature point group set; Determining the model points corresponding to the drone image group according to the environmental feature point group set and the preset three-dimensional environmental model; Determine the position of the drone corresponding to the model point; According to the drone position, performing a second correction process on the corrected position corresponding to the drone position to generate a corrected position; Combining the generated correction positions into a correction position sequence, and determining the correction position sequence as UAV trajectory information; Determining drone status information according to the drone trajectory information; An input unit, configured to input the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; The drone control intervention unit is configured to perform a drone control intervention operation on the drone in response to the drone predicted behavior information satisfying a preset behavior condition.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images