Unmanned aerial vehicle behavior monitoring method and device, electronic equipment and computer readable medium

Abstract

The embodiment of the invention discloses an unmanned aerial vehicle behavior monitoring method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: determining a monitoring range corresponding to each unmanned aerial vehicle monitoring device included in an unmanned aerial vehicle monitoring system; determining a monitoring total range corresponding to the unmanned aerial vehicle monitoring equipment; carrying out regional division processing on the total monitoring range; through at least one unmanned aerial vehicle monitoring device included in the unmanned aerial vehicle monitoring system, whether an unmanned aerial vehicle appears in a monitoring total range is monitored in real time; acquiring unmanned aerial vehicle state information and unmanned aerial vehicle trajectory information of the unmanned aerial vehicle; inputting the unmanned aerial vehicle state information and the unmanned aerial vehicle trajectory information into a pre-trained unmanned aerial vehicle behavior prediction model to obtain unmanned aerial vehicle prediction behavior information; and performing unmanned aerial vehicle control intervention operation on the unmanned aerial vehicle in response to the unmanned aerial vehicle predicted behavior information meeting a preset behavior condition. According to the embodiment, large deviation of the prediction result is avoided, and waste of unmanned aerial vehicle resources is avoided.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and computer-readable medium for monitoring drone behavior. Background Art

[0002] At present, with the rapid development of drone technology, drones have been widely used in various fields of life. How to monitor and control the behavior of drones has become an important research topic. At present, when monitoring the behavior of drones, the usual method is to determine the location of the drone through a single active detection device or a passive detection device, predict the behavior of the drone based on the location of the drone, and control the drone with dangerous behavior.

[0003] However, when using the above method to monitor the behavior of drones, the following technical problems often occur:

[0004] The drone behavior is predicted only by the drone's position without identifying features such as the drone's posture, which leads to incorrect recognition of the drone's behavior and waste of drone resources.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a drone behavior monitoring method, device, electronic device, and computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for monitoring drone behavior, the method comprising: determining a monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; determining a total monitoring range corresponding to the drone monitoring device based on the monitoring range set; performing area division processing on the total monitoring range based on preset area division information to generate a first monitoring range, a second monitoring range and a third monitoring range; monitoring in real time whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system; determining drone status information and drone trajectory information of the drone in response to detecting the presence of a drone within the total monitoring range; inputting the drone status information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; performing a drone control intervention operation on the drone in response to the drone predicted behavior information satisfying a preset behavior condition, wherein the preset behavior condition is that the drone predicted behavior information characterizes that the drone is flying toward the third monitoring range.

[0009] In a second aspect, some embodiments of the present disclosure provide a drone behavior monitoring device, the device comprising: a first determination unit, configured to determine the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; a second determination unit, configured to determine the total monitoring range corresponding to the drone monitoring device based on the monitoring range set; an area division unit, configured to perform area division processing on the total monitoring range according to preset area division information to generate a first monitoring range, a second monitoring range and a third monitoring range; a monitoring unit, configured to monitor in real time whether a drone appears in the total monitoring range through at least one drone monitoring device included in the drone monitoring system; a third determination unit, configured to determine the drone status information and the drone trajectory information of the drone in response to monitoring the presence of a drone in the total monitoring range; an input unit, configured to input the drone status information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; a drone control intervention unit, configured to perform a drone control intervention operation on the drone in response to the drone predicted behavior information satisfying a preset behavior condition.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the drone behavior monitoring method of some embodiments of the present disclosure, the waste of drone resources is avoided. Specifically, the reason for the waste of drone resources is that the drone behavior is predicted only by the position of the drone, and the characteristics such as the posture of the drone are not identified, resulting in the wrong recognition of the behavior of the drone, which in turn causes the waste of drone resources. Based on this, the drone behavior monitoring method of some embodiments of the present disclosure, first, determines the monitoring range corresponding to each drone monitoring device included in the above-mentioned drone monitoring system, and obtains a monitoring range set; according to the above-mentioned monitoring range set, determines the total monitoring range corresponding to the above-mentioned drone monitoring device. Thus, it can be determined that the range in which the drone monitoring system can monitor the drone. Secondly, according to the preset area division information, the above-mentioned total monitoring range is subjected to area division processing to generate a first monitoring range, a second monitoring range and a third monitoring range. Thus, the monitoring range can be divided into three levels. Then, through at least one drone monitoring device included in the above-mentioned drone monitoring system, real-time monitoring is performed to determine whether a drone appears in the above-mentioned total monitoring range. Thus, it can be determined whether a drone enters the monitoring range. Afterwards, in response to monitoring the presence of a drone in the above-mentioned total monitoring range, the drone state information and drone trajectory information of the drone are determined. In this way, the flight state and flight trajectory of the drone can be determined at the same time. Then, the drone state information and the drone trajectory information are input into the pre-trained drone behavior prediction model to obtain the drone predicted behavior information. In this way, the behavior of the drone can be predicted. Finally, in response to the drone predicted behavior information meeting the preset behavior conditions, the drone control intervention operation is performed on the drone. In this way, control intervention can be performed on drones whose behavior is predicted to be dangerous. Also, because the drone behavior is predicted by features such as flight state and flight trajectory, large deviations in the prediction results are avoided, thereby avoiding waste of drone resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the drone behavior monitoring method according to the present disclosure;

[0015] Figure 2 is a schematic diagram of the structure of some embodiments of the drone behavior monitoring device according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 The process 100 of some embodiments of the drone behavior monitoring method according to the present disclosure is shown. The drone behavior monitoring method comprises the following steps:

[0024] Step 101: determine the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set.

[0025] In some embodiments, the execution subject (e.g., a server) of the drone behavior monitoring method may determine the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set. The drone monitoring system includes at least one drone monitoring device. The drone monitoring device in the at least one drone monitoring device set includes a monitoring camera and a monitoring sensor. The monitoring sensor may be a sensor for transmitting a monitoring signal.

[0026] In practice, the monitoring range of each drone monitoring device included in the drone monitoring system can be determined by the following steps:

[0027] In the first step, for each drone monitoring device in the at least one drone monitoring device included in the drone monitoring system, the following processing steps are performed:

[0028] The first processing step is to determine the sensor radius of the monitoring sensor included in the unmanned monitoring device, wherein the sensor radius may be the farthest distance at which the monitoring sensor can transmit a signal.

[0029] The second processing step is to obtain the coordinate information of the drone monitoring device. In practice, the coordinate information of the drone monitoring device can be obtained from the target database through a wired connection or a wireless connection. The coordinate information can be the location and altitude information of the installation of the drone monitoring device.

[0030] The third processing step is to determine the sensor monitoring range corresponding to the drone monitoring device based on the coordinate information and the sensor radius. In practice, the sensor monitoring range corresponding to the drone monitoring device can be determined as a circular area with the coordinate represented by the coordinate information as the center and the sensor radius as the radius.

[0031] The fourth processing step is to obtain the camera information of the monitoring camera included in the above-mentioned drone monitoring device. The above-mentioned camera information includes the camera direction, the camera installation height and the camera pitch angle. The above-mentioned camera direction can be the direction in which the camera captures the picture.

[0032] The fifth processing step is to determine the camera monitoring range corresponding to the above-mentioned UAV monitoring device according to the camera installation height and camera pitch angle included in the above-mentioned camera information.

[0033] The sixth processing step is to merge the above-mentioned sensor monitoring range and the above-mentioned camera monitoring range to generate a merged monitoring range as the monitoring range.

[0034] In the second step, each generated monitoring range is determined as a monitoring range set.

[0035] Step 102, determining the total monitoring range corresponding to the UAV monitoring device according to the monitoring range set.

[0036] In some embodiments, the execution subject may determine the total monitoring range corresponding to the drone monitoring device according to the monitoring range set. In practice, the monitoring ranges included in the monitoring range set may be combined into the total monitoring range.

[0037] Step 103, according to the preset area division information, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range.

[0038] In some embodiments, the execution subject may perform regional division processing on the total monitoring range according to preset regional division information to generate a first monitoring range, a second monitoring range, and a third monitoring range. The first monitoring range may represent a safety range. The second monitoring range may represent an early warning range. The third monitoring range may represent a protection range.

[0039] In practice, the above total monitoring range can be divided into regions through the following steps:

[0040] The first step is to determine the first area range, the second area range and the third area range according to the first area range, the second area range and the third area range included in the preset area division information. The first area range is the drone identification area, the second area range is the drone warning area, and the third area range is the drone control area.

[0041] In the second step, the above-mentioned total monitoring range is divided into regions according to the above-mentioned first area range, second area range and third area range to generate the first monitoring range, the second monitoring range and the third monitoring range.

[0042] Step 104: monitor in real time whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system.

[0043] In some embodiments, the execution entity may monitor in real time whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system.

[0044] In practice, the following steps can be used to monitor in real time whether drones appear within the above monitoring range:

[0045] In the first step, in response to a monitoring sensor included in any one of the at least one UAV monitoring devices detecting the presence of a target object, an image of the target object captured by at least one monitoring camera is acquired to obtain a set of target object images.

[0046] In the second step, each target object image in the target object image set is processed by drone recognition to generate a recognition result and obtain a recognition result set. In practice, each target object image in the target object image set can be input into a pre-trained drone recognition model to obtain a recognition result. The drone recognition model can be a pre-trained convolutional neural network model.

[0047] In the third step, in response to the recognition result set satisfying the first preset condition, the target object is determined to be a target drone. The first preset condition may be that the number of recognition results in the recognition result set that represent the recognition of a drone is greater than or equal to a preset recognition number.

[0048] Step 105 , in response to detecting the presence of a drone within the total monitoring range, determining drone state information and drone trajectory information of the drone.

[0049] In some embodiments, the execution entity may determine the drone status information and the drone trajectory information of the drone in response to detecting the presence of a drone within the total monitoring range.

[0050] In practice, when a drone is positioned using only a single sensor, there is a deviation in the signal transmission angle, resulting in the identified drone position not matching the drone's actual position, incorrectly determining the drone's status, and subsequently causing incorrect control of the drone, resulting in a waste of drone resources.

[0051] In some optional implementations of some embodiments, the above-mentioned execution subject may determine the drone state information and drone trajectory information of the drone through the following steps:

[0052] In the first step, in response to detecting the presence of a drone within the above-mentioned monitoring total range, at least one monitoring sensor corresponding to the above-mentioned drone is determined as a target sensor group. The target sensor in the above-mentioned target sensor group may be a monitoring sensor that can detect the above-mentioned drone.

[0053] In the second step, for each target sensor in the target sensor group, the following determination steps are performed:

[0054] The first determining step is to control the target sensor to send a detection signal to the UAV and receive a reflected signal in real time.

[0055] The second determination step is to generate a UAV monitoring position sequence based on the direction of the received reflected signal and the signal transmission time.

[0056] The third step is to select a drone monitoring position sequence that satisfies a second preset condition from the generated drone monitoring position sequences as the target position sequence. The second preset condition may be to select a drone monitoring position sequence whose corresponding target sensor is closest to the drone.

[0057] In the fourth step, according to each drone monitoring position sequence excluding the target position sequence, a first correction process is performed on each target position in the target position sequence to generate a corrected position sequence. The first correction process may be to correct each target position in the target position sequence according to the center point of each drone monitoring position corresponding to the target position.

[0058] The fifth step is to obtain a drone image group corresponding to the above-mentioned drone taken by at least one monitoring camera to obtain a drone image group set.

[0059] Step 6: For each drone image group in the drone image group set, perform the following recognition steps:

[0060] The first recognition step is to recognize at least one environmental feature point of each drone image in each drone image included in the drone image group to generate an environmental feature point group and obtain an environmental feature point group set.

[0061] The second recognition step is to determine the model points corresponding to the above-mentioned UAV image group based on the above-mentioned environmental feature point group set and the preset three-dimensional environmental model.

[0062] The third recognition step is to determine the drone position corresponding to the above model point.

[0063] The fourth identification step is to perform a second correction process on the corrected position corresponding to the drone position according to the drone position to generate a corrected position.

[0064] The seventh step is to combine the generated correction positions into a correction position sequence, and determine the correction position sequence as the UAV trajectory information.

[0065] Step 8: Determine the drone status information based on the above drone trajectory information.

[0066] The above-mentioned first step to the eighth step are an inventive point of an embodiment of the present disclosure, and in combination with the following step "step 107", the technical problem "when positioning the drone only by a single sensor, there is a deviation in the signal transmission angle, resulting in the position of the identified drone not being consistent with the actual position of the drone, which in turn leads to the wrong control of the drone, resulting in a waste of drone resources". The reasons for the waste of drone resources are as follows: when positioning the drone by a single sensor, there is a deviation in the signal transmission angle, resulting in the position of the identified drone not being consistent with the actual position of the drone, which in turn leads to the wrong control of the drone, resulting in a waste of drone resources. If the above factors are solved, the effect of avoiding the waste of drone resources can be achieved. In order to achieve this effect, the present disclosure firstly, in response to monitoring the presence of a drone within the above-mentioned monitoring total range, determines at least one monitoring sensor corresponding to the above-mentioned drone as a target sensor group. Thus, the sensor that can monitor the drone can be determined. Second, for each target sensor in the above-mentioned target sensor group, the following determination steps are performed: real-time control of the above-mentioned target sensor to send a detection signal to the above-mentioned drone, and receive a reflection signal; according to the direction of the received reflection signal and the signal transmission time, a drone monitoring position sequence is generated. Thus, the captured drone position can be determined. Third, a drone monitoring position sequence that meets the second preset condition is selected from each drone monitoring position sequence generated as a target position sequence. Thus, a drone monitoring position sequence that best matches the actual position of the drone can be selected. Fourth, according to each drone monitoring position sequence except the above target position sequence, a first correction process is performed on each target position in the above target position sequence to generate a corrected position sequence. Thus, the captured drone position can be corrected. Fifth, a drone image group corresponding to the above drone captured by at least one monitoring camera is obtained to obtain a drone image group set. Thus, images of the drone in flight can be collected. Sixth, for each drone image group in the above drone image group set, the following recognition steps are performed: identifying at least one environmental feature point of each drone image in each drone image included in the above drone image group to generate an environmental feature point group to obtain an environmental feature point group set; determining the model point corresponding to the above drone image group according to the above environmental feature point group set and the preset three-dimensional environment model; determining the drone position corresponding to the above model point; and performing a second correction process on the corrected position corresponding to the above drone position according to the above drone position to generate a corrected position. Thus, the position of the drone can be corrected again through environmental comparison. Seventh, the generated corrected positions are combined into a corrected position sequence, and the corrected position sequence is determined as the drone trajectory information; the drone status information is determined according to the drone trajectory information. Thus, the current state of the drone can be determined.In combination with step 107, in response to the drone predicted behavior information satisfying the preset behavior condition, a drone control intervention operation is performed on the drone. Thus, erroneous control intervention operations on the drone are avoided, thereby avoiding waste of drone resources.

[0067] Step 106 , inputting the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information.

[0068] In some embodiments, the execution entity may input the drone status information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information.

[0069] Optionally, the above drone behavior prediction model can be trained through the following steps:

[0070] The first step is to obtain a sample set.

[0071] In some embodiments, the execution subject may obtain a sample set, wherein the samples in the sample set include sample drone state information and sample drone trajectory information, and sample drone predicted behavior information corresponding to the sample drone state information and sample drone trajectory information.

[0072] The second step is to select samples from the above sample set.

[0073] In some embodiments, the execution entity may select a sample from the sample set. Here, the execution entity may randomly select a sample from the sample set.

[0074] The third step is to input the above samples into the initial network model to obtain the drone prediction behavior information corresponding to the above samples.

[0075] In some embodiments, the execution subject may input the sample into an initial network model to obtain the drone prediction behavior information corresponding to the sample. The initial neural network may be a classification model that can obtain the drone prediction behavior information based on the drone state information and the drone trajectory information.

[0076] The fourth step is to determine the loss value between the above-mentioned drone prediction behavior information and the sample drone prediction behavior information included in the above-mentioned sample.

[0077] In some embodiments, the execution subject may determine a loss value between the drone prediction behavior information and the sample drone prediction behavior information included in the sample. In practice, the loss value between the drone prediction behavior information and the sample drone prediction behavior information included in the sample may be determined based on a preset loss function. For example, the preset loss function may be a cross entropy loss function.

[0078] In the fifth step, in response to the loss value being greater than or equal to a preset threshold, the network parameters of the initial network model are adjusted.

[0079] In some embodiments, the execution subject may adjust the network parameters of the initial network model in response to the loss value being greater than or equal to a preset threshold. Here, there is no restriction on the setting of the preset threshold. For example, the difference between the loss value and the preset threshold may be calculated to obtain the loss difference. On this basis, the error value is passed forward from the last layer of the model using back propagation, stochastic gradient descent and other methods to adjust the parameters of each layer. Of course, as needed, the network freezing (dropout) method may also be used to keep the network parameters of some layers unchanged and not adjust them. There is no limitation on this.

[0080] Optionally, in response to the loss value being less than the preset threshold, the initial network model is determined as a UAV behavior prediction model.

[0081] In some embodiments, the execution entity may determine the initial network model as a drone behavior prediction model in response to the loss value being less than the preset threshold.

[0082] Step 107 , in response to the drone predicted behavior information satisfying the preset behavior condition, performing a drone control intervention operation on the drone.

[0083] In some embodiments, the execution subject may perform a drone control intervention operation on the drone in response to the drone predicted behavior information satisfying a preset behavior condition. The preset behavior condition may be that the drone predicted behavior information indicates that the drone is flying toward the third monitoring range. The drone control intervention operation may be to shoot down the drone by physical means.

[0084] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the drone behavior monitoring method of some embodiments of the present disclosure, the waste of drone resources is avoided. Specifically, the reason for the waste of drone resources is that the drone behavior is predicted only by the position of the drone, and the characteristics such as the posture of the drone are not identified, resulting in the wrong recognition of the behavior of the drone, which in turn causes the waste of drone resources. Based on this, the drone behavior monitoring method of some embodiments of the present disclosure, first, determines the monitoring range corresponding to each drone monitoring device included in the above-mentioned drone monitoring system, and obtains a monitoring range set; according to the above-mentioned monitoring range set, determines the total monitoring range corresponding to the above-mentioned drone monitoring device. Thus, it can be determined that the range in which the drone monitoring system can monitor the drone. Secondly, according to the preset area division information, the above-mentioned total monitoring range is subjected to area division processing to generate a first monitoring range, a second monitoring range and a third monitoring range. Thus, the monitoring range can be divided into three levels. Then, through at least one drone monitoring device included in the above-mentioned drone monitoring system, real-time monitoring is performed to determine whether a drone appears in the above-mentioned total monitoring range. Thus, it can be determined whether a drone enters the monitoring range. Afterwards, in response to monitoring the presence of a drone in the above-mentioned total monitoring range, the drone state information and drone trajectory information of the drone are determined. In this way, the flight state and flight trajectory of the drone can be determined at the same time. Then, the drone state information and the drone trajectory information are input into the pre-trained drone behavior prediction model to obtain the drone predicted behavior information. In this way, the behavior of the drone can be predicted. Finally, in response to the drone predicted behavior information meeting the preset behavior conditions, the drone control intervention operation is performed on the drone. In this way, control intervention can be performed on drones whose behavior is predicted to be dangerous. Also, because the drone behavior is predicted by features such as flight state and flight trajectory, large deviations in the prediction results are avoided, thereby avoiding waste of drone resources.

[0085] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a drone behavior monitoring device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the drone behavior monitoring device can be specifically applied to various electronic devices.

[0086] like Figure 2As shown, the drone behavior monitoring device 200 of some embodiments includes: a first determination unit 201, a second determination unit 202, an area division unit 203, a monitoring unit 204, a third determination unit 205, an input unit 206 and a drone control intervention unit 207. Among them, the first determination unit 201 is configured to determine the monitoring range corresponding to each drone monitoring device included in the above-mentioned drone monitoring system to obtain a monitoring range set; the second determination unit 202 is configured to determine the total monitoring range corresponding to the above-mentioned drone monitoring device based on the above-mentioned monitoring range set; the area division unit 203 is configured to perform area division processing on the above-mentioned total monitoring range according to preset area division information to generate a first monitoring range, a second monitoring range and a third monitoring range; the monitoring unit 204 is configured to monitor in real time whether a drone appears in the above-mentioned total monitoring range through at least one drone monitoring device included in the above-mentioned drone monitoring system; the third determination unit 205 is configured to determine the drone status information and drone trajectory information of the drone in response to the detection of the presence of a drone in the above-mentioned total monitoring range; the input unit 206 is configured to input the above-mentioned drone status information and the above-mentioned drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; the drone control intervention unit 207 is configured to perform a drone control intervention operation on the above-mentioned drone in response to the above-mentioned drone predicted behavior information meeting the preset behavior condition.

[0087] It is understandable that the units recorded in the drone behavior monitoring device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the drone behavior monitoring device 200 and the units contained therein, and will not be repeated here.

[0088] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0089] like Figure 3As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0090] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0091] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0092] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0093] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0094] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: determines the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set. According to the monitoring range set, determines the total monitoring range corresponding to the drone monitoring device. According to the preset area division information, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range. Through at least one drone monitoring device included in the drone monitoring system, monitor in real time whether a drone appears in the total monitoring range. In response to monitoring the presence of a drone in the total monitoring range, determine the drone state information and drone trajectory information of the drone. Input the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information. In response to the drone predicted behavior information satisfying the preset behavior condition, perform drone control intervention operation on the drone, wherein the preset behavior condition is that the drone predicted behavior information indicates that the drone flies to the third monitoring range.

[0095] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0097] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a first determination unit, a second determination unit, an area division unit, a monitoring unit, a third determination unit, an input unit, and a UAV control intervention unit. Among them, the names of these units do not constitute a limitation on the unit itself in certain cases. For example, the first determination unit may also be described as "a unit for determining the monitoring range corresponding to each UAV monitoring device included in the above-mentioned UAV monitoring system, and obtaining a monitoring range set".

[0098] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0099] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A drone behavior monitoring method, applied to a drone monitoring system, wherein: The drone monitoring system includes at least one drone monitoring device, including: Determine a monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; Determine the total monitoring range corresponding to the UAV monitoring device according to the monitoring range set; According to the preset area division information, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range; Real-time monitoring of whether a drone appears within the total monitoring range by at least one drone monitoring device included in the drone monitoring system; In response to detecting the presence of a drone within the total monitoring range, determining drone state information and drone trajectory information of the drone; Inputting the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; In response to the drone predicted behavior information satisfying a preset behavior condition, a drone control intervention operation is performed on the drone, wherein the preset behavior condition is that the drone predicted behavior information indicates that the drone is flying toward the third monitoring range.

2. The method according to claim 1, wherein: The drone monitoring device in the at least one drone monitoring device comprises: a monitoring camera and a monitoring sensor; And, the determining of the monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set includes: For each drone monitoring device in the at least one drone monitoring device included in the drone monitoring system, the following processing steps are performed: Determining a sensor radius of a monitoring sensor included in the unmanned monitoring device; Obtaining coordinate information of the drone monitoring device; Determine the sensor monitoring range corresponding to the drone monitoring device according to the coordinate information and the sensor radius; Obtaining camera information of a monitoring camera included in the drone monitoring device, wherein the camera information includes a camera installation height and a camera pitch angle; Determine the camera monitoring range corresponding to the drone monitoring device according to the camera installation height and the camera pitch angle included in the camera information; The sensor monitoring range and the camera monitoring range are combined to generate a combined monitoring range as the monitoring range; The generated monitoring ranges are determined as a monitoring range set.

3. The method according to claim 1, wherein: The performing of area division processing on the total monitoring range according to the preset area division information to generate a first monitoring range, a second monitoring range and a third monitoring range includes: Determine a first area range, a second area range, and a third area range according to the first area range, the second area range, and the third area range included in the preset area division information, wherein the first area range is a drone identification area, the second area range is a drone early warning area, and the third area range is a drone control area; According to the first area range, the second area range and the third area range, the total monitoring range is divided into regions to generate a first monitoring range, a second monitoring range and a third monitoring range.

4. The method according to claim 1, wherein: The at least one drone monitoring device included in the drone monitoring system monitors in real time whether a drone appears within the total monitoring range, including: In response to a monitoring sensor included in any one of the at least one unmanned aerial vehicle monitoring devices detecting the presence of a target object, acquiring an image of the target object taken by at least one monitoring camera to obtain a set of target object images; Performing drone recognition processing on each target object image in the target object image set to generate a recognition result, thereby obtaining a recognition result set; In response to the recognition result set satisfying a first preset condition, the target object is determined to be a target drone.

5. The method according to claim 1, wherein: The drone behavior prediction model is trained by the following steps: Acquire a sample set, wherein the samples in the sample set include sample drone state information and sample drone trajectory information, and sample drone predicted behavior information corresponding to the sample drone state information and the sample drone trajectory information; Selecting a sample from the sample set; Inputting the sample into the initial network model to obtain the predicted behavior information of the drone corresponding to the sample; Determine a loss value between the drone prediction behavior information corresponding to the sample and the sample drone prediction behavior information included in the sample; In response to the loss value being greater than or equal to a preset threshold, adjusting the network parameters of the initial network model.

6. The method according to claim 5, wherein: The method further comprises: In response to the loss value being less than the preset threshold, the initial network model is determined as a drone behavior prediction model.

7. A drone behavior monitoring device, comprising: A first determining unit is configured to determine a monitoring range corresponding to each drone monitoring device included in the drone monitoring system to obtain a monitoring range set; A second determination unit is configured to determine a total monitoring range corresponding to the drone monitoring device according to the monitoring range set; A region division unit is configured to perform region division processing on the total monitoring range according to preset region division information to generate a first monitoring range, a second monitoring range and a third monitoring range; A monitoring unit configured to monitor in real time whether a drone appears within the total monitoring range through at least one drone monitoring device included in the drone monitoring system; a third determining unit, configured to determine drone state information and drone trajectory information of the drone in response to detecting the presence of a drone within the total monitoring range; An input unit, configured to input the drone state information and the drone trajectory information into a pre-trained drone behavior prediction model to obtain drone predicted behavior information; The drone control intervention unit is configured to perform a drone control intervention operation on the drone in response to the drone predicted behavior information satisfying a preset behavior condition.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images