Unmanned aerial vehicle control method and device based on unmanned aerial vehicle vision, equipment and medium

Abstract

The embodiment of the invention discloses an unmanned aerial vehicle control method and device based on unmanned aerial vehicle vision, equipment and a medium. A specific embodiment of the method comprises the following steps: controlling a signal transmitting device to transmit a signal in real time; controlling a recognition device to perform recognition processing on the unknown flyer to generate a recognition result; controlling the monitoring unmanned aerial vehicles in the unmanned aerial vehicle group to perform image tracking processing on the unknown unmanned aerial vehicles; carrying out positioning processing on the unknown unmanned aerial vehicle; generating a second unknown unmanned aerial vehicle trajectory, and correcting the first unknown unmanned aerial vehicle trajectory according to the second unknown unmanned aerial vehicle trajectory to obtain a corrected unmanned aerial vehicle trajectory; performing trajectory prediction processing on the unknown unmanned aerial vehicle to generate an unmanned aerial vehicle prediction trajectory; and controlling the unmanned aerial vehicle group or the intercepting device to intercept unknown unmanned aerial vehicles. According to the embodiment, the area safety of the detection area is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a drone control method, device, equipment, and medium based on drone vision. Background Art

[0002] With the continuous development of UAV technology, how to control unknown UAVs has become an important research topic. At present, when controlling unknown UAVs, the commonly used method is to control the unknown UAVs through electromagnetic interference.

[0003] However, when using the above method to control an unknown drone, the following technical problems often occur:

[0004] Unknown drones may have been illegally modified (for example, adding a signal anti-interference module), and electromagnetic interference cannot be used to effectively control the drone, resulting in low regional security in the detection area.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a drone control method, device, electronic device, and computer-readable medium based on drone vision to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a drone control method based on drone vision, the method comprising: controlling the signal transmitting device to transmit a signal in real time to monitor an unregistered drone; in response to receiving a reflection signal indicating that an unknown flying object has been monitored, controlling the identification device to identify the unknown flying object to generate an identification result; in response to the identification result indicating that the unknown flying object is an unknown drone, controlling a monitoring drone in the drone group to perform image tracking processing on the unknown drone to generate an unknown drone image sequence; positioning the unknown drone according to the reflection signal received in real time to generate a first unknown drone trajectory; generating a second unknown drone trajectory according to the positioning of the monitoring drone and the unknown drone image sequence, and correcting the first unknown drone trajectory according to the second unknown drone trajectory to obtain a corrected drone trajectory; performing trajectory prediction processing on the unknown drone according to the corrected drone trajectory to generate a predicted drone trajectory; controlling the drone group or the interception device to intercept the unknown drone according to the predicted drone trajectory.

[0009] In a second aspect, some embodiments of the present disclosure provide a drone control device based on drone vision, the device comprising: a first control unit, configured to control the signal transmitting device to transmit a signal in real time to monitor an unregistered drone; a second control unit, configured to control the identification device to perform identification processing on the unknown flying object in response to receiving a reflection signal representing that an unknown flying object has been monitored, so as to generate an identification result; a third control unit, configured to control a monitoring drone in the drone group to perform image tracking processing on the unknown drone in response to the recognition result representing that the unknown flying object is an unknown drone, so as to generate an unknown drone image sequence; a positioning unit, configured to The unknown UAV is positioned based on the reflected signal received in real time to generate a first unknown UAV trajectory; a generation unit is configured to generate a second unknown UAV trajectory based on the positioning of the monitoring UAV and the unknown UAV image sequence, and to correct the first unknown UAV trajectory based on the second unknown UAV trajectory to obtain a corrected UAV trajectory; a trajectory prediction unit is configured to perform trajectory prediction on the unknown UAV based on the corrected UAV trajectory to generate a predicted UAV trajectory; a fourth control unit is configured to control the UAV swarm or the interception device based on the predicted UAV trajectory to intercept the unknown UAV.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the drone control method based on drone vision of some embodiments of the present disclosure, the regional security of the detection area is improved. Specifically, the reason for the low regional security of the detection area is that the unknown drone may be illegally modified (for example, adding a signal anti-interference module), and the use of electromagnetic interference cannot effectively control the drone, resulting in low regional security of the detection area. Based on this, the drone control method based on drone vision of some embodiments of the present disclosure, first, controls the above-mentioned signal transmitting device to transmit a signal in real time to monitor unregistered drones. Thus, it is possible to detect in real time whether there are unknown drones in the detection area. Secondly, in response to receiving a reflection signal representing the monitoring of an unknown flying object, the above-mentioned identification device is controlled to identify the above-mentioned unknown flying object to generate an identification result. Then, in response to the above-mentioned identification result representing that the above-mentioned unknown flying object is an unknown drone, the monitoring drone in the above-mentioned drone group is controlled to perform image tracking processing on the above-mentioned unknown drone to generate an unknown drone image sequence. Thus, the unknown drone can be tracked and photographed by the associated drone to obtain a clearer image of the unknown drone. Afterwards, the unknown drone is positioned based on the reflected signal received in real time to generate a first unknown drone trajectory; the second unknown drone trajectory is generated based on the positioning of the monitoring drone and the unknown drone image sequence, and the first unknown drone trajectory is corrected based on the second unknown drone trajectory to obtain a corrected drone trajectory. Thus, the corrected drone flight trajectory can be determined. Then, based on the corrected drone trajectory, the unknown drone is subjected to trajectory prediction processing to generate a predicted drone trajectory. Thus, the trajectory of the unknown drone can be predicted based on the flight trajectory of the unknown drone. Finally, based on the predicted drone trajectory, the drone group or the interception device is controlled to intercept the unknown drone. Thus, the unknown drone can be intercepted, thereby improving the regional security of the detection area. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the drone control method based on drone vision according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of a drone control device based on drone vision according to the present disclosure;

[0016] Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure;

[0017] Figure 4 is a schematic diagram of a drone interception system according to some embodiments of the drone control method based on drone vision of the present disclosure;

[0018] Figure 5 It is a trajectory display test diagram according to some embodiments of the drone control method based on drone vision of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1 The flowchart 100 of some embodiments of the drone control method based on drone vision according to the present disclosure is shown. The drone control method based on drone vision includes the following steps:

[0026] Step 101, controlling the signal transmitting device to transmit signals in real time to monitor unregistered drones.

[0027] In some embodiments, the executing entity (e.g., a server) of the drone control method based on drone vision can control the above-mentioned signal transmitting device to transmit signals in real time to monitor unregistered drones. Among them, the above-mentioned drone control method based on drone vision can be applied to a drone control system. The above-mentioned drone control system may include a signal transmitting device, a drone swarm, an identification device, and an interception device. The above-mentioned signal transmitting device may be a detection radar. The above-mentioned drone swarm may be a drone swarm connected to the above-mentioned drone control system by wireless. The above-mentioned identification device may be a terminal with a shooting function, which is used to identify the photographed object. The above-mentioned interception device may be used to intercept objects in the air. Figure 4 FIG. 1 shows a schematic diagram of a drone interception system deployed in a server. Figure 4 As shown in the figure, this system platform adopts C / S structure design, can load GIS high-definition map, and has functions such as multi-target intrusion alarm, target precise positioning, display of target movement trajectory and flight control hand position, etc.

[0028] Step 102, in response to receiving a reflection signal indicating that an unknown flying object has been detected, controlling the identification device to perform identification processing on the unknown flying object to generate an identification result.

[0029] In some embodiments, the execution entity may control the identification device to perform identification processing on the unknown flying object in response to receiving a reflection signal indicating that an unknown flying object has been detected, so as to generate an identification result.

[0030] In practice, the following steps can be used to control the identification device to identify the unknown flying object:

[0031] The first step is to control the recognition device to photograph the unknown flying object to generate a photographed image sequence, wherein the photographing process may be to photograph the unknown flying object at intervals of a preset time.

[0032] In the second step, the recognition model inputs each captured image in the captured image sequence into a pre-trained drone recognition model to obtain a recognition result. The drone recognition model may be a pre-trained model for identifying drones. For example, the drone recognition model may be a classification model trained using a preset data set including drone images.

[0033] In the process of adopting technical solutions to solve the above-mentioned technical problems, the following technical problems often occur: when identifying whether a flying object is a drone, due to the small size of the flying object and the presence of a certain distance, the flying object cannot be accurately identified, resulting in the unknown drone being identified as other objects, which in turn leads to low regional security in the detection area.

[0034] In some optional implementations of some embodiments, the execution subject may identify the unknown flying object through the following steps:

[0035] In the first step, based on the captured image sequence, the following processing steps are performed:

[0036] The first processing step is to perform moving target recognition processing on the first captured image in the captured image sequence to generate a captured foreground target image. The moving target recognition processing may be to determine the moving foreground target through the ViBe algorithm.

[0037] The second processing step is to perform contour detection on the moving target displayed in the foreground target image to generate a detected target image. Here, the contour detection of the moving target can be performed using a contour detection algorithm based on OpenCV.

[0038] The third processing step is to determine the position of the moving target in the first captured image based on the detected target image to obtain image position information.

[0039] The fourth processing step is to segment the first captured image according to the image position information to generate a segmented captured image. In practice, the first captured image can be segmented with the position represented by the image position information as the center and a preset length as the radius to generate a segmented captured image.

[0040] The fifth processing step is to perform feature extraction processing on the segmented image to generate a moving target feature vector.

[0041] In practice, the following sub-steps can be used to perform feature extraction processing on the segmented image to generate a moving target feature vector:

[0042] The first sub-step is to perform gamma correction on the segmented captured image to generate a corrected captured image. Here, before performing gamma correction, the image size of the segmented captured image needs to be adjusted.

[0043] The second sub-step is to determine the gradient information of each pixel in the corrected captured image to obtain a gradient information set, wherein the gradient information is used to characterize the contour shape of the local image.

[0044] The third sub-step is to combine the pixels in the corrected captured image to generate at least one pixel unit.

[0045] The fourth sub-step is to determine a feature descriptor of each pixel unit in at least one pixel unit according to the above-mentioned gradient information set.

[0046] The fifth sub-step is to combine each preset number of pixel units in at least one pixel unit to generate at least one pixel block, and to connect in series the feature descriptors corresponding to the pixel units included in each pixel block to generate a combined feature descriptor.

[0047] The sixth sub-step is to concatenate the generated combined feature descriptors into a motion target feature vector.

[0048] The sixth processing step is to input the moving target feature vector into a pre-trained moving target recognition model to obtain a recognition result. The moving target recognition model may be a pre-trained nonlinear SVM model.

[0049] In the second step, in response to not identifying a moving target in the first captured image and the captured image sequence after deleting the first captured image is not empty, the captured image sequence after deleting the first captured image is used as the captured image sequence and the above processing steps are performed again.

[0050] The above-mentioned first step and second step are an inventive point of an embodiment of the present disclosure, and in combination with the following step "step 107", the technical problem "when identifying whether a flying object is a drone, the flying object cannot be accurately identified due to its small size and a certain distance, resulting in the unknown drone being identified as other objects, which in turn leads to low regional security in the detection area". The reasons for the low regional security of the detection area are as follows: when identifying whether a flying object is a drone, the flying object cannot be accurately identified due to its small size and a certain distance, resulting in the unknown drone being identified as other objects, which in turn leads to low regional security in the detection area. If the above factors are solved, the effect of improving the regional security of the detection area can be achieved. In order to achieve this effect, the present disclosure firstly performs the following processing steps based on the captured image sequence: the first captured image in the captured image sequence is processed for motion target recognition to generate a captured foreground target image. Thus, it can be determined whether the unknown flying object is in motion. Second, the moving target displayed in the captured foreground target image is processed for contour detection to generate a detected target image. Thus, the contour of the moving target can be determined. Third, according to the above-mentioned detected target image, the position of the above-mentioned moving target in the first captured image is determined to obtain the image position information. Thus, the position of the moving target in the captured image can be determined. Fourth, according to the above-mentioned image position information, the first captured image is segmented to generate a segmented captured image; the segmented captured image is subjected to feature extraction processing to generate a moving target feature vector. Thus, the feature information of the segmented image can be extracted, and the recognition range can be narrowed to the sub-image area, thereby improving the recognition speed and recognition accuracy. In addition, it can avoid identifying unknown drones as other objects, thereby improving the regional security of the detection area. Fifth, the segmented captured image is subjected to gamma correction processing to generate a corrected captured image; the gradient information of each pixel in the corrected captured image is determined to obtain a gradient information set. Thus, the feature information of the local image can be determined. Sixth, combine and process each pixel point in the corrected captured image to generate at least one pixel unit; determine the feature descriptor of each pixel unit in the at least one pixel unit according to the above-mentioned gradient information set; combine and process each preset number of pixel units in the at least one pixel unit to generate at least one pixel block, and connect the feature descriptors corresponding to each pixel unit included in each pixel block in series to generate a combined feature descriptor; connect the generated combined feature descriptors in series to form a moving target feature vector. In this way, a feature vector of an unknown flying object can be generated.Seventh, input the moving target feature vector into the pre-trained moving target recognition model to obtain the recognition result; in response to not recognizing the presence of a moving target in the first captured image, and the captured image sequence after deleting the first captured image is not empty, the captured image sequence after deleting the first captured image is used as the captured image sequence, and the above processing steps are performed again. In this way, the recognition process of the position flying object is completed, and because the recognition range of the unknown flying object is narrowed, the recognition speed and recognition accuracy are improved, thereby avoiding the unknown drone from being identified as other objects, and improving the regional safety of the detection area.

[0051] Step 103, in response to the above recognition result characterizing that the above unknown flying object is an unknown drone, controlling the monitoring drone in the above drone group to perform image tracking processing on the above unknown drone to generate an unknown drone image sequence.

[0052] In some embodiments, the execution subject may, in response to the recognition result indicating that the unknown flying object is an unknown drone, control the monitoring drone in the drone group to perform image tracking processing on the unknown drone to generate an unknown drone image sequence. The image tracking processing may be controlling the shooting device installed on the monitoring drone to perform real-time tracking and shooting of the unknown drone.

[0053] Step 104: Position the unknown UAV according to the reflected signal received in real time to generate a first unknown UAV trajectory.

[0054] In some embodiments, the execution subject may perform positioning processing on the unknown UAV according to the reflected signal received in real time to generate a first unknown UAV trajectory. The reflected signal may be a signal reflected back by the transmission signal of the detection radar after detecting the unknown UAV. Figure 5 The trajectory display test diagram shown can display the generated first unknown UAV trajectory in the system to facilitate observation of the UAV trajectory.

[0055] In practice, the execution subject may perform positioning processing on the unknown UAV through the following steps to generate a first unknown UAV trajectory:

[0056] The first step is to obtain a preset real-scene three-dimensional model. The preset real-scene three-dimensional model is a three-dimensional model of the signal transmission range corresponding to the signal transmission device. The signal transmission range can be the range of the signal that the signal transmission device can transmit.

[0057] The second step is to select at least one reflection signal from each reflection signal received in real time according to a preset time interval to obtain a reflection signal sequence. The preset time interval may be a preset time interval between selecting every two reflection signals. For example, the preset time interval may be 1 second.

[0058] In the third step, for each reflection signal in the reflection signal sequence, the position represented by the reflection signal is marked in the preset real-scene 3D model. Here, for each reflection signal in the reflection signal sequence, the coordinates and height represented by the reflection signal can be marked in the preset real-scene 3D model.

[0059] The fourth step is to generate a first unknown UAV trajectory according to each position marked in the above-mentioned preset real-scene three-dimensional model.

[0060] Step 105: generating a second unknown drone trajectory according to the positioning of the monitoring drone and the unknown drone image sequence, and correcting the first unknown drone trajectory according to the second unknown drone trajectory to obtain a corrected drone trajectory.

[0061] In some embodiments, the execution entity may generate a second unknown drone trajectory according to the positioning of the monitoring drone and the unknown drone image sequence, and perform correction processing on the first unknown drone trajectory according to the second unknown drone trajectory to obtain a corrected drone trajectory.

[0062] Step 106: perform trajectory prediction processing on the unknown UAV according to the corrected UAV trajectory to generate a predicted UAV trajectory.

[0063] In some embodiments, the execution entity may perform trajectory prediction processing on the unknown UAV according to the corrected UAV trajectory to generate a predicted UAV trajectory.

[0064] In practice, the above-mentioned execution subject can perform trajectory prediction processing on the above-mentioned unknown UAV through the following steps:

[0065] The first step is to determine a drone trajectory point set based on the above-mentioned corrected drone trajectory. In practice, a preset number of drone trajectory points can be selected from the above-mentioned corrected drone trajectory, and the above-mentioned preset number of drone trajectory points can be combined into a drone trajectory point set. Here, the above-mentioned execution subject can select drone trajectory points from the above-mentioned corrected drone trajectory at intervals of a preset time step to obtain a drone trajectory point set. The above-mentioned preset time step can be a pre-set time step of the above-mentioned unknown drone flight.

[0066] In the second step, for each drone trajectory point in the drone trajectory point set, determine the drone speed and drone displacement corresponding to the drone trajectory point. The drone speed may be the speed of the unknown drone at the drone trajectory point. The drone displacement may be the displacement of the unknown drone from the previous drone trajectory point to the current drone trajectory point.

[0067] The third step is to generate a drone speed sequence and a drone displacement sequence according to the determined drone speeds and drone displacements. In practice, the above execution entity can sort the drone speeds and drone displacements in chronological order according to the time of the drone trajectory points corresponding to the drone speeds and drone displacements to generate a drone speed sequence and a drone displacement sequence.

[0068] The fourth step is to normalize the above-mentioned UAV speed sequence and the above-mentioned UAV displacement sequence to generate a normalized UAV speed sequence and a normalized UAV displacement sequence.

[0069] The fifth step is to input the above normalized UAV velocity sequence and the above normalized UAV displacement sequence into the pre-trained UAV trajectory prediction model to obtain the UAV predicted trajectory.

[0070] Optionally, the above-mentioned drone trajectory prediction model can be trained through the following steps:

[0071] The first step is to obtain a sample set.

[0072] In some embodiments, the execution subject may obtain a sample set, wherein the samples in the sample set include a sample normalized drone speed sequence and a sample normalized drone displacement sequence, and a sample drone predicted trajectory corresponding to the sample normalized drone speed sequence and the sample normalized drone displacement sequence.

[0073] The second step is to select samples from the above sample set.

[0074] In some embodiments, the execution entity may select a sample from the sample set. Here, the execution entity may randomly select a sample from the sample set.

[0075] The third step is to input the above samples into the initial network model to obtain the predicted trajectory of the UAV corresponding to the above samples.

[0076] In some embodiments, the execution subject may input the sample into an initial network model to obtain a predicted trajectory of the drone corresponding to the sample. The initial neural network may be a generative model that can obtain the predicted trajectory of the drone based on a normalized drone velocity sequence and a normalized drone displacement sequence.

[0077] The fourth step is to determine the loss value between the predicted trajectory of the above-mentioned UAV and the predicted trajectory of the sample UAV included in the above-mentioned sample.

[0078] In some embodiments, the execution subject may determine a loss value between the predicted trajectory of the drone and the predicted trajectory of the sample drone included in the sample. In practice, the loss value between the predicted trajectory of the drone and the predicted trajectory of the sample drone included in the sample may be determined based on a preset loss function. For example, the preset loss function may be a cross entropy loss function.

[0079] In the fifth step, in response to the loss value being greater than or equal to a preset threshold, the network parameters of the initial network model are adjusted.

[0080] In some embodiments, the execution subject may adjust the network parameters of the initial network model in response to the loss value being greater than or equal to a preset threshold. Here, there is no restriction on the setting of the preset threshold. For example, the difference between the loss value and the preset threshold may be calculated to obtain the loss difference. On this basis, the error value is transferred from the last layer of the model to the front by back propagation, stochastic gradient descent and other methods to adjust the parameters of each layer. Of course, as needed, the network freezing (dropout) method may also be used to keep the network parameters of some layers unchanged without adjustment, and no limitation is made to this.

[0081] Optionally, in response to the loss value being less than the preset threshold, the initial network model is determined as a UAV trajectory prediction model.

[0082] In some embodiments, the execution entity may determine the initial network model as a UAV trajectory prediction model in response to the loss value being less than the preset threshold.

[0083] Step 107, based on the predicted trajectory of the drone, control the drone group or the interception device to intercept the unknown drone.

[0084] In some embodiments, the above-mentioned execution entity can control the above-mentioned drone group or the above-mentioned interception device according to the predicted trajectory of the above-mentioned drone to intercept the above-mentioned unknown drone.

[0085] In practice, the above-mentioned execution entity can intercept the above-mentioned unknown drone through the following steps:

[0086] The first step is to predict the size of the unknown drone based on the unknown drone image sequence to obtain a size prediction result. In practice, each unknown drone image in the unknown drone image sequence can be input into a pre-trained size prediction model to obtain a prediction result. Then, the average of the obtained prediction results is determined as the size prediction result.

[0087] The second step is to determine whether the predicted trajectory of the drone meets a preset trajectory condition. The preset trajectory condition may be that the predicted trajectory of the drone indicates that the unknown drone flies into the interception range corresponding to the interception device.

[0088] The third step is to control the interception device included in the UAV control system to perform a catapult interception operation on the unknown UAV in response to the UAV predicted trajectory satisfying the preset trajectory condition, according to the UAV predicted trajectory. The catapult interception operation may be to launch an interception net at the unknown UAV through the interception device.

[0089] Step 4: In response to the predicted trajectory of the drone not satisfying the preset trajectory condition, the drone group is controlled to dynamically intercept the unknown drone according to the size prediction result and the predicted trajectory of the drone. Here, in response to the size prediction result being greater than the preset drone interception size,

[0090] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the drone control method based on drone vision of some embodiments of the present disclosure, the regional security of the detection area is improved. Specifically, the reason for the low regional security of the detection area is that the unknown drone may be illegally modified (for example, adding a signal anti-interference module), and the use of electromagnetic interference cannot effectively control the drone, resulting in low regional security of the detection area. Based on this, the drone control method based on drone vision of some embodiments of the present disclosure, first, controls the above-mentioned signal transmitting device to transmit a signal in real time to monitor unregistered drones. Thus, it is possible to detect in real time whether there are unknown drones in the detection area. Secondly, in response to receiving a reflection signal representing the monitoring of an unknown flying object, the above-mentioned identification device is controlled to identify the above-mentioned unknown flying object to generate an identification result. Then, in response to the above-mentioned identification result representing that the above-mentioned unknown flying object is an unknown drone, the monitoring drone in the above-mentioned drone group is controlled to perform image tracking processing on the above-mentioned unknown drone to generate an unknown drone image sequence. Thus, the unknown drone can be tracked and photographed by the associated drone to obtain a clearer image of the unknown drone. Afterwards, the unknown drone is positioned based on the reflected signal received in real time to generate a first unknown drone trajectory; the second unknown drone trajectory is generated based on the positioning of the monitoring drone and the unknown drone image sequence, and the first unknown drone trajectory is corrected based on the second unknown drone trajectory to obtain a corrected drone trajectory. Thus, the corrected drone flight trajectory can be determined. Then, based on the corrected drone trajectory, the unknown drone is subjected to trajectory prediction processing to generate a predicted drone trajectory. Thus, the trajectory of the unknown drone can be predicted based on the flight trajectory of the unknown drone. Finally, based on the predicted drone trajectory, the drone group or the interception device is controlled to intercept the unknown drone. Thus, the unknown drone can be intercepted, thereby improving the regional security of the detection area.

[0091] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a drone control device based on drone vision. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the drone control device based on drone vision can be specifically applied to various electronic devices.

[0092] like Figure 2As shown, some embodiments of the drone control device 200 based on drone vision include: a first control unit 201, a second control unit 202, a third control unit 203, a positioning unit 204, a generation unit 205, a trajectory prediction unit 206 and a fourth control unit 207. The first control unit 201 is configured to control the signal transmitting device to transmit signals in real time to monitor unregistered drones; the second control unit 202 is configured to control the identification device to perform identification processing on the unknown flying object in response to receiving a reflection signal representing the monitoring of an unknown flying object to generate an identification result; the third control unit 203 is configured to control the monitoring drone in the drone group to perform image tracking processing on the unknown drone in response to the recognition result representing that the unknown flying object is an unknown drone to generate an unknown drone image sequence; the positioning unit 204 is configured to track the unknown drone according to the reflection signal received in real time. The UAV is positioned to generate a first unknown UAV trajectory; the generation unit 205 is configured to generate a second unknown UAV trajectory according to the positioning of the above-mentioned monitoring UAV and the above-mentioned unknown UAV image sequence, and to correct the above-mentioned first unknown UAV trajectory according to the above-mentioned second unknown UAV trajectory to obtain a corrected UAV trajectory; the trajectory prediction unit 206 is configured to perform trajectory prediction processing on the above-mentioned unknown UAV according to the above-mentioned corrected UAV trajectory to generate a UAV predicted trajectory; the fourth control unit 207 is configured to control the above-mentioned UAV group or the above-mentioned interception device according to the above-mentioned UAV predicted trajectory to intercept the above-mentioned unknown UAV.

[0093] It can be understood that the units recorded in the drone control device 200 based on drone vision are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the drone control device 200 based on drone vision and the units contained therein, and will not be repeated here.

[0094] Reference below Figure 3 , which shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0095] like Figure 3As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0096] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0097] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0098] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0099] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0100] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: controls the signal transmitting device to transmit signals in real time to monitor unregistered drones. In response to receiving a reflection signal indicating that an unknown flying object has been monitored, controls the identification device to perform identification processing on the unknown flying object to generate an identification result. In response to the identification result indicating that the unknown flying object is an unknown drone, controls the monitoring drone in the drone group to perform image tracking processing on the unknown drone to generate an unknown drone image sequence. According to the reflection signal received in real time, the unknown drone is positioned to generate a first unknown drone trajectory. According to the positioning of the monitoring drone and the unknown drone image sequence, a second unknown drone trajectory is generated, and according to the second unknown drone trajectory, the first unknown drone trajectory is corrected to obtain a corrected drone trajectory. According to the corrected drone trajectory, the unknown drone is subjected to trajectory prediction processing to generate a drone prediction trajectory. According to the predicted trajectory of the above-mentioned drone, the above-mentioned drone group or the above-mentioned interception device is controlled to intercept the above-mentioned unknown drone.

[0101] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising a first control unit, a second control unit, a third control unit, a positioning unit, a generation unit, a trajectory prediction unit and a fourth control unit. Among them, the names of these units do not constitute a limitation on the units themselves in certain circumstances, for example, the first control unit may also be described as "a unit that controls the above-mentioned signal transmitting device to transmit signals in real time to monitor unregistered drones".

[0104] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0105] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A drone control method based on drone vision, applied to a drone control system, wherein the drone control system includes a signal transmitting device, a drone group and an identification device, and the method includes: Controlling the signal transmitting device to transmit signals in real time to monitor unregistered drones; In response to receiving a reflection signal indicating that an unknown flying object has been detected, controlling the identification device to perform identification processing on the unknown flying object to generate an identification result; In response to the identification result indicating that the unknown flying object is an unknown drone, controlling a monitoring drone in the drone group to perform image tracking processing on the unknown drone to generate an unknown drone image sequence; Performing positioning processing on the unknown UAV according to the reflected signal received in real time to generate a first unknown UAV trajectory; Generate a second unknown drone trajectory according to the positioning of the monitoring drone and the unknown drone image sequence, and perform correction processing on the first unknown drone trajectory according to the second unknown drone trajectory to obtain a corrected drone trajectory; According to the corrected UAV trajectory, performing trajectory prediction processing on the unknown UAV to generate a predicted UAV trajectory; According to the predicted trajectory of the drone, the drone group or the interception device is controlled to intercept the unknown drone.

2. The method according to claim 1, wherein: In response to receiving the reflection signal indicating that the unknown flying object has been detected, controlling the identification device to perform identification processing on the unknown flying object to generate an identification result, including: Controlling the recognition device to photograph the unknown flying object to generate a photographed image sequence; Each captured image in the captured image sequence is input into a pre-trained drone recognition model to obtain a recognition result.

3. The method according to claim 1, wherein: The method of performing positioning processing on the unknown UAV according to the reflected signal received in real time to generate a first unknown UAV trajectory includes: Acquire a preset real-scene three-dimensional model, wherein the preset real-scene three-dimensional model is a three-dimensional model of a signal transmission range corresponding to the signal transmitting device; Selecting at least one reflection signal from each reflection signal received in real time according to a preset time interval to obtain a reflection signal sequence; For each reflection signal in the reflection signal sequence, marking a position represented by the reflection signal in the preset real-scene three-dimensional model; A first unknown UAV trajectory is generated according to each position marked in the preset real-scene three-dimensional model.

4. The method according to claim 1, wherein: The step of performing trajectory prediction processing on the unknown UAV according to the corrected UAV trajectory to generate a predicted UAV trajectory includes: Determining a set of drone trajectory points based on the corrected drone trajectory; For each drone trajectory point in the drone trajectory point set, determining a drone speed and a drone displacement corresponding to the drone trajectory point; Generate a UAV speed sequence and a UAV displacement sequence according to the determined UAV speeds and UAV displacements; Normalizing the UAV speed sequence and the UAV displacement sequence to generate a normalized UAV speed sequence and a normalized UAV displacement sequence; The normalized UAV velocity sequence and the normalized UAV displacement sequence are input into a pre-trained UAV trajectory prediction model to obtain a UAV predicted trajectory.

5. The method according to claim 4, wherein: The UAV trajectory prediction model is trained by the following steps: Acquire a sample set, wherein the samples in the sample set include a sample normalized UAV velocity sequence and a sample normalized UAV displacement sequence, and a sample UAV predicted trajectory corresponding to the sample normalized UAV velocity sequence and the sample normalized UAV displacement sequence; Selecting a sample from the sample set; Inputting the sample into the initial network model to obtain the predicted trajectory of the drone corresponding to the sample; Determine a loss value between a predicted trajectory of a drone corresponding to the sample and a predicted trajectory of a sample drone included in the sample; In response to the loss value being greater than or equal to a preset threshold, adjusting the network parameters of the initial network model.

6. The method according to claim 5, wherein: The method further comprises: In response to the loss value being less than the preset threshold, the initial network model is determined as a UAV trajectory prediction model.

7. The method according to claim 1, wherein: The drone control method further includes an interception device; and The controlling the drone group or the interception device to intercept the unknown drone according to the predicted trajectory of the drone includes: According to the unknown UAV image sequence, a size prediction is performed on the unknown UAV to obtain a size prediction result; Determining whether the predicted trajectory of the drone meets a preset trajectory condition; In response to the predicted trajectory of the drone satisfying the preset trajectory condition, controlling the interception device included in the drone control system to perform a ejection interception operation on the unknown drone according to the predicted trajectory of the drone; In response to the predicted trajectory of the drone not satisfying the preset trajectory condition, the drone group is controlled to perform a dynamic interception operation on the unknown drone according to the size prediction result and the predicted trajectory of the drone.

8. A drone control device based on drone vision, comprising: A first control unit is configured to control the signal transmitting device to transmit a signal in real time to monitor unregistered drones; A second control unit is configured to control the identification device to perform identification processing on the unknown flying object in response to receiving a reflection signal indicating that the unknown flying object has been detected, so as to generate an identification result; A third control unit is configured to control a monitoring drone in the drone group to perform image tracking processing on the unknown drone in response to the recognition result indicating that the unknown flying object is an unknown drone, so as to generate an unknown drone image sequence; A positioning unit is configured to perform positioning processing on the unknown UAV according to the reflected signal received in real time to generate a first unknown UAV trajectory; a generating unit configured to generate a second unknown drone trajectory according to the positioning of the monitoring drone and the unknown drone image sequence, and to perform correction processing on the first unknown drone trajectory according to the second unknown drone trajectory to obtain a corrected drone trajectory; A trajectory prediction unit is configured to perform trajectory prediction processing on the unknown UAV according to the corrected UAV trajectory to generate a predicted UAV trajectory; The fourth control unit is configured to control the drone group or the interception device to intercept the unknown drone according to the predicted trajectory of the drone.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images