Systems and methods for countering unmanned aerial vehicles

Abstract

The present invention relates to a system and method for countering unmanned aerial vehicles (UAVs), the method comprising: detecting unknown flying objects in a monitored area of ​​airspace; capturing images of the detected unknown flying objects; analyzing the captured images to classify the detected unknown flying objects; and determining whether the detected unknown flying objects include an unmanned aerial vehicle based on the analyzed images. In response to determining that the detected unknown flying objects include a UAV, suppressing one or more radio signals exchanged between the UAV and a user of the UAV until the UAV leaves the monitored area of ​​airspace.

Description

Technical Field

[0001] The present invention relates to the field of countering unmanned aerial vehicles (UAVs), and more particularly, to systems and methods for countering UAVs. Background Art

[0002] An unmanned vehicle is a vehicle equipped with an automated control system that allows the unmanned vehicle to move without active and sustained human intervention. Such unmanned vehicles may include, but are not limited to, unmanned ground transporters (e.g., cars), self-propelled robots (e.g., lunar rovers), and unmanned aerial vehicles (e.g., quadcopters). The rapid development of complex unmanned (robotic) systems and devices requires new solutions for defending against them.

[0003] More specifically, UAVs (also known as drones) are becoming more convenient for individuals to purchase, resulting in a significant increase in their use by the general public. Such UAVs represent civilian UAVs. In the case of improper operation, seemingly safe commercial and civilian (private) use of UAVs may bring many dangers. Such use may bring risks to life and property. In addition, UAVs may be used to infringe on the sanctity of business, education, sports, entertainment and government existence. UAVs (such as drones) can also be used to invade privacy or carry out terrorist and criminal activities. For example, UAVs may fly into residential areas, carry explosives, or, for example, deliver contraband to prisoners by flying above the prison ground. Therefore, there is a real need for an integrated system and method for detecting, tracking, identifying / classifying and blocking UAVs, especially civilian UAVs. Examples of civilian UAVs include, but are not limited to, drones of these companies such as DJI, FREEFLY, PARROT and XIAOMI.

[0004] One of the solutions to the aforementioned problems is to use systems that use radio frequency detection (radar). However, this solution has insurmountable difficulties in detecting and identifying small flying objects. Due to the small size of UAVs (small unmanned aerial vehicles, SUAVs) and their ability to fly at different altitudes and speeds, radar systems are generally unable to ensure the necessary level of detection of flying objects and their subsequent identification.

[0005] Another traditional solution to the above problems involves sensors, such as light identification, detection and ranging (LIDAR) devices. Compared with radar, LIDAR devices have the advantage that they provide more accurate position determination and have a smaller spot size, thus forming a more accurate image of the target. LIDAR is a device designed to detect, identify and determine the range of objects using light reflections.

[0006] It should be noted that civilian UAVs are often private property. The owner of a civilian UAV may not be aware of prohibitions against invading privacy rights or crossing the airspace boundaries of objects over which the UAV is flying. Therefore, a solution to counter unmanned aerial vehicles should effectively deal with the UAV without causing actual damage to it.

[0007] Another criterion to consider when designing solutions for countering UAVs is that the use of civilian UAVs typically occurs in the airspace above population centers (cities, urban-type settlements, villages, etc.), which again imposes a number of restrictions on acceptable methods of countering unmanned aerial vehicles. For example, a countermeasure solution may have to take into account the proximity of various municipal structures and the use of various devices using cellular networks that utilize municipal infrastructure.

[0008] Therefore, there is a real need to create solutions that effectively counteract any detected movement of unknown unmanned aerial vehicles within monitored areas of the airspace. Summary of the invention

[0009] Aspects of the present invention solve the above-mentioned problems and deficiencies known in the art. Aspects provide an integrated solution for detecting, classifying, identifying and resisting unmanned aerial vehicles, particularly civilian UAVs, without causing damage to them. This integrated solution can be used for government, commercial, private and public interests. One of the aspects of the present invention is to provide protection for specific areas of the airspace around an object on or next to which the system described herein has been installed to prevent UAVs, including in the airspace above population centers. In particular, the present invention relates to solutions for detecting and resisting unknown UAVs in the event that the unknown UAVs pass through the monitored airspace. An unknown UAV is a UAV that does not have permission to enter the monitored area of ​​the airspace.

[0010] The disclosed system performs at least: detecting moving and / or flying objects using a primary detection module, capturing detected objects using a recognition module, classifying detected objects based on at least one captured image using a control and classification module, and identifying the UAV using the control and classification module in the event that the detected object is determined to be a UAV. The disclosed system also takes the step of using a threat elimination module to resist unknown UAVs in the event that the UAV cannot be identified. The identification of the detected UAV can be based on a "friend or foe" procedure. If the UAV is not identified, then the UAV can be classified as an unknown UAV.

[0011] In one aspect of the invention, the disclosed solution can be integrated with an operational security system used at an object, the airspace around which needs to be protected against UAVs. As used herein, the term "object" broadly refers to any kind of facility (such as a house and a stadium), as well as conditionally designated spaces (such as an airport).

[0012] A first technical result of the present invention is to broaden the range of technical means for countering various UAVs, including countering various UAVs in the airspace above population centers, by probing the airspace, detecting and identifying UAVs, and taking measures to remove detected unknown UAVs from monitored areas of the airspace.

[0013] A second technical result of the present invention is to protect the airspace of protected objects (including objects located in population centers) against unknown UAVs by detecting, classifying and identifying UAVs, and then removing unknown UAVs from the airspace of the protected objects (if necessary).

[0014] As a variant embodiment of the present invention, a method for detecting and resisting an unknown unmanned aerial vehicle (UAV) is proposed, wherein the method includes the following steps: a primary detection module can be used to perform detection of unknown flying objects in a monitored area of ​​airspace. The primary detection module can be used to determine the spatial coordinates of the detected unknown flying object, and the determined spatial coordinates can be sent to the control and classification module. An identification module can be used to capture an image of the detected unknown flying object. The control and classification module can be used to classify the detected unknown flying object based on the analysis of at least one image obtained from the identification module. In response to determining that the unknown flying object is a UAV using the control and classification module, identification of the UAV can be performed. After determining that the UAV is an unknown UAV, a threat elimination module can be used to perform directional radio suppression of the UAV's control signal until the unknown UAV leaves the monitored area of ​​the airspace.

[0015] In another aspect of the invention, protection against UAVs may be performed in the airspace over population centers.

[0016] In another aspect of the invention, identification of unknown UAVs can be performed by detecting visual markers, GPS beacons, or RFID tags that indicate UAV ownership. In another aspect, machine learning models such as neural networks can be used during analysis to classify detected unknown objects.

[0017] In another aspect, prior to searching the airspace for flying objects, the spatial coordinates of the position of the primary detection module are determined.

[0018] In yet another aspect, the recognition module may include at least one camera for capturing the detected flying object, wherein the capturing is performed based on the spatial coordinates of the detected flying object.

[0019] On the other hand, LIDAR or GPS receivers may be used to perform primary detection of unknown flying objects.

[0020] In yet another aspect, the recognition module may be mounted on a swivel module, thereby enabling the camera to move 360 ​​degrees around its axis.

[0021] In another aspect, the identification module may include at least two cameras, wherein one camera is configured to perform gross measurements and a second camera enables multiple zoom-ins on detected flying objects.

[0022] As another variant aspect of the invention, tracking of the detected flying object can be performed, during which the second camera gradually approaches the detected object in order to obtain at least one image presenting the detected flying object at the required resolution, and the at least one image is sent to the control and classification module. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more exemplary aspects of the present invention and, together with the detailed description, serve to explain the principles and implementations of these exemplary aspects.

[0024] Figure 1 A high-level architecture of a system for detecting and countering UAVs is shown, with the possibility of implementing various different aspects.

[0025] Figure 2 An example of a prototype of stationary aspects of a system for detecting and countering UAVs in the airspace above population centers is shown.

[0026] Figure 3 An example of primary detection of unknown objects within a monitored area of ​​airspace according to aspects of the present invention is shown.

[0027] Figure 4 An example of capturing an unknown object using a camera with a zoom function is shown.

[0028] Figure 5 An example of the location of the various modules of the protection system against UAVs at an industrial object is shown.

[0029] Figure 6 A flow chart illustrating a method of protecting against UAVs in the airspace over population centers.

[0030] Figure 7 Examples of general purpose computer systems, personal computers, or servers according to aspects of the present invention are presented. DETAILED DESCRIPTION

[0031] Aspects of the present invention enable to solve the defects of known technology by proposing a system and method for detecting and resisting unmanned vehicles. As used herein, the term "unmanned vehicle" refers broadly to vehicles of this type such as ground transport vehicles and air transport vehicles. In a specific aspect, the ground transport vehicle may refer to a self-propelled car. In a specific aspect, the air transport vehicle may refer to an unmanned aerial vehicle. In a specific aspect of the present invention, detection and resistance may be performed at least for UAV. One of the advantages of the present invention is that the disclosed system can be used in population centers (cities) without causing damage to the infrastructure of the population center or to the UAV itself. Examples of such damage may include damage associated with UAVs hitting various objects (such as buildings) and antennas mounted thereon. Another example of such damage may include damage to the functional interruption of various devices caused by radio frequency suppression during the steps taken by the system against UAVs. In a non-limiting example, such suppression may include suppressing communication signals between the UAV and its operator. The disclosed system may enable exploitation within urban areas by countering UAVs using directional radio antennas for countering UAVs without disabling the UAV's onboard control systems and without restricting the operation of the UAV's flight mechanisms (e.g., propellers and thrusters), which allows the UAV to not crash into any objects.

[0032] Another advantage of the present invention is the use of LIDAR to perform at least the search and primary detection of flying objects in the monitored airspace. Thus, LIDAR determines clusters of points within a specific area by laser probing the airspace. The determined clusters can then be analyzed to identify foreign objects / unknown flying objects.

[0033] As mentioned above, the disclosed system can be designed to provide protection against various civilian UAVs. Examples of such UAVs can be quadcopters created by companies such as DJI (PHANTOM, INSPIRE, MAVIC, M600, S1000 models), FREEFLY (ALTA model), PARROT and XIAOMI, among many others.

[0034] According to various aspects, the present invention may be implemented in either a fixed form or a mobile form, and examples of its implementation are given below.

[0035] According to various aspects of the present invention, in order to achieve its purpose, it is desirable to install or deploy a system for detecting and resisting unmanned vehicles (UAVs in specific cases) directly on or next to an object for which UAV protection is required for the airspace around it. As used herein, the term "object for which unmanned vehicle protection is required" broadly refers to at least one of the following: some specific structures (e.g., stadiums, residences), entire infrastructures (e.g., airports, seaports, industrial parks), population centers (e.g., rural cities), places for holding public events, etc. Similarly, various different aspects make it possible to install the disclosed system on an immovable object (such as the roof of a building), or on a movable object (such as a car) equipped with a portion for installing the disclosed system. In addition, the disclosed system can also be integrated with existing security systems employed on objects (e.g., in airports, ports, and stadiums). After deploying a system for protecting the airspace of an object and preventing unmanned vehicles (UAVs in specific examples), the system can be started in a search mode for unmanned vehicles (particularly UAVs). The disclosed system also has the ability to adjust functionality based on the requirements of the location the system is located in. Examples of such requirements may include, but are not limited to, weather conditions, designated areas of airspace, and their boundaries.

[0036] In various aspects, depending on the particular configuration of the installed system, detection and / or threat elimination of multiple flying objects, including UAVs, within a monitored area can be performed sequentially or simultaneously. In addition, the disclosed system is capable of performing threat elimination of specific unmanned vehicles. In order to perform threat elimination of specific unmanned vehicles, the disclosed system can obtain information at least about the types and / or types of unmanned vehicles that need to be protected in a specific spatial area.

[0037] Furthermore, it will be appreciated that the invention is not limited to the examples shown in the accompanying drawings. More specifically, the functionality of the disclosed system is not limited to defending against UAVs, and the system can also be used in a similar manner to defend against other types of unmanned vehicles.

[0038] Figure 1A system 100 for detecting and counteracting UAVs is schematically shown. In a preferred aspect, the system 100 for detecting and counteracting UAVs (hereinafter referred to as a protection system 100 against UAVs) may include at least the following modules: a primary detection module 110, an identification module 120, a control and classification module 130, and a threat elimination module 140. In addition, each module uses software that enables their mutual interaction and the execution of their functions.

[0039] The primary detection module 110 may be configured to detect any moving and / or flying objects in the airspace and subsequently determine the spatial coordinates of the detected moving and / or flying objects. The primary detection module 110 may be configured to interact with at least the control and classification module 130. The interaction between these modules may include the transmission of data determined during the exploration of the airspace and the coordinates determined for the detected objects. The coordinates may include at least the azimuth orientation of the detected unknown flying object, the height of the detected unknown flying object, and the distance to the detected unknown flying object.

[0040] It should be noted that in implementing the protection system 100 against UAVs, the ability to search for and detect flying objects may be limited by the technical capabilities of the equipment used to manufacture the primary detection module 110, or by the defined boundaries of the monitored area of ​​airspace. Depending on the implementation and configuration of the primary detection module 110, detection of multiple flying objects may be performed simultaneously or sequentially (one at a time).

[0041] The primary detection module 110 may be configured to include a GPS (Global Positioning System) device and at least one of the following devices:

[0042] LIDAR equipment, which emits a laser beam capable of detecting objects in airspace;

[0043] A camera, such as a wide-angle camera;

[0044] an audio receiver, such as a microphone;

[0045] Radio frequency equipment (radar).

[0046] The aforementioned devices may contain software that allows them to obtain necessary information and interact with other modules of the protection system 100 for preventing UAVs. Such software may be designed and configured based on various aspects of the protection system 100 for preventing UAVs. It should be noted that the design and configuration of such software is outside the scope of the present invention. In a preferred aspect, the primary detection module 110 may include an omnidirectional LIDAR, which may use laser detection in conjunction with a GPS receiver to perform a search for UAVs. The laser detection may be performed in a scanning sector from 0 to 360 degrees along the horizontal axis and a scanning angle from 0 to 90 degrees along the vertical axis. The window along the vertical axis may reach at least 45 degrees. According to various aspects, the distance from the location where the UAV can be detected by the disclosed system to the UAV may vary, for example, in a range from about 0.2 km to about 5 km. At the same time, according to the technical capabilities of the primary detection module 110, the detection distance may be increased or decreased as needed.

[0047] On the other hand, the primary detection module 110 may include a LIDAR, a GPS receiver, and at least one camera. In this particular aspect, the primary detection may be achieved by both the LIDAR and the camera. It should be noted that the number of cameras installed depends on the airspace area to be protected and the necessary scanning sector (from about 0 degrees to about 360 degrees). Thus, for example, 10 (ten) cameras installed in a circle may meet the requirements of all-round visibility. The cameras connected to each other may communicate with at least the control and classification module 130, which enables searching for UAVs in parallel with the LIDAR.

[0048] In another aspect, the primary detection module 110 may additionally include at least one audio receiver, which may also perform a search for the UAV in conjunction with the control and classification module 130 .

[0049] Preliminary adjustment / calibration and further control of the primary detection module 110 may be performed using the control and classification module 130. Examples of calibration may include determining the spatial coordinates of the primary detection module 110 relative to other modules of the protection system 100 for preventing UAVs, or determining the spatial coordinates of the primary detection module 110 relative to an object whose surrounding airspace needs protection. Such control of the primary detection module 110 may include, for example, determining the boundaries of the airspace surrounding an object that needs to be protected from UAV flight.

[0050] The identification module 120 can be configured to: 1) capture detected flying objects based on data obtained from the control and classification module 130, where the data may include information about the spatial coordinates of the detected objects; 2) track the captured objects (observe their movement), and 3) transmit information about the captured objects to the control and classification module 130.

[0051] In one aspect, the recognition module 120 may include two cameras mounted on a slewing module. The first camera (e.g., a wide-angle camera) may enable tracking (video observation) of an object in a wide-angle field of view, and the second camera may have a narrower angle field of view. However, the second camera may have the ability to use a zoom function to obtain a higher resolution image of the target object. In other words, the first camera may be a so-called panoramic camera, and the second camera may be a zoom camera. In one aspect, the first camera may be a camera with a wide-angle lens, and the second camera may be a camera including a zoom lens with a high variable focal length, so that the scale of the image of the target object can be changed when shooting from the same point.

[0052] The control of the slewing module can be performed by the control and classification module 130, which can send a control command to the slewing module to rotate the recognition module 120 in the direction of the detected flying object, for example, a mounted camera. The second camera can allow changing (enlarging) the scale of the image of the flying object by optical zooming and / or by digital zooming.

[0053] It should be noted that the identification module 120 can also be configured to use at least one camera to perform its functions. To some extent, different aspects may depend on the technical characteristics of the camera and the airspace that needs to be protected from UAVs. In one aspect, when the primary detection module 110 is configured with at least one camera, the identification module 120 may only include a zoom camera.

[0054] In other aspects, the identification module 120 may be composed of or additionally include at least one of the following devices: a camera and a device with infrared thermal imaging and / or night vision capabilities. In addition, the identification module 120 may also include additional devices that help detect, monitor, and collect information about flying objects in the airspace for subsequent analysis.

[0055] The control and classification module 130 may also be configured to process data obtained from the primary detection module 110 and the recognition module 120, control the turnaround module and the modules 110, 120 and 140. The control and classification module 130 may also be configured to classify the detected flying object, wherein the classification may include at least analysis of the data obtained from the recognition module 120. In one aspect, the control and classification module 130 may employ a neural network, such as an artificial neural network (ANN), to make a decision on whether the detected object corresponds to a specific UAV type. As data exists for analysis, the recognition module 120 may provide various images of the target object according to the device that produces such an image. Thus, for example, a camera may provide at least one video frame, while a camera may provide a photo. In response to determining that the detected object is a UAV, the control and classification module 130 may send a request to the threat elimination module 140 to resist the detected UAV.

[0056] In the context of the present invention, "neural networks" may be primarily used to detect objects in various images, and may be used to classify detected objects. For example, the images may be video frames obtained from at least the recognition module 120. The neural network employed by the control and classification module 130 for analyzing the video frames may be a previously trained neural network, and may also have the ability to learn dynamically (on-the-fly) during its operation. The neural network may be trained using a list of prepared annotated video frames, which represents examples of different flying objects of interest, including but not limited to different classes and types of UAVs operating in different climatic conditions, with different foreshortening direction angles, and with different background lighting. In a particular aspect, the control and classification module 130 may use a convolutional neural network (CNN) to effectively identify objects in the analyzed images.

[0057] In one aspect, the control and classification module 130 can be a server (such as a graphics processing unit "GPU" server). In general, the server implementation can be understood as a computer system, such as the following combination Figure 7 The computer system. The computer system can be understood as a personal computer, such as a desktop computer, a laptop computer and a netbook, or an electronic device with wireless communication capabilities, such as a mobile phone, a smart phone and a tablet computer. The computer system can include various software, such as but not limited to: hardware, programs and / or system software.

[0058] The threat elimination module 140 can be configured to counter the detected UAV. As used herein, the term "counter" generally refers to escorting the UAV out of the monitored airspace. In one aspect, the threat elimination module 140 can use at least one antenna that can be configured to enable directional radio suppression of control signals used to control the detected UAV. Radio suppression can be performed continuously until the UAV leaves the monitored area of ​​the airspace. The antenna used by the threat elimination module 140 can produce radio suppression at least within a specific range (e.g., from about 20 degrees to about 25 degrees). The (one or more) antennas used can be able to interfere with control signals at a given distance within a specific frequency range. Therefore, for example, the suppression distance of the UAV can be at least 600 meters, the suppression power can be at least 100MW, and the frequency range emitted by the antenna can include at least the following frequencies: 900MHz, 1.2GHz, 2.4MHz, 5.8GHz, L1 and L2, and a white noise source can be used to suppress the signal of the communication channel of the UAV without interfering with the GPS receiver of the UAV.

[0059] In one aspect, the threat elimination module 140 may be arranged together with the identification module 120 on a turntable of a slewing module so that the position of the module may be changed according to the movement of the detected flying object.

[0060] Various aspects of the invention include both fixed and mobile implementations.Furthermore, in one aspect, the disclosed protection system 100 against UAVs can be integrated with existing security systems used at objects around which the airspace needs to be protected against UAVs.

[0061] When the protection system 100 against UAVs is integrated with the aforementioned security system, the protection system 100 against UAVs can be adapted using the control and classification module 130. During this adaptation, all available devices of the security system can be determined, such as additional cameras. In addition, all devices of both systems can be coordinated for the joint operation of the aforementioned modules of the protection system 100 against UAVs with respect to the detected devices. It should be noted that when the protection system against UAVs is integrated into the security system, the control and classification module 130 can be integrated using the control server of the security system by installing at least the software comprising the neural network.

[0062] exist Figure 2 The working scenario of the proposed system 100 is presented in the description of one aspect shown in .

[0063] Figure 2An example of a fixed form prototype of the UAV protection system 100 is shown. Therefore, the UAV protection system 100 may include at least a primary detection module 210, an identification and threat elimination module 250, and a data processing module 290. Then, the primary detection module performs the above combined Figure 1 The primary detection module 110 described herein may include a LIDAR 220, a GPS receiver 230, and a magnetometer ( Figure 2 The identification and threat elimination module 250 may perform the above combined Figure 1 The combined functions of the identification module 120 and the threat elimination module 140 described above. The identification and threat elimination module 250 can be installed on the slewing module 240. In this example, the identification and threat elimination module 250 can be implemented as two cameras: a first camera (panoramic camera) 260 and a second camera (camera with zoom function) 265. The identification and threat elimination module 250 can also include a set of radio antennas 270. The set of radio antennas 270 can include antennas for radio suppression, each of which operates at a specific radio frequency. The frequency or frequency range can be determined according to the frequency of the UAV operation. The data processing module 290 can be configured to execute the control and classification module 130 ( Figure 1 ) tasks and functions. In this example, the data processing module 290 may include a server that includes a computer system capable of processing large amounts of data, particularly graphic data. In one aspect, the data processing module 290 may be incorporated into a separate housing. Such an implementation enables sufficient mobility and ergonomic effectiveness of the protection system 100 against UAVs. Communication between the primary detection module 210, the identification and threat elimination module 250, and the data processing module 290 may be provided at least by wired communication (cable connection).

[0064] First, all the modules shown of the protection system 100 against UAVs can be installed directly on or next to the object whose surrounding airspace needs to be protected against UAVs. The protection system 100 against UAVs can then be started in UAV search mode. If necessary, the settings of the protection system 100 against UAVs can also be established according to the geographical location. Thus, for example, the settings can involve at least one of the following:

[0065] If the modules (particularly the primary detection module 210 and the identification and threat elimination module 250 ) are separate, determine the spatial coordinates of the protection system 100 against UAVs and its modules relative to each other;

[0066] Determine the boundaries of the airspace area that will provide protection against UAV flights;

[0067] · constructing data processing used by the data processing module 290 in its neural network classification / calculation, wherein the construction may also include training the neural network according to the operating conditions of the protection system 100 against UAVs; · forming an airspace area for the UAV to search for the protected object;

[0068] Areas within the monitored area that form the airspace that need to be protected from UAVs;

[0069] Using the primary detection module 210 , in particular with the aid of LIDAR 220 , a map of the location of objects within the monitored area of ​​the airspace is generated.

[0070] Thus, the protection system 100 against UAVs may be activated in a mode of searching for flying objects in a monitored area of ​​airspace.

[0071] Whenever any flying object appears in the monitored area of ​​the airspace, the primary detection module 210 can detect it using the LIDAR 220 and can determine the coordinates of the detected flying object. The primary detection module 210 can also be configured to send the determined coordinates to the data processing module 290.

[0072] In a variant aspect, the detection process of flying objects may include the following steps. LIDAR 220 may perform laser exploration of airspace, during which information is transmitted to data processing module 290. The information provided by LIDAR 220 may contain data about the position of each object in the form of a point within the monitored area of ​​airspace. The position of each point may be generated based on the reflection of the laser beam from each surface including the surface of the object. Data processing module 290 may determine the presence of flying objects based on the analysis of the information received from LIDAR 220. On the other hand, in which a map representing the airspace area has been generated, the generated map may be used to perform analysis. Next, LIDAR 220 may determine the spatial coordinates of the detected flying objects in cooperation with GPS receiver 230, and then the movement of the detected flying objects may also be tracked. The obtained data may be sent to data processing module 290 in real time. Figure 3 An example of detection and image capture of a flying object using LIDAR 220 is presented in FIG. 2 , where a flying object is detected in a square.

[0073] Return to reference Figure 2 Next, the data processing module 290 may send a command to the rotation module, and the rotation module may perform a rotation so that the identification and threat elimination module 250 is aligned with the direction of the detected unknown flying object. The data processing module 290 may also send the coordinates of the flying object to the identification and threat elimination module 250.

[0074] Using at least one of the aforementioned cameras, the identification and threat elimination module 250 can capture images of detected unknown flying objects, and then can track the flight movement of the detected flying objects. During the tracking of the detected flying objects, the identification and threat elimination module 250 can use the second camera 265 with a zoom function to perform zooming on the target object. These steps can provide at least one digital video frame with the detected object, which can be used to further identify the detected object. As described above, the second camera 265 can be a camera containing a lens with a high variable focal length, so that the scale of the image of the object can be changed during its shooting of the object from a single point. The first camera 260 can be a camera with a wide-angle lens. The second camera 265 can be configured to determine and track the direction of movement of the flying object, for example, so as not to lose the flying object from the line of sight. It should be mentioned that when there are two mentioned cameras 260 and 265 in the implementation of the identification and threat elimination module 250, the capture of the image of the flying object can be performed in a variety of ways. First, the image can be captured by two cameras at the same time. Secondly, image capture may first be performed by the first (panoramic) camera 260, and correction may then be performed by the second camera 265. Thirdly, image capture may first be performed by the second camera 265, which may continue to track the detected object (monitor its movement). Later, if necessary, the identification and threat elimination module 250 may use the first camera 260 for guidance. The effectiveness of each method of capturing an image of a flying object may depend on the distance to the flying object and the speed of movement of the flying object. It should be noted that the term "capturing an image of an object" in this article refers to at least an intermediate stage between the detection phase and the tracking phase in the process of processing information about an object.

[0075] In one aspect, the second camera 265 can first use the midpoint of the distance to the object in order to capture at least the flying object in the video frame (e.g., in the form of a dot). After capturing the first image, the necessary scale changes can be made to zoom in on the flying object in the video frame. Figure 4 An example of capturing an image of an unknown flying object using a camera 265 with a zoom function is presented in FIG. Figure 4 The captured flying object is shown after a zoom operation has been performed. It should be noted that the use of two different cameras within the identification and threat elimination module 250 enables a high degree of focus to be ensured on the unknown flying object and allows tracking of the unknown flying object during its motion.

[0076] In other aspects, the first camera 260 may also generate video frames having captured unknown flying objects, and send the generated video frames to the data processing module 290 .

[0077] After capturing the image, the identification and threat elimination module 250 may send at least one video frame with the captured object to the data processing module 290. It should be noted that during this process, the scale of the image of the flying object may be adjusted to produce at least one high-quality video frame. The data processing module 290 may determine whether a change in scale (enlargement or reduction) is required based on the analysis of the video frame during the identification of the detected flying object.

[0078] In one aspect, the data processing module 290 can use a neural network to analyze the acquired data. The neural network used can receive at least one video frame obtained from the identification and threat elimination module 250 as input to classify the detected unknown flying object. The input data can include at least a video frame received from at least one of the first camera 260 and / or the second camera 265, and information from other devices used by the identification and threat elimination module 250 or the primary detection module 210. Such devices can include at least LIDAR, audio receivers, and radars. Using the analysis results performed by the neural network, the data processing module 290 can make a decision on whether the detected unknown flying object meets at least one type of UAV or a specific model of UAV. If it is determined that the detected flying object is a UAV, the data processing module 290 can send a signal to the identification and threat elimination module 250 to indicate the need to resist the detected UAV.

[0079] As described above, the identification and threat elimination module 250 may also employ a set of radio antennas 270. In response to receiving a signal from the data processing module 290, the identification and threat elimination module 250 may orient the radio antennas 270 toward the detected UAV and may perform directional radio suppression on the UAV. In particular, the identification and threat elimination module 250 may perform suppression of control signals of the detected UAV until the UAV leaves the monitored airspace. The identification and threat elimination module 250 may also receive the type of radio frequency used or the desired radio frequency range for interference from the data processing module 290. The data processing module 290 may determine the radio frequency type and the desired range based on the identified UAV. When the detected UAV is removed from the monitored area of ​​the airspace, the identification and threat elimination module 250 may effectively complete its mission.

[0080] In a particular aspect of the UAV protection system 100, when the primary detection module 210 simultaneously detects two or more unknown flying objects, the data processing module 290 may be used to perform prioritization in order to determine the order of classification of the unknown flying objects and (if necessary) the subsequent threat elimination thereof. The prioritization may be performed based at least on a comparison of the distances to the detected multiple flying objects and a comparison of the closing speeds of the multiple flying objects.

[0081] On the other hand, the protection system 100 for preventing UAVs, after determining that the unknown flying object is a UAV, the disclosed system can optionally identify the traceability of the UAV and / or its ability to be in a monitored area of ​​the airspace. To this end, a "friend or foe" technique can be used. Thus, for example, the protection system 100 for preventing UAVs can perform additional analysis for identifying the UAV. The additional analysis can be based on the ability to identify visual markers on the body of the UAV, the use of infrared transmitters, RFID (radio frequency identification) tags, or GPS beacons. Thus, based on the detection of one of the aforementioned identifiers, the UAV can be identified as a "friend" or "enemy". For example, if the UAV has a GPS beacon, the protection system 100 for preventing UAVs can obtain coordinates from the UAV and can identify the UAV based on the coordinates. If it is determined that the UAV has permission to enter the monitored area, the system will not perform any resistance. Otherwise, if the UAV is identified as unknown, it can be resisted as described above.

[0082] In other aspects, the UAV-proof protection system 100 may include two or more primary detection modules 210 and identification and threat elimination modules 250. In this case, the UAV-proof protection system 100 may also enable capture and simultaneous threat elimination of two or more detected UAVs.

[0083] On the other hand, the monitored area of ​​the airspace can be divided into multiple sectors according to the distance from the protected object. Based on the divided sectors, the protection system 100 against UAVs can determine which module performs its function. For example, the primary detection module 210 can perform a search for flying objects in all sectors, while the identification and threat elimination module 250 can only cover nearby sectors. The nearby sectors can be determined based on the distance at which the identification and threat elimination module 250 can perform its functions. The sectors and configuration of one module or another module in the protection system 100 against UAVs can be set according to the technical capabilities of the modules and according to the need for protection against UAVs in one sector or another sector of the monitored area of ​​the airspace.

[0084] Figure 5 An example of the location of the individual modules of the protection system against UAVs at an industrial object is shown. In this example, modules 210 and 250 are spaced apart within the monitored area at the industrial object. It should be noted that the separation of the individual modules and their number depends on the industrial object itself that needs to be protected and on the technical capabilities of the respective modules. For example, Figure 5The industrial park shown in is located in a large area. Therefore, in order to effectively protect the industrial park, it is desirable to distribute these modules. Specifically, for example, the number of modules 250 may not be limited to one, but may be increased to four. When the modules 250 are arranged, they may be installed in different corners of the industrial object in four directions.

[0085] Figure 6 A flow chart of a method for protecting against UAVs in airspace over population centers is shown. In step 310, the primary detection module 110 performs primary detection of flying objects within a monitored area of ​​the airspace. The primary detection may be performed using at least a LIDAR 220, which may perform a search by laser probing the airspace. In this step, the LIDAR 220 may only be able to detect flying objects, but not identify them. Therefore, at this point, all detected flying objects are also unknown flying objects. Step 320 may be initiated in response to detecting at least one flying object within a monitored area of ​​the airspace.

[0086] In step 320, the primary detection module 110 may determine the spatial coordinates of each detected flying object, and may send the determined spatial coordinates to the control and classification module 130. The spatial coordinates of the detected flying object may include at least the azimuth orientation of the detected unknown flying object, the height of the detected unknown flying object, and the distance to the detected unknown flying object. The control and classification module 130 may send the received coordinates to the recognition module 120 to capture an image of the flying object.

[0087] In step 330, the recognition module 120 may capture an image of the detected flying object using at least one camera of the recognition module 120. After the image is captured, the flying object may be tracked and a video frame with the captured flying object may be generated by the recognition module 120. The recognition module 120 may send the captured image(s) to the control and classification module 130. In a preferred aspect, image capture may be performed using two cameras, wherein the first camera is a camera 260 with a wide angle lens and the second camera is a camera 265 with a zoom function. The two cameras are capable of capturing images of the flying object and transmitting video frames.

[0088] In step 340, the control and classification module 130 may be used to classify the detected object based on the analysis of at least one of the video frames obtained using at least one camera. In order to classify the detected object, the control and classification module 130 may use a neural network in the analysis of the video frame. The neural network may be trained in advance by a list of annotated test images, which represent different flying objects at different projection reductions and with various backgrounds. The neural network may be used to analyze each video frame obtained, and a decision may be generated as an output. The generated decision may contain information about the traceability of a specific flying object (including a UAV). In addition, if the flying object is determined to be a UAV, the decision may also contain information about the type and model of the UAV. In a particular aspect, if the flying object is determined to be an unknown object, the control and classification module 130 may output the generated decision to an operator of the protection system for preventing UAVs and may wait for a response, or the control and classification module 130 may perform additional analysis, which may be based on an analysis of the probability that the flying object belongs to a specific type of object. Based on this additional analysis, the control and classification module 130 can then generate a further decision as to whether the unknown object is a UAV. In another aspect, the control and classification module 130 can request additional video frames, and can analyze the additional video frames using a neural network until the control and classification module 130 can classify the detected flying object. In this case, if it is determined that the detected flying object is a UAV, step 350 can be performed.

[0089] In step 350, if the object is determined to be a UAV, the control signals of the UAV may be subjected to directional radio suppression by the threat elimination module 140 until the UAV leaves the monitored area of ​​the airspace. The suppression may be achieved using a series of radio antennas, such as the radio antenna 270. The suppression may be stopped after the detected UAV is removed from the monitored area of ​​the airspace. The protection system 100 against UAVs may continue to search further for the flying object.

[0090] In certain aspects, if the control and classification module 130 determines that the detected unknown flying object is a UAV, then in optional step 345, the UAV may be identified by the control and classification module 130. The identification of the UAV may include determining whether the UAV belongs to a UAV that is authorized to fly in a monitored area of ​​the airspace. The identification mechanism of the UAV may use, for example, a "friend or foe" identification technique. According to various aspects, the control and classification module 130 may use visual markings, infrared (IR) transmitters, RFID tags, or GPS beacons on the body of the UAV to identify the source of the UAV. If the detected unknown UAV belongs to a UAV that is authorized to fly in a monitored area of ​​the airspace, the protection system 100 against UAVs may stop capturing images and may stop tracking the UAV, but may allow the UAV to continue its movement in the airspace. Otherwise, if the UAV is not identified, step 350 may be performed.

[0091] Figure 7 is a block diagram illustrating a computer system 20 according to an exemplary aspect on which aspects of the systems and methods for detecting and countering UAVs may be implemented. The computer system 20 may represent Figure 1 and Figure 2 The UAV protection system 100 can be in the form of multiple computing devices or in the form of a single computing device, such as a desktop computer, a laptop computer, a handheld computer, a mobile computing device, a smart phone, a tablet computer, a server, a mainframe, an embedded device, and other forms of computing devices.

[0092] As shown, computer system 20 includes a central processing unit (CPU) 21, a system memory 22, and a system bus 23 connecting various system components, including a memory associated with the central processing unit 21. The system bus 23 may include a bus memory or a bus memory controller, a peripheral bus, and a local bus capable of interacting with any other bus architecture. Examples of buses may include PCI, ISA, PCI-Express, HyperTransport, and the like. TM (HyperTransport TM ), unlimited bandwidth TM (InfiniBand TM ), Serial ATA, I 2C, and other suitable interconnections. The central processing unit 21 (also referred to as a processor) may include a single group or multiple groups of processors having a single core or multiple cores. The processor 21 may execute one or more computer executable codes that implement the technology of the present invention. The system memory 22 may be any memory for storing data used herein and / or computer programs executable by the processor 21. The system memory 22 may include volatile memory (such as random access memory (Random Access Memory, RAM) 25) and non-volatile memory (such as read-only memory (Read-Only Memory, ROM) 24, flash memory, etc.) or any combination thereof. The basic input / output system (Basic Input / Output System, BIOS) 26 may store basic programs for transmitting information between elements of the computer system 20, such as those basic programs when the operating system is loaded using ROM 24.

[0093] The computer system 20 may include one or more storage devices, such as one or more removable storage devices 27, one or more non-removable storage devices 28, or a combination thereof. The one or more removable storage devices 27 and the one or more non-removable storage devices 28 are connected to the system bus 23 via a memory interface 32. In one aspect, the storage devices and corresponding computer-readable storage media are power-independent modules for storing computer instructions, data structures, program modules, and other data for the computer system 20. The system memory 22, the removable storage devices 27, and the non-removable storage devices 28 may use a variety of computer-readable storage media. Examples of computer-readable storage media include: machine memory, such as cache, SRAM, DRAM, zero-capacitor RAM, two-transistor RAM, eDRAM, EDO RAM, DDR RAM, EEPROM, NRAM, RRAM, SONOS, PRAM; flash memory or other storage technology, such as in a Solid State Drive (SSD) or a flash drive; magnetic cassettes, tapes, and disk storage, such as in a hard drive or floppy disk; optical storage, such as in a compact disc (CD-ROM) or a digital versatile disc (DVD); and any other medium that can be used to store the desired data and can be accessed by the computer system 20.

[0094] The system memory 22, removable storage device 27 and non-removable storage device 28 of computer system 20 can be used to store operating system 35, additional application programs 37, other program modules 38 and program data 39. Computer system 20 may include a peripheral interface 46 for transmitting data from input device 40, such as a keyboard, mouse, light pen, game controller, voice input device, touch input device, or other peripheral devices, such as a printer or scanner via one or more I / O ports, such as a serial port, parallel port, Universal Serial Bus (USB), or other peripheral interfaces. Display device 47 (such as one or more monitors, projectors or integrated displays) can also be connected to system bus 23 via output interface 48 (such as a video adapter). In addition to display device 47, computer system 20 can also be equipped with other peripheral output devices (not shown), such as speakers and other audio-visual devices.

[0095] The computer system 20 can work in a network environment using a network connection with one or more remote computers 49. The one or more remote computers 49 can be local computer workstations or servers, which include most or all of the elements described above when describing the properties of the computer system 20. Other devices can also be present in the computer network, such as but not limited to routers, websites, peer devices or other network nodes. The computer system 20 may include one or more network interfaces 51 or network adapters for communicating with the remote computers 49 via one or more networks, such as a local area computer network (LAN) 50, a wide area computer network (WAN), an intranet, and the Internet. Examples of the network interface 51 may include an Ethernet interface, a frame relay interface, a SONET (Synchronous Optical Network) interface, and a wireless interface.

[0096] Various aspects of the present invention may be systems, methods and / or computer program products. The computer program products may include one or more computer-readable storage media having computer-readable program instructions for causing a processor to perform various aspects of the present invention.

[0097] Computer readable storage medium can be a tangible device, and this tangible device can keep and store the program code of the form of instruction or data structure, and this program code can be accessed by the processor of computing device (such as computing system 20).Computer readable storage medium can be electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or its any suitable combination.As an example, this type of computer readable storage medium can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), flash memory, hard disk, portable computer disk, memory stick, floppy disk or even mechanical coding device, such as punch card or convex structure in the groove of recording instruction thereon.As used in this article, computer readable storage medium should not be regarded as temporary signal itself, temporary signal such as radio wave or other free propagation electromagnetic wave, electromagnetic wave propagated by waveguide or transmission medium or electric signal transmitted by wire.

[0098] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network interface in each computing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the corresponding computing device.

[0099] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (Instruction-Set-Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​and traditional programmed programming languages).Computer-readable program instructions (as independent software packages) can be performed completely on the user's computer, partly on the user's computer, partly on the user's computer and partly on a remote computer or completely on a remote computer or server.In the latter case, the remote computer can be connected to the user's computer through any type of network (including LAN or WAN), or can be connected to an external computer (for example, through the Internet).In some embodiments, electronic circuits (including for example programmable logic circuits, field programmable gate arrays (FPGAs) or programmable logic arrays (Programmable Logic Arrays, PLAs)) can be performed by utilizing the state information of computer-readable program instructions to personalize the electronic circuit, thereby performing aspects of the present invention.

[0100] In various aspects, the system and method described in the present invention can be processed according to modules. As used herein, the term "module" refers to, for example, the arrangement of the components of the equipment, components, or hardware (such as by application-specific integrated circuit (ASIC) or FPGA) of the real world, or refers to a combination of hardware and software, such as by a microprocessor system and an instruction set (the instruction set converts the microprocessor system into a special-purpose device when executed) that implements the module function. A module can also be implemented as a combination of two modules, wherein certain functions are promoted by hardware alone, and other functions are promoted by a combination of hardware and software. In some implementations, at least a portion of a module (and in some cases, all of a module) can be run on a processor of a computer system. Therefore, each module can be implemented with various suitable configurations, and should not be limited to any specific implementation illustrated herein.

[0101] For the sake of clarity, not all routine features of various aspects are disclosed herein. It should be appreciated that in the development of any practical implementation of the present invention, many specific implementation decisions must be made in order to achieve the developer's specific goals, and these specific goals will vary for different implementations and different developers. It should be understood that such development efforts can be complex and time-consuming, but are still a routine task of engineering for those of ordinary skill in the art who understand the advantages of the present invention.

[0102] In addition, it should be understood that the words or terms used herein are for the purpose of description rather than limitation, so the terms or terms of this specification should be interpreted by those skilled in the art according to the teachings and guidance provided herein in combination with the knowledge of (one or more) relevant technical personnel. In addition, it is not intended to attribute any term in this specification or claims to an uncommon or special meaning unless explicitly stated as such.

[0103] The various aspects disclosed herein include the present and future known equivalents of the known modules cited herein in an illustrative manner. In addition, although various aspects and applications have been shown and described, it will be apparent to those skilled in the art who understand the advantages of the present invention that more modifications than those mentioned above are feasible without departing from the inventive concept disclosed herein.

Claims

1. A method for protecting against unmanned aerial vehicles, the method comprising: detecting flying objects within a monitored area of ​​airspace and determining spatial coordinates of the detected flying objects using data received from sensors of a primary detection module, the sensors including a global positioning system device, a light recognition, detection and ranging device, a radar, an audio receiver and at least one camera, wherein the primary detection module is configured to detect the airspace to obtain information associated with the detected unknown flying objects; capturing one or more images of the detected unknown flying object based on the determined spatial coordinates using a recognition module having a first camera including a wide-angle camera and a second camera having a zoom function, wherein at least one of the first camera and the second camera is attached to a swivel base, and wherein the swivel base is configured to rotate the at least one camera 360 degrees around an axis of the camera, wherein the recognition module is configured to change the scale of the images of the unknown flying object captured by the first camera and the second camera; feeding data received from sensors of said primary detection module and said one or more images captured using said recognition module to an artificial neural network trained to classify said detected unknown flying objects; determining, based on the output of the artificial neural network, whether the detected unknown flying object includes an unmanned aerial vehicle; and In response to determining that the detected unknown flying objects include the unmanned aerial vehicle, one or more radio signals exchanged between the unmanned aerial vehicle and a user of the unmanned aerial vehicle are suppressed until the unmanned aerial vehicle exits the monitored area of ​​the airspace.

2. The method according to claim 1 further includes tracking the detected unknown flying object.

3. The method according to claim 1, wherein: Classification of the detected unknown flying object is performed using a trained neural network.

4. The method of claim 1, further comprising, in response to determining that the detected unknown flying object comprises an unmanned aerial vehicle, identifying the unmanned aerial vehicle.

5. The method according to claim 4, wherein: Identifying the unmanned aerial vehicle also includes detecting at least one of a visual marker, a GPS beacon, and an RFID tag indicative of ownership of the unmanned aerial vehicle.

6. The method according to claim 1, wherein: The detected unmanned aerial vehicles include unmanned aerial vehicles that are not authorized to fly within the monitored area of ​​the airspace.

7. A system for protecting against unmanned aerial vehicles, the system comprising: A hardware processor configured to: detecting flying objects within a monitored area of ​​airspace and determining spatial coordinates of the detected flying objects using data received from sensors of a primary detection module, the sensors including a global positioning system device, a light recognition, detection and ranging device, a radar, an audio receiver and at least one camera, wherein the primary detection module is configured to detect the airspace to obtain information associated with the detected unknown flying objects; capturing one or more images of the detected unknown flying object based on the determined spatial coordinates using a recognition module having a first camera including a wide-angle camera and a second camera having a zoom function, wherein at least one of the first camera and the second camera is attached to a swivel base, and wherein the swivel base is configured to rotate the at least one camera 360 degrees around an axis of the camera, wherein the recognition module is configured to change the scale of the images of the unknown flying object captured by the first camera and the second camera; feeding data received from sensors of said primary detection module and said one or more images captured using said recognition module to an artificial neural network trained to classify said detected unknown flying objects; determining, based on the output of the artificial neural network, whether the detected unknown flying object includes an unmanned aerial vehicle; and In response to determining that the detected unknown flying objects include the unmanned aerial vehicle, one or more radio signals exchanged between the unmanned aerial vehicle and a user of the unmanned aerial vehicle are suppressed until the unmanned aerial vehicle exits the monitored area of ​​the airspace.

8. The system according to claim 7, wherein: The hardware processor is also configured to track the detected unknown flying object.

9. The system according to claim 7, wherein: Classification of the detected unknown flying object is performed using a trained neural network.

10. The system according to claim 7, wherein: The hardware processor is further configured to, in response to determining that the detected unknown flying object includes an unmanned aerial vehicle, identify the unmanned aerial vehicle.

11. The system according to claim 10, wherein: The hardware processor configured to identify the unmanned aerial vehicle is further configured to detect at least one of a visual marker, a GPS beacon, and an RFID tag indicative of ownership of the unmanned aerial vehicle.

12. The system according to claim 7, wherein: The detected unmanned aerial vehicles include unmanned aerial vehicles that are not authorized to fly within the monitored area of ​​the airspace.

13. A non-transitory computer readable medium having stored thereon computer executable instructions for arming an unmanned aerial vehicle, the computer executable instructions comprising instructions for: detecting flying objects within a monitored area of ​​airspace and determining spatial coordinates of the detected flying objects using data received from sensors of a primary detection module, the sensors including a global positioning system device, a light recognition, detection and ranging device, a radar, an audio receiver and at least one camera, wherein the primary detection module is configured to detect the airspace to obtain information associated with the detected unknown flying objects; Using a recognition module having a first camera including a wide-angle camera and a second camera having a zoom function, one or more images of the detected unknown flying object are captured based on the determined spatial coordinates, wherein, at least one of the first camera and the second camera is attached to a swivel base, and wherein the swivel base is configured to rotate the at least one camera 360 degrees around an axis of the camera, wherein the recognition module is configured to change a scale of images of the unknown flying object captured by the first camera and the second camera; feeding data received from sensors of said primary detection module and said one or more images captured using said recognition module to an artificial neural network trained to classify said detected unknown flying objects; determining, based on the output of the artificial neural network, whether the detected unknown flying object includes an unmanned aerial vehicle; and In response to determining that the detected unknown flying objects include the unmanned aerial vehicle, one or more radio signals exchanged between the unmanned aerial vehicle and a user of the unmanned aerial vehicle are suppressed until the unmanned aerial vehicle exits the monitored area of ​​the airspace.

14. The non-transitory computer readable medium of claim 13, the computer executable instructions further comprising instructions for tracking the detected unknown flying object.

Images