Low-altitude wide-area security risk high-precision identification method based on reinforcement learning

Abstract

The invention discloses a reinforcement learning-based low-altitude wide-area security risk high-precision identification method, which comprises the following steps of: forming an unmanned aerial vehicle communication network by an unmanned aerial vehicle, a base station and a ground station, transmitting data acquired when the unmanned aerial vehicle executes a task in a specified airspace to the base station through the unmanned aerial vehicle communication network, and transferring the data to the ground station by the base station; and the ground station collects and detects the received data sent by the unmanned aerial vehicle so as to identify whether an abnormal event occurs. The accuracy and efficiency of risk identification and target identification are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of risk identification, in particular to a low-altitude wide-area security risk high-precision identification method based on reinforcement learning. BACKGROUND

[0002] There are many target detection algorithms based on reinforcement learning, but when these algorithms are directly applied to the unmanned aerial vehicle platform, problems such as low detection accuracy and easy loss of tracking may occur. Therefore, how to efficiently and accurately identify unmanned aerial vehicles has become one of the problems to be solved at present.

[0003] The existing photoelectric detection equipment can better cooperate with the microwave radar to track and classify targets, but the detection distance is relatively short and is easily disturbed by weather, building shielding and other factors. The existing radar detection equipment can complete the search, detection and tracking of various targets, but is easily disturbed by ground clutter and is not suitable for blocked situations, and electromagnetic pollution is serious. The existing radio detection equipment can discover and locate unmanned aerial vehicles by passively receiving radio signals between unmanned aerial vehicles and ground stations, but has relatively high environmental requirements and cannot have shielding.

[0004] The existing photoelectric detection equipment can better cooperate with the microwave radar to track and classify targets, but the detection distance is relatively short and is easily disturbed by weather, building shielding and other factors. The existing radar detection equipment can complete the search, detection and tracking of various targets, but is easily disturbed by ground clutter and is not suitable for blocked situations, and electromagnetic pollution is serious. The existing radio detection equipment can discover and locate unmanned aerial vehicles by passively receiving radio signals between unmanned aerial vehicles and ground stations, but has relatively high environmental requirements and cannot have shielding.

[0005] During the target identification process of the unmanned aerial vehicle, overlap and blurred feature information may occur, which cannot be effectively identified. Single base station detection and identification of unmanned aerial vehicles may have a blind area and cause target loss. The flight attitude and behavior information of cooperative unmanned aerial vehicles cannot be obtained, and the unmanned aerial vehicles cannot be controlled. The unmanned aerial vehicle crisis cannot be predicted and warned. SUMMARY

[0006] In view of this, the present application provides a low-altitude wide-area security risk high-precision identification method based on reinforcement learning.

[0007] The present application discloses a low-altitude wide-area security risk high-precision identification method based on reinforcement learning, which comprises: Step 1: The unmanned aerial vehicle, the base station and the ground station constitute an unmanned aerial vehicle communication network, and the data collected by the unmanned aerial vehicle during the execution of the task in the designated airspace is transmitted to the base station through the unmanned aerial vehicle communication network, and then transferred to the ground station by the base station. Step 2: The ground station collects and detects the data sent by the unmanned aerial vehicle to identify whether an abnormal event occurs.

[0008] Further, the step 1 comprises: establishing a local wireless network between the unmanned aerial vehicle and the base station, and any two nodes in the local wireless network can transmit data; regarding the unmanned aerial vehicle and the base station as nodes in the local wireless network; when the unmanned aerial vehicle enters the designated airspace to perform a task, actively searching for the local wireless network and updating the existing local wireless network to realize the calling of any unmanned aerial vehicle in the designated airspace at any time; when the unmanned aerial vehicle is in the blind area of the base station, establishing the connection between the local wireless network where the unmanned aerial vehicle is located and the adjacent local wireless network to enable the unmanned aerial vehicle to communicate with the base station through other local wireless networks; The unmanned aerial vehicle transmits the data obtained by the unmanned aerial vehicle through the base station to the ground station located in the same unmanned aerial vehicle communication network.

[0009] Further, the unmanned aerial vehicle communication network further comprises a unmanned aerial vehicle ad hoc network; the unmanned aerial vehicle ad hoc network is composed of a plurality of unmanned aerial vehicles and a plurality of ground stations; when a new unmanned aerial vehicle or ground station wants to join the unmanned aerial vehicle ad hoc network, the existing unmanned aerial vehicle ad hoc network is expanded by adding a new node in the unmanned aerial vehicle ad hoc network.

[0010] Further, the step 1 further comprises: realizing the real-time monitoring of the flight data of the unmanned aerial vehicle during the performance of the task of the unmanned aerial vehicle through the unmanned aerial vehicle data monitoring technology, the flight data comprising the flight mode, the flight attitude, the power load, the power information, the horizontal cruising speed, the vertical climbing speed, the height, the GPS positioning information and the camera holder angle.

[0011] Further, before the step 1, further comprising: Before the unmanned aerial vehicle flies, the flight parameters of the unmanned aerial vehicle are set, the flight parameters of the unmanned aerial vehicle comprising the flight height, the voltage, the communication interruption protection, the unmanned aerial vehicle remote controller calibration, the unmanned aerial vehicle GPS calibration, the maximum climbing speed of the unmanned aerial vehicle in the GPS mode, the flight speed and the maximum descending speed when returning.

[0012] Further, the step 2 comprises: The ground station collects the data returned by the unmanned aerial vehicle and detects the trajectory, time and place of the non-cooperative unmanned aerial vehicle to accurately identify the relevant information; if an abnormal event is found, the problem information is pushed to the commander and the operation personnel to enable the commander and the operation personnel to discover the problem in time and handle the corresponding plan; the abnormal event comprises the intrusion alarm and the deviation alarm; the problem information comprises the model and the number of the unmanned aerial vehicle where the abnormal event occurs.

[0013] Further, the accurate identification related information includes: Classify the data returned by the unmanned aerial vehicle through the trained classifier to identify the model of the data; If the trajectory, time, and place of the non-cooperative unmanned aerial vehicle do not match the preset content, an abnormal event occurs.

[0014] Further, the training process of the classifier includes: Extract video data from the data collected by the unmanned aerial vehicle and use it as a training sample, and convert the training sample into a negative sample pair, and use the negative sample for feature selection and extraction; the negative sample pair consists of two negative samples; the video data includes model, fuselage length, wing span, flight attitude, and airborne equipment; Preprocess the images in the video data, slide the window on the preprocessed images, extract features from the images in the window according to the negative sample pair, input the extracted features into the classifier for training, and obtain the trained classifier.

[0015] Due to the adoption of the above technical solutions, the present application has the following advantages: the present application can timely discover abnormal events and send the identification results related to the abnormal events (the target of the abnormal event) to relevant personnel, thereby improving the accuracy and efficiency of risk identification and target identification. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art according to these drawings.

[0017] Figure 1 A flowchart of a low-altitude wide-area safety risk high-precision identification method based on reinforcement learning according to an embodiment of the present application; Figure 2 A flowchart of the training of the classifier according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The present application is further described in conjunction with the drawings and embodiments. The described embodiments are only some of the embodiments of the present application, not all the embodiments. All other embodiments obtained by those skilled in the art should belong to the scope of protection of the embodiments of the present application.

[0019] The technical problems solved by the present application include: in the target identification process of the unmanned aerial vehicle, overlap and blurred feature information occur, which cannot be effectively identified. Single base station detection and identification of unmanned aerial vehicles have blind areas, and target loss problems occur. The flight attitude, behavior and other information of non-cooperative unmanned aerial vehicles cannot be obtained, and the unmanned aerial vehicles cannot be controlled. The crisis of the unmanned aerial vehicle cannot be predicted and warned.

[0020] Referring to Figure 1 The present application provides an embodiment of a low-altitude wide-area safety risk high-precision identification method based on reinforcement learning, which comprises: Step 1: The unmanned aerial vehicle, the base station and the ground station constitute an unmanned aerial vehicle communication network, and the data collected by the unmanned aerial vehicle in the specified airspace during task execution is transmitted to the base station through the unmanned aerial vehicle communication network, and then transferred to the ground station by the base station. Step 2: The ground station collects and detects the data sent by the unmanned aerial vehicle to identify whether an abnormal event occurs.

[0021] Optionally, the step 1 comprises: Establishing a local wireless network between the unmanned aerial vehicle and the base station, and enabling data transmission between any two nodes in the local wireless network; regarding the unmanned aerial vehicle and the base station as nodes in the local wireless network; When the unmanned aerial vehicle enters the specified airspace to perform the task, it actively searches for and joins the local wireless network, and updates the existing local wireless network to realize the calling of any unmanned aerial vehicle in the specified airspace at any time; when the unmanned aerial vehicle is in the blind area of the base station, the local wireless network in which the unmanned aerial vehicle is located is connected with the adjacent local wireless network, so that the unmanned aerial vehicle can communicate with the base station through other local wireless networks. The unmanned aerial vehicle transmits the data obtained by it to the ground station located in the same unmanned aerial vehicle communication network through the base station through the local wireless network.

[0022] Optionally, the unmanned aerial vehicle communication network further comprises an unmanned aerial vehicle ad hoc network; the unmanned aerial vehicle ad hoc network is composed of a plurality of unmanned aerial vehicles and a plurality of ground stations; when a new unmanned aerial vehicle or ground station wants to join the unmanned aerial vehicle ad hoc network, the existing unmanned aerial vehicle ad hoc network is expanded by adding a new node in the unmanned aerial vehicle ad hoc network.

[0023] Optionally, the step 1 further comprises: Real-time monitoring of various flight data of the unmanned aerial vehicle during task execution is realized by the unmanned aerial vehicle data monitoring technology, and the various flight data include flight mode, flight attitude, power load, power information, horizontal cruising speed, vertical climbing speed, height, GPS positioning information and camera gimbal angle.

[0024] Optionally, before the step 1, the method further comprises: Before the unmanned aerial vehicle flies, flight parameters of the unmanned aerial vehicle are set, the flight parameters of the unmanned aerial vehicle including flight height, voltage, communication interruption protection, unmanned aerial vehicle remote controller calibration, unmanned aerial vehicle GPS calibration, maximum climbing speed of the unmanned aerial vehicle in GPS mode, flight speed, maximum descending speed when returning.

[0025] Optionally, the step 2 comprises: The ground station collects data returned by the unmanned aerial vehicle and detects the trajectory, time and location of the non-cooperative unmanned aerial vehicle, accurately identifies relevant information; if an abnormal event is found, the commander and the operator are pushed to the problem information, so that the commander and the operator can discover the problem in time and handle the corresponding plan; the abnormal event includes intrusion alarm and deviation alarm; the problem information includes the model and number of the unmanned aerial vehicle where the abnormal event occurs.

[0026] Optionally, the accurate identification of relevant information comprises: The data returned by the unmanned aerial vehicle is classified by the trained classifier to identify the model in the data; If the trajectory, time and location of the non-cooperative unmanned aerial vehicle do not conform to the preset content, an abnormal event occurs in the task.

[0027] Optionally, the training process of the classifier comprises: Referring to Figure 2 Video data is extracted from the data collected by the unmanned aerial vehicle and used as a training sample, and the training sample is converted into a negative sample pair, and the negative sample is used for feature selection and extraction; the negative sample pair consists of two negative samples; the video data includes model, fuselage length, wing span, flight attitude and airborne equipment; The images in the video data are preprocessed, the window is slid on the preprocessed images, the images in the window are extracted according to the negative sample, and the extracted features are input into the classifier for training to obtain the trained classifier.

[0028] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them, although the present application has been described in detail with reference to the above examples, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered in the protection scope of the claims of the present application.

Claims

1. A high-precision identification method for low-altitude wide-area security risks based on reinforcement learning, characterized in that, include: Step 1: The drone, base station, and ground station form a drone communication network, and transmit the data collected by the drone during its mission in the designated airspace to the base station through the drone communication network, and then the base station relays it to the ground station; Step 2: The ground station collects and detects the data it receives from the drone to identify any abnormal events.

2. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 1, characterized in that, Step 1 includes: Establish a local wireless network between the drone and the base station, enabling data transmission between any two nodes in the local wireless network; both the drone and the base station are considered nodes in the local wireless network. When a drone enters a designated airspace to perform a mission, it actively searches for and joins the local wireless network and updates the existing local wireless network to enable it to call any drone in the designated airspace at any time. When a drone is in a base station blind spot, it establishes a connection between the local wireless network where the drone is located and its neighboring local wireless networks so that the drone can communicate with the base station through other local wireless networks. The drone transmits the data it acquires to the ground station located in the same drone communication network via a local wireless network.

3. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 2, characterized in that, The drone communication network also includes a drone self-organizing network; the drone self-organizing network consists of multiple drones and multiple ground stations; when a new drone or ground station wants to join the drone self-organizing network, the existing drone self-organizing network is expanded by adding nodes to the drone self-organizing network.

4. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 1, characterized in that, Step 1 further includes: By using drone data monitoring technology, various flight data of the drone can be monitored in real time during the drone's mission. These flight data include flight mode, flight attitude, power load, battery information, horizontal cruise speed, vertical climb speed, altitude, GPS positioning information, and camera gimbal angle.

5. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 1, characterized in that, Before step 1, the following are also included: Before the drone takes flight, its flight parameters are set, including flight altitude, voltage, communication interruption protection, drone remote controller calibration, drone GPS calibration, maximum climb rate in GPS mode, flight speed, and maximum descent rate during return flight.

6. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 1, characterized in that, Step 2 includes: The ground station collects and detects the trajectory, time, and location of non-cooperative drones from the data transmitted back by the drones, and accurately identifies relevant information. If an abnormal event is detected, the station pushes the problem information to the command personnel and operators so that the command personnel and operators can promptly discover the problem and take appropriate contingency measures. Abnormal events include intrusion alarms and yaw alarms. Problem information includes the model and serial number of the drone that caused the abnormal event.

7. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 6, characterized in that, The accurate identification of relevant information includes: The data transmitted back by the drone is classified by a trained classifier to identify the drone model in the data; If the trajectory, time, and location of the non-cooperative drone do not match the preset content, then an abnormal event has occurred in the mission.

8. The high-precision identification method for low-altitude wide-area security risks based on reinforcement learning according to claim 7, characterized in that, The training process of the classifier includes: Video data is extracted from the data collected by the drone and used as training samples. At the same time, the training samples are converted into negative sample pairs, and the negative samples are used for feature selection and extraction. A negative sample pair consists of two negative samples. The video data includes aircraft type, fuselage length, wingspan, flight attitude, and onboard equipment. The images in the video data are preprocessed. A window is slid across the preprocessed images. Features are extracted from the images in the window based on negative samples. The extracted features are then input into a classifier for training, resulting in a trained classifier.

Images