A multi-unmanned aerial vehicle based airborne intelligent processing system

Abstract

The application discloses a kind of based on multi-unmanned aerial vehicle's airborne intelligent processing system, by data processing module in each unmanned aerial vehicle, and with data processing module all communication connection data receiving module and situation autonomous perception module;Wherein, data receiving module is used to receive input data, data processing module is used to match the preset task template based on task list and target perception data, obtain task automatic control process data, generate payload control instruction based on task automatic control process data, situation autonomous perception module responds to multiple instructions of data processing module, respectively acquire multiple target perception data, multiple perception data include target detection identification positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, the present application is based on task template, parameterized cooperative reconnaissance task automatic driving mode, can assist multiple unmanned aerial vehicle system flexible configuration multiple combat task mode, so that unmanned aerial vehicle system combat has very strong flexibility.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle on-board data processing, and particularly relates to an on-board intelligent processing system based on multiple unmanned aerial vehicles. BACKGROUND

[0002] With the development of unmanned aerial vehicles, the application field of cluster unmanned aerial vehicles is gradually extensive. In the military field, unmanned aerial vehicles can be used for reconnaissance, attack, etc. In the civil field, cluster unmanned aerial vehicles can be used for environmental detection, emergency rescue, patrol monitoring, power inspection, etc. Cluster unmanned aerial vehicles have the characteristics of complex tasks, large number of unmanned aerial vehicles, multiple types of task loads, and limited communication bandwidth. In the traditional data processing mode, the ground command and control system is the only processing center of big data, receives multiple types of data transmitted by multiple unmanned aerial vehicles, and performs fusion processing and analysis, and the processing and analysis process completely depends on personnel. At present, the data processing mode with the ground as the only node cannot meet the use demand. The collaborative fusion of multiple task load data, the battlefield situation awareness of multiple source heterogeneous loads, and the bandwidth resource allocation are limited by bandwidth and cannot be completed by the ground control system. In addition, the intelligence of the unmanned aerial vehicle group platform is low in the aspects of multiple task load interface adaptation, storage scheduling distribution of multiple source task data, intelligent processing and automatic perception of multiple source load data, and collaborative reconnaissance control of loads. SUMMARY

[0003] The main purpose of the present application is to provide an on-board intelligent processing system based on multiple unmanned aerial vehicles, aiming to improve the automation and intelligence level of multiple task load interface adaptation, storage scheduling distribution of multiple source task data, intelligent processing and automatic perception of multiple source load data, and collaborative reconnaissance control of loads.

[0004] To achieve the above object, the application provides a multi-unmanned aerial vehicle (UAV) based airborne intelligent processing system, comprising: a data processing module arranged in each of the UAVs, and a data receiving module and a situation autonomous perception module in communication connection with the data processing module; wherein the data receiving module is configured to receive input data, the input data comprising source data, a target list of other UAVs and a task list, and send the source data, the target list of other UAVs and the task list to the data processing module, wherein the source data comprises optical image data, SAR image data, UAV pose information and payload attitude information; the data processing module is configured to receive input data, a target list of a local UAV and target perception data sent by the situation autonomous perception module, query a preset task template based on the task list to obtain task parameters, automatically calculate the task parameters based on the target perception data, obtain payload control instructions according to the task parameters, and distribute the payload control instructions to each payload; the situation autonomous perception module is configured to receive the input data, the target list of the local UAV and the payload control instructions, acquire a plurality of target perception data in response to a plurality of instructions of the data processing module, the plurality of target perception data comprising target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send the plurality of target perception data to the data processing module.

[0005] Optionally, the situation autonomous perception module comprises a target detection, identification and positioning sub-module configured to preprocess the optical image data and the SAR image data in response to a target detection and identification instruction to obtain pre-input data, process the pre-input data by calling a preset target detection algorithm to obtain first position information, and process the UAV pose information, the payload attitude information and the position of the target in the image based on a preset spatial coordinate conversion algorithm to obtain second position information, and send the first position information and the second position information to the data receiving module.

[0006] Optionally, the situation autonomous perception module further comprises a multi-source heterogeneous distributed perception sub-module configured to process the target list of other UAVs and the target list of the local UAV based on a preset target re-identification algorithm in response to a target association and fusion instruction to obtain a fusion target list, and send the fusion target list to the data processing module.

[0007] Optionally, the situation autonomous perception module further comprises an image guidance sub-module configured to acquire the second position information in response to an image guidance instruction, guide the UAV by using the second position information, correct an offset amount of the UAV payload relative to the first position information based on the first position information to obtain guidance information, and send the guidance information to the data processing module.

[0008] Optionally, the situation autonomous perception module further comprises a damage assessment submodule, configured to receive the optical image data and the SAR image data in response to a damage assessment instruction, obtain a damage assessment result of a target based on the optical image data and the SAR image data before and after the attack, and send the damage assessment result to the data processing module.

[0009] Optionally, the system further comprises a scene matching positioning module connected to the data processing module, configured to receive the visible light infrared video and the SAR image data sent by the data processing module, the locally stored map data, the unmanned aerial vehicle pose information and the payload attitude information under satellite denial conditions, in response to a scene matching instruction, call a preset image matching algorithm to process the visible light infrared video and the SAR image data and the map data, obtain target positioning information and image map matching information, and based on a rear intersection algorithm, process the unmanned aerial vehicle pose information, the payload attitude information and the image map matching information to obtain unmanned aerial vehicle positioning information, and send the unmanned aerial vehicle positioning information and the target positioning information to the data processing module.

[0010] Optionally, the system further comprises a heterogeneous reconnaissance information correlation tracking module connected to the data processing module, configured to respond to a reconnaissance tracking instruction, use a radar type unmanned aerial vehicle to conduct long distance reconnaissance on a target in a task area to obtain a first target list, use the target positioning information to guide an optoelectronic type unmanned aerial vehicle to the task area to obtain a second target list, process the first target list and the second target list based on a preset target detection and recognition algorithm to obtain a final target ID number, and send the target ID number to the data processing module.

[0011] Optionally, the system further comprises a weak network based data compression module connected to the situation autonomous perception module and the data processing module respectively, configured to receive the fusion target list, in response to an image guidance instruction, perform image low magnification compression on a center field region of a video image in the fusion target list and high magnification compression on the remaining regions to obtain compressed video, and send the compressed video to the data processing module.

[0012] Optionally, the system further comprises a heterogeneous payload service customization module connected to the data processing module, configured to obtain a payload state frame and the task list, parse the payload state frame to obtain a payload type, and obtain a customized payload service according to the task list and the payload type, and based on the customized payload service, call a pre-constructed target model and a corresponding detection and recognition algorithm.

[0013] Optionally, the data processing module further comprises a database management submodule, configured to sort data in a target list based on priority, and add, delete, modify and query data in the target list.

[0014] The unmanned aerial vehicle airborne intelligent processing system provided by the embodiment of the application comprises a data processing module arranged in each unmanned aerial vehicle, and a data receiving module and a situation autonomous perception module which are both in communication connection with the data processing module; the data receiving module is used for receiving input data, the input data comprising source data, a target list of other unmanned aerial vehicles and a task list, and sending the source data, the target list of other unmanned aerial vehicles and the task list to the data processing module, wherein the source data comprises optical image data, SAR image data, unmanned aerial vehicle pose information and payload attitude information; the data processing module is used for receiving the input data, a target list of a local unmanned aerial vehicle and target perception data sent by the situation autonomous perception module, matching a preset task template based on the task list and the target perception data, obtaining task automatic control flow data, generating payload control instructions based on the task automatic control flow data, and distributing the payload control instructions to each payload; the situation autonomous perception module is used for receiving the input data, the target list of the local unmanned aerial vehicle and the payload control instructions, acquiring a plurality of target perception data in response to a plurality of instructions of the data processing module, the plurality of perception data comprising target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and sending the plurality of target perception data to the data processing module; based on the task template and a parameterized cooperative reconnaissance task automatic driving mode, the unmanned aerial vehicle system can be flexibly configured with various combat task modes, so that the unmanned aerial vehicle system has strong flexibility in combat. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A structural block diagram of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application is shown in FIG. 1;

[0016] Figure 2 A task automation driving function implementation flowchart of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application is shown in FIG. 2;

[0017] Figure 3 A situation autonomous perception module structural block diagram of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application is shown in FIG. 3;

[0018] Figure 4 A heterogeneous reconnaissance information correlation tracking implementation flowchart of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application is shown in FIG. 4;

[0019] Figure 5 A scene matching positioning implementation flowchart of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application is shown in FIG. 5;

[0020] Figure 6 A data transmission implementation flowchart of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the application based on a weak network is shown in FIG. 6;

[0021] Figure 7 The heterogeneous load service customization module of the unmanned aerial vehicle airborne intelligent processing system provided by an embodiment of the present application.

[0022] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0024] The purpose of the present application is to use a high-computing-power, lightweight hardware integrated design framework for the airborne intelligent task processing system based on a multi-unmanned aerial vehicle group to solve the problems of low intelligent level of multi-task load replacement, multi-source task data storage scheduling distribution, multi-source load data intelligent processing automatic perception and task equipment cooperative reconnaissance control.

[0025] The airborne intelligent processing system, as an airborne processing device, has great advantages compared to ground centralized processing devices. After obtaining the reconnaissance data source, the airborne intelligent processing system only needs to perform automatic and intelligent processing on the airborne edge to generate target perception results, battlefield situation and other information, thereby greatly reducing the bandwidth demand and improving the intelligent level of the unmanned aerial vehicle.

[0026] Reference Figure 1 The present application provides an airborne intelligent processing system based on a multi-unmanned aerial vehicle, comprising: a data processing module 10 arranged in each of the unmanned aerial vehicles, and a data receiving module 20 and a situation autonomous perception module 30 which are both in communication connection with the data processing module 10, wherein the data receiving module 20 is used to receive input data, the input data including source data, a target list of other unmanned aerial vehicles and a task list, and send the source data, the target list of other unmanned aerial vehicles and the task list to the data processing module 10, wherein the source data includes optical image data, SAR image data, unmanned aerial vehicle pose information and load attitude information; the data processing module 10 is used to receive input data, a target list of a local machine and target perception data sent by the situation autonomous perception module 30, query a preset task template based on the task list to obtain task parameters, automatically calculate the task parameters based on the target perception data, obtain load control instructions according to the task parameters, and distribute the load control instructions to each load; the situation autonomous perception module 30 is used to receive the input data, the target list of the local machine and the load control instructions, acquire a plurality of target perception data in response to a plurality of instructions of the data processing module 10, the plurality of target perception data including target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send the plurality of target perception data to the data processing module 10.

[0027] Exemplarily, the airborne intelligent processing system receives input data from the outside through the data receiving module 20, wherein the input data can include a task list, inertial navigation attitude data, and reconnaissance data of various task devices. The airborne intelligent processing system obtains a target list, task automatic driving instructions, scene matching positioning data, and damage assessment data through the data processing module 10, and sends the above data to the unmanned aerial vehicle system. In addition, the airborne intelligent processing system also performs lightweight database management on the data in the target list generated by the situation autonomous perception module 30 in the system. The management of the database includes prioritizing the data in the target list, and adding, deleting, modifying, and querying the data in the target list.

[0028] In a specific embodiment, the data processing module 10 processes data in a multi-threaded manner, in other words, the reception, processing, and distribution of data all have independent thread tasks, wherein the data interaction between threads adopts a shared memory method. The unmanned aerial vehicle has a large-capacity storage disk design, and a SQLite database is deployed in the airborne intelligent processing system. The SQLite database can efficiently process the storage, query, and retrieval of target list data.

[0029] It should be noted that the airborne intelligent processing system arranged in each unmanned aerial vehicle can receive task lists, inertial navigation data, and airborne avionics data in real time through a CAN interface, receive reconnaissance data of an optical-electric load through an SDI interface, and receive reconnaissance data of a radar reconnaissance load through a LAN interface. After each interface of the airborne intelligent processing system receives data, the data is stored in the corresponding memory area through shared memory, and when each sub-thread detects data, data processing begins. The target list, scene matching positioning data, or damage assessment results generated after the data processing module 10 processes data can be stored in the database, and users can access the database through a SQLite interface. Users can retrieve and obtain data from the database by configuring confidence, detection type, and number. In addition, the airborne intelligent processing system also designs a gigabit network port, through which the airborne intelligent processing system can distribute target list, scene matching positioning data, or damage assessment result data to the unmanned aerial vehicle system.

[0030] In the embodiment, the airborne intelligent processing system receives the task list distributed by the unmanned aerial system, reads the current task mode, which can be persistent reconnaissance, guided attack, damage assessment or satellite guidance denial. The airborne intelligent processing system calculates the parameters for executing the task according to the payload reconnaissance requirements, for example, the parameters of the photoelectric type reconnaissance payload can be reconnaissance field of view, pitch angle, single machine width, etc. The airborne intelligent processing system formulates an automatic control process according to the system task stage, and finally forms a payload task instruction set including visible light field of view control, infrared field of view control, single machine width angle, pitch angle, etc. The unmanned aerial system automatically drives the task payload through the task instruction set to assist the cluster unmanned aerial system to perform task cooperative control and task cooperative planning.

[0031] Reference Figure 2 For example, the data processing module 10 receives the task list relayed by the unmanned aerial system through the network port, searches the pre-bound task template according to the task type, calls the configuration parameters of the template, and calculates the payload instructions of the current task. Taking the "persistent reconnaissance" task stage as an example, the specific implementation method is described. After the airborne intelligent processing system receives the task list, it is parsed through the protocol that the current task mode is persistent reconnaissance, and the task template calculation parameters are queried, including area size, area position, traversal level, target size estimation, target distance estimation, flight height, etc. The data processing module 10 automatically calculates the driving parameters according to the type and capability of the payload. Taking the single traversal task mode of the photoelectric reconnaissance payload as an example: in order to realize the photoelectric persistent reconnaissance task, the instruction parameters of the photoelectric equipment include pitch angle, reconnaissance field of view, and single machine width:

[0032] The pitch angle is calculated as follows: Where H is the flight height and L is the target distance.

[0033] The reconnaissance field of view is calculated as follows: Where A is the field of view, L is the target distance, a is the target size, m is the number of pixels, and n is the number of detector pixels.

[0034] The single machine width is calculated as follows: Where M is the single machine width angle, D is the area width, and fn is the number of aircrafts. The airborne intelligent processing system programs the driving parameters (F, A, M) obtained by calculation according to the protocol, judges whether the current position of the unmanned aerial vehicle is within the photoelectric action distance L from the task area, and if so, the instruction set is sent to the photoelectric equipment through the serial port. After receiving the instruction, the photoelectric equipment drives the corresponding payload component to execute the persistent reconnaissance task, and cooperates with other unmanned aerial vehicles in the formation to complete the persistent reconnaissance task.

[0035] Reference Figure 3In the embodiment of the present application, the situation autonomous perception module 30 comprises a target detection and identification positioning sub-module 301, and the heterogeneous load service customization module 301 is configured to preprocess the optical image data and the SAR image data in response to a target detection and identification instruction to obtain pre-input data, call a preset target detection algorithm to process the pre-input data to obtain first position information, and process the unmanned aerial vehicle pose information, the load attitude information and the position of the target in the image based on a preset spatial coordinate conversion algorithm to obtain second position information, and send the first position information and the second position information to the data receiving module 20.

[0036] Specifically, the situation autonomous perception module 30 intelligently processes battlefield reconnaissance information through photoelectric reconnaissance load and radar reconnaissance load, and the intelligent processing includes target detection and identification positioning, multi-source heterogeneous distributed perception, image guidance and damage assessment.

[0037] The target detection and identification positioning sub-module 301 receives visible light and infrared video forwarded by the data processing module 10 through the MIPI interface of the data processing module 10, and receives SAR images forwarded by the internal network interface. The situation autonomous perception module 30 calls a target detection and identification algorithm library to perform target detection and identification, and obtains a target detection and identification result, which includes a target category, a confidence and a pixel position of a single image (first position information). The target detection and identification positioning sub-module 301 uses the attitude of the photoelectric, a target ranging value, an unmanned aerial vehicle position and an attitude, and calculates an absolute position of a single target through spatial coordinate conversion. The matching result with a high-precision electronic map is used to calculate an absolute position of multiple targets (second position information). The target detection and identification positioning sub-module 301 sends a target list formed by the target detection result to the data processing module according to a target library format.

[0038] In actual execution, the target detection and identification positioning sub-module 301 takes the target detection and identification algorithm as the premise, detects and identifies the photoelectric / SAR image to obtain the category and position information of the target, and carries out tracking (photoelectric) and target positioning on the basis thereof. The implementation method is as follows: the target detection and identification positioning sub-module 301 receives optical image data, SAR image data, unmanned aerial vehicle pose information, and load attitude information from the data processing module 10. The target detection and identification positioning sub-module 301 receives a detection and identification instruction from the data processing module 10. The target detection and identification positioning sub-module 301 reads the optical image data and SAR image data, and pre-processes the image to obtain pre-input data, wherein the pre-input data is target detection algorithm input data. The target detection and identification positioning sub-module 301 calls the board card computing resource to execute the target detection algorithm to obtain the target type, target confidence, and target position in the image (first position information). The target detection and identification positioning sub-module 301 calculates the target geographical position (first position information) through the target position, unmanned aerial vehicle position and attitude, and load attitude information. The target detection and identification positioning sub-module 301 removes repeated targets for the detected target.

[0039] With reference to the foregoing Figure 3 In an embodiment of the present application, the situation autonomous perception module 30 further comprises a multi-source heterogeneous distributed perception sub-module 302, which is configured to process the target list of the other unmanned aerial vehicles and the target list of the local unmanned aerial vehicle based on a preset target re-identification algorithm in response to a target association fusion instruction, obtain a fusion target list, and send the fusion target list to the data processing module 10.

[0040] In actual implementation, the target association function of the situation autonomous perception module 30 realizes single-machine heterogeneous target fusion and multi-machine homogeneous target fusion. The implementation method is as follows: the situation autonomous perception module 30 receives the local target list, the feature list, and the target list and the feature list of the other unmanned aerial vehicles from the data processing module 10, and takes the local target features and the target features of the other unmanned aerial vehicles as the input of the target re-identification algorithm (ReID). The situation autonomous perception module 30 receives the target association fusion instruction forwarded by the data processing module, calls the computing resource to perform target re-identification, and obtains the photoelectric multi-machine target list after fusion and deduplication. The situation autonomous perception module 30 sends the fused target list to the data processing module.

[0041] The multi-source heterogeneous distributed perception sub-module 302 takes the result of single-machine detection and identification as a target list, takes the result of photoelectric / SAR detection and identification of the other unmanned aerial vehicles transmitted by the inter-machine communication device as another target list, and performs target re-identification on the two target identification results, and outputs the target reconnaissance result of the task area.

[0042] Reference Figure 4 In the embodiments of the present application, the unmanned aerial vehicle on-board intelligent processing system further comprises a heterogeneous reconnaissance information correlation tracking module 50, which is connected to the data processing module 10 and is configured to, in response to a reconnaissance tracking instruction, use a radar-type unmanned aerial vehicle to perform long-distance reconnaissance on targets in a task area to obtain a first target list, use the target positioning information to guide an optical-electrical type unmanned aerial vehicle to the task area to obtain a second target list, process the first target list and the second target list based on a preset target detection and recognition algorithm to obtain a final target ID number, and send the target ID number to the data processing module 10.

[0043] Exemplarily, the target information in the target list detected by each aircraft includes a target type and a target position, and the target position has a certain positioning error. Considering that the error sources of each aircraft are the same, the relative position error between the reconnaissance targets of the same aircraft is small, and the topological relationship of the target positions detected by each aircraft is used to realize the inter-aircraft target correlation. The target position topological relationship is obtained by calculating the position difference between two targets of the same aircraft. The target list data formed by the A unmanned aerial vehicle is as follows:

[0044] Target ID Target position Target slice A1 AT1 (B, L, H) T_Pic1 A2 AT2 (B, L, H) T_Pic2 A3 AT3 (B, L, H) T_Pic3 --- --- ---

[0045] The relative horizontal position of target 1 and target 2 is D (AT1，AT2) , the relative horizontal position of target 2 and target 3 is D (AT3，AT2) , and the relative horizontal position of target 1 and target 3 is D (AT1，AT3) . Thus, the relative distance interpolation of target 1 is calculated as follows:

[0046] The target list data formed by the B unmanned aerial vehicle is as follows:

[0047] Target ID Target position Target slice B1 BT1 (B, L, H) BT_Pic1 B3 BT2 (B, L, H) BT_Pic2 B3 BT3 (B, L, H) BT_Pic3 --- --- ---

[0048] The relative horizontal position of target 1 and target 2 detected by the B unmanned aerial vehicle is D BBT1，BT2) , the relative horizontal position of target 2 and target 3 is D (NT3，NT2) , and the relative horizontal position of target 1 and target 3 is D (BT1，BT3) . Thus, the relative distance interpolation of target 1 is calculated as follows:

[0049] If So the target 1 seen by B machine is the target 1 seen by A machine, and so on, to calculate all the target, determine the ID number of other targets. Then according to the part of the adjacent area information retained in the target image slice, through the registration of the relatively large scale scene, the secondary identification of the target is assisted to be realized. The registration method based on mutual information is adopted to realize the scene image registration between unmanned aerial vehicles. Finally, the target ID number is determined.

[0050] Reference Figure 3 And Figure 5 In the embodiment of the present application, the situation autonomous perception module 30 further comprises an image guidance sub-module 303, which is configured to acquire the second position information in response to an image guidance instruction, guide the unmanned aerial vehicle by using the second position information, correct the offset of the payload of the unmanned aerial vehicle relative to the first position information based on the first position information, obtain guidance information, and send the guidance information to the data processing module 10.

[0051] In actual execution, the situation autonomous perception module 30 receives a target guidance instruction, guides the photoelectric payload to complete geographic guidance. Then, the video photographed by the photoelectric device is detected, re-identified, and single-target tracked, and the photoelectric reconnaissance payload is continuously outputted with the offset of the target relative to the image center, and the photoelectric device is cooperated to complete the guidance.

[0052] Specifically, the image guidance sub-module 303 receives the image guidance instruction forwarded by the data processing module 10, reads the guidance target ID in the instruction, queries the database according to the target ID, obtains the target initial positioning and target slice features, and sends the target initial positioning information to the photoelectric payload according to the protocol. The photoelectric payload performs geographic guidance according to the positioning information, so that the target reappears in the image. The image guidance sub-module 303 receives the image data forwarded by the data processing module, continuously performs target detection, identification and tracking. When a new target is detected, the new target and the guidance target are re-identified; when it is determined that they are the same target, the target is continuously detected and tracked to obtain the deviation of the target from the image center pixel, and the deviation is sent to the data processing module 10 in real time.

[0053] Continue to refer to Figure 3 In the embodiment of the present application, the situation autonomous perception module 30 further comprises a damage assessment sub-module 304, which is configured to receive the optical image data and the SAR image data in response to a damage assessment instruction, obtain a damage assessment result of a target based on the optical image data and the SAR image data before and after the attack, and send the damage assessment result to the data processing module 10.

[0054] The damage assessment submodule 304 receives image data and damage assessment instructions from the data processing module. The target motion attribute damage is judged according to the motion before and after the attack. The damage proportion is evaluated according to the proportion of smoke and fire light in the target detection area after the attack. The vital part of the target is identified according to the detection result, and the physical part damage is evaluated.

[0055] The damage assessment submodule 304 can evaluate the damage degree of the battlefield through the change information of the reconnaissance image before and after the damage. After receiving the damage assessment instruction, the damage assessment submodule 304 calls the damage analysis and evaluation module to start the task.

[0056] The damage assessment submodule 304 realizes three damage assessment methods, including motion attribute damage assessment, damage proportion assessment and physical part damage assessment.

[0057] The motion attribute damage assessment can receive the automatic damage assessment instruction and image data forwarded by the data processing module. The target detection algorithm is called for detection and identification. The background motion interference is eliminated by registration, and the current motion attribute of the target is obtained by using the frame difference method. The motion attribute change of the target before and after the attack is output.

[0058] The physical part damage assessment can receive the automatic damage assessment instruction and image data forwarded by the data processing module. The target detection algorithm and vital part detection algorithm are called. The detection result of the vital part in the target frame is output.

[0059] The damage proportion assessment can receive the automatic damage assessment instruction and image data forwarded by the data processing module. The target list is received. The target frame position is obtained by the target detection algorithm before the attack. The position of smoke and fire light in the image is obtained by the smoke and fire light detection algorithm after the attack. It is judged whether the target is hit. If it is hit, the proportion of the damage area to the target area is continuously calculated by the change detection method.

[0060] Reference Figure 1 In the embodiment of the present application, the unmanned aerial vehicle on-board intelligent processing system further comprises a scene matching positioning module 40 connected to the data processing module 10, for receiving visible light infrared video and SAR image data, local storage map data, unmanned aerial vehicle pose information and load attitude information sent by the data processing module 10 under satellite denial conditions, responding to a scene matching instruction, calling a preset image matching algorithm to process the visible light infrared video and SAR image data and the map data, obtaining target positioning information and image map matching information, and based on the rear intersection algorithm, processing the unmanned aerial vehicle pose information, load attitude information and image map matching information to obtain unmanned aerial vehicle positioning information, and sending the unmanned aerial vehicle positioning information and the target positioning information to the data processing module 10.

[0061] In the satellite denial environment, the scene matching positioning module 40 receives the scene matching instruction forwarded by the data processing module, the scene matching positioning module 40 receives the original reconnaissance image forwarded by the data processing function module, the original reconnaissance image can include visible infrared video and SAR image, the high-precision electronic map stored locally is used as a matching base map, and the feature correspondence between the two images is obtained based on the image matching algorithm, so that the target positioning result can be calculated. The scene matching positioning module 40 uses the optical load attitude, the position and attitude of the unmanned aerial vehicle, and calculates the position of the unmanned aerial vehicle by the resection method. According to the protocol, the target and the unmanned aerial vehicle position are sent to the data processing module 10.

[0062] In the actual implementation process, the scene matching positioning module 40 receives the scene matching instruction and optical image data / SAR image data forwarded by the data processing module 10, reads the target detection result provided by the situation autonomous perception module 30; reads the local electronic map of the task processing board (tiff format map, precision 1 meter), and performs unmanned aerial vehicle coarse positioning through the unmanned aerial vehicle position, attitude and load attitude, and performs map clipping;

[0063] The scene matching positioning module 40 inputs the image data and the map clipping data into the matching algorithm, and calls the board card computing resource to perform matching algorithm reasoning to obtain the feature point pairing relationship between the image data and the electronic map. In the first matching, the local electronic map of the task processing board is read to traverse the scene matching. According to the geographical position characteristics of the electronic map, the target positioning result can be calculated through the feature point pairing relationship between the electronic map and the reconnaissance image, and high-precision positioning of the target is realized. In addition, through the feature point pairing relationship, the load attitude, the unmanned aerial vehicle attitude, and the resection method, the unmanned aerial vehicle positioning result can be calculated, and the scene matching positioning module 40 outputs the aircraft, target positioning result and time to the data processing module.

[0064] In the embodiment of the application, the unmanned aerial vehicle on-board intelligent processing system further comprises a weak network-based data compression module 60, which is connected with the situation autonomous perception module 30 and the data processing module 10 respectively, and is used for receiving the fusion target list, performing image low-magnification compression on the field center region of the video image in the fusion target list in response to the image guidance instruction, performing high-magnification compression on the remaining regions, obtaining compressed video, and sending the compressed video to the data processing module 10.

[0065] Reference Figure 6Specifically, the weak network data compression module 60 receives the target result output by the situation autonomous perception module 30, slices and compresses the target image, integrates the target ID, target position, attribute information and recognition confidence rate, and forms a target list according to the target list protocol. After the weak network data compression module 60 receives the image guidance instruction sent by the data processing module, the image low magnification compression is performed on the center region of the field of view in the video image, and the high magnification compression is performed on the remaining regions.

[0066] In actual implementation, after the weak network data compression module 60 receives the target result data forwarded by the situation autonomous perception module 30, the key information in the data is first extracted to form target simplified information, wherein the target simplified information includes target number, target type, target position (longitude, latitude and height), etc. Secondly, the width and height of the target in the target result data are each expanded by 20 pixels, and then the slicing is performed in the image according to the pixel position (target center point position, target width and height). Then, the sliced image is compressed by JPEG according to the image size control within a certain data size. Finally, the target simplified information and the target slice are combined to form a single target list, and are sent to the data processing module.

[0067] The weak network data compression module 60 adjusts the video window size according to the control instruction, and obtains the ROI image evaluation value in real time through information entropy statistics. On the basis of the original framework of the H.265 encoder, the weak network data compression module 60 realizes optimization by improving the H.265 encoder architecture under the premise of real-time judgment of the ROI local area image quality. The weak network data compression module 60 uses the Peak Signal Noise Ratio (PSNR) and Root Mean Square Error (MSE) methods for ROI image evaluation.

[0068] The weak network data compression module 60 determines whether the macro block is in the ROI range through the encoder. For the macro block of the ROI part, a lower quantization parameter is set, and the code rate control part is modified to obtain more bits. The non-ROI part obtains fewer bits. In this way, the quantization range is refined from one frame to one macro block.

[0069] Exemplarily, the weak network data compression module 60 can use the encoding mode of H.265 to perform H.265 compression after the data processing module receives and analyzes the video data. The width and height of the resolution instruction are read, and a new encoder is reinitialized by the video compression module according to the width and height. The new encoder is used for hard compression. The video data compressed by the new encoder is transmitted to the data processing module 10 through the shared memory.

[0070] Reference Figure 7In the embodiment of the present application, the unmanned aerial vehicle on-board intelligent processing system further comprises a heterogeneous payload service customization module 70 connected to the data processing module 10, for obtaining a payload state frame and the task list, parsing the payload state frame to obtain a payload type, and obtaining a customized payload service according to the task list and the payload type, and calling a pre-constructed target model and a corresponding detection and identification algorithm based on the customized payload service.

[0071] The heterogeneous payload service customization module 70 automatically identifies the payload type by parsing the device type code of the payload state frame. The heterogeneous payload service customization module 70 calls the control module of each payload according to the payload type. According to the task phase and the task target type sent by the data processing module 10, the corresponding algorithm and target model are called, wherein the algorithm includes scene matching, situation awareness, etc.

[0072] In actual execution, the heterogeneous payload service customization module 70 standardizes the design of the payload hanging point interface in the physical layer and the protocol layer for different types of unmanned aerial vehicle systems. The communication interface of the heterogeneous payload service customization module 70 adopts CAN bus, the image interface adopts network port design, and the same protocol framework is adopted. When the unmanned aerial vehicle system is powered on, the task processing board can automatically interpret the type of the payload through the state word in the communication protocol. The heterogeneous payload service customization module 70 realizes the customization of the payload service through the scheduling of the algorithm module and the loading of the target model according to the combat task phase and the task target type sent by the data processing module 10. The target model is mainly classified according to the reconnaissance source and the target type, the reconnaissance source includes visible light, infrared, radar SAR, the target type includes armored vehicles, aircraft, ships, typical buildings, vital parts, fire, etc., and different target models are loaded according to different reconnaissance payload types, task target types, and task modes. When the unmanned aerial vehicle is in system reconnaissance operation, the target detection and identification is realized by calling the task target model. When damage assessment is performed, the task target detection model, the vital part model, and the fire detection model are called. The heterogeneous payload service customization module 70 automatically schedules the corresponding algorithm module based on the task control process according to the payload type and the task type, including the situation awareness function module, the scene matching module in denial environment, and the damage assessment module. The specific algorithms include: typical target detection and identification based on artificial intelligence of photoelectricity, non-typical target detection and identification based on matching of photoelectricity, vital part detection and identification of photoelectricity, radar SAR target detection and identification, multi-target positioning, radar SAR scene matching positioning, photoelectricity scene matching positioning, radar SAR damage assessment, and photoelectricity damage assessment, etc.

[0073] In the embodiments of the present application, the data processing module 10 further comprises a database management submodule 101 for priority-based sorting of data in the target list, and adding, deleting, modifying and querying data in the target list

[0074] The above merely describes the preferred embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An unmanned aerial vehicle (UAV) onboard intelligent processing system, comprising: The utility model relates to a kind of unmanned aerial vehicle (UAV) task management system, including: Data processing module arranged in each of the unmanned aerial vehicle, and data receiving module and situation autonomous perception module are connected with the data processing module communication; Wherein, the data receiving module is used to receive input data, and the input data is sent to the data processing module, wherein the input data includes source data, target list and task list of other unmanned aerial vehicles, and the source data includes optical image data, SAR image data, unmanned aerial vehicle pose information, load attitude information; The data processing module is used to receive the input data, target list of local machine and target perception result data sent by the situation autonomous perception module, obtain task parameters based on the task list query preset task template, automatically calculate task parameters based on the target perception result data, obtain load control instruction according to the task parameters, and distribute the load control instruction to each load and the situation autonomous perception module; The situation autonomous perception module is used to receive the input data, target list of local machine and load control instruction, respectively obtain a plurality of target perception result data in response to a plurality of instructions of the data processing module, and send a plurality of target perception result data to the data processing module, wherein a plurality of target perception result data includes target detection and identification positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data; Further comprising: heterogeneous reconnaissance information association tracking module, connected with the data processing module, for responding to reconnaissance tracking instruction, using radar type unmanned aerial vehicle to carry out long-distance reconnaissance to target in task area, obtaining first target list, using the target positioning information to guide photoelectric type unmanned aerial vehicle to the task area, obtaining second target list, based on preset target detection and identification algorithm processing the first target list and the second target list, obtaining final target ID number, and sending the target ID number to the data processing module.

2. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, The situation autonomous perception module includes: Target detection and identification positioning submodule, for responding to target detection and identification instruction, pre-processing the optical image data and the SAR image data to obtain pre-input data, calling preset target detection algorithm to process the pre-input data to obtain first position information, and based on preset spatial coordinate conversion algorithm processing the unmanned aerial vehicle pose information, the load attitude information and the position of target in image to obtain second position information, and sending the first position information and the second position information to the data receiving module.

3. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, The situation autonomous perception module further includes: Multi-source heterogeneous distributed perception submodule, for responding to target association fusion instruction, based on preset target re-identification algorithm processing the target list of other unmanned aerial vehicles and the target list of local machine to obtain fusion target list, and sending the fusion target list to the data processing module.

4. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, The situation autonomous perception module further includes: An image guidance sub-module is configured to acquire second position information in response to an image guidance instruction, guide the UAV using the second position information, correct an offset of the UAV payload relative to the first position information based on the first position information to obtain guidance information, and send the guidance information to the data processing module.

5. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, The autonomous situation awareness module further comprises: A damage assessment sub-module is configured to receive the optical image data and the SAR image data in response to a damage assessment instruction, obtain a damage assessment result of a target based on the optical image data and the SAR image data before and after the attack, and send the damage assessment result to the data processing module.

6. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, Further comprising: A scene matching positioning module is connected to the data processing module and is configured to receive the visible light infrared video and the SAR image data, the locally stored map data, the UAV pose information, and the payload attitude information sent by the data processing module under satellite denial conditions, call a preset image matching algorithm to process the visible light infrared video and the SAR image data and the map data in response to a scene matching instruction, obtain target positioning information and image map matching information, process the UAV pose information, the payload attitude information, and the image map matching information based on a rear intersection algorithm to obtain UAV positioning information, and send the UAV positioning information and the target positioning information to the data processing module.

7. The unmanned aerial vehicle onboard intelligent processing system of claim 3, wherein, Further comprising: A weak network-based data compression module is connected to the autonomous situation awareness module and the data processing module and is configured to receive the fusion target list, perform image low-magnification compression on a field center region of a video image in the fusion target list and high-magnification compression on other regions in response to an image guidance instruction to obtain compressed video, and send the compressed video to the data processing module.

8. The unmanned aerial vehicle onboard intelligent processing system of claim 3, wherein, Further comprising: A heterogeneous payload service customization module is connected to the data processing module and is configured to acquire a payload state frame and the task list, analyze the payload state frame to obtain a payload type, obtain a customized payload service based on the task list and the payload type, and call a pre-built target model and a corresponding detection and recognition algorithm based on the customized payload service.

9. The unmanned aerial vehicle onboard intelligent processing system of claim 1, wherein, The data processing module further comprises: A database management sub-module is configured to sort data in a target list based on priority, and add, delete, modify, and query data in the target list.

Images