Airborne intelligent processing system based on multiple unmanned aerial vehicles

Abstract

The invention discloses an airborne intelligent processing system based on multiple unmanned aerial vehicles, and the system comprises data processing modules in all unmanned aerial vehicles, and a data receiving module and a situation autonomous perception module which are in communication connection with the data processing modules. Wherein the data receiving module is used for receiving input data, and the data processing module is used for matching a preset task template based on a task list and target sensing data to obtain task automatic control flow data, generating a load control instruction based on the task automatic control flow data, and sending the load control instruction to the server. The situation autonomous perception module responds to the multiple instructions of the data processing module, multiple pieces of target perception data are obtained, and the multiple pieces of perception data comprise target detection, recognition and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage evaluation data. And a parameterized cooperative reconnaissance task automatic driving mode can assist the multi-unmanned aerial vehicle system to flexibly configure various combat task modes, so that the combat of the unmanned aerial vehicle system has very high flexibility.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned aerial vehicle (UAV) airborne data processing, and in particular to an airborne intelligent processing system based on multiple UAVs. Background Art

[0002] With the development of drones, the application fields of swarm drones are becoming increasingly extensive. In the military field, drones can be used for reconnaissance and strikes. In the civilian field, swarm drones can be used for environmental testing, emergency rescue, patrol monitoring, power inspection, etc. Swarm drones have the characteristics of complex tasks, large number of drones, multiple types of mission payloads, and limited communication bandwidth. In the traditional data processing method, the ground command and control system, as the only processing center for big data, receives multiple data transmitted by multiple drones and performs fusion processing and analysis. The processing and analysis process is completely dependent on personnel. At present, the data processing mode with the ground as the only node can no longer meet the use needs. The collaborative fusion of multi-task payload data, battlefield situation awareness of multi-source heterogeneous payloads, and bandwidth resource allocation cannot be completed by the ground control system due to bandwidth limitations. In addition, the automation and intelligence level of the drone swarm platform is low in terms of multi-task payload interface adaptation, storage scheduling and distribution of multi-source mission data, intelligent processing and automatic perception of multi-source payload data, and payload collaborative reconnaissance control. Summary of the invention

[0003] The main purpose of this application is to provide an airborne intelligent processing system based on multiple UAVs, aiming to improve the automation and intelligence levels of multi-task payload interface adaptation, storage, scheduling and distribution of multi-source mission data, intelligent processing and automatic perception of multi-source payload data, and payload collaborative reconnaissance and control.

[0004] To achieve the above-mentioned purpose, the present application provides an airborne intelligent processing system based on multiple UAVs, comprising: a data processing module arranged in each of the UAVs, and a data receiving module and a situation autonomous perception module which are communicatively connected to the data processing module; wherein the data receiving module is used to receive input data, the input data includes source data, target lists and task lists of other UAVs, and send the source data, target lists and task lists of other UAVs to the data processing module, wherein the source data includes optical image data, SAR image data, UAV posture information, and payload posture information; the data processing module is used to receive input data, the target list of the local machine, and the target list of the local machine. The target perception data sent by the situation autonomous perception module is used to query the preset task template based on the task list to obtain task parameters, automatically calculate the task parameters based on the target perception data, obtain the load control instructions according to the task parameters, and distribute the load control instructions to each load; the situation autonomous perception module is used to receive the input data, the target list of the local machine and the load control instructions, respond to multiple instructions of the data processing module, respectively obtain multiple target perception data, the multiple perception data include target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send the multiple target perception data to the data processing module.

[0005] Optionally, the situation autonomous perception module includes: a target detection, identification and positioning submodule, which is used to pre-process the optical image data and the SAR image data in response to a target detection and identification instruction to obtain pre-input data, call a preset target detection algorithm to process the pre-input data to obtain first position information, and process the drone posture information, the payload posture information and the position of the target in the image based on a preset spatial coordinate conversion algorithm to obtain second position information, and send the first position information and the second position information to the data receiving module.

[0006] Optionally, the situational autonomous perception module also includes: a multi-source heterogeneous distributed perception sub-module, which is used to respond to a target association fusion instruction, process the target list of other drones and the target list of the local drone based on a preset target re-identification algorithm, obtain a fused target list, and send the fused target list to the data processing module.

[0007] Optionally, the situation autonomous perception module also includes: an image guidance submodule, which is used to respond to an image guidance instruction, obtain the second position information, use the second position information to guide the UAV, correct the offset of the UAV payload relative to the first position information based on the first position information, obtain guidance information, and send the guidance information to the data processing module.

[0008] Optionally, the situation autonomous perception module also includes: a damage assessment submodule, which is used to respond to a damage assessment instruction, receive the optical image data and the SAR image data, obtain a damage assessment result of the target based on the optical image data and the SAR image data before and after the strike, and send the damage assessment result to the data processing module.

[0009] Optionally, it also includes: a scene matching positioning module, connected to the data processing module, used to receive the visible light infrared video and SAR image data, locally stored map data, UAV posture information and payload attitude information sent by the data processing module under satellite denial conditions, and in response to a scene matching instruction, call a preset image matching algorithm to process the visible light infrared video and SAR image data, the map data, to obtain target positioning information and image map matching information, and process the UAV posture information, payload attitude information and image map matching information based on a rear handover algorithm to obtain UAV positioning information, and send the UAV positioning information and the target positioning information to the data processing module.

[0010] Optionally, it also includes: a heterogeneous reconnaissance information association tracking module, connected to the data processing module, used to respond to the reconnaissance tracking instruction, use the radar type UAV to perform long-range reconnaissance on the targets in the mission area, obtain a first target list, use the target positioning information to guide the optoelectronic type UAV to the mission area, obtain a second target list, process the first target list and the second target list based on a preset target detection and recognition algorithm, obtain a final target ID number, and send the target ID number to the data processing module.

[0011] Optionally, it also includes: a data compression module based on a weak network, which is respectively connected to the situation autonomous perception module and the data processing module, and is used to receive the fusion target list, and in response to an image guidance instruction, perform low-rate image compression on the central area of ​​the field of view of the video image in the fusion target list, and high-rate compression on the remaining areas to obtain a compressed video, and send the compressed video to the data processing module.

[0012] Optionally, it also includes: a heterogeneous payload service customization module, connected to the data processing module, used to obtain the payload status frame and the task list, parse the payload status frame, obtain the payload type, and obtain a customized payload service based on the task list and the payload type, and call a pre-built target model and a corresponding detection and recognition algorithm based on the customized payload service.

[0013] Optionally, the data processing module further includes: a database management submodule, which is used to sort the data in the target list based on priority, and to add, delete, modify and query the data in the target list.

[0014] An airborne intelligent processing system based on multiple UAVs proposed in an embodiment of the present application includes a data processing module arranged in each UAV, and a data receiving module and a situation autonomous perception module which are communicatively connected to the data processing module; wherein the data receiving module is used to receive input data, the input data includes source data, a target list and a task list of other UAVs, and send the source data, the target list and the task list of other UAVs to the data processing module, wherein the source data includes optical image data, SAR image data, UAV posture information, and payload posture information; the data processing module is used to receive input data, the target list of the local machine, and the target perception data sent by the situation autonomous perception module, and based on the matching of the task list and the target perception data The preset task template obtains the task automatic control process data, generates the payload control instruction based on the task automatic control process data, and distributes the payload control instruction to each payload; the situation autonomous perception module is used to receive input data, the local target list and payload control instructions, and respond to multiple instructions of the data processing module to obtain multiple target perception data respectively. The multiple perception data include target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send multiple target perception data to the data processing module. Based on the task template, the parameterized collaborative reconnaissance task automatic driving mode can assist multiple UAV systems to flexibly configure various combat mission modes, making the UAV system combat very flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A structural block diagram of a drone onboard intelligent processing system provided in one embodiment of the present application;

[0016] Figure 2 A flowchart for realizing the task automation driving function of the UAV airborne intelligent processing system provided in one embodiment of the present application;

[0017] Figure 3 This is a structural block diagram of the situation autonomous perception module of the UAV airborne intelligent processing system provided in one embodiment of the present application;

[0018] Figure 4 A flowchart for implementing heterogeneous reconnaissance information association tracking of a drone-mounted intelligent processing system provided in one embodiment of the present application;

[0019] Figure 5 A flowchart for implementing scene matching positioning of a drone onboard intelligent processing system provided in one embodiment of the present application;

[0020] Figure 6 A flowchart for realizing data transmission based on a weak network for a drone-mounted intelligent processing system provided in one embodiment of the present application;

[0021] Figure 7 A heterogeneous payload service customization module for a drone onboard intelligent processing system provided in one embodiment of the present application.

[0022] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] The purpose of the present invention is to propose an airborne intelligent task processing system based on a multi-UAV swarm, using a high-computing-power, lightweight hardware integrated design framework to solve the problems of multi-task payload replacement, storage, scheduling and distribution of multi-source task data, intelligent processing and automatic perception of multi-source payload data, and low intelligence level of collaborative reconnaissance and control of task equipment.

[0025] As an airborne processing device, the airborne intelligent processing system has huge advantages over ground centralized processing equipment. After obtaining the reconnaissance data source, the airborne intelligent processing system only needs to perform automated and intelligent processing on the airborne edge to generate target perception results, battlefield situation and other information, thereby greatly reducing bandwidth requirements and improving the intelligence level of drones.

[0026] refer to Figure 1 The present application provides an airborne intelligent processing system based on multiple UAVs, comprising: a data processing module 10 arranged in each of the UAVs, and a data receiving module 20 and a situation autonomous perception module 30 which are both communicatively connected to the data processing module 10, wherein the data receiving module 20 is used to receive input data, the input data including source data, target lists and task lists of other UAVs, and send the source data, target lists and task lists of other UAVs to the data processing module 10, wherein the source data includes optical image data, SAR image data, UAV posture information, and payload posture information; the data processing module 10 is used to receive input data, the target list of the local machine, and the target list of the local machine, and send the source data, the target list of other UAVs, and the task list of the local machine to the data processing module 10. The target perception data sent by the situation autonomous perception module 30 queries the preset task template based on the task list to obtain task parameters, automatically calculates the task parameters based on the target perception data, obtains the load control instructions according to the task parameters, and distributes the load control instructions to each load; the situation autonomous perception module 30 is used to receive the input data, the target list of the local machine and the load control instructions, and responds to multiple instructions of the data processing module 10 to respectively obtain multiple target perception data, the multiple perception data include target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send the multiple target perception data to the data processing module 10.

[0027] Exemplarily, the airborne intelligent processing system receives input data from the outside through the data receiving module 20, wherein the input data may include a task list, inertial navigation attitude data, and reconnaissance data of various task equipment. The airborne intelligent processing system obtains the target list, task automatic drive instructions, scene matching positioning data, and damage assessment data through the data processing module 10, and sends the above data to the UAV system. In addition, the airborne intelligent processing system also performs lightweight database management on the data in the target list generated by the situation autonomous perception module 30 in this system. The management of the database includes prioritizing the data in the target list, and adding, deleting, modifying, and querying the data in the target list.

[0028] In a specific embodiment, the data processing module 10 processes data in a multi-threaded manner. In other words, the receiving, processing and distribution of data all have independent thread tasks, wherein the data interaction between threads adopts a shared memory manner. The drone has a large-capacity storage disk design, and an SQLite database is deployed in the onboard intelligent processing system. The SQLite database can efficiently process the storage, query and retrieval of the target list data.

[0029] It should be noted that the airborne intelligent processing system set in each drone can receive the task list, inertial navigation data and airborne avionics data in real time through the CAN interface, the reconnaissance data of the optoelectronic load is received through the SDI interface, and the reconnaissance data of the radar reconnaissance load is received through the LAN interface. After receiving the data, each interface of the airborne intelligent processing system stores the data in the corresponding memory area through the shared memory, and starts data processing when each sub-thread detects that there is data. The target list, scene matching positioning data or damage assessment results generated after the data processing module 10 performs data processing can be stored in the database, and the user can access the database concurrently through the SQLite interface. The user can search the database and obtain data by configuring the confidence, detection type and number. In addition, the airborne intelligent processing system is also designed with a gigabit network port, through which the airborne intelligent processing system can distribute the target list, scene matching positioning data or damage assessment result data to the drone system.

[0030] In this embodiment, the airborne intelligent processing system receives the task list distributed by the UAV system and reads the current task mode, which can be persistent reconnaissance, guided strike, damage assessment or satellite-guided denial. The airborne intelligent processing system calculates the parameters for executing the task according to the payload reconnaissance requirements. For example, the parameters of the optoelectronic reconnaissance payload can be the reconnaissance field of view, pitch angle, single-machine width, etc. The airborne intelligent processing system formulates an automatic control process according to the system mission stage, and finally forms a payload task instruction set such as visible light field of view control, infrared field of view control, single-machine width angle, pitch angle, etc. The UAV system automatically drives the task payload through the task instruction set, and assists the cluster UAV system in task collaborative control and task collaborative planning.

[0031] refer to Figure 2 Exemplarily, the data processing module 10 receives the task list transferred by the UAV system through the network port, searches for the pre-bound task template according to the task type, retrieves the configuration parameters of the template, and calculates and determines the load instructions for this task. Taking the "persistent reconnaissance" task stage as an example, the specific implementation method is described. After receiving the task list, the airborne intelligent processing system parses through the protocol to find that the task mode is persistent reconnaissance. At the same time, the calculation parameters of the task template are queried, including: area, area location, traversal level, target size estimation, target distance estimation, flight altitude, etc. The data processing module 10 automatically calculates the drive parameters according to the load type and its capabilities. Taking the single traversal task mode of the optoelectronic reconnaissance load as an example: In order to realize the optoelectronic persistent reconnaissance task, the instruction parameters for driving the optoelectronic equipment include pitch angle, reconnaissance field of view, and single-machine width:

[0032] The pitch angle is calculated as follows: Among them, H is the flight altitude and L is the target distance.

[0033] The reconnaissance field of view is calculated as follows: Where A is the field of view, L is the target distance, a is the target size, m is the number of pixels, and n is the number of detector pixels.

[0034] The single machine width is calculated as follows: Where M is the single-machine width angle, D is the area width, and fn is the number of aircraft. The onboard intelligent processing system will calculate the drive parameters (F, A, M) according to the protocol programming instructions, and judge that if the current position of the UAV is within the photoelectric action distance L from the mission area, the instruction set will be sent to the photoelectric device through the serial port. After receiving the instruction, the photoelectric device will drive the corresponding load component to perform the persistent reconnaissance mission and cooperate with other UAVs in the formation to complete the persistent reconnaissance mission.

[0035] refer to Figure 3In an embodiment of the present invention, the situation autonomous perception module 30 includes a target detection, identification and positioning submodule 301. The heterogeneous payload service customization module 301 is used to pre-process the optical image data and the SAR image data in response to a target detection and identification instruction to obtain pre-input data, call a preset target detection algorithm to process the pre-input data to obtain first position information, and process the drone posture information, the payload posture information and the position of the target in the image based on a preset spatial coordinate conversion algorithm to obtain second position information, and send the first position information and the second position information to the data receiving module 20.

[0036] Specifically, the situational autonomous perception module 30 performs intelligent processing of battlefield reconnaissance information through optoelectronic reconnaissance payloads and radar reconnaissance payloads. The intelligent processing includes target detection, identification and positioning, multi-source heterogeneous distributed perception, image guidance and damage assessment.

[0037] Among them, the target detection, identification and positioning submodule 301 receives the visible light and infrared video forwarded by the data processing module 10 through the MIPI interface of the data processing module 10, as well as the SAR image forwarded through the internal network port. The situation autonomous perception module 30 calls the target detection and identification algorithm library to perform target detection and identification, and obtains the target detection and identification result, which includes the target category, confidence and pixel position of a single image (first position information). The target detection, identification and positioning submodule 301 uses the photoelectric posture, target ranging value, drone position and posture, and calculates the absolute position of a single target through spatial coordinate conversion. Using the matching results with the high-precision electronic map, the absolute position of multiple targets (second position information) is calculated. The target detection, identification and positioning submodule 301 forms a target list with the target detection result according to the target library format and sends it to the data processing module.

[0038] In the actual execution process, the target detection, identification and positioning submodule 301 is based on the target detection and identification algorithm, and obtains the category and location information of the target by detecting and identifying the optoelectronic / SAR image, and then tracks the target (optoelectronic) and locates the target on this basis. The implementation method is as follows: the target detection, identification and positioning submodule 301 receives the optical image data, SAR image data, drone posture information, and payload posture information from the data processing module 10. The target detection, identification and positioning submodule 301 receives the detection and identification instruction from the data processing module 10. The target detection, identification and positioning submodule 301 reads the optical image data and SAR image data and performs pre-processing of slicing and resizing the image to obtain pre-input data, wherein the pre-input data is the input data of the target detection algorithm. The target detection, identification and positioning submodule 301 calls the board computing power resources to execute the target detection algorithm to obtain the target type, target confidence and the position of the target in the image (first position information). The target detection, identification and positioning submodule 301 calculates the target geographic location (first position information) through the target position, drone position and posture and payload posture information. The target detection, identification and positioning submodule 301 removes duplicate targets from the detected targets.

[0039] Continue to refer Figure 3 In an embodiment of the present invention, the situation autonomous perception module 30 also includes a multi-source heterogeneous distributed perception sub-module 302. The multi-source heterogeneous distributed perception sub-module 302 is used to respond to the target association fusion instruction, process the target list of the other UAVs and the target list of the local UAV based on a preset target re-identification algorithm, obtain a fused target list, and send the fused target list to the data processing module 10.

[0040] In the actual implementation process, the target association function of the situation autonomous perception module 30 realizes the fusion of single-machine heterogeneous targets and multi-machine homogeneous targets. The implementation method is as follows: the situation autonomous perception module 30 receives the local target list, feature list and other drone target lists and feature lists from the data processing module 10, and uses the local target features and other drone target features as the target re-identification algorithm (ReID) input. The target association fusion instruction forwarded from the data processing module is received, and the target association fusion module calls the computing power resources for target re-identification to obtain the optoelectronic multi-machine target list after fusion and deduplication. The fused target list is sent to the data processing module.

[0041] The multi-source heterogeneous distributed perception submodule 302 uses the results of single-machine detection and identification as a target list, and uses the results of other UAV optoelectronic / SAR detection and identification transmitted by the inter-machine communication equipment as another target list. The multi-source heterogeneous distributed perception submodule 302 performs target re-identification on the two target identification results and outputs the target reconnaissance results of the mission area.

[0042] refer to Figure 4 In an embodiment of the present application, the UAV onboard intelligent processing system also includes a heterogeneous reconnaissance information association tracking module 50, which is connected to the data processing module 10 and is used to respond to a reconnaissance and tracking instruction, use a radar-type UAV to perform long-range reconnaissance on targets in the mission area, obtain a first target list, use the target positioning information to guide the optoelectronic UAV to the mission area, obtain a second target list, process the first target list and the second target list based on a preset target detection and recognition algorithm, obtain a final target ID number, and send the target ID number to the data processing module 10.

[0043] For example, the target information in the target list detected by each aircraft includes the target type and target position. There is a certain positioning error in the target position. Considering that the error source of each aircraft is the same, the relative position error between the targets detected by the same aircraft is small. The topological relationship of the target position detected by each aircraft is used to realize the target association between aircraft. The target position topological relationship is obtained by calculating the position difference between two targets on the same aircraft. The target list data formed by UAV A is as follows:

[0044] Target ID Target location Target Slice A1 AT1(B,L,H) T_Pic1 A2 AT2(B,L,H) T_Pic2 A3 AT3(B,L,H) T_Pic3 --- --- ---

[0045] The relative horizontal position of target 1 and target 2 is D (AT1，AT2) , the relative horizontal position of target 2 and target 3 is D (AT3，AT2) , the relative horizontal position of target 1 and target 3 is D (AT1，AT3) , then calculate the relative distance interpolation of target 1

[0046] The target list data formed by UAV B is as follows:

[0047] Target ID Target location Target Slice B1 BT1(B,L,H) BT_Pic1 B3 BT2(B,L,H) BT_Pic2 B3 BT3(B,L,H) BT_Pic3 --- --- ---

[0048] The relative horizontal position of target 1 and target 2 detected by UAV B is D BBT1，BT2) , the relative horizontal position of target 2 and target 3 is D (NT3，NT2) , the relative horizontal position of target 1 and target 3 is D (BT1，BT3) . Then calculate the relative distance interpolation of target 1

[0049] if Then the target 1 seen by the B machine is the target 1 seen by the A machine, and so on, traverse all the targets and calculate to determine the ID numbers of other targets. Then, based on the partial neighboring area information retained in the target image slice, the relatively large-scale scene registration is used to assist in the secondary recognition of the target. The mutual information-based registration method is used to achieve scene image registration between drones and aircraft. Finally, the target ID number is determined.

[0050] refer to Figure 3 and Figure 5 In an embodiment of the present invention, the situation autonomous perception module 30 also includes an image guidance submodule 303, which is used to respond to an image guidance instruction, obtain the second position information, use the second position information to guide the UAV, correct the offset of the UAV payload relative to the first position information based on the first position information, obtain guidance information, and send the guidance information to the data processing module 10.

[0051] In the actual execution process, the autonomous sensing module 30 receives the target guidance instruction and guides the optoelectronic payload to complete the geographic guidance. Then, the video shot by the optoelectronics is detected, re-identified and single-target tracked, and the offset of the target relative to the center of the image is continuously output to the optoelectronic reconnaissance payload, and the optoelectronic equipment is cooperated to complete the guidance.

[0052] Specifically, the image guidance submodule 303 receives the image guidance instruction forwarded by the data processing module 10, and reads the guidance target ID in the instruction. The database is queried according to the target ID to obtain the initial positioning of the target and the target slice characteristics, and the initial positioning information of the target is sent to the optoelectronic payload according to the protocol. The optoelectronic payload performs geographic guidance based on the positioning information so that the target reappears in the image. The image guidance submodule 303 receives the image data forwarded by the data processing module, and continuously performs target detection, identification and tracking. When a new target is detected, the new target is re-identified with the guidance target; when it is determined to be the same target, the target is continuously detected and tracked to obtain the pixel deviation between the target and the center of the image, and it is sent to the data processing module 10 in real time.

[0053] Continue to refer Figure 3 In an embodiment of the present application, the situation autonomous perception module 30 also includes a damage assessment submodule 304, which is used to respond to a damage assessment instruction, receive the optical image data and the SAR image data, obtain a damage assessment result of the target based on the optical image data and the SAR image data before and after the strike, and send the damage assessment result to the data processing module 10.

[0054] Among them, the damage assessment submodule 304 receives the image data and damage assessment instructions from the data processing module. The target motion attribute damage is judged based on the movement before and after the strike. The damage ratio is assessed based on the proportion of smoke and fire detection in the target detection area after the strike. The critical parts of the target are identified through the detection results, and the physical parts damage assessment is performed.

[0055] Exemplarily, the damage assessment submodule 304 implements the function of assessing the damage degree of the battlefield through the change information of the reconnaissance image before and after the damage. After obtaining the damage assessment instruction, the damage assessment submodule 304 calls the damage analysis and assessment module to start the task.

[0056] The damage assessment submodule 304 implements three damage assessment methods, including motion attribute damage assessment, damage ratio assessment and physical part damage assessment.

[0057] Among them, the motion attribute damage assessment can receive the automatic damage assessment instructions and image data forwarded by the data processing module. Call the target detection algorithm for detection and recognition, eliminate background motion interference through registration, and use the frame difference method to obtain the current motion attribute of the target. Output the change of the target's motion attribute before and after the attack.

[0058] The physical part damage assessment can receive the automatic damage assessment instructions and image data forwarded by the data processing module, call the target detection algorithm and the critical part detection algorithm, and output the critical part detection results in the target box.

[0059] The damage ratio assessment can receive the automatic damage assessment instructions forwarded by the data processing module, receive image data and target list. Before the strike, the target frame position is obtained through the target detection algorithm. After the strike, the smoke and fire detection algorithm is used to obtain the position of the smoke and fire in the image to determine whether the target is hit. If it is hit, the proportion of the damaged area to the target area is continuously calculated through the change detection method.

[0060] refer to Figure 1 In an embodiment of the present invention, the UAV airborne intelligent processing system further includes a scene matching positioning module 40, which is connected to the data processing module 10 and is used to receive the visible light infrared video and SAR image data, the locally stored map data, the UAV posture information and the payload posture information sent by the data processing module 10 under satellite denial conditions, and in response to a scene matching instruction, call a preset image matching algorithm to process the visible light infrared video and SAR image data and the map data to obtain target positioning information and image map matching information, and process the UAV posture information, the payload posture information and the image map matching information based on the rear handover algorithm to obtain the UAV positioning information, and send the UAV positioning information and the target positioning information to the data processing module 10.

[0061] Among them, in the satellite denial environment, the scene matching positioning module 40 receives the scene matching instruction forwarded by the data processing module, and the scene matching positioning module 40 receives the original reconnaissance image forwarded by the data processing function module. The original reconnaissance image can include visible light infrared video and SAR image, and uses the high-precision electronic map stored locally as the matching base map, and obtains the feature correspondence between the two images based on the image matching algorithm, so as to calculate the target positioning result. The scene matching positioning module 40 uses the photoelectric payload attitude, the drone position and attitude, and calculates the drone position through the rear intersection method. According to the protocol, the target and drone positions are sent to the data processing module 10.

[0062] In the actual implementation process, the scene matching positioning module 40 receives the scene matching instruction and optical image data / SAR image data forwarded by the data processing module 10, reads the target detection result provided by the situation autonomous perception module 30; reads the local electronic map of the task processing board (tiff format map, accuracy 1 meter), and performs rough positioning of the drone through the drone position, attitude and payload attitude, and performs map clipping;

[0063] The scene matching and positioning module 40 uses the image data and the map clipping data as the matching algorithm input, and calls the board computing resources to perform matching algorithm reasoning to obtain the feature point pairing relationship between the image data and the electronic map. Among them, during the first matching, the local electronic map of the task processing board is read to traverse the scene matching. According to the geographical location characteristics of the electronic map, the target positioning result can be calculated through the feature point pairing relationship between the electronic map and the reconnaissance image, so as to achieve high-precision positioning of the target. In addition, through the feature point pairing relationship, the payload posture, and the drone posture, the drone positioning result can be calculated using the rear intersection method, and the scene matching and positioning module 40 outputs the aircraft, target positioning result and time to the data processing module.

[0064] In an embodiment of the present invention, the UAV onboard intelligent processing system also includes a weak network-based data compression module 60, which is respectively connected to the situation autonomous perception module 30 and the data processing module 10, and is used to receive the fusion target list, and in response to the image guidance instruction, perform low-rate image compression on the central area of ​​the field of view of the video image in the fusion target list, and high-rate compression on the remaining areas to obtain a compressed video, and send the compressed video to the data processing module 10.

[0065] refer to Figure 6Specifically, the weak network data compression module 60 receives the target result output by the situation autonomous perception module 30, slices and compresses the target image, integrates the target ID, target position, attribute information and recognition confidence rate, and forms a target list according to the target list protocol. After receiving the image guidance instruction sent by the data processing module, the weak network data compression module 60 performs low-rate image compression on the central area of ​​the field of view in the video image, and high-rate compression on the remaining areas.

[0066] In the actual implementation process, after the weak network data compression module 60 receives the target result data forwarded by the situation autonomous perception module 30, it first extracts the key information in the data to form target simplified information, where the target simplified information includes target number, target type, target position (longitude, latitude, altitude), etc.; secondly, after expanding the width and height of the target in the target result data by 20 pixels each, it slices the image according to the pixel position (target center point position, target width and height), and then controls the slice image to perform JPEG compression within a certain data size according to the image size; finally, the target simplified information and target slices are combined into a single target list and sent to the data processing module.

[0067] The data compression module 60 of the weak network adjusts the video window size according to the control instruction, and obtains the ROI image evaluation value in real time through information entropy statistics. Based on the original framework of the H.265 encoder, the data compression module 60 of the weak network is optimized by improving the H.265 encoder architecture under the premise of real-time judgment of the image quality of the ROI local area. The data compression module 60 of the weak network uses Peak Signal Noise Ratio (PSNR) and Root Mean Square Error (MSE) to evaluate the ROI image.

[0068] The data compression module 60 of the weak network determines whether the macroblock is within the ROI range through the encoder. For the macroblock in the ROI part, a lower quantization parameter is set, and the bit rate control part is modified to obtain more bits, while the non-ROI part obtains fewer bits. In this way, the quantization range is refined from one frame to one macroblock.

[0069] Exemplarily, the data compression module 60 of the weak network can use the H.265 encoding method, and perform H.265 compression after the data processing module receives and parses the video data. Read the width and height of the resolution instruction, the video compression module reinitializes a new encoder according to the width and height, and uses the new encoder for hard compression; the video data compressed by the new encoder is transmitted to the data processing module 10 through the shared memory as a compressed code stream.

[0070] refer to Figure 7In an embodiment of the present invention, the UAV onboard intelligent processing system also includes a heterogeneous payload service customization module 70, which is connected to the data processing module 10, and is used to obtain the payload status frame and the task list, parse the payload status frame, obtain the payload type, and obtain a customized payload service according to the task list and the payload type, and call the pre-built target model and the corresponding detection and recognition algorithm based on the customized payload service.

[0071] The heterogeneous load service customization module 70 parses the device type code of the load status frame and automatically identifies the load type. The heterogeneous load service customization module 70 calls the control module of each load according to the load type. According to the task phase and task target type sent by the data processing module 10, the corresponding algorithm and target model are called, wherein the algorithm includes scene matching, situation awareness, etc.

[0072] In the actual execution process, the heterogeneous payload service customization module 70 performs standardized design on the physical layer and protocol layer for the payload hanging point interface for different types of UAV systems. The communication interface of the heterogeneous payload service customization module 70 adopts CAN bus, and the image interface adopts network port design, using the same protocol framework. When the UAV system is powered on, the task processing board can automatically judge the type of payload through the status word in the communication protocol. The heterogeneous payload service customization module 70 realizes the service customization of the payload through the scheduling of the algorithm module and the loading of the target model according to the combat mission stage and the mission target type sent by the data processing module 10. Among them, the target model is mainly classified according to the reconnaissance source and target type. The reconnaissance source includes visible light, infrared, and radar SAR. The target types include armored vehicles, aircraft, ships, typical buildings, key parts, fire, etc. Different target models are called for loading according to different reconnaissance payload types, mission target types, and mission modes. When the UAV is performing system reconnaissance operations, the mission target model is called to realize target detection and identification. During damage assessment, the mission target detection model, key part model, and fire detection model are called. The heterogeneous payload service customization module 70 automatically schedules the corresponding algorithm modules according to the payload type, task type and based on the task control process, including the situation awareness function module, the scene matching module in the denied environment and the damage assessment module. The specific algorithms include: optoelectronic typical target detection and recognition based on artificial intelligence, optoelectronic atypical target detection and recognition based on matching, optoelectronic critical part detection and recognition, radar SAR target detection and recognition, multi-target positioning, radar SAR scene matching positioning, optoelectronic scene matching positioning, radar SAR damage assessment, optoelectronic damage assessment, etc.

[0073] In the embodiment of the present application, the data processing module 10 further includes a database management submodule 101 for sorting the data in the target list based on priority, and for adding, deleting, modifying and querying the data in the target list.

[0074] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An airborne intelligent processing system based on multiple drones, characterized in that: include: A data processing module disposed in each of the drones, and a data receiving module and a situation autonomous perception module that are both communicatively connected to the data processing module; The data receiving module is used to receive input data, the input data includes source data, target lists and task lists of other drones, and send the source data, target lists and task lists of other drones to the data processing module, wherein the source data includes optical image data, SAR image data, drone posture information, and payload posture information; The data processing module is used to receive input data, the target list of the local machine and the target perception data sent by the situation autonomous perception module, query the preset task template based on the task list to obtain task parameters, automatically calculate the task parameters based on the target perception data, obtain the load control instructions according to the task parameters, and distribute the load control instructions to each load; The situation autonomous perception module is used to receive the input data, the target list of the aircraft and the payload control instructions, and in response to multiple instructions of the data processing module, respectively obtain multiple target perception data, the multiple perception data include target detection, identification and positioning data, multi-source heterogeneous image fusion data, image guidance data and damage assessment data, and send the multiple target perception data to the data processing module.

2. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: The situation autonomous perception module includes: The target detection, identification and positioning submodule is used to pre-process the optical image data and the SAR image data in response to the target detection and identification instruction to obtain pre-input data, call a preset target detection algorithm to process the pre-input data to obtain first position information, and process the UAV posture information, the payload posture information and the position of the target in the image based on a preset spatial coordinate conversion algorithm to obtain second position information, and send the first position information and the second position information to the data receiving module.

3. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: The situation autonomous perception module also includes: The multi-source heterogeneous distributed perception submodule is used to respond to the target association fusion instruction, process the target list of the other UAVs and the target list of the local UAV based on a preset target re-identification algorithm, obtain a fused target list, and send the fused target list to the data processing module.

4. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: The situation autonomous perception module also includes: The image guidance submodule is used to respond to the image guidance instruction, obtain the second position information, use the second position information to guide the UAV, correct the offset of the UAV payload relative to the first position information based on the first position information, obtain guidance information, and send the guidance information to the data processing module.

5. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: The situation autonomous perception module also includes: The damage assessment submodule is used to respond to the damage assessment instruction, receive the optical image data and the SAR image data, obtain a damage assessment result of the target based on the optical image data and the SAR image data before and after the strike, and send the damage assessment result to the data processing module.

6. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: Also includes: The scene matching positioning module is connected to the data processing module and is used to receive the visible light infrared video and SAR image data, the locally stored map data, the UAV posture information and the payload posture information sent by the data processing module under satellite denial conditions, and in response to the scene matching instruction, call a preset image matching algorithm to process the visible light infrared video and SAR image data and the map data to obtain target positioning information and image map matching information, and process the UAV posture information, payload posture information and image map matching information based on the rear handover algorithm to obtain UAV positioning information, and send the UAV positioning information and the target positioning information to the data processing module.

7. The multi-UAV airborne intelligent processing system according to claim 3, characterized in that: Also includes: A heterogeneous reconnaissance information association tracking module is connected to the data processing module and is used to respond to the reconnaissance tracking instruction, use the radar-type UAV to perform long-range reconnaissance on the targets in the mission area, obtain a first target list, use the target positioning information to guide the optoelectronic UAV to the mission area, obtain a second target list, process the first target list and the second target list based on a preset target detection and recognition algorithm, obtain a final target ID number, and send the target ID number to the data processing module.

8. The multi-UAV airborne intelligent processing system according to claim 3, characterized in that: Also includes: The data compression module based on the weak network is respectively connected to the situation autonomous perception module and the data processing module, and is used to receive the fusion target list, respond to the image guidance instruction, perform low-rate image compression on the central area of ​​the field of view of the video image in the fusion target list, and high-rate compression on the remaining areas to obtain a compressed video, and send the compressed video to the data processing module.

9. The multi-UAV airborne intelligent processing system according to claim 3, characterized in that: Also includes: A heterogeneous payload service customization module is connected to the data processing module, and is used to obtain a payload status frame and the task list, parse the payload status frame, obtain a payload type, and obtain a customized payload service according to the task list and the payload type, and call a pre-built target model and a corresponding detection and recognition algorithm based on the customized payload service.

10. The multi-UAV airborne intelligent processing system according to claim 1, characterized in that: The data processing module also includes: The database management submodule is used to sort the data in the target list based on priority, and to add, delete, modify and query the data in the target list.

Images