High-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system

Abstract

The invention discloses a high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system, and the method comprises the steps: cooperatively arranging a multi-node unmanned aerial vehicle in a flight path key section, obtaining a flight body sequence image through the optical measurement of an unmanned aerial vehicle array, and returning multi-source measurement data through combining with an air-ground integrated communication network, and real-time / afterward fusion is carried out on optical measurement and in-situ measurement based on a unified space-time reference, so that high-precision calculation and dynamic perception of full flight path parameters such as the position, the attitude and the speed of the high-rotation flight body are realized. According to the invention, the technical problems of global monitoring and precise measurement of the motion state of the flying body in a high-dynamic and high-overload environment are effectively solved, and the method has the advantages of flexible deployment, strong anti-interference performance and high measurement precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of high-rotation flight parameter measurement technology, in particular to a high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system. BACKGROUND

[0002] In the research and application of high-rotation flight bodies, precise measurement of flight parameters and global perception of motion states are key technical bottlenecks to be broken through. Accurate acquisition of the pose parameters of high-rotation flight bodies is directly related to core links such as flight trajectory dynamic calibration, anti-interference performance optimization and damage effectiveness evaluation. However, due to the extreme overload condition in the high-dynamic environment, the sensors in the traditional target range measurement means often cannot realize the full-range measurement of flight parameters and the global perception of motion states, resulting in that the existing technology cannot meet the demand of high-precision flight parameter measurement.

[0003] In recent years, through the deep integration of high-speed imaging load and precise image processing algorithm, the target range optical measurement technology realizes the accurate measurement of the motion parameters of flight bodies, and plays an important supporting role in the improvement of high-rotation flight body striking precision and forward research. Compared with traditional optical measurement methods, unmanned aerial vehicle optical measurement technology has the advantages of low-cost deployment, flexible station deployment, multi-platform collaborative networking and strong survival ability in strong confrontation environment, and has become a new technology direction for high-rotation flight parameter measurement.

[0004] However, for the high-dynamic flight characteristics of high-rotation flight bodies, how to scientifically deploy the unmanned aerial vehicle measurement array in a complex battlefield environment, break through the technical bottlenecks of feature extraction and space-time registration of high-rotation high-speed moving targets in image sequences, and realize high-precision fusion of unmanned aerial vehicle measurement data and traditional target range in-situ measurement data under the unified space-time reference, so as to complete the precise measurement of full flight parameters and the global perception of motion states, is still a technical problem to be solved at present.

[0005] At present, there is no systematic solution for this scene, and the related technical blank seriously restricts the improvement of the actual combat effectiveness of high-rotation flight bodies. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system, which realizes the precise measurement of high-rotation flight trajectory parameters and the global perception of motion states.

[0007] A high-rotation flight parameter unmanned aerial vehicle array optical measurement system, comprising a launch area measurement unmanned aerial vehicle array 100, a middle section measurement unmanned aerial vehicle array 200, a landing point area measurement unmanned aerial vehicle array 300, a communication unmanned aerial vehicle array 400, a computing center 500 installed on a ground station and a weather balloon 600;

[0008] The launch area measurement UAV array 100 comprises at least three UAVs; at least one UAV is deployed directly behind the launch point to monitor the attitude, speed and position of the high-rotation flying body in the yaw direction after launch; at least two UAVs are deployed in front of the launch point and symmetrically distributed on both sides of the launch surface to monitor the attitude, speed and position of the high-rotation flying body in the direction of flight and height after launch.

[0009] The middle section measurement UAV array 200 comprises a plurality of UAVs, which are respectively deployed at different positions in front of the launch point and symmetrically distributed on both sides of the launch surface at the same distance from the launch surface, for monitoring the attitude, speed and position of the high-rotation flying body during flight.

[0010] The landing point area measurement UAV array 300 comprises at least two UAVs, at least one of which is deployed directly above the preset landing point to detect the landing point position; at least one is deployed in front of the preset landing point of the high-rotation flying body to monitor the parachute opening during soft recovery.

[0011] The communication UAV array 400 is used to provide relay nodes for the communication link, and maintains communication with the in-situ measurement device on the high-rotation flying body to be measured, the launch area measurement UAV array 100, the middle section measurement UAV array 200, the landing point area measurement UAV array 300 throughout the whole process, and transmits measurement information to the ground station in real time to form an air-ground communication link for real-time networking communication.

[0012] The computing center 500 receives optical measurement data from the launch area measurement UAV array 100, the middle section measurement UAV array 200 and the landing point area measurement UAV array 300 in real time through the communication UAV array 400, and processes the data in real time or after the event.

[0013] The weather balloon 600 carries instruments to ascend and return the current atmospheric pressure, temperature, humidity, wind direction and wind speed.

[0014] Preferably, the launch area measurement UAV array 100, the middle section measurement UAV array 200 and the landing point area measurement UAV array 300 comprise optical measurement load, communication module and measurement UAV flight control unit; the optical measurement load comprises a visible light camera for capturing sequential images of the high-rotation flying body; the communication module uses microwave or laser communication technology to establish a downlink with the communication UAV array unit, and is responsible for sending the original data or pre-processed data obtained by the data acquisition and storage unit to the outside; the measurement UAV flight control unit comprises a GNSS receiver, an inertial measurement unit and a flight control computer, which receives the flight route instructions from the computing center 500 and accurately controls the position and attitude of the UAV in the air.

[0015] Preferably, the communication UAV array 400 includes a communication UAV flight control unit and a core communication unit. The communication UAV flight control unit controls the UAVs to form a specific formation in the predetermined airspace to build a stable communication network coverage. The core communication unit is a multi-band, multi-protocol, software-defined intelligent communication system, which includes three links and a MESH network inside the array. The downlink with the measurement array transmits video and image data from multiple measurement UAVs through a high-speed microwave or laser communication link. The uplink with the computing center transmits all integrated data to the ground computing center 500 through satellite communication. The UAVs in the array exchange data and control information through the MESH network to achieve load balancing and link redundancy.

[0016] Preferably, the computing center 500 includes a real-time processing module, a post-processing module, a communication hub, and a cluster control module. The real-time processing module is responsible for information fusion of in-situ measurement data and light measurement data of the measurement UAV array received by the communication UAV array unit under a unified space-time reference through real-time filtering. The post-processing module is responsible for converting the fusion processing of the measurement data into parameter fusion estimation after the test is completed, giving high-precision estimation of the flight parameters of the high-speed rotating body and the errors of the measurement UAV array and the in-situ measurement system, and calibrating the measurement device. The communication hub realizes bidirectional data transmission between the computing center and the communication UAV. The cluster control module realizes UAV route planning, task allocation, and command delivery to the UAV array base station 700, realizing UAV take-off, formation adjustment, and landing.

[0017] Preferably, it also includes a UAV array base station 700 for realizing automatic storage, charging, and take-off and landing scheduling of the UAVs, supporting continuous operation of the cluster.

[0018] Preferably, the UAV array base station 700 comprises a hangar 701, an automatic take-off and landing module 702, an intelligent charging module 703, a state monitoring system 704, and a mechanical scheduling mechanism 705. The hangar 701 is made of waterproof and dustproof metal structure to protect the electronic components of the UAV. The automatic take-off and landing module 702 is a lifting platform with guiding marks on the surface, which can realize precise take-off and landing of the UAV in cooperation with the visual positioning module of the UAV, and is also equipped with a light guiding system to provide take-off and landing guidance for the UAV in night or low visibility environment. The intelligent charging module 703 is a wireless charging seat that can support automatic wireless charging of the UAV. The state monitoring system 704 includes an environmental sensor for monitoring the temperature, humidity and other environmental conditions in the hangar, and a UAV state sensor for monitoring the battery capacity and motor speed of the UAV, which is used to monitor the health of the system. The mechanical scheduling mechanism 705 can automatically close the movable hatch of the hangar when the system is idle or in harsh environment, realizing the sealing protection of the hangar. When the system starts or the UAV takes off, the hatch is automatically opened to provide a take-off and landing channel for the UAV.

[0019] Preferably, it also comprises an RTK base station unit 800, which provides high-precision positioning reference for the UAV cluster through real-time dynamic difference technology.

[0020] A light measurement method of a high-rotation flight parameter UAV array light measurement system, comprising:

[0021] Step S31, before light measurement, release the weather balloon 600 to detect atmospheric parameters, and transmit atmospheric parameters such as air pressure, temperature and humidity, and wind direction and speed under current atmospheric conditions back to the calculation center 500;

[0022] Step S32, before light measurement, hover the launch area measurement UAV array 100, the middle section measurement UAV array 200, and the landing point area measurement UAV array 300 at the set position;

[0023] Step S33, arrange the communication UAV array 400 to form a network, and establish a communication link:

[0024] Arrange the UAV array network as a relay node on the communication link, receive the measurement data on the in-situ measurement device on the high-rotation flying body and the measurement data of the measurement UAV array throughout the journey, and form a data transmission link of the in-situ measurement system, the UAV light measurement system, and the ground calculation center 500;

[0025] Step S34, according to the proposed scheme, test in the target field, transmit the in-situ measurement data received by the communication relay node in the communication UAV array 400 and the UAV optical measurement data back to the calculation center 500, estimate the full flight trajectory parameters of the high-rotation flying body and the measurement system error through information fusion and filtering under the same space-time reference, and feedback and calibrate the measurement device;

[0026] Step S35, detecting the landing state and performing recovery.

[0027] At the landing position, the landing position is evaluated by using the landing area measurement UAV array 300 and the preset flight trajectory to provide guidance for the arrangement of the measurement UAV network and the recovery after landing; the landing position and attitude of the flying body are accurately measured by using the landing area measurement UAV array 300, and the calculation center is transmitted back through the UAV communication link.

[0028] A light measurement method of a high-rotation flight parameter UAV array light measurement system, comprising:

[0029] Step S21, data preprocessing:

[0030] First, the original video data transmitted back by the UAV is decoded and processed to convert it into a continuous image frame sequence. Then, each frame of image is subjected to size standardization processing to uniformly adjust all frames to a preset fixed resolution. Then, the standardized image is subjected to USM sharpening processing, which specifically includes: performing Gaussian blur processing on the image to obtain a low-frequency component, subtracting the low-frequency component from the original image to obtain high-frequency detail information, setting an appropriate threshold to filter noise, and then weighting and superimposing the high-frequency details back to the original image according to a preset sharpening intensity coefficient;

[0031] Step S22, frame alignment and registration processing:

[0032] A unified space-time coordinate system needs to be established for frame alignment and registration processing; the first frame of the video sequence is selected as the reference coordinate system reference image. For each subsequent frame of image, first convert it and the reference frame into grayscale images, then calculate their cross power spectrum, and determine the translation parameters between the frames by detecting the phase correlation peak value of the cross power spectrum; the RANSAC algorithm is used to optimize the preliminary estimated transformation parameters; according to the optimized transformation parameters, affine transformation is applied to the current frame to align it with the reference frame; for the image boundary missing area generated after the transformation, mirror filling method is used for compensation processing to ensure the integrity of the image;

[0033] Step S23, background modeling and moving target detection:

[0034] A plurality of background sample images are pre-acquired, and a background model is constructed by weighted average, wherein the weight of each background image is dynamically adjusted according to the closeness of the acquisition time and the detection time; the same pre-processing procedure as the input image is applied to the constructed background model, that is, step S21; in the detection stage, the current frame and the background model are first converted into grayscale images, and the absolute difference image between the two is calculated, and then an adaptive threshold method based on the Otsu algorithm is used to binarize and segment the difference image, wherein the threshold sensitivity coefficient is automatically adjusted according to the scene complexity; the binarized image is sequentially subjected to morphological open operation to remove noise points, and then closed operation is performed to fill the internal cavities of the target;

[0035] Finally, the connected component analysis algorithm is used to extract the foreground region meeting the preset area threshold, and the minimum circumscribed rectangle of each connected component is calculated as the target bounding box;

[0036] Step S24, flight body enhancement and feature extraction:

[0037] According to the known actual physical size of the flight body and the geometric parameters of the imaging system, the theoretical projection size of the target in the image is calculated; based on this, the boundary of the detected target region is expanded, and the expansion amount is adjusted according to the target motion speed and inter-frame motion;

[0038] An edge sharpening process is performed on the enhanced image using a Laplacian operator to highlight the contour features of the target, and the accurate position information of the target in each frame image is recorded, including the top-left corner coordinates, width, height and center point coordinates of the bounding box;

[0039] Step S25, multi-frame fusion and trajectory generation:

[0040] First, time domain median filtering is performed on all pre-processed image frames to generate a static background image; then the enhanced target ROI in each frame is fused into the background image using an alpha blending method according to the center coordinate position;

[0041] For the target center points in consecutive frames, a cubic B-spline curve is used for interpolation connection to generate a smooth motion trajectory line;

[0042] Step S26, flight body motion analysis:

[0043] Based on the boundary box coordinate sequence of the target in each frame, tail feature points that can stably represent the motion state of the target are selected; the position changes of these feature points in consecutive frames are analyzed, and the central difference method is used to calculate the instantaneous velocity component, wherein the time interval is determined according to the frame rate of the video;

[0044] The horizontal speed and the vertical speed are vector synthesized to obtain the actual motion speed of the target;

[0045] Step S27: flight body optical measurement data fusion:

[0046] The Kalman filter is used to fuse the optical measurement data and other measurement system data such as in-situ measurement and radar measurement, to estimate the full flight trajectory parameters of the flight body and measurement system error, and to feedback and calibrate the measurement device.

[0047] The present application has the following beneficial effects:

[0048] The present application discloses a high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system, which is characterized in that: a plurality of unmanned aerial vehicles are cooperatively arranged in a key section of a flight trajectory, a sequence of images of a flight body is acquired by relying on unmanned aerial vehicle array optical measurement, multi-source measurement data is returned in combination with air-ground integrated communication network, optical measurement and in-situ measurement are fused in real time / after the event based on a unified space-time reference, and high-precision calculation and dynamic perception of full flight trajectory parameters such as position, attitude and speed of a high-rotation flight body are realized. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a high-rotation flight parameter unmanned aerial vehicle array optical measurement system schematic diagram according to the first embodiment of the present application;

[0050] Figure 2 is a measurement unmanned aerial vehicle array and a middle section measurement unmanned aerial vehicle array arrangement schematic diagram of the high-rotation flight parameter unmanned aerial vehicle array optical measurement system according to the first embodiment of the present application;

[0051] Figure 3 is a high-rotation flight parameter unmanned aerial vehicle array optical measurement device schematic diagram according to the second embodiment of the present application;

[0052] Figure 4 is a high-rotation flight parameter unmanned aerial vehicle array optical measurement device structure diagram according to the second embodiment of the present application;

[0053] Figure 5 is a high-rotation flight parameter unmanned aerial vehicle array optical measurement device structure diagram according to the second embodiment of the present application;

[0054] Figure 6 is a high-rotation flight parameter unmanned aerial vehicle array optical measurement method flow chart according to the third embodiment of the present application;

[0055] Figure 7is a flow chart of the high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system actual combat experiment scheme according to the fourth embodiment of the present application;

[0056] Figure 8 is a top view of the launching area measurement unmanned aerial vehicle array and the middle section measurement unmanned aerial vehicle array arrangement in the high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system actual combat experiment scheme according to the fourth embodiment of the present application;

[0057] Figure 9 is a side view of the launching area measurement unmanned aerial vehicle array and the middle section measurement unmanned aerial vehicle array arrangement in the high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system actual combat experiment scheme according to the fourth embodiment of the present application;

[0058] Figure 10 is a side view of the landing point area measurement unmanned aerial vehicle array and the middle section measurement unmanned aerial vehicle array arrangement in the high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system actual combat experiment scheme according to the fourth embodiment of the present application. DETAILED DESCRIPTION

[0059] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0060] Embodiment 1:

[0061] The present embodiment provides a high-rotation flight parameter unmanned aerial vehicle array optical measurement system, as shown in Figure 1 , including a launching area measurement unmanned aerial vehicle array 100, a middle section measurement unmanned aerial vehicle array 200, a landing point area measurement unmanned aerial vehicle array 300, a communication unmanned aerial vehicle array 400, a computing center 500 installed on a ground station, and a weather balloon 600.

[0062] The launching area measurement unmanned aerial vehicle array 100 includes three unmanned aerial vehicles 101, 102 and 103, as shown in Figure 2 , the 103 is deployed L1 meters behind the launching point, with a height of h b meters from the ground, for monitoring the attitude, speed and position of the high-rotation flight body after launching; the 101 and 102 are respectively deployed L2 meters in front of the launching point, symmetrically distributed on both sides of the launching surface, with a distance of x b meters from the launching surface, and a height of h b meters from the ground, for monitoring the attitude, speed and position of the high-rotation flight body after launching; the L1 and L2 parameters are set according to actual conditions. x b , h b parameters are calculated from the L1, L2 parameters and the seven-degree-of-freedom flight trajectory.

[0063] The mid-section measurement UAV array 200 includes several UAVs 201, 202, 203, 204, ..., such as Figure 2 As shown, they are deployed directly in front of the launch point at L... mn At a distance of (n = 1, 2, ...) meters, they are symmetrically distributed on both sides of the launch surface, at a distance of x from the launch surface. mn (n = 1, 2, ...) meters, height h mn (n = 1, 2, ...) meters, used to monitor the attitude, speed, and position of high-speed spinning objects during flight. L mn The parameters should be set according to the actual situation. mn h mn By L mn The parameters and the seven-degree-of-freedom flight trajectory were calculated.

[0064] The landing area measurement UAV array 300 includes two UAVs 301 and 302, such as Figure 2 As shown, 301 is deployed directly above the pre-set landing point to detect the landing location. 302 is deployed in front of the pre-set landing point of the high-speed spinning vehicle to monitor parachute deployment during soft recovery.

[0065] The communication UAV array 400 is used to provide relay nodes for the communication link, and can maintain communication with the in-situ measurement device and the measurement UAV array throughout the process, and transmit the measurement information to the ground workstation in real time to form an air-to-ground communication link for real-time network communication.

[0066] The computing center 500 installed on the ground workstation receives optical measurement data from the measurement drone arrays 100, 200, and 300 in real time via the communication drone array 400, and processes it in real time or post-processing.

[0067] The weather balloon 600 can carry instruments up and transmit back the current atmospheric pressure, temperature, humidity, wind direction and wind speed. It is generally released before the experiment begins.

[0068] Example 2:

[0069] This embodiment provides a high-rotational-parameter unmanned aerial vehicle (UAV) array optical measurement device, such as... Figure 3 and Figure 4 As shown, it includes a weather balloon 600, a drone array base station 700, a measurement drone array unit, an RTK base station 800, a communication drone array unit, and a computing center 500.

[0070] The measuring UAV array unit, i.e. the launching area measuring UAV array 100, the middle section measuring UAV array 200, and the landing area measuring UAV array 300, mainly includes an optical measuring load, a communication module, and a measuring UAV flight control unit. The optical measuring load is the core of the device, including a high-speed, high-resolution visible light camera, which is used to capture sequential images of the high-rotation flying body. The communication module is responsible for sending the raw data or pre-processed data obtained by the data acquisition storage unit to the outside. It establishes a downlink with the communication UAV array unit, usually using high-speed, low-latency microwave or laser communication technology. The measuring UAV flight control unit includes an integrated high-precision GNSS receiver, an inertial measurement unit, and a flight control computer. It receives the flight path instructions from the computing center and accurately controls the position and attitude of the UAV in the air.

[0071] The communication UAV array unit is a component of the communication UAV array 400, which is an airborne mobile communication network, including a communication UAV flight control unit and a core communication unit. The communication UAV flight control unit controls the UAVs to form a specific formation in the predetermined airspace, and constructs a stable communication network coverage. The core communication unit is a multi-band, multi-protocol, software-defined intelligent communication system, including three links and a MESH network inside the array. The downlink with the measuring array receives massive video and image data from multiple measuring UAVs through a high-speed microwave or laser communication link. The uplink with the computing center integrates all the data and transmits it to the ground computing center through a large-bandwidth satellite communication. The MESH network inside the array exchanges data and control information between the UAVs, achieving load balancing and link redundancy, and single-node failure does not affect the relay function of the entire network.

[0072] The computing center 500 is responsible for data reception, processing, and fusion. It includes a real-time processing module, a post-processing module, a communication hub, and a cluster control module. The real-time processing module is responsible for information fusion of the in-situ measurement data received by the communication UAV array unit and the optical measurement data of the measuring UAV array under a unified space-time reference through real-time filtering. The post-processing module is responsible for converting the fusion processing of the measurement data into parameter fusion estimation after the test is completed, giving high-precision estimation of the flight parameters of the high-rotation flying body and the errors of the measuring UAV array and the in-situ measurement system, and calibrating the measuring device. The communication hub realizes bidirectional data transmission between the computing center and the communication UAV. The cluster control module realizes UAV flight path planning, task allocation, and sends instructions to the UAV array base station 700, realizing the take-off, formation adjustment, and landing of the UAV.

[0073] The weather balloon 600 can carry a radiosonde to ascend and return the current atmospheric pressure, temperature, humidity, wind direction, and wind speed, and is usually released before the experiment begins.

[0074] The UAV array base station 700 can realize automatic storage, charging, take-off and landing scheduling of UAVs, support continuous operation of clusters, etc. Figure 5 As shown, it mainly includes hangar 701, automatic take-off and landing module 702, intelligent charging module 703, state monitoring system 704, mechanical scheduling mechanism 705. The hangar 701 adopts waterproof and dustproof metal structure to protect the electronic elements of the UAV; the automatic take-off and landing module 702 is a lifting platform, the surface of which is provided with a guide mark, which can cooperate with the visual positioning module of the UAV to realize accurate take-off and landing of the UAV, and is also provided with a light guiding system to provide take-off and landing guidance for the UAV in night or low visibility environment; the intelligent charging module 703 is a wireless charging seat which can support automatic wireless charging of the UAV. The state monitoring system 704 includes an environmental sensor for monitoring the temperature, humidity and other environmental states in the hangar and a UAV state sensor for monitoring the battery capacity and motor speed of the UAV, which is used to monitor the health of the system. The mechanical scheduling mechanism 705 can automatically close the movable door of the hangar when the system is idle or in harsh environment, so as to realize the sealing protection of the hangar; when the system starts or the UAV takes off, the door is automatically opened to provide a take-off channel for the UAV.

[0075] The RTK base station unit 800 provides high-precision positioning reference for the UAV cluster through real-time dynamic difference technology. It mainly includes a satellite receiver, a high-gain antenna and a difference signal transmitter.

[0076] Embodiment 3:

[0077] The embodiment provides a high-rotation flight parameter UAV array light measurement method, as shown in Figure 6 When processing the data transmitted back by the measurement UAV, the following steps are included:

[0078] Step S21, data preprocessing.

[0079] First, the original video data transmitted back by the UAV is decoded and processed to convert it into a continuous image frame sequence. Then, each frame of image is subjected to size standardization processing, and all frames are uniformly adjusted to a preset fixed resolution. Then, the standardized image is subjected to USM sharpening processing, i.e. formula (1), which specifically includes: performing Gaussian blur processing on the image to obtain a low-frequency component, subtracting the low-frequency component from the original image to obtain high-frequency detail information, setting an appropriate threshold to filter noise, and then weighting and superimposing the high-frequency details back to the original image according to a preset sharpening intensity coefficient.

[0080] I sharp (x,y)=I orig (x,y)+λ·(I orig (x,y)-(G σ *Iorig )(x,y)) (1)

[0081] where I orig (x,y) is the pixel value of the original image at coordinate (x,y). G σ is the Gaussian blur kernel with standard deviation σ, * is the convolution operation, and λ is the sharpening intensity coefficient.

[0082] Finally, the sharpened image is inverted to enhance the visibility of dark objects, i.e., formula (2).

[0083] I enhanced (x,y) = 255 - I sharp (x,y) (2)

[0084] where 255 is the maximum pixel value of an 8-bit image.

[0085] Step S22, frame alignment registration processing.

[0086] When the UAV is hovering, the camera view angle may shift slightly due to factors such as airflow disturbance and gimbal micro-motion. To eliminate image displacement caused by camera movement and establish a unified space-time coordinate system, frame alignment registration processing is required.

[0087] The first frame of the video sequence is selected as the reference coordinate system reference image. For each subsequent frame, first convert it to a grayscale image with the reference frame, then calculate the cross-power spectrum, i.e., formula (3).

[0088]

[0089] where I1 is the first frame image, I2 is the frame to be registered, F{·} is the two-dimensional Fourier transform, F * {·} is the complex conjugate of the Fourier transform, (u,v) is the pixel coordinate in the frequency domain, and R(u,v) is the calculated normalized cross-power spectrum. This method eliminates the effects of light changes by frequency domain correlation and retains phase difference information for displacement detection.

[0090] The translation parameters between frames are determined by detecting the phase correlation peak of the cross-power spectrum, i.e., formula (4).

[0091] (Δx,Δy) = argmax (x,y) |F -1 {R(u,v)}(x,y) | (4)

[0092] where F -1 {·} is the inverse transform of the two-dimensional Fourier transform, and argmax (x,y)To find the coordinate position with the maximum function value, (Δx, Δy) is the detected inter-frame translation.

[0093] To improve the registration accuracy, the RANSAC algorithm is used to optimize the estimated transformation parameters and eliminate the influence of outliers. According to the optimized transformation parameters, the affine transformation is applied to the current frame to make it accurately aligned with the reference frame. For the missing area of the image boundary generated after the transformation, the mirror filling method is used for compensation processing to ensure the integrity of the image.

[0094] Step S23, background modeling and moving target detection.

[0095] A plurality of background sample images are collected in advance, and a background model is constructed by weighted averaging, wherein the weight of each background image is dynamically adjusted according to the closeness between its collection time and detection time. The same pre-processing procedure as the input image is applied to the constructed background model, that is, step S21. In the detection stage, the current frame and the background model are first converted into grayscale images, and the absolute difference image between the two is calculated. Then, an adaptive threshold method based on the Otsu algorithm is used to binarize and segment the difference image, wherein the threshold sensitivity coefficient is automatically adjusted according to the scene complexity. The binarized image is sequentially subjected to morphological opening operation to remove noise points, and then subjected to morphological closing operation to fill the internal cavities of the target, that is, formula (5).

[0096]

[0097] In the formula, D is the original image, and S is the structural element, which is a disc-shaped kernel here. The morphological opening operation is a morphological opening operation, that is, erosion first, and then expansion. The morphological closing operation is a morphological closing operation, that is, expansion first, and then erosion.

[0098] Finally, the connected component analysis algorithm is used to extract the foreground region that meets the preset area threshold, and the minimum circumscribed rectangle of each connected component is calculated as the target bounding box.

[0099] Step S24, flight body enhancement and feature extraction.

[0100] According to the known actual physical size of the flight body and the geometric parameters of the imaging system, the theoretical projection size of the target in the image is calculated. Based on this, the boundary of the detected target region is expanded, and the expansion amount is adjusted according to the target motion speed and inter-frame motion. The ROI is extracted in the expanded region, that is, formula (6):

[0101]

[0102] In the formula, L is the actual length of the flight body, θ is the included angle between the flight body and the optical axis, z is the shooting distance, p y p z is the pixel size of the camera.

[0103] The ROI is enhanced in resolution by using a deep learning-based super-resolution reconstruction algorithm. Then, the enhanced image is edge sharpened using a Laplacian operator to highlight the contour features of the target. The accurate position information of the target in each frame of image is recorded, including the top-left corner coordinates, width, height and center point coordinates of the bounding box, etc.

[0104] Step S25, multi-frame fusion and trajectory generation.

[0105] First, the time domain median filter is performed on all preprocessed image frames to generate a static background image. Then, the enhanced target ROI in each frame is fused into the background image according to the center coordinate position by using alpha blending, i.e. formula (7).

[0106] I comp (x,y)=(1-α)I bg (x,y)+αI obj (x,y) (7)

[0107] In the formula, I comp (x,y) is the blended picture, I bg (x,y) is the background picture, I obj (x,y) is the detected target picture, and a is the transparency.

[0108] For the center point of the target in consecutive frames, a cubic B-spline curve is used for interpolation connection to generate a smooth motion trajectory.

[0109] Step S26, flight body motion analysis.

[0110] Based on the sequence of bounding box coordinates of the target in each frame, tail feature points that can stably represent the motion state of the target are selected. The position changes of these feature points in consecutive frames are analyzed, and the central difference method is used to calculate the instantaneous velocity components, wherein the time interval is determined according to the frame rate of the video, i.e. formula (8).

[0111]

[0112] In the formula, f s is the frame rate of the video.

[0113] The calculated horizontal and vertical velocities are vector synthesized to obtain the actual motion velocity of the target, i.e. formula (9).

[0114]

[0115] Step S27: Flight body photometric data fusion

[0116] The Kalman filter is used to fuse the optical measurement data with in-situ measurement, radar measurement and other measurement system data, to give high-precision estimation of the full flight trajectory parameters of the flying body and measurement system error, and to perform feedback calibration on the measurement device.

[0117] Embodiment 4:

[0118] The embodiment provides a real combat experiment scheme of a high-rotation flight parameter unmanned aerial vehicle array optical measurement method, device and system, as shown in Figure 7 The unmanned aerial vehicle array is used to measure the high-rotation flight parameters of the high-rotation flying body during the flight test of the high-rotation flying body in the target range, and includes the following steps:

[0119] Step S31, a meteorological balloon 600 is released to detect atmospheric parameters.

[0120] Before the test is performed, the meteorological balloon 600 carrying a radio sonde is released, and the atmospheric parameters such as air pressure, temperature and humidity and wind direction and wind speed under the current atmospheric condition are transmitted back to the calculation center 500, so that the meteorological data for the real combat test is bound.

[0121] Step S32, the measurement unmanned aerial vehicle array 100, 200, 300 is arranged.

[0122] Before starting the test, the unmanned aerial vehicles required for the test need to be hovered at the specified positions, as shown in Figure 8 and Figure 9 .

[0123] The three unmanned aerial vehicles 101, 102 and 103 of the launch area measurement unmanned aerial vehicle array 1 take off from the H1 point, as shown in Figure 6 , Figure 7 , wherein the 103 vertically ascends h b , then advances x0 in the negative direction of the y axis, and then hovers. b The 102 vertically ascends h b , then advances L1+L2 in the positive direction of the x axis, then advances x0+x in the negative direction of the y axis, and then hovers. b The 101 vertically ascends h b , then advances L1+L2 in the positive direction of the x axis, then advances x0-x in the negative direction of the y axis, and then hovers.

[0124] The 2n unmanned aerial vehicles 201, 202 of the middle section measurement unmanned aerial vehicle array 2 take off from the H1 point, as shown in Figure 8 and Figure 9 , wherein the 202 vertically ascends h mn , then advances L1+L in the positive direction of the x axis, then advances x0+x in the negative direction of the y axis, and then hovers. mn The 201 vertically ascends h mn , then advances L1+L in the positive direction of the x axis, then advances x0+x in the negative direction of the y axis, and then hovers. b The 201 vertically ascends h mnRear edge y-axis negative direction forward x0-x mn Rear hover.

[0125] Wherein the drop zone measurement unmanned aerial vehicle array 3 of 2 unmanned aerial vehicles 301, 302 takes off from the preset drop point H2, as Figure 10 301 is vertically raised h e1 Rear hover, 302 is vertically raised h e2 Rear edge x-axis negative direction forward x e Rear hover.

[0126] Step S33, the communication unmanned aerial vehicle array 400 is arranged to form a network, and a communication link is established.

[0127] The unmanned aerial vehicle array is arranged to form a network as a relay node on the communication link, receives the measurement data on the in-situ measurement device on the high-rotation flying body and the measurement data of the measurement unmanned aerial vehicle array throughout the journey, and forms a data transmission link of the in-situ measurement system, the unmanned aerial vehicle optical measurement system and the ground computing center.

[0128] Step S34, real combat test is carried out, and the measurement data is collected in real time.

[0129] According to the proposed test scheme, the in-situ measurement data received by the unmanned aerial vehicle communication relay node and the unmanned aerial vehicle optical measurement data are transmitted back to the computing center 500, the full flight trajectory parameters of the flying body and the measurement system error are given high-precision estimation through information fusion and filtering under the same space-time reference, and the measurement device is feedback calibrated.

[0130] Step S35, the drop point landing state is detected, and recovery is carried out.

[0131] At the drop point position, the drop zone measurement unmanned aerial vehicle array 300 and the preset flight trajectory are used to evaluate the drop point position, so as to provide guidance for the arrangement of the measurement unmanned aerial vehicle network and the recovery after landing. The drop zone measurement unmanned aerial vehicle array 300 is used to accurately measure the landing position and attitude of the flying body, and the unmanned aerial vehicle communication link is used to transmit back to the computing center.

[0132] In summary, the above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A high spin flight parameter drone array light measurement system, characterized in that, The array of launch area measuring UAVs (100), the array of midcourse measuring UAVs (200), the array of impact point area measuring UAVs (300), the array of communication UAVs (400), the computing center (500) installed on the ground station, and the meteorological balloon (600); The array of launch area measuring UAVs (100) comprises at least three UAVs; at least one UAV is deployed right behind the launch point to monitor the attitude, speed, and position of the high-rotation flying body in the yaw direction after launch; at least two UAVs are deployed right in front of the launch point and symmetrically distributed on both sides of the launch surface to monitor the attitude, speed, and position of the high-rotation flying body in the direction of flight and height after launch; The array of midcourse measuring UAVs (200) comprises a plurality of UAVs, which are respectively deployed at different positions in front of the launch point and symmetrically distributed on both sides of the launch surface at the same distance from the launch surface to monitor the attitude, speed, and position of the high-rotation flying body during flight; The array of impact point area measuring UAVs (300) comprises at least two UAVs, at least one of which is deployed directly above the preset impact point to detect the impact point position; at least one is deployed in front of the preset impact point of the high-rotation flying body to monitor the parachute opening during soft recovery; The array of communication UAVs (400) is used to provide relay nodes for the communication link, and maintains communication with the in-situ measurement device on the high-rotation flying body, the array of launch area measuring UAVs (100), the array of midcourse measuring UAVs (200), and the array of impact point area measuring UAVs (300) throughout the journey, and transmits measurement information to the ground station in real time to form an air-ground communication link for real-time networking communication; The computing center (500) receives the photometric data of the array of launch area measuring UAVs (100), the array of midcourse measuring UAVs (200), and the array of impact point area measuring UAVs (300) in real time through the array of communication UAVs (400) and processes it in real time or after the event; The meteorological balloon (600) carries instruments to ascend and return the current atmospheric pressure, temperature, humidity, wind direction, and wind speed.

2. A high spin flight parameter UAV array LIDAR system as claimed in claim 1, wherein, The array of launch area measuring UAVs (100), the array of midcourse measuring UAVs (200), and the array of impact point area measuring UAVs (300) comprise optical measurement payloads, communication modules, and measurement UAV flight control units; the optical measurement payload comprises a visible light camera for capturing sequential images of the high-rotation flying body; the communication module uses microwave or laser communication technology to establish a downlink with the array of communication UAVs unit and is responsible for sending the original data or pre-processed data obtained by the data acquisition storage unit to the outside; the measurement UAV flight control unit comprises a GNSS receiver, an inertial measurement unit, and a flight control computer, which receives the flight path instructions from the computing center (500) to accurately control the position and attitude of the UAV in the air.

3. A high spin flight parameter UAV array LIDAR system as in claim 1, wherein, The communication unmanned aerial vehicle array (400) comprises a communication unmanned aerial vehicle flight control unit and a core communication unit. The communication unmanned aerial vehicle flight control unit controls the unmanned aerial vehicles to form a specific formation in a predetermined airspace, and constructs a stable communication network coverage. The core communication unit is a multi-band, multi-protocol, software-defined intelligent communication system, which comprises three links and a MESH network inside the array. The downlink with the measurement array is transmitted through a high-speed microwave or laser communication link, and the video and image data transmitted by multiple measurement unmanned aerial vehicles are received in parallel. The uplink with the computing center integrates all the data, and then transmits the data to the ground computing center (500) through satellite communication. The unmanned aerial vehicles inside the array exchange data and control information through the MESH network wireless networking, so as to realize load balancing and link redundancy.

4. A high spin flight parameter UAV array LIDAR system as in claim 1, wherein, The computing center (500) comprises a real-time processing module, a post-processing module, a communication hub and a cluster control module. The real-time processing module is responsible for information fusion of in-situ measurement data received by the communication unmanned aerial vehicle array unit and optical measurement data of the measurement unmanned aerial vehicle array under a unified space-time reference through a real-time filtering scheme. The post-processing module is responsible for converting the fusion processing of the measurement data into the fusion estimation of parameters after the test is completed, giving high-precision estimation of the flight parameters of the high-rotation flying body and the errors of the measurement unmanned aerial vehicle array and the in-situ measurement system, and calibrating the measurement device. The communication hub realizes bidirectional data transmission between the computing center and the communication unmanned aerial vehicle. The cluster control module realizes flight path planning, task allocation of the unmanned aerial vehicle, and issues instructions to the unmanned aerial vehicle array base station (700), so as to realize take-off, formation adjustment and landing of the unmanned aerial vehicle.

5. A high spin flight parameter UAV array LIDAR system as in claim 1, wherein, The unmanned aerial vehicle array base station (700) is further included, which is used for realizing automatic storage, charging, take-off and landing scheduling of the unmanned aerial vehicle, and supporting continuous operation of the cluster.

6. A high spin flight parameter UAV array LIDAR system as claimed in claim 5, wherein, The unmanned aerial vehicle array base station (700) comprises a hangar (701), an automatic take-off and landing module (702), an intelligent charging module (703), a state monitoring system (704) and a mechanical scheduling mechanism (705). The hangar (701) adopts a waterproof and dustproof metal structure to protect electronic elements of the unmanned aerial vehicle. The automatic take-off and landing module (702) is a lifting platform, which is provided with a guide mark on the surface and can realize precise take-off and landing of the unmanned aerial vehicle in cooperation with a visual positioning module of the unmanned aerial vehicle. The lifting platform is also provided with a light guiding system to provide take-off and landing guidance for the unmanned aerial vehicle in night or low-visibility environment. The intelligent charging module (703) is a wireless charging seat, which can support automatic wireless charging of the unmanned aerial vehicle. The state monitoring system (704) comprises an environment sensor for monitoring temperature, humidity and other environmental states in the hangar and an unmanned aerial vehicle state sensor for monitoring battery capacity and motor speed of the unmanned aerial vehicle, which are used for monitoring health conditions of the system. The mechanical scheduling mechanism (705) can automatically close the movable door of the hangar in an idle state or in a harsh environment, so as to realize sealing protection of the hangar. When the system is started or the unmanned aerial vehicle takes off and lands, the movable door is automatically opened to provide a take-off and landing channel for the unmanned aerial vehicle.

7. A high spin flight parameter UAV array LIDAR system as in claim 1, wherein, The RTK base station unit (800) is further included, which provides a high-precision positioning reference for the unmanned aerial vehicle cluster through real-time dynamic difference technology.

8. An optical measurement method based on the high-spin flight parameter unmanned aerial vehicle array optical measurement system according to any one of claims 1 to 7, characterized in that, The RTK base station unit (800) is further included, which provides a high-precision positioning reference for the unmanned aerial vehicle cluster through real-time dynamic difference technology. Step S31, before the light measurement, release the weather balloon (600) to detect the atmospheric parameters, and return the atmospheric parameters such as air pressure, temperature and humidity, and wind direction and speed to the calculation center (500) under the current atmospheric conditions; Step S32, before the light measurement, the launch area measurement UAV array (100), the middle section measurement UAV array (200), and the landing point area measurement UAV array (300) are suspended at the set position; Step S33, the communication UAV array (400) is arranged to form a network, and a communication link is established: The UAV array network is arranged as a relay node on the communication link, receives the measurement data on the in-situ measurement device on the high-speed rotating flying body and the measurement data of the measurement UAV array, forms a data transmission link of the in-situ measurement system, the UAV light measurement system, and the ground calculation center (500), and transmits the data to the calculation center (500); Step S34, according to the proposed scheme, the in-situ measurement data received by the communication relay node in the communication UAV array (400) and the UAV optical measurement data are returned to the calculation center (500), the full flight trajectory parameters of the high-speed rotating flying body and the measurement system error are estimated through information fusion and filtering under the same space-time reference, and feedback calibration is performed on the measurement device; Step S35, the landing point is detected and the target state is detected, and the recovery is performed; At the landing point, the landing point area measurement UAV array (300) and the preset flight trajectory are used to evaluate the landing point, to provide guidance for the arrangement of the measurement UAV network and the recovery after the target is hit, the landing point area measurement UAV array (300) is used to accurately measure the landing position and attitude of the flying body, and the UAV communication link is used to return the data to the calculation center.

9. An optical measurement method based on the high-spin flight parameter unmanned aerial vehicle array optical measurement system according to any one of claims 1 to 7, characterized in that, Comprise: Step S21, data preprocessing. First, the original video data transmitted by the UAV is decoded and processed to convert it into a continuous image frame sequence; then, each frame image is standardized in size to adjust all frames to a preset fixed resolution; then, the standardized image is subjected to USM sharpening processing, which specifically includes: performing Gaussian blur processing on the image to obtain a low-frequency component, subtracting the low-frequency component from the original image to obtain high-frequency detail information, filtering noise, and then weighting and superimposing the high-frequency details back to the original image according to a preset sharpening intensity coefficient; Step S22, frame alignment and registration processing: A unified space-time coordinate system needs to be established, and frame alignment and registration processing is required; the first frame of the video sequence is selected as the reference coordinate system reference image; for each subsequent frame, first convert it and the reference frame into grayscale images, then calculate their cross power spectrum, and determine the translation parameters between the frames by detecting the phase correlation peak value of the cross power spectrum; the RANSAC algorithm is used to optimize the preliminary estimated transformation parameters; according to the optimized transformation parameters, affine transformation is applied to the current frame to align it with the reference frame; for the image boundary missing area generated after the transformation, a mirror filling method is used for compensation processing to ensure the integrity of the image; Step S23, background modeling and motion target detection: A plurality of background sample images are collected in advance, and a background model is constructed by weighted average, wherein the weight of each background image is dynamically adjusted according to the closeness of the collection time and the detection time; the same pre-processing procedure as the input image is applied to the constructed background model, that is, step S21; in the detection stage, the current frame and the background model are first converted into gray images, and the absolute difference image between the two is calculated, and then an adaptive threshold method based on the Otsu algorithm is used to binarize and segment the difference image, wherein the threshold sensitivity coefficient is automatically adjusted according to the scene complexity; the binarized image is sequentially subjected to morphological open operation to remove noise points, and then closed operation is performed to fill the internal cavities of the target; Finally, the connected component analysis algorithm is used to extract the foreground region meeting the preset area threshold, and the minimum circumscribed rectangle of each connected component is calculated as the target bounding box; Step S24, flight body enhancement and feature extraction: According to the known actual physical size of the flight body and the geometric parameters of the imaging system, the theoretical projection size of the target in the image is calculated; based on this, the boundary of the detected target region is expanded, and the expansion amount is dynamically adjusted according to the target motion speed and inter-frame motion; An edge sharpening process is performed on the enhanced image using a Laplacian operator to highlight the contour features of the target, while recording the accurate position information of the target in each frame of image, including the top-left corner coordinates, width, height and center point coordinates of the bounding box; Step S25, multi-frame fusion and trajectory generation: First, time domain median filtering is performed on all pre-processed image frames to generate a static background image; then the enhanced target ROI in each frame is fused into the background image in an alpha blending manner according to the center coordinate position; For the target center points in consecutive frames, a cubic B-spline curve is used for interpolation connection to generate a smooth motion trajectory line; Step S26, flight body motion analysis: Based on the boundary box coordinate sequence of the target in each frame, tail feature points that can stably represent the motion state of the target are selected; the position changes of these feature points in consecutive frames are analyzed, and the central difference method is used to calculate the instantaneous velocity components, wherein the time interval is determined according to the frame rate of the video; The calculated horizontal and vertical velocities are vector synthesized to obtain the actual motion velocity of the target; Step S27: Fusion of photometric data of flight body: The Kalman filter is used to fuse the photometric data with other measurement system data such as in-situ measurement and radar measurement, to estimate the full flight trajectory parameters of the flight body and the measurement system error, and to feedback and calibrate the measurement device.

Images