A large field of view three-dimensional density field reconstruction method and measurement system based on space-based multi-platform array

Abstract

The application discloses a large-view-field three-dimensional density field reconstruction method and a measuring system based on an air-based multi-platform array, and belongs to the technical field of flow field measurement.The air-based multi-platform array is an airborne platform carrying at least five cameras;the air-based multi-platform array synchronously collects natural background images without flow field disturbance and disturbance images with flow field disturbance, and the natural background images without flow field disturbance are taken as reference images;the air-based multi-platform takes the reference images under the field angles of the cameras as the reference, estimates the displacement of each pixel point in the corresponding disturbance image through an optical flow algorithm, and obtains the displacement field of each disturbance image;the NeDF model is constructed through a multi-layer perception mechanism, the three-dimensional refractive index gradient is reconstructed from the displacement field based on the NeDF model;the three-dimensional Poisson equation is solved by using the three-dimensional refractive index gradient field, and the three-dimensional refractive index distribution is obtained;the Gladstone-Dale equation is solved by using the three-dimensional refractive index field, and the three-dimensional density field of the flow field is obtained.The application realizes high-fidelity three-dimensional density field reconstruction of a super-large-range flow field by using a natural background.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of flow field measurement and computer vision, and particularly relates to a large field of view three-dimensional density field reconstruction method and a measuring system based on a space-based multi-platform array. BACKGROUND

[0002] Background Oriented Schlieren (BOS) is a mature non-contact optical measurement method, which quantitatively inverts the density field information by observing the apparent displacement of the background pattern caused by light passing through the non-uniform density field. Tomographic BOS (TBOS) combines BOS and tomographic imaging technology, which can realize three-dimensional reconstruction of the flow field through synchronous observation from different angles by multiple cameras.

[0003] However, the existing TBOS technology is mainly applied in the controllable environment in the laboratory. When it is tried to be extended to the super-large scale, open outdoor environment, such as aircraft wake, industrial emissions, atmospheric phenomena, etc., there are still the following technical bottlenecks: under laboratory conditions, the synchronization of multiple cameras usually depends on the signal line hardware parallel connection, but this way is complex in wiring and poor in flexibility, which is difficult to adapt to large-scale, distributed outdoor measurement scenarios; accordingly, in large-scale outdoor observation, it is often necessary to rely on unmanned aerial vehicles, balloons and other airborne platforms to carry cameras to form an observation array, how to ensure high-precision, low-delay synchronization triggering in a wireless environment is still one of the key difficulties; on the other hand, the traditional background schlieren method relies on a random dot background plate made by hand, which can realize a unified spatial reference through a calibration plate in the laboratory, but when the observation range is expanded to a super-large scale, the size requirement of the background plate and the calibration plate is extremely large, and there are significant difficulties in manufacturing and arrangement, which is almost impossible to realize in actual outdoor environment. Therefore, how to establish a unified space-time reference without a calibration plate and ensure the geometric consistency of multi-camera data is a core problem that must be solved to realize large-scale tomographic reconstruction.

[0004] Therefore, there is an urgent need for a technical solution that can systematically solve the above displacement extraction and space-time synchronization challenges to realize accurate and reliable quantitative measurement of three-dimensional density field in large-scale, real outdoor environment. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a large field of view three-dimensional density field reconstruction method and a measuring system based on a space-based multi-platform array, which realizes high-fidelity three-dimensional density field reconstruction of a super-large range flow field by using a natural background, breaks through the field of view limitation of traditional tomographic technology, and can perform high-precision, three-dimensional density field quantitative measurement on unsteady flow field.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] A large field of view three-dimensional density field reconstruction method based on space-based multi-platform array, comprising the following steps:

[0008] Step S1: synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance by using a space-based multi-platform array, taking the natural background images without flow field disturbance as reference images; the space-based multi-platform array is an airborne platform carrying several cameras;

[0009] Step S2: taking the reference images under the field of view of each camera as a reference, estimating the displacement of each pixel point in the corresponding disturbance image to obtain the displacement field of each disturbance image;

[0010] Step S3: constructing a NeDF model through a multi-layer perception mechanism, and reconstructing a three-dimensional refractive index gradient from the displacement field based on the NeDF model to obtain a three-dimensional refractive index gradient field;

[0011] Step S4: solving a three-dimensional Poisson equation by using the three-dimensional refractive index gradient field to obtain a three-dimensional refractive index distribution;

[0012] Step S5: solving a Gladstone-Dale equation by using the three-dimensional refractive index distribution to obtain a three-dimensional density field of the flow field.

[0013] Further, the fields of view of all the cameras converge on the flow field region to be measured, and the lines of sight of all the cameras pass through the flow field region to be measured and fall on the natural background.

[0014] Further, step S2 comprises the following substeps:

[0015] Step S2.1: correcting the distortion of the corresponding reference image and disturbance image by using the camera intrinsic parameters, and performing gray scale normalization processing on the distortion-corrected reference image and disturbance image respectively to obtain a standardized reference image and a standardized disturbance image;

[0016] Step S2.2: constructing an optical flow constraint equation according to the standardized reference image and the corresponding standardized disturbance image, and obtaining a two-dimensional displacement field of the disturbance image relative to the reference image by using a multi-resolution image pyramid iterative optimization strategy.

[0017] Further, the construction process of the multi-resolution image pyramid is as follows: setting the number of layers of the image pyramid as n, and the resolution of the image pyramid gradually decreases from the bottom layer to the top layer, wherein the resolution of the bottom layer of the image pyramid is the resolution of the standardized reference image and the standardized disturbance image , and the resolution of each remaining layer of the image pyramid is represented as , S represents a scale factor, .

[0018] Further, step S2.2 comprises the following substeps:

[0019] Step S2.2.1: down-sampling the normalized reference image and the corresponding normalized perturbed image respectively to obtain the top layer of the image pyramid, constructing an over-determined equation set of optical flow constraint equation for the sampling pixels of the down-sampled normalized reference image and the corresponding normalized perturbed image, solving by using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image;

[0020] Step S2.2.2: up-sampling the image entering the next layer of the image pyramid, correcting the displacement of the normalized perturbed image relative to the normalized reference image according to the displacement increment obtained in the last layer;

[0021] Step S2.2.3: for each pixel on the normalized reference image, searching the corresponding position on the displacement-corrected normalized perturbed image, performing pixel interpolation processing to obtain a registration image corresponding to each pixel on the normalized reference image, and calculating the residual error between the normalized reference image and the registration image;

[0022] Step S2.2.4: constructing an over-determined equation set of optical flow constraint equation for the sampling pixels of the up-sampled reference image and the corresponding perturbed image, solving by using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image;

[0023] Step S2.2.5: repeating steps S2.2.2-S2.2.4 until the residual error converges to obtain the optimal displacement increment of the bottom layer of the image pyramid, and entering the next layer of the image pyramid;

[0024] Step S2.2.6: repeating steps S2.2.2-S2.2.5 until reaching the bottom layer of the image pyramid to obtain the two-dimensional displacement field of the perturbed image relative to the reference image.

[0025] Further, step S3 includes the following sub-steps:

[0026] Step S3.1: determining the real light deflection angle disturbed by the flow field according to the displacement field of the perturbed image;

[0027] Step S3.2: constructing a NeDF model through a multi-layer perception mechanism;

[0028] Step S3.3: inputting the three-dimensional coordinates of the point to be measured in the flow field into the NeDF model after position encoding, and outputting the three-dimensional refractive index gradient of the flow field to be measured;

[0029] Step S3.4: performing forward integration on the three-dimensional refractive index gradient of the flow field to be measured to predict the light deflection angle of the flow field to be measured, calculating a mean square error loss function through the real light deflection angle and the predicted light deflection angle, and updating the parameters of the NeDF model.

[0030] Step S3.5: Repeat step S3.3-step S3.4 until the mean square error loss function converges, obtaining the three-dimensional refractive index gradient of the flow field to be measured.

[0031] Further, the specific process of step S3.1 is:

[0032] For a point on the natural background without flow field disturbance C The imaging position of the point on the image plane is point A When the light ray of the point C passes through the flow field, according to the displacement field of the corresponding disturbance image, the imaging position of the point C after refraction deflection on the image plane is point .

[0033] Trace the point backwards along the light propagation direction, through the lens center of the camera lens, and intersect with the center plane of the flow field to be measured at point B .

[0034] Connect the point B with the point C to obtain the propagation direction vector of the light ray before entering the flow field , calculate the real light deflection angle , wherein represents the propagation direction vector of the light ray after entering the flow field.

[0035] Further, the prediction process of the light deflection angle of the point is:

[0036]

[0037] Wherein, represents the predicted value of the light deflection angle of the point, represents the three-dimensional refractive index gradient of the flow field to be measured, , represents the refractive index gradient of the flow field to be measured in the x axis direction, represents the refractive index gradient of the flow field to be measured in the y axis direction, represents the refractive index gradient of the flow field to be measured in the z axis direction. ray represents a light ray passing through the flow field to be measured along the light path of the point.

[0038] Further, the camera on the air-based multi-platform array is jointly optimized for external parameters:

[0039]

[0040] Wherein, denotes an extrinsic joint optimization function, denotes a rotation matrix of a camera denotes a natural texture feature point in a overlapped field of view image taken by denotes a rotation matrix of a camera on an aerial multi-platform array, denotes a translation vector of a camera on an aerial multi-platform array, denotes a three-dimensional coordinate of a natural texture feature point, denotes an intrinsic matrix of a camera on an aerial multi-platform array, denotes a perspective projection function.

[0041] Further, the present application also provides a three-dimensional density field measurement system applying the three-dimensional density field reconstruction method based on an aerial multi-platform array, comprising: an aerial multi-platform array, a displacement field calculation module, a three-dimensional refractive index gradient prediction module and a three-dimensional density field calculation module.

[0042] The aerial multi-platform array is used for synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance, and taking the natural background images without flow field disturbance as reference images.

[0043] The displacement field calculation module estimates the displacement of each pixel point in the corresponding disturbance image based on the reference image, and obtains the displacement field of each disturbance image.

[0044] The three-dimensional refractive index gradient prediction module reconstructs the three-dimensional refractive index gradient from the displacement field based on the NeDF model.

[0045] The three-dimensional density field calculation module determines the three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and obtains the three-dimensional density field of the flow field in combination with the Gladstone-Dale equation.

[0046] Compared with the prior art, the present application has the following beneficial effects: the present application adopts natural scenes as the background based on the large field of view three-dimensional density field reconstruction method and measurement system of the air-based multi-platform array, can avoid the construction of artificial background panels, significantly reduces the experimental cost and deployment complexity, while ensuring large-scale observation, has good adaptability and flexibility, and is especially suitable for outdoor environments where background panels cannot be deployed; at the same time, the NeDF model is constructed through a multi-layer perception mechanism to reconstruct the three-dimensional refractive index gradient, fuses position encoding and layered sampling, accurately captures flow field details, avoids information loss, does not need a pre-trained model, realizes high-fidelity three-dimensional refractive index gradient field reconstruction with low-dimensional parameterization, can adapt to sparse and limited viewing angle conditions, and can still realize high-quality reconstruction when the number of cameras is limited. The present application breaks through the field of view limitation of traditional tomography technology, can perform high-precision three-dimensional quantitative measurement on unsteady flow fields, has important scientific value and engineering application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the large field of view three-dimensional density field reconstruction method based on the air-based multi-platform array of the present application;

[0048] Figure 2 The structural schematic diagram of the air-based multi-platform array in the present application;

[0049] Figure 3 The schematic diagram of the real light deflection angle disturbed by the flow field in the present application;

[0050] Figure 4 The training flowchart of the NeDF model in the present application. DETAILED DESCRIPTION

[0051] The technical solutions of the present application will be further explained and described below in combination with the drawings.

[0052] As Figure 1 The flowchart of the large field of view three-dimensional density field reconstruction method based on the air-based multi-platform array of the present application, the large field of view three-dimensional density field reconstruction method includes the following steps:

[0053] Step S1: as Figure 2The air-based multi-platform array is an airborne platform carrying at least five cameras, the cameras are arranged on unmanned aerial vehicles, the unmanned aerial vehicles are arranged around the flow field to be measured, the camera posture is adjusted by the unmanned aerial vehicle holder, and the height of the unmanned aerial vehicle is adjusted so that the fields of view of all the cameras are jointly intersected in the flow field to be measured, and the lines of sight of all the cameras fall on the natural background with rich texture features through the flow field to be measured, so that the optical flow calculation failure caused by insufficient or repeated background texture is avoided. After the position and posture adjustment is completed, the unmanned aerial vehicle and the camera remain fixed during the entire experiment. The air-based multi-platform array is used to synchronously collect natural background images without flow field disturbance and disturbance images with flow field disturbance, and the natural background images without flow field disturbance are used as reference images.

[0054] In one technical solution of the present application, when the flow field to be measured does not generate disturbance, a synchronous trigger instruction is sent by the ground server to trigger the multiple cameras to simultaneously shoot natural background images, a plurality of images are continuously collected, the plurality of images collected by each camera are averaged at the pixel level to obtain an average image, and the average image is used as a reference image for subsequent processing. The flow field disturbance source is started, and the synchronous trigger instruction is sent again to continuously collect a plurality of disturbance images with flow field disturbance.

[0055] In one technical solution of the present application, the multiple cameras are calibrated for internal and external parameters, and the calibration process is divided into two stages of laboratory pre-calibration and on-site optimization to ensure the accuracy and robustness of the final internal and external parameters.

[0056] Before the air-based multi-platform array is deployed, each camera in the air-based multi-platform array needs to be independently pre-calibrated for internal parameters in a ground controlled environment. 10 to 50 frames of effective images of the checkerboard calibration board can be collected for each camera. When shooting, the calibration board is slowly moved and its posture is adjusted in the field of view of the camera to cover different pitch angles, yaw angles and roll angles, so that the feature points of the calibration board are completely and clearly covered in the entire field of view, including the center, edge and corners of the image. The entire process needs to be carried out under stable and uniform lighting conditions to avoid reflection or shadow, and appropriate shutter speed is set to prevent image blur. For the image set collected for each camera, a mature camera calibration algorithm is used to calculate the initial internal parameters, including focal length, principal point coordinates and distortion coefficients.

[0057] In the field integrated calibration process, firstly, the natural texture feature points are extracted from the images collected by the multi-camera in the ground overlapping field of view, the relative external parameter calibration between the multi-cameras is realized by matching the natural texture feature points and further solving the rotation matrix and the translation vector of each camera by using the PnP method, so that the relative spatial position relationship between the cameras is obtained. After the alignment of the external parameters by the above method, the joint optimization of the multi-camera external parameters is used by Bundle Adjustment, so that the geometric constraints between the multi-angles can be fully utilized, the overall consistency and accuracy of the external parameter estimation are improved, and the influence of the single camera calibration error on the overall system is reduced.

[0058] The process of the joint optimization of the multi-camera external parameters in the application is as follows:

[0059]

[0060] Among them, represents the joint optimization function of the external parameters, represents the rotation matrix of the camera on the air-based multi-platform array, represents the first natural texture feature point in the overlapping field of view image shot by the camera on the air-based multi-platform array, represents the translation vector of the camera on the air-based multi-platform array, represents the rotation matrix of the camera on the air-based multi-platform array, represents the translation vector of the camera on the air-based multi-platform array, represents the three-dimensional coordinates of the first natural texture feature point, represents the intrinsic matrix of the camera on the air-based multi-platform array, represents the perspective projection function. Subsequently, not less than 5 to 10 ground control points (GCPs) which are easy to identify are laid on the ground in the overlapping area of the cameras, and the three-dimensional coordinates of the GCPs in the preset world coordinate system are obtained by using high-precision measuring equipment such as real-time dynamic GNSS or total station. Based on the GCPs, the relative calibration results of the multi-camera are converted to the world coordinate system, the introduction of the absolute scale and the determination of the coordinate reference are realized, and the global external parameter relationship of the multi-camera in the unified world coordinate system is established. Step S2: Taking the reference image under the field of view angle of each camera as the reference, the displacement of each pixel point in the corresponding perturbation image is estimated by using the optical flow algorithm, and the displacement field of each perturbation image is obtained; including the following sub-steps:

[0061] Step S3: Taking the reference image under the field of view angle of each camera as the reference, the displacement of each pixel point in the corresponding perturbation image is estimated by using the optical flow algorithm, and the displacement field of each perturbation image is obtained; including the following sub-steps:

[0062] Step S4: Taking the reference image under the field of view angle of each camera as the reference, the displacement of each pixel point in the corresponding perturbation image is estimated by using the optical flow algorithm, and the displacement field of each perturbation image is obtained; including the following sub-steps:

[0063] ​​Step S2.1: the internal parameter matrix and distortion coefficient of the camera are used for distortion correction of the corresponding reference image and perturbation image, so that the influence of radial distortion, tangential distortion and other lens effects on the pixel geometric position is eliminated, and the corresponding relationship between the image coordinates and the real physical space is ensured to be accurate and reliable. And the reference image and the perturbation image after distortion correction are subjected to gray scale normalization processing respectively to obtain the standardized reference image and the standardized perturbation image, which can effectively suppress the damage of light change, imaging noise and exposure difference to the brightness consistency assumption, and improve the robustness of the optical flow calculation.

[0064] The process of gray scale normalization processing in the application is:

[0065]

[0066] Among them, represents the gray value of the reference image or the perturbation image after distortion correction at the pixel position , and respectively represent the minimum value and the maximum value of the gray value of the reference image or the perturbation image after distortion correction, and through the linear transformation, the gray scale range of the image is mapped to the [0, 1] interval.

[0067] Step S2.2: constructing an optical flow constraint equation according to the standardized reference image and the corresponding standardized perturbation image , wherein, , respectively are the gradients of the pixel points in the spatial direction , of the standardized reference image, is the gray scale change rate in the time direction, is the displacement component of the point in the standardized perturbation image. Since one optical flow constraint equation cannot solve two unknowns and , additional constraints need to be introduced for solving, and the application adopts an iterative optimization strategy of a multi-resolution graph pyramid to obtain a two-dimensional displacement field of the perturbation image relative to the reference image.

[0068] The construction process of the multi-resolution graph pyramid of the application is: the number of layers of the graph pyramid is set to n, and the resolution of the graph pyramid gradually decreases from the bottom layer to the top layer, so that large displacement becomes small displacement in the graph pyramid layer of coarse resolution, facilitating layer-by-layer estimation. Among them, the bottom layer resolution of the graph pyramid is the resolution of the standardized reference image and the standardized perturbation image , and the resolution of each remaining layer of the graph pyramid is represented as , S represents a scale factor. .

[0069] The acquisition process of the two-dimensional displacement field of the perturbed image relative to the reference image in the application is as follows:

[0070] Step S2.2.1: down-sampling the normalized reference image and the corresponding normalized perturbed image respectively to obtain the top layer of the image pyramid, assuming that the displacement increment of all pixels in a sliding window is equal, constructing an over-determined equation group of the optical flow constraint equation for the sampling pixels of the down-sampled normalized reference image and the corresponding normalized perturbed image.

[0071]

[0072] wherein, I(x,y) represents the pixel value of the normalized reference image at the point (x,y) ;

[0073] using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image.

[0074] Step S2.2.2: up-sampling the image entering the next layer of the image pyramid, correcting the displacement of the normalized perturbed image relative to the normalized reference image according to the displacement increment obtained in the last layer.

[0075] Step S2.2.3: for each pixel on the normalized reference image, searching for the corresponding position on the displacement-corrected normalized perturbed image. Since the fractional coordinates are often not integer coordinates, the pixel value cannot be directly taken, and pixel interpolation processing needs to be performed on the normalized perturbed image. I 2 to obtain the gray value, so that the registration image corresponding to each pixel on the normalized reference image is obtained, and the residual error of the normalized reference image and the registration image is calculated. wherein, I(x,y) represents the gray value of the normalized reference image.

[0076] Step S2.2.4: constructing an over-determined equation group of the optical flow constraint equation for the sampling pixels of the up-sampled reference image and the corresponding perturbed image, using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image.

[0077] Step S2.2.5: repeating the steps S2.2.2-S2.2.4 until the residual error converges, obtaining the optimal displacement increment of the bottom layer of the image pyramid, and entering the next layer of the image pyramid.

[0078] ​​​​​​​​​​Step S2.2.6: Repeat steps S2.2.2-S2.2.5 until the bottom layer of the pyramid is reached, obtaining a two-dimensional displacement field of the perturbed image relative to the reference image.

[0079] Step S3: Building a NeDF model through a multi-layer perception mechanism, reconstructing a three-dimensional refractive index gradient from the displacement field based on the NeDF model, fusing position encoding and hierarchical sampling, accurately capturing flow field details, avoiding information loss, without pre-training a model, achieving high-fidelity three-dimensional refractive index gradient field reconstruction with low-dimensional parameterization, and being able to adapt to sparse and limited viewing angle conditions, and still achieving high-quality reconstruction when the number of cameras is limited. Specifically, the following sub-steps are included:

[0080] Step S3.1: determining the real light deflection angle disturbed by the flow field according to the displacement field of the perturbed image, specifically, as Figure 3 For a point on the natural background without flow field disturbance C The imaging position on the image plane is point A When the light ray of the point C passes through the flow field, the imaging position of the point C after refraction deflection on the image plane is point ;

[0081] Reversely tracing the point along the light propagation direction, passing through the lens center of the camera lens, and intersecting with the point B on the flow field midplane;

[0082] Connecting the point B with the point C to obtain the propagation direction vector of the light ray before entering the flow field, and calculating the real light deflection angle , wherein represents the propagation direction vector of the light ray after entering the flow field.

[0083] Step S3.2: as Figure 4 , building a NeDF model through a multi-layer perception mechanism, considering the calculation time and reconstruction accuracy, the present application adopts a multi-layer perception mechanism with 3 hidden layers as the NeDF model;

[0084] Step S3.3: position encoding is performed on the three-dimensional coordinates of a preset point in the flow field to be measured to realize dimensionality increase, input the NeDF model to improve the expression ability of the NeDF model, each layer of the hidden layer contains 256 neurons, and output the three-dimensional refractive index gradient of the flow field to be measured;

[0085] Step S3.4: Perform forward integration on the three-dimensional refractive index gradient of the flow field under test to predict the light deflection angle at that point, and then use the actual light deflection angle... Supervised training allows the NeDF model to gradually learn the accurate three-dimensional refractive index gradient. Specifically, the mean squared error loss function is calculated by constructing the true ray deflection angle and the predicted ray deflection angle to measure the predicted deflection angle. Angle of deflection from real light Update the NeDF model parameters based on the differences between them;

[0086] The prediction process for the light ray deflection angle at a point in this invention is as follows:

[0087]

[0088] in, The predicted value representing the angle of light refracting at a point. This represents the three-dimensional refractive index gradient of the flow field under test. , Indicates the flow field to be measured at The refractive index gradient along the axial direction, Indicates the flow field to be measured at The refractive index gradient along the axial direction, Indicates the flow field to be measured at The refractive index gradient along the axial direction; ray This represents a ray of light that passes through the flow field to be measured along the light path from the point.

[0089] The mean square error loss function in this invention The calculation process is as follows:

[0090]

[0091] in, This indicates the number of points used in the training. This represents the predicted value of the light refraction angle of the flow field under test. This represents the actual angle of refraction of light rays.

[0092] Step S3.5: Repeat steps S3.3-S3.4 until the mean square error loss function converges to obtain the three-dimensional refractive index gradient of the flow field to be measured.

[0093] Step S4: Solve the three-dimensional Poisson equation using the three-dimensional refractive index gradient field to obtain the three-dimensional refractive index distribution. Specifically, based on the relationship between the three-dimensional refractive index gradient and the refractive index distribution, a three-dimensional Poisson equation is established. By numerically solving this equation, the complete three-dimensional refractive index distribution is obtained. The three-dimensional Poisson equation is expressed as:

[0094]

[0095] wherein, denotes a Laplacian operator, denotes a gradient operator, is a three-dimensional refractive index distribution to be solved, is a known refractive index gradient field.

[0096] Step S5: solving the Gladstone-Dale equation by using the three-dimensional refractive index distribution to obtain a three-dimensional density field of the flow field; according to the standard atmospheric parameter table with a temperature of 25℃ and an air pressure of 101.325kPa, the Gladstone-Dale constant is selected to solve the Gladstone-Dale equation wherein ， is a three-dimensional density field of the flow field to be solved.

[0097] The large field-of-view three-dimensional density field reconstruction method based on the air-based multi-platform array utilizes a natural background to realize high-fidelity three-dimensional density field reconstruction of a super-large range flow field, and breaks through the limitations of the traditional BOS technology in the field and scale, and can perform three-dimensional, instantaneous or non-contact measurement on an open space flow field in a range of several meters to several hundred meters.

[0098] In one technical solution of the present application, a large field-of-view three-dimensional density field measurement system based on an air-based multi-platform array is also provided, comprising: an air-based multi-platform array, a displacement field calculation module, a three-dimensional refractive index gradient prediction module and a three-dimensional density field calculation module.

[0099] The air-based multi-platform array is used for synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance, and taking the natural background images without flow field disturbance as reference images.

[0100] The displacement field calculation module estimates the displacement of each pixel point in the corresponding disturbance image based on the reference images to obtain a displacement field of each disturbance image.

[0101] The three-dimensional refractive index gradient prediction module reconstructs a three-dimensional refractive index gradient from the displacement field based on the NeDF model.

[0102] The three-dimensional density field calculation module determines a three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and then obtains a three-dimensional density field of the flow field in combination with the Gladstone-Dale equation.

[0103] In one technical solution of the present application, a computer readable storage medium is also provided, which stores a computer program, and the computer program makes a computer execute the large-scale three-dimensional density field reconstruction method based on the air-based multi-platform array.

[0104] In one technical solution of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the method for reconstructing a large-scale three-dimensional density field based on a space-based multi-platform array is implemented.

[0105] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium which can contain or store programs for use by or in connection with an instruction execution system, apparatus or device. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer storage medium can include one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0106] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0107] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application shall be considered within the protection scope of the present application.

Claims

1. A large field of view three-dimensional density field reconstruction method based on space-based multi-platform array, characterized in that, The method comprises the following steps: Step S1: synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance by using the air-based multi-platform array, and taking the natural background images without flow field disturbance as reference images; The air-based multi-platform array is an airborne platform carrying several cameras; Step S2: taking the reference images under the field of view of each camera as the reference, estimating the displacement of each pixel point in the corresponding disturbance image, and obtaining the displacement field of each disturbance image; Step S3: constructing a NeDF model through a multi-layer perception mechanism, reconstructing a three-dimensional refractive index gradient from the displacement field based on the NeDF model, and obtaining a three-dimensional refractive index gradient field; the step S3 comprises the following sub-steps: Step S3.1: determining the real light deflection angle of the flow field disturbance according to the displacement field of the disturbance image; Step S3.2: constructing a NeDF model through a multi-layer perception mechanism; Step S3.3: inputting the three-dimensional coordinates of the point in the flow field to be measured into the NeDF model after position coding, and outputting the three-dimensional refractive index gradient of the flow field to be measured; Step S3.4: performing forward integration on the three-dimensional refractive index gradient of the flow field to be measured, predicting the light deflection angle of the flow field to be measured, calculating a mean square error loss function through the real light deflection angle and the predicted light deflection angle, and updating the parameters of the NeDF model; Step S3.5: repeating the step S3.3 to the step S3.4 until the mean square error loss function converges, and obtaining the three-dimensional refractive index gradient of the flow field to be measured; Step S4: solving a three-dimensional Poisson equation by using the three-dimensional refractive index gradient field, and obtaining a three-dimensional refractive index distribution; Step S5: solving a Gladstone-Dale equation by using the three-dimensional refractive index distribution, and obtaining a three-dimensional density field of the flow field.

2. The method according to claim 1, wherein, The fields of view of all the cameras converge on the flow field to be measured, and the lines of sight of all the cameras pass through the flow field to be measured and fall on the natural background.

3. The method according to claim 1, wherein, The step S2 comprises the following sub-steps: Step S2.1: performing distortion correction on the corresponding reference image and disturbance image by using the camera intrinsic parameters, and performing gray scale normalization on the distortion-corrected reference image and disturbance image respectively, to obtain a standardized reference image and a standardized disturbance image; Step S2.2: constructing an optical flow constraint equation according to the standardized reference image and the corresponding standardized disturbance image, and obtaining a two-dimensional displacement field of the disturbance image relative to the reference image by using a multi-resolution image pyramid iterative optimization strategy.

4. The method according to claim 3, wherein, The construction process of the multi-resolution graph pyramid is as follows: setting the number of layers of the graph pyramid as n, the resolution of the graph pyramid gradually decreases from the bottom layer to the top layer, wherein the resolution of the bottom layer of the graph pyramid is the resolution of the standardized reference image and the standardized disturbance image The resolution of each remaining layer of the graph pyramid is represented as S represents a scale factor, .

5. The method according to claim 4, wherein, The step S2.2 comprises the following sub-steps: Step S2.2.1: performing downsampling on the standardized reference image and the corresponding standardized disturbance image respectively to obtain the top layer of the image pyramid, constructing an overdetermined equation set of the optical flow constraint equation by using the sampled pixels of the standardized reference image and the sampled pixels of the corresponding standardized disturbance image obtained by downsampling, and solving the overdetermined equation set by using the least square method to obtain the displacement increment of the standardized disturbance image relative to the standardized reference image; Step S2.2.2: performing upsampling on the image entering the next layer of the image pyramid, and correcting the displacement of the standardized disturbance image relative to the standardized reference image according to the displacement increment obtained in the previous layer; Step S2.2.3: For each pixel on the normalized reference image, find the corresponding position on the displacement-corrected normalized perturbation image, perform pixel interpolation to obtain a registration image corresponding to each pixel on the normalized reference image, and calculate the residual error between the normalized reference image and the registration image; Step S2.2.4: Construct an overdetermined equation set of optical flow constraint equations for the sampling pixels of the up-sampled reference image and the sampling pixels of the corresponding perturbation image, and solve the equation set using the least square method to obtain the displacement increment of the normalized perturbation image relative to the normalized reference image; Step S2.2.5: Repeat steps S2.2.2-S2.2.4 until the residual error converges, and obtain the optimal displacement increment at the bottom of the image pyramid, and enter the next layer of the image pyramid; Step S2.2.6: Repeat steps S2.2.2-S2.2.5 until the bottom layer of the image pyramid is reached, and obtain the two-dimensional displacement field of the perturbation image relative to the reference image.

6. The method according to claim 1, wherein, The specific process of step S3.1 is as follows: For a point on the natural background without flow field disturbance C The imaging position on the image plane is a point A When the point C After the light ray of the point passes through the flow field, the imaging position of the point on the image plane is determined according to the displacement field of the corresponding disturbance image C After refraction deflection, the imaging position on the image plane is a point ; The point Reverse tracing along the light propagation direction, through the lens center of the camera lens, and intersecting the center plane of the flow field to be measured at point B ; The point B is connected with the point C , to obtain the propagation direction vector of the light ray before entering the flow field , and the real light ray deflection angle is calculated , wherein represents the propagation direction vector of the light ray after entering the flow field.

7. The method according to claim 1, wherein, The prediction process of the ray deflection angle of the point is as follows: wherein, represents a predicted value of a light ray deflection angle of a point, represents a three-dimensional refractive index gradient of the flow field to be measured, , represents a refractive index gradient of the flow field to be measured in the x axis direction, represents a refractive index gradient of the flow field to be measured in the y axis direction, represents a refractive index gradient of the flow field to be measured in the z axis direction; ray represents a light ray of a point passing through the flow field to be measured along an optical path.

8. The method according to claim 1, wherein, The camera on the air-based multi-platform array is jointly optimized for external parameters: in, This represents the extrinsic parameter joint optimization function. Indicates camera on a space-based multi-platform array The first overlapping field of view image captured Each natural texture feature point Indicates camera on a space-based multi-platform array The rotation matrix, Indicates camera on a space-based multi-platform array The translation vector, Indicates the first The three-dimensional coordinates of each natural texture feature point Indicates camera on a space-based multi-platform array The intrinsic parameter matrix, This represents the perspective projection function.

9. A three-dimensional density field measurement system using the large field of view three-dimensional density field reconstruction method based on the array of space-based multiple platforms according to any one of claims 1-8, characterized in that, It includes: An air-based multi-platform array, a displacement field calculation module, a three-dimensional refractive index gradient prediction module, and a three-dimensional density field calculation module; The air-based multi-platform array is used to synchronously collect natural background images without flow field disturbance and perturbation images with flow field disturbance, and the natural background images without flow field disturbance are used as reference images; The displacement field calculation module estimates the displacement of each pixel point in the corresponding perturbation image based on the reference image to obtain the displacement field of each perturbation image; The three-dimensional refractive index gradient prediction module reconstructs the three-dimensional refractive index gradient from the displacement field based on the NeDF model; The three-dimensional density field calculation module determines the three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and then combines the Gladstone-Dale equation to obtain the three-dimensional density field of the flow field.

Images