Space-based multi-platform array-based large-view-field three-dimensional density field reconstruction method and measurement system

Abstract

The invention discloses a large-view-field three-dimensional density field reconstruction method and measurement system based on an air-based multi-platform array. The method comprises the following steps: the air-based multi-platform array is an airborne platform carrying at least five cameras; the air-based multi-platform array synchronously collects a natural background image without flow field disturbance and a disturbance image with flow field disturbance, and the natural background image without flow field disturbance serves as a reference image; the space-based multi-platform estimates the displacement of each pixel point in the corresponding disturbance image through an optical flow algorithm by taking the reference image under each field angle of the camera as a benchmark to obtain a displacement field of each disturbance image; a NeDF model is established through the multi-layer sensor, and a three-dimensional refractive index gradient is reconstructed from the displacement field based on the NeDF model; solving a three-dimensional Poisson equation by using the three-dimensional refractive index gradient field to obtain three-dimensional refractive index distribution; and solving a Gladstone-Dale equation by using the three-dimensional refractive index field to obtain a three-dimensional density field of the flow field. According to the invention, super-large-range flow field high-fidelity three-dimensional density field reconstruction is realized by using a natural background.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of flow field measurement and computer vision, and in particular, relates to a large field of view three-dimensional density field reconstruction method and measurement system based on a space-based multi-platform array. BACKGROUND

[0002] Background Oriented Schlieren (BOS) is a mature non-contact optical measurement method, which quantitatively inverts the density field information by observing the apparent displacement of the background pattern caused by light passing through the non-uniform density field. Tomographic BOS (TBOS) combines BOS and tomographic imaging technology, which can realize three-dimensional reconstruction of the flow field through multi-camera synchronous observation from different angles.

[0003] However, the existing TBOS technology is mainly applied in the controllable environment in the laboratory. When it is tried to be extended to the super-large scale, open outdoor environment, such as aircraft wake, industrial emissions, atmospheric phenomena, etc., there are still the following technical bottlenecks: under laboratory conditions, the synchronization of multiple cameras usually depends on signal line hardware parallel connection, but this way is complex in wiring and poor in flexibility, which is difficult to adapt to large-scale, distributed outdoor measurement scenarios; accordingly, in large-scale outdoor observation, it is often necessary to rely on unmanned aerial vehicles, balloons and other airborne platforms to carry cameras to form an observation array, how to ensure high-precision, low-delay synchronization triggering in a wireless environment is still one of the key difficulties; on the other hand, the traditional background schlieren method relies on a random dot background plate made by hand, which can realize a unified spatial reference through a calibration plate in the laboratory, but when the observation range is expanded to a super-large scale, the size requirement of the background plate and the calibration plate is extremely large, and the manufacturing and arrangement are difficult, which is almost impossible to realize in actual outdoor environment. Therefore, how to establish a unified space-time reference without a calibration plate and ensure the geometric consistency of multi-camera data is a core problem that must be solved to realize large-scale tomographic reconstruction.

[0004] Therefore, there is an urgent need for a technical solution that can systematically solve the above displacement extraction and space-time synchronization challenges to realize accurate and reliable quantitative measurement of three-dimensional density field in large-scale, real outdoor environment. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a large field of view three-dimensional density field reconstruction method and measurement system based on a space-based multi-platform array, which realizes high-fidelity three-dimensional density field reconstruction of a super-large range flow field using a natural background, breaks through the field of view limitation of traditional tomographic technology, and can perform high-precision, three-dimensional density field quantitative measurement on unsteady flow field.

[0006] To achieve the above purpose, the present application adopts the following technical solutions: A large field of view three-dimensional density field reconstruction method based on air-based multi-platform array, comprising the following steps: Step S1: synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance by using an air-based multi-platform array, taking the natural background images without flow field disturbance as reference images; the air-based multi-platform array is an airborne platform carrying several cameras; Step S2: taking the reference images under the field of view of each camera as a reference, estimating the displacement of each pixel point in the corresponding disturbance image to obtain the displacement field of each disturbance image; Step S3: constructing a NeDF model through a multi-layer perception mechanism, and reconstructing a three-dimensional refractive index gradient from the displacement field based on the NeDF model to obtain a three-dimensional refractive index gradient field; Step S4: solving a three-dimensional Poisson equation by using the three-dimensional refractive index gradient field to obtain a three-dimensional refractive index distribution; Step S5: solving a Gladstone-Dale equation by using the three-dimensional refractive index distribution to obtain a three-dimensional density field of the flow field.

[0007] Further, the fields of view of all the cameras converge on the flow field region to be measured, and the lines of sight of all the cameras pass through the flow field region to be measured and fall on the natural background.

[0008] Further, step S2 comprises the following sub-steps: Step S2.1: correcting the distortion of the corresponding reference image and disturbance image by using the camera intrinsic parameters, and performing gray scale normalization processing on the distortion-corrected reference image and disturbance image to obtain a standardized reference image and a standardized disturbance image; Step S2.2: constructing an optical flow constraint equation according to the standardized reference image and the corresponding standardized disturbance image, and obtaining a two-dimensional displacement field of the disturbance image relative to the reference image by using a multi-resolution image pyramid iterative optimization strategy.

[0009] Further, the construction process of the multi-resolution image pyramid is as follows: setting the number of layers of the image pyramid as n, and the resolution of the image pyramid gradually decreases from the bottom layer to the top layer, wherein the resolution of the bottom layer of the image pyramid is the resolution of the standardized reference image and the standardized disturbance image , and the resolution of each remaining layer of the image pyramid is represented as , S represents a scale factor, .

[0010] Further, step S2.2 comprises the following sub-steps: Step S2.2.1: down-sampling the normalized reference image and the corresponding normalized perturbed image respectively to obtain the top layer of the image pyramid, constructing an over-determined equation set of optical flow constraint equation for the sampling pixels of the down-sampled normalized reference image and the corresponding normalized perturbed image, solving by using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image; Step S2.2.2: up-sampling the image entering the next layer of the image pyramid, correcting the displacement of the normalized perturbed image relative to the normalized reference image according to the displacement increment obtained in the last layer; Step S2.2.3: for each pixel on the normalized reference image, searching for the corresponding position on the displacement-corrected normalized perturbed image, performing pixel interpolation processing to obtain a registration image corresponding to each pixel on the normalized reference image, and calculating the residual error between the normalized reference image and the registration image; Step S2.2.4: constructing an over-determined equation set of optical flow constraint equation for the sampling pixels of the up-sampled reference image and the corresponding perturbed image, solving by using the least square method to obtain the displacement increment of the normalized perturbed image relative to the normalized reference image; Step S2.2.5: repeating steps S2.2.2-S2.2.4 until the residual error converges, obtaining the optimal displacement increment in the bottom layer of the image pyramid, and entering the next layer of the image pyramid; Step S2.2.6: repeating steps S2.2.2-S2.2.5 until reaching the bottom layer of the image pyramid, obtaining the two-dimensional displacement field of the perturbed image relative to the reference image.

[0011] Further, step S3 includes the following sub-steps: Step S3.1: determining the real light deflection angle disturbed by the flow field according to the displacement field of the perturbed image; Step S3.2: constructing a NeDF model through a multi-layer perception mechanism; Step S3.3: inputting the three-dimensional coordinates of the point to be measured in the flow field into the NeDF model after position encoding, and outputting the three-dimensional refractive index gradient of the flow field to be measured; Step S3.4: performing forward integration on the three-dimensional refractive index gradient of the flow field to be measured to predict the light deflection angle of the flow field to be measured, calculating a mean square error loss function through the real light deflection angle and the predicted light deflection angle, and updating the parameters of the NeDF model; Step S3.5: repeating steps S3.3-S3.4 until the mean square error loss function converges, obtaining the three-dimensional refractive index gradient of the flow field to be measured.

[0012] Further, the specific process of step S3.1 is as follows: For a point on a natural background without flow field disturbanceC the imaging position of the point A on the image plane is point C ; C the imaging position of the point on the image plane after refraction deflection is point ; ; B ; ; B ; C ; ; ; ;

[0013] Further, the prediction process of the ray deflection angle of the point is:

[0014] wherein, represents the predicted value of the ray deflection angle of the point, represents the three-dimensional refractive index gradient of the flow field to be measured, , represents the refractive index gradient of the flow field to be measured in the x-axis direction, x represents the refractive index gradient of the flow field to be measured in the y-axis direction, represents the refractive index gradient of the flow field to be measured in the z-axis direction, y represents the refractive index gradient of the flow field to be measured in the x-axis direction, represents the refractive index gradient of the flow field to be measured in the y-axis direction, z represents the refractive index gradient of the flow field to be measured in the z-axis direction; ray represents a ray of the point passing through the flow field to be measured along the optical path.

[0015] Further, the extrinsic parameters of the cameras on the air-based multi-platform array are jointly optimized:

[0016] wherein, represents the extrinsic parameter joint optimization function, represents the natural texture feature point in the first overlapping field of view image taken by the camera on the air-based multi-platform array, represents the rotation matrix of the camera on the air-based multi-platform array, represents the translation vector of the camera on the air-based multi-platform array, represents the natural texture feature point in the first overlapping field of view image taken by the camera on the air-based multi-platform array, represents the rotation matrix of the camera on the air-based multi-platform array, represents the translation vector of the camera on the air-based multi-platform array, represents the natural texture feature point in the first overlapping field of view image taken by the camera on the air-based multi-platform array, represents the natural texture feature point in the first overlapping field of view image taken by the camera on the air-based multi-platform array, Three-dimensional coordinates of a natural texture feature point, an intrinsic matrix of a camera on the air-based multi-platform array, a perspective projection function.

[0017] Further, the application also provides a three-dimensional density field measurement system applying the large field-of-view three-dimensional density field reconstruction method based on the air-based multi-platform array, comprising: an air-based multi-platform array, a displacement field calculation module, a three-dimensional refractive index gradient prediction module and a three-dimensional density field calculation module. The air-based multi-platform array is used for synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance, and taking the natural background images without flow field disturbance as reference images. The displacement field calculation module estimates the displacement of each pixel point in the corresponding disturbance image based on the reference image, to obtain the displacement field of each disturbance image. The three-dimensional refractive index gradient prediction module reconstructs the three-dimensional refractive index gradient from the displacement field based on the NeDF model. The three-dimensional density field calculation module determines the three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and then combines the Gladstone-Dale equation to obtain the three-dimensional density field of the flow field. Compared with the prior art, the application has the following beneficial effects: the large field-of-view three-dimensional density field reconstruction method and measurement system based on the air-based multi-platform array adopt a natural scene as a background, can avoid the construction of an artificial background board, significantly reduce the experimental cost and deployment complexity, ensure large-scale observation, have good adaptability and flexibility, and are especially suitable for outdoor environments where a background board cannot be arranged; meanwhile, the NeDF model is constructed through a multi-layer perception mechanism to reconstruct the three-dimensional refractive index gradient, position encoding and hierarchical sampling are fused, flow field details are accurately captured, information loss is avoided, a pre-trained model is not needed, high-fidelity three-dimensional refractive index gradient field reconstruction is realized with low-dimensional parameterization, and the method can adapt to sparse and limited viewing angle conditions and still realize high-quality reconstruction when the number of cameras is limited. The application breaks through the field-of-view limitation of traditional tomography technology, can perform high-precision three-dimensional quantitative measurement on unsteady flow fields, and has important scientific value and engineering application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 a flowchart of the large field-of-view three-dimensional density field reconstruction method based on the air-based multi-platform array of the application; Figure 2 a structural schematic diagram of the air-based multi-platform array in the application; Figure 3 a schematic diagram of the deflection angle of a real light ray disturbed by a flow field in the application; Figure 4 ​A schematic diagram of a training process of the NeDF model in the application. DETAILED DESCRIPTION

[0019] The technical solutions of the application will be further explained in combination with the accompanying drawings.

[0020] As Figure 1 A flowchart of the large field-of-view three-dimensional density field reconstruction method based on the air-based multi-platform array of the application, which comprises the following steps: Step S1: As Figure 2 The air-based multi-platform array is an airborne platform carrying at least five cameras. The cameras can be deployed on a UAV, and the UAV is deployed around the flow field to be measured. The camera attitude is adjusted by the UAV gimbal, and the height of the UAV is adjusted so that the fields of view of all cameras are collectively intersected in the flow field area to be measured, and the lines of sight of all cameras pass through the flow field area to be measured and fall on a natural background with rich texture features, thereby avoiding the calculation failure of optical flow caused by insufficient or repeated background texture. After completing the position and attitude adjustment, the UAV and the camera remain fixed during the entire experiment. The air-based multi-platform array is used to synchronously collect natural background images without flow field disturbance and disturbance images with flow field disturbance, and the natural background images without flow field disturbance are used as reference images.

[0021] In one technical solution of the application, when the flow field to be measured does not generate disturbance, a ground server sends a synchronous trigger instruction to trigger multiple cameras to simultaneously shoot natural background images, continuously collects multiple frames of images, performs average processing on the multiple frames of images collected by each camera at the pixel level to obtain an average image, and uses the average image as a reference image for subsequent processing. Start the flow field disturbance source, and send the synchronous trigger instruction again to continuously collect multiple frames of disturbance images with flow field disturbance.

[0022] In one technical solution of the application, the multiple cameras are calibrated for internal and external parameters. The calibration process is divided into two stages of laboratory pre-calibration and on-site optimization to ensure the accuracy and robustness of the final internal and external parameters. Before the air-based multi-platform array is deployed, the internal parameters of each camera in the air-based multi-platform array are independently pre-calibrated in a ground controlled environment. 10 to 50 frames of effective images of the chessboard calibration board can be collected for each camera. When shooting, the calibration board is slowly moved and its attitude is adjusted in the field of view of the camera to cover different pitch angles, yaw angles and roll angles, so that the feature points of the calibration board are completely and clearly covered in the entire field of view area, including the center, edges and corners of the image. The entire process needs to be carried out under stable and uniform lighting conditions to avoid reflection or shadow, and appropriate shutter speed is set to prevent image blur. For the image set collected for each camera, a mature camera calibration algorithm is used to calculate the initial internal parameters, including focal length, principal point coordinates and distortion coefficients.

[0023] In the integrated on-site calibration process, images acquired by multiple cameras in an overlapping field of view on the ground are first used to extract natural texture feature points. These feature points are then matched, and the rotation matrix and translation vector of each camera are further calculated using the PnP method to achieve relative extrinsic parameter calibration between the multiple cameras, thereby obtaining their relative spatial relationships. After aligning the extrinsic parameters using the above method, Bundle Adjustment is used for joint optimization of multi-camera extrinsic parameters. This fully utilizes the geometric constraints between multiple viewpoints, improving the overall consistency and accuracy of extrinsic parameter estimation, thus reducing the impact of single-camera calibration errors on the overall system.

[0024] The process of multi-camera extrinsic parameter joint optimization in this invention is as follows:

[0025] in, This represents the extrinsic parameter joint optimization function. Indicates camera on a space-based multi-platform array The first overlapping field of view image captured Each natural texture feature point Indicates camera on a space-based multi-platform array The rotation matrix, Indicates camera on a space-based multi-platform array The translation vector, Indicates the first The three-dimensional coordinates of each natural texture feature point Indicates camera on a space-based multi-platform array The intrinsic parameter matrix, This represents the perspective projection function.

[0026] Subsequently, no fewer than 5 to 10 easily identifiable ground control points (GCPs) are deployed on the ground in the camera overlap area, and their three-dimensional coordinates in a preset world coordinate system are obtained using high-precision surveying equipment such as real-time dynamic GNSS or total station. Based on these GCPs, the relative calibration results of the multiple cameras are uniformly transformed to the world coordinate system, realizing the introduction of absolute scale and the determination of coordinate reference, thereby establishing the global extrinsic parameter relationship of the multiple cameras in a unified world coordinate system.

[0027] Step S2: Using the reference images at each field of view of the camera as a reference, estimate the displacement of each pixel in the corresponding perturbed image using an optical flow algorithm to obtain the displacement field of each perturbed image; including the following sub-steps: Step S2.1: the internal parameter matrix and distortion coefficient of the camera are used for distortion correction of the corresponding reference image and perturbation image, so that the influence of radial distortion, tangential distortion and other lens effects on the pixel geometric position is eliminated, and the correspondence between the image coordinates and the real physical space is ensured to be accurate and reliable. And the reference image and the perturbation image after distortion correction are subjected to gray scale normalization processing respectively to obtain the standardized reference image and the standardized perturbation image, which can effectively suppress the damage of light change, imaging noise and exposure difference to the brightness consistency assumption, and improve the robustness of optical flow calculation.

[0028] The process of gray scale normalization processing in the application is:

[0029] Among them, , represents the gray value of the reference image or the perturbation image after distortion correction at the pixel position , and respectively represent the minimum value and the maximum value of the gray value of the reference image or the perturbation image after distortion correction, and through the linear transformation, the gray scale range of the image is mapped to the [0, 1] interval.

[0030] Step S2.2: constructing an optical flow constraint equation according to the standardized reference image and the corresponding standardized perturbation image , wherein, , respectively are the gradients of the pixel points in the spatial direction , , is the gray scale change rate in the time direction, is the displacement component of the point in the standardized perturbation image. Since one optical flow constraint equation cannot solve two unknowns and , additional constraints need to be introduced for solving, and the application adopts an iterative optimization strategy of a multi-resolution graph pyramid to obtain a two-dimensional displacement field of the perturbation image relative to the reference image.

[0031] The construction process of the multi-resolution graph pyramid in the application is: the number of layers of the graph pyramid is set to n, and the resolution of the graph pyramid gradually decreases from the bottom layer to the top layer, so that large displacement becomes small displacement in the graph pyramid layer of coarse resolution, facilitating layer-by-layer estimation. Among them, the bottom layer resolution of the graph pyramid is the resolution of the standardized reference image and the standardized perturbation image , and the resolution of each remaining layer of the graph pyramid is represented as , S represents a scale factor. .

[0032] The acquisition process of the two-dimensional displacement field of the perturbation image relative to the reference image in the application is: Step S2.2.1: Downsample the standardized reference image and the corresponding standardized perturbation image respectively to obtain the top layer of the graph pyramid. Assume that in a... Within the sliding window, the displacement increment of all pixels For the sampled pixels of the downsampled normalized reference image and the corresponding normalized perturbation image, an overdetermined set of optical flow constraint equations is constructed:

[0033] in, Indicating the first normalized reference image One point; The displacement increment of the normalized perturbation image relative to the normalized reference image is obtained by using the least squares method. ; Step S2.2.2: Upsample the image entering the next layer of the graph pyramid, and correct the displacement of the normalized perturbation image relative to the normalized reference image based on the displacement increment obtained from the previous layer. ; Step S2.2.3: For each pixel on the normalized reference image Find the corresponding position on the displacement-corrected normalized perturbation image. ,because Since the coordinates are often not integers, pixel values ​​cannot be directly obtained; a standardized perturbation image is required. I 2. Perform pixel interpolation to obtain grayscale values. This yields a registered image that corresponds one-to-one with the pixels of the normalized reference image. The residual between the normalized reference image and the registered image is then calculated. ,in, Represents the grayscale value of the standardized reference image; Step S2.2.4: For the sampled pixels of the upsampled reference image and the sampled pixels of the corresponding perturbation image, construct an overdetermined set of optical flow constraint equations, solve them using the least squares method, and obtain the displacement increment of the normalized perturbation image relative to the normalized reference image. Step S2.2.5: Repeat steps S2.2.2-S2.2.4 until the residuals converge, obtain the optimal displacement increment under the graph pyramid of this layer, and enter the next layer of the graph pyramid; Step S2.2.6: Repeat steps S2.2.2-S2.2.5 until the bottom layer of the graph pyramid is reached to obtain the two-dimensional displacement field of the perturbation image relative to the reference image.

[0034] Step S3: building a NeDF model through a multi-layer perception mechanism, reconstructing a three-dimensional refractive index gradient from a displacement field based on the NeDF model, fusing position encoding and hierarchical sampling, accurately capturing flow field details, avoiding information loss, without pre-training a model, achieving high-fidelity three-dimensional refractive index gradient field reconstruction with low-dimensional parameterization, and being able to adapt to sparse and limited viewing angle conditions, and still achieving high-quality reconstruction when the number of cameras is limited. Specifically, the following sub-steps are included: Step S3.1: determining the real ray deflection angle of the flow field disturbance according to the displacement field of the perturbed image, specifically as follows: Figure 3 For a point on the natural background without flow field disturbance C The imaging position of the point on the image plane is point A When the light ray of the point C passes through the flow field, the real ray deflection angle of the point C is determined according to the displacement field of the corresponding perturbed image. ; The point is traced back along the propagation direction of the light ray, passes through the lens center of the camera lens, and intersects with the point B on the flow field line plane. The point B is connected with the point C to obtain the propagation direction vector of the light ray before entering the flow field , and the real ray deflection angle is calculated, wherein represents the propagation direction vector of the light ray after entering the flow field.

[0035] Step S3.2: as shown in Figure 4 , a NeDF model is built through a multi-layer perception mechanism, and a three-layer hidden layer multi-layer perception mechanism is used as the NeDF model in the present application in consideration of the calculation time and reconstruction accuracy. Step S3.3: the three-dimensional coordinates of a preset point in the flow field to be measured are position-encoded to realize dimension upgrading, and input into the NeDF model to improve the expression ability of the NeDF model, wherein each layer of the hidden layer contains 256 neurons, and the three-dimensional refractive index gradient of the flow field to be measured is output; Step S3.4: the three-dimensional refractive index gradient of the flow field to be measured is forward-integrated to predict the ray deflection angle of the point, and the real ray deflection angle supervises the training of the NeDF model to gradually learn the accurate three-dimensional refractive index gradient, specifically, a mean square error loss function is calculated through the real ray deflection angle and the predicted ray deflection angle to measure the difference between the predicted deflection angle and the real ray deflection angle , and the parameters of the NeDF model are updated. The prediction process of the ray deflection angle of the point in the application is as follows:

[0036] wherein, represents the predicted value of the ray deflection angle of the point, represents the three-dimensional refractive index gradient of the flow field to be measured, , represents the refractive index gradient of the flow field to be measured in the direction of the x-axis, represents the refractive index gradient of the flow field to be measured in the direction of the y-axis, represents the refractive index gradient of the flow field to be measured in the direction of the z-axis, represents the refractive index gradient of the flow field to be measured in the direction of the x-axis, represents the refractive index gradient of the flow field to be measured in the direction of the y-axis, represents the refractive index gradient of the flow field to be measured in the direction of the z-axis. ray represents a ray of light passing through the flow field to be measured along the optical path.

[0037] The calculation process of the mean square error loss function in the application is as follows:

[0038] wherein, represents the number of points participating in training, represents the predicted value of the ray deflection angle of the flow field to be measured, represents the true ray deflection angle.

[0039] Step S3.5: Repeat steps S3.3-S3.4 until the mean square error loss function converges, and obtain the three-dimensional refractive index gradient of the flow field to be measured.

[0040] Step S4: Solve the three-dimensional Poisson equation using the three-dimensional refractive index gradient field to obtain the three-dimensional refractive index distribution; specifically, based on the relationship between the three-dimensional refractive index gradient and the refractive index distribution, a three-dimensional Poisson equation is established, and the complete three-dimensional refractive index distribution is obtained by numerically solving the equation. Wherein, the three-dimensional Poisson equation is expressed as:

[0041] wherein, represents the Laplacian operator, represents the gradient operator, is the three-dimensional refractive index distribution to be solved, is the known refractive index gradient field.

[0042] Step S5: Solve the Gladstone-Dale equation using the three-dimensional refractive index distribution to obtain the three-dimensional density field of the flow field; according to the standard atmospheric parameter table with a temperature of 25℃ and an air pressure of 101.325kPa, the Gladstone-Dale constant solving the Gladstone-Dale equation wherein ， is a three-dimensional density field of the flow field to be solved.

[0043] The present application is based on a large field-of-view three-dimensional density field reconstruction method of an air-based multi-platform array, which uses a natural background to realize high-fidelity three-dimensional density field reconstruction of a super-large range flow field, breaks through the limitations of traditional BOS technology in terms of site and scale, and can perform three-dimensional, instantaneous or non-contact measurement on an open space flow field in a range of several meters to several hundred meters.

[0044] In one technical solution of the present application, a large field-of-view three-dimensional density field measurement system based on an air-based multi-platform array is also provided, comprising: an air-based multi-platform array, a displacement field calculation module, a three-dimensional refractive index gradient prediction module, and a three-dimensional density field calculation module. The air-based multi-platform array is used to synchronously collect natural background images without flow field disturbance and disturbance images with flow field disturbance, and the natural background images without flow field disturbance are used as reference images. The displacement field calculation module estimates the displacement of each pixel point in the corresponding disturbance image based on the reference image, and obtains the displacement field of each disturbance image. The three-dimensional refractive index gradient prediction module reconstructs the three-dimensional refractive index gradient from the displacement field based on the NeDF model. The three-dimensional density field calculation module determines the three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and then obtains the three-dimensional density field of the flow field in combination with the Gladstone-Dale equation.

[0045] In one technical solution of the present application, a computer readable storage medium is also provided, which stores a computer program, and the computer program causes a computer to execute the large-scale three-dimensional density field reconstruction method based on an air-based multi-platform array.

[0046] In one technical solution of the present application, an electronic device is also provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the large-scale three-dimensional density field reconstruction method based on an air-based multi-platform array is realized.

[0047] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium which can contain or store programs for use by or in connection with an instruction execution system, apparatus or device. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of computer storage medium can include one or more wires, portable computer disks, hard drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0048] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0049] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application shall be considered within the protection scope of the present application.

Claims

1. A large field of view three-dimensional density field reconstruction method based on space-based multi-platform array, characterized in that, The method comprises the following steps: Step S1: synchronously collecting natural background images without flow field disturbance and disturbance images with flow field disturbance by using the air-based multi-platform array, and taking the natural background images without flow field disturbance as reference images; The air-based multi-platform array is an airborne platform carrying several cameras; Step S2: taking the reference images under the field of view of each camera as the reference, estimating the displacement of each pixel point in the corresponding disturbance image, and obtaining the displacement field of each disturbance image; Step S3: constructing a NeDF model through a multi-layer perception mechanism, reconstructing a three-dimensional refractive index gradient from the displacement field based on the NeDF model, and obtaining a three-dimensional refractive index gradient field; Step S4: solving a three-dimensional Poisson equation by using the three-dimensional refractive index gradient field, and obtaining a three-dimensional refractive index distribution; Step S5: solving a Gladstone-Dale equation by using the three-dimensional refractive index distribution, and obtaining a three-dimensional density field of the flow field.

2. The method according to claim 1, wherein, The fields of view of all the cameras converge on the flow field region to be measured, and the lines of sight of all the cameras pass through the flow field region to be measured and fall on the natural background.

3. The method according to claim 1, wherein, Step S2 comprises the following sub-steps: Step S2.1: performing distortion correction on the corresponding reference images and disturbance images by using camera intrinsic parameters, and performing gray scale normalization processing on the distortion-corrected reference images and disturbance images respectively to obtain standardized reference images and standardized disturbance images; Step S2.2: constructing an optical flow constraint equation according to the standardized reference images and the corresponding standardized disturbance images, and obtaining a two-dimensional displacement field of the disturbance image relative to the reference image by using a multi-resolution image pyramid iterative optimization strategy.

4. The method according to claim 3, wherein, The construction process of the multi-resolution graph pyramid is as follows: setting the number of layers of the graph pyramid as n, the resolution of the graph pyramid gradually decreases from the bottom layer to the top layer, wherein the resolution of the bottom layer of the graph pyramid is the resolution of the standardized reference image and the standardized disturbance image The resolution of each remaining layer of the graph pyramid is represented as S represents a scale factor, .

5. The method according to claim 4, wherein, Step S2.2 comprises the following sub-steps: Step S2.2.1: performing downsampling on the standardized reference images and the corresponding standardized disturbance images respectively to obtain the top layer of the image pyramid, constructing an over-determined equation set of the optical flow constraint equation for the sampled pixels of the standardized reference images and the sampled pixels of the corresponding standardized disturbance images obtained by downsampling, and solving by using the least square method to obtain the displacement increment of the standardized disturbance image relative to the standardized reference image; Step S2.2.2: performing upsampling on the images entering the next layer of the image pyramid, and correcting the displacement of the standardized disturbance image relative to the standardized reference image according to the displacement increment obtained in the previous layer; Step S2.2.3: for each pixel on the standardized reference image, searching for the corresponding position on the displacement-corrected standardized disturbance image, performing pixel interpolation processing to obtain a registration image corresponding to each pixel on the standardized reference image, and calculating the residual error between the standardized reference image and the registration image; Step S2.2.4: constructing an over-determined equation set of the optical flow constraint equation for the sampled pixels of the upsampled reference images and the sampled pixels of the corresponding disturbance images, and solving by using the least square method to obtain the displacement increment of the standardized disturbance image relative to the standardized reference image; Step S2.2.5: repeating steps S2.2.2-S2.2.4 until the residual error converges, obtaining the optimal displacement increment of the current layer of the image pyramid, and entering the next layer of the image pyramid. Step S2.2.6: repeat step S2.2.2-step S2.2.5 until the bottom layer of the pyramid is reached, obtaining the two-dimensional displacement field of the perturbed image relative to the reference image.

6. The method according to claim 1, wherein, Step S3 includes the following sub-steps: Step S3.1: determining the real light deflection angle of the flow field perturbed according to the displacement field of the perturbed image; Step S3.2: constructing a NeDF model through a multi-layer perception mechanism; Step S3.3: inputting the three-dimensional coordinates of the points in the flow field to be measured into the NeDF model after position encoding, and outputting the three-dimensional refractive index gradient of the flow field to be measured; Step S3.4: forward integrating the three-dimensional refractive index gradient of the flow field to be measured, predicting the light deflection angle of the flow field to be measured, calculating the mean square error loss function through the real light deflection angle and the predicted light deflection angle, and updating the NeDF model parameters; Step S3.5: repeat step S3.3-step S3.4 until the mean square error loss function converges, and obtain the three-dimensional refractive index gradient of the flow field to be measured.

7. The method according to claim 6, wherein, The specific process of step S3.1 is: For a point on the natural background without flow field disturbance C The imaging position on the image plane is a point A When the light ray of the point C After the light ray of the point C The imaging position on the image plane is a point ; The point Reverse tracing along the light propagation direction, through the lens center of the camera lens, and intersecting with the center plane of the flow field to be measured at point B ; The point B is connected with the point C , to obtain the propagation direction vector of the light ray before entering the flow field , and the real light ray deflection angle is calculated , wherein represents the propagation direction vector of the light ray after entering the flow field.

8. The method according to claim 6, wherein, The prediction process of the light deflection angle of the point is: in, The predicted value representing the angle of light refracting at a point. This represents the three-dimensional refractive index gradient of the flow field under test. , Indicates the flow field to be measured at x The refractive index gradient along the axial direction, Indicates the flow field to be measured at y The refractive index gradient along the axial direction, Indicates the flow field to be measured at z The refractive index gradient along the axial direction; ray This represents a ray of light that passes through the flow field under test along the light path from the point.

9. The method of claim 1, wherein, Jointly optimizing the external parameters of the cameras on the air-based multi-platform array: wherein, denotes an extrinsic joint optimization function, denotes a rotation matrix of a camera on the aerial multi-platform array, denotes a natural texture feature point in the overlapped field of view image denotes a rotation matrix of a camera on the aerial multi-platform array, denotes a translation vector of a camera on the aerial multi-platform array, denotes a three-dimensional coordinate of a natural texture feature point denotes a natural texture feature point in the overlapped field of view image denotes an intrinsic matrix of a camera on the aerial multi-platform array, denotes a perspective projection function.

10. A three-dimensional density field measurement system using the large field of view three-dimensional density field reconstruction method based on the array of space-based multiple platforms according to any one of claims 1-9, characterized in that, Including: Air-based multi-platform array, displacement field calculation module, three-dimensional refractive index gradient prediction module, and three-dimensional density field calculation module; The air-based multi-platform array is used to synchronously collect natural background images without flow field perturbation and perturbed images with flow field perturbation, and the natural background image without flow field perturbation is used as a reference image; The displacement field calculation module estimates the displacement of each pixel point in the corresponding perturbed image based on the reference image, obtaining the displacement field of each perturbed image; The three-dimensional refractive index gradient prediction module reconstructs the three-dimensional refractive index gradient from the displacement field based on the NeDF model; The three-dimensional density field calculation module determines the three-dimensional refractive index distribution according to the three-dimensional refractive index gradient, and then obtains the three-dimensional density field of the flow field in combination with the Gladstone-Dale equation.

Images