A lightweight method for reducing the amount of calculation in high-speed aircraft background schlieren imaging

Abstract

The application discloses a lightweight method for reducing calculation amount in high-speed aircraft background schlieren imaging, and belongs to the field of high-speed aircraft air disturbance flow field visualization. The method comprises the following steps: inputting a background image without an aircraft and a background schlieren image with the aircraft; compressing a two-dimensional image to a one-dimensional signal through background difference projection to determine a target frame; predicting the flight direction aircraft vertex coordinates and the heading angle in the normalized target frame by using a trained lightweight neural network; linearly mapping the predicted aircraft vertex coordinates and the heading angle to the original image space to obtain the original aircraft vertex coordinates and the heading angle; and determining a to-be-calculated region through linear segmentation based on the original aircraft vertex coordinates and the heading angle. The method dynamically locks the physical disturbance sensitive domain through the core ideas of lightweight flight target positioning, vertex-heading geometric model and frame linear segmentation, greatly reduces the calculation load and the algorithm power demand on the premise of guaranteeing the integrity of the shock wave pattern.

Description

Technical Field

[0001] This invention belongs to the field of visualization of air disturbance flow fields of high-speed aircraft, and more specifically, relates to a lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft. Background Technology

[0002] Schlieren imaging technology for high-speed aircraft visualizes disturbed flow fields by capturing changes in air refractive index caused by shock waves, and is a core method for analyzing the aerodynamic characteristics of supersonic flight. With the increasing demand for space-based detection and high-resolution imaging, the daily data volume of schlieren systems can reach terabytes (TB), requiring intensive optical flow calculations and other measures to extract particle displacements from full-frame image sequences to obtain background schlieren images. Under high data volumes, traditional full-frame processing methods consume significant computational resources. Some experiments capturing supersonic flow field shock waves against a solar flare background demonstrate that processing data from a single experiment requires long-term computation using large-scale computing clusters, severely limiting real-time analysis and engineering applications.

[0003] Existing computational optimization schemes primarily alleviate computational pressure by reducing image resolution or frame rate, but this leads to the loss of weak schlieren features. Since background schlieren technology relies on precise capture of particle displacement, it is difficult to obtain particle displacement at the algorithmic level when the image resolution is low, thus failing to obtain schlieren images. Another common method is to utilize hardware such as GPUs to achieve parallel acceleration of computation. While this approach can significantly improve processing speed, its high power consumption fundamentally contradicts the strict power constraints of space-based platforms such as spaceborne systems, and its computational load remains essentially unchanged. To reduce invalid computation, recent research has attempted to introduce moving region segmentation techniques. These methods rely on static background modeling or fixed threshold segmentation, resulting in a high missegmentation rate in complex scenarios such as dynamic cloud interference and moving objects on the ground. Existing wind tunnel tests have shown that traditional segmentation algorithms have high response delays to the wake edges of high-speed aircraft, failing to meet the millisecond-level dynamic tracking requirements of high-speed aircraft, and instead increasing additional computational overhead due to the segmentation algorithm itself. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a lightweight method for reducing computational load in high-speed aircraft background schlieren imaging, thereby solving the problem of excessive resource consumption for full-area particle displacement calculation in high-speed aircraft background schlieren imaging.

[0005] To achieve the above objectives, this invention provides a lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft, comprising the following steps:

[0006] Input a background image without the aircraft and a background schlieren image containing the aircraft;

[0007] The background image without the aircraft and the background schlieren image with the aircraft are compressed into a one-dimensional signal by background difference projection. The target box is determined based on the one-dimensional signal and the target box is normalized.

[0008] Within the normalized bounding box obtained in the above steps, a coordinate system is defined, called the normalized coordinate system. The origin of this coordinate system is located at the upper left corner of the normalized bounding box, the coordinate axes are aligned with the original image coordinate axes, and the coordinate range is equal to the size of the bounding box.

[0009] Within the normalized target box, a trained lightweight neural network is used to predict the aircraft vertex coordinates and orientation angle in the flight direction;

[0010] The predicted vertex coordinates and orientation angle of the aircraft are linearly mapped to the original image space to obtain the original vertex coordinates and orientation angle of the aircraft.

[0011] Based on the original vertex coordinates and orientation angle of the aircraft, a boundary line representing the maximum front of the shock wave is constructed. By calculating the position symbol Sc of the center point of the target box relative to this boundary line, all pixels in the background schlieren map containing the aircraft that have the same symbol as Sc are selected, thus forming the region to be calculated.

[0012] The present invention also provides an electronic device, comprising: a computer-readable storage medium and a processor;

[0013] The computer-readable storage medium is used to store executable instructions;

[0014] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the above-described method.

[0015] The present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to perform the above-described method.

[0016] The present invention also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the above-described method.

[0017] Compared with the prior art, the above-described technical solutions conceived in this invention can achieve the following beneficial effects:

[0018] (1) This invention proposes a lightweight adaptive computational region selection method architecture based on target localization. This method dynamically locks the sensitive domain of physical disturbance through the core ideas of lightweight flight target localization, vertex-orientation geometric model and linear segmentation of the frame. Under the premise of ensuring the integrity of shock wave morphology, it greatly reduces the computational load and significantly reduces the computing power requirement.

[0019] (2) To address the problem of high computational redundancy in full-frame two-dimensional traversal during target localization, and to avoid the resource consumption complexity of the traditional sliding window method, this invention employs a background difference projection method for rapid target box localization. Sensitive region localization is achieved by utilizing the pixel summation and difference between the image and the static background in the row and column directions, compressing the two-dimensional search problem into one-dimensional signal analysis. Based on the density gradient perturbation characteristics caused by aircraft motion, this method extracts projection difference extrema points through dynamic threshold segmentation, quickly outputting the aircraft boundary coordinates without requiring sliding window traversal or the extensive data collection, labeling, and model training involved in deep learning methods.

[0020] (3) This invention proposes a fast vertex-orientation evaluation method for high-speed flying targets based on fixed-size image input. The method aims to detect the front vertex and instantaneous flight orientation of the aircraft. It employs a lightweight neural network within a normalized target bounding box to efficiently predict the aircraft's vertex and orientation. The network input is an image within a standardized target bounding box that has been proportionally scaled down to a smaller number of pixels (e.g., 64) and padded with zeros on the shorter sides. A neural network structure capable of regressing arbitrary aircraft vertex coordinates and orientation angles is constructed, and the neural network is trained using a simulated dataset of aircraft vertex and orientation data. Due to the small input image size (e.g., 64×64), this method exhibits minimal latency compared to traditional corner detection and orientation detection methods. Finally, the vertex coordinates and orientation of the aircraft in the entire image are obtained by combining the target's location coordinates with the overall image.

[0021] (4) Determination of the region to be calculated based on simplified linear segmentation. This invention constructs a straight line equation passing through the vertex, consistent with the perpendicular direction of the target orientation, based on the target's vertex and orientation. Then, it performs binary classification of the region to be calculated by determining whether the pixel coordinates in the image belong to the left or right side of the line. Compared to traditional image segmentation methods, this step involves minimal computation, significantly reducing computational delay. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft, provided by the present invention.

[0023] Figure 2 This is Example 1 of the mask for the aircraft vertex, orientation, and region to be calculated in Embodiment 1 of the present invention.

[0024] Figure 3 This is Example 2 of the mask for the aircraft vertex, orientation, and region to be calculated in Embodiment 1 of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0026] This embodiment describes a lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft, as follows: Figure 1 As shown:

[0027] Step 1: Based on the input background image Ib without the aircraft and the background schlieren image I with the aircraft, the background image without the aircraft and the background schlieren image with the aircraft are compressed into a one-dimensional signal by background difference projection, and the target box is determined according to the one-dimensional signal.

[0028] Step 2: Normalize the target box to L×L, where L is the side length of the target box (number of pixels). Preferably, L=64.

[0029] Step 3: Use the trained lightweight neural network to determine the vertices (xp0, yp0) and orientation angle (denoted as angle, counterclockwise from the positive X-axis) of the aircraft within the normalized target box. Preferably, radians are used as the unit.

[0030] Step 4: Linearly map the predicted aircraft vertex coordinates and orientation angle to the original image space to obtain the original aircraft vertex coordinates (xp, yp) and orientation angle;

[0031] Step 5: Based on the original vertex coordinates (xp, yp) and orientation angle of the aircraft, construct a boundary line representing the maximum front of the shock wave. By calculating the position symbol Sc of the center point of the target box relative to this boundary line, filter out all pixels in the background schlieren map containing the aircraft that have the same symbol Sc, thus forming the region to be calculated.

[0032] Specifically, the process of determining the target bounding box using dimensionality reduction in step 1 is as follows:

[0033] Step 1.1: Calculate DifX = abs(sumX(I) - sumX(Ib)) and DifY = abs(sumY(I) - sumY(Ib)), where abs() is the absolute value function, sumX(I) and sumY(I) are the results of summing pixel values of the background schlieren image I containing the aircraft in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) respectively; sumX(Ib) and sumY(Ib) are the results of summing pixel values of the background image Ib without the aircraft in the horizontal direction (X-axis direction) and the vertical direction (Y-axis direction) respectively; obvious disturbances will occur in the density gradient where the aircraft appears, and peak values will appear in the corresponding rows and columns of the projection difference signals DifX and DifY. Just find the peak / valley values in a one-dimensional array to determine the approximate left and right boundaries (xa, xb) and upper and lower boundaries (ya, yb) of the aircraft, turning the two-dimensional traversal problem into a one-dimensional threshold search to reduce the computational amount and avoid the high computational amount generated by the traditional full-frame sliding window method.

[0034] Step 1.2: Find the first coordinate (xa) and the last coordinate (xb) that satisfy the condition DifX > Th1 × max(DifX), where Th1 is a threshold between 0 and 1, preferably Th1 = 0.5, that is, the first and last coordinates that satisfy the condition that the DifX value is greater than half of its maximum value;

[0035] Step 1.3: Find the first coordinate (ya) and the last coordinate (yb) that satisfy the condition DifY > Th1 × max(DifY). Similar to Step 1.2, preferably, Th1 = 0.5, that is, the first and last coordinates that satisfy the condition that the DifY value is greater than half of its maximum value;

[0036] Step 1.4: Find the first coordinate that satisfies the conditions DifX < Th2 × max(DifX) and DifY < Th2 × max(DifY) going forward from xa and ya, where Th2 is a threshold between 0 and 1, preferably Th2 = 0.1, that is, the coordinate that satisfies the conditions that the DifX value is less than one-tenth of its maximum value and the DifY value is less than one-tenth of its maximum value, denoted as x1, y1;

[0037] Step 1.5: Find the first coordinate that satisfies the conditions DifX < Th2 × max(DifX) and DifY < Th2 × max(DifY) going backward from xb and yb. Similar to Step 1.4, preferably, Th2 = 0.1, that is, the coordinate that satisfies the conditions that the DifX value is less than one-tenth of its maximum value and the DifY value is less than one-tenth of its maximum value, denoted as x2, y2.

[0038] Th1 and Th2 therein can be adjusted and determined according to the actual situation.

[0039] Specifically, the process of normalizing the target box to L×L in step 2 is as follows:

[0040] Step 2.1: Scale the image by the longer side to L pixels and record the scaling factor a1 = Len / L, where Len is the original length of the longer side of the image;

[0041] Step 2.2: Fill the short side with 0 to L pixels from the right and bottom.

[0042] Specifically, the process of determining the vertices (xp0, yp0) and orientation angle (angle, counterclockwise angle with respect to the positive X-axis) of the aircraft within the normalized target box using the trained neural network model in step 3 is as follows:

[0043] Step 3.1: Select or construct a neural network model that can regress the vertex and orientation, such as the lightweight MobileNet model, ShuffleNet model, or SqueezeNet model. These models have simple structures, low computational cost, and fast inference speed, making them suitable for efficient prediction of the vertex and orientation of aircraft.

[0044] Step 3.2: Train the neural network model using the normalized bounding box data; the training dataset can be constructed by overlaying various aircraft models with arbitrary orientations and positions on any image background, recording the vertices and orientations of the aircraft in each image as data labels. Since the input image size of the neural network is fixed, combined with a large amount of data, the trained regression model has good versatility.

[0045] Step 3.3: Use a neural network model trained with normalized target bounding box data as input data to predict the aircraft's vertex (xp0, yp0) and orientation angle (angle, the angle counterclockwise from the positive X-axis).

[0046] Specifically, the process of calculating the image-level vertex coordinates (xp, yp) in step 4 is as follows:

[0047] Step 4.1: xp = xp0×a1 +x1;

[0048] Step 4.2: yp = yp0×a1 +y1.

[0049] Specifically, the process of filtering the region to be calculated based on vertices and orientation in step 5 is as follows:

[0050] Step 5.1: Using the perpendicular line pointing in the direction as the slope, obtain the equation of the line passing through the vertex (xp, yp), which can be specifically expressed as:

[0051] y−yp=tan(angle+pi / 2)⋅(x−xp)

[0052] Where angle is the angle of the aircraft's orientation counterclockwise from the positive X-axis, tan() is the tangent function, and pi represents 180 radians.

[0053] Step 5.2: Transform the line into the general form Ax + By + C = 0;

[0054] Step 5.3: Substitute the center point of the target box (0.5×(x1 +x2), 0.5×(y1 +y2)) into the equation of the line and record the sign of the equation result;

[0055] Step 5.4: For the point whose displacement is to be calculated, substitute it into the equation of the straight line. If the positive or negative value is the same as the center point of the target box, keep it; otherwise, discard it.

[0056] Example 1:

[0057] This invention provides a lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft. This case study utilizes the Python programming language, employs the OpenCV library for image processing, and utilizes the PyTorch library to build and train the neural network model. The experimental data consists of schlieren detection data from a high-speed aircraft, including 100 background images (Ib) and 100 background schlieren images (I).

[0058] Experimental results show that the present invention can accurately determine the region to be calculated, and the computational load is reduced by an average of approximately 64% compared to traditional methods. For detailed comparisons of the experimental results of the present invention, please refer to the following figures. Figure 2 and Figure 3 ,in Figure 2 The general form of the perpendicular bisector equation is -41x - 287y + 225582 = 0, which reduces the computational cost by 70.61%. Figure 3 The general form of the equation for the perpendicular bisector is 189x - 13y - 54489 = 0, which reduces the computational cost by 57.55%.

[0059] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight method for reducing computational load in background schlieren imaging of high-speed aircraft, characterized in that, It includes the following steps: Input the background image without the aircraft and the background schlieren image with the aircraft; Compress the background image without the aircraft and the background schlieren image with the aircraft into one-dimensional signals through background differential projection, including: summing the pixel values of the background image without the aircraft and the background schlieren image with the aircraft in the horizontal and vertical directions respectively, taking the difference between the summation results in the horizontal and vertical directions respectively, and taking the absolute value of the difference to obtain the differential signal DifX in the horizontal direction and the differential signal DifY in the vertical direction as the compressed one-dimensional signals; determining the target box according to the one-dimensional signals and normalizing the target box; Predict the vertex coordinates and orientation angles of the aircraft in the flight direction within the normalized target box using the trained lightweight neural network; Linearly map the predicted vertex coordinates and orientation angles of the aircraft to the original image space to obtain the original vertex coordinates and orientation angles of the aircraft; Based on the original vertex coordinates and orientation angles of the aircraft, construct a dividing line representing the maximum front of the shock wave. By calculating the position symbol Sc of the coordinate value of the center point of the target box relative to this dividing line, screen out all pixel points in the background schlieren image with the aircraft that have the same symbol as Sc, which constitutes the area to be calculated.

2. The method according to claim 1, characterized in that, The determining the target box according to the one-dimensional signals includes: Finding the first coordinate xa and the last coordinate xb that satisfy the condition DifX>Th1×max(DifX), where Th1 is a threshold between 0 and 1 and max(DifX) is the maximum value of DifX; Finding the first coordinate ya and the last coordinate yb that satisfy the condition DifY>Th1×max(DifY), where max(DifY) is the maximum value of DifY; Finding the first coordinate that satisfies DifX<Th2×max(DifX) and DifY<Th2×max(DifY) from xa and ya forward, denoted as x1 and y1, where Th2 is a threshold between 0 and 1; Finding the first coordinate that satisfies DifX<Th2×max(DifX) and DifY<Th2×max(DifY) from xb and yb backward, denoted as x2 and y2; Determining the target box from x1 and y1, x2 and y2.

3. The method according to claim 2, characterized in that, The predicting the vertex coordinates and orientation angles of the aircraft in the flight direction within the normalized target box using the trained lightweight neural network includes: Selecting the lightweight neural network; Training the neural network model using the training data within the normalized target box; the training data set is constructed by superimposing various aircraft models with arbitrary orientations and positions on an arbitrary image background, and recording the vertices and orientations of the aircraft in each image as data labels; Using the data of the normalized target box as input and inputting it into the trained lightweight neural network to predict the vertex (xp0, yp0) and orientation angle of the aircraft.

4. The method according to claim 3, characterized in that, The original vertex coordinates (xp, yp) of the aircraft are expressed as: xp = xp0×a1 +x1 yp = yp0×a1 +y1 Where the scaling factor a1 = Len / L, Len is the original length of the long side of the image, L is the side length of the target box, and the original orientation angle of the aircraft remains unchanged.

5. The method according to claim 4, characterized in that, The original vertex coordinates and orientation angle of the aircraft are used to construct a boundary line representing the maximum shock front. By calculating the position symbol Sc of the target box center point relative to this boundary line, all pixels in the background schlieren map containing the aircraft that have the same symbol Sc are selected, thus forming the region to be calculated, including: Using the perpendicular line to the angle as its slope, we obtain the equation of the line passing through the vertex (xp, yp), which is expressed as: and yp=tan(angle+pi / 2) (x xp) Where angle is the angle of the aircraft's orientation counterclockwise from the positive X-axis, tan() is the tangent function, and pi represents 180 radians. Transform the straight line into the general form Ax + By + C = 0; Substitute the center point of the target box (0.5×(x1 +x2), 0.5×(y1 +y2)) into the equation of the line and record the sign of the equation result; For a point whose displacement is to be calculated, substitute it into the equation of the straight line. If the positive or negative value is the same as the center point of the target box, keep it; otherwise, discard it.

6. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method as described in any one of claims 1-5.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-5.

Images