Visualization method and device based on fractal-like graph background schlieren
By using fractal-like graphics and affine transformation methods in background pattern technology, the problem of insufficient measurement precision in the prior art is solved, and higher accuracy and fineness are achieved, and it is suitable for complex flows such as engine tail jets. field measurement.
Patent Information
- Application Number
- CN202410935312.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-12
AI Technical Summary
When measuring the engine tail jet field, existing background pattern technology is difficult to measure comprehensively and accurately within a large area due to the local inaccuracy and insufficient precision introduced by random point distribution.
The background pattern method based on fractal-like graphics is adopted. By obtaining fractal-like graphics with self-similarity as the background image, and image displacement extraction is performed using affine transformed digital image correlation method to generate a more refined visual image.
The accuracy and precision of the measurement results are improved, the inaccuracy and point size introduced in the randomly distributed dot map are avoided on the fineness, and the visualization ability of jets of different flow velocities is enhanced.
Smart Images

Figure CN118941443A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of aviation technology, and in particular to a visualization method and device based on fractal-like graphic background schlieren. Background Art
[0002] Since the engine tail jet flow field has the characteristics of high temperature, ultra-high speed, large gradient and large range, the existing contact measurement methods represented by thermocouples and non-contact measurement methods represented by radiation method and spectroscopy are difficult to comprehensively and accurately measure the tail jet flow field over a large area due to limited measurement range and complex optical layout. Since the supersonic flow of the engine tail jet has obvious gas compressibility effect and is accompanied by strong aerodynamic heat, it is very suitable for research using background schlieren technology. Background schlieren technology is based on the principle that the refractive index of light in different density fields is different. Only a camera and a background are needed to dynamically measure the flow field, which greatly simplifies the structure and optical layout of the measuring device. Compared with other measurement technologies, background schlieren technology can make up for the limitations of traditional measurement methods in field of view, spatiotemporal resolution and layout, and can better meet the needs of engineering measurement.
[0003] In the prior art, when using the background schlieren technique to measure the jet, the background generally adopts a randomly distributed dot graph, and the background image displacement before and after the jet is calculated by cross-correlation or optical flow method, thereby forming a jet visualization result. Due to the variability of the number of random points in the query window (i.e., the search area when calculating the background image displacement), this point distribution will introduce local inaccuracies in the estimated displacement field. In addition, the random point background belongs to a sparse graph, and the size of its points directly affects the fineness of the jet visualization. Too large background points are prone to missing jet details, while too small background points are prone to over-deformation in a strong gradient flow field, thereby increasing the difficulty of extracting the displacement field. Summary of the invention
[0004] The object of the present invention is to provide a method and device for visualizing fractal-like graphic background schlieren to improve the accuracy and precision of test results.
[0005] In a first aspect, an embodiment of the present invention provides a visualization method based on fractal-like graphic background schlieren, comprising:
[0006] Acquire a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, respectively, and the background image is a fractal-like graphic with a self-similar pattern;
[0007] Based on the digital image correlation method of affine transformation, image displacement extraction is performed on the first image and the second image to obtain an image displacement vector;
[0008] Based on the image displacement vector, a visual image of the target to be measured is generated.
[0009] Furthermore, the background image includes a fractal graphic, or a synthetic fractal graphic obtained by synthesizing a sparse graph with geometric self-similarity and a dense graph with statistical self-similarity.
[0010] Furthermore, the synthetic fractal graphics are synthesized in the following manner:
[0011] Obtaining sparse graphs based on geometric self-similarity and dense graphs based on statistical self-similarity;
[0012] Sort the pixels in the sparse image and the dense image respectively according to the image intensity, to obtain a first sorting result corresponding to the sparse image and a second sorting result corresponding to the dense image;
[0013] Based on the first sorting result and the second sorting result, the image intensity of the sparse graph is adjusted to obtain an adjusted sparse graph; wherein the intensity of each pixel in the adjusted sparse graph is equal to the intensity of the pixel at the same sorting position in the dense graph;
[0014] The adjusted sparse graph and dense graph are weighted summed to obtain a synthetic fractal graph.
[0015] Furthermore, the sparse graph includes a semi-random point graph generated by a Poisson disk sampling algorithm, a sinusoidal fringe graph generated by a two-dimensional sinusoidal distribution, a plane filling curve graph generated by a Lindenmayer system, or a dendritic growth graph generated by a diffusion-limited aggregation model, and the dense graph includes a wavelet noise graph generated by upsampling and downsampling a Gaussian noise image.
[0016] Furthermore, based on the digital image correlation method of affine transformation, image displacement extraction is performed on the first image and the second image to obtain an image displacement vector, including:
[0017] Traversing each pixel in the first image;
[0018] For the traversed current pixel point, obtain the reference sub-region where the current pixel point is located, and determine the displacement value of the current pixel point based on the reference sub-region and the target sub-region matching the reference sub-region on the second image; wherein the reference sub-region is the neighborhood of the current pixel point, and the reference sub-region and the target sub-region have an affine transformation mapping relationship on the coordinates of the pixel point;
[0019] Until all the pixel points in the first image are traversed, an image displacement vector composed of the displacement values of the pixel points is obtained.
[0020] Further, based on the reference sub-region and the target sub-region on the second image that matches the reference sub-region, determining the displacement value of the current pixel point includes:
[0021] The Newton-Raphson iteration method is used to solve the following calculation formula under the first-order affine transformation to obtain the incremental displacement parameter vector Δp when the displacement parameter vector p of the current pixel is expanded by the second-order Taylor series. Based on the incremental displacement parameter vector Δp, the displacement parameter vector p is obtained by iterative update, and u and v in the displacement parameter vector p are determined as the displacement values of the current pixel:
[0022]
[0023]
[0024]
[0025] p=(u,u x ,u y ,v,v x , v y ) T
[0026] in, is the similarity C ZNSSD (p) is the gradient at p, is the similarity C ZNSSD (p) Hessian matrix at p, x = (x, y) T is the coordinate of the pixel in the reference sub-area, Δx=(Δx, Δy) T , Δx and Δy are the pixel distances from the pixel in the reference sub-region to the current pixel, W(Δx; p) is the affine transformation function that maps the coordinates of the pixel in the reference sub-region to the target sub-region, f(x) and g(x+W(Δx; p)) represent the grayscale values at the coordinate x of the reference sub-region and the coordinate x+W(Δx; p) of the target sub-region, respectively. are the mean grayscale values of each pixel in the reference sub-area and the mean grayscale values of each pixel in the target sub-area, respectively. Δf and Δg are the standard deviations of the grayscale values of each pixel in the reference sub-area and the standard deviations of the grayscale values of each pixel in the target sub-area, respectively. The reference sub-area contains (2M+1)×(2M+1) pixels centered on the current pixel. u and v are the displacement values of the current pixel in the x and y directions, respectively. u x 、u y 、v x 、v y is the first-order displacement gradient.
[0027] Furthermore, based on the image displacement vector, a visual image of the target to be measured is generated, including:
[0028] The displacement values of each pixel point in the image displacement vector in two directions are converted into angle values and radial distance values in the polar coordinate system, and the angle value, radial distance value and 1 of each pixel point in the image displacement vector are used as the hue value, brightness value and saturation value of the pixel point respectively to obtain a visual image of the target to be measured.
[0029] In a second aspect, an embodiment of the present invention further provides a visualization device based on fractal-like graphic background schlieren, comprising:
[0030] An acquisition module is used to acquire a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with a self-similar pattern;
[0031] An extraction module, used for performing image displacement extraction on the first image and the second image based on a digital image correlation method of affine transformation to obtain an image displacement vector;
[0032] The generation module is used to generate a visual image of the target to be measured based on the image displacement vector.
[0033] In a third aspect, an embodiment of the present invention further provides an image processing device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the visualization method based on fractal-like graphic background schlieren of the first aspect is implemented.
[0034] In a fourth aspect, an embodiment of the present invention further provides a visualization system based on background schlieren of quasi-fractal graphics, comprising the image processing device of the third aspect, and also comprising a background plate and an image acquisition module; the background plate and the image acquisition module are respectively arranged on both sides of the target to be measured, the background image arranged on the background plate is a quasi-fractal graphic with a self-similar pattern, the image acquisition module is connected to the image processing device, and the image acquisition module is used to collect the background image on the background plate in an environment without a target to be measured and in an environment with a target to be measured, respectively, and send both the collected first image and the second image to the image processing device.
[0035] The visualization method and device based on fractal-like background schlieren provided by the embodiment of the present invention can obtain the first image and the second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting the background image on the background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with self-similar pattern; then, based on the digital image correlation method of affine transformation, the first image and the second image are subjected to image displacement extraction to obtain the image displacement vector; and then based on the image displacement vector, a visualization image of the target to be measured is generated. In this way, when the background schlieren technology is used to measure the target to be measured of the engine tail jet, a fractal-like graphic with self-similar pattern is used as the background image, and the image displacement is extracted based on the digital image correlation method of affine transformation. The self-similarity of the pattern can add more features that are easy to be identified to the background image. The digital image correlation method based on affine transformation can avoid the problems of outliers, roughness and high-frequency information filtering, and at the same time avoid the inaccuracy introduced by the variability of the number of random points in the inquiry window in the randomly distributed dot map and the influence of the point size on the precision and extraction difficulty, thereby improving the accuracy and precision of the test results. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 A structural schematic diagram of a visualization system based on fractal-like graphic background schlieren provided by an embodiment of the present invention;
[0038] Figure 2 A schematic flow chart of a visualization method based on fractal-like graphic background schlieren provided by an embodiment of the present invention;
[0039] Figure 3 A variety of background images provided for embodiments of the present invention;
[0040] Figure 4 Visualized images corresponding to various background images provided in the embodiments of the present invention;
[0041] Figure 5 Visual image comparison results corresponding to various displacement extraction methods provided in the embodiments of the present invention;
[0042] Figure 6 A schematic diagram of the structure of a visualization device based on fractal-like graphic background schlieren provided by an embodiment of the present invention;
[0043] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0044] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0045] At present, when the background schlieren technology is used to measure the jet, the background generally adopts a randomly distributed dot pattern. The variability of the number of random points in the inquiry window will introduce local inaccuracy, and the size of the points affects the fineness and extraction difficulty. Based on this, an embodiment of the present invention provides a visualization method and device based on the background schlieren of a fractal-like graphic. A fractal-like graphic with self-similarity in the pattern is used as the background image. The inherent self-affine characteristics of the fractal-like graphic are utilized, and an affine transformation-based digital image correlation method is used to perform image displacement extraction. More features can be added to the background image for easy identification, thereby improving the visualization capability of jets with different flow rates under different measurement resolution requirements and different imaging environments.
[0046] The embodiment of the present invention provides a visualization system based on fractal-like graphic background schlieren, see Figure 1 The structure diagram of a visualization system based on fractal-like graphic background schlieren is shown, the system includes an image processing device 11, a background plate 12 and an image acquisition module 13; the background plate 12 and the image acquisition module 13 are respectively arranged on the target to be measured (i.e. Figure 1 The background image set on the background plate 12 is a fractal-like graphic with self-similarity. The background plate 12 is used to provide deflection information generated when light passes through the flow field. The image acquisition module 13 is connected to the image processing device 11. The image acquisition module 13 is used to collect the background image on the background plate 12 in an environment without a target to be measured and in an environment with a target to be measured, and send the collected first image and second image to the image processing device 11. The image processing device 11 considers the self-affine characteristics of the background image, and extracts and visualizes the image displacement field based on the first image and the second image.
[0047] The above-mentioned target to be measured can be the flow field to be measured of the engine tail jet, or it can be the shock wave of ammunition explosion, or it can be a shock wave (i.e., a strong compression wave in supersonic gas) or a flame, etc. The embodiment of the present invention does not limit the target to be measured.
[0048] To facilitate understanding of this embodiment, a visualization method based on fractal-like graphic background schlieren disclosed in an embodiment of the present invention is introduced in detail below.
[0049] The embodiment of the present invention provides a method for visualizing fractal-like graphic background schlieren, which can be executed by the above-mentioned image processing device 11. Figure 2 The flowchart of a method for visualizing fractal-like graphic background schlieren is shown, and the method mainly includes the following steps S210 to S230:
[0050] Step S210, obtaining a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, respectively, and the background image is a fractal-like graphic with a self-similar pattern.
[0051] Quasi-fractal graphics are graphics with certain geometric self-similarity and / or statistical self-similarity, and quasi-fractal graphics include fractal graphics with strict data definition. Optionally, the above-mentioned background image may include a fractal graphic, or a synthetic fractal graphic (i.e., a quasi-fractal graphic) obtained by synthesizing a sparse graph with geometric self-similarity and a dense graph with statistical self-similarity. Fractal graphics are a kind of graphics with self-similar properties, that is, they present similar structures at different scales. The pattern of the synthetic fractal graphics maintains both geometric self-similarity and statistical self-similarity within a certain range. Among them, a sparse graph refers to a graph with few edges and many points in the graph; a dense graph refers to a graph with many edges and few points in the graph.
[0052] For ease of understanding, the synthesis method of the above-mentioned synthetic fractal graphics is introduced below.
[0053] The above-mentioned synthetic fractal graphics are synthesized in the following manner: obtaining a sparse graph based on geometric self-similarity and a dense graph with statistical self-similarity; sorting each pixel in the sparse graph and the dense graph according to the image intensity, respectively, to obtain a first sorting result corresponding to the sparse graph and a second sorting result corresponding to the dense graph; adjusting the image intensity of the sparse graph based on the first sorting result and the second sorting result, to obtain an adjusted sparse graph; wherein the intensity of each pixel in the adjusted sparse graph is equal to the intensity of the pixel at the same sorting position in the dense graph; performing weighted summation processing on the adjusted sparse graph and the dense graph, to obtain a synthetic fractal graphics. The synthetic fractal graphics obtained in this way maintain both geometric self-similarity and statistical self-similarity within a certain range.
[0054] Furthermore, the above-mentioned sparse graph may include a semi-random point graph generated by a Poisson disk sampling algorithm, a sinusoidal fringe graph generated by a two-dimensional sinusoidal distribution, a plane filling curve graph generated by a Lindenmayer system (referred to as L-system), or a dendritic growth graph generated by a diffusion-limited aggregation model, and the dense graph includes a wavelet noise graph generated by upsampling and downsampling a Gaussian noise image.
[0055] In some possible embodiments, the pixels of the sparse map can be sorted first according to the grayscale value, and then the pixels of the dense map can be sorted; or the pixels of the dense map can be sorted first according to the grayscale value, and then the pixels of the sparse map can be sorted. The above-mentioned step of adjusting the image intensity of the sparse map can be implemented by the following process: for each pixel in the sparse map, the corresponding pixel of the same sorting position in the dense map is found based on the sorting position of the pixel, and the grayscale value of the pixel is adjusted to be the same as the grayscale value of the corresponding pixel. When the adjusted sparse map and dense map are weighted and summed, the weights used (including the weights corresponding to the sparse map and the weights corresponding to the dense map) depend on the measurement conditions: when the image noise of the image collected by the image acquisition module 13 is large, the weight of the dense map can be reduced; conversely, when the spatial resolution of the sparse map is too sparse for the target and the imaging quality is good, the weight of the dense map can be increased. The weight can be an empirical value set manually, or it can be determined based on the image noise of the image collected by the image acquisition module 13 and the spatial resolution of the sparse map. The embodiment of the present invention does not limit the specific determination method of the weight. For example, the following scoring method is used: the correspondence between image noise and score and the relationship between spatial resolution and score are preset, the image noise and the spatial resolution of the sparse graph are converted into corresponding scores respectively, and the weight of the sparse graph and the weight of the dense graph are determined based on the sum of the scores of the two (i.e., the total score) and the correspondence between the total score and the weight. In addition, the weight can also be output through artificial intelligence, for example, the weight is determined by a trained deep learning model.
[0056] Step S220: performing image displacement extraction on the first image and the second image based on the digital image correlation method of affine transformation to obtain an image displacement vector.
[0057] In this embodiment, since the pattern of the background image has self-similarity, the background image has a self-affine property. Considering the self-affine property, a digital image correlation method based on affine transformation can be used to extract image displacement. This extraction method can enhance the robustness of the measurement result.
[0058] In some possible embodiments, the above step S220 may be implemented by the following sub-steps:
[0059] Sub-step 1: traverse each pixel in the first image.
[0060] All pixels in the first image may be traversed, or some pixels in the first image may be traversed. The pixels of the first image may be traversed in a preset traversal manner, and the traversed pixels may also be referred to as points of interest. The traversal manner may be set according to actual needs and is not limited here. For example, the traversal may be performed in a left-to-right or top-to-bottom order.
[0061] Sub-step 2, for the current pixel point traversed, obtain the reference sub-area where the current pixel point is located, and determine the displacement value of the current pixel point based on the reference sub-area and the target sub-area matching the reference sub-area on the second image; wherein the reference sub-area is the neighborhood of the current pixel point, and the reference sub-area and the target sub-area have an affine transformation mapping relationship on the coordinates of the pixel point.
[0062] The zero-mean normalized sum of squared differences can be used to quantitatively evaluate the similarity between the reference sub-region and the deformed sub-region in the second image, and the deformed sub-region with the highest similarity is determined as the matching target sub-region, thereby obtaining the displacement value between the current pixel and the center point of the target sub-region. If the neighboring points of the point of interest in the reference sub-region are still the neighboring points of the corresponding pixel points in the target sub-region, the coordinates of the points around the center of the reference sub-region can be mapped to the points in the target sub-region according to the affine transformation. When performing affine transformation mapping, the order of the affine transformation can be selected according to actual needs. The higher the order, the more image geometric deformation types that can be described, and the more complex image deformation can be described, but the number of unknown quantities is also greater, and the difficulty of solving is greater. For example, if there is only rigid translation in the reference sub-region and the deformed sub-region, a zero-order affine transformation can be used; if there are additional shape transformations between the reference sub-region and the deformed sub-region on the basis of translation, such as rotation, shearing and scaling, a first-order affine transformation can be used; if there are more complex deformation states, such as bending and warping, a second-order affine transformation can be used. The unknown number related to the displacement in the similarity can be defined as a displacement parameter vector p, so that the problem of solving the point displacement value of the current pixel can be converted into the problem of solving the displacement parameter vector p.
[0063] In a possible implementation, the displacement value of the current pixel is determined by the following process:
[0064] The Newton-Raphson iteration method (also known as the Newton-Raphson iteration method) is used to solve the following calculation formula under the first-order affine transformation to obtain the incremental displacement parameter vector Δp when the displacement parameter vector p of the current pixel point is expanded by the second-order Taylor series. Based on the incremental displacement parameter vector Δp, the displacement parameter vector p is obtained by iterative updating, and u and v in the displacement parameter vector p are determined as the displacement values of the current pixel point:
[0065]
[0066]
[0067]
[0068] p=(u,u x ,u y ,v,v x , v y ) T
[0069] in, is the similarity C ZNSSD (p) is the gradient at p, is the similarity C ZNSSD (p) Hessian matrix at p, x = (x, y) T is the coordinate of the pixel in the reference sub-area, Δx=(ΔxΔy) T , Δx and Δy are the pixel distances from the pixel in the reference sub-region to the current pixel, W(Δx; p) is the affine transformation function that maps the coordinates of the pixel in the reference sub-region to the target sub-region, f(x) and g(x+W(Δx; p)) represent the grayscale values at the coordinate x of the reference sub-region and the coordinate x+W(Δx; p) of the target sub-region, respectively. are the mean grayscale values of each pixel in the reference sub-area and the mean grayscale values of each pixel in the target sub-area, respectively. Δf and Δg are the standard deviations of the grayscale values of each pixel in the reference sub-area and the standard deviations of the grayscale values of each pixel in the target sub-area, respectively. The reference sub-area contains (2M+1)×(2M+1) pixels centered on the current pixel. u and v are the displacement values of the current pixel in the x and y directions, respectively. u x 、u y 、v x 、v y is the first-order displacement gradient.
[0070] It should be noted that if a second-order affine transformation or an n-order affine transformation is adopted, it is only necessary to modify the affine transformation function W(Δx; p).
[0071] Sub-step 3, until all pixel points in the first image are traversed, an image displacement vector composed of the displacement values of each pixel point is obtained.
[0072] Step S230: generating a visualized image of the target to be measured based on the image displacement vector.
[0073] Each pixel point in the image displacement vector includes two-dimensional data in the Cartesian coordinate system. In order to realize the visualization of the target to be measured, the image displacement vector in the Cartesian coordinate system can be transformed into the polar coordinate system to obtain the angle and radial distance. Then, the angle, radial distance and vector 1 (i.e., a vector with all elements being 1) are brought into the HSV (H refers to hue, S refers to saturation, and V refers to brightness) color coding to serve as hue, brightness and saturation respectively to obtain a visualized image.
[0074] Based on this, the above step S230 can be implemented through the following process: convert the displacement values of each pixel point in the image displacement vector in two directions into angle values and radial distance values in the polar coordinate system, and use the angle value, radial distance value and 1 of each pixel point in the image displacement vector as the hue value, brightness value and saturation value of the pixel point respectively to obtain a visual image of the target to be measured.
[0075] The visualization method and system based on fractal-like background schlieren provided by the embodiment of the present invention can obtain the first image and the second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting the background image on the background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with self-similar pattern; then, based on the digital image correlation method of affine transformation, the first image and the second image are subjected to image displacement extraction to obtain the image displacement vector; and then, based on the image displacement vector, a visualization image of the target to be measured is generated. In this way, when the background schlieren technology is used to measure the target to be measured of the engine tail jet, a fractal-like graphic with self-similar pattern is used as the background image, and the image displacement is extracted based on the digital image correlation method of affine transformation. The self-similarity of the pattern can add more features that are easy to be identified to the background image. The digital image correlation method based on affine transformation can avoid the problems of outliers, roughness and high-frequency information filtering, and at the same time avoid the inaccuracy introduced by the variability of the number of random points in the inquiry window in the randomly distributed dot map and the influence of the size of the points on the precision and extraction difficulty, thereby improving the accuracy and precision of the test results.
[0076] For ease of understanding, the visualization system and method based on fractal-like graphic background schlieren are introduced in detail below, taking the flow field to be measured in which the target to be measured is the engine tail jet as an example.
[0077] In order to add more features to the background image for easy identification, an embodiment of the present invention proposes a background schlieren measurement system and method based on synthetic fractal graphics, which improves the system's visualization capability for jets with different flow rates under different measurement resolution requirements.
[0078] Specifically, Figure 1As shown, the system includes a background plate 12 with a synthetic fractal image as the background image, an image acquisition module 13, and an image processing device 11 that considers the fractal self-affine feature; the background plate 12 and the image acquisition module 13 are respectively placed on both sides of the flow field of the nozzle to be tested 14. The nozzle to be tested 14 is used to generate the flow field to be tested of the engine tail jet.
[0079] The background plate 12 is used to provide the deflection information generated when the light passes through the flow field. A fractal-like image with certain similar features is synthesized using a synthetic fractal algorithm as the background image on the background plate. The background image includes but is not limited to Figure 3 The image shown. Figure 3 (a) is a semi-random point graph generated by the Poisson disk sampling algorithm; Figure 3 (b) is a sinusoidal fringe pattern generated by a two-dimensional sinusoidal distribution; Figure 3 (c) is the plane filling curve diagram generated by the L-system; Figure 3 (d) is the dendritic growth diagram generated by the diffusion-limited aggregation model. Figure 3 (a) to (d) in the figure all have certain geometric self-similarity, and Figure 3 (c) in has a strict fractal mathematical definition, while Figure 3 The different irregular branches in (d) also reflect the complexity of the pattern. Figure 3 (e) in the figure is a wavelet noise image (i.e., wavelet image) generated by upsampling and downsampling the Gaussian noise image. It has statistical self-similarity, and the pattern can have similar statistical characteristics at different scales. Figure 3 (a) to (d) in the figure belong to sparse graphs, which are highly resistant to noise but have constant spatial resolution. Figure 3 (e) in the figure belongs to a dense graph, which has weak noise resistance but has multi-scale spatial resolution. Figure 3 (f) to (h) in the figure are a new type of fractal-like graphic background image generated by combining sparse graphs and dense graphs, which can achieve both geometric self-similarity and statistical self-similarity of the pattern within a certain range. The synthesis method is as follows:
[0080] Step 1: Sort the pixels of the selected sparse graph and dense graph respectively according to the grayscale values of the pixels, and make the sorting strength of the sparse graph equal to the sorting strength of the dense graph. The sorting strength here means that the grayscale values of the pixels at the same sorting position are the same.
[0081] Step 2: Perform weighted addition of the sparse graph and the dense graph after the sorting intensity adjustment to generate a synthetic fractal graph. The weight of the weighted addition depends on the measurement conditions: when the image noise of the image collected by the image acquisition module 13 is large, the weight of the dense graph can be reduced; conversely, when the spatial resolution of the sparse graph is too sparse for the target and the imaging quality is good, the weight of the dense graph can be increased.
[0082] According to the above synthesis method, Figure 3 (f) in the figure is a wavelet-dot graph that combines a semi-random dot graph and a wavelet graph. Figure 3 (g) in the figure is a wavelet-sinusoidal fringe image that combines a sine fringe image and a wavelet image. Figure 3 (h) in the figure is a wavelet-filling curve graph that combines a plane filling curve graph and a wavelet graph.
[0083] The image acquisition module 13 is used to acquire background images on the background plate 12 in an environment without a flow field to be measured and in an environment with a flow field to be measured. Then the image processing device 11 extracts the displacement of the acquired image. The image displacement extraction method adopts a digital image correlation technology, taking into account the self-affine characteristics of fractal graphics, that is, similar patterns can be transformed through affine transformation. The steps of image displacement extraction are as follows:
[0084] Step 1: predefine a sub-region centered on the pixel of interest (called the reference sub-region) on the background image without the flow field to be measured (i.e., the first image), and search for its corresponding target sub-region on the background image with the flow field to be measured (i.e., the second image). The zero-mean normalized sum of squared differences is used to quantitatively evaluate the similarity between the reference sub-region and the target sub-region:
[0085]
[0086] Where x = (x, y) T is the coordinate of the pixel in the reference sub-region. W(x) is the displacement of the coordinate x of the reference sub-region. f(x) and g(x+W(x)) represent the image grayscale values at the coordinate x of the reference sub-region and the coordinate x+W(x) of the target sub-region, respectively. If the pixel of the reference sub-region is located at a sub-pixel position, the image intensity (i.e., image grayscale value) at that position is estimated using bicubic B-spline interpolation, which is a bicubic interpolation with cubic B-spline as the basis function. are the means of f(x) and g(x+W(x)), respectively, and Δf and Δg are the standard deviations of f(x) and g(x+W(x)). The reference sub-area Ω can be defined as a square area containing (2M+1)×(2M+1) pixels centered at the point of interest x.
[0087] Assuming that the neighboring points of the interest point x in the reference subregion are still the neighboring points at the target subregion x+W(x), the coordinates of the points around the center of the reference subregion can be mapped to the points in the target subregion according to the affine transformation:
[0088] x′=x0+Δx+ξ(x0+Δx,y0+Δy)
[0089] y′=y0+Δy+η(x0+Δx,y0+Δy) (2)
[0090] Where Δx and Δy are the pixel distances from the point Q(x,y) to the subregion center P(x0,y0), x = x0 + Δx, y = y0 + Δy. ξ(x,y) and η(x,y) form the displacement W(x) of the subregion center P(x0,y0), x = (x,y). If only rigid translation exists in the reference subregion and the deformed subregion in the second image, a zero-order affine transformation is used:
[0091] ξ0(x0+Δx,y0+Δy)=u
[0092] η0(x0+Δx,y0+Δy)=v (3)
[0093] On the basis of translation, if there are additional shape transformations between the reference subregion and the deformed subregion, such as rotation, shearing and scaling, a first-order affine transformation can be used:
[0094] ξ1(x0+Δx,y0+Δy)=u+u x Δx+u y Δy
[0095] η0(x0+Δx,y0+Δy)=v+v x Δx+v y Δy (4)
[0096] If there are more complex deformation states, including bending and warping, a second-order affine transformation is used:
[0097]
[0098] Wherein, u and v in equations (3) to (5) are the displacement components in the x and y directions of the reference sub-area center point P(x0, y0), respectively. x 、u y 、v x 、v y is the first-order displacement gradient, u xx 、u xy 、u yy 、v xx 、v xy 、v yy is the second-order displacement gradient.
[0099] Step 2: Define the unknown number related to the displacement in step 1 as a displacement parameter vector p. For example, for a first-order affine transformation, p = (u, u x ,u y ,v,v x , v y ) T Then discretize equation (1) into:
[0100]
[0101] Where Δx = (Δx, Δy) T is the local coordinate in the subregion centered at point x, and W(Δx; p) is the affine transformation function that maps the displacement of the local point in the reference subregion to the target subregion, such as for a first-order affine transformation:
[0102]
[0103] Step 3: Minimize equation (7) to obtain the optimal parameter vector p. A second-order Taylor series expansion can be performed around p, and the partial derivative of the expansion with respect to the incremental displacement parameter vector Δp is taken. The position where the partial derivative is zero is considered to be the optimal parameter vector p:
[0104]
[0105] in, is the similarity C ZNSSD (p) is the gradient at p, is the similarity C ZNSSD (p) The Hessian matrix at p. Through the Newton-Raphson iteration method, we first solve equation (8) to obtain the incremental parameter vector Δp, and then iteratively update the parameter p until p converges.
[0106] Step 4: Select a new point of interest, and obtain the displacement value at the point according to steps 1 to 3. Repeat this process until all points of interest in the first image are traversed.
[0107] Figure 4 The visualization results of the image displacement field of the supersonic jet at a certain moment under different background images are shown. The steps to obtain the visualization results are as follows: first, the image displacement vector (u, v) in the Cartesian coordinate system is transformed into the polar coordinate system to obtain the angle and radial distance; then the angle, radial distance and vector 1 are brought into the HSV color code to serve as hue, brightness and saturation respectively. Figure 4The two-dimensional color disk on the far right represents the image displacement vector values corresponding to different colors and different saturations. It can be seen that when the resolution of the sparse background (i.e., the background image using a sparse graph) (such as a plane filling curve) is too low compared to the flow field deflection, the new fractal-like graph (such as a wavelet-filling curve) can improve the accuracy of displacement estimation.
[0108] Figure 5 The displacement extraction results of random point images, wavelet images, and sinusoidal fringe images by different image displacement extraction methods are compared. Figure 4 In addition to the digital image correlation method based on affine transformation, the existing image displacement extraction methods compared include the cross-correlation method based on particle image velocimetry, the optical flow method, and the Fourier transform profile method. It can be seen that even under the same fractal background, the visualization results of the digital image correlation method based on affine transformation avoid the problems of outliers, roughness, and high-frequency information filtering as much as possible, and can adapt to more fractal backgrounds (i.e., background graphics using fractal graphics).
[0109] In summary, the embodiments of the present invention have the following beneficial effects:
[0110] 1. For the first time, we proposed to use fractal-like graphics as the background image of the system. Fractal-like graphics can add more features to the background image for easy identification due to their geometric self-similarity, statistical self-similarity, multi-scale and complexity. In addition, different fractal-like graphics have different advantages in resisting large displacement, high noise, high blur and low resolution, and can adapt to more complex test environments.
[0111] 2. A general method for generating new fractal-like graphics is proposed, which inherits both the noise resistance of sparse graphs and the scale invariance of dense graphs. The new graphics (i.e., synthetic fractal graphics) can maintain both geometric self-similarity and statistical self-similarity within a certain range. The weighted nature of the generation method allows users to strike a balance between noise resistance and scale invariance, making the background image more versatile.
[0112] 3. Taking advantage of the inherent self-affine properties of fractal-like graphics, the image displacement algorithm shows enhanced robustness in background schlieren jet measurements by combining affine transformation.
[0113] Corresponding to the above-mentioned visualization method based on the background schlieren of a fractal-like graphic, an embodiment of the present invention further provides a visualization device based on the background schlieren of a fractal-like graphic. Figure 6 A structural schematic diagram of a visualization device based on fractal-like graphic background schlieren is shown, the device comprising:
[0114] An acquisition module 61 is used to acquire a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with a self-similar pattern;
[0115] An extraction module 62, configured to extract image displacement of the first image and the second image based on a digital image correlation method of affine transformation to obtain an image displacement vector;
[0116] The generating module 63 is used to generate a visual image of the target to be measured based on the image displacement vector.
[0117] The visualization device based on fractal-like background schlieren provided by the embodiment of the present invention can obtain the first image and the second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting the background image on the background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with self-similar pattern; then, based on the digital image correlation method of affine transformation, the first image and the second image are subjected to image displacement extraction to obtain the image displacement vector; and then, based on the image displacement vector, a visualization image of the target to be measured is generated. In this way, when the background schlieren technology is used to measure the target to be measured of the engine tail jet, a fractal-like graphic with self-similar pattern is used as the background image, and the image displacement is extracted based on the digital image correlation method of affine transformation. The self-similarity of the pattern can add more features that are easy to be identified to the background image. The digital image correlation method based on affine transformation can avoid the problems of outliers, roughness and high-frequency information filtering, and at the same time avoid the inaccuracy introduced by the variability of the number of random points in the inquiry window in the randomly distributed dot map and the influence of the point size on the precision and extraction difficulty, thereby improving the accuracy and precision of the test results.
[0118] Furthermore, the above-mentioned background image includes a fractal graphic, or a synthetic fractal graphic obtained by synthesizing a sparse graph with geometric self-similarity and a dense graph with statistical self-similarity.
[0119] In some possible embodiments, the above-mentioned device also includes a synthesis module, which is used to: obtain a sparse graph based on geometric self-similarity and a dense graph with statistical self-similarity; sort each pixel point in the sparse graph and the dense graph according to the image intensity to obtain a first sorting result corresponding to the sparse graph and a second sorting result corresponding to the dense graph; based on the first sorting result and the second sorting result, adjust the image intensity of the sparse graph to obtain an adjusted sparse graph; wherein the intensity of each pixel point in the adjusted sparse graph is equal to the intensity of the pixel point at the same sorting position in the dense graph; and perform weighted summation processing on the adjusted sparse graph and the dense graph to obtain a synthesized fractal graph.
[0120] Furthermore, the above-mentioned sparse graph includes a semi-random point graph generated by a Poisson disk sampling algorithm, a sinusoidal fringe graph generated by a two-dimensional sinusoidal distribution, a plane filling curve graph generated by a Lindenmayer system, or a dendritic growth graph generated by a diffusion-limited aggregation model, and the dense graph includes a wavelet noise graph generated by upsampling and downsampling a Gaussian noise image.
[0121] Furthermore, the above-mentioned extraction module 62 is specifically used to: traverse each pixel point in the first image; for the current pixel point traversed, obtain the reference sub-area where the current pixel point is located, and determine the displacement value of the current pixel point based on the reference sub-area and the target sub-area matching the reference sub-area on the second image; wherein, the reference sub-area is the neighborhood of the current pixel point, and the reference sub-area and the target sub-area have an affine transformation mapping relationship on the coordinates of the pixel point; until all the pixel points in the first image are traversed, an image displacement vector composed of the displacement values of each pixel point is obtained.
[0122] Furthermore, the extraction module 62 is also used to solve the following calculation formula under the first-order affine transformation by the Newton-Raphson iteration method to obtain the incremental displacement parameter vector Δp when the displacement parameter vector p of the current pixel point is expanded by the second-order Taylor series, and based on the incremental displacement parameter vector Δp, obtain the displacement parameter vector p by iterative updating, and determine u and v in the displacement parameter vector p as the displacement values of the current pixel point:
[0123]
[0124]
[0125] p=(u,u x ,u y ,v,v x , v y ) T
[0126] in, is the similarity C ZNSSD (p) is the gradient at p, is the similarity C ZNSSD (p) Hessian matrix at p, x = (x, y) T is the coordinate of the pixel in the reference sub-area, Δx=(Δx, Δy) T , Δx and Δy are the pixel distances from the pixel in the reference sub-region to the current pixel, W(Δx; p) is the affine transformation function that maps the coordinates of the pixel in the reference sub-region to the target sub-region, f(x) and g(x+W(Δx; p)) represent the grayscale values at the coordinate x of the reference sub-region and the coordinate x+W(Δx; p) of the target sub-region, respectively. are the mean grayscale values of each pixel in the reference sub-area and the mean grayscale values of each pixel in the target sub-area, respectively. Δf and Δg are the standard deviations of the grayscale values of each pixel in the reference sub-area and the standard deviations of the grayscale values of each pixel in the target sub-area, respectively. The reference sub-area contains (2M+1)×(2M+1) pixels centered on the current pixel. u and v are the displacement values of the current pixel in the x and y directions, respectively. u x 、u y 、v x 、v y is the first-order displacement gradient.
[0127] Furthermore, the above-mentioned generation module 63 is specifically used to: convert the displacement values of each pixel point in the image displacement vector in two directions into angle values and radial distance values in the polar coordinate system, and use the angle value, radial distance value and 1 of each pixel point in the image displacement vector as the hue value, brightness value and saturation value of the pixel point, respectively, to obtain a visual image of the target to be measured.
[0128] The visualization device based on fractal-like background schlieren provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned visualization method based on fractal-like background schlieren. For the sake of brief description, for parts not mentioned in the embodiment of the visualization device based on fractal-like background schlieren, reference may be made to the corresponding contents in the aforementioned visualization method based on fractal-like background schlieren.
[0129] like Figure 7 As shown, an image processing device 700 provided by an embodiment of the present invention includes: a processor 701, a memory 702 and a bus, the memory 702 stores a computer program that can be run on the processor 701, when the image processing device 700 is running, the processor 701 and the memory 702 communicate through the bus, and the processor 701 executes the computer program to realize the above-mentioned visualization method based on the fractal-like graphic background schlieren.
[0130] Specifically, the memory 702 and the processor 701 can be general-purpose memories and processors, which are not specifically limited here.
[0131] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the visualization method based on fractal-like graphic background schlieren in the previous method embodiment is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.
[0132] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.
[0133] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.
[0134] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0135] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A visualization method based on fractal-like graphic background schlieren, characterized in that: include: Acquire a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, respectively, and the background image is a fractal-like graphic with a self-similar pattern; Based on the digital image correlation method of affine transformation, performing image displacement extraction on the first image and the second image to obtain an image displacement vector; Based on the image displacement vector, a visualized image of the target to be measured is generated.
2. The visualization method based on fractal-like graphic background schlieren according to claim 1, characterized in that: The background image includes a fractal graph, or a synthetic fractal graph obtained by synthesizing a sparse graph with geometric self-similarity and a dense graph with statistical self-similarity.
3. The visualization method based on fractal-like graphic background schlieren according to claim 2, characterized in that: The synthetic fractal graphics are synthesized in the following manner: Obtaining sparse graphs based on geometric self-similarity and dense graphs based on statistical self-similarity; Sort the pixels in the sparse graph and the dense graph respectively according to the image intensity to obtain a first sorting result corresponding to the sparse graph and a second sorting result corresponding to the dense graph; Based on the first sorting result and the second sorting result, intensity adjustment is performed on the image intensity of the sparse graph to obtain an adjusted sparse graph; wherein the intensity of each pixel point in the adjusted sparse graph is equal to the intensity of the pixel point at the same sorting position in the dense graph; The adjusted sparse graph and the dense graph are subjected to weighted summation to obtain the composite fractal graph.
4. The visualization method based on fractal-like graphic background schlieren according to claim 2 or 3, characterized in that: The sparse graph includes a semi-random point graph generated by a Poisson disk sampling algorithm, a sinusoidal fringe graph generated by a two-dimensional sinusoidal distribution, a plane filling curve graph generated by a Lindenmayer system, or a dendritic growth graph generated by a diffusion-limited aggregation model. The dense graph includes a wavelet noise graph generated by upsampling and downsampling a Gaussian noise image.
5. The visualization method based on fractal-like graphic background schlieren according to claim 1, characterized in that: The digital image correlation method based on affine transformation performs image displacement extraction on the first image and the second image to obtain an image displacement vector, including: Traversing each pixel point in the first image; For the traversed current pixel point, obtain a reference sub-region where the current pixel point is located, and determine a displacement value of the current pixel point based on the reference sub-region and a target sub-region matching the reference sub-region on the second image; wherein the reference sub-region is a neighborhood of the current pixel point, and the reference sub-region and the target sub-region have an affine transformation mapping relationship in the coordinates of the pixel point; Until all the pixel points in the first image are traversed, an image displacement vector composed of the displacement values of the pixel points is obtained.
6. The visualization method based on fractal-like graphic background schlieren according to claim 5, characterized in that: The determining the displacement value of the current pixel point based on the reference sub-region and a target sub-region on the second image that matches the reference sub-region comprises: The following calculation formula under the first-order affine transformation is solved by the Newton-Raphson iteration method to obtain the incremental displacement parameter vector Δp when the displacement parameter vector p of the current pixel point is expanded by the second-order Taylor series, and based on the incremental displacement parameter vector Δp, the displacement parameter vector p is obtained by iterative updating, and u and v in the displacement parameter vector p are determined as the displacement values of the current pixel point: p=(u,u x ,u y ,v,v x ,v y ) T in, is the similarity C ZNSSD (p) is the gradient at p, is the similarity C ZNSSD (p) Hessian matrix at p, x = (x, y) T is the coordinate of the pixel point in the reference sub-area, Δx=(Δx, Δy) T , Δx and Δy are the pixel distances from the pixel in the reference sub-region to the current pixel, W(Δx; p) is the affine transformation function that maps the coordinates of the pixel in the reference sub-region to the target sub-region, f(x) and g(x+W(Δx; p)) represent the grayscale values at the coordinate x of the reference sub-region and the coordinate x+W(Δx; p) of the target sub-region, respectively. are the mean grayscale values of each pixel in the reference sub-area and the mean grayscale values of each pixel in the target sub-area, Δf and Δg are the standard deviations of the grayscale values of each pixel in the reference sub-area and the standard deviations of the grayscale values of each pixel in the target sub-area, respectively. The reference sub-area contains (2M+1)×(2M+1) pixels centered on the current pixel. u and v are the displacement values of the current pixel in the x and y directions, respectively. u x 、u y 、v x 、v y is the first-order displacement gradient.
7. The visualization method based on fractal-like graphic background schlieren according to claim 1, characterized in that: The step of generating a visual image of the target to be measured based on the image displacement vector includes: The displacement values of each pixel point in the image displacement vector in two directions are converted into angle values and radial distance values in the polar coordinate system, and the angle value, radial distance value and 1 of each pixel point in the image displacement vector are respectively used as the hue value, brightness value and saturation value of the pixel point to obtain a visualized image of the target to be measured.
8. A visualization device based on fractal-like graphic background schlieren, characterized in that: include: An acquisition module, used for acquiring a first image and a second image corresponding to the target to be measured; wherein the first image and the second image are respectively obtained by collecting a background image on a background plate in an environment without the target to be measured and in an environment with the target to be measured, and the background image is a fractal-like graphic with a self-similar pattern; An extraction module, configured to extract image displacement of the first image and the second image based on a digital image correlation method of affine transformation to obtain an image displacement vector; A generating module is used to generate a visual image of the target to be measured based on the image displacement vector.
9. An image processing device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the visualization method based on fractal-like graphic background schlieren according to any one of claims 1 to 7 is implemented.
10. A visualization system based on fractal-like graphic background schlieren, characterized in that: The image processing device comprises the image processing device as described in claim 9, and further comprises a background plate and an image acquisition module; the background plate and the image acquisition module are respectively arranged on both sides of the target to be measured, the background image arranged on the background plate is a fractal-like graphic with a self-similar pattern, the image acquisition module is connected to the image processing device, and the image acquisition module is used to respectively acquire the background image on the background plate in an environment without the target to be measured and in an environment with the target to be measured, and send both the acquired first image and the second image to the image processing device.
Citation Information
Cited By
Prefabricated floor slab shape design method and system, storage medium and electronic equipment
CN119989491A
Prefabricated assembly floor slab type design method, system, storage medium and electronic equipment
CN119989491B
