Digital image airship skin measurement method based on image pyramid and rectangular sub-area

Abstract

The present invention provides a digital image airship skin measurement method based on an image pyramid and rectangular subregions. The method comprises the following steps: obtaining a reference image and a target image; calculating, based on the reference image and the target image, integer pixel displacement values ​​associated with the digital image using an image pyramid and a cross-correlation method; and calculating sub-pixel deformation parameter values ​​associated with the digital image based on the integer pixel displacement values ​​and a pair of rectangular subregions. The digital image airship skin measurement method based on an image pyramid and rectangular subregions provided by the present invention can measure airship skins, improves measurement accuracy and speed, and enhances the reliability of real-time airship skin measurement.

Description

Technical Field

[0001] The present invention relates to the technical field of airship skin detection, in particular to a digital image airship skin measurement method based on image pyramid and rectangular sub-areas. Background Art

[0002] Near-space refers to the area between 20 and 100 kilometers above the Earth's surface, lying between the flight altitudes of conventional aircraft and the orbital altitudes of satellites. Compared to other aircraft, developing aircraft in this promising area holds significant military and civilian applications, potentially enabling long-term, fixed-point surveys, environmental monitoring, and emergency rescue operations. Stratospheric airships are a common type of near-space vehicle, effectively filling the gap between the flight altitudes of aircraft and spacecraft. They rely primarily on air buoyancy to hover at a designated altitude and maintain a fixed station.

[0003] However, the development of near-space vehicles presents numerous difficulties and challenges. Due to the unique high-altitude environment, vehicles require sufficient aerodynamic lift and environmental adaptability. When an airship hovers for extended periods, solar radiation causes internal temperature fluctuations, which in turn causes the gas inside to expand and contract. Under the influence of gravity and fluctuating internal and external air pressures, the envelope material is susceptible to significant deformation. Material failure can lead to an airship accident. Therefore, monitoring skin material deformation is crucial to extending the life of the airship and minimizing losses.

[0004] Existing technologies often use sensors to monitor skin material deformation, primarily through resistance strain gauges. While these offer high accuracy and mature technology, they require specialized circuitry and are susceptible to environmental influences. Furthermore, the airship's surface is flexible, so direct contact with the strain gauges inevitably creates an impact. If the strain gauges fall off or the contact is uneven, the measurement results will be erroneous. Therefore, a digital image-based airship skin measurement method based on image pyramids and rectangular subregions is highly desirable. Summary of the Invention

[0005] The purpose of the present invention is to provide a digital image airship skin measurement method based on image pyramid and rectangular sub-area, which can realize the measurement of airship skin, improve the accuracy and speed of measurement, and enhance the reliability of real-time measurement of airship skin.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A digital image airship skin measurement method based on image pyramid and rectangular sub-areas includes the following steps:

[0008] Step 1: Obtain reference image and target image;

[0009] Step 2: Based on the reference image and the target image, the digital image related integer pixel displacement value is calculated based on the image pyramid and cross-correlation method;

[0010] Step 3: Calculate the sub-pixel deformation parameter value related to the digital image based on the integer pixel displacement value and a pair of rectangular sub-regions.

[0011] Optionally, in step 1, the digital image-related integer pixel displacement values ​​are calculated based on the reference image and the target image using an image pyramid and a cross-correlation method, specifically:

[0012] Construct an image pyramid with the same number of layers for the reference image and the target image, and perform fast Fourier correlation calculation on the image with the lowest resolution between the reference image and the target image. The value with the strongest correlation obtains the integer pixel displacement (u1, v1), which is the displacement of the center point of the entire image. The calculated pixel displacement (u1, v1) is used as the initial value of each calculation point searched in the next layer. When calculating the next layer, the calculation point coordinates and the corresponding displacement are doubled. In the nkth layer, a square area with the calculation point coordinate as the center and a side length of (k+1)×2r+1 is selected on the reference image. According to the displacement value of the previous layer, the calculation point coordinates in the target image are determined. A square area with a side length of (k+1)×2r+1 is also taken on the target image for cross-correlation. By recalculating the cross-correlation of each layer, it is used to eliminate the error of the previous transmission, and the displacement is gradually transmitted to the next layer until the last layer. The displacement result is:

[0013]

[0014] Where u and v are the displacements of a calculation point on the image caused by material deformation, i.e., the displacement of whole pixels. u is in the horizontal direction, v is in the vertical direction, and u1,...,u is n and v1,...,v n is the integer pixel displacement at each level of the image pyramid.

[0015] Optionally, in step 3, a sub-pixel deformation parameter value related to the digital image is calculated based on the integer pixel displacement value and a pair of rectangular sub-regions, specifically:

[0016] Obtain the target image, divide it based on a pair of rectangular sub-areas, and use the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2) for calculation, where W1 and H2 are the number of pixels in the horizontal direction, H1 and W2 are the number of pixels in the vertical direction, and W1=W2, H1=H2, obtain the integer pixel displacement, and calculate based on the integer pixel displacement by minimizing the ZNSSD criterion, and use the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2) to obtain the incremental deformation parameter Δp1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter Δp2 of the rectangular sub-area H2×W2 (W2>H2), and according to the inverse incremental deformation function W -1 (ξ,Δp) and the deformation function W(ξ,p) respectively obtain the deformation parameter p1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter p2 of the rectangular sub-area H2×W2 (W2>H2), which are:

[0017] p1=[u1,u x1 ,u y1 ,v1,v x1 ,v y1 ] T

[0018] p2=[u2,u x2 ,u y2 ,v2,v x2 ,v y2 ] T

[0019] The deformation parameter p of the target image is calculated based on the deformation parameter p1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter p2 of the rectangular sub-area H2×W2 (W2>H2):

[0020]

[0021] According to the deformation parameter p, the incremental deformation parameter Δp is obtained as:

[0022]

[0023] Where, and are the average values ​​of u1, u2 and v1, v2 respectively, u x1 、u y1 、v x1 and v y1 represents the gradient of the horizontal and vertical displacement of the rectangular sub-area W1×H1 (W1>H1), u x2 、u y2 、v x2 and vy2 Represents the gradient of the horizontal and vertical displacement of the rectangular sub-area H2×W2 (W2>H2), Δu x1 , Δv x1 , Δu y2 and Δv y2 u x1 、v x1 、u y2 and v y2 The change in the deformation parameter p and the incremental deformation parameter Δp is used to determine whether they meet the convergence condition. If so, the deformation parameter p is output. Otherwise, the calculation is continued for the two rectangular sub-areas until the convergence condition is met. The convergence condition is:

[0024]

[0025] Where Δx′ is the horizontal offset of all relative calculation points in the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2), and Δy′ is the horizontal offset of all relative calculation points in the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2). 2× The vertical offsets of all relative calculation points in the rectangular sub-area W2 (W2>H2). In the initial calculation, the displacements u1, u2, v1 and v2 are all derived from integer pixel displacements, and the other parameters are 0.

[0026] According to a specific embodiment provided by the present invention, the present invention discloses the following technical effects: the present invention provides a digital image airship skin measurement method based on an image pyramid and rectangular sub-regions, the method comprising obtaining a reference image and a target image, calculating, based on the reference image and the target image, integer pixel displacement values ​​related to the digital image using an image pyramid and a cross-correlation method, and calculating sub-pixel deformation parameter values ​​related to the digital image based on the integer pixel displacement values ​​and a pair of rectangular sub-regions; the method uses an integer pixel method for estimation, which can achieve accurate integer pixel estimation, while also accelerating the efficiency of sub-pixel matching and avoiding the failure of the sub-pixel search method under large deformations; the integer pixel method uses an image pyramid for layer-by-layer transmission, with the top layer of the pyramid performing calculations on the entire image, capable of measuring large deformations; based on fast Fourier cross-correlation calculations, it can achieve good calculation results and fast calculation advantages under light changes and noise interference; sub-pixel calculation is used to obtain higher measurement accuracy, and uses a pair of rectangular sub-regions. Since each rectangular sub-region has high calculation accuracy in a single direction, high-precision calculation of calculation points is achieved by using a pair of rectangular sub-regions in a complementary manner. At the same time, a pair of sub-regions can be calculated separately before each deformation is merged, which can improve efficiency through parallel calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 This is a flow chart of a digital image airship skin measurement method based on image pyramid and rectangular sub-areas according to an embodiment of the present invention;

[0029] Figure 2 This is a flow chart of calculating the relative whole pixel displacement value of a digital image based on the image pyramid and cross-correlation method;

[0030] Figure 3 Schematic diagram of the process of calculating the sub-pixel deformation parameter value related to a digital image based on the integer pixel displacement value and a pair of rectangular sub-regions. DETAILED DESCRIPTION

[0031] The purpose of the present invention is to provide a digital image airship skin measurement method based on image pyramid and rectangular sub-area, which can realize the measurement of airship skin, improve the accuracy and speed of measurement, and enhance the reliability of real-time measurement of airship skin.

[0032] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] like Figure 1 As shown, the digital image airship skin measurement method based on image pyramid and rectangular sub-areas provided by the embodiment of the present invention includes the following steps:

[0034] Step 1: Obtain reference image and target image;

[0035] Step 2: Based on the reference image and the target image, the digital image related integer pixel displacement value is calculated based on the image pyramid and cross-correlation method;

[0036] Step 3: Calculate the sub-pixel deformation parameter value related to the digital image based on the integer pixel displacement value and a pair of rectangular sub-regions.

[0037] like Figure 2 As shown, in step 1, the digital image-related integer pixel displacement value is calculated based on the reference image and the target image based on the image pyramid and cross-correlation method, specifically:

[0038] Construct an image pyramid with the same number of layers for the reference image and the target image, and perform fast Fourier correlation calculation on the image with the lowest resolution between the reference image and the target image. The value with the strongest correlation obtains the integer pixel displacement (u1, v1), which is the displacement of the center point of the entire image. The calculated pixel displacement (u1, v1) is used as the initial value of each calculation point searched in the next layer. When calculating the next layer, the calculation point coordinates and the corresponding displacement are doubled. In the nkth layer, a square area with the calculation point coordinate as the center and a side length of (k+1)×2r+1 is selected on the reference image. According to the displacement value of the previous layer, the calculation point coordinates in the target image are determined. A square area with a side length of (k+1)×2r+1 is also taken on the target image for cross-correlation. By recalculating the cross-correlation of each layer, it is used to eliminate the error of the previous transmission, and the displacement is gradually transmitted to the next layer until the last layer. The displacement result is:

[0039]

[0040] Where u and v are the displacements of a calculation point on the image caused by material deformation, i.e., the displacement of whole pixels. u is in the horizontal direction, v is in the vertical direction, and u1,...,u is n and v1,...,v n is the integer pixel displacement at each level of the image pyramid.

[0041] Detailed description: First, starting from the nth layer, fast Fourier correlation calculation is performed on the reference image and the image with the lowest resolution in the target image. Among them, the value with the strongest correlation obtains the integer pixel displacement (u1, v1), which is the displacement of the center point of the entire image. The integer pixel displacement (u1, v1) is used as the initial value of each calculation point searched in the next layer. Due to the size change, the calculation point coordinates and the corresponding displacement need to be doubled in the next layer. In the n-1th layer, a square area with the calculation point coordinate as the center and a side length of 4r+1 is selected as the reference image information. According to the displacement value of the previous layer, the coordinates of the calculation point in the target image are determined. On the target image, a square area with a side length of 4r+1 is also taken for cross-correlation to obtain the offset value (u2, v2). In the n-2th layer, a square area with the calculation point coordinate as the center and a side length of 6r+1 is selected in the reference image. According to the displacement value of the previous layer, the coordinates of the calculation point in the target image are determined. On the target image, a square area with a side length of 6r+1 is also taken for cross-correlation to obtain the new offset value ( u3 ,v3), and continue in sequence until reaching the 0th layer, and obtain the final displacement result as shown in the above formula. In addition, when multiple calculation points need to perform integer pixel estimation, the integer pixel displacement obtained by the cross-correlation calculation of the nth layer (u1, v1), which can improve the calculation speed. The integer pixel search method can avoid the failure of the sub-pixel search method under large deformation. At the same time, the accurate integer pixel search method can also speed up the sub-pixel search. Therefore, the displacement result obtained by the integer pixel search is used as the calculation initial value of the sub-pixel matching method.

[0042] like Figure 3 As shown, in step 3, the sub-pixel deformation parameter value related to the digital image is calculated based on the integer pixel displacement value and a pair of rectangular sub-regions, specifically:

[0043] According to the calculation process of the reverse combined Gauss-Newton algorithm, the incremental deformation parameter Δ is calculated in each iteration. p , and then update the incremental deformation function W -1 (ξ,Δ p ), and finally update the deformation function W (ξ, p ) to obtain new deformation parameters again p , the specific steps are as follows:

[0044] Get the target image and divide it into two rectangular sub-areas using W 1× H1(W 1＞ H1) rectangular sub-area and H 2× W2(W 2＞ H2) is calculated. Before the calculation, it is explained that the first-order function of the digital image correlation method can be expressed as:

[0045]

[0046] Among them, x' and y' represent the position of the point after deformation, x and y represent the position of the point before deformation, u and v represent the displacement in the horizontal and vertical directions respectively, u x 、u y 、v x and v y Represents the displacement gradients of u and v in the horizontal and vertical directions respectively, Δx and Δy represent the distances of a point (x, y) in the sub-area from the center point of the sub-area in the horizontal and vertical directions, and the deformation parameters are usually expressed in the form of vectors as p = [u, u x ,u y ,v,v x ,v y ], express the first-order shape function in matrix form:

[0047]

[0048] The deformation function is defined in matrix form as:

[0049]

[0050] where ξ=(Δx,Δy ,1 ) T is the offset value from the center of the sub-area, and the incremental deformation function is defined as:

[0051]

[0052] According to the ICGN method, the incremental deformation function is applied to the reference sub-area. Therefore, the ZNSSD criterion is defined as follows:

[0053]

[0054] and is the average gray value of the sub-region, where

[0055]

[0056]

[0057]

[0058]

[0059] Where N is the number of pixels in the sub-area, for f(W(ξ, p )) Perform a first-order Taylor expansion approximation to obtain:

[0060]

[0061] Where, The gradient of the reference sub-region is obtained by the convolution operator. is the Jacobian matrix, so the ZNSSD criterion is about Δ p Perform partial derivatives to obtain extreme values, and solve the formula using the Gauss-Newton iteration method to obtain the incremental deformation parameters:

[0062]

[0063] Where H is the Hessian matrix of the deformation parameters,

[0064]

[0065] Thus, the reverse incremental deformation function is obtained:

[0066]

[0067] Apply the reverse incremental deformation function to the deformation function for update, that is, W(ξ,p)=W(ξ,p)W -1 (ξ,p);

[0068] W1 and H2 are the number of pixels in the horizontal direction, H1 and W2 are the number of pixels in the vertical direction, and W1=W2, H1=H2, obtain the integer pixel displacement, and according to the integer pixel displacement, use the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2) to calculate respectively, and obtain the incremental deformation parameter Δp1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter Δp2 of the rectangular sub-area H2×W2 (W2>H2), according to the inverse incremental deformation function W -1 (ξ,Δp) and the deformation function W(ξ,p) respectively obtain the deformation parameter p1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter p2 of the rectangular sub-area H2×W2 (W2>H2), which are:

[0069] p1=[u1,u x1 ,u y1 ,v1,v x1 ,v y1 ] T

[0070] p2=[u2,u x2 ,u y2 ,v2,v x2 ,v y2 ] T

[0071] The deformation parameter p of the target image is calculated based on the deformation parameter p1 of the rectangular sub-area W1×H1 (W1>H1) and the incremental deformation parameter p2 of the rectangular sub-area H2×W2 (W2>H2):

[0072]

[0073] According to the deformation parameter p, the incremental deformation parameter Δp is obtained as:

[0074]

[0075] Where, and are the average values ​​of u1, u2 and v1, v2 respectively, u x1 、u y1 、v x1 and v y1 represents the gradient of the horizontal and vertical displacement of the rectangular sub-area W1×H1 (W1>H1), u x2 、u y2 、v x2 and v y2 Represents the gradient of the horizontal and vertical displacement of the rectangular sub-area H2×W2 (W2>H2), Δu x1, Δv x1 , Δu y2 and Δv y2 u x1 、v x1 、u y2 and v y2 The change in the deformation parameter p and the incremental deformation parameter Δp is used to determine whether they meet the convergence condition. If so, the deformation parameter p is output. Otherwise, the calculation is continued for the two rectangular sub-areas until the convergence condition is met. The convergence condition is:

[0076]

[0077] Where Δx′ is the horizontal offset of all relative calculation points in the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2), and Δy′ is the vertical offset of all relative calculation points in the rectangular sub-area W1×H1 (W1>H1) and the rectangular sub-area H2×W2 (W2>H2). During the initial calculation, the displacements u1, u2, v1, and v2 are all derived from integer pixel displacements, and the other parameters are 0. Since the two rectangular sub-areas of a calculation point are not associated before each calculation of the combined deformation, parallel computing can be used to speed up the calculation process.

[0078] The present invention provides a digital image airship skin measurement method based on an image pyramid and rectangular sub-areas. The method comprises obtaining a reference image and a target image, calculating integer pixel displacement values ​​associated with the digital image based on the reference image and the target image using an image pyramid and a cross-correlation method, and calculating sub-pixel deformation parameter values ​​associated with the digital image based on the integer pixel displacement values ​​and a pair of rectangular sub-areas. The method uses an integer pixel method for estimation, which can achieve accurate integer pixel estimation, while also accelerating the efficiency of sub-pixel matching and avoiding the failure of sub-pixel search methods under large deformations. The integer pixel method uses an image pyramid for layer-by-layer transmission, with the top layer of the pyramid performing calculations on the entire image, capable of measuring large deformations. Based on fast Fourier cross-correlation calculations, the method has good calculation effects and the advantages of fast calculations under light changes and noise interference. Sub-pixel calculation is used to obtain higher measurement accuracy. A pair of rectangular sub-areas is used. Since each rectangular sub-area has high calculation accuracy in a single direction, high-precision calculation of calculation points is achieved by utilizing a complementary method of the pair of rectangular sub-areas. Simultaneously, the pair of sub-areas can be calculated separately before each deformation is combined, thereby improving efficiency through parallel calculations.

[0079] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A digital image airship skin measurement method based on image pyramid and rectangular sub-areas, characterized in that: The steps include: Step 1: Obtain reference image and target image; Step 2: Based on the reference image and the target image, the digital image related integer pixel displacement value is calculated based on the image pyramid and cross-correlation method; Step 3: Calculate the sub-pixel deformation parameter value related to the digital image based on the integer pixel displacement value and a pair of rectangular sub-regions; In step 3, the sub-pixel deformation parameter value of the digital image is calculated based on the integer pixel displacement value and a pair of rectangular sub-regions, specifically: Obtain the target image, divide it based on a pair of rectangular sub-areas, and use the rectangular sub-area W1×H1 and the rectangular sub-area H2×W2 for calculation, where W1 and H2 are the number of pixels in the horizontal direction, H1 and W2 are the number of pixels in the vertical direction, and W1=W2, H1=H2, W1>H1, W2>H2, obtain the integer pixel displacement, and calculate based on the integer pixel displacement by minimizing the ZNSSD criterion, using the rectangular sub-area W1×H1 and the rectangular sub-area H2×W2 to obtain the incremental deformation parameter Δp1 of the rectangular sub-area W1×H1 and the incremental deformation parameter Δp2 of the rectangular sub-area H2×W2, according to the inverse incremental deformation function W -1 (ξ,Δp) and the deformation function W(ξ,p) respectively obtain the deformation parameter p1 of the rectangular sub-area W1×H1 and the incremental deformation parameter p2 of the rectangular sub-area H2×W2, which are: p1=[u1,u x1 ,u y1 ,v1,v x1 ,v y1 ] T p2=[u2,u x2 ,u y2 ,v2,v x2 ,v y2 ] T The deformation parameter p of the target image is calculated based on the deformation parameter p1 of the rectangular sub-area W1×H1 and the incremental deformation parameter p2 of the rectangular sub-area H2×W2: According to the deformation parameter p, the incremental deformation parameter Δp is obtained as: Where, and are the average values ​​of u1, u2 and v1, v2 respectively, u x1 、u y1 、v x1 and v y1 Represents the gradient of the horizontal and vertical displacement of the rectangular sub-area W1×H1, u x2 、u y2 、v x2 and v y2 Represents the gradient of the horizontal and vertical displacement of the H2×W2 rectangular sub-area, Δu x1 , Δv x1 , Δu y2 and Δv y2 u x1 、v x1 、u y2 and v y2 The change in the deformation parameter p and the incremental deformation parameter Δp is used to determine whether they meet the convergence condition. If so, the deformation parameter p is output. Otherwise, the calculation is continued for the two rectangular sub-areas until the convergence condition is met. The convergence condition is: Where Δx′ is the horizontal offset of all relative calculation points in the rectangular sub-area of ​​W1×H1 and the rectangular sub-area of ​​H2×W2, and Δy′ is the vertical offset of all relative calculation points in the rectangular sub-area of ​​W1×H1 and the rectangular sub-area of ​​H2×W2. During the initial calculation, the displacements u1, u2, v1, and v2 are all derived from integer pixel displacements, and the other parameters are 0.

2. The digital image airship skin measurement method based on image pyramid and rectangular sub-areas according to claim 1, characterized in that: In step 1, the digital image-related integer pixel displacement values ​​are calculated based on the reference image and the target image using the image pyramid and cross-correlation method, specifically: Construct an image pyramid with the same number of layers for the reference image and the target image, and perform fast Fourier correlation calculation on the image with the lowest resolution between the reference image and the target image. The value with the strongest correlation obtains the integer pixel displacement (u1, v1), which is the displacement of the center point of the entire image. The calculated pixel displacement (u1, v1) is used as the initial value of each calculation point searched in the next layer. When calculating the next layer, the calculation point coordinates and the corresponding displacement are doubled. In the nkth layer, a square area with the calculation point coordinate as the center and a side length of (k+1)×2r+1 is selected on the reference image. According to the displacement value of the previous layer, the calculation point coordinates in the target image are determined. A square area with a side length of (k+1)×2r+1 is also taken on the target image for cross-correlation. By recalculating the cross-correlation of each layer, it is used to eliminate the error of the previous transmission, and the displacement is gradually transmitted to the next layer until the last layer. The displacement result is: Where u and v are the displacements of a calculation point on the image caused by material deformation, i.e., the displacement of whole pixels. u is in the horizontal direction, v is in the vertical direction, and u1,...,u is n and v1,...,v n is the integer pixel displacement at each level of the image pyramid.

Images