Online measurement method and device for icing ice shape in cloud and mist field based on binocular vision

Abstract

The invention belongs to the technical field of aerospace optical measurement, and particularly relates to a cloud field icing ice shape online measurement method and device based on binocular vision, and the method comprises the following steps: enabling a dark field illumination system to face a measured ice surface in a cloud field environment, and enabling the dark field illumination system to face the measured ice surface; the dark field illumination system projects dark field illumination light to the detected ice surface; a binocular camera is adopted to continuously collect icing process images of the detected ice surface projected with the dark field illumination light, and a real-time icing image pair sequence of the detected ice surface is obtained; extracting and matching surface feature points of the icing image pair sequence of the real-time detected ice surface to obtain matched surface feature points, and obtaining external parameters of the binocular camera based on the matched surface feature points; and based on a binocular vision principle, performing three-dimensional reconstruction on the real-time measured ice surface according to the matched surface feature points, the external parameters of the binocular camera and the internal parameters of the binocular camera, and realizing three-dimensional measurement of the real-time measured ice surface.

Description

Technical Field

[0001] The invention relates to the technical field of aerospace optical measurement, and in particular to an online measurement method and device for icing and ice shape in a cloud and fog field based on binocular vision. Background Art

[0002] Due to the immaturity of early ice measurement technology and the limitations of methods, as well as the low demand for experimental ice shape data, most ice measurements were made using simple direct contact measurement methods, including the hot knife method and the casting method. With the deepening of ice research, the real-time growth process of ice has attracted much attention. Optical methods have been widely used due to their real-time, high-precision and non-contact advantages.

[0003] There are three commonly used ice type measurement techniques: the first is to directly photograph the ice type and qualitatively analyze the shape and structure of the ice. This method not only lacks quantitative data, but is also highly subjective. The second is to measure the thickness of the ice layer by manual tracing. This method obtains ice type data by cutting off a small piece of ice and tracing the outline of the ice shape on a cardboard template cut into the airfoil profile with a pencil, and then uses a handheld digitizer to convert the pencil traced content point by point into digital coordinates to achieve ice type measurement. Although more advanced digital scanners with digital software have been developed, it is still difficult to change its shortcomings such as interrupting the test, destroying the ice type, and large random measurement errors. The third method is to use binocular vision to achieve ice type measurement based on the principles of laser positioning and multi-angle imaging. However, most of this method is still limited to two-dimensional ice thickness measurement. In the study of three-dimensional measurement, it is only limited to the measurement of ice type of regular-shaped clear ice. It is mainly through single laser line irradiation to verify the feasibility of laser line scanning, lacking follow-up work and practicality.

[0004] In the prior art, the visible laser line scanning method, infrared laser point scanning method, infrared laser line scanning method and optical flow method can be used to reconstruct the ice type in three dimensions. However, in the visible laser line scanning method, the ice layer needs to be frosted during the visible laser line reconstruction process, which greatly affects the continuity and accuracy of the measurement; in the infrared laser point scanning method and infrared laser line scanning method, the infrared laser scanning process is limited in two aspects. On the one hand, it is limited by the low resolution of the infrared camera and cannot capture the ice type characteristics well; on the other hand, it has high requirements for the reflection characteristics of the ice water environment and is greatly affected by the environment, and cannot be used in complex conditions such as cloud and fog fields. At the same time, the existing measurement method cannot continuously and non-contactly measure the three-dimensional shape of ice as it grows in the cloud and fog field of the icing wind tunnel. It is greatly disturbed by the background such as the cloud and fog field during the measurement process, and the ice needs to be surface treated during laser scanning measurement. Summary of the invention

[0005] The purpose of the present invention is to provide an online measurement method for ice shape in a cloud and fog field based on binocular vision, so as to solve the problem in the prior art that the binocular vision method cannot continuously measure the three-dimensional shape of ice as it grows in a cloud and fog field.

[0006] In order to solve the above problems, the present invention proposes an online measurement method for ice shape in a cloud and fog field based on binocular vision. The technical solution adopted is: An online measurement method for ice shape in a cloud and fog field based on binocular vision comprises the following steps: S1, in a foggy environment, arrange the dark field lighting system facing the ice surface to be measured, and project dark field lighting light onto the ice surface to be measured; S2, using a binocular camera to continuously collect images of the freezing process of the ice surface under test projected with dark field illumination light, to obtain a real-time freezing image pair sequence of the ice surface under test; S3, extracting and matching surface feature points of a sequence of ice images of the real-time ice surface to be measured, obtaining matched surface feature points, and obtaining external parameters of the binocular camera based on the matched surface feature points; S4, based on the binocular vision principle, performs three-dimensional reconstruction of the real-time measured ice surface according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera, thereby realizing real-time three-dimensional measurement of the measured ice surface.

[0007] Furthermore, between S2 and S3, it also includes: Performing image correction preprocessing on the real-time icing image pair sequence of the measured ice surface to obtain a corrected real-time icing image pair sequence of the measured ice surface; the image correction preprocessing includes distortion correction; In S3, the surface feature points of the real-time ice image sequence of the measured ice surface are extracted and matched to obtain the matched surface feature points, which are specifically: Surface feature points are extracted and matched on the corrected real-time ice image of the measured ice surface to obtain matched surface feature points.

[0008] Furthermore, the extraction and matching of surface feature points of the corrected real-time ice image sequence of the measured ice surface to obtain the matched surface feature points specifically includes: The SIFT multi-scale feature extraction algorithm is used to extract the surface feature points of the sequence of the corrected real-time ice image of the measured ice surface. The surface feature points are matched by the KNN feature point matching method and the random sampling consistency algorithm to obtain the matched surface feature points.

[0009] Furthermore, the SIFT multi-scale feature extraction algorithm includes: Firstly, a nonlinear gamma transform is performed on the corrected real-time ice image pair sequence of the measured ice surface to obtain an enhanced real-time ice image pair sequence of the measured ice surface; then, a Gaussian difference pyramid is constructed based on the enhanced real-time ice image pair sequence of the measured ice surface to obtain a Gaussian difference image, and the scale space extreme value of each pixel point in the Gaussian difference image is detected to obtain the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface; finally, a feature point descriptor is generated based on the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface to complete the extraction of surface feature points of the enhanced real-time ice image pair sequence of the measured ice surface.

[0010] Further, in S3, the step of acquiring the external parameters of the binocular camera based on the matched surface feature points specifically includes: Based on the matched surface feature points and combined with the internal parameters of the stereo camera, the incremental SFM algorithm is used to determine the external parameters of the stereo camera. Furthermore, the incremental SFM algorithm includes: Based on the matched surface feature points, a sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points is obtained; the sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points is reconstructed in combination with the internal parameters of the binocular camera to obtain the initial external parameters of the binocular camera; the initial external parameters of the binocular camera are iteratively processed and optimized to obtain the external parameters of the binocular camera.

[0011] Furthermore, the iterative processing and optimization processing of the initial external parameters of the binocular camera to obtain the external parameters of the binocular camera includes: iterative processing of the initial external parameters of the binocular camera, and optimization processing using a bundle adjustment method and a random sampling consistency algorithm to obtain the external parameters of the binocular camera.

[0012] Further, in S4, the three-dimensional reconstruction of the measured ice surface based on the binocular vision principle and according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera specifically includes: based on the binocular vision principle, according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera, a stereo matching algorithm is used to construct a local point cloud of the ice image of each measured ice surface, and the spatial relationship between adjacent image pairs is obtained by matching the ice images of the measured ice surface, and a global point cloud of the measured ice surface is established, thereby realizing the three-dimensional reconstruction of the measured ice surface.

[0013] The present invention also provides an online measurement device for executing the above-mentioned online measurement method of ice shape in cloud and fog fields based on binocular vision, comprising: a cloud and fog field simulation system, a dark field lighting system, an image acquisition system and a data processing system, wherein the cloud and fog field simulation system is arranged facing the ice surface to be measured, and is used to simulate the cloud and fog field environment; the dark field lighting system is arranged facing the ice surface to be measured, and is used to project dark field illumination light onto the ice surface to be measured; the image acquisition system comprises a binocular camera, and the binocular camera is arranged on the side of the ice surface to be measured, and is used to continuously acquire images of the freezing process of the ice surface to be measured with dark field illumination light projected, so as to obtain a real-time sequence of freezing image pairs of the ice surface to be measured; the data processing system receives the real-time sequence of freezing image pairs of the ice surface to be measured acquired by the binocular camera, and performs data processing according to steps S3-S4 to obtain three-dimensional measurement of the ice surface to be measured.

[0014] Furthermore, the online measurement device also includes an optical platform, on which a horizontal guide rail is arranged, and the binocular camera moves away from or close to the dark field illumination system via the horizontal guide rail.

[0015] Beneficial effects: The present invention is an improved invention. Aiming at the high transparency of clear ice, the present invention adopts a dark field lighting system to avoid the scattering problem of laser in foggy fields and ice surfaces. The dark field lighting method forms an image with a dark background and clear ice surface features, avoiding overexposure caused by strong reflection of the ice surface caused by bright field shooting, and can obtain more surface features. It solves the problem that the image of the iced wing surface taken by a camera with a small aperture lens under ambient light conditions is dark, resulting in the inability to extract features. In addition, a binocular camera is used to continuously collect images of the icing process of the measured ice surface projected with dark field lighting light in a foggy field environment, and images of the icing process of clear ice, frost ice, and mixed ice in foggy fields are obtained, which has the advantages of non-contact measurement, fast measurement speed, high precision, and large range. At the same time, by reconstructing the real-time three-dimensional image of the ice surface and realizing real-time three-dimensional measurement of the ice surface, the ice thickness and shape information is obtained, and then the real-time ice surface related information of the freezing process of the ice surface is realized, which solves the problem that the binocular vision method in the prior art cannot continuously measure the three-dimensional shape of ice as it grows in a cloud field.

[0016] Between S2 and S3, the method further includes: performing image correction preprocessing on the real-time icing image pair sequence of the measured ice surface to obtain a corrected real-time icing image pair sequence of the measured ice surface; the image correction preprocessing includes distortion correction; In S3, the surface feature points of the real-time ice image sequence of the ice surface to be measured are extracted and matched, and the matched surface feature points include: The surface feature points of the corrected real-time ice image of the measured ice surface are extracted and matched to obtain the matched surface feature points. The image intensity is enhanced through the above image correction preprocessing, which facilitates the collection of clearer feature points.

[0017] The extraction and matching of surface feature points of the corrected real-time ice image sequence of the measured ice surface to obtain matched surface feature points specifically includes: using the SIFT multi-scale feature extraction algorithm to extract the surface feature points of the corrected real-time ice image sequence of the measured ice surface, and then matching the surface feature points by using the KNN feature point matching method and the random sampling consistency algorithm to obtain matched surface feature points, thereby improving the degree of matching of the feature points and thereby improving the accuracy of spatial relationship estimation.

[0018] The SIFT multi-scale feature extraction algorithm includes: first, performing a nonlinear gamma transform on a corrected real-time ice image pair sequence of the measured ice surface to obtain an enhanced real-time ice image pair sequence of the measured ice surface; then, constructing a Gaussian difference pyramid based on the enhanced real-time ice image pair sequence of the measured ice surface to obtain a scale-transformed image pair sequence, and detecting the scale space extreme value of each pixel point in the scale-transformed image pair sequence, thereby obtaining the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface; finally, generating a feature point descriptor based on the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface to complete the extraction of surface feature points of the enhanced real-time ice image pair sequence of the measured ice surface, and through nonlinear gamma transform, improving the contrast of the image, reducing the influence of illumination, making the image easy to observe, and highlighting the feature points that are easy to be ignored.

[0019] In S3, the acquiring of the external parameters of the binocular camera based on the matched surface feature points specifically includes: based on the matched surface feature points, in combination with the internal parameters of the binocular camera, an incremental SFM algorithm is used to determine the external parameters of the binocular camera to improve the accuracy of the three-dimensional reconstruction of ice.

[0020] The incremental SFM algorithm includes: based on matched surface feature points, obtaining a sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points; reconstructing the sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points in combination with internal parameters of the binocular camera to obtain initial external parameters of the binocular camera; iteratively processing and optimizing the initial external parameters of the binocular camera to obtain the external parameters of the binocular camera, thereby improving the accuracy of the external parameters of the binocular camera.

[0021] The initial external parameters of the binocular camera are iteratively processed and optimized to obtain the external parameters of the binocular camera, including: iteratively processing the initial external parameters of the binocular camera, and optimizing them using a bundle adjustment method and a random sampling consistency algorithm to obtain the external parameters of the binocular camera, and using a random sampling consistency algorithm to reduce noise and improve matching reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the spatial coordinates of the characteristic points in the binocular vision principle in the online measurement method for ice shape in a cloud and fog field based on binocular vision of the present invention; Figure 2 It is a schematic diagram of pixel coordinates of feature points in the left and right images in the binocular vision principle in the online measurement method for ice shape in a cloud and fog field based on binocular vision of the present invention; Figure 3 It is a binocular vision principle flow chart of the on-line measurement method of ice shape in a cloud and fog field based on binocular vision of the present invention; Figure 4 It is the original image in the online measurement method of ice shape in a cloud and fog field based on binocular vision of the present invention; Figure 5 It is an image transformed from the original image in the on-line measurement method for ice shape in a cloud and fog field based on binocular vision of the present invention; Figure 6 It is a schematic diagram of the construction principle of the Gaussian difference pyramid in the online measurement method of ice shape in the cloud and fog field based on binocular vision of the present invention; Figure 7 It is a schematic diagram of the position of the extreme point in the online measurement method of ice shape in the cloud and fog field based on binocular vision of the present invention; Figure 8 It is a flow chart of the incremental SFM method in the online measurement method of ice shape in the cloud and fog field based on binocular vision of the present invention; Fig. 9 It is a schematic diagram of reprojection error in the online measurement method of ice shape in a cloud and fog field based on binocular vision of the present invention; Fig.10 It is a structural schematic diagram of an online measuring device in an online measuring device for ice shape in a cloud and fog field based on binocular vision of the present invention; Fig.11 It is a schematic diagram of online measurement in the online measurement device for ice shape in a cloud and fog field based on binocular vision of the present invention; In the figure, 1, camera #1, 2, camera #2, 3, yellow light, 4, ice surface under test, 5, wind tunnel exit, 6, droplet, 7, wing model, 8, flange base, 9, optical platform, 10, electric guide rail. DETAILED DESCRIPTION

[0023] As cited in the background technology, the binocular vision method in the prior art cannot continuously measure the three-dimensional shape of ice as it grows in a foggy field. Therefore, the present invention provides an online ice shape measurement method for foggy fields based on binocular vision, comprising the following steps: First, in a foggy field environment, a dark field lighting system is arranged facing the ice surface to be measured, and the dark field lighting system projects dark field lighting light to the ice surface to be measured, so as to provide a dark field environment and a foggy field environment, thereby avoiding the problem of laser scattering in the foggy field and the ice surface; secondly, a binocular camera is used to continuously collect images of the freezing process of the ice surface to be measured projected with dark field lighting light, so as to obtain a real-time sequence of ice image pairs of the ice surface to be measured, and obtain the bright ice, frost ice, and mixed ice in the foggy field. Ice formation process image; then, the surface feature points of the real-time ice surface to be measured are extracted and matched for the sequence of ice images to obtain the matched surface feature points, and the external parameters of the binocular camera are obtained based on the matched surface feature points to obtain the three-dimensional coordinates of the surface feature points; finally, based on the principle of binocular vision, the real-time ice surface to be measured is reconstructed in three dimensions according to the matched surface feature points, the external parameters of the binocular camera and the internal parameters of the binocular camera, so as to realize the real-time three-dimensional measurement of the ice surface to be measured, and realize the non-contact continuous measurement of the three-dimensional shape of ice as it grows in the cloud and fog field. Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] The following describes the online measurement method and device for ice shape in a cloud and fog field based on binocular vision according to an embodiment of the present application with reference to the accompanying drawings.

[0026] Combine the following Figure 1 , the online measurement method of ice shape in cloud and fog fields based on binocular vision provided by this application is described in detail.

[0027] See also Figure 1 The present invention provides an online measurement method for ice shape in a foggy field based on binocular vision, comprising the following steps: first, in a foggy field environment, a dark field lighting system is arranged facing the ice surface to be measured, and the dark field lighting system projects dark field lighting light to the ice surface to be measured; second, a binocular camera is used to continuously collect images of the freezing process of the ice surface to be measured projected with dark field lighting light, and a real-time freezing image pair sequence of the ice surface to be measured is obtained; then, surface feature points of the real-time freezing image pair sequence of the ice surface to be measured are extracted and matched, and matched surface feature points are obtained, and binocular camera external parameters are obtained based on the matched surface feature points; finally, based on the principle of binocular vision, the real-time ice surface to be measured is reconstructed in three dimensions according to the matched surface feature points, binocular camera external parameters, and binocular camera internal parameters, so as to realize real-time three-dimensional measurement of the ice surface to be measured. Specifically, in view of the occlusion phenomenon in foggy fields, the dark field lighting system uses yellow light illumination. Since yellow light has a shorter wavelength, it can reduce the diffraction of light in clouds and fog, allowing more light to pass through the air with more droplets. At the same time, since the scattering intensity of yellow light is five times that of red light, the yellow light can evenly illuminate the ice surface after passing through the scattering lampshade on the head of the fill light and the cloud field.

[0028] See also Figure 1 and Figure 2 The online measurement method of ice shape in a cloud and fog field based on binocular vision of the present invention is based on binocular vision method. The basic principle of binocular vision method is that when an object is imaged on a photosensitive element, it is converted from three-dimensional to two-dimensional, so its depth information in space is lost, and the binocular vision method can restore the depth information of the object to achieve the positioning and measurement of the object. Based on the parallax principle, the difference between the pixel coordinate values ​​of the left and right images of a point in space is used to calculate the spatial coordinates of the feature point through the triangulation principle, wherein the left and right images are the images corresponding to the left and right cameras.

[0029] Specifically: Assume there is a feature point in the space , the pixel coordinates of the spatial feature point in the left and right images are , , the baseline B is the origin of the coordinate system of the left and right cameras The distance between the two triangles can be known from the geometric relationship 1 of similar triangles: Formula 1 In formula 1, The focal length of the left and right cameras are the spatial coordinate values ​​of the spatial feature point P; B is the origin of the coordinate system of the left and right cameras distance.

[0030] Then the three-dimensional coordinates of the spatial feature point P are shown in Formula 2: Formula 2 In formula 2, B is the origin of the coordinate system of the left and right cameras The distance The focal length of the left and right cameras is the pixel point of the spatial feature point P in the left image The horizontal axis of is the pixel point of the spatial feature point P in the left image The ordinate of , D is the disparity of the spatial feature point P in the left and right images, as shown in Formula 3: Formula 3 In formula 3, is the pixel point of the spatial feature point P in the left image The horizontal axis of is the pixel point of the spatial feature point P in the right image The vertical coordinate of .

[0031] From the above formula, we can know that as long as we know the internal and external parameters of the binocular camera, we can find the three-dimensional coordinates of the spatial feature point P. If we can obtain the three-dimensional coordinates of multiple points in space, we can reconstruct the original three-dimensional shape of the object. Among them, the internal parameters of the binocular camera are mainly determined by the binocular camera's own chip, lens focal length, etc. Therefore, the technology of three-dimensional reconstruction of the ice surface mainly lies in the selection of the spatial feature point P in the image taken by the binocular camera and the accurate acquisition of the camera's external parameters.

[0032] In a specific embodiment of the present application, between step 2 and step 3, the binocular vision-based online measurement method for ice and ice shapes in a cloud and fog field further includes: performing image correction preprocessing on a real-time ice image pair sequence of the measured ice surface to obtain a corrected real-time ice image pair sequence of the measured ice surface; the image correction preprocessing includes distortion correction. At this time, the surface feature points of the real-time ice image pair sequence of the measured ice surface are extracted and matched to obtain matched surface feature points, specifically: the surface feature points of the corrected real-time ice image of the measured ice surface are extracted and matched to obtain matched surface feature points.

[0033] Here, the surface feature points of the corrected real-time ice image pair sequence of the measured ice surface are extracted and matched to obtain the matched surface feature points, specifically including: using the scale invariant feature transform (SIFT) multi-scale feature extraction algorithm to extract the surface feature points of the corrected real-time ice image pair sequence of the measured ice surface, and then matching the surface feature points through the K-Nearest Neighbor (KNN) feature point matching method and the random sampling consistency algorithm to obtain the matched surface feature points.

[0034] Specifically, Figure 3 As shown, a binocular camera is used to capture left camera images and right camera images to obtain an image pair sequence, that is, a real-time icing image pair sequence of the measured ice surface; secondly, the real-time icing image pair sequence of the measured ice surface is input, and the real-time icing image pair sequence of the measured ice surface is distorted and corrected to obtain a corrected real-time icing image pair sequence of the measured ice surface; then, the SIFT multi-scale feature extraction algorithm is used to extract surface feature points of the corrected real-time icing image pair sequence of the measured ice surface, and the KNN feature point matching method is used in combination with the random sampling consistency algorithm to improve the accuracy of spatial relationship estimation, complete the matching of surface feature points, and obtain matched surface feature points.

[0035] In a specific embodiment of the present application, the SIFT multi-scale feature extraction algorithm includes: first, performing a nonlinear gamma transform on a corrected real-time ice image pair sequence of the measured ice surface to obtain an enhanced real-time ice image pair sequence of the measured ice surface; then, constructing a Gaussian difference pyramid based on the enhanced real-time ice image pair sequence of the measured ice surface to obtain a scale-transformed image pair sequence, and detecting the scale space extreme value of each pixel point in the scale-transformed image pair sequence to obtain the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface; finally, generating a feature point descriptor based on the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface to complete the extraction of surface feature points of the enhanced real-time ice image pair sequence of the measured ice surface.

[0036] The ice images collected from different viewing angles and times have texture, projection, luminosity and nonlinear intensity changes due to the influence of cloud and fog occlusion and camera placement, and some prominent ice features in the image will cause light to be directly reflected into the camera due to the angle relationship, resulting in overexposure. It will be difficult to match multiple images by directly obtaining feature points based on the grayscale extreme values ​​and gradient relationships of the image. The traditional SIFT multi-scale feature extraction algorithm needs to downsample and Gaussian blur the image when extracting extreme points during scale transformation, which will make it difficult to extract feature points from the overexposed part due to similar brightness. Therefore, the present application improves the SIFT multi-scale feature extraction algorithm, that is, performs a nonlinear gamma transform on the ice image, expands the high grayscale level of the image, compresses the low grayscale level, improves the contrast of the image, and reduces the impact of light, making the image easy to observe and highlighting feature points that are easy to overlook.

[0037] Specifically, Figure 3 , 4 As shown in , 5, 6, and 7, the specific process of extracting and matching the surface feature points of the real-time ice image sequence of the measured ice surface to obtain the matched surface feature points is as follows: First, the corrected real-time ice image pair sequence of the measured ice surface is subjected to nonlinear gamma transformation to obtain an enhanced real-time ice image pair sequence of the measured ice surface. That is, in order to highlight the features of the brighter part of the ice image, improve the accuracy of subsequent processing, make the image easy to observe, and highlight the feature points that are easy to be ignored, a nonlinear gamma transformation is used, wherein the nonlinear gamma transformation is shown in Formula 4: Formula 4 In formula 4, The real-time ice image of the ice surface to be measured for correction is compared with the images in the sequence. The gray value at the pixel, To enhance the real-time ice image of the measured ice surface, the transformed sequence The gray value at the pixel, is the compensation coefficient, which is used to ensure that The grayscale value at the pixel is between 0-255. is the gamma value, which is used to control the image contrast gain. When the grayscale is low, the image will be stretched and the grayscale will be compressed. When , the opposite is true.

[0038] In this application, when the distortion correction process is performed on the real-time ice image sequence of the measured ice surface, since the ice surface reflects light strongly and the image brightness is large, the present application takes When the value is 2, the higher grayscale or brighter parts of the image are stretched, which enhances the details of the bright parts of the image and helps the feature extraction operation.

[0039] Secondly, scale space extreme value detection: In order to detect extreme points in images at different scales to find potential key points, it is necessary to construct a Gaussian difference pyramid (Difference Of Gaussian, DOG), specifically: (1) The enhanced real-time ice image sequence of the measured ice surface is convolved with Gaussian kernels of different scales to construct a Gaussian blurred image sequence, where the Gaussian convolution kernel is shown in Formula 5: Formula 5 In formula 5, is the circumference of a circle, is the standard deviation of the Gaussian distribution, x and y are the plane coordinates of any point in the enhanced real-time ice image of the measured ice surface, and e is a natural constant; The enhanced real-time ice image of the measured ice surface is convolved with the Gaussian kernel to obtain a Gaussian blurred image as shown in Formula 6: Formula 6 In formula 6, To enhance the real-time ice image of the measured ice surface in coordinates The gray value at the pixel point is is the Gaussian convolution kernel.

[0040] Here, the adjacent layers The first image of each layer is obtained by downsampling the third to last image of the previous layer, thus obtaining a multi-scale Gaussian pyramid.

[0041] (2) Calculate the difference image between adjacent scales to form a Gaussian difference pyramid and obtain a Gaussian difference image, where the Gaussian difference image is shown in Formula 7: Formula 7 In formula 7, is the Gaussian difference image, ) is the standard deviation of the Gaussian distribution. The Gaussian convolution kernel, The standard deviation of the Gaussian distribution is The Gaussian convolution kernel, To enhance the real-time ice image of the measured ice surface in coordinates The gray value at the pixel point, ) is the standard deviation of the Gaussian distribution. Gaussian blurred image.

[0042] In the Gaussian difference pyramid, the extreme points of each pixel in the Gaussian difference image in its spatial and scale neighborhood are detected, that is, the extreme points of the pixels in the neighborhood are detected. All points are big or all are small.

[0043] (3) Perform three-dimensional quadratic fitting on the extreme points in the Gaussian difference image to obtain the positions of the extreme points in the enhanced real-time ice image sequence of the measured ice surface. At the same time, filter out low-contrast feature points and unstable edge response points to enhance matching stability and improve noise resistance.

[0044] Then, based on the positions of the extreme points in the enhanced real-time ice image sequence of the measured ice surface, feature point descriptors are generated to complete the extraction of surface feature points of the enhanced real-time ice image sequence of the measured ice surface. In order to achieve rotation invariance, one or more directions need to be assigned to each key point, specifically: (a) In the neighborhood of the key point, the gradient magnitude and direction of each pixel are calculated and these direction values ​​are filled into the histogram; (b) Select the main peak direction in the histogram as the main direction of the key point; if the difference between other peak directions and the main peak is within a certain range, they can also be used as additional directions; (c) In the neighborhood of the key point, the gradient histograms of multiple regions are calculated, and these histograms are combined into a feature vector. Usually, the neighborhood of the key point is divided into 4×4 sub-regions, and the gradient histograms in 8 directions are calculated for each sub-region, thereby forming a 128-dimensional feature vector, which is the feature vector of the key point, that is, a 128-dimensional feature descriptor. Since the feature vector mainly depends on the gradient of each point in the local area, under the influence of light and fog, although the grayscale size of the image as a whole changes, it has little effect on the local gradient, and has the characteristic of illumination invariance.

[0045] Finally, the surface feature points are matched by the KNN feature point matching method and the random sampling consistency algorithm. In the feature matching, the obtained 128-dimensional feature descriptor is used to measure the similarity of the feature point pairs and generate matching point pairs. The screening criterion for feature point matching is the Euclidean geometric distance between feature points, and the calculation expression is shown in Formula 8: Formula 8 In formula 8, and are any two feature points in any two images in the sequence of enhanced real-time ice images of the measured ice surface, and The enhanced real-time ice image of the ice surface being measured is respectively any two feature points in any two images in the sequence and feature descriptor, n is 128.

[0046] Select the two feature points with the smallest Euclidean distance, calculate the ratio r of the two smallest Euclidean distances, and compare it with the specific threshold T, as shown in Formula 9: Formula 9 In formula 9, and They are the Euclidean geometric distances of the nearest neighbor and the next nearest neighbor of the spatial feature point, respectively. If the ratio r is less than a specific threshold T (usually between 0.4 and 0.6), the two points are considered to be matching points, otherwise they are eliminated.

[0047] In a specific embodiment of the present application, obtaining the external parameters of the binocular camera based on the matched surface feature points specifically includes: based on the matched surface feature points, combined with the internal parameters of the binocular camera, using the incremental motion recovery structure (Structure From Motion, SFM) algorithm to determine the external parameters of the binocular camera. Among them, the incremental SFM algorithm includes: based on the matched surface feature points, obtaining a real-time sequence of ice image pairs of the measured ice surface with the largest number of matching surface feature points; combining the internal parameters of the binocular camera to reconstruct the real-time sequence of ice image pairs of the measured ice surface with the largest number of matching surface feature points to obtain the initial external parameters of the binocular camera; iteratively processing and optimizing the initial external parameters of the binocular camera, thereby obtaining the external parameters of the binocular camera. Here, iteratively processing and optimizing the initial external parameters of the binocular camera to obtain the external parameters of the binocular camera includes: iteratively processing the initial external parameters of the binocular camera, and optimizing them using the bundle adjustment method and the random sampling consistency algorithm at the same time, thereby obtaining the external parameters of the binocular camera.

[0048] Specifically, see Figure 8 and Fig. 9 The present invention uses the incremental SFM method to extract the external parameters of the binocular camera, mainly using multiple images from different perspectives to infer the three-dimensional structure of the object and the motion trajectory of the camera. The external parameters of the binocular camera mainly include the 3*3 rotation matrix R and the 1*3 translation vector T. The internal parameters of the binocular camera include the focal length f and two radial distortion parameters. and The geometric scene provides each trajectory point X in the trajectory j , through the projection equation, the trajectory point X j Projected onto the two-dimensional plane image of the camera, the projection point is obtained, and the reprojection error is the distance between the projection point and the real point on the two-dimensional plane image of the camera. For n viewing angles and m trajectories, the target optimization equation of the reprojection error is shown in Equation 10: Formula 10 In formula 10, is the cumulative sum of the projection errors of the binocular camera trajectory when observing trajectory j at the corresponding viewing angle i. Take 1, otherwise take 0, For perspective, For the trajectory, To project to Camera image The coordinates of the point, The pixel coordinates of the point; where, Determined based on the camera's internal parameters; The camera image refers to the image captured by multiple view cameras at different viewpoints during the real-time movement of the left and right cameras.

[0049] Specifically, an image pair in a sequence of icing images of the measured ice surface with the largest number of matching surface feature points is selected as an initial image pair, the cumulative sum of projection errors of the binocular camera trajectory observed at the corresponding viewing angle i is calculated, and based on the internal parameters of the binocular camera, the cumulative sum of projection errors is optimized by using a bundle adjustment method to obtain the initial external parameters of the binocular camera, that is, the binocular camera parameters are estimated; then, based on the initial external parameters of the binocular camera, a new sequence of icing images of the measured ice surface with the largest number of matching surface feature points is added, and the sequence of icing images of the measured ice surface with the largest number of matching surface feature points is iteratively processed, that is, the optimal camera and scene parameters are obtained by combining the bundle adjustment method, that is, the external parameters are optimized, and at the same time, a random sampling consistency algorithm is used to reduce noise and improve the reliability of matching, so as to obtain the external parameters of the binocular camera.

[0050] In a specific embodiment of the present application, Figure 3 As shown, based on the principle of binocular vision, the three-dimensional reconstruction of the measured ice surface is carried out according to the matched surface feature points, the external parameters of the binocular camera and the internal parameters of the binocular camera. Specifically, the following steps are performed: based on the principle of binocular vision, according to the matched surface feature points, the external parameters of the binocular camera and the internal parameters of the binocular camera, a stereo matching algorithm is used to construct a local point cloud of the ice image of each measured ice surface, and the spatial relationship between adjacent image pairs is obtained by matching the ice images of the measured ice surface, and a global point cloud of the measured ice surface is established, thereby realizing the three-dimensional reconstruction of the measured ice surface.

[0051] Specifically, according to the basic principles of the binocular vision method, based on the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera, the spatial relationship between adjacent image pairs is obtained through image matching, that is, the three-dimensional coordinates of the feature points are obtained by using the stereo matching algorithm, thereby obtaining a sparse point cloud; then a dense point cloud can be obtained by obtaining a depth map; after removing the noise of the point cloud data that is far away from the surface or not on the surface in the dense point cloud, a triangulated network is constructed between adjacent points based on the point cloud data, that is, a global point cloud of the measured ice surface is established, and then the measured ice surface is reconstructed in three dimensions; finally, a more complete three-dimensional model of the ice surface can be obtained through tip smoothing and texture mapping.

[0052] The present application also provides an online measurement device for executing the above-mentioned online measurement method for ice shape in a cloud and fog field based on binocular vision, such as Fig.10 and Fig.11As shown, it includes a cloud and fog field simulation system, a dark field lighting system, an image acquisition system and a data processing system. The cloud and fog field simulation system is arranged facing the ice surface 4 to be measured, and is used to simulate the cloud and fog field environment; the dark field lighting system is arranged facing the ice surface 4 to be measured, and is used to project the dark field lighting light to the ice surface 4 to be measured; the image acquisition system includes a binocular camera, and the binocular camera system is arranged on the side of the ice surface 4 to be measured, and is used to continuously collect the freezing process images of the ice surface 4 to be measured with the dark field lighting light projected, and obtain the real-time freezing image pair sequence of the ice surface 4 to be measured; the data processing system receives the real-time freezing image pair sequence of the ice surface 4 to be measured collected by the binocular camera, and performs data processing according to the following two steps to obtain the three-dimensional measurement of the ice surface 4 to be measured. Among them, the online measurement device also includes an optical platform 9, and a horizontal guide rail is arranged on the optical platform 9, and the binocular camera moves away from or close to the dark field lighting system through the horizontal guide rail.

[0053] Specifically, Fig.10 and Fig.11 As shown, taking the ice measurement on the wing model 7 as an example, the present application designs an experimental platform for online measurement of ice shape, which is placed at the wind tunnel exit 5. The online measurement device for ice shape in a cloud and fog field based on binocular vision includes: an optical platform 9, a flange base 8 is installed on the optical platform 9, and the wing model 7 is installed on the flange base 8. The cloud and fog field simulation system is the wind tunnel exit 5, which is arranged facing the wing model 7, and it produces droplets 6 to form a measured ice surface 4 on the wing model 7; the dark field lighting system is yellow light 3, using JINBEI EF-150 fill light as the light source, which is installed through a bracket; the yellow light 3 shines on the measured ice surface 4 from the incoming flow side of the measured surface of the wing model 7 at a low angle, and most of the light does not enter the camera lens after being reflected by the measured ice surface 4 and the wing surface, and a small part of the light enters the camera due to surface diffuse reflection, diffraction, etc. Since the wavelength of yellow light 3 is shorter, the diffraction phenomenon of light in the fog can be reduced, allowing more light to pass through the air with more droplets 6; at the same time, since the scattering intensity of yellow light 3 is five times that of red light, after being affected by the scattering lampshade of the fill light head and the fog field, the light can evenly illuminate the ice surface 4 to be measured.

[0054] The image acquisition system includes binocular cameras, namely, 1# camera 1 and 2# camera 2, which are installed on the side of the flange base 8, 10mm apart, and shoot the same area of ​​the measured ice surface 4 on the wing model 7 at a small angle. The shooting direction of 1# camera 1 close to the yellow light 3 is greater than 45° with the irradiation direction of the yellow light 3, and the shooting angle between the two cameras is less than 10°. Among them, the two cameras are industrial cameras with the same parameters, using global exposure chips to realize mobile image acquisition, with a resolution of 4096 × 3000 pixels and a frame rate of 23.5. A lens with a focal length of 75mm is installed, and the subtle features of the ice surface can be captured at a shooting distance of 200-500mm. The aperture is set to a small aperture of F11 to obtain a larger depth of field range to shoot the curved wing surface. The optical platform 9 is provided with a horizontal guide rail with a stroke of 600 mm, and the flange base 8 is installed on the horizontal movable guide rail, so that the binocular camera can be moved away from or close to the dark field illumination system through the horizontal guide rail; at the same time, it is connected to the optical vibration isolation platform through a scissor lift table, and the height adjustment can be completed according to the actual situation.

[0055] During use, the binocular camera system scans and shoots through the bottom horizontal rail, which is an electric rail 10. The shooting range is 60mm from the leading edge of the wing, and the scanning process is 1s. Although the icing process is an unsteady dynamic process, the maximum icing speed is about 0.1mm / s, and the ice surface changes are small during the scanning time, so they can be ignored. This allows for uninterrupted icing measurement in the cloud and fog field.

[0056] Here, those skilled in the art can understand that the specific measurement method in the above-mentioned cloud and fog field icing ice shape online measurement device based on binocular vision has been referred to above. Figures 1 to 9 The description of the online measurement method of ice shape in cloud and fog fields based on binocular vision has been introduced in detail, so its repeated description will be omitted.

[0057] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. The patent protection scope of the present invention shall be based on the claims. All equivalent structural changes made using the contents of the description and drawings of the present invention should also be included in the protection scope of the present invention.

Claims

1. An online measurement method for ice shape in a cloud and fog field based on binocular vision, characterized in that: The following steps are involved: S1, in a foggy environment, arrange the dark field lighting system facing the ice surface to be measured, and project dark field lighting light onto the ice surface to be measured; S2, using a binocular camera to continuously collect images of the freezing process of the ice surface under test projected with dark field illumination light, to obtain a real-time freezing image pair sequence of the ice surface under test; S3, extracting and matching surface feature points of a sequence of ice images of the real-time ice surface to be measured, obtaining matched surface feature points, and obtaining external parameters of the binocular camera based on the matched surface feature points; S4, based on the binocular vision principle, performs three-dimensional reconstruction of the real-time measured ice surface according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera, thereby realizing real-time three-dimensional measurement of the measured ice surface.

2. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 1 is characterized in that: Between S2 and S3, it also includes: Performing image correction preprocessing on the real-time icing image pair sequence of the measured ice surface to obtain a corrected real-time icing image pair sequence of the measured ice surface; the image correction preprocessing includes distortion correction; In S3, the surface feature points of the real-time ice image sequence of the measured ice surface are extracted and matched to obtain the matched surface feature points, which are specifically: Surface feature points are extracted and matched on the corrected real-time ice image of the measured ice surface to obtain matched surface feature points.

3. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 2 is characterized in that: The extraction and matching of surface feature points of the corrected real-time ice image sequence of the measured ice surface to obtain the matched surface feature points specifically includes: The SIFT multi-scale feature extraction algorithm is used to extract the surface feature points of the sequence of the corrected real-time ice image of the measured ice surface. The surface feature points are matched by the KNN feature point matching method and the random sampling consistency algorithm to obtain the matched surface feature points.

4. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 3 is characterized in that: The SIFT multi-scale feature extraction algorithm includes: Firstly, a nonlinear gamma transform is performed on the corrected real-time ice image pair sequence of the measured ice surface to obtain an enhanced real-time ice image pair sequence of the measured ice surface; then, a Gaussian difference pyramid is constructed based on the enhanced real-time ice image pair sequence of the measured ice surface to obtain a Gaussian difference image, and the scale space extreme value of each pixel point in the Gaussian difference image is detected to obtain the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface; finally, a feature point descriptor is generated based on the position of the extreme value point in the enhanced real-time ice image pair sequence of the measured ice surface to complete the extraction of surface feature points of the enhanced real-time ice image pair sequence of the measured ice surface.

5. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 1 is characterized in that: In S3, the step of acquiring the external parameters of the binocular camera based on the matched surface feature points specifically includes: Based on the matched surface feature points and combined with the internal parameters of the stereo camera, the incremental SFM algorithm is used to determine the external parameters of the stereo camera.

6. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 5 is characterized in that: The incremental SFM algorithm includes: Based on the matched surface feature points, a sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points is obtained; the sequence of real-time icing image pairs of the measured ice surface with the largest number of matching surface feature points is reconstructed in combination with the internal parameters of the binocular camera to obtain the initial external parameters of the binocular camera; the initial external parameters of the binocular camera are iteratively processed and optimized to obtain the external parameters of the binocular camera.

7. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 6 is characterized in that: The iterative processing and optimization processing of the initial external parameters of the binocular camera to obtain the external parameters of the binocular camera includes: iterative processing of the initial external parameters of the binocular camera, and optimization processing using a bundle adjustment method and a random sampling consistency algorithm to obtain the external parameters of the binocular camera.

8. The method for online measurement of ice shape in a cloud and fog field based on binocular vision according to claim 1 is characterized in that: In S4, the three-dimensional reconstruction of the measured ice surface based on the binocular vision principle and according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera specifically includes: based on the binocular vision principle, according to the matched surface feature points, external parameters of the binocular camera and internal parameters of the binocular camera, a stereo matching algorithm is used to construct a local point cloud of the ice image of each measured ice surface, and the spatial relationship between adjacent image pairs is obtained by matching the ice images of the measured ice surface, and a global point cloud of the measured ice surface is established, thereby realizing the three-dimensional reconstruction of the measured ice surface.

9. An online measuring device for executing the online measuring method for ice shape in a cloud and fog field based on binocular vision as claimed in any one of claims 1 to 8, characterized in that: include: A cloud and fog field simulation system, a dark field lighting system, an image acquisition system and a data processing system, wherein the cloud and fog field simulation system is arranged facing the ice surface (4) to be measured and is used to simulate the cloud and fog field environment; the dark field lighting system is arranged facing the ice surface (4) to be measured and is used to project dark field lighting light onto the ice surface (4) to be measured; the image acquisition system comprises a binocular camera, wherein the binocular camera is arranged on the side of the ice surface (4) to be measured and is used to continuously acquire images of the freezing process of the ice surface (4) to be measured with the dark field lighting light projected thereon, so as to obtain a real-time freezing image pair sequence of the ice surface (4) to be measured; and the data processing system receives the real-time freezing image pair sequence of the ice surface (4) to be measured acquired by the binocular camera and performs data processing according to steps S3-S4 to obtain three-dimensional measurement of the ice surface (4) to be measured.

10. The binocular vision-based online measurement device for ice shape in a cloud and fog field according to claim 9, characterized in that: The online measurement device further comprises an optical platform (9), on which a horizontal guide rail is arranged, and the binocular camera moves away from or close to the dark field illumination system via the horizontal guide rail.

Images