Method for three-dimensional reconstruction of two-dimensional infrared images of an aeroengine blade under monocular vision

Abstract

The application discloses a method for three-dimensional reconstruction of two-dimensional infrared images of an aero-engine blade under single visual angle, and relates to the technical field of computer vision, wherein the conversion from a two-dimensional temperature field to a three-dimensional temperature field is realized by means of a spatial position mapping algorithm for infrared temperature images under single visual angle on the premise of three-dimensional modeling of an aero-engine. The application has the characteristics of high efficiency, precision and low cost without providing multiple visual angle infrared images; the spatial position mapping of infrared images is combined with a three-dimensional model of a measured object for the first time to realize three-dimensional temperature field reconstruction of the measured object, which provides a new solution for temperature measurement and control of aero-engine blades and has a positive significance for promoting the development of the aviation industry.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a method for three-dimensional reconstruction of two-dimensional infrared images of an aero-engine blade under a single visual angle. BACKGROUND

[0002] The rapid measurement of temperature distribution in the process of shell ground tunnel experiment and engine test in aviation and space tasks is increasingly urgent. The contact temperature measurement method obviously cannot meet the requirements in terms of temperature measurement range and dynamic response frequency. The existing radiation method for temperature measurement needs to well solve the problem between temperature measurement accuracy and object surface emissivity. The online measurement of flame temperature and temperature distribution of a tail nozzle is urgently needed in the fields of rocket engine, turbine engine and ramjet, which can improve the design of the engine and evaluate the performance of the engine, and has become a difficult problem in aerospace technology.

[0003] The temperature field detection technology can be applied in the field of temperature testing of thermal barrier coating materials and other materials related to aero-engine and ground gas turbine, and fills the technical gap of temperature testing by using thin film temperature sensor technology at present in China. The technology can increase the success rate of small space wall temperature testing, reduce the testing modification cost, and has greater economic benefits. At the same time, the technology can be applied to the wall temperature testing of other engine parts, core engines and whole machines, or applied to other engine testing projects requiring rapid temperature testing response.

[0004] The temperature measuring instrument composed of thermocouple, electric measuring instrument and connecting wire is widely used for contact temperature measurement. The thermocouple is a closed loop composed of two different conductors or semiconductors, and the two end points are respectively in different temperature environments. When the environment temperature reaches thermal equilibrium, the loop will generate electromotive force, i.e. thermoelectric potential. After calibration, it can be used to measure temperature. In principle, thermocouple temperature measurement has high accuracy and repeatability, and it is widely used for temperature measurement in the range of 100-1600 ℃. Thermocouples made of special materials can also measure higher or lower temperatures, and are widely used in the fields of industrial production and scientific research of coal combustion. In large coal-fired boilers, high-temperature thermocouples are generally used to measure the temperature of high-temperature flue gas in the furnace point by point. However, due to its ability to measure only "point" temperature and the limitation of high-temperature performance of the thermistor element material, it cannot realize real-time online temperature measurement of the high-temperature surface to be measured for a long time.

[0005] The infrared radiation temperature measurement method is a method for determining the flame temperature in the engine by measuring the radiation energy of specific wavelength emitted by the flue gas composition in the combustion process. The principle of infrared temperature measurement technology is radiation temperature measurement, which belongs to non-contact measurement. In recent years, the performance has been continuously improved, the function has been continuously enhanced, and it has developed rapidly. Compared with the thermocouple temperature measurement method, the infrared temperature measurement method mainly has the following advantages:

[0006] 1. Non-contact and long-distance measurement. The infrared thermometer determines the temperature of the object by measuring the infrared radiation of the object surface, without contacting the measured object, and can measure the global temperature of the object surface in real time. Moreover, the measurement method can obtain the temperature of the object at a long distance, is less affected by the environment, and is suitable for measuring the temperature in various harsh environments.

[0007] 2. High accuracy and wide temperature measurement range. The measurement method only receives infrared radiation of a specific wavelength, does not affect the temperature field to be measured, has high measurement accuracy, and can measure infrared waves of specific wavelengths emitted by different substances with adjustable wavelength range.

[0008] 3. Short response time and fast reaction speed. The temperature can be determined as soon as the infrared radiation of the target is received, the response time is in the order of milliseconds or even microseconds, and real-time online monitoring and remote control can be performed. Moreover, the use and maintenance are convenient.

[0009] However, the infrared thermometer only obtains two-dimensional temperature images, which can only reflect local and planar temperature information, and the temperature distribution information of the temperature field in the depth direction is compressed. If three-dimensional temperature distribution is to be obtained, the measurer mainly obtains the depth information of the object by multi-angle and multi-position infrared shooting, which makes the entire measurement system complicated and cumbersome. In the data processing process, multi-angle infrared pictures need to be fused, and the data processing cost is high.

[0010] Therefore, the skilled in the art is committed to developing a method for three-dimensional reconstruction of two-dimensional infrared images of an aero-engine blade under single-angle, to overcome the problems existing in the prior art. SUMMARY

[0011] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present application is to provide a method for three-dimensional temperature field reconstruction with high efficiency, precision and low cost, to solve the problems of the prior art, such as the need for multi-position, complicated system and high cost for three-dimensional reconstruction of the temperature field.

[0012] To achieve the above-mentioned purpose, the present application provides a method for three-dimensional reconstruction of two-dimensional infrared images of an aero-engine blade under single-angle, which comprises the following steps:

[0013] Step 1, 3D modeling of the measured aero-engine blade is performed, and a single-angle infrared image of the aero-engine blade is obtained, wherein the aero-engine blade occupies m pixel rows in the infrared image;

[0014] Step 2, according to the 3D model constructed in step 1, three-dimensional point cloud data of the model surface is obtained, and a total of M three-dimensional points are taken;

[0015] Step 3, according to the infrared image in step 1, obtaining the peripheral contour pixel position of the infrared image of the aero-engine blade;

[0016] Step 4, determining the normal vector of the plane where the infrared camera is imaged and the unit vector of the pixel row and the pixel column of the image formed;

[0017] Step 5, according to the unit vector of the pixel column of the infrared image obtained in step 4, calculating the projection value a of each point in step 2 on the unit vector of the pixel column of the infrared image i , and sorting, where i=1,…,M, the projection value a i ranges from [p,q];

[0018] Step 6, according to the number of pixel rows m contained in the infrared image, equally dividing the projection value range [p,q] into m intervals, and the three-dimensional points in the same interval correspond to the same row of pixels in the infrared image;

[0019] Step 7, respectively calculating the projection value b of all three-dimensional points in each interval in step 6 on the unit vector of the pixel row of the infrared image obtained in step 4 j,r , wherein the projection value b j,r ranges from [h,k], j represents the jth row of pixels occupied by the aero-engine blade in the infrared image, and r represents the rth three-dimensional point in the interval; according to the relative position of the projection value b j,r in the projection value b j,r range [h,k], the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image is calculated;

[0020] Step 8, judging whether each three-dimensional point is blocked or not, if there is blocking, the point cannot be photographed by the infrared camera, i.e. cannot be given temperature information;

[0021] Step 9, according to steps 6, 7 and 8, determining the pixel position of any one three-dimensional point in the corresponding point of the infrared image of the aero-engine blade, and determining its temperature;

[0022] Step 10, after completing the acquisition of the infrared temperature information of each three-dimensional point, performing surface reconstruction processing on the three-dimensional point cloud.

[0023] Further, the peripheral contour pixel position of the infrared image of the aero-engine blade in step 3 needs to be filtered and binarized first, and then the image contour is located.

[0024] Further, the image contour is located by means of OpenCV library.

[0025] Further, in step 3, when the peripheral contour pixel position of the infrared image of the aero-engine blade is obtained, each row of pixels records a left end point and a right end point, i.e. [Lj ,R j ], where j represents the j-th row of pixels of the portion of the image occupied by the aircraft engine blades in the captured infrared image.

[0026] Furthermore, in step 7, according to b j,r The relative position in the range [h, k] is used to calculate the coordinates of the pixel column in the infrared image corresponding to the three-dimensional point. The calculation formula is: in Indicates a floor operation.

[0027] Furthermore, the determination of the normal vector of the plane where the infrared camera is imaging and the unit vectors of the pixel rows and pixel columns of the image in step 4 is performed according to the shooting position and angle of the infrared camera.

[0028] Furthermore, the determination of whether each three-dimensional point is blocked in step 8 is performed by estimating the normal vector of the point cloud.

[0029] Furthermore, the normal vector estimation of the point cloud is performed with the help of the Open3D library.

[0030] Furthermore, the normal vector of the point cloud is estimated to determine whether each three-dimensional point is occluded. If the inner product of the normal vector of the three-dimensional point and the normal vector of the imaging plane is greater than a predetermined threshold, occlusion exists.

[0031] Furthermore, the surface reconstruction of the three-dimensional point cloud adopts a rolling ball method.

[0032] The advantages of the present invention are that it combines the spatial position mapping of infrared images and the three-dimensional model of the object to be measured for the first time to realize the reconstruction of the three-dimensional temperature field of the object to be measured, providing a new solution for the temperature measurement and control of aircraft engine blades, and has positive significance for promoting the development of the aviation industry.

[0033] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a real picture of an aero-engine blade according to a preferred embodiment of the present invention;

[0035] Figure 2 This is a picture of a 3D model of an aero-engine blade according to a preferred embodiment of the present invention;

[0036] Figure 3 This is an infrared image of an aircraft engine blade according to a preferred embodiment of the present invention;

[0037] Figure 4 is a 3D model surface point cloud picture of an aero-engine blade of a preferred embodiment of the present application;

[0038] Figure 5 is an aero-engine blade infrared image contour map of a preferred embodiment of the present application;

[0039] Figure 6 is an aero-engine blade surface three-dimensional point cloud normal vector of a preferred embodiment of the present application;

[0040] Figure 7 is an aero-engine blade temperature field reconstruction map of a preferred embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application can be embodied in many different forms and is not limited to the embodiments set forth herein. The preferred embodiments of the present application are described below with reference to the accompanying drawings.

[0042] In the drawings, components having the same structure are denoted by the same reference numerals, and components having similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present application is not limited to the size and thickness of each component. In order to make the drawing clearer, the thickness of some components is exaggerated in some places.

[0043] The present application provides a method for three-dimensional reconstruction of a two-dimensional infrared image of an aero-engine blade under monocular perspective, which belongs to the field of image processing. The conversion of a two-dimensional temperature field to a three-dimensional temperature field is achieved by means of a spatial position mapping algorithm on the infrared temperature image under monocular perspective on the premise of three-dimensional modeling of the aero-engine.

[0044] The specific technical solution implementation steps are as follows:

[0045] Step 1, 3D modeling of the measured aero-engine blade is performed, and a single perspective infrared image of the aero-engine blade is obtained (assuming that the aero-engine blade occupies m pixel rows in the infrared image);

[0046] Step 2, according to the 3D model constructed in step 1, three-dimensional point cloud data of the model surface is obtained, a total of M three-dimensional points;

[0047] Step 3, according to the infrared image in step 1, the peripheral contour pixel position of the aero-engine blade infrared image is obtained (each row of pixels records a left end point and a right end point, i.e. [L j ,R j ], where j represents the jth row of pixels occupied by the aero-engine blade in the photographed infrared image);

[0048] Preferably, the peripheral contour pixel position of the aero-engine blade infrared image in step 3 needs to be filtered and binarized first, and then the image contour is located by means of Open Source Computer Vision Library (OpenCV);

[0049] Step 4, according to the position and angle of the infrared camera shooting, the normal vector of the imaging plane of the infrared camera and the unit vector of the pixel row and pixel column of the image formed are determined;

[0050] Step 5, according to the unit vector of the pixel column of the infrared image obtained in step 4, the projection value a of each point in step 2 on the unit vector of the pixel column of the infrared image is calculated i (i takes the value of 1~M), and is sorted (the projection value ranges from [p, q]);

[0051] Step 6, according to the number of pixel rows m contained in the infrared image, the projection value ([p, q]) is equally divided into m intervals, and the three-dimensional points in the same interval correspond to the same row of pixels in the infrared image;

[0052] Step 7, the projection value b of all three-dimensional points in each interval in step 6 on the unit vector of the pixel row of the infrared image obtained in step 4 is calculated respectively j,r (the projection value ranges from [h, k], where j represents the jth row of pixels occupied by the aero-engine blade in the infrared image, and r represents the rth three-dimensional point in the interval), according to the relative position of b j,r in the range of [h, k], the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image is calculated;

[0053] Preferably, the relative position of b j,r in the range of [h, k] is calculated, and the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image is calculated, and the calculation formula is (where represents the floor operation);

[0054] Step 8, it is judged whether each three-dimensional point is shielded or not, if there is shielding, the point cannot be shot by the infrared camera, that is, cannot be given temperature information;

[0055] Preferably, whether each three-dimensional point is shielded or not is judged by means of normal vector estimation of point cloud by Open3D library, if the inner product of the normal vector of the three-dimensional point and the normal vector of the imaging plane is greater than a certain threshold (threshold), there is shielding;

[0056] Step 9, according to steps 6, 7 and 8, the pixel position of any three-dimensional point in the corresponding point of the infrared image of the aero-engine blade can be determined, and the temperature thereof can be determined;

[0057] Step 10, after the infrared temperature information of each three-dimensional point is acquired, the surface reconstruction processing is performed on the three-dimensional point cloud;

[0058] Preferably, the surface reconstruction of the three-dimensional point cloud is performed by using the rolling ball method.

[0059] Embodiment

[0060] The embodiment provides a method for three-dimensional reconstruction of a two-dimensional infrared image of an aero-engine blade under single-view, and the specific steps are as follows:

[0061] Step 1, a 3D model of the aero-engine blade to be measured is constructed, and a single-view infrared image of the aero-engine is acquired, as shown in Figure 1 、 Figure 2 and Figure 3 ;

[0062] Step 2, according to the 3D model constructed in step 1, three-dimensional point cloud data of the model surface is acquired, and a total of 30000 three-dimensional points are taken, and the point cloud is as shown in Figure 4 ;

[0063] Step 3, according to the infrared image in step 1, the peripheral contour pixel position of the infrared image of the aero-engine is acquired (each row of pixels records a left end point and a right end point, that is, [L j ,R j ], wherein j represents the jth row of pixels of the aero-engine blade in the part of the image shot by the infrared image); the contour diagram is as shown in Figure 5 ;

[0064] Step 4, according to the shooting position and angle of the infrared camera, the normal vector of the plane where the infrared camera is imaged and the unit vectors of the pixel row and the pixel column of the formed image are determined; in the embodiment, the normal vector is (0, 1, 0), the unit vector of the pixel column direction of the formed image is (0, 0, -1), and the unit vector of the pixel row direction of the formed image is (1, 0, 0);

[0065] Step 5, according to the unit vector of the infrared image pixel column obtained in step 4, the projection value a i (i takes values 1-30000) of each point in step 2 on the unit vector of the infrared image pixel column is calculated and sorted (the projection value ranges from 0 to 200, and the unit is mm);

[0066] Step 6, according to the number of pixel rows 436 contained in the infrared image, the projection value ([0, 200]) is equally divided into 436 intervals, and the three-dimensional points in the same interval correspond to the same row of pixels in the infrared image;

[0067] Step 7, the projection value b of all three-dimensional points in each interval in step 6 on the unit vector of the pixel row of the infrared image obtained in step 4 is calculated respectively j,r (the projection value range is [h, k], where j represents the jth row of pixels corresponding to the infrared image of the aero-engine, and r represents the rth three-dimensional point in the interval), according to b j,r the relative position in the range [h, k], and the formula “ (where represents the floor operation)” is used to calculate the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image;

[0068] Step 8, set the threshold value (threshold) to -0.1, and judge whether each three-dimensional point is blocked according to whether the inner product of the three-dimensional point normal vector and the imaging surface normal vector is greater than the threshold value; if the inner product value is greater than the threshold value, it means that the point is blocked, that is, it cannot be assigned temperature information; the three-dimensional point normal vector is as shown in Figure 6 ;

[0069] Step 9, according to steps 6, 7 and 8, the pixel position of any three-dimensional point in the infrared image corresponding to the aero-engine can be determined, and its temperature can be determined;

[0070] Step 10, after each three-dimensional point completes the acquisition of infrared temperature information, the surface reconstruction of the three-dimensional point cloud is performed, and the reconstruction result is as shown in Figure 7 .

[0071] Analysis of the reconstruction result of the aero-engine infrared temperature field:

[0072] In the embodiment, accurate position mapping between the two-dimensional infrared plane temperature image photographed by the infrared camera and the surface of the three-dimensional model is successfully realized, so that three-dimensional reconstruction of the temperature field is realized. From the reconstruction result of Figure 7 , it can be seen that the infrared image does not appear distorted or stretched, and the reconstruction effect is good.

[0073] The method for three-dimensional reconstruction of the temperature field of the aero-engine blade under single view provided by the application does not need to provide multiple view infrared images, and has the characteristics of high efficiency, accuracy and low cost.

[0074] The application first combines the infrared image spatial position mapping and the three-dimensional model of the measured object to realize the three-dimensional temperature field reconstruction of the measured object, which provides a new solution for the temperature measurement and control of the aero-engine blade, and has a positive significance for promoting the development of the aviation industry.

[0075] The preferred embodiments of the application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art in light of the teachings above without departing from the spirit of the application. It is, therefore, to be understood that what is desired to be protected by letters patent is set forth in the appended claims.

Claims

1. A method for three-dimensional reconstruction of two-dimensional infrared images of an aeroengine blade under monoscopic view, characterized in that, The method comprises the following steps: Step 1: 3D modeling of the measured aero-engine blade is performed, and an aero-engine blade single-view infrared image is obtained, wherein the aero-engine blade occupies m pixel rows in the infrared image; Step 2: According to the 3D model constructed in step 1, three-dimensional point cloud data of the model surface is obtained, and M three-dimensional points are obtained; Step 3: According to the infrared image in step 1, the peripheral contour pixel position of the aero-engine blade infrared image is obtained; In the step 3, when the peripheral profile pixel position of the aero-engine blade infrared image is obtained, a left end point and a right end point, i.e., [L j ,R j ] are recorded for each row of pixels, wherein j represents the jth row of pixels of the aero-engine blade occupied part of the image. Step 4: The normal vector of the plane where the infrared camera is imaged and the unit vectors of the pixel rows and columns of the formed image are determined; Step 5, calculate the projection value a of each point in the pixel column unit vector of the infrared image in step 2 according to the pixel column unit vector of the infrared image obtained in step 4 i , and sort, where i = 1, …, M, the projection value a i ranges in [p, q]; Step 6: According to the pixel row number m contained in the infrared image, the projection value range [p, q] is equally divided into m intervals, and the three-dimensional points in the same interval correspond to the same row of pixels in the infrared image; Step 7, calculate the projection value b of all three-dimensional points in each interval in step 6 on the unit vector of the pixel row of the infrared image obtained in step 4 respectively j,r Wherein, the projection value b j,r The range is [h, k], j represents the corresponding infrared image pixel row of the aero-engine blade, and r represents the rth three-dimensional point in the interval; according to b j,r The relative position in the projection value b j,r The range [h, k], the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image is calculated. The step 7 according to b j,r The relative position in the range of [h, k] is calculated, and the coordinate of the pixel column corresponding to the three-dimensional point in the infrared image is calculated according to the formula Wherein Indicates the floor operation. Step 8: Determine whether each three-dimensional point is blocked, if there is a block, the point cannot be photographed by the infrared camera, that is, it cannot be given temperature information; Step 9: According to steps 6, 7 and 8, determine the pixel position of any three-dimensional point in the aero-engine blade infrared image corresponding point, and determine its temperature; Step 10: After obtaining the infrared temperature information of each three-dimensional point, the surface reconstruction of the three-dimensional point cloud is performed.

2. The method for 3D reconstruction of 2D infrared images of aeroengine blades under monoscopic view according to claim 1, characterized in that, In step 3, the peripheral contour pixel position of the aero-engine blade infrared image is obtained, which needs to be filtered and binarized first, and then the image contour is located.

3. The method of claim 2, wherein, The image contour is located by means of OpenCV library.

4. The method of claim 1, wherein, In step 4, the normal vector of the plane where the infrared camera is imaged and the unit vectors of the pixel rows and columns of the formed image are determined according to the infrared camera shooting position and angle.

5. The method for 2D infrared image 3D reconstruction of aeroengine blade under monoscopic view of claim 1, wherein, In step 8, whether each three-dimensional point is blocked is determined by estimating the normal vector of the point cloud.

6. The method of claim 5, wherein, The normal vector of the point cloud is estimated by means of Open3D library.

7. The method of claim 6, wherein, In step 8, whether each three-dimensional point is blocked is determined by estimating the normal vector of the point cloud.

8. The method for 2D infrared image 3D reconstruction of aeroengine blade under monoscopic view of claim 1, wherein, If the inner product of the three-dimensional point normal vector and the imaging plane normal vector is greater than a predetermined threshold, it is determined that the three-dimensional point is blocked. The surface reconstruction of the three-dimensional point cloud is performed by using the rolling sphere method.

Images