Method for thermal radiation surface reconstruction and correction based on curvature flow and control point screening

Abstract

The application discloses a thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening, which comprises the following steps: obtaining a thermal radiation degraded image and performing pretreatment; calculating the local average curvature and gradient amplitude of each point in the image, iteratively updating the height of each point on the curved surface through the curvature flow operation, and simultaneously combining the maximum iteration number and the convergence criterion to adaptively terminate iteration; performing morphological top hat transformation on the image processed by the curvature flow, and performing down-sampling to obtain thermal radiation curved surface control points; calculating the difference between the local plane normal vector of each control point and the neighborhood control points, obtaining Laplace curvature, and screening out the control points less than the curvature threshold to form the final control points together with the boundary points; interpolating and reconstructing the thermal radiation curved surface through Delaunay triangulation, and performing Lowess smoothing processing to obtain a continuous and smooth thermal radiation curved surface; and subtracting the thermal radiation curved surface from the thermal radiation degraded image to obtain a correction image. The application can correct the thermal radiation degraded image with high-light targets or abnormal extreme values.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of aerodynamic optical thermal radiation effect correction, and particularly relates to a thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening. BACKGROUND

[0002] In a high-temperature and strong-airflow environment, the high temperature of the window surface of an aircraft infrared imaging device will cause radiation distortion, making it difficult for an optical detector to accurately obtain target information. The method of using deep learning to establish a thermal radiation image degradation model and a correction parameter database to correct the thermal radiation degradation image has great limitations. The physical means such as imaging window coating and cooling adopted not only increase the design difficulty and cost, but also cannot significantly improve the imaging clarity.

[0003] Meanwhile, due to the complexity and diversity of the imaging conditions, imaging equipment and imaging targets of high-speed aircraft, the applicability of various correction algorithms is relatively narrow, and they are generally used for 8-bit low dynamic range degradation images, but when applied to 12-bit, 16-bit and other high dynamic range degradation images with saturated bright targets or abnormal values, overfitting or overcompensation may occur, so that high-quality clear images cannot be restored.

[0004] The existing traditional correction algorithm uses control points for curved surface fitting. Once the control points are not selected properly, the curved surface calculation result will be affected by the control points located at the target or noise position. Meanwhile, no matter how high the order of the curved surface is, it will still be strictly constrained by the mathematical model, and problems such as excessive calculation and long time consumption may also occur. SUMMARY

[0005] The purpose of the present application is to provide a thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening, which can be used for the correction of high dynamic range aerodynamic optical thermal radiation effect degradation images with saturated bright targets.

[0006] The technical solutions adopted by the present application are as follows:

[0007] The first aspect of the present application provides a thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening, which comprises the following steps:

[0008] S1, obtaining a thermal radiation degradation image and pre-processing the thermal radiation degradation image;

[0009] S2, based on the pre-processed image, calculating the local average curvature and gradient amplitude of each point in the image, iteratively updating the height of each point of the curved surface through the curvature flow operation, and simultaneously combining the maximum iteration number and the convergence criterion to adaptively terminate the iteration to obtain the image after curvature flow processing;

[0010] S3, performing morphological top-hat transformation on the image processed by the curvature flow to suppress local abnormal extreme values in the image, extracting a thermal radiation feature image, and performing down-sampling on the thermal radiation feature image to obtain thermal radiation curved surface control points;

[0011] S4, determining local plane normal vectors of all control points of the thermal radiation curved surface;

[0012] S5, calculating differences between the local plane normal vector of each control point and local plane normal vectors of neighboring control points of the control point to obtain Laplacian curvatures of the control points, screening out control points with a Laplacian curvature less than a curvature threshold value to form final control points of the thermal radiation curved surface together with boundary points;

[0013] S6, reconstructing the thermal radiation curved surface through Delaunay triangulation interpolation according to the final control points of the thermal radiation curved surface to obtain a preliminary thermal radiation curved surface, and performing Lowess smoothing processing on the preliminary thermal radiation curved surface to obtain a continuous and smooth thermal radiation curved surface;

[0014] S7, subtracting the continuous and smooth thermal radiation curved surface from the thermal radiation degeneration image to realize correction of the thermal radiation degeneration image, and obtaining a corrected image.

[0015] In the above scheme, the preprocessing includes gray scale conversion, normalization and boundary filling.

[0016] In the above scheme, step S2 specifically includes:

[0017] Based on the preprocessed image Z, local average curvatures and gradient amplitudes H of each point in the image are calculated:

[0018]

[0019]

[0020] In the formula, H and are the local average curvatures and gradient amplitudes of each point in the image Z; , , , , are the first-order partial derivatives and second-order partial derivatives of each point in the image Z;

[0021] Taking each point as a unit, the local average curvature H and the gradient amplitude of the point are combined to iteratively update the curved surface height:

[0022]

[0023] In the formula, is the surface after one iteration update; is the current surface, and the initial value is the value of each point in the preprocessed image Z; is the time step;

[0024] The maximum amplitude of the surface change between two adjacent iterations is calculated, and if it is less than a set threshold value, or the current iteration number reaches a preset maximum iteration number, the iteration is terminated to obtain the curvature flow processed image; otherwise, the local average curvature and gradient amplitude of each point in the graph are recalculated based on the iteration updated surface, and the iteration is continued.

[0025] In the above scheme, after each iteration update, it is judged whether the value of each point is between 0 and 1, if yes, the original value is kept, if greater than 1, it is set to 1, and if less than 0, it is set to 0.

[0026] In the above scheme, step S5 specifically includes:

[0027] The Euclidean difference average between the local plane normal vector of each control point and the local plane normal vector of its neighborhood control point is calculated as the local curvature index of the control point:

[0028]

[0029] In the formula, represents the Laplace curvature of the control point; represents the neighborhood control point set of the control point, represents the number of control points in the neighborhood control point set; represents the local plane normal vector of the control point, represents the local plane normal vector of the control point in the neighborhood control point set;

[0030] According to the set curvature threshold , the control points with Laplace curvature less than the curvature threshold are screened out to form the final control points of the thermal radiation surface together with the boundary points:

[0031]

[0032] In the formula, is the final control point set of the thermal radiation surface, including all control points with Laplace curvature less than the curvature threshold and the boundary point set.

[0033] ​​In the above scheme, according to the final control points of the thermal radiation curved surface, the thermal radiation curved surface is reconstructed by Delaunay triangulation interpolation to obtain a preliminary thermal radiation curved surface, comprising:

[0034] According to the final control points of the thermal radiation curved surface, a Delaunay triangular mesh is constructed with the final control points as the vertices by using a Delaunay triangulation method, and the entire curved surface is divided into several non-overlapping triangles;

[0035] For all points in the thermal radiation degenerated image, if there is a corresponding final control point, the value of the corresponding final control point is taken; if there is no corresponding final control point, the point is an interpolation point; for the interpolation point, the triangle to which the interpolation point belongs is determined, and the coordinates and gray values of the three vertices of the triangle are used to calculate the gray value of the interpolation point by weighted average according to the area

[0036]

[0037] In the formula, represents the value of the interpolation point ; represents the value of the three vertices of the triangle to which the interpolation point belongs; is the interpolation weight corresponding to each vertex, is the area of the sub-triangle formed by the interpolation point and the remaining two vertices of the triangle;

[0038] After traversing all the interpolation points, a continuous curved surface, i.e. a preliminary thermal radiation curved surface, is formed.

[0039] In the above scheme, the Lowess smoothing processing is:

[0040]

[0041]

[0042] In the formula, is the value of the point in the preliminary thermal radiation curved surface after Lowess smoothing processing; represents all points in the local neighborhood of the point , is the radius of the local neighborhood ; represents the value of the neighborhood point ; represents the weight of the neighborhood point .

[0043] In the above scheme, step S7 further comprises:

[0044] ​​The luminance of the point in the corrected image whose luminance exceeds the luminance threshold is set to the luminance threshold, and the low-luminance area in the image is enhanced through linear gray scale stretching.

[0045] According to a second aspect of the present application, a computer device is provided, comprising a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions being executed by the processor to implement the steps of the thermal radiation surface reconstruction and correction method based on curvature flow and control point screening according to any one of the first aspect.

[0046] According to a third aspect of the present application, a computer readable storage medium is provided, having stored thereon programs or instructions, the programs or instructions being executed by a processor to implement the steps of the thermal radiation surface reconstruction and correction method based on curvature flow and control point screening according to any one of the first aspect.

[0047] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0048] The present application provides a thermal radiation surface reconstruction and correction method based on curvature flow and control point screening, which can avoid the influence of saturated high-brightness targets and outliers on the thermal radiation surface calculation process when processing high dynamic range degraded images, and screen the selected control points according to their curvature characteristics before surface reconstruction, avoiding the use of unreasonable control points for calculation to affect the correction results. At the same time, the surface reconstruction is performed by the weighted interpolation method using irregular control points, and the interpolation points are only affected by the adjacent control points, avoiding the constraints of mathematical models. This method can correct different types of aerodynamic optical thermal radiation effects, especially for high dynamic range thermal radiation degraded images with abnormal high-brightness targets or more noise and targets, which will have better results and can improve the image signal-to-noise ratio, preparing for the realization of target recognition and detection operation. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 A flowchart of a thermal radiation surface reconstruction and correction method based on curvature flow and control point screening is provided for the embodiments of the present application;

[0050] Figure 2 A 16-bit high dynamic range thermal radiation degraded image with saturated high-brightness targets is provided for the embodiments of the present application;

[0051] Figure 3 An image after curvature flow processing of the image is provided for the embodiments of the present application;

[0052] Figure 4 A thermal radiation feature map after morphological top-hat transformation to suppress high-brightness targets is provided for the embodiments of the present application;

[0053] Figure 5 A control point image obtained by down-sampling a thermal radiation feature image is provided for an embodiment of the present application;

[0054] Figure 6 A screened control point image is provided for an embodiment of the present application;

[0055] Figure 7 A thermal radiation surface image obtained after reconstruction is provided for an embodiment of the present application;

[0056] Figure 8 A three-dimensional surface image corresponding to the thermal radiation surface image obtained after reconstruction is provided for an embodiment of the present application;

[0057] Figure 9 A thermal radiation surface corrected image is provided for an embodiment of the present application;

[0058] Figure 10 A three-dimensional surface image corresponding to the thermal radiation surface corrected image is provided for an embodiment of the present application;

[0059] Figure 11 A corrected image with enhanced display effect is provided for an embodiment of the present application;

[0060] Figure 12 A three-dimensional surface image corresponding to the corrected image with enhanced display effect is provided for an embodiment of the present application;

[0061] Figure 13 A hardware structure schematic diagram of a computer device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0063] It is apparent that the drawings in the following description merely show some examples or embodiments of the present application, and the present application can be applied to other similar situations without creative labor by those skilled in the art based on these drawings. In addition, it can be understood that, although the efforts made in the development process can be complex and lengthy, some modifications, such as design, manufacture or production, etc. based on the technical content disclosed in the present application, are only routine technical means for those skilled in the art related to the disclosure of the present application, and should not be understood as insufficient disclosure of the present application.

[0064] In the present application, the term "embodiment" means that the specific features, structures or properties described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0065] Unless otherwise defined, the technical terms or scientific terms involved in the present application should be understood as the usual meaning by those skilled in the art in the technical field to which the present application belongs. The terms "one", "a", "an", "the", and similar words involved in the present application do not represent quantity limitation, but can represent singular or plural. The terms "include", "contain", "have", and any variations thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or units, but can also include steps or units not listed, or can also include other steps or units inherent to the process, method, product or device. The terms "connected", "connected", "coupled" and similar words involved in the present application are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The term "multiple" refers to two or more. The association between the associated objects is described by the term "and / or", which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects. The terms "first", "second", "third", etc. involved in the present application are only to distinguish similar objects, and do not represent a specific order for the objects.

[0066] The application provides a thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening, comprising the following steps: firstly, inputting a thermal radiation degradation image and processing parameters, pre-processing the thermal radiation degradation image, including gray scale conversion, normalization and boundary filling; then, iteratively updating the image curved surface height through the local average curvature and gradient amplitude of the pre-processed image pixel points, filtering and smoothing the abrupt details on the curved surface, and simultaneously suppressing the local abnormal extreme value in the image through morphological top-hat transformation, so as to extract a thermal radiation feature image; obtaining the thermal radiation curved surface control points through downsampling, extracting the local curved surface normal vector based on the control point normal vector estimation method; subsequently, calculating the control point and its neighborhood normal vector difference to obtain the Laplace curvature of the control point, and screening out the low curvature points and boundary points; on this basis, using the Delaunay triangulation interpolation curved surface reconstruction and Lowess smoothing algorithm to construct a continuous and smooth thermal radiation curved surface; finally, removing the curved surface from the input image to realize the thermal radiation effect correction of the degradation image, and simultaneously suppressing the highlight target and enhancing the dark part of the corrected image to achieve better display effect.

[0067] The method is simple and has obvious effect, and can be used for the correction of a high dynamic range aerodynamic optical thermal radiation effect degradation image with a saturated bright target.

[0068] As shown in Figure 1 The thermal radiation curved surface reconstruction and correction method based on curvature flow and control point screening provided by the application comprises the following steps:

[0069] S1, pre-processing the image, including gray scale conversion, normalization and boundary filling.

[0070] S2, calculating the local average curvature and gradient amplitude of each point of the image, iteratively updating the height of each point of the curved surface through the curvature flow operation, and simultaneously combining the maximum iteration number and convergence criterion to adaptively terminate the iteration, so as to filter and smooth the abrupt details on the image curved surface.

[0071] Specifically, the local average curvature and gradient amplitude of each point in the pre-processed image three-dimensional curved surface are calculated, at each curved surface point, the height value is updated based on the current local curvature characteristics and the time step, the entire curved surface is iteratively updated while the numerical truncation is applied to prevent numerical overflow, and the maximum iteration number and convergence criterion are combined to adaptively terminate the iteration, so as to filter and smooth the abrupt details on the image curved surface, and preliminarily extract the thermal radiation feature image.

[0072] S3, then, on the image obtained in step S2, local abnormal extreme values in the image are suppressed by morphological top-hat transformation, thereby extracting a thermal radiation feature image, and down-sampling is performed to obtain thermal radiation curved surface control points.

[0073] Specifically, for the preliminarily extracted thermal radiation feature image, first, a morphological structural element is constructed as a local operation template of the image, and by using the template, local abnormal extreme value regions in the image with a size smaller than the structural element are extracted by top-hat transformation TopHat algorithm, i.e., original image minus morphological opening operation result; then, the regions are removed from the original image, thereby extracting a thermal radiation feature image of background brightness change trend caused by thermal effect after smoothing scene target edges and details, and down-sampling is performed to select pixel points on a uniform grid as thermal radiation curved surface control points.

[0074] S4, a local point set of the thermal radiation curved surface control points is extracted by a radius search method based on Euclidean distance, and a local plane is fitted based on principal component analysis method, and a local plane normal vector is extracted.

[0075] Specifically, the Euclidean distances of the thermal radiation curved surface control points to other points are calculated, and points within a set radius range are selected as the neighborhood point set thereof. A normal vector estimation method based on local neighborhood principal component analysis is adopted, a feature vector corresponding to the minimum eigenvalue is extracted as a local tangent plane normal vector by eigenvalue decomposition of a covariance matrix, and is used for calculating the normal vector difference in the subsequent step.

[0076] S5, the normal vector difference of the control points and the control points in the field thereof is calculated, a Laplace curvature is obtained, and according to a set curvature threshold, low curvature points and boundary points are screened out as final control points.

[0077] Specifically, the difference between the normal vector of each control point and the normal vectors in the neighborhood thereof, i.e., the Laplace curvature, is calculated as a local curvature index of the point. By using the index, high curvature control points located on scene targets and details are filtered out, and control points on the image edge are forcibly retained, thereby constructing a stable and complete background control point set for subsequent thermal radiation curved surface reconstruction.

[0078] S6, according to the screened control points, a thermal radiation curved surface is reconstructed by Delaunay triangulation interpolation, and a Lowess smoothing algorithm is used to eliminate local noise and jumps, thereby obtaining a continuous and smooth thermal radiation curved surface.

[0079] Specifically, taking the selected control points as vertices, a Delaunay triangular mesh is constructed to divide the whole plane into several non-overlapping triangles, for each regular grid point, the Delaunay triangle in which the grid point is located is found, and the gray value of the grid point is calculated by using the coordinates and gray values of the vertices of the triangle and performing a weighted average according to the area, so as to form a continuous surface, and finally, the local noise and jumps in the continuous surface are eliminated by using a Lowess smoothing algorithm, so that a continuous and smooth thermal radiation surface is obtained.

[0080] S7, the reconstructed thermal radiation surface is removed from the degraded image, so that the correction of the degraded image is realized, and meanwhile, the highlight target is suppressed and the dark part is enhanced in the corrected image, so that a better display effect is realized.

[0081] Specifically, the reconstructed thermal radiation surface is removed from the degraded image to obtain an image after thermal radiation effect correction, then a highlight threshold is set to cut off and suppress the abnormally highlighted area, and finally, the image is subjected to linear gray stretching to enhance the visual effect of low-luminance details, so that the balance of the dynamic range of the image is improved and the visual quality is enhanced.

[0082] By implementing the method of the present application, when processing a high dynamic range degraded image, the influence of saturated highlight targets and abnormal values on the thermal radiation surface calculation process can be avoided, and before the surface is reconstructed, the selected control points are screened according to their curvature characteristics, so that the phenomenon of using unreasonable control points to affect the correction result is avoided, and meanwhile, the surface is reconstructed by using the weighted interpolation method with irregular control points, so that the interpolation points are only affected by the adjacent control points, and the constraint of the mathematical model is avoided. The method can realize the correction of different types of aerodynamic optical thermal radiation effects, and especially for high dynamic range thermal radiation degraded images with abnormal highlight targets or more noise and targets, the method will have better effect and can improve the signal-to-noise ratio of the image, so as to prepare for the realization of target recognition and detection operation.

[0083] The embodiment of the present application also provides a thermal radiation surface reconstruction and correction method based on curvature flow and control point screening, and the algorithm flow is as shown in Figure 1 The method comprises the following steps:

[0084] S1, pre-processing the image, including gray conversion, normalization and boundary filling;

[0085] S2, calculating the local average curvature and gradient amplitude of each point of the image, iteratively updating the height of each point of the surface by curvature flow operation, and simultaneously combining the maximum iteration number and convergence criterion to adaptively terminate iteration, so as to realize the filtering and smoothing of the sudden detail part on the image surface;

[0086] S3, then the image obtained in step S2 is subjected to morphological top-hat transformation to suppress local abnormal extreme values in the image, thereby extracting a thermal radiation feature image, and down-sampling is performed to obtain thermal radiation curved surface control points;

[0087] S4, a local point set of the thermal radiation curved surface control points is extracted by a radius search method based on Euclidean distance, a local plane is fitted based on principal component analysis, and a local plane normal vector is extracted;

[0088] S5, the difference between the control points and the normal vectors of the domain control points is calculated to obtain Laplace curvature, and low curvature points and boundary points are screened out as final control points according to a set curvature threshold;

[0089] S6, according to the screened control points, the thermal radiation curved surface is reconstructed by Delaunay triangulation interpolation, and local noise and jumps are eliminated by using a Lowess smoothing algorithm, thereby obtaining a continuous and smooth thermal radiation curved surface;

[0090] S7, the reconstructed thermal radiation curved surface is removed from the degraded image, thereby realizing correction of the degraded image, and meanwhile, highlight targets are suppressed and dark parts are enhanced in the corrected image, thereby realizing better display effect.

[0091] In some embodiments, the method in step S1 is specifically: pre-processing a 256x256 size 16-bit degraded image as shown in FIG. 1, including gray scale conversion, normalization and boundary padding, wherein the 16-bit gray scale value of the image data is normalized to 0-1 double-precision floating point type, facilitating subsequent processing and calculation. Figure 2

[0092] In some embodiments, the method in step S2 is specifically: performing iterative processing of the image Z obtained by pre-processing by curvature flow to suppress high-frequency noise and local abnormalities, thereby filtering and smoothing the details of the image surface mutation, while retaining the smooth change trend in the background caused by thermal radiation.

[0093] First, the local average curvature and gradient amplitude H of each point of the three-dimensional curved surface of the image Z are calculated:

[0094]

[0095]

[0096] wherein, H and are matrices containing the average curvature and gradient amplitude of each point of the image Z; , , ​, , , are the first and second order partial derivatives of image Z at each point, since the way of calculating the derivatives is usually based on the difference of the neighborhood in space, i.e. using the 3x3 neighborhood composed of the up, down, left, right and diagonal positions of the pixel to approximate, which means that the local average curvature of a point H is actually calculated by the second order derivative approximation of the pixel gray scale in its local neighborhood, which reflects the bending characteristics of the local area around the point.

[0097] Then, taking each point as a unit, combining the gradient amplitude , the average curvature and the set time step (the embodiment is set to 0.4), the surface height is iteratively updated:

[0098]

[0099] wherein, is the surface of the new iteration, is the current surface (the initial value is the normalized gray scale value of each point in the pre-processed image Z), and the process represents that the height value (i.e. the normalized gray scale value) of each point of the image is changed slightly in each iteration according to its local curvature form. When , it indicates that the local is concave, and the surface height will increase; when , it indicates that the local is convex, and the surface height will decrease; when , it indicates that the local is flat, and the surface height is not updated.

[0100] In addition, numerical truncation needs to be applied at the same time to ensure that the pixel value is kept in the interval , to prevent numerical overflow:

[0101]

[0102] i.e. by calculating the maximum amplitude of the surface change between the adjacent two iterations , when the change is less than the set threshold (the embodiment is set to 1e-5), or the number of iterations reaches the set maximum number of iterations (the embodiment is set to 20), it is considered that the algorithm converges and is terminated in advance, and the image after curvature flow processing is obtained as shown in Figure 3 .

[0103] It should be noted that when the maximum amplitude of the surface change between the adjacent two iterations is less than the set threshold , the change amplitude of all points is also less than the set threshold .

[0104] In some embodiments, the method in step S3 specifically involves: setting a morphological structuring element (in this embodiment, a circular template with a radius of 15) to define the scale of local image operations, and then applying the local operation scale to the image. The TopHat morphological transformation is performed by subtracting the morphological opening operation result from the original image, thereby finding the image... The local extrema regions smaller than the structuring element are identified. The opening operation involves first eroding the image, then dilating it. This removes bright areas larger than the structuring element, retaining only local extrema smaller than the structuring element.

[0105] Then from the image Subtracting this local extremum from the image removes bright targets or anomalous extrema regions that curvature flow operations cannot filter out, resulting in a thermal radiation feature image that retains only the brightness variation trend caused by thermal radiation in the background, free from target interference. Figure 4 As shown, downsampling is performed on a regular uniform grid, that is, sampling at a fixed step size n (set to 0.3 in this embodiment) at a fixed ratio every approximately 10*n pixels at equal intervals in the row and column directions to construct a regular uniform network, i.e., a set of control points for the thermal radiation surface, as shown. Figure 5 As shown.

[0106] In some embodiments, the method in step S4 specifically involves: for each control point of the thermal radiation surface... Choose a radius with its center as r (In this embodiment, it is set to 2) the set of points within the neighborhood. Principal component analysis was performed on the neighborhood point set. Using the geometric center of the neighborhood point set (i.e., the average position of these points) as a reference, the offset of each neighborhood point relative to the center was calculated, and a covariance matrix was constructed. :

[0107]

[0108]

[0109]

[0110] in, Control points and neighboring points The Euclidean distance between them; , represents the geometric center of the neighborhood points, that is, the average position of the neighborhood points; It is the covariance matrix of the local point set, which reflects the spatial distribution characteristics of the local point set in three directions (x, y, z axes).

[0111] Subsequently, the covariance matrix was analyzed. Perform eigenvalue decomposition and take the eigenvector corresponding to the smallest eigenvalue as the control point. Normal vector of the local plane By iterating through each point, we can obtain the set of normal vectors for all points. .

[0112] In some embodiments, the method in step S5 specifically involves: calculating the mean Euclidean difference between the normal vector of each control point and its neighboring normal vectors. That is, the Laplace curvature, which serves as a local curvature index at that point:

[0113]

[0114] According to the preset threshold (In this embodiment, the value is set to 0.7). Points with smaller curvature are selected, while the boundary point set is retained to supplement the edge constraints of the control points, thus obtaining the final control point set of the selected thermal radiation surface. ,like Figure 6 As shown:

[0115]

[0116] Final control point set of thermal radiation surface It includes all Laplace curvatures Less than the curvature threshold control points It also includes the set of boundary points.

[0117] In some embodiments, the method in step S6 specifically involves: processing the final control point set of the selected thermal radiation surface. The Delaunay triangulation method is used to construct a Delaunay triangular mesh with control points as vertices, dividing the entire plane into several non-overlapping triangles.

[0118] For each grid point in the regular grid (a 256×256 grid of degraded image size) In other words, interpolation points (i.e., all pixels in the original degraded image that do not correspond to the final control points) are located in the Delaunay triangle based on their coordinates. Then, using the coordinates and grayscale values ​​of the three vertices of the triangle containing that point, a weighted average is calculated based on the area to determine the grayscale value of that grid point. :

[0119]

[0120] in, This represents the grayscale values ​​of the three vertices of the triangle containing that point. The interpolation weight corresponding to each vertex, The area of the sub-triangle formed by the point and two vertices of the triangle.

[0121] After traversing all the interpolation points, the corresponding gray matrix of the entire regular grid is generated, forming a continuous surface , that is, the preliminary thermal radiation surface. Moreover, through interpolation, the size of the surface is also restored to the original image size.

[0122] Finally, the surface is subjected to Lowess smoothing to eliminate local noise and jumps, and a continuous and smooth thermal radiation surface bias is finally obtained, as shown in Figure 7 and Figure 8 . The Lowess smoothing process is as follows:

[0123]

[0124] The formula indicates that the weighted average is calculated using a Gaussian kernel weight within the local neighborhood of each point (set to 5 in this embodiment). Different weights are given to the points in the neighborhood according to the distance from the center point, thereby smoothing the local noise and jumps.

[0125] In some embodiments, the method in step S6 is specifically: removing the calculated thermal radiation surface from the input degraded image, thereby realizing correction of the thermal radiation effect and obtaining a corrected image, as shown in Figure 9 and Figure 10 . Since the high pixel value of the saturated highlight target existing in the image will inhibit the display range of the low brightness area, in order to improve the visualization quality of the corrected image, the brightness range is adjusted and processed. First, a brightness threshold is set (set to 0.7 in this embodiment), and the part with a brightness higher than the threshold is truncated to prevent the local overexposed area from affecting the overall display effect. Then, the low brightness area in the image is enhanced through linear gray stretching, and the gray scale range of [0, 0.5] is stretched to [0, 1], enhancing the display effect of dark details. Finally, the corrected image realizes correction of the thermal radiation effect and optimization of the brightness of the foreground target, as shown in Figure 11 and Figure 12 .

[0126] It should be noted that the steps shown in the above flow or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0127] In combination with Figure 1The method for reconstructing and correcting thermal radiation curved surface based on curvature flow and control point screening of the described embodiments of the present application can be implemented by a computer device. Figure 13 The figure is a schematic diagram of the hardware structure of the computer device of the embodiments of the present application. As shown in the figure, the device can include a processor 301 and a memory 302 storing computer program instructions. Figure 13

[0128] Specifically, the processor 301 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0129] ​The memory 302 can include a mass storage for data or instructions. By way of example, and without limitation, the memory 302 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a Digital Versatile Disk (DVD), a Blu-Ray, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 302 can be removable and / or non-removable (or fixed) as appropriate. The memory 302 can be internal or external as appropriate. In certain embodiments, the memory 302 is a Non-Volatile Memory. In certain embodiments, the memory 302 includes a Read-Only Memory (ROM) and a Random-Access Memory (RAM). The ROM can be a Mask-Programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended Data Output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0130] The memory 302 can be used to store or buffer various data files required for processing and / or communication, and possible computer program instructions executed by the processor 301.

[0131] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any of the above-mentioned embodiments of the heat radiation curved surface reconstruction and correction method based on curvature flow and control point screening.

[0132] In some embodiments, the computer device can further include a communication interface 303 and a bus 300. As shown, the processor 301, the memory 302, and the communication interface 303 are connected through the bus 300 and complete communication with each other. Figure 13

[0133] The communication interface 303 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 303 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations, etc.

[0134] ​Bus 300 includes hardware, software, or both, to couple components of the computer device to each other and to couple components to other components in the environment. Bus 300 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, etc. By way of example and not limitation, bus 300 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, bus 300 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0135] The computer device can execute the thermal radiation surface reconstruction and correction method based on curvature flow and control point screening in the embodiments of the present application, thereby realizing the combination of Figure 1 The described thermal radiation surface reconstruction and correction method based on curvature flow and control point screening.

[0136] Furthermore, in conjunction with the thermal radiation surface reconstruction and correction method based on curvature flow and control point screening in the above embodiments, this invention can be implemented using a computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the thermal radiation surface reconstruction and correction methods based on curvature flow and control point screening in the above embodiments.

[0137] In summary, this invention provides a thermal radiation surface reconstruction and correction method based on curvature flow and control point selection. It can be used to correct images with high dynamic range aero-optical-thermal effects that degrade the image due to saturated bright targets. It avoids the defects of abnormal extreme values ​​affecting the correction quality in traditional correction algorithms, and also avoids the situation where improper selection of control points affects the correction effect in traditional correction methods. It effectively removes the thermal radiation effect of the image and improves the image quality and clarity.

[0138] It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. In addition, depending on the implementation needs, the various steps / components described in this invention can be broken down into more steps / components, or two or more steps / components or parts of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0139] It will be readily understood by those skilled in the art that the above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A method for thermal radiation surface reconstruction and correction based on curvature flow and control point screening, characterized in that, The method comprises: S1, obtaining a thermal radiation degradation image and preprocessing the thermal radiation degradation image; S2, based on the preprocessed image, calculating the local average curvature and gradient amplitude of each point in the graph, iteratively updating the height of each point on the surface through curvature flow operation, and simultaneously combining the maximum iteration number and convergence criterion to adaptively terminate iteration to obtain the curvature flow processed image; S3, performing morphological top-hat transformation on the curvature flow processed image to suppress local abnormal extreme values in the image, extracting a thermal radiation feature image, and down-sampling the thermal radiation feature image to obtain thermal radiation surface control points; S4, determining the local plane normal vector of all control points of the thermal radiation surface; S5, calculating the difference between the local plane normal vector of each control point and the local plane normal vector of its neighborhood control points to obtain the Laplace curvature of the control point, and screening out the control points with a Laplace curvature less than a curvature threshold value to form the final control points of the thermal radiation surface together with the boundary points; S6, according to the final control points of the thermal radiation surface, reconstructing the thermal radiation surface through Delaunay triangulation interpolation to obtain a preliminary thermal radiation surface, and performing Lowess smoothing processing on the preliminary thermal radiation surface to obtain a continuous and smooth thermal radiation surface; S7, subtracting the continuous and smooth thermal radiation surface from the thermal radiation degradation image to correct the thermal radiation degradation image and obtain a corrected image; Wherein, according to the final control points of the thermal radiation surface, the thermal radiation surface is reconstructed through Delaunay triangulation interpolation to obtain a preliminary thermal radiation surface, which comprises: According to the final control points of the thermal radiation surface, a Delaunay triangular mesh is constructed with the final control points as vertices by using the Delaunay triangulation method, and the entire surface is divided into several non-overlapping triangles; For all points in the thermal radiation degradation image, if there is a corresponding final control point, the value of the corresponding final control point is taken; if there is no corresponding final control point, it is taken as an interpolation point; for the interpolation point, determine its belonging triangle, and calculate the gray value of the interpolation point by weighted average according to the area of the triangle and the gray values of the three vertices of the triangle; After traversing all interpolation points, a continuous surface is formed, which is the preliminary thermal radiation surface.

2. The method according to claim 1, wherein, The preprocessing includes gray scale conversion, normalization and boundary filling.

3. The method according to claim 1, wherein, Step S2 specifically comprises: Based on the pre-processed image Z, the local mean curvature and the gradient magnitude H of each point in the graph are calculated : wherein H and are the local mean curvature and gradient magnitude, respectively, of each point in the image Z; , , , , are the first and second partial derivatives, respectively, of each point in the image Z; In units of each point, local mean curvature of the point H and gradient magnitude , iteratively update the surface height: In the formula, is the surface after one iteration update; is the current surface, whose initial value is the value of each point in the preprocessed image Z; is the time step; Calculate the maximum amplitude of surface change between adjacent two iterations, if it is less than a set threshold, or the current iteration number reaches a preset maximum iteration number, then terminate the iteration to obtain the curvature flow processed image; otherwise, based on the iteratively updated surface, recalculate the local average curvature and gradient amplitude of each point in the graph, and continue iteration.

4. The method according to claim 3, wherein, After each iteration update, it is judged whether the value of each point is between 0 and 1, if yes, the original value is kept, if greater than 1, it is set to 1, and if less than 0, it is set to 0.

5. The method of claim 1, wherein, Step S5 specifically comprises: calculating the average of the Euclidean differences between the local planar normal vector of each control point and the local planar normal vectors of its neighborhood control points as a local curvature indicator of this control point: wherein denotes the Laplacian curvature of the control point; denotes the set of neighbor control points of the control point, denotes the number of control points in the set of neighbor control points; denotes the local plane normal vector of the control point, denotes the local plane normal vector of the control point in the set of neighbor control points; According to the set curvature threshold , the control points with Laplace curvature less than the curvature threshold are screened out, and the boundary points are composed of the final control points of the thermal radiation surface wherein is the set of final control points of the thermal radiation surface, including all Laplacian curvature control points with curvature less than a curvature threshold and the set of boundary points.​ 6. The method of claim 1, wherein, The gray value of the interpolation point is calculated as: wherein denotes the value of the interpolation point ; denotes the values of the three vertices of the triangle to which the interpolation point belongs; is the interpolation weight corresponding to each vertex, is the area of the sub-triangle formed by the interpolation point and the remaining two vertices of the triangle.

7. The method of claim 1, wherein the method further comprises: The Lowess smoothing processing is: wherein is the midpoint of the preliminary thermal radiation surface is the value after Lowess smoothing; denotes a point of the local neighborhood of all points , is the radius of the local neighborhood ; denotes the value of the neighborhood point ; denotes the weight of the neighborhood point .

8. The method of claim 1, wherein, Step S7 further comprises: Setting the brightness of the points in the corrected image whose brightness exceeds a brightness threshold to the brightness threshold, and enhancing the low brightness area in the image through linear gray scale stretching.

9. A computer device, comprising: It comprises: A processor and a memory, the memory stores programs or instructions executable on the processor, the programs or instructions are executed by the processor to implement the steps of the method for thermal radiation surface reconstruction and correction based on curvature flow and control point screening according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, A processor and a memory, the memory stores programs or instructions executable on the processor, the programs or instructions are executed by the processor to implement the steps of the method for thermal radiation surface reconstruction and correction based on curvature flow and control point screening according to any one of claims 1 to 8. A processor and a memory, the memory stores programs or instructions executable on the processor, the programs or instructions are executed by the processor to implement the steps of the method for thermal radiation surface reconstruction and correction based on curvature flow and control point screening according to any one of claims 1 to 8.

Images