Multi-scale intrinsic empty spectral feature extraction method and system based on relative total variation

Abstract

The application relates to the technical field of image processing, and discloses a multi-scale intrinsic hyperspectral feature extraction method and system based on relative total variation. The method comprises the following steps: obtaining a representative feature image of an unmanned aerial vehicle (UAV) airborne insulator hyperspectral image after black and white correction and dimension reduction along a spectrum dimension, constructing a relative total variation energy function to extract a multi-scale intrinsic structure image, stacking and fusing spectral features pixel by pixel to construct an intrinsic hyperspectral feature, and outputting the intrinsic hyperspectral feature for insulator surface state detection. The application solves the problems that in insulator hyperspectral feature extraction, glaze texture and defect structure are mixed and difficult to separate, single scale cannot consider coarse and fine defects, and hyperspectral information dimensions are inconsistent and directly spliced to be unbalanced, and improves the distinguishability and discrimination accuracy of the extracted intrinsic hyperspectral feature on the states of insulator surface contamination, aging and damage.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for extracting multi-scale intrinsic spatial spectrum features based on relative total variation. Background Technology

[0002] UAV-borne hyperspectral imaging has been widely used in the inspection of insulators of power lines. After push-broom imaging of the insulators, the airborne hyperspectral imager obtains a three-dimensional data cube containing two-dimensional space and one-dimensional wavelength. Each pixel has reflectance readings in hundreds of consecutive bands, containing both spatial morphology and spectral information. Compared with ordinary visible light imaging, it is more conducive to distinguishing dirt, aging and damage on the surface of the insulator. In order to extract discriminative features, the current approach usually performs principal component transformation or band selection along the spectral dimension to reduce the dimensionality. Then, the spatial texture is extracted from the dimensionality reduction result using traditional gradient operators, edge detection or single-scale filtering. The spectral information and spatial information are then simply concatenated and fed into a classifier to distinguish the surface conditions.

[0003] Existing technologies have several shortcomings in extracting surface features of insulators. The glaze surface of insulators itself contains a large number of fine textures and reflective spots unrelated to defects. Traditional gradient operators and edge detection cannot distinguish whether the gradient comes from the edge of a real defect or from the glaze texture. Both are retained, and the texture is mistakenly identified as a defect during discrimination. Single-scale filtering can only match one spatial span. If the scale is too small, the overall outline of the umbrella skirt and large areas of dirt will be fragmented. If the scale is too large, pinpoint damage and fine lines will be smoothed out. It is difficult to simultaneously take into account multiple defects of varying thicknesses. Spectral features and spatial features come from the reflectance domain and gradient domain, respectively. Their numerical magnitudes and physical meanings are different. After simple concatenation, the larger value overwhelms the other, and the discrimination is dominated by the difference in magnitude rather than the difference in the actual state.

[0004] The aforementioned problem of mixed texture and defect gradients requires the extraction process to separate the structure from the texture based on the consistency of gradient direction within the neighborhood, rather than simply considering gradient magnitude. This leads to the technical challenge of constructing a metric that responds to texture and structure in opposite directions to suppress glaze texture and preserve the main defect structure. Beyond this metric, the vast differences in the range of insulator surface defects from fine lines to skirt contours mean that a single scale cannot cover everything. This leads to the technical challenge of how to make the same metric respond separately in a set of spatial ranges from small to large, obtaining a multi-level structural expression that is complementary from fine to coarse. Even with this multi-level structural expression, the multi-scale structural information from the gradient domain still needs to be integrated with the spectral information from the reflectance domain for discrimination. However, directly splicing the two with inconsistent dimensions will result in imbalance. This further leads to the technical challenge of how to unify the multi-scale structural information and spectral information to the same comparable scale and splice them into a dimensionally consistent intrinsic spatial-spectral feature that balances both types of information. Summary of the Invention

[0005] This application provides a multi-scale intrinsic spatial spectrum feature extraction method and system based on relative total variation, which solves the problems of difficult differentiation between glaze texture and defect structure, inability of a single scale to take into account both coarse and fine defects, and imbalance caused by direct splicing of inconsistent spatial spectrum information dimensions in the hyperspectral feature extraction of insulators. It improves the distinguishability and accuracy of the extracted intrinsic spatial spectrum features in identifying the surface dirt, aging and damage of insulators.

[0006] Firstly, this application provides a method for extracting multi-scale intrinsic spatial spectrum features based on relative total variation, wherein the method includes: Step S1: Acquire hyperspectral images of insulators collected by UAV, perform black and white correction preprocessing, reduce the dimensionality of all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. Step S2: For each representative feature image, construct a relative total variation energy function constrained by the separation of structure and texture. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, extract the intrinsic structure image that preserves the main structure while suppressing texture and noise. Adjust the smoothing parameter and window scale in the relative total variation energy function to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. Step S3: Stack the intrinsic structure images of each representative feature image at each scale pixel by pixel along the spectral dimension, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. Step S4: Output the intrinsic spatial spectrum features as the extraction result for the detection of the surface condition of the insulator.

[0007] Secondly, this application provides a multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation, the multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation includes: The dimension reduction module is used to acquire hyperspectral images of insulators collected by UAVs. After black and white correction preprocessing, dimension reduction is performed on all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. The extraction module is used to construct a relative total variation energy function constrained by the separation of structure and texture for each representative feature image. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, the intrinsic structure image that preserves the main structure while suppressing texture and noise is extracted. The smoothing parameter and window scale in the relative total variation energy function are adjusted to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. The fusion module is used to stack the intrinsic structure images of each representative feature image at each scale along the spectral dimension pixel by pixel, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. The output module is used to output the intrinsic spatial spectral features as the extraction result for the detection of the surface condition of the insulator.

[0008] Thirdly, a multi-scale intrinsic spatial spectrum feature extraction device based on relative total variation is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory to cause the multi-scale intrinsic spatial spectrum feature extraction device based on relative total variation to execute the above-described multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation.

[0009] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the above-described multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation.

[0010] The technical solution provided in this application has achieved practical and beneficial results in the specific field of hyperspectral surface condition detection of insulators. Dimensionality reduction is first performed along the spectral dimension on all bands after black-and-white correction, resulting in several representative feature images that are uncorrelated and carry the main energy. This compresses hundreds of highly redundant original bands, which amplify each other's noise, into a few images with concentrated information. Subsequent structure extraction is performed only on these representative feature images, avoiding redundant calculations caused by band-by-band processing and reducing the interference of weak signal band noise on structure discrimination. For each representative feature image, a relative total variation energy function constrained by the separation of structure and texture is constructed. The intrinsic structure image is extracted by solving the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window. The key contribution of this technique is that it shifts the discrimination criterion from the magnitude of the gradient to the consistency of the gradient direction in the neighborhood. The cancellation of positive and negative gradients in the texture area makes the ratio too large and thus is suppressed, while the consistent direction of the defect edges makes the ratio too small and thus is preserved. Therefore, the fine texture and reflective spots inherent in the insulator glaze are erased, while the main structure of real defects such as dirt boundaries, aging blocks and broken contours are completely preserved, thus resolving the problem of the difficulty in distinguishing between texture and defect gradients from the source.

[0011] Based on this, by adjusting the smoothing parameter and window scale in the relative total variation energy function, multi-scale intrinsic structure images reflecting structural information at different scales are obtained. The same separation metric responds to different spatial ranges from small to large. Small scales retain pinpoint damage and fine lines, while large scales outline umbrella-like contours and large areas of dirt. Various defects of varying thickness are thus properly represented, compensating for the shortcomings of single-scale approaches. The intrinsic structure images of each representative feature image at each scale are stacked pixel by pixel along the spectral dimension and then fused with spectral features to construct intrinsic spatial-spectral features. This allows the multi-level structural information from the gradient domain and the spectral distribution from the reflectance domain to be juxtaposed on the same pixel. Spatial morphology and spectral fingerprints corroborate each other, avoiding the imbalance caused by directly splicing two types of information with vastly different magnitudes, which would suppress each other. In this specific application, the directional sensitivity of the relative total variation metric perfectly matches the imaging characteristics of the insulator glaze surface, which has dense texture and sparse defects. The combination of multi-scale and spatial-spectral fusion also meets the inspection needs of defects with large spans and diverse states. It is these algorithm features and the adaptation to the insulator surface condition detection scenario that together support the distinguishability and reliability of this application in terms of dirt, aging and damage. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of an embodiment of the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation in this application. Detailed Implementation

[0014] This application provides a method and system for multi-scale intrinsic spatial spectrum feature extraction based on relative total variation. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation in this application includes: Step S1: Acquire hyperspectral images of insulators collected by UAV, perform black and white correction preprocessing, reduce the dimensionality of all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. Specifically, the raw data obtained by airborne hyperspectral imagers typically consists of one to two hundred consecutive bands in the wavelength dimension. Adjacent bands have highly similar gray levels, making direct computation redundant and amplifying noise. Principal component transformation (PCM) is performed along the spectral dimension to reduce this redundancy. After removing the mean from the spectral vector of each pixel, the overall image covariance matrix is ​​calculated. Eigenvalue decomposition is then performed on the covariance matrix, and the eigenvalues ​​are arranged from largest to smallest. The variance contribution rate of each principal component is calculated, and the top few principal components whose cumulative contribution rate reaches a set threshold are selected. The original high-dimensional data is projected onto the directions of these principal components, resulting in a significantly smaller number of representative feature images than the original bands. Each representative feature image is a two-dimensional grayscale image of the same size as the original image, carrying the spatial information with the highest energy and no correlation from the original data. All of these images constitute the spatial-spectral data to be processed.

[0016] Step S2: For each representative feature image, construct a relative total variation energy function constrained by the separation of structure and texture. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, extract the intrinsic structure image that preserves the main structure while suppressing texture and noise. Adjust the smoothing parameter and window scale in the relative total variation energy function to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. Specifically, for each representative feature image, the relative total variation energy function is solved. The energy function consists of two parts: a fidelity term and a regularization term. The fidelity term constrains the output to not deviate from the input image, while the regularization term is the ratio of the total variation of the window to the intrinsic variation of the window. A Gaussian window is opened with each pixel as the center. The horizontal and vertical gradients of each point within the window are weighted according to their Gaussian distance from the center, and the absolute values ​​are accumulated to obtain the total variation of the window. The signed summation and the absolute value of the summation are then used to obtain the intrinsic variation of the window. In texture regions, the gradient directions within the window are chaotic, with positive and negative gradients canceling each other out, resulting in a small intrinsic variation and a large ratio. The regularization term strongly suppresses this. In edge regions, the gradient directions are consistent, the two gradients are close, and the ratio is small. The regularization term hardly affects the edges. Iterative reweighted least squares is used to repeatedly update the weights and solve the linear equation system. After convergence, the output retains the main outline and removes fine textures, resulting in an intrinsic structure image. The smoothing parameter controls the weight of the regularization term relative to the fidelity term; a larger value results in a flatter output and a coarser structure. The Gaussian window scale controls the spatial range of the statistical gradient; a larger scale results in a larger identified structure. By applying a set of smoothing parameters and window scales from small to large, a family of multi-scale intrinsic structure images from fine to coarse is obtained.

[0017] Step S3: Stack the intrinsic structure images of each representative feature image at each scale pixel by pixel along the spectral dimension, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. Specifically, assuming eight representative feature images at four scales, each pixel corresponds to 32 intrinsic structure values ​​(8 x 4) on the structural side. These values ​​are arranged into a multi-scale structural feature vector according to a fixed order of spectral band and scale. Simultaneously, the values ​​of the same pixel along the spectral dimension in the spatial spectral data to be processed are arranged into a spectral feature vector. These two types of vectors respectively characterize the spatial morphology and spectral distribution of the surface. The structural values ​​come from the gradient domain, and the spectral values ​​come from the reflection intensity. Their dimensions and numerical ranges are different. After subtracting their own mean from each vector and dividing by their own standard deviation, and normalizing them to the same dimensionless scale, the two vectors are concatenated to obtain the pixel-by-pixel intrinsic spatial spectral features. The entire image is then compiled into a feature cube with the number of channels equal to the sum of the structural dimension and the spectral dimension.

[0018] Step S4: Output the intrinsic spatial spectrum features as the extraction result for the detection of the surface condition of the insulator.

[0019] Specifically, each pixel of the intrinsic spatial spectral feature cube is a feature vector that simultaneously contains spatial structure and spectral information. By expanding the entire cube pixel by pixel and sending it to the discrimination stage, the state of the insulator surface can be distinguished point by point, such as normal glaze, attached dirt, glaze aging, and exposed glaze. The state annotation map with the same size as the original image is output, thus completing the detection of the insulator surface state.

[0020] In one specific embodiment, step S1 includes: Push-broom imaging of insulators of collector lines was performed using an airborne hyperspectral imager mounted on a drone to obtain raw hyperspectral data. Under the same environment as when acquiring the original hyperspectral data, the full white calibration data of the standard white board was acquired, and the full black calibration data under the lens occlusion state was acquired. The full white calibration data and the full black calibration data were used to perform black and white correction on the original hyperspectral data to eliminate the device's dark current and noise in the weak light intensity band, and obtain the corrected hyperspectral image. The corrected hyperspectral image is subjected to principal component transformation along the spectral dimension. The top principal components whose cumulative contribution rate reaches a set threshold are selected as representative feature images to obtain the spatial spectrum data to be processed, which consists of several representative feature images.

[0021] Specifically, the airborne hyperspectral imager scans line by line along the flight direction, imaging the insulator of the collector line as a three-dimensional data cube consisting of a two-dimensional space and a one-dimensional wavelength. Each pixel has a raw grayscale reading across one or two hundred consecutive bands. The raw readings contain detector dark current and uneven device response, and cannot directly reflect the true reflectivity. Therefore, under the same illumination, integration time, and lens parameters as when photographing the insulator, a frame of full-white calibration data is first acquired by pointing the image at a standard white board, and then a frame of full-black calibration data is acquired by covering the lens. The two frames of calibration data correspond one-to-one with the insulator data in each band. The original grayscale of the insulator is subtracted from the calibration grayscale of the all-black band in the same band to deduct the dark current, and then divided by the difference between the calibration grayscale of the all-white and all-black bands in the same band to normalize the sensor gain. The quotient is the corrected relative reflectivity, which falls between zero and one. In bands with weak light intensity, the difference between all-white and all-black is very small, and the noise is amplified when divided. For these bands with too small a difference, they are directly eliminated or the weight is reduced to obtain a corrected hyperspectral image that eliminates dark current and suppresses noise in weak signal bands.

[0022] Even after correction, the hyperspectral image still has one to two hundred bands in the wavelength dimension. Adjacent bands have highly similar reflectance, resulting in high redundancy and amplifying random noise. Principal component transformation is applied along the spectral dimension to reduce this redundancy. The spectral reading of each pixel is considered as a vector of one to two hundred dimensions. The mean of all pixel vectors in the entire image is calculated, and the mean is removed pixel by pixel. The covariance between bands is then calculated to obtain a square matrix of covariance with dimensions equal to the number of bands. The matrix elements reflect the degree of co-variance fluctuation between any two bands. Eigenvalue decomposition of the covariance matrix yields a set of eigenvalues ​​and mutually orthogonal eigenvectors. The eigenvalues ​​represent the data variance along the corresponding direction. The eigenvalues ​​are sorted from largest to smallest. Each eigenvalue is divided by the sum of all eigenvalues ​​to calculate the variance contribution rate of a single principal component. These are accumulated from largest to smallest until the cumulative contribution rate reaches 99.5%, at which point the eigenvectors are truncated. The first few eigenvectors before this point are used as projection directions, and the mean-removed spectral vectors are projected onto these directions. Each projection result corresponds to a principal component. The values ​​of the same principal component at each pixel in the whole image are rearranged back to two dimensions according to their original spatial positions, thus obtaining a representative feature image of the same size as the original image. The number of such images is determined by the number of principal components taken, usually only a few to a dozen. All of them constitute unrelated spatial spectrum data to be processed, which carries the main energy.

[0023] In one specific embodiment, in step S2, the relative total variation energy function is composed of a weighted sum of a data fidelity term and a relative total variation regularization term; The relative total variation regularization term is represented by the ratio of the total variation of the window in the pixel neighborhood to the intrinsic variation of the window, where the total variation of the window is the weighted sum of the gradient magnitudes in the neighborhood, and the intrinsic variation of the window is the magnitude of the weighted sum of the gradients in the neighborhood. This makes the texture region have a larger ratio because the positive and negative gradients cancel each other out, and the structural edges have a smaller ratio because the gradient directions are consistent. With the goal of making the intrinsic structure image approximate the original representative feature image while minimizing the ratio, the relative total variation energy function is solved by iterative reweighted least squares method to obtain the intrinsic structure image that suppresses texture and noise and preserves the main structure.

[0024] Specifically, the relative total variation energy function is constructed pixel-by-pixel for each representative feature image. It is obtained by weighting and summing the data fidelity term and the relative total variation regularization term according to the smoothing parameter. The goal is to find an output image that is as close as possible to the original representative feature image while minimizing the regularization term; this output image is the intrinsic structure image. The data fidelity term measures the sum of squares of the pixel-by-pixel gray-level differences between the intrinsic structure image and the original representative feature image, constraining the output to not deviate from the original content. The relative total variation regularization term is represented at each pixel by dividing the total variation of the window within its neighborhood window by the intrinsic variation of the window. The algorithm for the total variation of the window is to calculate the horizontal and vertical gray-level gradients for each neighboring point within the window centered on the current pixel and take their absolute values. The absolute values ​​are weighted according to the Gaussian distance from the neighboring point to the center and then accumulated to reflect the total amount of gray-level fluctuations within the window. Regardless of the direction of the gradient, only its magnitude is accumulated. The window-based intrinsic variation algorithm first sums the horizontal and vertical gradients within the same window with the same sign and Gaussian weights, then takes the absolute value of the sum. Gradients in opposite directions cancel each other out during the summation phase. The smoothing parameter is the weight of the regularization term in the total energy. A larger value indicates a stronger bias towards suppressing the regularization term, resulting in a smoother output and a coarser preserved structure; a smaller value indicates a closer resemblance to the original image and the preservation of more details.

[0025] Within the texture area window, the gradient directions are chaotic, alternating between positive and negative. The total variation of the window is larger due to taking absolute values ​​one by one, while the inherent variation of the window is smaller due to the mutual cancellation of signed summation. Dividing the two results in an oversized ratio. The regularization term applies strong suppression to such areas, smoothing out texture and fine noise. At the structural edges, the gradient directions within the window are basically consistent. Taking absolute values ​​first and then accumulating them, as well as taking signed summation first and then taking absolute values, yield similar results with a smaller ratio. The regularization term remains almost unchanged, preserving the main outline and edges. This ratio-type regularization term is nonlinear with respect to the image to be determined. It is solved by iterative reweighted least squares. In each round, the total variation and intrinsic variation of the window at each pixel are estimated using the image from the previous round. Based on this, a set of spatial adaptive weights are calculated after linearizing the ratio. The weights of texture regions are large and the weights of edge regions are small. A very small positive number is superimposed on the denominator of the division of the weights to prevent numerical overflow. After fixing this set of weights, the energy function degenerates into a sparse linear equation system with the image to be determined as the unknown. The solution yields a new round of image estimation. The weights are then updated with the new image to enter the next round. This process is repeated until the difference between the images of two adjacent rounds is less than the set tolerance and converges. The final result is the intrinsic structure image that suppresses texture and noise and preserves the main structure.

[0026] In one specific embodiment, both the total variation of the window and the inherent variation of the window are calculated within a Gaussian window centered on the current pixel, and the spatial scale of the Gaussian window determines the coarseness of the extracted structure. When constructing multi-scale intrinsic structure images, a set of Gaussian window spatial scales and a corresponding set of smoothing parameters are set in ascending order. The larger the scale and the larger the smoothing parameters, the coarser the extracted structure, thus obtaining multi-scale intrinsic structure images arranged from fine to coarse.

[0027] Specifically, the total variation and intrinsic variation of the window are not simply accumulated within an equally weighted square neighborhood, but are calculated in a weighted manner within a Gaussian window centered on the current pixel. Each neighboring point within the window is first substituted into the Gaussian function according to its Euclidean distance to the center pixel to obtain a weight. The closer the distance, the closer the weight is to one; the farther the distance, the faster the weight decays to near zero. The absolute value accumulation and signed accumulation of the gradients are multiplied by this weight before summing. The spatial scale of the Gaussian window is the standard deviation of the Gaussian function. When the standard deviation is set small, only a few points immediately adjacent to the current pixel receive considerable weight, and the statistical gray-level fluctuations are limited to a very small spatial span. Thus, the gradients of fine textures and small-sized defects are included in the ratio judgment, and the output intrinsic structure image retains fine structures. When the standard deviation is set large, more distant neighbors also receive non-negligible weights, and the statistical range is extended to a larger spatial span. Small-sized fluctuations are smoothed in the large-scale weighted average, and only gray-level changes with a sufficiently large span dominate the ratio, and the output image retains only coarse structures.

[0028] When constructing multi-scale intrinsic structure images, a set of Gaussian window spatial scales are set in ascending order. Adjacent scales are magnified step by step at a fixed ratio, such as taking a set of standard deviations that double step by step, like three pixels, six pixels, twelve pixels, and twenty-four pixels. This ensures that the structural span dominated by each of the four scales successively covers fine lines, small spots, umbrella skirt edges, and finally the main body of the insulator. The spans between levels do not overlap too much or leave gaps. Each scale is also paired with a smoothing parameter that changes in the same direction. When the scale is small, the smoothing parameter is set to a small value to weaken the regularization term, making the output closer to the original image and retaining more details. When the scale is large, the smoothing parameter is set to a large value to strengthen the regularization term, making the output flatter and leaving only coarse structures. In the first intrinsic structure image obtained by solving with the minimum scale and the minimum smoothing parameter, small-span undulations such as fine glaze lines and needle-like damage are clearly distinguishable. In the last image obtained by solving with the maximum scale and the maximum smoothing parameter, the fine lines are completely erased, leaving only coarse-span structures such as the umbrella skirt outline and large areas of dirt. Substituting the two ends confirms the direction that the larger the scale and smoothing parameter, the coarser the structure remains. The two middle levels transition sequentially. The four images arranged from small to large scale constitute a multi-scale intrinsic structure image from fine to coarse.

[0029] In one specific embodiment, in step S3: For each pixel, the intrinsic structure values ​​of the pixel in each representative feature image and at each scale are arranged in order of spectral band and scale to form a multi-scale structural feature vector that characterizes the spatial structure. The original spectral vector of the pixel in the spectral dimension or its normalized result is used as the spectral feature vector. The multi-scale structural feature vector and the spectral feature vector are concatenated and then normalized to obtain the intrinsic spatial-spectral features that simultaneously characterize the spatial structure and spectral distribution.

[0030] Specifically, for each pixel, its intrinsic structure values ​​across all representative feature images and scales are extracted and arranged in a column according to a pre-agreed spectral band and scale order. If there are eight representative feature images and four scales, then each pixel yields eight by four, totaling thirty-two intrinsic structure values. These are arranged with spectral bands as the outer layer and scales as the inner layer; that is, the structure values ​​from the first representative feature image (four scales) are placed first, followed by the four from the second, and so on up to the eighth image. This results in a thirty-two-dimensional multi-scale structure feature vector, where each component corresponds to a specific spectral band and scale. All pixels in the entire image follow the same arrangement rule, ensuring that the same dimension represents the same spectral band and scale combination at any pixel. The spectral values ​​of the same pixel are taken in a separate column. This can be achieved either by directly arranging the vector using the corrected relative reflectance of each band, or by first normalizing the spectral vector by its modulus to eliminate overall brightness differences. The resulting spectral feature vector has a dimension equal to the number of participating bands or the number of principal components, characterizing the spectral distribution of the pixel.

[0031] The multi-scale structural feature vectors originate from the gradient domain, and their values ​​fluctuate significantly depending on the edge strength. The spectral feature vectors, on the other hand, originate from reflection intensity, with values ​​concentrated around zero to one. Since their dimensions and numerical amplitudes are inconsistent, directly concatenating them end-to-end for discrimination would cause the larger structural component to overwhelm the spectral component, leading to an imbalance. Therefore, a uniform normalization process is applied after concatenation. Concatenation involves sequentially linking the 32-dimensional structural vector and the spectral vector to form a longer vector. Normalization is performed component-by-component, calculating the mean and standard deviation of each component across all pixels in the entire image. Subtracting the mean from the component and dividing by its standard deviation maps each component to a dimensionless scale with a zero mean and consistent dispersion. This brings the structural and spectral segments into the same comparable range, eliminating numerical magnitude differences while preserving the relative fluctuations within each component. The long vector after component-wise normalization is the intrinsic spatial spectral feature of the pixel. By restoring the intrinsic spatial spectral features of each pixel in the entire image to their original spatial positions, a feature cube is formed that has the number of channels equal to the sum of the structural dimension and the spectral dimension, and simultaneously carries spatial structural information and spectral distribution information. If a pixel is missing some structural values ​​due to being located at the edge of the image, it is supplemented with the corresponding component of its nearest effective neighbor pixel before participating in normalization, ensuring that the dimensions of each intrinsic spatial spectral feature are uniform.

[0032] In one specific embodiment, the number of representative feature images selected is eight, and the number of scales of the multi-scale intrinsic structure images in step S2 is four.

[0033] In one specific embodiment, step S4 includes: The intrinsic spatial spectrum features are input into a pre-trained classifier or defect detection model to determine the category of the insulator surface region pixel by pixel. Based on the discrimination results, the system identifies the dirt, aging, or damage status of the insulator surface and outputs the detection results of the insulator surface status.

[0034] Specifically, the intrinsic spatial spectral feature cube is decomposed pixel by pixel according to its spatial location. Each pixel corresponds to a feature vector with a length equal to the sum of the structural dimension and the spectral dimension. The pixels of the entire image are arranged into a two-dimensional matrix with pixels as rows and feature components as columns, and fed into a pre-trained classifier or defect detection model. The model parameters are determined by training with labeled samples before inference. The labeled samples used for training are the intrinsic spatial spectral features of several hyperspectral images of insulators calculated using the same process described above. Each pixel is manually labeled with one of four true categories: normal glaze, attached dirt, glaze aging, and exposed clay. During the training phase, these feature vectors, along with their true category labels, are fed into the model in pairs. The model calculates the discrimination response for each category based on the input features, compares it with the true labels to obtain the discrimination error, and repeatedly adjusts the internal parameters in the direction of reducing the error until the error converges. After that, the parameters are fixed. During the inference phase, the model independently calculates the response value of each of the four categories to which each decomposed pixel feature vector belongs. The category with the largest response value is taken as the discrimination category of that pixel. After pixel-by-pixel processing, each pixel obtains a definite category assignment.

[0035] Four categories are bound to the physical state of the insulator surface during the training and labeling phase. Normal glaze corresponds to pixels with intact glaze, smooth spectrum, and no abnormal absorption. Attached dirt corresponds to pixels with accumulated dust or salt spray, overall low reflectivity, and patchy undulations in the structural domain. Glaze aging corresponds to pixels with glaze layer deterioration after long-term operation and shift in reflection characteristics in specific bands. Damage and exposed body correspond to pixels with glaze peeling off and exposing the body, and strong gradient abrupt changes in the edge structural domain. The category labels given by the model are translated into specific surface states through this correspondence. By restoring the category labels obtained from pixel-by-pixel discrimination to their original spatial positions, a state labeling map of the same size as the input hyperspectral image of the insulator is obtained, with each pixel representing one of the four states. On the map, the dirty area, aging area, and damaged area are displayed in contiguous patches, with the normal area as the background. This state labeling map represents the detection result of the insulator surface state. Inspectors use it to locate the surface area where defective pixels are located and determine the overall health status of the insulator.

[0036] The above describes the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation in the embodiments of this application. The following describes the multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation in the embodiments of this application. One embodiment of the multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation in the embodiments of this application includes: The dimension reduction module is used to acquire hyperspectral images of insulators collected by UAVs. After black and white correction preprocessing, dimension reduction is performed on all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. The extraction module is used to construct a relative total variation energy function constrained by the separation of structure and texture for each representative feature image. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, the intrinsic structure image that preserves the main structure while suppressing texture and noise is extracted. The smoothing parameter and window scale in the relative total variation energy function are adjusted to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. The fusion module is used to stack the intrinsic structure images of each representative feature image at each scale along the spectral dimension pixel by pixel, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. The output module is used to output the intrinsic spatial spectral features as the extraction result for the detection of the surface condition of the insulator.

[0037] This invention also provides a multi-scale intrinsic spatial spectrum feature extraction device based on relative total variation, which can be a server. The device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the above-described method.

[0038] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation.

[0039] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0040] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a multi-scale intrinsic spatial spectrum feature extraction device based on relative total variation (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0041] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting multi-scale intrinsic spatial spectrum features based on relative total variation, characterized in that, The method includes: Step S1: Acquire hyperspectral images of insulators collected by UAV, perform black and white correction preprocessing, reduce the dimensionality of all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. Step S2: For each representative feature image, construct a relative total variation energy function constrained by the separation of structure and texture. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, extract the intrinsic structure image that preserves the main structure while suppressing texture and noise. Adjust the smoothing parameter and window scale in the relative total variation energy function to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. Step S3: Stack the intrinsic structure images of each representative feature image at each scale pixel by pixel along the spectral dimension, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. Step S4: Output the intrinsic spatial spectrum features as the extraction result for the detection of the surface condition of the insulator.

2. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 1, characterized in that, Step S1 includes: Push-broom imaging of insulators of collector lines was performed using an airborne hyperspectral imager mounted on a drone to obtain raw hyperspectral data. Under the same environment as when acquiring the original hyperspectral data, the full white calibration data of the standard white board was acquired, and the full black calibration data under the lens occlusion state was acquired. The full white calibration data and the full black calibration data were used to perform black and white correction on the original hyperspectral data to eliminate the device's dark current and noise in the weak light intensity band, and obtain the corrected hyperspectral image. The corrected hyperspectral image is subjected to principal component transformation along the spectral dimension. The top principal components whose cumulative contribution rate reaches a set threshold are selected as representative feature images to obtain the spatial spectrum data to be processed, which consists of several representative feature images.

3. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 1, characterized in that, In step S2, the relative total variation energy function is composed of a weighted sum of a data fidelity term and a relative total variation regularization term. The relative total variation regularization term is represented by the ratio of the total variation of the window in the pixel neighborhood to the intrinsic variation of the window, where the total variation of the window is the weighted sum of the gradient magnitudes in the neighborhood, and the intrinsic variation of the window is the magnitude of the weighted sum of the gradients in the neighborhood. This makes the texture region have a larger ratio because the positive and negative gradients cancel each other out, and the structural edges have a smaller ratio because the gradient directions are consistent. With the goal of making the intrinsic structure image approximate the original representative feature image while minimizing the ratio, the relative total variation energy function is solved by iterative reweighted least squares method to obtain the intrinsic structure image that suppresses texture and noise and preserves the main structure.

4. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 3, characterized in that, Both the total variation of the window and the inherent variation of the window are calculated within a Gaussian window centered on the current pixel. The spatial scale of the Gaussian window determines the coarseness of the extracted structure. When constructing multi-scale intrinsic structure images, a set of Gaussian window spatial scales and a corresponding set of smoothing parameters are set in ascending order. The larger the scale and the larger the smoothing parameters, the coarser the extracted structure, thus obtaining multi-scale intrinsic structure images arranged from fine to coarse.

5. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 1, characterized in that, In step S3: For each pixel, the intrinsic structure values ​​of the pixel in each representative feature image and at each scale are arranged in order of spectral band and scale to form a multi-scale structural feature vector that characterizes the spatial structure. The original spectral vector of the pixel in the spectral dimension or its normalized result is used as the spectral feature vector. The multi-scale structural feature vector and the spectral feature vector are concatenated and then normalized to obtain the intrinsic spatial-spectral features that simultaneously characterize the spatial structure and spectral distribution.

6. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 1, characterized in that, The number of representative feature images selected in step S1 is eight, and the number of scales of the multi-scale intrinsic structure images in step S2 is four.

7. The multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation according to claim 1, characterized in that, Step S4 includes: The intrinsic spatial spectrum features are input into a pre-trained classifier or defect detection model to determine the category of the insulator surface region pixel by pixel. Based on the discrimination results, the system identifies the dirt, aging, or damage status of the insulator surface and outputs the detection results of the insulator surface status.

8. A multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation, characterized in that, For implementing the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation as described in any one of claims 1-7, the multi-scale intrinsic spatial spectrum feature extraction system based on relative total variation comprises: The dimension reduction module is used to acquire hyperspectral images of insulators collected by UAVs. After black and white correction preprocessing, dimension reduction is performed on all bands along the spectral dimension to obtain spatial spectrum data to be processed, which consists of several representative feature images. The extraction module is used to construct a relative total variation energy function constrained by the separation of structure and texture for each representative feature image. By solving for the optimal solution that minimizes the ratio of the total variation of the window to the intrinsic variation of the window, the intrinsic structure image that preserves the main structure while suppressing texture and noise is extracted. The smoothing parameter and window scale in the relative total variation energy function are adjusted to obtain a multi-scale intrinsic structure image that reflects structural information at different scales. The fusion module is used to stack the intrinsic structure images of each representative feature image at each scale along the spectral dimension pixel by pixel, and fuse them with the spectral features extracted from the spatial spectral data to be processed in the spectral dimension to construct intrinsic spatial spectral features that simultaneously characterize spatial structure and spectral distribution. The output module is used to output the intrinsic spatial spectral features as the extraction result for the detection of the surface condition of the insulator.

9. A multi-scale intrinsic spatial spectrum feature extraction device based on relative total variation, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it causes the processor to perform the multi-scale intrinsic spatial spectrum feature extraction method based on relative total variation as described in any one of claims 1 to 7.

Images