Composite insulator shed surface aging defect detection device based on multispectral fusion

Abstract

The application relates to the field of industrial vision, and discloses a composite insulator umbrella skirt surface aging defect detection device based on multispectral fusion, which comprises the following steps: controlling a multispectral camera to perform wide-area scanning to obtain an original multispectral data cube; extracting an insulator suspected aging region of interest, obtaining an initial defect category through a visual algorithm, and calculating a three-dimensional surface normal vector; calculating an optimal observation angle based on the normal vector and a pre-stored refractive index, and controlling a locking mechanism to adjust the camera pose; deducing a target characteristic waveband subset according to the initial category, tuning a light filter and a light supplement array to perform secondary targeted collection, and obtaining a narrowband multispectral image; and inputting physical information neural network, and outputting category prediction and physical inversion parameters. The application eliminates mirror reflection noise from the source, overcomes the limitation of fixed waveband blind sampling, penetrates surface stains and false images by using physical constraints, directly inverts the mechanism decay of insulating materials, and significantly reduces the interference of complex environmental factors.

Description

Technical Field

[0001] This invention relates to the field of industrial vision technology, specifically to a device for detecting aging defects on the surface of composite insulator skirts based on multispectral fusion. Background Technology

[0002] Composite insulators are widely used in power transmission and transformation systems. Operating outdoors for extended periods, the surface of the insulator skirts can develop aging defects such as powdering and microcracks due to ultraviolet radiation, high-voltage electric fields, and complex weather conditions. Timely and accurate detection of these aging defects is crucial for maintaining the safe and stable operation of the power grid.

[0003] Currently, non-contact inspection of aging defects in insulators mainly relies on machine vision equipment or conventional multispectral imaging systems. Because the sheds of composite insulators have irregular curved surfaces and are mostly made of polymer materials such as silicone rubber, their surfaces are prone to specular reflection under natural or artificial light. This localized highlighting directly masks the true texture of the shed surface, causing optical acquisition equipment to lose defect features at the source. Furthermore, conventional multispectral inspection equipment typically uses continuous scanning with fixed wavelengths for image acquisition. This method generates a large amount of redundant data and cannot extract differentiated spectral bands for specific aging stages or defect types, resulting in a low signal-to-noise ratio for the target defect signal.

[0004] Furthermore, outdoor insulators are often covered with varying degrees of dirt. Existing deep learning-based visual detection algorithms are prone to misjudging these images because surface dirt and early, slight aging are visually similar. Current data-driven models primarily rely on image pixel features, lacking underlying constraints on the physical mechanisms of photochemical degradation in insulating materials. Faced with complex outdoor light drift and unknown dirt evolution, pure visual algorithms struggle to distinguish between surface deposits and substantial decay of the substrate material, thus reducing the accuracy of aging defect detection and the system's adaptability. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multispectral fusion-based device for detecting aging defects on the surface of composite insulator skirts. This device solves the technical problem that existing multispectral visual inspection systems are hampered by reflective surfaces of insulator skirts, redundant acquisition in fixed bands, and interference from surface contamination, making it difficult to accurately identify the true aging defects of the substrate material.

[0006] To achieve the above objectives, the present invention provides a multispectral fusion-based device for detecting aging defects on the surface of composite insulator skirts, applicable to the field of industrial vision. The device includes a multispectral camera, a mechanism-optical pose tracking and locking mechanism, a tunable filter, an active supplementary lighting array, and a visual intelligent processing center. The visual intelligent processing center is electrically connected to the multispectral camera, the mechanism-optical pose tracking and locking mechanism, the tunable filter, and the active supplementary lighting array. The visual intelligent processing center is configured as follows: The multispectral camera is controlled to perform a wide-area scan to acquire the original multispectral data cube of the insulator; Based on the original multispectral data cube, a region of interest (ROI) suspected of aging is extracted. An AI vision algorithm is used to perform preliminary visual measurements on the ROI suspected of aging to obtain an initial defect category. The three-dimensional surface normal vector of the ROI suspected of aging is then calculated. The optimal observation angle is calculated based on the three-dimensional surface normal vector and the pre-stored physical refractive index of the insulating material. A driving command is generated to control the mechanism-optical pose follower locking mechanism to adjust the pose of the multispectral camera so that the angle between the observation optical axis of the multispectral camera and the three-dimensional surface normal vector satisfies the optimal observation angle. Based on the initial defect category, the target feature band subset is deduced, and the tunable filter and the active fill light array are tuned according to the target feature band subset. The multispectral camera is then controlled to perform secondary targeted acquisition to obtain a customized narrowband multispectral image. The customized narrowband multispectral image is input into a pre-configured physical information neural network, which outputs a defect category prediction containing confidence probability and physical inversion parameters, thereby achieving high-precision visual detection of aging defects in insulators.

[0007] Furthermore, the step of extracting the suspected aging region of interest based on the original multispectral data cube specifically includes: the visual intelligent processing center uses a spatial and spectral joint smoothing operator to eliminate environmental distribution noise in the original multispectral data cube, and performs radiometric calibration correction in combination with standard reflectance calibration coefficients to generate a preprocessed multispectral image, and extracts the suspected aging region of interest based on the preprocessed multispectral image.

[0008] Furthermore, during the calculation of the three-dimensional surface normal vector, the visual intelligent processing center also simultaneously calculates the incident vector of the main light source in the current environment; specifically, this includes: extracting the center point of the suspected aging region of interest, calculating the normal vector at the center point as the three-dimensional surface normal vector, locating the extreme point of the global brightness distribution of the original multispectral data cube to extract the incident vector of the main light source, and using the quotient of the dot product and the L2 norm product of the incident vector of the main light source and the three-dimensional surface normal vector to solve the inverse cosine function to obtain the actual ambient light incident angle, which is used to evaluate the light interference intensity of the current visual detection environment.

[0009] Furthermore, after receiving the driving command, the mechanism-optical pose tracking locking mechanism converts the initial observation vector of the multispectral camera into a target observation vector by solving the rigid body transformation matrix containing the three-dimensional rotation matrix and translation vector, so that the angle between the target observation vector and the three-dimensional surface normal vector is equal to the optimal observation angle, which is the Brewster angle that eliminates the specular reflection component of the insulating material surface.

[0010] Furthermore, the process of deducing the target feature band subset includes: comparing the features of the polymer photochemical degradation spectrum matrix pre-stored in the input device with the initial defect category, selecting a specific number of extremely narrow frequency bands with the maximum information entropy for the initial defect category from the entire band, and fusing them to construct the target feature band subset.

[0011] Furthermore, the global joint optimization loss function of the physical information neural network during training includes a data fitting loss term and a physical consistency constraint loss term. The data fitting loss term is calculated by the cross-entropy between the predicted classification distribution corresponding to the defect category prediction output by the physical information neural network and the real data labels corresponding to the training samples. The physical consistency constraint loss term is calculated by the sum of squares of the differences between the actual observed spectral reflectance under the target feature band subset and the output value of the theoretical optical reflectance physical model. The value of the global joint optimization loss function is equal to the data fitting loss term plus the physical consistency constraint loss term multiplied by a preset balanced weight coefficient. Its core calculation model is expressed as follows:

[0012] in, This represents the value of the global joint optimization loss function. This indicates that the classification distribution is predicted by the model. With real data labels The calculated data fitting loss term, This indicates the preset balance weight coefficient. This represents the physical consistency constraint loss term. The mathematical mechanism of this physical consistency constraint loss term is expressed as follows:

[0013] in, Represents a subset of the target's feature bands. Indicates at wavelength The actual observed spectral reflectance extracted below, This represents a theoretical optical reflection physical model, which is essentially a bidirectional reflection distribution function characterizing the physical laws governing the scattering and absorption of photons inside a polymer insulating medium. The surface chemical component concentration parameter represents the output of the hidden layer of the physical information neural network. The parameter representing the defect erosion depth jointly predicted by the physical information neural network. This represents the optimal observation angle.

[0014] Furthermore, after acquiring the defect category prediction and physical inversion parameters, the visual intelligence processing center constructs a composite aging severity scoring function to calculate the aging state level of the insulator. The composite aging severity scoring function consists of a weighted sum of three dimensions, including: a first penalty component after normalizing and mapping the surface chemical component concentration parameter, an exponential second penalty component constructed with the defect erosion depth parameter as the independent variable, and a third probability component characterizing the confidence probability of the defect category prediction.

[0015] Furthermore, the active supplementary lighting array includes an adaptive spatial-spectral coupling projection unit. During the secondary targeted acquisition, the active supplementary lighting array projects a colored multi-frequency structured light onto the surface of the insulator under test through the adaptive spatial-spectral coupling projection unit. This light is composed of a microscopic topological pattern and a specific narrowband spectrum within the target characteristic band subset, which are interwoven and superimposed to obtain the underlying data with forced alignment between the spatial structure and spectral absorption.

[0016] Furthermore, the visual intelligence processing center is also equipped with a closed-loop adaptive backtracking module; when the internal calculation variance of the physical constraints generated by the physical information neural network in the forward inference stage exceeds a preset threshold, the closed-loop adaptive backtracking module packages the current customized narrowband multispectral image and the corresponding device pose and ambient light parameters into a heterogeneous feature data package, and automatically triggers an online incremental update of the physical information neural network.

[0017] This invention provides a device for detecting aging defects on the surface of composite insulator skirts based on multispectral fusion. It has the following beneficial effects: 1. To address the problem of aging characteristics being obscured by reflections on the curved surface of an umbrella skirt under complex lighting conditions, this invention utilizes a mechanism-optical pose tracking and locking mechanism to actively deflect the camera's optical axis to the optimal observation angle based on the calculated three-dimensional normal vector. This solution directly intervenes in the optical reflection path at the mechanical motion level, eliminating optical noise caused by specular reflections at the data acquisition source.

[0018] 2. To address the issues of redundant interference and weak targeting in fixed-band acquisition, this invention reverse-engineers the target characteristic band based on preliminary screening results and dynamically tunes the filter and supplementary light array to perform secondary targeted acquisition. This closed-loop mechanism of hardware and software collaboration overcomes the limitations of blind acquisition in fixed bands, enabling the front-end hardware to become an adaptive physical probe for specific aging processes, accurately capturing high signal-to-noise ratio defect spectra.

[0019] 3. Addressing the technical bottleneck of easily leading to misjudgments of "foreign objects with the same spectrum" due to surface contamination and early slight aging, this invention embeds the optical reflection laws of polymer materials as physical constraints into a physical information neural network. This design ensures that the model is constrained by the physical laws governing material degradation when processing visual data, effectively penetrating surface contamination artifacts and directly inverting the mechanism of insulating material decay, thus reducing the interference of complex environmental factors on detection accuracy. Attached Figure Description

[0020] Figure 1 This is a system structure block diagram of the present invention; Figure 2 This is a flowchart of the wide-area scanning and data preprocessing mechanism of the present invention; Figure 3 This is a flowchart of the suspected feature coarse screening and three-dimensional geometric pose calculation of the present invention; Figure 4 This is a flowchart of the material optical boundary-based follow-up locking control of the present invention; Figure 5 This is a flowchart of the prior art-driven band extrapolation and targeted secondary acquisition process of the present invention. Figure 6 This is a flowchart illustrating the physical information neural network architecture and optimization deduction process of the present invention. Figure 7 This is a flowchart of the multidimensional quantitative evaluation and closed-loop adaptive evolution mechanism of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see the appendix Figure 1 - Appendix Figure 7 This invention provides a device for detecting aging defects on the surface of composite insulator skirts based on multispectral fusion. The device includes a multispectral camera, a mechanism-optical pose tracking locking mechanism, a tunable filter, an active supplementary light array, and a visual intelligent processing center.

[0023] The multispectral camera is fixedly mounted on the end effector of the mechanistic-optical pose tracking and locking mechanism, enabling multi-degree-of-freedom attitude deflection control in space. In specific industrial applications, the mechanistic-optical pose tracking and locking mechanism can be implemented using a multi-axis servo gimbal or a six-degree-of-freedom industrial robotic arm. The tunable filter is integrated into the optical path input of the multispectral camera for dynamically selecting transmission spectra in specific wavelength bands. The active fill light array is arranged parallel to the field of view axis of the multispectral camera. The visual intelligent processing center is electrically connected to the multispectral camera, the mechanistic-optical pose tracking and locking mechanism, the tunable filter, and the active fill light array via a high-speed data bus, and is responsible for the signaling interaction and data processing of the entire link.

[0024] In this embodiment of the invention, the specific execution logic of wide-area scanning and data preprocessing includes: the visual intelligent processing center controls the multispectral camera to perform wide-area scanning to obtain the original multispectral data cube of the insulator; the spatial and spectral joint smoothing operator is used to eliminate environmental distribution noise in the original multispectral data cube, and radiometric calibration correction is performed in combination with the standard reflectance calibration coefficient to generate a preprocessed multispectral image.

[0025] After system startup, the visual intelligence processing center sends the first-level control command to the multispectral camera. Based on this command, the multispectral camera operates in a preset wideband scanning mode, scanning bands covering the visible, near-infrared, and ultraviolet light ranges, thereby acquiring the original multispectral data cube of the surface of the composite insulator skirt under test. This original multispectral data cube is defined as... ,in, For two-dimensional pixel coordinates in space, For the wavelength parameter of the continuous spectrum.

[0026] In complex industrial environments, uneven illumination and dark current in sensors can lead to severe Gaussian noise and impulse noise superimposed on the acquired data cube. After acquiring the original multispectral data cube, the visual intelligent processing center invokes its internally residing spatial and spectral joint smoothing operator to perform first-order preprocessing. This operator uses a three-dimensional joint filtering kernel to smooth noise in the spatial domain while strictly preserving the gradient information of the insulator skirt edge texture, and maintaining the structural integrity of spectral absorption peaks between adjacent bands in the spectral dimension.

[0027] Subsequently, the visual intelligence processing center incorporates standard reflectance calibration data to perform radiometric calibration correction on the filtered multispectral data. This process precisely maps the dimensionless digital grayscale values ​​acquired by the device into absolute spectral reflectance, reflecting the inherent physical properties of the insulator material.

[0028] The overall computational logic for generating preprocessed multispectral images is represented as follows:

[0029] In this calculation formula, This represents the preprocessed multispectral image output after denoising and calibration. Indicates a joint spatial and spectral smoothing operator; Indicates a specific wavelength The standard reflection calibration coefficient.

[0030] Furthermore, the standard reflection calibration coefficients in the above radiation calibration process are calculated based on the system reference calibration data. Their mathematical relationship is expressed as follows:

[0031] In this formula, Indicates the standard reflective whiteboard at wavelength Theoretical value of absolute reflectance at that location; This represents the quantized value of the spectral response intensity obtained by the multispectral camera in real-time imaging of the standard reflective white board under the same ambient light illumination. This represents the system's inherent dark current background response value captured by the multispectral camera with the light inlet channel closed. By eliminating the dark current background and normalizing it based on standard reflectance, the system establishes a rigorous data foundation for subsequent physical parameter inversion.

[0032] In this embodiment of the invention, in order to accurately strip away complex backgrounds and locate the measurement area of ​​physical defects, the system performs subsequent calculations based on the generated preprocessed multispectral image.

[0033] In this embodiment of the invention, in order to accurately strip away complex backgrounds and locate the measurement area of ​​physical defects, the system extracts the suspected aging region of interest based on the generated preprocessed multispectral image, performs preliminary visual measurement on the suspected aging region of interest using an AI vision algorithm to obtain an initial defect category, and calculates the three-dimensional surface normal vector of the suspected aging region of interest and simultaneously calculates the incident vector of the main light source of the current environment.

[0034] After receiving the preprocessed multispectral image, the visual intelligence processing center inputs it into an internally configured lightweight segmentation network. This network focuses on the multimodal features of the insulator skirt surface, quickly separating the background environment from the defect-free intact areas, and outputting regions of interest containing suspected aging features (such as mild chalking, microcracks, etc.), while simultaneously outputting the initial defect category attributes of the regions of interest.

[0035] The network output is converted into a binary mask matrix, defined as follows: In this mask matrix, the set of pixel coordinates with a value of 1 constitutes the effective computational domain for subsequent precise physical targeting analysis.

[0036] To perform optical reflection physics analysis, geometric interference caused by the irregular curvature of the umbrella skirt must be eliminated. The visual intelligent processing center extracts the geometric center point of the effective mask region as the reference coordinates and calls the image depth estimation algorithm to reconstruct the local three-dimensional surface topology including the mask center point based on the principle of binocular parallax or structured light phase shift.

[0037] Based on the reconstructed 3D curved surface topology, the visual intelligence processing center extracts the spatial normal vector at the center point of the mask through micro-facet cross-section fitting calculation. This vector is defined as the 3D surface normal vector required for subsequent calculations.

[0038] Simultaneously, the visual intelligence processing center extracts the incident vector of the main light source based on the extreme points of the global brightness distribution of the original multispectral data cube. Specifically, the algorithm extracts the coordinates of the extreme point with the strongest specular response as the specular mapping center of the light source by traversing the global brightness gradient of the multispectral data cube in the spatial domain, thereby accurately calculating the relative incident direction of the main light source illuminating the center point of the mask.

[0039] Finally, the visual intelligence processing center constructs a local coordinate system, establishing the geometric projection relationship between the three-dimensional surface normal vector and the incident vector of the main light source. The spatial constraint relationship of the actual ambient light incident angle is expressed as:

[0040] In this geometric constraint formula, Indicates the actual ambient light incidence angle; This represents the normal vector at the center point of the extracted suspected aging region of interest, i.e., the three-dimensional surface normal vector; Represents the incident vector of the main light source in the current environment; operator Represents the dot product operation of two spatial vectors; This represents the L2 norm of the corresponding spatial vector, i.e., the physical magnitude of the vector. The calculation of this angle parameter provides crucial boundary conditions for subsequent identification and avoidance of specular highlight blind spots.

[0041] In this embodiment, in order to eliminate the interference of specular reflection caused by the irregular polymer umbrella-shaped surface at the physical level, the system actively implements mechanical pose feedback locking based on the optical boundary properties of the insulating material.

[0042] In this embodiment of the invention, the mechanism-optical pose tracking and locking control logic includes: calculating the optimal observation angle based on the three-dimensional surface normal vector and the pre-stored physical refractive index of the insulating material, and generating a driving command; controlling the mechanism-optical pose tracking and locking mechanism to convert the initial observation vector of the multispectral camera into a target observation vector by solving a rigid body transformation matrix containing a three-dimensional rotation matrix and a translation vector after receiving the driving command, so that the angle between the target observation vector and the three-dimensional surface normal vector is equal to the optimal observation angle.

[0043] In this invention, the inherent physical refractive index parameter of the composite insulator matrix material is pre-installed inside the visual intelligence processing center. According to Fresnel's law of reflection of electromagnetic waves at different medium interfaces, when the angle between the observation lines meets specific conditions, the specular reflection component of the medium surface is reduced to a minimum, and at this time, the diffuse reflection light information scattered and transmitted from within the material is maximized.

[0044] The system uses the physical refractive index to calculate the ideal optical boundary conditions and establish the optimal observation angle. This optimal observation angle is specifically configured as a Brewster angle capable of eliminating the specular reflection component from the corresponding insulating material surface.

[0045] After determining the optimal observation angle, the visual intelligence processing center acquires the initial observation pose parameters of the multispectral camera. Combining the previously calculated 3D surface normal vectors, the system plans the target motion pose of the camera in the global coordinate system and calculates the spatial rigid body transformation matrix required to transition from the initial pose to the target pose.

[0046] The attitude transformation logic of the spatial displacement of the multispectral camera is expressed as follows:

[0047] In this attitude transformation formula, This represents the initial observation vector of the multispectral camera before it performs mechanical deflection; This represents the target observation vector where the principal optical axis of the camera is located after the pose adjustment; Represents a three-dimensional rotation matrix that drives the camera to undergo spatial deflection; This represents the translation vector that matches the movement of the optical center of the multispectral camera.

[0048] Based on the calculated rigid body transformation matrix, the visual intelligence processing center generates corresponding low-level servo drive commands and sends them to the mechanism-optical pose tracking and locking mechanism. The mechanism-optical pose tracking and locking mechanism responds to these drive commands, controlling the multispectral camera at the end effector to precisely execute spatial translation and angular deflection actions.

[0049] During this deflection process, the camera's final target observation vector is strictly constrained by the following optical boundary conditions:

[0050] In this constraint equation, This represents the optimal observation angle based on the physical refractive index configuration; This represents the adjusted target observation vector from the multispectral camera. Represents the 3D surface normal vector of the region of interest suspected of aging, calculated; operator Represents the inner product operation of spatial vectors; It represents the magnitude of the L2 norm of the corresponding spatial vector.

[0051] Through the aforementioned rigid body transformation and pose locking mechanism, the device forces the camera's imaging optical path to be aligned with the optimal observation channel derived from purely physical laws before the secondary image acquisition action occurs, thus blocking the blinding artifacts caused by complex environmental lighting from the source physical link.

[0052] In this embodiment, to avoid data redundancy and background interference caused by blind acquisition across the entire band, the system performs dynamic frequency band extrapolation and hardware-coordinated targeted acquisition based on the coarse screening results and prior physical knowledge base.

[0053] The system has a pre-stored polymer photochemical degradation spectrum matrix. This spectrum matrix is ​​a priori database constructed based on high-precision spectral scanning results of composite insulator standard samples at different standard aging stages, through data extraction and feature clustering. It fully records the specific spectral absorption patterns of various types of physical defects.

[0054] In this embodiment of the invention, the execution logic of targeted deduction and acquisition includes: deducing a subset of target feature bands based on the initial defect category, tuning the tunable filter and the active supplementary light array according to the subset of target feature bands, controlling the multispectral camera to perform secondary targeted acquisition, and acquiring a customized narrowband multispectral image; wherein, the process of deducing the subset of target feature bands includes: comparing the features of the initial defect category in the polymer photochemical degradation spectrum matrix pre-stored in the device, selecting a specific number of extremely narrow frequency bands with the maximum information entropy for the initial defect category from the entire band, and fusing them to construct the subset of target feature bands.

[0055] The visual intelligence processing center acquires the initial defect category attributes of the suspected aging region of interest output by the lightweight segmentation network, denoted as... The initial defect category attribute. The system inputs the polymer photochemical degradation spectrum matrix for feature querying and mapping. It traverses the global continuous spectral sequence, calculates the information gain and discrimination of each band for the specific type of defect, and then selects a specific number of extremely narrow frequency bands with the maximum information entropy for the initial defect category.

[0056] The aforementioned selected extremely narrow frequency bands are fused to construct a subset of target feature bands, whose logical set expression is as follows:

[0057] In this set expression, This represents a subset of target feature bands derived for the initial defect category; This represents the initial defect category attribute parameter; This indicates the total number of extremely narrow frequency bands extracted through screening; Indicates the first element contained in the set. A discrete characteristic wavelength parameter with high information entropy.

[0058] After acquiring the target feature band subset, the visual intelligence processing center generates hardware collaborative control signals and sends them to the tunable filter and active filler array. Upon receiving the signals, the tunable filter changes the birefringence state of its internal liquid crystal layer, ensuring that its physical transmission passband is precisely aligned with the discrete wavelength parameters contained in the target feature band subset, thus blocking other redundant spectra from entering the photosensitive element.

[0059] Synchronously, the adaptive spatial-spectral coupling projection unit configured inside the active supplementary lighting array responds to control signals. At the instant the multispectral camera is about to perform secondary targeted acquisition, this projection unit emits an active probe light field onto the surface of the insulator being measured. This light field pattern is formed by the interweaving and superposition of a specific microscopic topological geometry pattern with a specific narrowband spectrum within the subset of the target's characteristic wavelength bands, constituting a colored multi-frequency structured light.

[0060] While maintaining the optimal observation angle, the multispectral camera utilizes a high signal-to-noise ratio target region illuminated by colored multi-frequency structured light. Through a tunable filter that has already undergone band switching, it performs photon interception in an extremely narrow frequency band. Through the aforementioned dynamic opto-mechanical-electrical coordinated process, the system ultimately acquires a customized narrowband multispectral image, achieving deep physical alignment between the underlying spatial structural features and specific chemical absorption spectra.

[0061] In this embodiment, to overcome the limitation that pure data-driven models are easily affected by surface contamination, this system introduces explicit optical scattering mechanism constraints into the conventional deep learning architecture, and realizes closed-loop collaboration between data and natural laws by constructing a physical information neural network.

[0062] In this embodiment of the invention, the processing logic based on the physical information neural network includes: inputting the customized narrowband multispectral image into a pre-configured physical information neural network, and outputting defect category predictions containing confidence probabilities and physical inversion parameters, thereby achieving high-precision visual detection; wherein the physical information neural network uses a global joint optimization loss function containing data fitting loss terms and physical consistency constraint loss terms for gradient updates during training.

[0063] The system control unit feeds the acquired customized narrowband multispectral images forward into a pre-configured physical information neural network. The network's topology is expanded based on the feature extraction backbone, with a classification head branch and a regression head branch connected in parallel at the tensor output of its deep features. The classification head branch uses a conventional probability mapping structure to output predicted distribution parameters for defect attributes; the regression head branch forces the output of physical domain-level scalars through linear rectification or smoothing nonlinear activation functions, specifically surface chemical component concentration parameters and defect erosion depth parameters.

[0064] This invention constructs a training mechanism that combines data fitting with physical laws as constraints. The mathematical framework of its global joint optimization loss function is expressed as follows:

[0065] In this total loss calculation model, This represents the value of the global joint optimization loss function generated by the system in each training iteration; This represents the basic data fitting loss term, which is the predicted classification distribution output by the model's classification head. With real data labels The cross-entropy between them is calculated. Represents the preset balance weight coefficient that controls the strength of physical constraints; This represents the physical consistency constraint loss term, used to measure the degree of deviation between the network inference results and the actual physical laws.

[0066] To calculate the aforementioned physical consistency constraint loss term, the visual intelligence processing center is equipped with a theoretical optical reflection physical model, which is essentially a micro-surface bidirectional reflection distribution function that characterizes the scattering and attenuation law of photons entering the interior of a polymer insulating medium.

[0067] The detailed computational derivation of the physical consistency constraint loss term is as follows:

[0068] In this constraint equation, This represents a subset of the target feature bands derived from the previous process; This represents the current discrete wavelength parameter when traversing the subset of target feature bands; This represents the absolute reflectance of the actual observed spectrum captured by a multispectral camera at a specific wavelength. This represents the theoretical optical reflection physical model; This represents the surface chemical component concentration parameter matrix derived from the regression head branch of the physical information neural network; This represents a scalar parameter indicating the erosion depth of surface defects on insulators, jointly predicted by a physical information neural network. This indicates that the optimal observation angle is locked during the second data acquisition. This represents the summation operation of the squared difference using the L2 norm.

[0069] Through the aforementioned global joint optimization mechanism, the physical information neural network must find the optimal weight space that can both fit the visual feature annotations and strictly obey the physical laws of optical scattering when performing error backpropagation updates. This strategy utilizes the absorption mechanism of polymer media for specific spectra as a law barrier, directly preventing the algorithm from mistaking surface-attached dirt artifacts for decay of the matrix material mechanism.

[0070] In this embodiment, in order to achieve quantitative classification of the mechanism decay of insulating materials and ensure the diagnostic robustness of the detection system under long-term complex operating conditions, the system performs multi-dimensional quantitative evaluation and adaptive algorithm evolution based on inversion parameters.

[0071] In this invention, the logic of the multidimensional quantitative evaluation and adaptive evolution mechanism includes: after the visual intelligent processing center obtains the defect category prediction and physical inversion parameters, it constructs a composite aging severity scoring function to calculate the aging state level of the insulator; when the internal calculation variance of the physical constraints generated by the physical information neural network in the forward inference stage exceeds a preset threshold, the closed-loop adaptive backtracking module configured in the visual intelligent processing center packages the current customized narrowband multispectral image and the corresponding device pose and ambient light parameters into a heterogeneous feature data package, and automatically triggers the online incremental update of the physical information neural network.

[0072] After acquiring the defect category prediction, surface chemical component concentration parameter, and defect erosion depth parameter from the physical information neural network output, the visual intelligence processing center calls the built-in composite aging severity scoring function. This function integrates features from three independent dimensions: chemical material composition variation, physical spatial erosion degree, and algorithm classification confidence, to perform a global comprehensive calculation of the insulator's current aging state level.

[0073] The mathematical model of the composite aging severity scoring function is expressed as follows:

[0074] In this comprehensive scoring model, This represents the calculated output of the composite aging severity score function value; Indicates the concentration parameter of surface chemical components The first penalty component obtained after normalization mapping; Indicated by the defect erosion depth parameter The exponential second penalty component is constructed for the independent variable, where This is the preset depth sensitivity coefficient; This represents the third probability component that characterizes the predicted confidence probability of the defect category; , and These are the weighting factors corresponding to each of the above independent components in the global evaluation system.

[0075] In dynamic, long-cycle industrial inspections, the system constantly faces challenges from ambient light drift and the evolution of unknown contamination. To address this, a closed-loop adaptive backtracking module is additionally configured within the visual intelligence processing center. This module monitors in real time the internal calculation variance of physical constraints generated by the physical information neural network during the forward inference process. When the fitting residual between the actual observed spectrum of the input sample and the theoretical optical reflection physical model increases sharply, causing this internal calculation variance to exceed the system's set characterization tolerance threshold, it indicates that the current equipment has encountered novel environmental disturbances or rare aging topologies that exceed the boundaries of the existing prior knowledge base.

[0076] Once the variance boundary threshold is triggered, the closed-loop adaptive backtracking module immediately intervenes, encapsulating the customized narrowband multispectral image that triggered the anomaly, the extracted 3D surface normal vector, and the corresponding spatial illumination environment parameters into an independent heterogeneous feature data package. Subsequently, the visual intelligence processing center injects this heterogeneous feature data package into the dynamic calculation queue and activates the online incremental update program for the physical information neural network. This mechanism forces the network to fine-tune the gradient of local weights based on high-information-entropy edge scene data, enabling the system to achieve adaptive correction and evolution driven by physical laws without human intervention.

[0077] It is important to note that the original multispectral data cubes and related training samples involved in this invention were all collected from industrial composite insulator equipment in power systems. The data sources do not contain any personal privacy information, biometric information, or sensitive geographic mapping data involving national security. The model training and inference process of the physical information neural network in this invention is based solely on the physical / chemical properties of polymer materials, and the algorithm logic does not contain any discriminatory bias against human social attributes. This device and its onboard AI algorithm strictly comply with relevant laws and regulations such as the Data Security Law throughout the entire application implementation process. Its technical purpose is to improve the safety and reliability of industrial power grid operation, which has a positive promoting effect on the public interest and conforms to the ethical compliance standards of the artificial intelligence industry.

[0078] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multispectral fusion-based device for detecting aging defects on the surface of composite insulator skirts, applied in the field of industrial vision, characterized in that... The system includes a multispectral camera, a mechanistic-optical pose tracking and locking mechanism, a tunable filter, an active illumination array, and a visual intelligence processing center. The visual intelligence processing center is electrically connected to the multispectral camera, the mechanistic-optical pose tracking and locking mechanism, the tunable filter, and the active illumination array. The visual intelligence processing center is configured as follows: The multispectral camera is controlled to perform a wide-area scan to acquire the original multispectral data cube of the insulator; Based on the original multispectral data cube, a region of interest (ROI) suspected of aging is extracted. An AI vision algorithm is used to perform preliminary visual measurements on the ROI suspected of aging to obtain an initial defect category. The three-dimensional surface normal vector of the ROI suspected of aging is then calculated. The optimal observation angle is calculated based on the three-dimensional surface normal vector and the pre-stored physical refractive index of the insulating material. A driving command is generated to control the mechanism-optical pose follower locking mechanism to adjust the pose of the multispectral camera so that the angle between the observation optical axis of the multispectral camera and the three-dimensional surface normal vector satisfies the optimal observation angle. Based on the initial defect category, the target feature band subset is deduced, and the tunable filter and the active fill light array are tuned according to the target feature band subset. The multispectral camera is then controlled to perform secondary targeted acquisition to obtain a customized narrowband multispectral image. The customized narrowband multispectral image is input into a pre-configured physical information neural network, which outputs a defect category prediction containing confidence probability and physical inversion parameters, thereby achieving high-precision visual detection of aging defects in insulators.

2. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, The step of extracting the suspected aging region of interest based on the original multispectral data cube specifically includes: the visual intelligent processing center using a spatial and spectral joint smoothing operator to eliminate environmental distribution noise in the original multispectral data cube, and combining it with the standard reflectance calibration coefficient for radiometric calibration correction to generate a preprocessed multispectral image, and extracting the suspected aging region of interest based on the preprocessed multispectral image.

3. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, During the calculation of the three-dimensional surface normal vector, the visual intelligence processing center also simultaneously calculates the incident vector of the main light source in the current environment. Specifically, this includes: extracting the center point of the suspected aging region of interest, calculating the normal vector at the center point as the three-dimensional surface normal vector, locating the extreme point of the global brightness distribution of the original multispectral data cube to extract the incident vector of the main light source, and using the quotient of the dot product and the L2 norm product of the incident vector of the main light source and the three-dimensional surface normal vector to solve the inverse cosine function and obtain the actual ambient light incident angle, which is used to evaluate the light interference intensity of the current visual detection environment.

4. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, After receiving the driving command, the mechanism-optical pose tracking locking mechanism converts the initial observation vector of the multispectral camera into the target observation vector by solving the rigid body transformation matrix containing the three-dimensional rotation matrix and translation vector, so that the angle between the target observation vector and the three-dimensional surface normal vector is equal to the optimal observation angle, which is the Brewster angle that eliminates the specular reflection component of the insulating material surface.

5. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, The process of deducing the target feature band subset includes: inputting the initial defect category into the polymer photochemical degradation spectrum matrix pre-stored in the device for feature comparison, selecting a specific number of extremely narrow frequency bands with the maximum information entropy for the initial defect category from the entire band, and fusing them to construct the target feature band subset.

6. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, The global joint optimization loss function of the physical information neural network during training includes a data fitting loss term and a physical consistency constraint loss term. The data fitting loss term is calculated by the cross-entropy between the predicted classification distribution corresponding to the defect category prediction output by the physical information neural network and the real data labels corresponding to the training samples. The physical consistency constraint loss term is calculated by the sum of squares of the differences between the actual observed spectral reflectance under the target feature band subset and the output value of the theoretical optical reflectance physical model. The value of the global joint optimization loss function is equal to the data fitting loss term plus the physical consistency constraint loss term multiplied by a preset balanced weight coefficient.

7. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 6, characterized in that, The theoretical optical reflection physical model is a two-way reflection distribution function that characterizes the physical laws of photon scattering and absorption inside a polymer insulating medium; the physical inversion parameters include surface chemical component concentration parameters and defect erosion depth parameters; the independent variable input features of the two-way reflection distribution function include the wavelength corresponding to the target feature band subset, the surface chemical component concentration parameters derived from the hidden layer of the physical information neural network, the defect erosion depth parameters jointly predicted by the physical information neural network, and the optimal observation angle.

8. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 7, characterized in that, After acquiring the defect category prediction and physical inversion parameters, the visual intelligence processing center constructs a composite aging severity scoring function to calculate the aging state level of the insulator. The composite aging severity scoring function consists of a weighted sum of three dimensions: a first penalty component after normalizing and mapping the surface chemical component concentration parameter, an exponential second penalty component constructed with the defect erosion depth parameter as the independent variable, and a third probability component characterizing the prediction confidence probability of the defect category.

9. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, The active supplementary lighting array includes an adaptive spatial-spectral coupling projection unit. During the secondary targeted acquisition, the active supplementary lighting array projects a colored multi-frequency structured light onto the surface of the insulator under test through the adaptive spatial-spectral coupling projection unit. This light is composed of a microscopic topological pattern and a specific narrowband spectrum within the target characteristic band subset, which are interwoven and superimposed to obtain the underlying data with forced alignment between the spatial structure and spectral absorption.

10. The multispectral fusion-based composite insulator skirt surface aging defect detection device according to claim 1, characterized in that, The visual intelligence processing center is also equipped with a closed-loop adaptive backtracking module. When the internal calculation variance of the physical constraints generated by the physical information neural network in the forward inference stage exceeds a preset threshold, the closed-loop adaptive backtracking module packages the current customized narrowband multispectral image and the corresponding device pose and ambient light parameters into a heterogeneous feature data package and automatically triggers an online incremental update of the physical information neural network.

Images