Power equipment monitoring and early warning system based on intelligent inspection

Abstract

The invention discloses a power equipment monitoring and early warning system based on intelligent patrol, and relates to the technical field of power system intelligent monitoring and safety early warning, the system fuses visible light, infrared, ultraviolet, acoustics, vibration and other multi-source signals, innovatively encodes multi-modal features in a complex manner, realizes high-speed and high-dimensional feature extraction through an optical convolution chip, and realizes high-precision and high-dimensional feature extraction. And the early perception capability of hidden dangers is greatly improved. The adaptive complex neural network is introduced, the model structure and parameter dynamic optimization, self-learning and self-evolution functions are achieved, the fault recognition accuracy is remarkably improved, and false alarm and missing alarm are reduced. The system supports cloud side end distributed collaboration, data real-time convergence and multi-scene visualization, has hierarchical deployment and online adaptive capabilities, and adapts to intelligent operation and maintenance requirements of multiple types of power stations. The system has the advantages of high performance, high intelligence, easiness in integration, expandability and the like, is prominent in innovativeness, and can be widely applied to intelligent monitoring and operation and maintenance in electric power and related fields.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring and safety early warning of power systems, and in particular to an electric power equipment monitoring and early warning system based on intelligent patrol. Background Art

[0002] As power systems continue to expand and become more intelligent, safe equipment operation and early detection of hidden dangers have become core requirements for operations and maintenance. Existing power equipment monitoring and early warning systems often rely on single sensors or local image analysis, making it difficult to achieve comprehensive, real-time, and highly sensitive intelligent monitoring.

[0003] Chinese invention patent CN114627388B discloses a device and method for detecting foreign objects on power transmission lines. This system automatically identifies foreign objects through image acquisition and local edge processing, and uploads alarm signals to a cloud server. This solution effectively reduces network bandwidth pressure and energy consumption, improving the intelligence of power transmission line inspections. However, this system primarily focuses on detecting foreign objects on transmission lines, with a single function. It struggles to achieve deep integration and early warning of the multi-dimensional status of the equipment itself (such as overheating, discharge, looseness, and vibration). Furthermore, relying solely on a single modality (image) is susceptible to external interference such as weather and light, limiting its ability to detect potential hazards.

[0004] Chinese invention patent CN106559653B discloses a safety monitoring and early warning system for near-electrical operations at power construction sites. This system uses intelligent image processing and wireless communication technology to achieve safe distance monitoring and dynamic alarms within the construction area. This solution can effectively ensure personal safety in special working conditions such as high-altitude operations, and has the advantages of simple deployment and strong environmental adaptability. However, this system primarily monitors the position of people and objects in construction scenarios and does not involve automatic identification and active early warning of the operating status of power equipment, complex hidden dangers, and trending faults. Furthermore, it relies heavily on static image segmentation and safe distance determination, failing to meet the requirements for multimodal, full-lifecycle intelligent monitoring of equipment.

[0005] In summary, the above design achieves basic anomaly identification and alarm through local intelligent processing and image analysis, and improves the automation level of the power system. However, it still has the following limitations: the monitoring dimension is limited, and a single mode is mainly used, which makes it difficult to capture deep anomalies of multiple types and multiple physical quantities of the equipment itself (such as partial discharge, micro-vibration, chronic degradation, etc.); the ability to identify trend hazards and early minor anomalies is weak, and it is impossible to achieve full life cycle status perception and prediction; the system scalability and intelligent upgrade capabilities are insufficient, and there is a lack of self-learning, continuous model optimization and multi-site cloud-edge collaboration capabilities; real-time and anti-interference performance are limited, and the ability to adapt to environmental changes (such as severe weather and complex working conditions) is not strong. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and propose an intelligent patrol-based power equipment monitoring and early warning system to address the above-mentioned problems. This intelligent power monitoring and early warning system integrates multimodal intelligent patrols, deep fusion of multi-source complex signals, efficient acceleration of optical convolution, adaptive AI dynamic optimization, and cloud-based distributed collaborative self-learning. This system achieves comprehensive, highly sensitive, robust, and continuously optimized health monitoring and proactive risk management of power equipment, significantly improving the intelligent operation and maintenance level and inherent safety of the power system.

[0007] The purpose of the present invention is to achieve the following technical solutions: a power equipment monitoring and early warning system based on intelligent patrol, comprising: Multimodal sensor terminals are used to synchronously collect raw signals from multiple sources, including visible light, infrared, ultraviolet, acoustics, and vibration, at substations or transmission lines, and to uniformly timestamp different types of signals. The data preprocessing and complex mapping module is used to perform amplitude normalization, phase unwrapping, Hilbert transform or other time-frequency analysis on the collected multi-source signals, and convert each type of signal into a complex signal stream containing amplitude components and phase components; The complex-valued optical convolution acceleration unit includes a microcomb laser, an optical splitter, a fine spectrum shaping module, and a multi-channel Mach-Zehnder electro-optical modulator. The fine spectrum shaping module independently adjusts the amplitude and / or phase of each optical comb frequency line according to the convolution kernel weight, physically maps the convolution kernel weight to the optical carrier to achieve weight loading, and modulates the real and imaginary parts of the input complex signal onto different wavelength channels respectively. The input signal and weight are placed in a dispersion-compensating fiber delay array. Through delay configuration, each channel forms a time-stepping relationship, achieving sliding window convolution superposition of the input signal stream and the weight matrix in the physical domain, and outputting multiple parallel convolution results. Photoelectric detection module, used to synchronously collect the convolution results of each channel and output the complete complex convolution results through differential processing; The edge AI processing module is used to splice the convolution results according to the spatial and channel dimensions to form a multi-dimensional complex feature tensor, and then perform feature fusion, anomaly identification and risk classification, and upload the analysis results to the cloud platform; A cloud-based big data analysis and management platform that aggregates historical complex features and equipment health records from multiple sites to enable trend analysis, fault prediction, and remote expert-assisted decision-making.

[0008] The multimodal sensing terminal includes infrared thermal imagers and ultraviolet corona detectors arranged in a linear or planar array. The array acoustic sensor covers the range of 20Hz to 30kHz. Each sensor ensures spatiotemporal data coordination through a clock synchronization mechanism.

[0009] The data preprocessing and complex mapping module includes a dedicated signal processing chip that can use time-frequency analysis on the harmonic characteristics of acoustic and vibration signals, encode signal intensity as mode, harmonic frequency shift or transient characteristics as phase, achieve complex high-sensitivity encoding of local anomalies, and automatically distribute abnormal high-frequency signals to independent data channels.

[0010] The complex-valued optical convolution acceleration unit uses an on-chip integrated microring resonator array to generate multi-wavelength comb lasers. After spectral shaping, the convolution kernel weights correspond to odd and even wavelength channels with real and imaginary parts respectively. The real and imaginary parts of the input complex signal are modulated in parallel to the corresponding optical carriers through multiple Mach-Zehnder electro-optical modulators, achieving a one-to-one correspondence between weights and signals in the optical domain.

[0011] During the convolution operation, after the optical signal passes through the dispersion-compensating fiber delay array, each wavelength channel forms a time step in turn, realizing the sliding of the convolution window on the input data stream. The output of all channels is synchronously collected by the photoelectric detection module and differentiated by a balanced photoelectric detector to realize the output of the complex convolution result with positive and negative weights.

[0012] The edge AI processing module includes an adaptive complex-valued convolutional neural network, which has the ability to automatically adjust the number of convolution layer cores, activation function type and feature fusion strategy, and can dynamically optimize model parameters based on device operation data to improve the accuracy of anomaly recognition.

[0013] When the edge AI processing module identifies an anomaly, it can automatically adjust the alarm threshold and inspection frequency, and upload relevant abnormal samples and judgment criteria to the cloud-based big data analysis and management platform for continuous model optimization and expert-assisted decision-making.

[0014] The cloud platform supports real-time aggregation and multi-scenario visualization of multi-terminal data, and has functions such as equipment health big data management, risk trend prediction, fault sequence playback and regional collaborative early warning. It also supports remote access and interaction from mobile terminals or web pages.

[0015] The system supports multi-level distributed deployment and modular expansion, and can flexibly adapt to the actual application needs of different power sites, substations or transmission lines.

[0016] The system has online self-learning capabilities and can continuously correct complex neural network model parameters, convolution weights and early warning criteria based on actual inspection history, achieving continuous adaptive optimization for different equipment and environments.

[0017] The beneficial effects of the present invention are: 1. Multiple signals, including visible light, infrared, ultraviolet, acoustic, and vibration, are uniformly encoded in the complex domain and convolutional feature fusion is implemented at the physical level. This multimodal complex fusion approach enables fine-grained identification of equipment operating status, not only detecting traditional amplitude anomalies but also accurately sensing hidden defects such as subtle phase shifts, harmonic characteristics, and transient disturbances. This is particularly suitable for early fault detection and trend monitoring in complex operating conditions such as partial discharge, poor contact, mechanical looseness, hot spots, and environmental noise interference.

[0018] 2. Utilizing a complex-valued optical convolution accelerator, it can perform high-dimensional sliding convolutions on dozens or even hundreds of channels in nanoseconds to microseconds, significantly exceeding the throughput limits of purely electronic convolution operations. In scenarios where multiple sites and terminals are online simultaneously, the system can seamlessly perform convolution analysis and feature extraction on large-scale, long-term multimodal signals, effectively enabling large-scale, synchronized health monitoring of equipment. A physical sliding window mechanism further enhances convolution efficiency, enabling seamless capture of fault signals in both time and space.

[0019] 3. The edge AI processing module incorporates an adaptive complex neural network that not only automatically adjusts convolution kernels, activation functions, and fusion strategies based on operational data, but also features continuous self-learning and dynamic optimization capabilities. The system automatically optimizes alarm thresholds and inspection frequencies based on device history, inspection results, and environmental changes, enabling intelligent prediction of risk trends and self-balancing of sensitivity, significantly reducing the risk of human misjudgment, operational intervention, and data traffic pressure. Automatic closed-loop collection of abnormal samples and cloud-based integration allow the system to evolve with field realities, continuously improving recognition accuracy and generalization capabilities.

[0020] 4. The cloud platform supports real-time data aggregation from multiple sites and terminals, regional collaborative risk management, and multi-scenario health visualization. The system features three-level distributed deployment and modular scalability, supporting standalone sites, regional centralized control, and full-network cloud models to meet the specific needs of various substations, transmission lines, and power companies. Any equipment or site upgrade, expansion, or migration does not require a complete system reconfiguration, resulting in strong engineering adaptability and significantly reduced expansion and operation and maintenance costs.

[0021] 5. Operations and maintenance personnel and experts can remotely access equipment lifecycle health records, abnormal trend curves, fault playback animations, and judgment tracing processes at any time through the app / website, achieving transparent and traceable information across all scenarios. Once a regional or network-wide risk is identified, the system automatically conducts multi-site coordinated analysis and batch alerts, enhancing proactive and coordinated operations and maintenance responses, and supporting integrated intelligent operations and maintenance.

[0022] 6. The cloud platform automatically aggregates network-wide inspection samples and criteria. Based on global optimization and model consistency goals, it periodically trains and distributes next-generation complex AI models and optimal parameters, supporting full-scenario adaptation across devices, environments, and fault types. It also supports tracking the "aging curve" of equipment throughout its lifecycle and self-learning and adjusting alarm strategies, effectively reducing missed detections and false alarms and promoting the continuous evolution of the intelligent power operation and maintenance system.

[0023] 7. The solution is based entirely on standard industrial sensors, mainstream optical communication and signal processing hardware, and an open AI platform. It allows for modular deployment and phased expansion, facilitating integration with existing power automation, dispatching, and asset management systems. It offers simplified maintenance and flexible upgrades, significantly reducing the engineering barriers and costs of both new and retrofit operations and maintenance projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a timing diagram of the present invention; Figure 2 This is a system interaction diagram of the present invention. DETAILED DESCRIPTION

[0025] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0026] It is to be noted that the directions of "left", "right", "up", "down", "front", "back", "inside" and "outside" in the following schemes are all relative directions and are not listed here one by one.

[0027] Example 1 like Figure 1 and Figure 2 As shown, this embodiment discloses a multimodal intelligent monitoring and early warning system for power equipment based on a complex-valued optical convolution accelerator. It is applicable to critical power infrastructure such as substations, transmission lines, and switchgear. The system consists of the following functional units: a multimodal sensor terminal, a data preprocessing and complex mapping module, a complex-valued optical convolution accelerator, a photoelectric detection module, an edge AI processing module, and a cloud-based big data analysis and management platform.

[0028] Multimodal sensing terminals utilize linear or area array infrared thermal imagers (e.g., 640×480 pixels, with a temperature measurement accuracy of ±2°C) and are deployed in key heat-spot locations such as transformers, high-voltage switches, and busbars. Ultraviolet corona detectors utilize highly sensitive UV C-band detectors (wavelength 180–280 nm) and are deployed at high-voltage insulation and conductor connection points for early detection of partial discharge. Array acoustic sensors utilize a MEMS microphone array (1664 elements) covering a wide frequency range of 20 Hz to 30 kHz and are located within substation cabinets, switch housings, and beneath insulators to detect partial discharge, noise, and mechanical impact. Vibration acceleration sensors utilize highly sensitive IEPE accelerometers and are affixed to key mechanical structures such as transformer bodies and switch brackets to capture signals of equipment vibration, impact, and looseness. Visible light cameras (1080p or higher) are used for on-site environmental and equipment surface inspections, assisting with visual identification.

[0029] All sensor signals are input to a dedicated controller with a built-in temperature-compensated crystal oscillator or GPS timing module to achieve cross-modal unified clock synchronization better than 1ms.

[0030] The controller periodically sends sampling / trigger commands (the typical sampling period can be set to 1 to 10 seconds, and high-frequency sampling in milliseconds is supported in special situations). All data packets carry precise timestamps to facilitate subsequent data fusion and traceability.

[0031] The data preprocessing and complex number mapping module adopts FPGA+high-speed AD / DA module or high-performance DSP system, which has the ability to process multiple signals in parallel and cache at high speed.

[0032] All analog signals are filtered by hardware low-pass / band-pass filtering and combined with soft algorithms (such as median filtering and detrending algorithms) to eliminate electromagnetic interference and on-site power frequency background. The amplitudes are normalized to the range of 0 to 1 to facilitate cross-modal comparison.

[0033] Signal feature extraction and complex coding: Short-time Fourier transform (STFT) is used to analyze the main frequency band and transient signals of acoustic / vibration signals, and energy distribution, main harmonics, instantaneous power, etc. are extracted in real time.

[0034] The envelope amplitude and instantaneous phase of the signal are obtained by Hilbert transform, which are used as the complex modulus and phase respectively, to obtain a complex characteristic sequence with a time resolution better than 10ms.

[0035] Automatically assign independent multiple channels to detected abnormal events such as shocks and high-frequency pulses to improve subsequent recognition sensitivity.

[0036] Infrared / ultraviolet image signals are statistically analyzed using sliding window partitions to extract physical quantities such as mean, extreme values, and variance from significant areas of the image, such as temperature rise and discharge. Combined with pixel temporal variations, the temporal-spatial features are encoded into a complex sequence (amplitude represents signal intensity, and instantaneous rate of change represents phase) using methods such as the continuous wavelet transform.

[0037] Visible light signals use visual processing algorithms such as OpenCV to detect the surface status and appearance damage of key equipment, and abnormal areas are converted into independent complex feature channels.

[0038] All modal features are output as a complex data stream with a unified format. Each channel of data contains metadata such as [acquisition time, spatial position, amplitude, phase, and abnormality mark].

[0039] The complex-valued optical convolution acceleration unit uses an integrated silicon-based microring resonator array to output a multi-wavelength comb laser with a free spectral range (FSR) of typically 25100 GHz (supporting more than 1040 wavelength channels).

[0040] The laser output is divided into multiple paths (such as 2 or 4 paths) by a low-loss optical splitter and enters an independent fine spectrum shaping module respectively.

[0041] Fine spectrum shaping modules (such as AWG arrays and MEMS optical tunable filters) apply amplitude and / or phase modulation to each wavelength line separately, and perform physical mapping strictly according to the convolution kernel weight matrix in the neural network.

[0042] Supports real-time weight adjustment to facilitate switching between different monitoring tasks and devices.

[0043] Signal modulation and complex multiplication are implemented: The real and imaginary components of the complex input signal are modulated onto odd and even wavelength channels, respectively, using independent Mach-Zehnder electro-optical modulators (MZMs). The MZM modulation rate can reach 10–50 GHz. The input component and weight component of each wavelength channel are directly multiplied within the same channel using optical intensity or coherent interferometry. For the four-quadrant multiplication of complex convolution (Re{X}×Re{W}, Im{X}×Im{W}, Re{X}×Im{W}, and Im{X}×Re{W}), multiple MZMs can be used in parallel, or the input signal can be recombined via an optical combiner structure, with the corresponding complex sub-terms then being implemented in subsequent channels.

[0044] Sliding window convolution mapping: All weighted and input-signal modulated optical signals pass sequentially through a dispersion-compensating fiber delay array. Each channel is set with a different optical path delay to achieve time stepping, simulate a digital sliding convolution window, and output all convolution results in parallel. The output terminal can simultaneously obtain complex convolution features at different spatial and temporal locations.

[0045] The photodetection module uses highly sensitive InGaAs or Si photodetectors (bandwidth 10-50 GHz), with each convolution output channel equipped with an independent detector. Balanced photodetectors sample both positive and negative weighted channels. Hardware differential operations directly obtain the positive and negative components of the complex convolution, improving anomaly detection sensitivity and noise immunity. A synchronous sampling card (typically 1-10 GSps sampling rate) digitizes all outputs and features a multi-channel timing alignment buffer to ensure fully parallel, high-bandwidth data transmission.

[0046] The edge AI processing module uses an ARM SoC, X86 embedded host, or GPU edge computing device, with efficient tensor computing and storage capabilities. All convolution results are spliced ​​into a multi-dimensional complex feature tensor (e.g., a five-dimensional tensor of [position × convolution kernel × time × modulus × phase]) based on time, space, and channel information.

[0047] Intelligent algorithms run multi-layer convolutional neural networks (supporting complex weights and activation functions) to automatically cluster tensor features, segment anomalies, and identify types. Combining empirical rules and threshold models, they assess health status levels, identify fault types (such as overheating, partial discharge, looseness, and insulation degradation), and generate graded warnings. Edge self-learning is supported, dynamically fine-tuning model weights based on historical samples to improve adaptability to complex operating conditions. Key features, logs, and warning results are packaged and sent to the cloud. Local operations personnel can also query device status on-site via the interface or app.

[0048] The cloud-based big data analysis and management platform adopts multi-site integration: integrating all historical complex features, health assessments and alarm logs uploaded by multiple substations, lines, switches and other sites, and unifying database management.

[0049] The cloud-based deep learning and statistical analysis module models network-wide device operational data to identify abnormal trends, cross-site similarities, and periodic hidden dangers. This module integrates external data such as meteorological, environmental, and load data to provide more comprehensive device health management and risk prediction. Experts can remotely access single- and multi-site health records, trend curves, and abnormal event details. The system automatically generates health assessment reports, inspection recommendations, maintenance work orders, and emergency response plans, all accessible 24 / 7 via the app and web.

[0050] Working process Typical application scenarios (taking substation high-voltage switch monitoring as an example): The multimodal sensor automatically collects infrared temperature distribution, ultraviolet partial discharge images, acoustic pulses / noise, vibration signals, and visible light surface images according to the set cycle.

[0051] The controller uniformly issues sampling instructions, and all original signals are synchronously timestamped with high-precision timestamps. After anti-interference filtering and normalization, they flow into the signal processing chip.

[0052] Acoustic and vibration signals are processed through Hilbert transform and STFT to obtain amplitude and phase components. Special events such as shocks and partial discharges are separated into independent channels. After infrared / ultraviolet image processing, the features are converted into complex codes.

[0053] Multiple complex signals are fed into the optical convolution accelerator chip in parallel, and weights can be switched and loaded online based on typical fault templates such as overheating, discharge, and looseness. After full-channel optical domain sliding convolution processing, the results are output in large batches.

[0054] The photoelectric detection module digitizes all convolution outputs and transmits them in parallel to the edge AI via a high-speed synchronous acquisition card.

[0055] The AI ​​processing module splices tensors, runs recognition algorithms, determines the equipment health level and fault risk type in real time, outputs graded warning results, and generates detailed event logs.

[0056] Logs and warnings are automatically uploaded to the cloud, and cloud models synchronously analyze trends, pushing comprehensive assessment reports and expert advice to the operations and maintenance team.

[0057] The sliding optical convolution and feature fusion formula of multimodal complex signal stream and complex weights are as follows: Multimodal complex signal composite coding, assuming that the k-th acquisition point (position, time, sensor type joint index) corresponds to the multimodal original signal: Among them, MMM is the number of acquisition modes (such as infrared, ultraviolet, acoustic, vibration, etc.), S k (m) is the mth modal sampling value.

[0058] Define the multimodal complex signal as: in: α m ,β m is the cross-modal fusion weight (which can be adaptively adjusted according to the scene and type); Φ k (m) is the instantaneous phase of the mth modal signal (or other deep features such as rate of change, time series envelope, etc.); j is the imaginary unit.

[0059] Note: This can unify the amplitude (intensity) and characteristic change rate / phase information of the multimodal signal into a single complex sequence Z k , which is beneficial to subsequent physical complex convolution.

[0060] Assume that the complex weight template (for example, representing a typical failure or health mode) is: Where L is the convolution window length, each W l =a l +jb l (a l ,b l Real number, amplitude and phase can be adjusted independently).

[0061] Sliding convolution output definition (nth position convolution output): Introducing the "mode-phase interactive gating mechanism" to give the convolution window adaptive capabilities Where σ(⋅) is the Sigmoid gating function, and λ1 and λ2 are learnable coefficients.

[0062] Final weighted complex convolution output: In this way, the output of each position is jointly affected by the complex structure of the current amplitude, phase and weight of the input signal, and the sensitivity to abnormal features is dynamically adjusted by adaptive gating, thereby improving noise resistance and adaptability to non-stationary faults.

[0063] Feature fusion and AI decision-making Output tensor Y n final It can be spliced ​​in multiple dimensions such as space, time, and convolution kernels, and fed into subsequent complex AI models (such as complex convolutional neural networks, or customized complex gated feature fusion networks) to achieve multi-target intelligent analysis such as health grading, trend prediction, and anomaly classification.

[0064] The optical convolution acceleration unit achieves convolution rates at the multi-TOPS level, adapting to real-time, full-coverage monitoring of multiple sites and large-scale power scenarios. Deep, complex features such as amplitude, phase, and harmonics are uniformly mapped, enhancing the ability to identify and track potential hazards such as early discharge, insulation degradation, loosening, and overheating. Dispersion delay combined with multi-wavelength multiplexing enables optical domain sliding window convolution, accurately capturing subtle dynamic changes in equipment operating status and reducing the probability of missed detection. Hardware differential output and AI intelligent judgment greatly enhance anti-interference and fault tolerance in complex operating conditions, automatically adapting to different equipment, environments, and scenarios. Local AI enables real-time response, while the cloud supports multi-source big data trend prediction and expert remote assistance, enabling intelligent early warning and closed-loop management. All modules are based on mature industrial components and mainstream signal processing / communication / optoelectronic hardware, allowing for phased and modular integration and deployment, simplifying maintenance and keeping costs manageable.

[0065] All involved sensors, signal processors, and optical components (micro-comb laser, AWG, MZM, dispersion-compensating optical fiber, and photodetectors) can be selected from mature models on the market.

[0066] Digital processing and AI algorithms can be implemented based on mainstream industrial computing platforms such as FPGA, embedded ARM, GPU, and the engineering integration solution has been verified by applications in multiple fields.

[0067] It supports data docking and integration with existing power automation systems (such as SCADA and EMS), facilitating actual deployment and large-scale promotion.

[0068] Example 2: like Figure 1 and Figure 2 As shown in Figure 1, based on Example 1, this example innovates and upgrades the edge AI processing module, focusing on the introduction of an adaptive complex-valued convolutional neural network, and combining dynamic alarms with self-learning mechanisms to achieve a higher level of intelligent operation and maintenance of power equipment. The system structure is the same as before, including multimodal sensor terminals, signal preprocessing, optical convolution accelerators, photoelectric detection, edge AI, cloud platforms, etc., but the edge AI module has the following new functions: Adaptive complex neural network model: The network structure, number of cores, activation function, and feature fusion strategy can be automatically adjusted according to the task, scenario, and data status; Online dynamic optimization and self-learning capabilities: Model parameters and feature weights can be adjusted based on actual operating data to continuously improve recognition accuracy; Automatic adjustment of alarm thresholds and inspection frequency: The AI ​​model automatically optimizes alarm thresholds and equipment inspection cycles based on risk and abnormality probability, improving the timeliness of warnings and reducing false alarms and missed alarms. Abnormal samples and criteria are uploaded to the cloud: The original data, features and AI criteria of abnormal events are automatically collected and uploaded to the cloud for subsequent model retraining and expert assistance.

[0069] The complex feature tensor input of the adaptive complex-valued convolutional neural network structure comes from the multi-channel output of the optical convolution module (see Example 1) and is input into the local AI in the form of a complex tensor.

[0070] The number of convolutional layer kernels is adaptively increased or decreased according to the complexity of input data features and operation and maintenance objectives (such as monitoring equipment type, warning type, historical recognition difficulty, etc.) (for example, through adaptive neural structure search (NAS) or dynamic pruning / addition of kernels during training).

[0071] The activation function is variable and supports traditional complex ReLU, ModReLU, zReLU and other activation functions. The model can automatically select different layers or different types of activation functions to adapt to different feature distributions.

[0072] It supports multi-scale, multi-modal complex feature concatenation, gated fusion, attention mechanism and other methods. The model can dynamically weigh different fusion strategies to improve the discrimination accuracy.

[0073] During long-term field operation, the model continuously collects new data and uses technologies such as transfer learning and online fine-tuning to dynamically correct weight parameters to adapt to equipment aging, environmental changes, and new failure modes.

[0074] Dynamic alarm thresholds and inspection strategy adjustments: After each identification or trend assessment, the AI ​​module compares the warning history with the actual inspection situation. If it finds an increase in the false alarm / missed alarm rate, it automatically fine-tunes the threshold parameters.

[0075] The system supports multi-level warning strategies (such as early warning, risk warning, and emergency alarm). AI can adjust the judgment threshold of each level according to operating trends to ensure that the warning is both sensitive and not excessive.

[0076] The inspection frequency is adaptive. Under normal conditions, the system operates according to the basic inspection cycle. If AI detects that the probability of abnormality is gradually increasing or fluctuating violently, it will automatically increase the sampling frequency and shorten the inspection cycle to capture risk changes in a timely manner. After the risk is eliminated, it will automatically return to the energy-saving inspection frequency to reduce equipment load and data traffic.

[0077] Abnormal sample and judgment upload mechanism: When edge AI identifies suspicious or unseen anomalies (such as low model confidence or features deviating from the normal range), the system automatically packages the original complex signals, feature tensors, AI judgment labels and reasoning process logs in the acquisition window.

[0078] The collected data includes several sampling periods before and after the anomaly occurs to ensure that the abnormal development dynamics are fully captured.

[0079] In addition to raw data, AI can automatically upload detailed criteria used to determine anomalies, including model parameters, convolution kernel features, fusion weights, and gating status. All abnormal samples and criteria are packaged and uploaded to the cloud-based big data platform according to a set strategy (either in real time or in scheduled batches).

[0080] Cloud-based collaborative optimization: The cloud platform receives abnormal samples and criteria, combines multi-site historical data, and performs deep model retraining and expert annotation correction. The cloud platform can then distribute the optimized models, parameters, and thresholds back to each site, enabling system self-evolution and network-wide intelligent level iteration.

[0081] Working process Take the high-voltage switch of a substation as an example: Multimodal data acquisition, preprocessing and complex convolution operations are as in Example 1; The convolution output tensor is input into an adaptive complex neural network, and the model automatically adjusts the network structure (such as the number of convolutional layers, activation function type, and fusion method) according to the current data state and task requirements; The AI ​​model performs health classification and risk trend prediction on the current status of the high-voltage switch and outputs preliminary alarms; If the model detects an anomaly (such as low confidence, blurred judgment boundaries, or abnormally strong features), it automatically fine-tunes the alarm threshold, increases the inspection frequency, and initiates abnormal data collection; Abnormal samples and model criteria are automatically packaged and uploaded to the cloud, where they are aggregated for expert review or subsequent in-depth training. Distribute new models or parameters in the cloud to achieve network-wide self-learning and automatic upgrades for device health management.

[0082] The core structure and gating mechanism of the adaptive complex convolutional neural network. Assume that the input multimodal complex feature tensor is: Where N is the number of space / device nodes, T is the time window length, and F is the feature dimension.

[0083] Adaptive convolution kernel generation (dynamic structure adjustment) For the lth layer of convolution, the number of each convolution kernel is K (l) It can be adaptively updated by the following formula: K0 (l) : Default number of cores : The average change rate of AI output risk scores in the recent period : The rate of change of the model feature distribution entropy (the larger the rate, the more anomalies there are) γ1,γ2: structure adaptive weights σ(⋅): sigmoid gating function, ensuring the output is smooth between 0 and 1 illustrate: When the risk becomes greater or the feature distribution is abnormal, the number of convolution kernels is automatically increased to improve the model's representation power. Otherwise, it is reduced to save energy and reduce consumption.

[0084] Dynamic activation function selection (layer-level variable activation), for the lth layer, dynamically select the activation function: η 1, η2: Adaptive gating parameter Feature fusion gating formula (multi-modal gating fusion) performs gating fusion on the convolution outputs of each modality: in: O i (m) is the convolution output of the iiith spatial position and the mth mode w md ,v md : Modal fusion weight (trainable or adaptive adjustment) illustrate: The contribution of different modalities to the fusion results is automatically gated according to the strength of the current abnormal feature, with weak modalities downgraded and strong modalities enhanced, thereby improving the robustness of abnormality recognition.

[0085] Dynamic alarm threshold and inspection frequency self-learning update Assume the AI ​​output confidence is S risk (t) (t) (0~1), the alarm threshold is θ(t), and the inspection cycle is C(t).

[0086] FPR(t): false alarm rate of the current cycle, FNR(t): missed alarm rate δ1, δ2: Adaptive learning step size (can be fine-tuned based on historical data) If the risk score is high, the inspection cycle will be automatically shortened (increased sampling); after the risk is eliminated, the cycle will be extended to save energy.

[0087] Abnormal sample triggering and upload criteria If S risk (t)>θ(t) That triggers: Automatically package the Z, AI convolution features, activation status, and decision boundaries within the window and upload them to the cloud; The cloud uses abnormal samples for retraining and network-wide model upgrades.

[0088] The system can automatically adapt to a variety of devices, scenarios, and environmental changes without frequent manual intervention, greatly improving operational efficiency and intelligence. Through self-learning and self-adjustment, alarms are more accurate, false alarm / missing alarm rates are extremely low, and inspection frequency is intelligently adjusted based on risk, achieving both efficient energy saving and safety. Any unseen anomalies are automatically captured, uploaded, and shared across the entire network, effectively accumulating an anomaly sample library, continuously optimizing the entire system model, and promoting the evolution of industry knowledge. The complex convolutional network algorithm can be implemented based on mainstream AI frameworks (such as PyTorch and TensorFlow complex extensions). The model structure, parameters, and thresholds can be adaptively adjusted by both hardware and software, facilitating large-scale deployment and upgrades.

[0089] Example 3: like Figure 1 and Figure 2 As shown, this embodiment is based on the aforementioned multimodal intelligent monitoring and early warning system (Examples 1 and 2), and introduces a large-scale multi-site, multi-terminal distributed collaboration mechanism on the cloud platform to support flexible hierarchical deployment and full-process data closed-loop management, thereby improving the system's intelligence level and engineering scalability.

[0090] The system architecture mainly includes: field end (multimodal acquisition, optical convolution and AI edge analysis, see Examples 1 and 2), regional edge nodes (such as local centralized control stations or station-side servers), cloud big data platform, and multi-terminal access (operation and maintenance personnel app, web management terminal, expert decision-making terminal, API open interface). Multi-terminal data is aggregated and visualized in real time on the cloud for multiple scenarios. Edge AI modules at all on-site sites upload health characteristics, alarm events, and inspection raw data packets to the cloud platform in real time or near real time via secure links (5G / Industrial Ethernet / VPN).

[0091] The cloud uses high-performance distributed databases (such as time series database TSDB+NoSQL) to ensure the aggregation and consistency management of high-concurrency, multi-source, and heterogeneous data.

[0092] The data flows through a unified data processing and normalization engine, and all data are tagged with multi-dimensional labels such as site, device, time, and data type to facilitate retrieval and big data analysis.

[0093] Provides a multi-terminal-adaptable visual management interface, supporting desktop web terminals, operation and maintenance mobile apps, expert analysis terminals, and automatic API data flows.

[0094] The interface display includes: health overview of each site, multimodal trend chart of a single device, abnormal time series playback, convolution feature map, risk distribution heat map, device map positioning, etc.

[0095] It supports one-click retrieval of historical data, anomaly tracing trajectory animation, and linked playback of original signal waveform / convolution output / AI judgment results to assist expert decision-making.

[0096] Supports a multi-role account permission system, where operations, management, experts, equipment manufacturers, etc. can customize access to data and functions.

[0097] APP / WEB can receive push alerts, check device status, remotely issue diagnostic instructions or request expert assistance at any time.

[0098] Regional collaboration and hierarchical distributed deployment support multi-level deployment modes, such as: single-site independent operation, regional multi-site centralized control collaboration, and unified cloud-based management of the entire network.

[0099] Each region can add local edge servers as needed to achieve local caching of big data, resume transmission from breakpoints, and on-site response to real-time alarms, ensuring the reliable operation of the system under network fluctuations or high security requirements.

[0100] Supports three-level management of "site-region-cloud", and can be flexibly expanded to any number of sites or new monitoring equipment modules.

[0101] The cloud system supports multi-tenant and multi-project partition management, and flexibly allocates data and analysis resources to different operation and maintenance teams, regional companies or subcontractors.

[0102] The system can dynamically allocate cloud computing, storage, and analysis resources based on the number of devices, data size, and exception rate, improving stability and response speed under large-scale concurrency.

[0103] Cross-region / site collaborative early warning: When a specific type of hidden danger occurs in a certain area (such as lightning strikes, power grid fluctuations, regional equipment aging, etc.), the system automatically associates the data of surrounding sites for trend analysis and early warning coordination.

[0104] It supports hierarchical alarm notification and scheduling, and cross-site hidden dangers can be pushed to relevant responsible persons and experts in batches with one click.

[0105] Through online self-learning and adaptive model optimization, the cloud platform periodically collects abnormal samples, the latest AI judgment criteria and recognition results uploaded by each site, automatically aggregates historical inspection data from the entire network, and builds a large-scale sample library.

[0106] Based on this sample library, the system periodically triggers the retraining of the complex convolutional neural network, including weight fine-tuning, feature gating optimization, and automatic generation of anomaly criteria.

[0107] Transfer learning and federated learning can be combined to achieve full-network model evolution under the conditions of data privacy isolation at different sites.

[0108] Once the new model is completed, it is automatically distributed to each on-site / regional node, enabling adaptive upgrades of local edge AI without the need for manual batch distribution.

[0109] For different equipment types and environmental conditions (such as plateaus, coastal areas, deserts, etc.), the system automatically matches the optimal model parameters, convolution kernels, and alarm thresholds based on historical operations and abnormal distribution.

[0110] It supports self-learning and correction of the "aging curve" during the life cycle of the equipment, long-term tracking of equipment status changes, dynamic adaptation of the optimal judgment criteria, and reduction of false alarms and omissions.

[0111] Working process Take a provincial regional power grid as an example: Each substation, line, switch cabinet and other equipment on-site regularly uploads multimodal acquisition, optical convolution, and AI recognition results.

[0112] Regional nodes aggregate data locally to achieve high-frequency local alarms and emergency responses, and important data and events are uploaded to the cloud simultaneously.

[0113] The cloud aggregates the health characteristics, risk trends, and alarm logs of all devices in real time and automatically archives them into a multi-dimensional database of sites, devices, and events.

[0114] Operations and maintenance personnel can view the regional / network-wide health overview at any time through the mobile app. Once an abnormality is pushed, they can click to replay the entire abnormality evolution process, including multimodal signals and AI recognition process.

[0115] When a general abnormal trend appears in the region, the cloud automatically conducts horizontal analysis of similar sites and equipment history to generate collaborative early warning reports and expert operation and maintenance recommendations.

[0116] Every month / quarter, the cloud automatically triggers network-wide model retraining to generate a new generation of complex neural network models, which are then distributed to each site to achieve the evolution of intelligent operation and maintenance capabilities.

[0117] Distributed cloud-based adaptive multi-site model optimization and intelligent push, multi-site data aggregation and global feature aggregation Assume that the set of all sites connected to the cloud platform is S={s1,s2,...,s M}, upload the inspection feature tensor Fs at each site period (t) ∈C Ns×T×K , where N s is the number of devices at the site, T is the window length, and K is the feature dimension.

[0118] Define the cloud global aggregation tensor as: Concat is a cross-site primary key alignment and concatenation operation.

[0119] Regional collaborative anomaly perception and linkage gating For each critical fault type c, the global regional risk score is defined as: where ψc(⋅) is the AI ​​discriminant function for fault type c (complex CNN output probability, etc. can be used).

[0120] Collaborative gating notification function: σ(⋅) is the Sigmoid gating function λ1,λ2 are global / local weights (dynamically adjustable) when (dynamic threshold), automatically push regional collaborative warnings to all relevant terminals to achieve linkage and proactive risk management.

[0121] Online self-learning and model distribution formula Define the abnormal sample set uploaded by each site S as , global aggregate exception set .

[0122] Self-learning loss optimization: : Cloud-based adaptive model, : Local model for each site DJS is Jensen-Shannon divergence, which is used to measure the difference in output distribution between local and cloud (federated distillation or consistency constraints) ζ: global-local consistency weight After optimization, the model parameters Θ∗ are dynamically distributed: Evaluation of the site's current model mismatch or anomaly rate (such as false positives, missed negatives, and sample drift indicators) ξ: Site upgrade trigger threshold The cloud-edge-end three-tier architecture enables on-demand distributed expansion and unified intelligent upgrades from a single site to the entire network, significantly improving O&M efficiency and intelligence. Full-scenario data aggregation and full process traceability support multi-role remote collaboration and decision-making, improving response speed and management transparency. Models, weights, and criteria are continuously and automatically revised to adapt to equipment aging, new faults, and changing environments, ensuring the system is always in optimal operating condition and reducing manual intervention. All functions can be implemented using mainstream industrial IoT, cloud platforms, and big data / AI technology stacks, easily integrating with existing enterprise IT, scheduling, and O&M systems.

[0123] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A power equipment monitoring and early warning system based on intelligent patrol, characterized in that: include: Multimodal sensor terminals are used to synchronously collect raw signals from multiple sources, including visible light, infrared, ultraviolet, acoustics, and vibration, at substations or transmission lines, and to uniformly timestamp different types of signals. The data preprocessing and complex mapping module is used to perform amplitude normalization, phase unwrapping, Hilbert transform or other time-frequency analysis on the collected multi-source signals, and convert each type of signal into a complex signal stream containing amplitude components and phase components; A complex-valued optical convolution acceleration unit includes a microcomb laser, an optical splitter, a fine spectrum shaping module, and a multi-channel Mach-Zehnder electro-optical modulator. The fine spectrum shaping module independently adjusts the amplitude and / or phase of each optical comb frequency line according to the convolution kernel weight, physically maps the convolution kernel weight to the optical carrier to achieve weight loading, and modulates the real and imaginary parts of the input complex signal onto different wavelength channels respectively. The input signal and weight are configured in a dispersion-compensating fiber delay array to form a time-stepping relationship between each channel, thereby implementing a sliding window convolution superposition of the input signal stream and the weight matrix in the physical domain and outputting multiple parallel convolution results. Photoelectric detection module, used to synchronously collect the convolution results of each channel and output the complete complex convolution results through differential processing; The edge AI processing module is used to splice the convolution results according to the spatial and channel dimensions to form a multi-dimensional complex feature tensor, and then perform feature fusion, anomaly identification and risk classification, and upload the analysis results to the cloud platform; A cloud-based big data analysis and management platform that aggregates historical complex features and equipment health records from multiple sites to enable trend analysis, fault prediction, and remote expert-assisted decision-making.

2. The power equipment monitoring and early warning system based on intelligent patrol according to claim 1 is characterized by: The multimodal sensing terminal includes infrared thermal imagers and ultraviolet corona detectors arranged in a linear or planar array. The array acoustic sensor covers the range of 20Hz to 30kHz. Each sensor ensures spatiotemporal data coordination through a clock synchronization mechanism.

3. The power equipment monitoring and early warning system based on intelligent patrol according to claim 2 is characterized by: The data preprocessing and complex mapping module includes a dedicated signal processing chip that can use time-frequency analysis on the harmonic characteristics of acoustic and vibration signals, encode signal intensity as mode, harmonic frequency shift or transient characteristics as phase, achieve complex high-sensitivity encoding of local anomalies, and automatically distribute abnormal high-frequency signals to independent data channels.

4. The power equipment monitoring and early warning system based on intelligent patrol according to claim 3 is characterized by: The complex-valued optical convolution acceleration unit uses an on-chip integrated microring resonator array to generate multi-wavelength comb lasers. After spectral shaping, the convolution kernel weights correspond to odd and even wavelength channels with their real and imaginary parts respectively. The real and imaginary parts of the input complex signal are modulated in parallel to the corresponding optical carriers through multi-channel Mach-Zehnder electro-optical modulators, achieving a one-to-one correspondence between weights and signals in the optical domain.

5. The power equipment monitoring and early warning system based on intelligent patrol according to claim 4 is characterized by: In the convolution operation, after the optical signal passes through the dispersion-compensating fiber delay array, each wavelength channel forms a time step in sequence, realizing the sliding of the convolution window on the input data stream. The output of all channels is synchronously collected by the photoelectric detection module and differentiated by a balanced photoelectric detector to realize the output of the complex convolution result with positive and negative weights.

6. The power equipment monitoring and early warning system based on intelligent patrol according to claim 1 is characterized by: The edge AI processing module includes an adaptive complex-valued convolutional neural network, which has the ability to automatically adjust the number of convolution layer cores, activation function type and feature fusion strategy, and can dynamically optimize model parameters based on device operation data to improve the accuracy of anomaly recognition.

7. The power equipment monitoring and early warning system based on intelligent patrol according to claim 6 is characterized by: When the edge AI processing module identifies an anomaly, it can automatically adjust the alarm threshold and inspection frequency, and upload relevant abnormal samples and criteria to the cloud-based big data analysis and management platform for continuous model optimization and expert-assisted decision-making.

8. The power equipment monitoring and early warning system based on intelligent patrol according to claim 7 is characterized by: The cloud platform supports real-time aggregation and multi-scenario visualization of multi-terminal data, and has functions such as equipment health big data management, risk trend prediction, fault sequence playback and regional collaborative early warning, and supports remote access and interaction from mobile terminals or web pages.

9. The power equipment monitoring and early warning system based on intelligent patrol according to claim 8 is characterized by: The system supports multi-level distributed deployment and modular expansion, and can flexibly adapt to the actual application needs of different power sites, substations or transmission lines.

10. The power equipment monitoring and early warning system based on intelligent patrol according to claim 7, characterized in that: The system has online self-learning capabilities and can continuously correct complex neural network model parameters, convolution weights and early warning criteria based on actual inspection history, thereby achieving continuous adaptive optimization for different equipment and environments.

Images