Aerial optical cable defect detection method based on multi-mode sensing fusion and related equipment

Abstract

The invention discloses an aerial optical cable defect detection method based on multi-mode sensing fusion and related equipment, and relates to the field of data processing. The method comprises the following steps: acquiring multi-modal sensing data along an aerial optical cable through sensing equipment of different modals; performing time and space alignment processing on the distributed fiber bragg grating strain data, the acoustic emission signal data and the visual image data to obtain multi-mode time and space alignment data; extracting feature data of each mode based on the multi-mode space-time alignment data; performing fusion processing on the feature data of each modal through an attention mechanism to obtain fused feature data; determining a defect identification result of the aerial optical cable according to the fusion feature data, wherein the defect identification result comprises a defect type and a position of the defect on the optical cable; obtaining the result confidence of the defect recognition result; and generating a final optical cable defect detection result according to the result confidence. According to the invention, the problem that the current aerial optical cable defect detection has relatively large missed detection and false detection risks can be relieved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an overhead optical cable defect detection method based on multimodal sensing fusion and related equipment. Background Art

[0002] Overhead optical cables, which transmit optical signals by installing fiber-optic communication cables in the air via towers or supports, are a common method of laying optical cables in the power and communications sectors. To ensure stable operation of communication networks, improve power system reliability, and avoid economic losses and safety incidents, overhead optical cables require essential defect detection.

[0003] Related technologies primarily rely on manual inspections and regular maintenance to detect defects in overhead optical cables. Inspectors regularly conduct visual inspections along the cable route, using telescopes to observe the cable's appearance, sheath condition, the presence of hanging objects, and tower tilt. For defects that are difficult to detect visually, simpler detection equipment, such as an optical time-domain reflectometer (OTDR), may be needed to test the integrity of the fiber link. However, OTDRs are primarily used to locate fiber breaks or attenuation points and cannot directly detect physical damage or potential risks on the outside of the cable.

[0004] It can be seen that the overhead optical cable defect detection method of related technologies is highly dependent on manual experience, has low detection efficiency, and has a large risk of missed detection and false detection. Summary of the Invention

[0005] In response to the above technical problems and defects, the purpose of the present invention is to provide an overhead optical cable defect detection method and related equipment based on multimodal sensor fusion, which can alleviate the problem that the current overhead optical cable defect detection has a large risk of missed detection and false detection.

[0006] To achieve the above objectives, in a first aspect, the present invention provides an overhead optical cable defect detection method based on multimodal sensing fusion, comprising: acquiring multimodal sensing data along the overhead optical cable through sensing equipment of different modalities, the sensing equipment comprising a distributed fiber Bragg grating sensor, an acoustic emission sensor, and an image acquisition device, the multimodal sensing data comprising distributed fiber Bragg grating strain data, acoustic emission signal data, and visual image data; performing time and space alignment processing on the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to obtain multimodal time-space alignment data; extracting feature data of each modality based on the multimodal time-space alignment data; fusing the feature data of each modality through an attention mechanism to obtain fused feature data; determining a defect recognition result of the overhead optical cable based on the fused feature data, the defect recognition result comprising a defect type and a location of the defect on the optical cable; obtaining a result confidence of the defect recognition result; and generating a final optical cable defect detection result based on the result confidence.

[0007] By integrating multi-source heterogeneous data such as strain, acoustic emission, and visual images acquired by distributed fiber Bragg grating sensors, acoustic emission sensors, and image acquisition devices, this method achieves comprehensive perception of the status of overhead optical cables, thereby effectively reducing the risk of missed or false detection of overhead optical cable defects. Its core technical advantages lie in: first, precise temporal and spatial alignment of different modal data to ensure information consistency; second, feature extraction of each modal data to convert the raw data into more discriminative information; and most importantly, the introduction of an attention mechanism to intelligently fuse these feature data, enabling the system to dynamically focus on the modal information most critical for defect identification, effectively overcoming the limitations of a single sensor and improving the ability to perceive complex defects. Ultimately, based on the fused feature data, this method can accurately determine the defect type and specific location on the cable, and provide confidence in the results, thereby generating reliable defect detection results. This method significantly improves the accuracy, robustness, and intelligence of overhead optical cable defect detection, enabling earlier and more comprehensive detection of potential faults. It provides strong technical support for the preventive maintenance and safe operation of optical cables, and reduces operation and maintenance costs and accident risks.

[0008] Optionally, in some embodiments, the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data are subjected to time and space alignment processing to obtain multimodal time-space aligned data, including: constructing a unified spatial reference coordinate system corresponding to the length of the overhead optical cable based on the physical position information of the overhead optical cable and the deployment position of each sensor; mapping the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to corresponding position points on the unified spatial reference coordinate system according to the unified spatial reference coordinate system and the deployment position of each sensor, to obtain spatially aligned strain data, acoustic emission data, and image data; performing time synchronization registration on the spatially aligned strain data, acoustic emission data, and image data according to the timestamp information when each sensor collects data, to obtain strain data, acoustic emission data, and image data with unified time-space coordinates; and generating the multimodal time-space aligned data based on the strain data, acoustic emission data, and image data with unified time-space coordinates.

[0009] Optionally, in some embodiments, the feature data of each mode is extracted based on the multimodal space-time alignment data, including: performing time-frequency analysis on the distributed fiber Bragg grating strain data in the multimodal space-time alignment data to obtain feature data characterizing the strain state of the optical cable; performing event detection and feature extraction on the acoustic emission signal data in the multimodal space-time alignment data to obtain feature data characterizing the vibration or mechanical damage of the optical cable; performing image feature extraction on the visual image data in the multimodal space-time alignment data to obtain feature data characterizing the surface state of the optical cable.

[0010] Optionally, in some embodiments, the feature data of each modality are fused through the attention mechanism to obtain fused feature data, including: generating an attention weight value corresponding to the feature data of each modality based on the feature data of each modality; performing weighted calculation on the corresponding feature data of each modality according to the attention weight value to obtain weighted feature data of each modality; and generating the fused feature data based on the weighted feature data of each modality.

[0011] Optionally, in some embodiments, the feature data of each modality is based on the generation of attention weight values ​​corresponding to the feature data of each modality, including: performing feature transformation on the feature data of each modality respectively to obtain the transformed feature vector corresponding to each modality; merging and calculating the transformed feature vectors corresponding to each modality to obtain an energy score reflecting the importance of each modal feature; and normalizing the energy score to obtain the attention weight value corresponding to the feature data of each modality.

[0012] Optionally, in some embodiments, the defect identification result of the overhead optical cable is determined based on the fused feature data, and the defect identification result includes the defect type and the position of the defect on the optical cable, including: inputting the fused feature data into a trained defect classification model to obtain a judgment result of whether the overhead optical cable has a defect and the type of defect that exists; extracting spatial correlation feature information associated with the spatial position of the optical cable from the fused feature data; determining the specific position coordinates of the defect on the overhead optical cable based on the spatial correlation feature information; and generating the defect identification result by combining the judgment result and the specific position coordinates.

[0013] Optionally, in some embodiments, obtaining the result confidence of the defect identification result includes: calculating a first confidence of the defect type identification result and a second confidence of the defect position identification result based on the fusion feature data, defect classification model and position calculation result used to generate the defect identification result; and generating the result confidence of the defect identification result based on the first confidence and the second confidence.

[0014] In a second aspect, an embodiment of the present invention provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in the first aspect or the second aspect, and any possible implementation of the first aspect or the second aspect.

[0015] In a third aspect, the present invention provides a computer-readable storage medium comprising instructions, which, when executed on the electronic device, enables the electronic device to execute the method described in the first aspect or the second aspect, and any possible implementation of the first aspect or the second aspect.

[0016] In a fourth aspect, the present invention provides a computer program product comprising instructions, which, when the computer program product is run on the electronic device, enables the electronic device to execute the method described in the first aspect or the second aspect, and any possible implementation of the first aspect or the second aspect.

[0017] It is understood that the electronic device provided in the second aspect, the storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided by the present invention. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 11 is a flow chart of a method for detecting defects in an overhead optical cable based on multimodal sensing fusion according to an embodiment of the present invention; Figure 2 It is a schematic diagram of the architecture of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The terms used in the following embodiments of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. As used in the specification of the present invention, the singular expressions "a," "an," "above," "the," and "this" are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used in the present invention refers to any and all possible combinations of one or more of the listed items.

[0020] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying relative importance or implicitly indicating the quantity of the technical features indicated. Thus, a feature designated "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, unless otherwise specified, "plurality" means two or more.

[0021] It should also be noted that, unless otherwise clearly specified and limited, in the embodiments of the present invention, terms such as "setting" and "connection" should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal connection of two components; it can be a wired communication connection or a wireless communication connection. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. The embodiments of the present invention are described in detail below.

[0022] To address the shortcomings mentioned in the background technology, an embodiment of the present invention provides an overhead optical cable defect detection method based on multimodal sensor fusion. By integrating multi-source heterogeneous data such as strain, acoustic emission, and visual images acquired by distributed fiber Bragg grating sensors, acoustic emission sensors, and image acquisition devices, this method achieves comprehensive perception of the overhead optical cable status, thereby effectively reducing the risk of missed or false detection of overhead optical cable defects. Its core technical advantages lie in: first, precise temporal and spatial alignment of different modal data to ensure information consistency; second, feature extraction of each modal data to transform the raw data into more discriminative information; and most importantly, the intelligent fusion of these feature data through the introduction of an attention mechanism, enabling the system to dynamically focus on the modal information most critical for defect identification, effectively overcoming the limitations of a single sensor and improving the perception of complex defects. Ultimately, based on the fused feature data, the method can accurately determine the defect type and specific location on the cable, and provide a confidence level for the result, thereby generating reliable defect detection results.

[0023] This embodiment significantly improves the accuracy, robustness and intelligence of overhead optical cable defect detection through this method, can detect potential faults earlier and more comprehensively, provides strong technical support for preventive maintenance and safe operation of optical cables, and reduces operation and maintenance costs and accident risks.

[0024] Next, combine Figure 1 , to specifically illustrate the method of this embodiment, the method can be performed by an optical cable defect detection device (hereinafter referred to as the device or the detection device). The method specifically includes the following steps: Step 201: Acquire multimodal sensing data along the overhead optical cable using sensing equipment of different modes, wherein the sensing equipment includes a distributed fiber Bragg grating sensor, an acoustic emission sensor, and an image acquisition device, and the multimodal sensing data includes distributed fiber Bragg grating strain data, acoustic emission signal data, and visual image data.

[0025] Specifically, it is necessary to rationally deploy distributed fiber Bragg grating (FBG) sensors, acoustic emission (AE) sensors, and image acquisition devices based on the characteristics of overhead optical cables.

[0026] For distributed fiber Bragg grating (FBG) sensors, considering the length and environmental complexity of overhead optical cables, FBG arrays or FBGs can be integrated in series inside or outside the cable sheath. For example, FBG sensing fibers can be pre-embedded during the cable manufacturing process, or auxiliary optical cables carrying FBG sensors can be attached to the surface of the overhead cable through binding or winding. These FBG sensors are distributed along the cable axis, forming a quasi-distributed sensing network that monitors strain changes along the cable in real time. When the cable is subjected to tension, bending, twisting, vibration, or temperature changes, the FBG Bragg wavelength shifts. These wavelength shifts can be precisely measured using a demodulator (such as a broadband light source and optical spectrum analyzer), allowing strain data at different locations on the cable to be calculated, thus generating distributed fiber Bragg grating strain data (hereinafter referred to as FBG strain data).

[0027] Acoustic emission sensors are primarily used to capture transient elastic wave signals occurring inside or outside optical cables. These signals are typically associated with material micro-damage, crack propagation, friction, and discharge. AE sensors can utilize piezoelectric ceramic sensors, which are fixed to the optical cable surface or supporting structure at regular intervals (e.g., every tens or hundreds of meters). High-frequency data acquisition cards sample and digitize the received acoustic emission signals in real time to generate acoustic emission data. Because acoustic emission signals decay rapidly and are susceptible to interference from ambient noise, the density and sensitivity of the sensors, as well as signal preprocessing (such as filtering and amplification), are crucial.

[0028] As for image acquisition devices, given that overhead optical cables are typically installed at high altitudes and face complex environments, a variety of methods can be used to acquire visual image data. For example, drones equipped with high-resolution visible light cameras and / or infrared thermal imagers can be used to conduct regular or on-demand inspections along the optical cable route, capturing images of the cable exterior, poles, towers, insulators, and other components. Drone inspections offer the advantages of efficiency, flexibility, and wide coverage. Furthermore, fixed high-definition cameras can be deployed at key locations (such as poles and corners) for continuous or periodic monitoring. In special circumstances, the use of inspection robots or vehicle-mounted aerial work platforms equipped with image acquisition equipment for manual assisted inspections can also be considered. These image acquisition devices require image stabilization, autofocus, and high resolution to ensure clear and effective visual information.

[0029] Ultimately, through these sensing devices of different modalities, the optical cable defect detection equipment can obtain the operating status data of the overhead optical cable in a comprehensive and multi-dimensional manner, laying the foundation for subsequent intelligent analysis.

[0030] Step 202 : performing time and space alignment processing on the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to obtain multimodal time and space alignment data.

[0031] This step aims to eliminate the temporal and spatial inconsistencies that may exist when different sensing devices collect data, ensuring the accuracy of subsequent feature extraction and fusion.

[0032] Time alignment primarily addresses differences in data acquisition start times, sampling frequencies, and transmission delays across sensors. Specific implementation strategies include the following: First, all sensor devices (FBG demodulators, AE acquisition systems, and image acquisition devices) should be equipped with high-precision GPS timing modules or Network Time Protocol (NTP) servers to ensure that their internal clocks are synchronized with a unified standard time source (such as an atomic clock or satellite time). This ensures that each data record is accurately timestamped globally. After data acquisition is complete, data can be synchronized based on these timestamps. For data with inconsistent sampling frequencies, resampling techniques can be used to unify data from all modalities to the lowest common multiple or a preset unified sampling frequency. For example, low-frequency data can be interpolated to a high frequency, or high-frequency data can be downsampled to a low frequency, using methods such as linear interpolation, spline interpolation, or nearest neighbor interpolation.

[0033] In addition, data transmission delay and processing delay need to be considered. By establishing a transmission model from each sensor to the data fusion center or performing calibration tests in advance, these fixed or variable delays can be compensated to ensure that the data is aligned on the time axis at the millisecond or even microsecond level.

[0034] Spatial alignment solves the problem of spatial position inconsistency reflected by different sensor data.

[0035] For FBG strain data, since the FBG sensors are pre-placed on the optical cable, their spatial locations are fixed and known. During data acquisition, the number or physical location of each FBG is associated with the strain data.

[0036] For acoustic emission data, AE sensors are also fixed to the fiber optic cable at known intervals, and their spatial positions are also predetermined. The key is to correlate the location of the AE event source with a specific position on the fiber optic cable. This can be achieved using acoustic emission source localization algorithms (such as time difference of arrival and triangulation). These algorithms use the time difference between multiple AE sensors receiving the same event signal to infer the precise location of the sound source on the fiber optic cable.

[0037] Spatial alignment of visual image data is more complex. If drone inspections are used, the drones themselves are equipped with high-precision RTK-GPS modules and inertial measurement units (IMUs) to obtain precise 3D position and attitude information in real time. Geotagging of images allows the location of each image to be associated with the geographic coordinates of the fiber optic cable.

[0038] In addition, computer vision techniques such as feature point matching (SIFT, SURF, etc.), image registration, and SLAM (Simultaneous Localization and Mapping) can be used to stitch continuous image sequences into a panoramic view or 3D model of the optical cable route and match it with pre-established optical cable geographic information system (GIS) data. By mapping the known spatial positions of the FBG and AE sensors onto this optical cable GIS model or image stitching results, precise alignment of all modal data in a unified spatial coordinate system can be achieved.

[0039] For example, if an image shows an object hanging somewhere on the optical cable, the FBG data can provide strain information at the location of the object, and the AE data can provide acoustic emission events caused by friction or impact of the object. Through spatiotemporal alignment, these different modal data can be accurately associated with the same physical location and time period of the optical cable, forming a complete multimodal alignment dataset, laying a solid foundation for subsequent feature extraction and fusion.

[0040] After multimodal spatiotemporal alignment of data, data preprocessing operations such as noise filtering and signal enhancement are typically required. This is because sensor data acquired in real-world environments is often subject to various noise interferences, such as electromagnetic interference, environmental vibration, and sensor noise. These noises can mask true defect signals, reducing the accuracy of subsequent feature extraction and defect identification, and potentially leading to false positives or missed detections.

[0041] Noise filtering aims to remove these irrelevant random fluctuations, purifying the signal and making the valid information more prominent. Meanwhile, signal enhancement processing focuses on amplifying potentially weak but indicative defect signals, such as acoustic emission signals from early-stage minor damage or subtle changes in fiber strain. By improving the signal-to-noise ratio, the enhanced signal more clearly reflects the true condition of the optical cable, providing high-quality input for subsequent feature extraction and attention-based fusion, significantly improving the sensitivity and reliability of the entire defect detection system.

[0042] Step 203: Extract feature data of each modality based on the multimodal spatiotemporal alignment data.

[0043] Specifically, after receiving multi-modal raw data that has been precisely aligned in time and space, the overhead optical cable defect detection equipment does not directly fuse the raw data. Instead, it first conducts an in-depth analysis of the data of each mode to extract key information, namely feature data, that can effectively characterize the optical cable status and potential defects.

[0044] For distributed fiber Bragg grating (FBG) strain data, the detection equipment can process the long-term strain data series of each FBG sensing point. This includes but is not limited to: extracting statistical features such as average strain, strain variance, strain peak value, and strain change rate to reflect the overall stress state and strain fluctuation of the optical cable; performing frequency domain analysis, such as Fourier transform, to identify the characteristic frequencies and amplitudes of the optical cable under wind vibration, ice and snow loads, or structural vibration. These frequencies may be related to fatigue damage or resonance of the optical cable; and identifying local strain concentration points or strain mutation points, which are usually direct manifestations of optical cable structural damage (such as sheath damage and uneven stress on the internal optical fiber).

[0045] The equipment can use digital signal processing (DSP) algorithms and time-frequency analysis techniques to extract these features. For acoustic emission (AE) signal data, the equipment will focus on identifying and quantifying AE events from continuous AE waveforms and extracting their features. This includes: event count, that is, the number of AE events occurring per unit time. A high event rate may indicate continued damage development; amplitude, which reflects the energy of the AE event. High-amplitude events are usually associated with more severe damage; rise time and duration. These parameters help distinguish different types of sound sources (such as crack propagation, friction, and partial discharge); and frequency domain features, such as peak frequency, center frequency, and frequency band energy distribution. These can help the equipment identify the physical mechanism of the AE event (for example, the AE signal of a fiber break usually has a specific frequency range). The equipment will use threshold triggering, waveform envelope analysis, fast Fourier transform (FFT), and other technologies to extract these AE features.

[0046] For visual image data, the device can utilize advanced computer vision and deep learning techniques for feature extraction. This includes image preprocessing (such as denoising and enhancement) to improve image quality, followed by feature extraction using trained convolutional neural network (CNN) models. These CNN models can automatically learn and extract high-level semantic features from images. For example, they can identify visual anomalies such as wear, cracks, expansion, deformation, hanging objects (such as bird nests, kites, and foreign objects) on optical cable sheaths, ice and snow coverage, tilted towers, and damaged insulators. The outputs of intermediate CNN layers or the output before fully connected layers are typically used as feature vectors for the image. These feature vectors encode the visual information in the image in a compact and discriminative manner.

[0047] Through these targeted feature extraction processes, the overhead optical cable defect detection equipment can transform raw, heterogeneous multimodal data into a unified, structured, and informative feature representation, laying a solid foundation for subsequent intelligent fusion and defect identification.

[0048] Step 204: fuse the feature data of each modality through an attention mechanism to obtain fused feature data.

[0049] The purpose of this step is to overcome the limitations of traditional simple splicing or weighted fusion, so that the device can intelligently identify and utilize the complementarity and correlation between different modal features.

[0050] After obtaining the FBG strain, acoustic emission, and visual image features in step 203, the device activates its built-in attention mechanism (Self-Attention). This attention mechanism typically consists of one or more neural network layers. Its core concept is to enable the device to dynamically learn the importance of different modal features and adaptively assign weights based on the context of the current input data.

[0051] In practice, the device can initially concatenate or process feature vectors from different modalities in parallel to form a comprehensive feature representation. An attention mechanism will then be applied to this comprehensive feature representation. For example, the device can employ a self-attention mechanism, allowing the model to simultaneously consider features from other modalities while processing features from one modality, thereby capturing the inherent dependencies between features from different modalities.

[0052] In some embodiments, a cross-attention mechanism is used for multimodal fusion. Multiple attention heads can be designed, allowing, for example, visual features to "focus" on corresponding FBG strain features and AE event features, and vice versa. This means that when the device detects obvious sheath damage in the image, it will tend to give higher weight to the visual features; when a high-frequency acoustic emission event is detected but not visually obvious, it will focus more on the AE features; and when the optical cable undergoes long-term, slow deformation, the importance of FBG strain features will be highlighted. This dynamic weighting and information interaction capability enables the device to intelligently adjust the emphasis on different modal information based on the defect type.

[0053] The output of the attention mechanism is a new, weighted and transformed "fused feature data." This fused feature vector contains key information from all modalities, intelligently filtered and emphasized by the device, making it more discriminative. For example, if a dimension in the fused feature data is assigned a high weight, it indicates that the original modal feature corresponding to that dimension is crucial for identifying the current defect.

[0054] In this way, the aerial fiber optic cable defect detection equipment can effectively integrate the advantages of heterogeneous data sources, avoid the limitations of a single modality, and significantly improve the accuracy and robustness of detecting various complex and hidden aerial fiber optic cable defects. This fused feature data will serve as input for subsequent defect recognition models, enabling more accurate judgments.

[0055] Step 205: Determine a defect recognition result of the overhead optical cable based on the fused feature data. The defect recognition result includes a defect type and a location of the defect on the optical cable.

[0056] Specifically, the device's built-in defect recognition model analyzes the fused feature data. This model is typically a deep learning model trained with a large amount of labeled data, such as a multi-layer perceptron (MLP), a recurrent neural network (RNN) (if the fused features contain time series information), a convolutional neural network (CNN) (if the fused features can be mapped to an image-like structure), or a more complex Transformer model. The model's training goal is to learn the mapping relationship between the fused feature data and various known defect types (such as sheath damage, fiber breakage, stress concentration, ice and snow coverage, foreign object entanglement, tower tilt, vibration fatigue, etc.).

[0057] When the device receives new fused feature data, the model outputs one or more predictions. To determine the defect type, the model typically outputs a probability distribution, indicating the likelihood that the input data belongs to each defect type. The device selects the type with the highest probability as its initial defect type judgment. For example, if the model outputs a 95% probability for "sheath damage" and a 3% probability for "ice and snow covering," the device will identify the defect as sheath damage.

[0058] As for the location of the defect on the optical cable, this is a regression problem or a more sophisticated classification problem. Since precise spatial alignment has been performed in step 202, the spatial information of the defect is already implicit in the fused feature data. The model can regress the precise coordinates of the defect (for example, the number of meters from the starting point of the optical cable, or the longitude and latitude coordinates) directly from the fused features through learning. Alternatively, the device can pre-divide the optical cable into multiple small spatial segments, and then the model predicts which spatial segment the defect belongs to, and then combines the high-resolution image for secondary precise positioning. For example, if the fused feature data indicates that there is stress concentration in a certain section of the optical cable, the device will combine the FBG strain data and the positioning results of the AE event of the section, as well as the visual image of the section, to jointly determine the precise location of the defect. In addition, for certain defects, such as ice and snow covering, the impact may cover a longer section of the optical cable, and the device will output the starting and ending positions of a defect area.

[0059] Through the application of this deep learning model, overhead optical cable defect detection equipment can achieve automatic, high-precision identification and positioning of various complex defects, greatly improving detection efficiency and accuracy.

[0060] In this embodiment, training the defect recognition model is an iterative optimization process. First, a large-scale, high-quality training dataset must be constructed, containing a large number of fused feature data samples. Each sample is precisely labeled with its corresponding defect type (e.g., no defect, sheath damage, fiber break, etc.) and the exact location of the defect on the cable. This fused feature data is collected by multimodal sensors (such as FBG, AE, and vision sensors) and aligned, normalized, and fused during data preprocessing. Deep learning architectures are typically used for model selection. Convolutional neural networks (CNNs) are suitable for processing features with local correlations (such as visual information), recurrent neural networks (RNNs) or Transformer models are suitable for processing sequential data (such as time-series FBG or AE data), and multi-layer perceptrons (MLPs) can serve as the final classifier for the fused features. The training process typically adopts a supervised learning paradigm, where the model predicts the defect type based on the input fused feature data. The prediction is then compared with the true label and a loss function (such as cross-entropy loss) is calculated, which quantifies the difference between the prediction and the true value. Next, optimization algorithms (such as Adam and SGD) are used to adjust the model's internal parameters (weights and biases) by backpropagating the gradient of the loss function to minimize the loss function. This process is repeated for multiple epochs until the model achieves satisfactory performance on the validation set or the loss function converges. To improve the model's generalization capabilities, various regularization techniques (such as Dropout and L2 regularization) and data augmentation methods are also employed. Ultimately, the trained model is able to map new, unseen fused feature data to corresponding defect types and output a confidence score.

[0061] Step 206: Obtain the confidence level of the defect recognition result.

[0062] In this embodiment, while the defect recognition model outputs the defect type and location, the device also calculates and provides a confidence score for the result. This confidence score reflects the model's degree of certainty or trustworthiness in its predictions. For deep learning-based models, the confidence score can typically be obtained directly from the model's output layer. For example, in classification tasks, the probability value output by the model (after processing with the softmax function) can serve as the confidence score. Higher probability values ​​indicate greater confidence in the model's prediction.

[0063] Furthermore, more robust confidence levels can be calculated from the predictions of multiple models through methods such as ensemble learning or Bayesian deep learning. For example, if multiple sub-models all give the same prediction, the confidence level is higher. This confidence level is an important basis for subsequent decision-making, as it tells operators how likely the identified defect is to be real and how confident the model is in this judgment.

[0064] Step 207: Generate a final optical cable defect detection result based on the result confidence level.

[0065] Specifically, if the confidence level of a defect identification result reaches a preset high threshold (e.g., above 90%), the device can directly mark the result as a "confirmed defect" and automatically trigger the corresponding alarm mechanism or maintenance work order. If the confidence level is in the medium range (e.g., 60%-90%), the device may mark the result as a "suspected defect" or "defect awaiting review" and recommend further manual verification or more detailed local inspection to avoid false positives or missed negatives. If the confidence level is below a low threshold (e.g., below 60%), the device may deem the result unreliable and not report it or use it only as reference information to reduce invalid alarms.

[0066] The final inspection results are presented in a clear and structured format, including defect type, precise location, occurrence time, confidence level, and related raw data segments (such as images, strain curves, and acoustic emission waveforms), facilitating rapid response and troubleshooting by maintenance personnel.

[0067] This embodiment utilizes the aforementioned method for aerial optical cable defect detection based on multimodal sensor fusion. By integrating heterogeneous data from multiple sources, including distributed fiber Bragg grating strain, acoustic emission signals, and visual images, and leveraging advanced spatiotemporal alignment, feature extraction, and attention-based fusion techniques, this method enhances the comprehensiveness, accuracy, and intelligence of aerial optical cable defect detection. Compared to single-modality detection, this embodiment overcomes the limitations of each sensor and achieves information complementarity. For example, visual images can intuitively detect surface damage, FBG strain data can accurately capture internal stress changes, and acoustic emission signals can monitor the onset and development of microscopic material damage in real time. Through intelligent recognition of fused features using a deep learning model, this method not only accurately determines defect type (such as sheath damage, fiber breakage, and stress concentration), but also precisely locates the defect on the cable, provides confidence in the result, and effectively distinguishes between real defects and environmental interference. This greatly improves the sensitivity and robustness of defect detection, reduces missed and false detections of defects, and enables optical cable operation and maintenance personnel to discover and handle potential faults more promptly and accurately, thereby effectively preventing optical cable accidents, ensuring the stable operation of communication networks, and reducing operation and maintenance costs. It has significant technological advancement and practical application value.

[0068] This embodiment also provides an overhead optical cable defect detection method based on multimodal sensing fusion, which specifically includes the following steps: S301, acquiring multimodal sensing data along the overhead optical cable through sensing devices of different modes.

[0069] This step can refer to the description of the above embodiment and will not be repeated here.

[0070] S302: Constructing a unified spatial reference coordinate system corresponding to the length of the aerial optical cable based on the physical location information of the aerial optical cable and the deployment positions of the sensors.

[0071] This step aims to establish a standardized and unified linear spatial reference system for the entire aerial optical cable.

[0072] Specifically, the starting point of the optical cable must first be determined as the zero point of the spatial coordinate system (for example, 0 meters). The physical path of the optical cable is then extended, defining its length in meters. This means that every point on the optical cable will be assigned a unique, precise physical location coordinate. For example, if the total length of the optical cable is 1000 meters, the coordinate system will extend from 0 meters to 1000 meters. The establishment of this coordinate system is the basis for the spatial alignment of all subsequent sensor data. It provides a common "map" that enables data collected from different sensors at different locations to be accurately mapped to corresponding physical points on the optical cable, thereby achieving cross-sensor geographic correlation. Without this unified coordinate system, data from various sensors cannot be effectively integrated and compared in the spatial dimension, making it impossible to determine whether different signals originate from the same area of ​​the optical cable.

[0073] S303: Map the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to corresponding position points on the unified spatial reference coordinate system according to the unified spatial reference coordinate system and the deployment positions of the sensors, to obtain spatially aligned strain data, acoustic emission data, and image data.

[0074] This step accurately maps the raw data collected by each sensor to the previously constructed unified spatial reference coordinate system based on its actual deployment location on the optical cable or the location of the detected event.

[0075] For distributed fiber Bragg grating (FBG) strain data, since FBG sensors are usually pre-installed at specific discrete points on the optical cable, their strain data can be directly associated with their known physical positions in a unified coordinate system.

[0076] For acoustic emission (AE) signal data, when the AE sensor detects an event, it is necessary to determine the precise location of the AE event on the optical cable through a single-sensor position marker or a multi-sensor positioning algorithm (such as the arrival time difference method), and then map this location into a unified coordinate system.

[0077] For visual image data, this typically involves processing the GPS positioning information, odometer data, or visual odometry technology of image acquisition devices (such as drones or inspection robots) to determine the starting and ending positions of the fiber optic cable segment covered by each image or video frame in a unified coordinate system, thereby matching visual features (such as cracks and damage) with specific spatial locations on the fiber optic cable.

[0078] Through this step, data from all modalities are given clear spatial labels.

[0079] S304 , performing time synchronization registration on the spatially aligned strain data, acoustic emission data, and image data according to timestamp information when each sensor collects data, to obtain strain data, acoustic emission data, and image data with unified time and space coordinates.

[0080] The goal of time synchronization is to ensure that spatially mapped data from different modalities are precisely aligned in the temporal dimension. This requires that all sensing devices record high-precision timestamps during data acquisition. This is typically accomplished by synchronizing the clocks of all sensors and data acquisition systems using technologies such as the Network Time Protocol (NTP) or Global Positioning System (GPS) timing to ensure a consistent time base. During data processing, data points from all modalities are aligned according to their timestamps, using a unified time axis as a reference. If different sensors acquire data at different frequencies, interpolation, resampling, or nearest neighbor matching may be required to align all data points to a common time point. For example, if FBG data is acquired every second and acoustic emission data is acquired every millisecond, the acoustic emission data needs to be aggregated or sampled to the same frequency as the FBG data, or correlated using a time window.

[0081] This step ensures that in subsequent analysis, we can accurately correlate simultaneous events that occur at the same physical location but are captured by different sensors, thereby revealing the combined manifestation of the defect.

[0082] S305 : Generate the multimodal spatiotemporal alignment data based on the strain data, acoustic emission data, and image data having unified spatiotemporal coordinates.

[0083] After spatial mapping and temporal registration, the final step is to integrate all aligned sensor data into a unified, structured, multimodal spatiotemporal dataset. This can be accomplished by constructing a multidimensional data structure (such as a tensor, table, or custom data object) in which each entry, or "data point," represents a composite observation of a specific spatial location on the cable at a specific point in time. This composite observation encompasses information from all modalities (FBG strain, acoustic emission activity, and visual image features) collected at the same spatiotemporal point. For example, a data point might include "At time T, 50 meters down the cable, the FBG strain was X, the number of acoustic emission events was Y, and the visual image displayed Z features." This structured dataset serves as direct input for subsequent feature extraction and attention fusion, ensuring that all relevant information is correctly correlated, providing a high-quality foundation for deep learning models to understand the complex multimodal characteristics of cable defects.

[0084] S306: Extract feature data of each modality based on the multimodal spatiotemporal alignment data. Specifically, the process includes: S3061: Perform time-frequency analysis on the distributed fiber Bragg grating strain data in the multi-modal time-space alignment data to obtain characteristic data representing the strain state of the optical cable.

[0085] After receiving the spatiotemporally aligned FBG strain data, the cable strain not only contains static load information that varies slowly over time (such as deadweight, ice and snow loads, and prestressing), but also transient or periodic dynamic strains caused by wind vibration, flutter, vehicle vibration, or internal damage. Therefore, simple time-domain or frequency-domain analysis is insufficient to fully capture its characteristics. Time-frequency analysis methods such as short-time Fourier transform (STFT), wavelet transform (WT), or Hilbert-Huang transform (HHT) can simultaneously reveal the signal distribution in time and frequency, providing richer insights.

[0086] In specific implementation, the strain data for each FBG sensing point within a continuous time window is first sliced. For each time window, the STFT is applied to obtain the temporal variation of its spectrum, thereby identifying the vibration frequencies and intensities present at different moments. Wavelet transforms are more suitable for analyzing non-stationary signals. They can capture transient characteristics and local details of the signal using basis functions of different scales. This is particularly effective for detecting sudden strain events (such as the microslip before a fiber break) or nonlinear behavior in the strain signal. From these time-frequency analysis results, various features can be extracted, including but not limited to: average strain value (reflecting static load), strain fluctuation range (reflecting dynamic load intensity), dominant vibration frequency and amplitude (indicating specific vibration modes), energy distribution in specific frequency bands (distinguishing different types of vibration sources), strain change rate (indicating loading or unloading rates), and nonlinear characteristics of the strain signal (which may be associated with material damage or fatigue). These features collectively construct a multi-dimensional description of the strain state, providing a quantitative basis for subsequent defect identification. For example, persistently high strain may indicate overload, while abnormal vibration at a specific frequency may indicate structural loosening or fatigue.

[0087] S3062: Perform event detection and feature extraction on the acoustic emission signal data in the multimodal spatiotemporal alignment data to obtain feature data representing optical cable vibration or mechanical damage.

[0088] Event detection and feature extraction from acoustic emission signal data aim to identify transient elastic waves generated by microscopic damage within optical cables (such as fiber breakage, matrix cracking, coating delamination, and friction) from continuous acoustic emission (AE) waveforms and quantify their characteristics. This process begins with the critical event detection step. Since AE signals in real environments are often drowned out by ambient noise, effective algorithms are required to separate true AE events from background noise. Commonly used methods include fixed thresholds, adaptive thresholds (dynamically adjusted based on noise levels), short-term average / long-term average (STA / LTA) algorithms (which compare short-term and long-term signal energy to determine event occurrence), or energy-based detection methods. Once an AE event is detected, feature extraction is performed on each individual AE waveform. These features are designed to quantify the physical properties of the AE event and thus infer its source mechanism.

[0089] In this embodiment, the AE characteristics include: peak amplitude (the maximum signal intensity of the AE event, usually related to the severity of the injury), energy (the total energy released by the AE event, which is also an important indicator of the degree of injury), duration (the time from the start to the end of the AE event, which can distinguish different types of injuries), count (the number of times the signal exceeds the preset threshold, reflecting the frequency of injury activity), rise time (the time from the first threshold crossing to the peak amplitude of the signal, which can distinguish brittle fracture from plastic deformation), and frequency characteristics (by performing Fourier transform on the AE waveform, the center frequency, peak frequency, main frequency band, etc. are extracted. The AE signals generated by different damage mechanisms often have different frequency distributions).

[0090] In addition, waveform shape features (such as form factor and kurtosis) can also be extracted. These rich AE features can help distinguish damage caused by different reasons such as fiber breakage inside the cable, sheath wear, internal stress release, or external impact, providing direct evidence for defect type judgment and severity assessment.

[0091] S3063: Perform image feature extraction on the visual image data in the multimodal spatiotemporal alignment data to obtain feature data representing the surface state of the optical cable.

[0092] After acquiring a spatiotemporally aligned visual image, image preprocessing may be required, including denoising, brightness / contrast adjustment, and geometric correction, to improve image quality and the accuracy of subsequent feature extraction. Image feature extraction methods can be divided into traditional image processing methods and deep learning-based methods. Traditional methods include: texture feature extraction (such as the gray-level co-occurrence matrix (GLCM), which is used to quantify the roughness, contrast, and uniformity of an image and can identify surface changes such as corrosion and wear), edge and corner detection (such as the Canny and Sobel operators, which are used to detect cracks and damaged edges), shape features (such as circularity and aspect ratio, which are used to quantify the geometry of foreign matter and bulges), and color features (such as color histograms and color moments, which are used to detect stains and discoloration).

[0093] However, deep learning methods, particularly convolutional neural networks (CNNs), demonstrate greater capabilities for complex and diverse optical cable defects. By training CNN models, high-level, abstract, and discriminative features can be learned directly from raw images. This includes object detection (e.g., using models like YOLO and Faster R-CNN to identify and locate specific defects such as broken strands, damaged sheaths, hanging objects, ice hooks, bird nests, and tower corrosion, and outputting their categories, locations, and confidence scores); image segmentation (e.g., using models like U-Net and Mask R-CNN to accurately segment defect regions using pixel-level masks, providing precise information on the defect's shape and area); and image classification (classifying the entire image or local regions to determine the presence of a specific defect type). The extracted features can be feature maps from CNN intermediate layers (as high-dimensional feature vectors) or structured data from detection or segmentation results (e.g., defect category, bounding box coordinates, area, perimeter, etc.). These visual features directly reflect visible damage or anomalies on the exterior of the optical cable and are essential for assessing its health.

[0094] S307: Generate an attention weight value corresponding to each modal feature data based on the feature data of each modality.

[0095] Specifically, the distributed fiber Bragg grating (FBG) strain features, acoustic emission (AE) features, and visual image features extracted in the previous step are first taken as input. Next, an attention network or mechanism is introduced. The core idea is to allow the model to autonomously learn which modalities are more informative for the final defect judgment or prediction in a specific context. This can be achieved in various ways: one feasible approach is to use a multi-layer perceptron (MLP) or a more complex Transformer encoder structure. Specifically, the feature vectors of all modalities can be concatenated or initially processed through independent MLP layers. These processed features are then input into a shared attention module. This attention module typically consists of one or more fully connected layers (or self-attention layers) and ultimately outputs a set of weights that match the feature dimensions of each modal. These weights are typically normalized using a softmax function to ensure that their sum is 1. This can be interpreted as the relative importance or "attention" of each modality in the current input. For example, when a fiber optic cable is broken, the AE signal features may be given a higher weight; when detecting a sheath damage, the visual image features may be given a higher weight. This mechanism enables the model to flexibly adjust the emphasis on different modal information according to the actual situation of the input data, avoiding the information loss or interference that may be caused by simple averaging or fixed weight fusion methods.

[0096] In some embodiments, step S307 may specifically include: S3071: Perform feature transformation on the feature data of each mode to obtain a transformed feature vector corresponding to each mode.

[0097] The purpose of this step is to map raw feature data from different modalities, potentially with different dimensions and distributions, into a common feature space that is more suitable for attention calculations. In implementation, a feature transformation module is independently applied to the feature data of each modality (e.g., FBG strain features, acoustic emission features, visual image features). This module typically consists of one or more fully connected layers (i.e., linear transformations) and a nonlinear activation function (e.g., ReLU).

[0098] For example, if the dimension of the FBG feature is D FBG , the dimension of the acoustic emission characteristic is D AE , the dimension of the visual feature is D Visual , then we can design three independent MLPs (multi-layer perceptrons) to convert them into a common embedding dimension D embed .Right now: F FBG ′=MLP FBG (F FBG ); F AE ′=MLP AE (F AE ); F Visual ′=MLP Visual (F Visual ); Among them, F FBG 、F AE 、F Visual is the original feature data, and F FBG ′、F AE ′、F Visual ′ is the transformed eigenvector, and their dimensions are unified to D embed This transformation not only unifies the dimension of features, but also enhances the expressiveness of features through nonlinear activation functions, enabling them to better capture complex patterns within the modality and provide richer semantic information for subsequent attention calculations.

[0099] Furthermore, this independent transformation also allows the model to specifically optimize and refine the features of each modality before computing attention.

[0100] S3072: Calculate the transformed eigenvectors corresponding to the modes in combination to obtain an energy score reflecting the importance of each modal feature.

[0101] After obtaining the transformed eigenvectors of each mode (F FBG ′、FAE ′、F Visual ′), this step aims to evaluate the contribution of each modality to the current task through a shared mechanism, thereby generating an “energy score” or “relevance score”.

[0102] In practice, one possible approach is to input these transformed feature vectors into an attention score function. This function is usually a simple fully connected layer (or a small MLP) that receives a feature vector as input and outputs a scalar value, which is the energy score of the modality. For example: Score FBG =W score ∙F FBG ′+b score ; Score AE =W score ∙F AE ′+b score ; Score Visual =W score ∙F Visual ′+b score ; Here, W score and b score are the weight matrix and bias terms shared by all modalities. The purpose of these shared parameters is to allow the model to learn a universal "importance estimator" that fairly compares the contributions of different modalities. A higher energy score indicates a more important modality for identifying optical cable defects in the current context. These scores can be positive or negative; their absolute magnitude reflects importance, while their sign may be related to task-specific preferences.

[0103] This step is the core of the attention mechanism, which maps high-dimensional feature vectors to low-dimensional, comparable importance measures through a learnable function.

[0104] S3073: Normalize the energy scores to obtain attention weight values ​​corresponding to the feature data of each modality.

[0105] After calculating the energy score of each mode (Score FBG 、Score AE 、Score Visual ), and then convert it into attention weight values ​​that can be used for weighted fusion.

[0106] When implementing this, the most common approach is to apply the Softmax function. The Softmax function converts any set of real-valued values ​​into a probability distribution, where all output values ​​are between 0 and 1 and their sum is 1. The specific calculation is as follows:

[0107]

[0108]

[0109] Through Softmax normalization, we get the attention weight value corresponding to each modality (Weight FBG 、Weight AE 、Weight Visual ). These weight values ​​directly reflect the relative importance of each modality in the current context. For example, if Weight FBG is close to 1, while the other weights are close to 0, which means that the model considers the FBG strain data to be the most important source of information at present.

[0110] This normalization ensures the rationality of the attention weights, allowing them to be directly used for weighted summation or weighted multiplication of the original modal features, thereby achieving effective information fusion. This process is fully learnable. During training, the model automatically adjusts the parameter values ​​of Wscore and bscore according to the task objectives (such as defect recognition accuracy) to generate the optimal attention weights.

[0111] S308: Perform weighted calculation on the corresponding modal feature data according to the attention weight value to obtain weighted modal feature data.

[0112] This step applies these weights to the original modal feature data to highlight important information and suppress unimportant or noisy information.

[0113] In specific implementation, for each modal feature vector, perform element-by-element or broadcast multiplication with the corresponding attention weight value. For example, if the FBG strain feature vector is F FBG , and its attention weight is w FBG , then the weighted FBG feature is F FBG ′=w FBG ∙F FBG .

[0114] The same operation is applied to the acoustic emission eigenvector F AE and the visual image feature vector F Visual , and obtain the weighted F AE ′=w AE ∙F AE and F Visual ′=w Visual ∙F Visual .

[0115] The multiplication here can be a simple scalar multiplication (if the weight is a scalar) or a more complex matrix multiplication (if the weight itself is a matrix, used to weight the features internally). Through this weighting operation, the values ​​of the modal features that are judged to be more important by the attention mechanism will be amplified, thus dominating the subsequent fusion; while the values ​​of the modal features that are considered unimportant will be reduced, reducing their impact on the final fusion result.

[0116] This process effectively filters and emphasizes information, allowing the model to focus more on the modal information most relevant to defect identification, thereby improving the effectiveness of fusion and its anti-interference ability.

[0117] S309: Generate the fused feature data based on the weighted feature data of each modality.

[0118] This step aims to integrate the various modal feature data after attention weighting into a unified, high-dimensional fusion feature vector, which will serve as the input for subsequent defect recognition or classification tasks. When implementing, the most direct and commonly used method is to concatenate all the weighted modal feature vectors. For example, the weighted FBG feature F FBG ′, weighted acoustic emission characteristics F AE ′ and weighted visual image features F Visual ′ is connected along the feature dimension to form a longer fusion feature vector F fusion =[F FBG ′;F AE ′;F Visual ′]. This concatenation operation is simple and effective, which preserves the weighted information of all modalities and places them in the same feature space.

[0119] In addition to simple concatenation, more complex fusion strategies can be adopted. For example, an additional fully connected layer or multi-layer perceptron is used to perform nonlinear transformations on the concatenated features to learn more complex interactions between modalities. Alternatively, more advanced fusion modules such as Transformer-based fusion layers can be used to allow cross-attention calculations between feature vectors of different modalities, thereby capturing deeper inter-modal dependencies.

[0120] Regardless of the approach used, the ultimate goal is to generate a fused feature representation that comprehensively and effectively represents the current state of the optical cable. This fused feature vector incorporates information from different physical sensing principles, and the importance of this information has been adaptively adjusted through an attention mechanism. This provides richer and more discriminative input for subsequent defect recognition models, significantly improving the accuracy and robustness of defect detection.

[0121] S310: Input the fused feature data into the trained defect classification model to obtain a judgment result of whether the overhead optical cable has defects and the type of defects.

[0122] Specifically, the unified feature vector obtained by fusion through the attention mechanism in the previous step is first used as input. This fused feature vector already contains key information from multiple modalities such as fiber strain, acoustic emission, and visual images, and the importance of this information has been adaptively adjusted.

[0123] This fused feature vector is then fed into a pre-trained defect classification model. This model is typically a deep learning network, such as a multi-layer perceptron (MLP), convolutional neural network (CNN), or a recurrent neural network (RNN) variant. The specific choice depends on the structure of the fused features and the complexity of the task. During the model training phase, it learns from a large amount of fused feature data labeled with defect types (e.g., no defect, sheath damage, fiber breakage, corrosion, etc.) to identify the characteristic patterns corresponding to different defect types.

[0124] In actual applications, when new fused feature data is input, the model will output a probability distribution about whether there is a defect in the optical cable (binary classification) and, if so, what type of defect it is (multi-classification) based on the learned pattern.

[0125] Ultimately, the device will select the category with the highest probability as the final judgment result based on these probabilities, thereby realizing automatic identification of optical cable defects.

[0126] S311, extracting spatial correlation feature information associated with the spatial position of the optical cable from the fused feature data.

[0127] This step aims to identify and extract clues from the fused feature data that can indicate the specific location of defects on the optical cable. Although the fused feature data is a comprehensive representation, its constituent modalities (such as visual images, FBG strain data, and acoustic emission data) inherently carry rich spatial information.

[0128] During implementation, it may be necessary to trace back to the source of the fused features or to specifically design the fusion process to preserve or encode spatial information. For example, in visual image modalities, the pixel coordinates or bounding box information of the defect area obtained by the target detection algorithm can be directly associated with the physical location in the image; for distributed fiber optic strain sensors (FBGs), the physical location of each FBG sensor is known, and its strain data is naturally associated with a specific optical cable segment; for acoustic emission data, multiple acoustic emission sensors can be located by the difference in acoustic wave propagation time.

[0129] Therefore, extracting spatially relevant feature information is not simply "cutting out" a portion from the fused features. Instead, it refers to obtaining specific data points, coordinates, sensor IDs, or relative distance information that clearly point to spatial locations from the original modal features or encoding them during the fusion process. This information may exist in the fused features as sub-feature vectors, metadata tags, or specific encoding methods, providing the basis for subsequent precise location determination.

[0130] S312: Determine the specific location coordinates of the defect on the overhead optical cable based on the spatial correlation feature information.

[0131] The goal of this step is to convert it into precise, measurable defect location coordinates. The specific implementation method depends on the source modality of the spatially correlated feature information. For example, if the spatial information primarily comes from the defect bounding box in a visual image, it is necessary to combine the camera's calibration parameters, installation location, and the optical cable's geometric model to map the pixel coordinates in the image to the actual three-dimensional spatial coordinates of the optical cable or the linear distance along the cable (for example, how many meters from the starting point). If the spatial information comes from an FBG sensor, the defect location may be limited to the area where one or several FBG sensors are located. More precise positioning may require interpolation or machine learning-based predictions based on the strain distribution pattern. If the spatial information comes from an acoustic emission sensor, the precise location of the sound source (i.e., the defect) can be calculated using a triangulation algorithm based on the time difference between the sound waves received by multiple sensors.

[0132] In the case of multimodality, spatial information from different modalities can be cross-validated or fused (for example, through methods such as Kalman filtering) to improve positioning accuracy and robustness. The final output is a clear defect location coordinate that can be used for on-site repair or recording, such as the distance from a utility pole, GPS coordinates, or mileage on a fiber optic cable.

[0133] S313: Combining the judgment result and the specific location coordinates, generate the defect recognition result.

[0134] The purpose of this step is to integrate the two key pieces of information obtained in the previous step, "what is the defect" and "where the defect occurred", to form a complete, meaningful and actionable defect report.

[0135] During implementation, the device logically associates and encapsulates the defect type determination output by the defect classification model (for example, "sheath damage," "fiber breakage," or "no defect") with the specific defect location coordinates determined through spatial correlation feature information (for example, "150.2 meters from pole A" or "GPS coordinates: X degrees north latitude, Y degrees east longitude"). The final defect identification results can be presented in various forms, such as: a structured data record (such as a JSON object or database entry) containing information such as defect ID, type, location, timestamp, and confidence level; a visual report that marks the defect point and displays its type on the optical cable route map; or a direct alert to operations and maintenance personnel, clearly stating that "a certain type of defect has occurred at a certain location on a certain optical cable." This comprehensive result serves as a crucial basis for operations and maintenance personnel to troubleshoot, schedule repairs, and manage assets. It transforms abstract sensor data and complex model reasoning into directly usable on-site action guidelines, significantly improving the efficiency and accuracy of optical cable operations and maintenance.

[0136] S314 , based on the fused feature data, defect classification model, and position calculation result used to generate the defect identification result, respectively calculate a first confidence level of the defect type identification result and a second confidence level of the defect position identification result.

[0137] This step aims to quantify the reliability of the model's defect type and location determination. The initial confidence level of the defect type identification result is typically derived directly from the output of the trained defect classification model. When a deep learning classification model (for example, one using Softmax as the final activation function) classifies fused feature data, it outputs a probability value for each possible defect type. These probabilities sum to 1. The probability value corresponding to the defect type ultimately predicted by the model represents the confidence level of the defect type identification result.

[0138] For example, if the model predicts that the probability of "sheath being broken" is 0.95, then the first confidence level is 0.95, which means that the model is very confident in this judgment.

[0139] The defect location identification result refers to the identification of the specific location of the defect on the overhead optical cable. The calculation method for the second confidence level of the defect location identification result is more dependent on the specific location determination method. If the location is predicted by a regression model, the confidence level may be related to the variance or mean square error of the prediction; if the location is determined by multi-sensor triangulation (such as acoustic emission), the confidence level may be related to the residual error of the positioning algorithm, the geometric dilution of precision (GDOP), or the number and layout of sensors involved in positioning; if the location is inferred based on image processing or FBG sensor data, the confidence level may be related to the clarity of the image recognition, the spacing of the FBG sensors, or the data noise level. Essentially, the second confidence level reflects the uncertainty or precision in the location calculation process. The greater the precision and the lower the uncertainty, the higher the confidence level.

[0140] S315: Generate a result confidence level of the defect recognition result according to the first confidence level and the second confidence level.

[0141] The purpose of this step is to combine the confidence level of defect type identification (first confidence level) and the confidence level of defect location identification (second confidence level) to form a single metric representing the overall reliability of the defect identification result. There are several ways to generate the confidence level of the result, and the specific choice depends on the application scenario's emphasis on type and location accuracy. A simple and commonly used method is to take a weighted average of the two, namely: The confidence level of the result = w1 × first confidence level + w2 × second confidence level, where w1 and w2 are weights, and w1 + w2 = 1. If higher accuracy is required for defect type judgment, w1 can be given a larger weight; conversely, if higher position accuracy is required, w2 can be given a larger weight.

[0142] Another approach is to multiply the two: result confidence = first confidence × second confidence. This approach means that the overall confidence level is high only when both the type and location confidence levels are high. Low confidence in any one link significantly lowers the overall confidence level.

[0143] Another method is to take the minimum of the two, that is: result confidence = min(first confidence, second confidence); this means that the overall confidence is limited by the most uncertain link.

[0144] Through this comprehensive calculation, the equipment can provide a quantitative reliability index for each identified defect, which provides an important reference for subsequent decision-making (for example, whether to send someone for inspection immediately or whether to initiate emergency repair procedures), avoiding misjudgment or misoperation caused by high confidence in a single dimension.

[0145] S316: Generate a final optical cable defect detection result based on the result confidence level.

[0146] This step can refer to the description of the above embodiment and will not be repeated here.

[0147] During the implementation of the method of this embodiment, in order to achieve high coverage (i.e., accurate time and space alignment of the entire length of the overhead optical cable), it is necessary to build a unified spatial reference coordinate system and deploy high-precision positioning equipment (such as GPS or time synchronization equipment) to ensure seamless mapping and synchronization of multimodal data over long-distance optical cables.

[0148] However, such comprehensive coverage will lead to a significant increase in hardware costs (such as the purchase of dedicated sensors) and energy consumption, especially in large-scale or remote scenarios. The dense deployment of high-precision equipment and the resource requirements of synchronization protocols significantly increase the system investment cost, thus creating a contradiction between coverage breadth and economic benefits under limited budget constraints.

[0149] This conflict directly affects the practicality of the method. For example, in monitoring optical cables over 10 kilometers, the surge in costs may limit the scale of deployment, while insufficient optimization may sacrifice detection accuracy in key areas.

[0150] Therefore, this embodiment also provides a solution, which specifically includes the following steps: Step S2021: Obtain physical structural information and environmental risk information for the overhead optical cables. Specifically, the detection equipment proactively acquires physical structural information of the overhead optical cables, including cable length, connection point locations, and tower coordinates, through an integrated sensor network and GIS system. Furthermore, the equipment integrates weather stations and temperature and humidity sensors to collect real-time environmental risk information, such as wind speed, temperature fluctuations, and hail warning data. This data forms a structured dataset to support subsequent zoning decisions.

[0151] In step S2022, the overhead optical cable is segmented into multiple independent sections based on the physical structure information and environmental risk information. Specifically, based on the physical structure information and environmental risk information output in step S2021, the detection device uses a built-in algorithm engine to intelligently segment the optical cable into multiple independent sections. The boundaries of each section are defined by risk hotspots (such as splice points or high wind speed areas), ensuring that the zoning logic is closely linked to the risk distribution.

[0152] Step S2023 assigns a risk level to each independent section. Specifically, the inspection equipment assigns a risk level to each independent section. Using a pre-trained machine learning model, the equipment analyzes the section's historical defect rate and real-time environmental data, outputting high, medium, or low risk labels. For example, sections near joints are labeled high risk, while straight sections are labeled low risk. This provides a precise prioritization basis for equipment deployment.

[0153] In step S2024, based on the risk level, high-precision positioning equipment is deployed in high-risk areas, and low-cost positioning equipment is deployed in non-high-risk areas. Specifically, based on the risk level assigned in step S2023, the detection equipment automatically deploys high-precision positioning equipment, such as RTK-GPS modules, in high-risk areas to achieve centimeter-level positioning, while simultaneously deploying low-cost equipment, such as IMU sensors, in non-high-risk areas. This dynamic resource scheduling reduces hardware costs and maintains coverage integrity.

[0154] In step S2025, the high-precision positioning device and the low-cost positioning device are used to perform spatial mapping and temporal synchronization. Specifically, the detection device utilizes the deployed high-precision and low-cost devices to perform spatial mapping and temporal synchronization. Spatial mapping uses a coordinate transformation algorithm to align sensor data to a unified coordinate system. Temporal synchronization uses GPS clocks or the NTP protocol to ensure consistent timestamps across modal data, generating a high-precision spatiotemporal alignment output.

[0155] Step S2026: Output the multimodal spatiotemporal alignment data after partition alignment. Specifically, the inspection equipment outputs the multimodal spatiotemporal alignment data after partition alignment, integrating the spatial mapping and time synchronization results to form a structured dataset, including strain, acoustic emission, and image data corresponding to the optical cable location points. This data is directly input into the subsequent defect detection process to support multimodal fusion analysis.

[0156] This embodiment uses the above steps, the core function of which is to significantly reduce system costs while maintaining high coverage through intelligent zoning and differentiated resource allocation. These steps establish a closed-loop optimization process: by integrating the physical structure of the optical cable with environmental risk information, intelligent zoning based on risk hotspots (such as prioritizing the segmentation of joint points) is achieved, laying the foundation for the subsequent precise allocation of resources; positioning equipment is dynamically allocated based on risk level (such as deploying RTK-GPS in high-risk areas and IMU in low-risk areas), focusing limited resources on key sections to avoid high-cost investment across the entire area; and spatial and temporal alignment is performed through zoning to ensure the reliability of output data while reducing hardware costs by 40%-60%. Its technical significance lies in: Resolve core contradictions: Directly resolve the conflict between coverage and cost; Improved practicality: Makes large-scale fiber optic cable monitoring (e.g., 100+ km power grid) feasible within a limited budget; Enhanced robustness: Risk-driven resource allocation improves the system's adaptability to complex environments (such as sudden electromagnetic interference).

[0157] The method provided in the above embodiment can be executed by an optical cable defect detection device, which is an electronic device. The following describes the electronic device in the embodiment of the present invention from the perspective of hardware processing. Figure 2, which is a schematic diagram of a physical device structure of an electronic device in an embodiment of the present invention.

[0158] It should be noted that Figure 2 The structure of the electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0159] like Figure 2 As shown, the electronic device includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes, such as the methods described in the above embodiments, based on programs stored in a read-only memory (ROM) 402 or programs loaded from a storage unit 408 into a random access memory (RAM) 403. RAM 403 also stores various programs and data required for system operation. CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to bus 404.

[0160] The following components are connected to the input / output (I / O) interface 405: an input section 406 including an audio input device, push button switches, and the like; an output section 407 including a display, an audio output device, indicator lights, and the like; a storage section 408 including a hard disk and the like; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the input / output (I / O) interface 405 as needed. Removable media 411, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 410 as needed, so that computer programs read from the media can be installed in the storage section 408 as needed.

[0161] In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 409 and / or installed from removable media 411. When executed by the central processing unit (CPU) 401, the computer program performs the various functions defined in the present invention.

[0162] It should be noted that specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings.

[0164] Specifically, the electronic device of this embodiment includes a processor and a memory, the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to enable the electronic device to execute the method provided by the above embodiment.

[0165] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not incorporated into the electronic device. The storage medium carries one or more computer programs, and when executed by a processor of the electronic device, the electronic device implements the methods provided in the above embodiments.

[0166] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

[0167] As used in the above embodiments, the term “when” may be interpreted to mean “if” or “after” or “in response to determining that” or “in response to detecting that”, depending on the context. Similarly, the phrases “upon determining that” or “if (stated condition or event) is detected” may be interpreted to mean “if determining that” or “in response to determining that” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0168] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for detecting defects in overhead optical cables based on multimodal sensing fusion, characterized in that: include: Acquiring multimodal sensing data along the overhead optical cable using sensing equipment of different modes, wherein the sensing equipment includes a distributed fiber Bragg grating sensor, an acoustic emission sensor, and an image acquisition device, and the multimodal sensing data includes distributed fiber Bragg grating strain data, acoustic emission signal data, and visual image data; Performing time and space alignment processing on the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to obtain multimodal time and space alignment data; Extracting feature data of each modality based on the multimodal spatiotemporal alignment data; The feature data of each modality are fused through an attention mechanism to obtain fused feature data; Determining a defect identification result of the overhead optical cable according to the fused feature data, wherein the defect identification result includes a defect type and a location of the defect on the optical cable; Obtaining a confidence level of the defect identification result; A final optical cable defect detection result is generated according to the result confidence level.

2. The method according to claim 1, characterized in that The performing time and space alignment processing on the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data to obtain multimodal time and space alignment data includes: Based on the physical location information of the overhead optical cable and the deployment positions of the sensors, a unified spatial reference coordinate system corresponding to the length of the overhead optical cable is constructed; According to the unified spatial reference coordinate system and the deployment positions of the sensors, the distributed fiber Bragg grating strain data, the acoustic emission signal data, and the visual image data are respectively mapped to corresponding position points on the unified spatial reference coordinate system to obtain spatially aligned strain data, acoustic emission data, and image data; Performing time synchronization registration on the spatially aligned strain data, acoustic emission data, and image data according to timestamp information when each sensor collects data, to obtain strain data, acoustic emission data, and image data with unified time and space coordinates; The multimodal spatiotemporal alignment data is generated based on the strain data, acoustic emission data and image data with unified spatiotemporal coordinates.

3. The method according to claim 1, characterized in that The extracting feature data of each modality based on the multimodal spatiotemporal alignment data includes: Performing time-frequency analysis on the distributed fiber Bragg grating strain data in the multimodal time-space alignment data to obtain characteristic data representing the strain state of the optical cable; Performing event detection and feature extraction on the acoustic emission signal data in the multimodal spatiotemporal alignment data to obtain feature data representing optical cable vibration or mechanical damage; Image feature extraction is performed on the visual image data in the multimodal spatiotemporal alignment data to obtain feature data representing the surface state of the optical cable.

4. The method according to claim 1, wherein The fusing of the feature data of each modality by the attention mechanism to obtain fused feature data includes: Based on the feature data of each modality, generating an attention weight value corresponding to the feature data of each modality; Performing weighted calculation on the corresponding modal feature data according to the attention weight value to obtain weighted modal feature data; The fused feature data is generated based on the weighted feature data of each modality.

5. The method according to claim 4, characterized in that The generating, based on the feature data of each modality, an attention weight value corresponding to the feature data of each modality includes: Performing feature transformation on the feature data of each mode to obtain a transformed feature vector corresponding to each mode; Merging and calculating the transformed eigenvectors corresponding to the modes to obtain an energy score reflecting the importance of each modal feature; The energy scores are normalized to obtain attention weight values ​​corresponding to the feature data of each modality.

6. The method according to claim 1, characterized in that The step of determining a defect identification result of the overhead optical cable based on the fused feature data, wherein the defect identification result includes a defect type and a location of the defect on the optical cable, includes: Inputting the fused feature data into the trained defect classification model to obtain a judgment result of whether the overhead optical cable has defects and the type of defects that exist; Extracting spatial correlation feature information associated with the spatial position of the optical cable from the fused feature data; Determining specific location coordinates of the defect on the aerial optical cable based on the spatial correlation feature information; The defect recognition result is generated by combining the judgment result and the specific location coordinates.

7. The method according to claim 1, characterized in that The obtaining of the result confidence of the defect identification result includes: Calculating a first confidence level of the defect type identification result and a second confidence level of the defect location identification result based on the fused feature data, the defect classification model, and the position calculation result used to generate the defect identification result; A result confidence level of the defect recognition result is generated according to the first confidence level and the second confidence level.

8. An electronic device, characterized in that: including one or more processors and memory; The memory is coupled to the one or more processors, and is configured to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 7.

Images