An intelligent electric meter ar operation and maintenance method based on cross-modal data association

Abstract

The application discloses an intelligent electric meter AR operation and maintenance method based on cross-modal data association, and relates to the technical field of electric power intelligent operation and maintenance.The specific steps of the method are as follows: a double backup clock synchronization mechanism is used to collect power and visual double modal data and mark high-precision time stamps, and when synchronization fails, a local crystal oscillator is started for calibration; power modal electrical fingerprints and visual modal indicator light flickering time sequence features are extracted respectively; cross-modal time sequence matching and phase difference secondary determination are used to establish unique mapping of electric meters and data, support interference suppression and fault tolerance matching; non-intrusive load decoupling and electric power data AR space anchoring are completed, and when visual positioning drifts, electrical fingerprints are used for recalibration; a dynamic safety avoidance radius is generated based on the running state of equipment, three-level risk early warning and visual prompting are realized; and the application realizes label-free binding of electric meter data through double backup clock synchronization and cross-modal matching, and combines electrical fingerprint assisted positioning and dynamic early warning, so that the accuracy and safety of operation and maintenance can be improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent power operation and maintenance technology, specifically to an AR operation and maintenance method for smart meters based on cross-modal data association. Background Technology

[0002] Smart meters are now widely used in residential communities, industrial and commercial parks, and various power distribution rooms. On-site operation and maintenance (O&M) is a crucial link in ensuring the stable operation of the power system, and its efficiency and safety directly affect the reliability of power supply. In recent years, augmented reality (AR) technology has begun to be used in power O&M, providing O&M personnel with intuitive data support by overlaying virtual information onto real devices. However, most current AR O&M solutions rely on physical tags or simple image recognition to locate devices. In the complex environments of actual sites, inaccurate or unidentifiable identification often occurs, failing to cover various complex O&M scenarios.

[0003] Traditional AR-based power maintenance solutions have several prominent problems. First, they rely heavily on physical tags such as QR codes and RFID. If these tags are dirty, obstructed, or detached, they cannot be used to identify devices. Second, the mapping between power data and physical devices needs to be manually configured in advance in the background. When many meters of the same model are installed densely, it is easy to mix up the data. Third, safety warnings use fixed safety distances that do not adjust with the actual operating status of the equipment, making protection relatively passive. Finally, visual positioning technology is prone to errors when there are obstructions or sudden changes in lighting, and AR tags become unstable, failing to meet the needs of long-term maintenance. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an AR operation and maintenance method for smart meters based on cross-modal data association. This method achieves microsecond-level time alignment between power line optimized PTP and GPS / BeiDou dual-backup clock synchronization mechanism. Combined with cross-modal matching of electrical pulse and indicator flashing timing and secondary phase difference determination, it automatically establishes a tagless binding between the meter entity and power data, without relying on physical tags such as QR codes and RFID, and without the need for manual configuration of corresponding relationships in advance. At the same time, it adopts electrical fingerprint-assisted visual positioning recalibration technology, combined with hierarchical spatial mapping and dynamic safety early warning model, to solve the problems of easy drift and fixed safety early warning range of traditional visual positioning. It can transform abstract power information into intuitive AR visualization content, helping operation and maintenance personnel to quickly locate faults and proactively perceive safety risks, effectively improving the efficiency and safety of on-site operation and maintenance.

[0005] To solve the above-mentioned technical problems, this invention provides the following technical solution: an AR operation and maintenance method for smart meters based on cross-modal data association, the specific steps of which are as follows: S1: Employs a dual-backup clock synchronization mechanism consisting of Power Line Optimized PTP Precision Time Protocol and GPS / BeiDou time synchronization. It collects the first-mode power data from the smart meter and the second-mode visual data from the AR terminal. All data is uniformly marked with a high-precision timestamp to ensure strict alignment of the two modes of data in the time dimension, providing a reliable time reference for subsequent cross-modal matching. When both backup synchronizations fail simultaneously, the local crystal oscillator calibration mechanism of the smart meter is activated to avoid data matching errors caused by clock out-of-sync, ensuring that the system can still operate normally under extreme conditions. S2: Extract the electrical fingerprint feature vector of the power mode locally at the smart meter terminal, and extract the indicator flashing timing feature vector of the visual mode at the edge of the AR terminal to realize distributed processing of feature extraction, reduce data transmission bandwidth occupation, and reduce cloud computing pressure; S3: Calculate the cross-correlation function between the power pulse sequence and the indicator light flashing sequence. After a second phase difference determination, establish a unique mapping relationship between the meter entity and the power data, effectively avoiding misjudgment problems caused by a single matching method and significantly improving the matching accuracy. Employ a peak filtering algorithm to suppress multi-meter flashing interference. Dynamically adjust the matching confidence threshold based on ambient light intensity and harmonic noise level. Establish a stable mapping after three consecutive consistent matching results to adapt to interference conditions in different field environments and ensure accurate differentiation of adjacent meters even in densely installed scenarios. When the indicator light cannot be collected, automatically switch to a dual matching mode of electrical fingerprint and meter panel appearance features to achieve fault tolerance and avoid device recognition failure due to indicator light damage or obstruction. S4: A non-intrusive load monitoring algorithm is used to decouple the total current and identify the switching and operating status of downstream electrical appliances. Detailed load composition information can be obtained without the need for additional sensors, reducing system modification costs. The power data is projected onto the AR terminal screen coordinate system through a coordinate transformation matrix to achieve spatial anchoring, ensuring that virtual data and physical devices correspond accurately in space, improving the intuitive perception of maintenance personnel. When visual SLAM drifts and visual features are insufficient, electrical fingerprinting is used for SLAM recalibration to solve the stability problem of pure visual positioning in occlusion and sudden light changes, ensuring that AR tags are always accurately superimposed. S5: Based on real-time voltage levels and contact temperature rise trends, a virtual safety avoidance radius is generated through a dynamic safety radius calculation model. The safety range is dynamically adjusted according to the actual operating status of the equipment, which is more reasonable and protective than a fixed safety distance. The distance between the operator's hand and the equipment is detected and classified into three levels of risk. When entering the corresponding risk domain, a graded visual warning is triggered to realize a gradient warning of risk and avoid excessive interference with normal operation and maintenance.

[0006] Furthermore, in step S1, the power line optimized PTP precise time protocol adopts a sliding window timestamp filtering and phase compensation algorithm, with a synchronization error of <100μs, achieving microsecond-level synchronization accuracy and meeting the stringent time requirements for cross-modal timing matching; the local crystal oscillator calibration mechanism of the electricity meter uses the high-precision crystal oscillator of the AR terminal to calibrate the crystal oscillator of the electricity meter in real time, ensuring that the synchronization error is ≤1ms within 30 minutes, and can still maintain effective synchronization for a relatively long time even when all external synchronization sources fail, thus providing sufficient time for fault diagnosis.

[0007] Furthermore, in step S2, the electrical fingerprint feature vector includes six dimensions of features: active power, reactive power, effective current value, voltage phase, total harmonic distortion rate, and contact temperature rise. The combination of these multi-dimensional features forms a unique device identification, ensuring that the electrical fingerprints of different meters have sufficient distinguishability. The indicator light flashing timing feature vector includes three dimensions of features: flashing frequency, duty cycle, and continuous 5-second flashing sequence. The inherent flashing pattern of the indicator light of the meter is used as a visual recognition feature, without requiring any hardware modification.

[0008] Furthermore, in step S3, the core steps of the peak filtering algorithm are: retaining the maximum peak value in the cross-correlation function calculation results, automatically excluding the matching results corresponding to all other peak values, quickly filtering out interference signals generated by adjacent meters, and greatly improving matching efficiency; The core rule for dynamically adjusting the matching confidence threshold is: when the ambient light intensity is <1000 lux or the harmonic noise is >5%, the matching confidence threshold is automatically adjusted from 0.85 to 0.8. In harsh environments, the matching threshold is appropriately lowered to avoid matching failures caused by environmental interference.

[0009] Furthermore, in step S3, the core step of the phase difference secondary determination is to extract the time offset corresponding to the maximum peak value of the cross-correlation function, compare the time offset with the time offset of the local power pulse of the meter, and determine that the current visual entity and the power data source are the same meter when the time error between the two is <5ms. Through the dual time verification mechanism, the mismatch problem in cross-modal matching is completely eliminated.

[0010] Furthermore, in step S3, the core rule of the dual matching mode is: simultaneously calculate the cosine similarity between the currently acquired electrical fingerprint and the pre-stored electrical fingerprint, and the matching degree between the currently acquired electrical surface panel contour features and the pre-stored appearance features; when the cosine similarity is ≥0.9 and the appearance feature matching degree is ≥0.85, the matching is determined to be successful, and the two independent features simultaneously meet the matching conditions, ensuring the reliability of recognition in the scenario of indicator light failure.

[0011] Furthermore, in step S4, the non-intrusive load monitoring algorithm matches the total current signal with the characteristic current templates of various electrical appliances pre-stored in the digital twin model library, and quickly identifies the load type based on the pre-stored templates, without the need for complex model training, making deployment convenient; The spatial anchoring adopts a hierarchical mapping logic, which specifically includes: anchoring the basic electrical parameter data of the electricity meter to the meter body area, anchoring the load composition data of the downstream electrical appliances to the corresponding line branch area, and anchoring the line health data to the corresponding contact area. Different types of data correspond to different physical locations, making the information presentation more intuitive and clear, and facilitating maintenance personnel to quickly locate problems.

[0012] Furthermore, in step S4, the triggering condition for SLAM recalibration is that the SLAM positioning error is >5cm and the number of effective visual feature points extracted in 3 consecutive frames is <10. Recalibration is only triggered when the positioning quality has indeed deteriorated, so as to avoid frequent calibration affecting the stability of the system. The core steps of SLAM recalibration are as follows: traverse the electrical fingerprint vectors of all meters with established mapping relationships, calculate the cosine similarity, select the meter with the highest similarity (≥0.9) as the matching target, use the known world coordinates of the meter to correct the camera extrinsic matrix of SLAM, and use verified reliable device coordinates to correct the positioning deviation, thus completing relocation without relying on visual features; after each successful match, the spatial anchor point coordinates of the corresponding meter are automatically updated, and after accumulating 10 matches, a permanent stable anchor point is generated. As the number of uses increases, the positioning accuracy continues to improve, eventually reaching a stable state that requires no calibration; when the total harmonic distortion rate is detected to be >5%, the AR screen automatically generates a red indicator arrow pointing to the corresponding connected appliance, and simultaneously displays the appliance's operating data for the past 24 hours, proactively alerting abnormal devices and their historical operating trends, helping maintenance personnel quickly troubleshoot the root cause of the fault.

[0013] Furthermore, in step S5, the output value of the dynamic safety radius calculation model is linearly positively correlated with the voltage level and the contact temperature rise trend. The higher the voltage and the more obvious the temperature rise, the larger the safety avoidance radius and the protection level is automatically improved. The AR terminal's depth camera detects the gesture coordinates of the maintenance personnel and calculates the Euclidean distance between the gesture coordinates and the equipment center coordinates as the distance between the maintenance personnel's hand and the equipment. Real-time detection of personnel position can be achieved without wearing additional positioning equipment, which is convenient to use. The core rules of the three-level risk classification are as follows: the distance > dynamic safety avoidance radius is the safe zone; the distance > 1 / 2 of the dynamic safety avoidance radius and ≤ the dynamic safety avoidance radius is the warning zone; the distance ≤ 1 / 2 of the dynamic safety avoidance radius is the danger zone. When entering the warning zone, a yellow semi-transparent warning circle is superimposed; when entering the danger zone, a red flashing warning circle is superimposed, a voice alarm and terminal vibration are triggered, and unnecessary AR interactive operations are frozen. Different risk levels correspond to different intensity of warning methods, which can serve as a safety reminder without causing excessive interference to normal operation.

[0014] On the other hand, the smart meter AR operation and maintenance system based on cross-modal data association is characterized by comprising: Clock synchronization and data acquisition module: used to realize dual backup clock synchronization, acquire power and visual modal data and mark high-precision timestamps, start local crystal oscillator calibration when synchronization fails, and provide a unified time reference and reliable data source for the entire system; Edge feature extraction module: used to extract feature vectors of power mode and visual mode at the meter and AR terminal respectively, realize the distributed deployment of computing resources, and improve the overall response speed of the system; Cross-modal dynamic association module: It is used to realize the label-free unique matching of electricity meter entities and power data and the suppression of multi-meter interference. It is the core functional module of the system and solves the key problem of automatic association between equipment and data. Load decoupling and spatial anchoring module: used to realize non-intrusive load decoupling and AR spatial mapping of power data, transforming abstract power data into intuitive spatial visualization information; Dynamic safety early warning module: used to generate dynamic safety avoidance radius and trigger graded visual warnings, providing proactive safety protection for on-site operation and maintenance personnel and reducing operational risks.

[0015] Compared with existing technologies, this AR operation and maintenance method for smart meters based on cross-modal data association has the following advantages: I. This invention utilizes a power line-optimized dual-backup clock synchronization mechanism, combined with cross-modal matching of electrical pulses and indicator light flashing timing, and a phase difference secondary determination method, to achieve automatic, tag-free binding between the physical meter and power data. This solves the problem of traditional solutions relying on physical tags and requiring manual configuration of the corresponding relationships in advance. Even with interference from adjacent meters or indicator light malfunctions, device identification can still be completed, ensuring the system can operate normally in complex field environments.

[0016] Second, this invention utilizes electrical fingerprint-assisted visual positioning recalibration technology, combined with hierarchical spatial mapping and a dynamic safety early warning model, to achieve precise spatial fusion of power data and physical entities. This solves the problems of easy drift and fixed safety early warning range in traditional visual positioning. It can transform abstract power information into intuitive AR visualization content, helping maintenance personnel quickly locate faults and proactively perceive potential safety risks, making on-site maintenance work smoother and safer.

[0017] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 This is a schematic diagram of the overall process of the AR operation and maintenance method for smart meters based on cross-modal data association of the present invention; Figure 2 This is a schematic diagram of the cross-modal dynamic correlation and phase difference secondary determination process in this invention; Figure 3 This is a schematic diagram of the dynamic safety avoidance radius calculation and three-level risk classification process in this invention. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] This embodiment provides a smart meter AR operation and maintenance method based on cross-modal data association. By optimizing the PTP power line and synchronizing with GPS / BeiDou dual backup clocks, microsecond-level time alignment between power and visual data is achieved. By using cross-correlation matching of electrical pulses and indicator light flashing timing and secondary phase difference determination, a tagless binding between the meter entity and power data is automatically established. Combined with electrical fingerprint-assisted visual positioning recalibration and dynamic security early warning, it solves the problems of traditional AR operation and maintenance relying on physical tags, data confusion, and unstable positioning, and is adaptable to various on-site operation and maintenance scenarios.

[0022] like Figure 1As shown, the overall process of this embodiment includes five main steps: clock synchronization and data acquisition, edge feature extraction, cross-modal dynamic association, load decoupling and spatial anchoring, and dynamic security early warning.

[0023] This embodiment operates in a distributed system consisting of an industrial-grade AR terminal and a State Grid standard smart meter. The AR terminal is equipped with an 8-megapixel depth camera, a high-precision crystal oscillator with a frequency deviation of <0.1ppm, and an edge computing unit with a computing power of 2TOPS. The smart meter supports the DL / T645 power line communication protocol and has local data processing and storage capabilities. After the system is powered on, each module operates collaboratively according to the established process.

[0024] After system startup, the Power Line Optimized (PTP) Precision Time Protocol is prioritized as the primary synchronization source. This protocol employs a sliding window timestamp filtering and phase compensation algorithm. The sliding window size is fixed at 10 consecutive timestamps. The filtering process first calculates the average of the 10 timestamps, removes outliers with a deviation exceeding 100μs from the average, and then calculates the final synchronization time using the remaining valid timestamps. The phase compensation algorithm establishes a correspondence between delay and length by measuring the transmission delay of different power line lengths, and corrects the signal delay during transmission in real time, ultimately achieving a synchronization error of <100μs under the primary synchronization method. GPS / BeiDou time synchronization serves as a backup synchronization source, automatically switching to the backup source if the primary synchronization source fails to synchronize three times consecutively. When both synchronization sources fail simultaneously, the local crystal oscillator calibration mechanism of the meter is immediately activated. The high-precision crystal oscillator of the AR terminal monitors the frequency deviation of the meter's crystal oscillator in real time. When the deviation exceeds 5ppm, a calibration signal is sent to the meter to correct the oscillation frequency of the meter's crystal oscillator. The calibration operation is performed every 30 seconds to ensure that the synchronization error does not exceed 1ms within 30 minutes. After synchronization, the smart meter collects first-mode power data at a frequency of 50 times per second, including basic electrical parameters such as voltage, current, active power, and reactive power. The AR terminal collects second-mode visual data at a frequency of 25 frames per second, including the electrical surface image and depth information. All collected data are marked with a uniform high-precision timestamp with an accuracy of 1μs.

[0025] The smart meter performs local preprocessing on the collected electricity data. First, it applies a moving average filter to the raw data, with a window size of 5 sampling points, to remove high-frequency noise. Then, it performs min-max normalization on the filtered data, using the following formula: in, These are the normalized eigenvalues, ranging from 0 to 1; x These are the original eigenvalues; This represents the historical minimum value of this dimension feature; This represents the historical maximum value of this dimension feature.

[0026] After normalization, six dimensions of electrical fingerprint feature vectors are extracted: active power, reactive power, RMS current, voltage phase, total harmonic distortion (THD), and contact temperature rise. The AR terminal edge processes the acquired video stream in real time. First, an adaptive threshold segmentation algorithm identifies the indicator light areas on the electrical surface panel, with the threshold automatically adjusted based on ambient light intensity. Then, the on / off state of the indicator lights is recorded frame by frame, with on state marked as 1 and off state as 0, generating a binary flashing sequence. Based on the continuous flashing sequence data, three dimensions of indicator light flashing timing feature vectors are extracted: flashing frequency (the number of times the indicator light turns on and off per unit time); duty cycle (the proportion of the on state duration to the total time); and a continuous 5-second flashing sequence, which is a 125-frame binary sequence corresponding to 125 frames within 5 seconds.

[0027] After receiving the feature vectors of the power mode and the visual mode, the AR terminal first extracts pulses from the power data. When the current surge amplitude exceeds 10% of the rated current, it is identified as a power pulse, the timestamp of the pulse occurrence is recorded, and a power pulse sequence is generated. Then, the cross-correlation function between the power pulse sequence and the indicator light flashing sequence is calculated, as shown in the following formula: in, The cross-correlation function value; This is the time offset; for t The value of the power pulse signal at any given time is 1 when there is a pulse and 0 when there is no pulse. for t The indicator light flashes with a signal value of 1 when it is on and 0 when it is off. N Given the sequence length, take 10 consecutive seconds of data, for a total of 250 sampling points.

[0028] All peak values ​​in the cross-correlation function calculation results are extracted. A peak filtering algorithm is used to retain the largest peak value and automatically exclude all matching results corresponding to other peak values. The peak value threshold is 0.6; only peak values ​​greater than this threshold are retained. Then, a secondary phase difference determination is performed, extracting the time offset corresponding to the largest peak value. This time offset is compared with the power pulse occurrence time offset recorded locally by the meter. When the time error between the two is less than 5ms, the current visual entity and the power data source are determined to be the same meter. Figure 2As shown, the cross-modal dynamic correlation process sequentially performs cross-correlation calculation, peak filtering, and phase difference secondary determination, and dynamically adjusts the matching confidence threshold according to environmental conditions. The system monitors ambient light intensity and harmonic noise levels in real time and dynamically adjusts the matching confidence threshold. When the ambient light intensity is <1000 lux or the harmonic noise is >5%, the matching confidence threshold is automatically adjusted from 0.85 to 0.8. After three consecutive consistent matching results, a stable mapping relationship between the meter entity and the power data is established. When the indicator light cannot be collected due to damage, obstruction, or other reasons, the system automatically switches to dual matching mode. This mode calculates two similarities simultaneously: one is the cosine similarity between the currently collected electrical fingerprint and the pre-stored electrical fingerprint, as shown in the following formula: in, This represents the cosine similarity, with values ​​ranging from -1 to 1. The closer the value is to 1, the higher the similarity. This represents the i-th dimension feature value of the currently acquired electrical fingerprint; For pre-stored electrical fingerprints i The first feature is the feature value. The second is the matching degree between the currently collected electrode surface panel contour features and the pre-stored appearance features. The contour features are extracted using Hu invariant moments, and the matching degree is the reciprocal of the Euclidean distance between the two Hu moment vectors. When the cosine similarity is ≥0.9 and the appearance feature matching degree is ≥0.85, the match is considered successful.

[0029] After establishing a stable mapping relationship, the system employs a non-intrusive load monitoring algorithm to decouple the total current. The collected total current signal is matched with characteristic current templates of various electrical appliances pre-stored in the digital twin model library. The similarity between the two is calculated using a dynamic time warping algorithm, as shown in the following formula: in, This represents the dynamic time-normalized distance between two sequences; a smaller value indicates a higher similarity. X The total current sequence to be matched has a length of [value missing]. m ; Y The template current sequence has a length of n ; The Euclidean distance between corresponding points in the two sequences is given. The template with the highest similarity is selected as the matching result to identify the on / off state and operating state of the connected electrical appliances. The power data in the world coordinate system is projected onto the AR terminal screen coordinate system using a coordinate transformation matrix to achieve spatial anchoring. The coordinate transformation consists of two steps: first, the world coordinates are converted to camera coordinates, and then the camera coordinates are converted to screen coordinates. The transformation formula is as follows: in, These are screen pixel coordinates; KThis is the camera intrinsic parameter matrix, which includes camera intrinsic parameters such as focal length and principal point coordinates; R Let be the rotation matrix, describing the camera's pose; t Let be the translation vector, describing the position of the camera; , , () represents the three-dimensional coordinates in the world coordinate system.

[0030] Spatial anchoring employs a hierarchical mapping logic: basic electrical parameter data of the meter is anchored to the meter body area and displayed in the upper right corner of the meter panel; the load composition data of connected appliances is anchored to the corresponding line branch area and displayed next to the terminal block; and line health data is anchored to the corresponding contact area, with different colors representing health status. When visual SLAM drifts and visual features are insufficient, SLAM recalibration is initiated. The trigger condition is a SLAM positioning error > 5cm and < 10 valid visual feature points extracted in three consecutive frames. The recalibration process iterates through the electrical fingerprint vectors of all meters with established mapping relationships, calculates cosine similarity, and selects the meter with the highest similarity (≥ 0.9) as the matching target. Using the meter's known world coordinates and corresponding image feature points, the PnP algorithm is used to solve for the camera's extrinsic parameter matrix, correcting the SLAM pose estimation results. After each successful match, the spatial anchor point coordinates of the corresponding meter are automatically updated by taking the average of the coordinates from the last 10 matches. After 10 matches, a permanent stable anchor point is generated. Once generated, the permanent anchor point will not change with subsequent matching results. When the total harmonic distortion rate is detected to be greater than 5%, the AR screen automatically generates a red indicator arrow pointing to the corresponding connected appliance and simultaneously displays the appliance's operating data curve for the past 24 hours.

[0031] Based on real-time voltage levels and contact temperature rise trends, the system generates a virtual safety avoidance radius through a dynamic safety radius calculation model, as shown in the following formula: in, R For dynamic safety avoidance radius; The basic safety radius is 0.5m for low-voltage meters; U represents the real-time voltage level. This is the difference between the contact temperature rise and the ambient temperature. This is the voltage proportionality coefficient, taken as 0.001m / V, meaning that for every 1V increase in voltage, the safety radius increases by 0.001m; The temperature rise proportionality coefficient is set to 0.01 m / ℃, indicating that for every 1℃ increase in contact temperature, the safety radius increases by 0.01 m. The output value shows a linear positive correlation with both the voltage level and the contact temperature rise trend. The gesture coordinates of the maintenance personnel are detected by the depth camera of the AR terminal, and the Euclidean distance between the gesture coordinates and the equipment center coordinates is calculated as the distance between the maintenance personnel's hand and the equipment. The formula is as follows: in, d The Euclidean distance between two points; , , ( ) represents the three-dimensional coordinates of the maintenance personnel's hands; , , () represents the three-dimensional coordinates of the equipment center.

[0032] like Figure 3 As shown, a three-tiered risk classification is implemented based on distance: Distance > dynamic safety avoidance radius is the safe zone, triggering no warnings; distance > 1 / 2 of the dynamic safety avoidance radius but ≤ the dynamic safety avoidance radius is the warning zone, with a yellow semi-transparent warning circle overlaid on the AR screen; distance ≤ 1 / 2 of the dynamic safety avoidance radius is the danger zone, with a red flashing warning circle overlaid on the AR screen, triggering both voice alarm and terminal vibration, and freezing unnecessary AR interactive operations. Unnecessary interactive operations include parameter modifications, firmware upgrades, and other operations that may affect device operation; only data viewing functionality is retained.

[0033] In the corresponding smart meter AR operation and maintenance system, the clock synchronization and data acquisition module performs all the functions of the aforementioned clock synchronization and data acquisition stage; the edge feature extraction module performs all the functions of the feature extraction stage; the cross-modal dynamic association module performs all the functions of the cross-modal matching stage; the load decoupling and spatial anchoring module performs all the functions of the load decoupling and spatial mapping stage; and the dynamic safety early warning module performs all the functions of the safety early warning stage. Data transmission between modules is conducted via local Wi-Fi in binary format with a transmission latency of <20ms. When data transmission is interrupted, each module caches the data locally, and automatically synchronizes the cached data upon resumption of transmission.

[0034] This embodiment fully implements an AR-based smart meter maintenance method based on cross-modal data association. It eliminates the need for any physical tags or manual configuration of device-data mappings beforehand. Even in scenarios with damaged, detached, or obstructed tags, or with densely packed multiple meters, it can still accurately identify meters and bind corresponding data. The electrical fingerprint-assisted visual positioning recalibration technology solves the drift problem of traditional visual SLAM in occlusion and sudden light changes, ensuring long-term stability of AR tags. The dynamic safety warning mechanism adjusts the safety range based on the real-time operating status of the device, which better meets actual maintenance needs than a fixed safety distance. This method is applicable to various smart meter deployment scenarios such as residential communities, industrial and commercial parks, and outdoor power distribution rooms, and can also be extended to the maintenance of other power equipment such as transformers and switchgear.

[0035] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A smart meter AR operation and maintenance method based on cross-modal data association, characterized in that, The specific steps of this method are as follows: S1: It adopts a dual-backup clock synchronization mechanism consisting of the power line optimized PTP precise time protocol and GPS / BeiDou time synchronization to collect the first modal data of power from smart meters and the second modal data of vision from AR terminals. All data are uniformly marked with high-precision timestamps. When both backup synchronization fails simultaneously, the local crystal oscillator calibration mechanism of the electricity meter is activated. S2: Extract the electrical fingerprint feature vector of the power mode locally at the smart meter terminal, and extract the indicator flashing timing feature vector of the visual mode at the edge of the AR terminal. S3: Calculate the cross-correlation function between the power pulse sequence and the indicator light flashing sequence, and establish a unique mapping relationship between the meter entity and the power data after secondary phase difference determination; A peak filtering algorithm is used to suppress multi-meter flickering interference. The matching confidence threshold is dynamically adjusted according to the ambient light intensity and harmonic noise level. A stable mapping is established after three consecutive matching results are consistent. When the indicator light cannot be collected, it automatically switches to a dual matching mode of electrical fingerprint and electrical surface panel appearance features. S4: A non-intrusive load monitoring algorithm is used to decouple the total current and identify the switching and operating status of the downstream electrical appliances; the power data is projected onto the AR terminal screen coordinate system through a coordinate transformation matrix to achieve spatial anchoring; when visual SLAM drifts and visual features are insufficient, electrical fingerprinting is used for SLAM recalibration. S5: Based on real-time voltage level and contact temperature rise trend, a virtual safety avoidance radius is generated through a dynamic safety radius calculation model; the distance between the operator's hand and the equipment is detected and classified into three levels of risk, and a graded visual warning is triggered when entering the corresponding risk domain.

2. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S1, the Power Line Optimized PTP Precision Time Protocol uses a sliding window timestamp filtering and phase compensation algorithm, with a synchronization error of <100μs; the local crystal oscillator calibration mechanism of the electricity meter uses the high-precision crystal oscillator of the AR terminal to calibrate the crystal oscillator of the electricity meter in real time, ensuring that the synchronization error is ≤1ms within 30 minutes.

3. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S2, the electrical fingerprint feature vector includes six dimensions of features: active power, reactive power, effective current value, voltage phase, total harmonic distortion rate, and contact temperature rise; the indicator light flashing timing feature vector includes three dimensions of features: flashing frequency, duty cycle, and continuous 5s flashing sequence.

4. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S3, the core step of the peak filtering algorithm is to retain the maximum peak value in the cross-correlation function calculation results and automatically exclude the matching results corresponding to all other peak values. The core rule for dynamically adjusting the matching confidence threshold is: when the ambient light intensity is <1000 lux or the harmonic noise is >5%, the matching confidence threshold will be automatically adjusted from 0.85 to 0.

8.

5. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S3, the core step of the phase difference secondary determination is to extract the time offset corresponding to the maximum peak value of the cross-correlation function, compare the time offset with the time offset of the local power pulse of the meter, and determine that the current visual entity and the power data source are the same meter when the time error between the two is <5ms.

6. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S3, the core rule of the dual matching mode is to simultaneously calculate the cosine similarity between the currently acquired electrical fingerprint and the pre-stored electrical fingerprint, and the matching degree between the currently acquired electrical surface panel contour features and the pre-stored appearance features. A successful match is determined when the cosine similarity is ≥0.9 and the appearance feature matching degree is ≥0.

85.

7. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S4, the non-intrusive load monitoring algorithm matches the total current signal with the characteristic current templates of various electrical appliances pre-stored in the digital twin model library. The spatial anchoring adopts a hierarchical mapping logic, specifically including: anchoring the basic electrical parameter data of the electricity meter to the meter body area, anchoring the load composition data of the connected electrical appliances to the corresponding line branch area, and anchoring the line health data to the corresponding contact area.

8. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S4, the triggering condition for SLAM recalibration is that the SLAM positioning error is >5cm and the number of effective visual feature points extracted in 3 consecutive frames is <10. The core steps of SLAM recalibration are as follows: traverse the electrical fingerprint vectors of all meters with established mapping relationships, calculate the cosine similarity, select the meter with the highest similarity (≥0.9) as the matching target, and use the known world coordinates of the meter to correct the camera extrinsic matrix of SLAM; after each successful match, automatically update the spatial anchor point coordinates of the corresponding meter, and generate a permanent stable anchor point after accumulating 10 matches. When the total harmonic distortion rate is detected to be greater than 5%, the AR screen automatically generates a red indicator arrow pointing to the corresponding hanging appliance and simultaneously displays the appliance's operating data for the past 24 hours.

9. The AR operation and maintenance method for smart meters based on cross-modal data association according to claim 1, characterized in that, In step S5, the output value of the dynamic safety radius calculation model is linearly positively correlated with the voltage level and the contact temperature rise trend; the gesture coordinates of the maintenance personnel are detected by the depth camera of the AR terminal, and the Euclidean distance between the gesture coordinates and the center coordinates of the equipment is calculated as the distance between the maintenance personnel's hand and the equipment. The core rules of the three-level risk classification are as follows: a distance greater than the dynamic safety avoidance radius is a safe zone; a distance greater than 1 / 2 of the dynamic safety avoidance radius and less than or equal to the dynamic safety avoidance radius is a warning zone; a distance less than or equal to 1 / 2 of the dynamic safety avoidance radius is a danger zone; when entering the warning zone, a yellow semi-transparent warning circle is superimposed; when entering the danger zone, a red flashing warning circle is superimposed, a voice alarm and terminal vibration are triggered, and unnecessary AR interactive operations are frozen.

10. A smart meter AR operation and maintenance system based on cross-modal data association, applying the method described in any one of claims 1-9, characterized in that, include: Clock synchronization and data acquisition module: used to realize dual backup clock synchronization, acquire power and visual modal data and mark them with high-precision timestamps, and start local crystal oscillator calibration when synchronization fails; Edge feature extraction module: used to extract feature vectors of power mode and visual mode at the meter end and AR terminal respectively; Cross-modal dynamic association module: used to achieve label-free unique matching between electricity meter entities and power data and to suppress interference from multiple meters; Load decoupling and spatial anchoring module: used to achieve non-intrusive load decoupling and AR spatial mapping of power data; Dynamic safety warning module: used to generate dynamic safety avoidance radius and trigger graded visual warnings.

Images