Intelligent grounding wire state monitoring method based on multi-modal information fusion

Abstract

The invention relates to the technical field of power equipment monitoring, in particular to a ground wire state intelligent monitoring method based on multi-modal information fusion, which comprises the following steps: acquiring ground wire multi-modal monitoring data including image data, inertial sensing data and time sequence task data; a deep learning target detection algorithm is adopted for the image data to identify key components in the image, wherein the key components at least comprise a wire clamp, a connecting bolt and a hanging point; the preliminary judgment is based on the identified key components, the spatial relation features, geometric features and visual features of all the components are analyzed, and a preliminary result of the state of the grounding wire is generated according to a preset judgment rule; fusion verification: performing multi-modal fusion verification on the judged preliminary result, the attitude data and the time sequence task data, and outputting a final state judgment result; according to the invention, by fusing the image data, the inertial sensing data and the time sequence task data, accurate judgment and graded early warning of the hitching state of the ground wire are realized, and the safety risk of electric power overhaul operation is reduced.

Description

Technical Field

[0001] This invention relates to the field of power equipment monitoring technology, specifically to an intelligent monitoring method for grounding wire status based on multimodal information fusion. Background Technology

[0002] Grounding wires are crucial safety devices during power system maintenance. Their function is to conduct residual charge from the equipment under maintenance to the earth, preventing electric shock to workers. However, in actual operations, human error (such as missing connection or removal) or environmental factors (such as wind or vibration causing loose connections) can easily lead to problems like poor grounding wire connections or missing connections, potentially causing electric shock accidents that seriously threaten the lives of workers and the stable operation of the power system.

[0003] Existing grounding wire status monitoring methods mainly rely on manual inspection or single-sensor monitoring: manual inspection is inefficient and difficult to achieve comprehensive monitoring in complex operating environments; single-sensor monitoring (such as image recognition alone) is susceptible to environmental interference. For example, image recognition alone may lead to misjudgments of key components due to light obstruction, resulting in insufficient monitoring accuracy and inability to accurately determine the true status of the grounding wire. These methods fail to meet the high reliability requirements of power systems.

[0004] Therefore, it is necessary to design an intelligent monitoring method that integrates multimodal information to improve the above-mentioned problems. Summary of the Invention

[0005] To address the problems of existing technologies, this invention provides an intelligent monitoring method for grounding wire status based on multimodal information fusion, comprising the following steps: Image acquisition: Acquire multimodal monitoring data of the grounding wire, including image data, inertial sensing data, and time-series task data; Data processing: A deep learning object detection algorithm is used to identify key components in the image data. The key components include at least wire clamps, connecting bolts, and hanging points. Preliminary judgment: Based on the identified key components, the spatial relationship characteristics, geometric characteristics and visual characteristics of each component are analyzed, and a preliminary result of the grounding wire status is generated according to the preset judgment rules; Fusion verification: The preliminary judgment results are fused with attitude data and temporal task data for multimodal verification, and the final state judgment result is output. Warning Trigger: If the final status determination result is an abnormal status, the corresponding warning information will be triggered.

[0006] Furthermore, the deep learning object detection algorithm is either YOLO or SSD deep learning object detection algorithm.

[0007] Furthermore, the clamp attitude data includes the clamp's tilt angle, sway amplitude, and acceleration change rate data.

[0008] Furthermore, the time-series task data includes task stage data for "connecting the grounding wire" and "removing the grounding wire" as well as corresponding expected status data.

[0009] Furthermore, the preset determination rules include: Correctly connected: The angle between the wire clamp and the connection point is within the preset range, the gap between them is less than or equal to the preset threshold, and the connecting bolts are locked. Missing hook: The wire clamp is suspended and there is no matching hook point in the image; Poor connection: The clamp is in contact with a non-target connection point, or the contact angle or gap between the clamp and the connection point exceeds the preset range. Removed: The wireless clip at the original attachment point has been removed, and it matches the scheduled mission time.

[0010] Furthermore, the logic for the multimodal fusion verification is as follows: When the initial determination is "correctly connected", the attitude data must meet the preset threshold and be consistent with the expected state data in the "connecting the ground wire" stage; When initially determined to be "removed", it must be consistent with the expected status data of the "removal of ground wire" stage and the characteristics of the original wireless clip at the connection point; If the initial judgment is "missing" or "poorly attached", further verification is required based on attitude data.

[0011] Furthermore, the warning information includes: Level 1 warning: This includes the risk of short circuit due to poor connection, which will trigger an audible and visual alarm, SMS push or telephone notification; Level 2 warning: This includes situations where the number of missed hang-ups exceeds the first preset time threshold, or the number of missed removals exceeds the second preset time threshold, triggering an APP push or SMS reminder; Level 3 warning: This includes warnings when the cable clamp tilt angle approaches the safety threshold or the battery level is lower than the set threshold, which will only be displayed on the app.

[0012] The beneficial effects of this invention are: This invention presents an intelligent monitoring method for grounding wire status based on multimodal information fusion. By integrating image data, inertial sensor data, and time-series task data, it overcomes the limitations of single data sources, effectively avoids misjudgments caused by environmental interference, and achieves an accuracy rate of over 98%. It pre-sets multiple status judgment rules, covering common issues such as missing grounding wires and poor connections, meeting the full-scenario monitoring needs of power maintenance operations. Differentiated early warning strategies are formulated based on risk levels, ensuring rapid response to high-risk hazards while avoiding resource waste caused by excessive early warnings. Data collection, status judgment, and early warning push can be completed without manual intervention, significantly improving monitoring efficiency and reducing labor costs. Attached Figure Description

[0013] Figure 1 This is a flowchart of the intelligent monitoring method for grounding wire status based on multimodal information fusion according to the present invention; Figure 2 This is a schematic diagram of the intelligent monitoring system for grounding wire status of the present invention; Figure 3 This is a schematic diagram of the installation of the line-end monitoring device of the present invention; Figure 4 A schematic diagram of the structure of the housing of the line-end monitoring device of the present invention; Figure label: In the diagram: 1-Intelligent device box, 2-Line end monitoring device, 3-Insulating rod. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] Please see Figure 1-4 A grounding wire status intelligent monitoring method based on multimodal information fusion includes the following steps: Image acquisition: acquiring multimodal monitoring data of the grounding wire, including image data, inertial sensing data and time-series task data; the image data is a visual image of the grounding wire clamp and the connection point, the inertial sensing data is clamp attitude data, and the time-series task data comes from the task management terminal; Data processing: Deep learning object detection algorithms are used to identify key components in the image data. The key components include at least wire clamps, connecting bolts, and mounting points (such as busbars or grounding wire connectors). Preliminary judgment: Based on the identified key components, the spatial relationship characteristics, geometric characteristics and visual characteristics of each component are analyzed, and a preliminary result of the grounding wire status is generated according to the preset judgment rules; Fusion verification: The preliminary judgment results are fused with attitude data and temporal task data for multimodal verification, and the final state judgment result is output. Warning Trigger: If the final status determination result is an abnormal status, the corresponding warning information will be triggered.

[0016] It should be noted that the image data is captured by a miniature high-definition camera, taking clear images of the grounding wire and its surrounding environment, including the clamp, connecting bolts, and mounting points; the inertial sensing data is collected by an inertial measurement unit installed on the clamp, including attitude data such as the clamp's tilt angle, sway amplitude, and rate of change of acceleration; the timing task data is entered by the operator via PC / APP, including the start and end times of the "connecting the grounding wire" and "removing the grounding wire" tasks, as well as the expected status data for each stage (such as the expected clamp and mounting point to be in contact during the "connecting the grounding wire" stage).

[0017] Data processing employs deep learning object detection algorithms (YOLO, You Only Look Once) or (SSD, Single Shot MultiBox Detector) to process image data and identify key components (clips, bolts, and mounting points) in the images. These algorithms are characterized by strong real-time performance and high detection accuracy, enabling them to quickly locate key components in complex backgrounds and output the position coordinates, dimensions, and other geometric features of each component. The preliminary judgment step is based on the identified key components, extracting the spatial relationship features (such as the relative position of the clamp and the hanging point), geometric features (such as the fitting gap), and visual features (such as the locking status of the connecting bolts) of each component, and generating a preliminary judgment result by combining them with preset judgment rules. The preset judgment rules clearly define the judgment criteria for four core states: "correctly hung," "missing," "poorly hung," and "removed," ensuring the standardization of the preliminary judgment. To avoid misjudgment from a single data source, the fusion verification step involves performing multimodal fusion verification with attitude data and time-series task data: If the initial determination is "correctly connected," it is necessary to verify whether the attitude data (tilt angle, sway amplitude, etc.) are within the preset thresholds (e.g., tilt angle ≤ 5°, sway amplitude ≤ 3mm) and consistent with the expected state data in the "connecting the ground wire" stage. After confirming there are no abnormalities, the final result "correctly connected" is output. If the initial determination is "removed," it is necessary to verify that this state matches the planned time of the "removing the ground wire" task, and that no wire clamp features are detected at the original connection point for more than 3 seconds to avoid misjudgment due to image occlusion. If the initial determination is "missed connection" or "poor connection," it is necessary to combine the fluctuation characteristics of the attitude data (e.g., acceleration change rate > 0.5m / s²). 2 (And continue for 2 seconds) for further verification, and output the final result after confirming the anomaly; Furthermore, the deep learning object detection algorithm is either YOLO or SSD deep learning object detection algorithm; Furthermore, the clamp attitude data includes the clamp's tilt angle, sway amplitude, and acceleration change rate data, and is filtered to eliminate data noise. When the attitude data exceeds a preset threshold, an abnormal alarm is triggered. Furthermore, the time-series task data includes task stage data for "connecting the grounding wire" and "removing the grounding wire" as well as corresponding expected status data.

[0018] Furthermore, the preset determination rules include: Correctly connected: The angle between the wire clamp and the connection point is within the preset range, the gap between them is less than or equal to the preset threshold, and the connecting bolts are locked. Missing hook: The wire clamp is suspended and there is no matching hook point in the image; Poor connection: The clamp is in contact with a non-target connection point, or the contact angle or gap between the clamp and the connection point exceeds the preset range. Removed: The wireless clip at the original attachment point has been removed, and it matches the scheduled mission time.

[0019] Furthermore, the logic of the multimodal fusion verification is as follows: when it is initially determined that "it has been correctly attached", the attitude data must meet the preset threshold and be consistent with the expected state data of the "attaching ground wire" stage; when it is initially determined that "it has been removed", it must be consistent with the expected state data of the "removing ground wire" stage and have the wireless clip characteristics of the original attachment point; when it is initially determined that "it has been missed" or "it has been poorly attached", it needs to be further verified in combination with attitude data.

[0020] Furthermore, the early warning information includes a tiered early warning mechanism: Level 1 warning: This includes the risk of short circuit due to poor connection, which will trigger an audible and visual alarm, SMS push or telephone notification; Level 2 warning: This includes situations where the number of missed hang-ups exceeds the first preset time threshold, or the number of missed removals exceeds the second preset time threshold, triggering an APP push or SMS reminder; Level 3 warning: This includes situations where the cable clamp tilt angle is close to the safety threshold or the battery level is lower than the set threshold, and the warning message is only displayed on the APP. It should be noted that the warning triggering steps are based on the final status judgment result, triggering three levels of warnings according to the severity: Level 1 warning is for high-risk scenarios where poor connection leads to short circuits. In addition to on-site audible and visual alarms, SMS and telephone notifications are simultaneously sent to management personnel to ensure rapid response; Level 2 warning is for medium-risk scenarios where missed connection exceeds the first preset time threshold (e.g., 10 minutes) or missed disconnection exceeds the second preset time threshold (e.g., 30 minutes). The APP and SMS will remind the operators to handle the situation in a timely manner; Level 3 warning is for low-risk scenarios where the clamp tilt angle is close to the safety threshold (e.g., the tilt angle reaches 4°) or the equipment battery power is below 20%. The warning information is only displayed on the management APP to avoid excessive warnings.

[0021] The intelligent monitoring system for grounding wire status based on multimodal information fusion includes a wire end monitoring device 2, a data processing device, a back-end server, and a task management terminal; The line end monitoring device 2 can be detachably installed on the insulating rod 3 of the grounding wire clamp. It has a built-in IMU sensor and a miniature high-definition camera for real-time acquisition of image data, inertial sensing data and timing task data of the grounding wire. The data processing device is wirelessly connected to the line end monitoring device 2 via the Star Flash communication protocol. It is used to receive and process the image data, inertial sensing data and timing task data of the grounding wire, and communicate with the remote management platform via the 5G network. The backend server is used to store monitoring data, run deep learning target detection algorithms and multimodal fusion judgment logic, and generate monitoring reports and early warning information. The task management terminal includes a PC / APP client, which is used to input time-series task data, issue task plans based on the time-series task data, view real-time monitoring status, and receive early warning information.

[0022] The line end monitoring device 2 and the data processing device are installed in the smart device box 1. The smart device box 1 has a dedicated compartment for accommodating the line end monitoring device 2 and the data processing device. Charging contacts are provided at the bottom of the compartment for centralized charging and management of the line end monitoring device 2 and the data processing device. It should be noted that the grounding wire status monitoring terminal device includes three independent grounding rod structures for phases A, B, and C. Each grounding rod structure includes a clamp and an insulating rod 3. The clamp is located at the top of the insulating rod 3. The inner side of the clamp is provided with two opposing clamping hooks. The back side of the clamp is provided with a grounding wire connecting bolt. The grounding wire is connected to the grounding wire on the grounding wire connecting bolt. The grounding wires of the three phases are connected to the same grounding rod. Furthermore, the line-end monitoring device 2 includes: The outer shell is made of high-strength lightweight composite material. The outer shell is equipped with an arc-shaped clamping claw that can be adapted to different types of grounding wires and a rectangular shell. The end of the arc-shaped clamping claw is equipped with a press-type unlocking button, which can realize quick disassembly and assembly without tools and without damaging the surface of the wire clamp. The arc-shaped clamping claw is located outside the rectangular shell, and the clamping claw and the rectangular shell are integrally formed. A miniature high-definition camera is provided. A rectangular housing has an installation opening at one end corresponding to the camera. The main body of the miniature high-definition camera is installed inside the rectangular housing. The lens of the camera extends out of the rectangular housing through the installation opening and is located at the front end of the wire clamp, used to collect image data between the wire clamp and the connection point. The miniature high-definition camera is equipped with a motor-driven automatic opening and closing protective cover. The wire end monitoring device 2 is equipped with a main control unit. A motor drive assembly and a slide rail structure are connected to the outer periphery of the installation opening of the rectangular housing. The automatic opening and closing protective cover is opened and closed through the motor drive assembly and the slide rail structure. The automatic opening and closing protective cover is electrically connected to the main control unit, enabling the automatic opening and closing protective cover to open when the miniature high-definition camera is started and to close when the camera is not in operation. A star-flash communication module is installed inside the rectangular housing. The star-flash communication module is communicatively connected to the line-end monitoring device 2 to realize wireless communication with the data processing device. The IMU sensor is an inertial measurement unit, installed inside the rectangular housing. It includes an accelerometer and a gyroscope, and is used to collect attitude data of the cable clamp and the attachment point. The attitude data includes the tilt angle, sway amplitude and acceleration change rate of the cable clamp. When the attitude data exceeds a preset threshold, an abnormal alarm is triggered. Magnetic or snap-on charging contacts are located at the bottom of the rectangular housing; Furthermore, the miniature high-definition camera has 8 megapixels and an F2.0 aperture, and is installed 5-8cm directly in front of the cable clip attachment point to accurately acquire image data of the cable clip; It should be noted that a miniature high-definition camera (8 megapixels, F2.0 aperture) is installed 5-8cm directly in front of the cable clamp connection point. The lens is covered with a scratch-resistant hydrophobic coating (hardness up to 9H), and a motor-driven automatic protective cover is installed (closed when not in operation, and opens within 0.5 seconds when in operation). The rectangular housing has an installation opening at one end corresponding to the miniature high-definition camera. The main body of the miniature high-definition camera is installed inside the rectangular housing, and the lens of the miniature high-definition camera extends out of the rectangular housing through the installation opening and is located at the front end of the cable clamp, used to collect image data of the cable clamp and the connection point. The miniature high-definition camera is equipped with a motor-driven automatic opening and closing protective cover. The cable end monitoring device 2 is equipped with a main control unit. The outer periphery of the installation opening of the rectangular housing is connected to a motor drive component and a slide rail structure (electromagnetic motor, transmission slide rail). The automatic protective cover is opened and closed through the motor drive component and the slide rail structure. The automatic opening and closing protective cover is electrically connected to the main control unit, realizing the automatic opening and closing of the protective cover when the miniature high-definition camera is started and the automatic opening and closing of the protective cover when the miniature high-definition camera is not in operation (sleep or charging). Key components are identified using deep learning object detection algorithms. Based on these key components, the spatial relationship features (relative position, fitting angle), geometric features (fitting gap), and visual features (locking status of connecting bolts) between the clamp and the mounting point are analyzed. Furthermore, the data processing device is networked with at least three line-end monitoring devices 2; the data processing device is equipped with an adjustable 5G external antenna and a star flash antenna, which are combined with the star flash communication module to ensure stable and reliable data transmission when networked with multiple line-end monitoring devices 2; The data processing device is equipped with a status indicator light group on its surface, which includes a power indicator light, a communication indicator light, and an alarm indicator light. The data processing device is equipped with a power management circuit and a main power supply circuit. The main control unit is equipped with a reset pin. The power indicator light is electrically connected to the power management circuit of the data processing device to display the device's power supply status. The communication indicator light is electrically connected to the 5G external antenna and the StarSignal communication module to distinguish between the 5G network communication status and the StarSignal networking communication status. The alarm indicator light is electrically connected to the main control unit of the data processing device to trigger an alarm when abnormal data is detected at the receiving end or when the device itself malfunctions. The surface of the data processing device is also equipped with a physical button group, which includes an independent power switch button and a reset button. The power switch button is connected in series in the main power supply circuit of the data processing device to manually control the power supply of the device. The reset button is electrically connected to the reset pin of the main control unit of the data processing device to trigger the main control unit to reset, so that the data processing device returns to its initial operating state.

[0023] It should be noted that the data processing device has a status indicator light group on its surface, which includes a power indicator light, a communication indicator light, and an alarm indicator light. The power indicator light is electrically connected to the power management circuit of the data processing device and is used to display the current power supply status of the device (e.g., normally lit when powered on, flashing when low battery, and off when power is off). The communication indicator light is electrically connected to the 5G external antenna and the StarSignal communication module, respectively, and is used to distinguish between the 5G network communication status (e.g., normally lit when network connection is successful, flashing when data transmission is successful, and off when communication is interrupted) and the StarSignal networking communication status (e.g., normally lit when networking is successful, and flashing when the device is offline). The alarm indicator light is electrically connected to the main control unit of the data processing device. When the received line-end monitoring data is abnormal (e.g., attitude exceeds threshold, image recognition is abnormal) or When the data processing equipment itself malfunctions (such as antenna failure or module offline), the alarm indicator light will flash. The data processing equipment also has a physical button group on its surface, including an independent power switch and a reset button. The power switch is connected in series in the main power supply circuit of the data processing equipment and is used to manually control the power supply to the equipment (a single press turns on the power and starts the data processing equipment; a second press turns off the power and shuts it down). The reset button is electrically connected to the reset pin of the main control unit of the data processing equipment. When the data processing equipment experiences abnormal conditions such as program freeze or communication interruption, pressing the reset button will trigger the main control unit's reset circuit to generate a reset signal, restoring the data processing equipment to its initial operating state and clearing the abnormal fault.

[0024] Furthermore, the smart device box 1 also includes: The access control module uses electronic locks and RFID identification technology to control the opening permissions of the smart device box 1 and record operation logs. The power management module has a built-in high-capacity lithium battery and uses an intelligent charging algorithm to charge the monitoring device 2 and data processing device inside the intelligent device box 1 independently or synchronously. The status monitoring module is electrically connected to and works in conjunction with the access control module and the power management module to achieve multi-dimensional status monitoring and abnormal alarms. It should be noted that the smart device box 1 also includes an access control module, a power management module, and a status monitoring module. The structure and function of each module are adapted as follows: Access Control Module: The module adopts a combination of electronic lock and RFID identification technology. The electronic lock is installed at the connection between the door and the body of the smart device box 1, and the RFID identification component is integrated into the inside of the door or the control panel of the box. Only authorized personnel with RFID cards can unlock the electronic lock to open the box. The module also automatically records the time of each opening and closing of the box, the information of the authorized personnel, and the duration of the operation, forming an unalterable operation log to achieve access control and operation traceability.

[0025] Power Management Module: Built-in high-capacity lithium battery as the core power supply, configured with intelligent charging algorithm (integrating multi-stage constant current and constant voltage charging, multi-device load balancing, battery health protection and adaptive switching logic for charging status); Each dedicated compartment in the intelligent device box 1 is equipped with charging contacts at the bottom. The power management module provides independent or synchronous charging modes for the line end monitoring device 2 and data processing device placed in the compartment through these contacts. During the charging process, the current and voltage parameters are adjusted in real time to balance charging efficiency, safety protection and battery life.

[0026] The status monitoring module is electrically connected to and works in conjunction with the access control module and power management module to achieve multi-dimensional status monitoring and anomaly alarms. Equipment in-place status monitoring: In-place detection contacts are installed in each dedicated compartment, and capacitive sensors or infrared beam sensors are used to determine whether the line end monitoring device 2 and data processing device are fully returned to their positions through logical verification of circuit on / off signals and auxiliary sensing signals. Battery power monitoring: A data transmission channel is established through the charging contacts to receive battery power, voltage and other data transmitted by the built-in battery management system of the device inside the box. After being corrected by the built-in calibration algorithm, the battery status of the device is accurately obtained. Cabinet opening and closing status monitoring: A magnetic proximity switch (including permanent magnet and reed switch / Hall sensor) is installed at the connection between the cabinet door and the cabinet to monitor the opening and closing status of the cabinet. After the cabinet is closed, ensure that the status monitoring module is in good contact and start charging. Abnormal alarm function: Preset abnormal judgment thresholds such as missing equipment, not returned to its original position, low battery, charging failure, illegal opening of the cabinet, and door not closing for timeout. When an abnormal state is detected, a local alarm is triggered (corresponding to the indicator light changing color and flashing, and the buzzer sounding) and a remote alarm are triggered (alarm information is pushed to the remote management platform through the 5G network of the data processing equipment, and the staff's APP pop-up window or telephone notification are linked simultaneously). The alarm level is set according to the degree of abnormal risk.

[0027] Furthermore, the charging contacts of the line-end monitoring device 2 adopt a recessed structure, and the groove depth of the recessed structure is 2-3mm; It should be noted that the recessed structure of the charging contacts on the line-end monitoring device 2 is designed to ensure charging reliability, improve contact protection, and extend the device's lifespan. Specifically, the recessed design prevents the charging contacts from being exposed on the rectangular housing surface; instead, they are embedded in a contact mounting groove at the bottom of the housing (the groove depth is typically 2-3mm). This prevents direct collisions and friction between the contacts and external hard objects (such as tools, other equipment, or the ground) during disassembly, transport, or storage, preventing deformation, wear, or oxidation of the contact metal sheets and ensuring good conductivity even after long-term use. It also prevents accidental short circuits caused by operator touches or corrosion from sweat or dust, improving safety. The recessed mounting groove also acts as a "guide and positioning" mechanism, fitting snugly with the raised charging contacts at the bottom of the smart device box 1 compartment. When the device is placed in the smart device box 1 compartment, the raised contacts can quickly and accurately align along the inner wall of the recessed groove, avoiding poor charging contact due to misalignment. The sidewalls of the charging tank restrict lateral movement of the equipment within the compartment, ensuring a tight seal between the contacts during charging. This prevents connection disruptions due to slight shaking of the enclosure or vibrations during transport, guaranteeing continuous charging. The line-end monitoring device 2, designed for use in dusty, rainy, and other complex outdoor environments, features a recessed structure combined with a sealed housing design to enhance contact protection. It effectively prevents dust, mud, rainwater, and other foreign objects from entering the contact area, avoiding short circuits and increased contact resistance caused by debris accumulation, thus meeting the overall IP65 protection rating requirements of the equipment. It also reduces the contact area between moisture and corrosive gases in the air, lowering the probability of contact oxidation and rust, ensuring stable charging even in harsh environments such as high humidity and salt spray.

[0028] Furthermore, the IMU sensor has an angular accuracy of 0.01° and is used to collect the attitude data of the cable clamp. This attitude data includes tilt angle, sway amplitude, and acceleration rate of change. An abnormal alarm is triggered when the attitude data exceeds a preset threshold. Specific implementation process: 1. Deployment of the data acquisition module Install a miniature high-definition camera (1080P resolution, 30fps frame rate) and an inertial measurement unit (100Hz sampling rate) on the grounding wire clamp. The camera should be facing the connection point to ensure clear imaging of key components. Operators should be equipped with a PC / APP (mobile APP or dedicated equipment) to input timing task data.

[0029] Data transmission and storage: The monitoring data is transmitted to the back-end server in real time using a 5G / 4G network. The storage module uses a cloud database with a storage period of 1 year, supporting data backtracking and analysis.

[0030] Algorithm parameter settings: Deep learning object detection algorithm: YOLOv8 model is used. The training set contains 5000 ground wire images under different environments (covering sunny days, rainy days, nighttime and other scenes). The model detection confidence threshold is set to 0.8. Preset thresholds: The preset contact angle range is 0°-5°, the preset contact gap threshold is 2mm, the first preset time threshold (missed hanging) is 10 minutes, the second preset time threshold (missed removal) is 30 minutes, and the attitude fluctuation threshold (rate of change of acceleration) is 0.5m / s². 2 .

[0031] 2. Workflow Example (1) The operator enters the task of "hanging ground wire" through PC / APP terminal, sets the task start time to T0, and the expected state is "the wire clamp is in contact with the hanging point and the bolt is locked"; (2) The grounding wire image was captured by a miniature high-definition camera. The YOLOv8 algorithm was used to identify the clamp, connecting bolt and hanging point. The clamp and hanging point were found to have a contact angle of 3° and a contact gap of 1mm. The connecting bolt was locked. It was initially determined that the wire was "correctly hung". (3) The inertial measurement unit collected data showing that the wire clamp tilt angle was 2°, the sway amplitude was 1.5 mm, and the rate of change of acceleration was 0.2 m / s². 2 If the preset attitude threshold is met and the expected state is consistent with the "grounding wire" stage, the final result "correctly connected" will be output after fusion verification. (4) If after 1 hour, the inertial measurement unit detects that the tilt angle of the clamp changes to 6° and the rate of change of acceleration is 0.8 m / s² 2 The miniature high-definition camera captured a 3mm gap between the wire clip and the mounting point, which was initially determined to be a "poor mounting". After verification, the anomaly was confirmed and a level one warning was triggered: the on-site audible and visual alarm was activated, and the management personnel received SMS and telephone notifications. (5) After the operator finishes processing, the task of “removing the ground wire” is entered through the PC / APP terminal. The system detects the wireless clip feature at the original attachment point and matches the task schedule time, outputs the final result of “removed”, and the warning is lifted.

[0032] It is worth noting that (1) this application uses multimodal data fusion technology to deeply integrate image data, inertial sensing data and time-series task data to form a triple verification mechanism of "visual features, posture features and task time-series features", which makes up for the limitations of a single data source. At the image data level, YOLO or SSD deep learning target detection algorithms are used to accurately identify key components such as clamps, connecting bolts and hanging points in complex environments such as sunny days, rainy days and nights, and output visual features such as component position coordinates and geometric dimensions to provide a basis for state judgment; at the inertial sensing data level, by installing an inertial measurement unit (IMU) on the clamp, attitude data such as clamp tilt angle, sway amplitude and acceleration change rate are collected in real time to capture the dynamic state changes of the clamp and avoid subtle anomalies that cannot be reflected by static images; at the time-series task data level, the task stages and expected states of "connecting the ground wire" and "removing the ground wire" are entered through the PC / APP terminal to provide time dimension rationality verification for the monitoring results. For example, when the system initially determines that the clamp is "correctly connected" through image recognition, it further verifies whether the inertial sensing data meets preset thresholds (such as tilt angle ≤ 5°, sway amplitude ≤ 3mm). Simultaneously, it confirms that the current time is within the "grounding wire connection" task phase and is consistent with the expected state. This triple verification complements and constrains each other, effectively eliminating misjudgments caused by environmental interference. Actual test data shows that this multimodal fusion mechanism can improve the accuracy of grounding wire status determination to over 98%. Even in severe environments such as heavy rain and strong winds, the accuracy remains above 95%, exceeding the performance level of single monitoring methods and solving the problem of "insufficient monitoring accuracy" in the background technology. (2) This application constructs a state determination system that fully covers the entire life cycle of the grounding wire by pre-setting multiple state determination rules, covering four key states: “correctly connected”, “missing connection”, “poor connection”, and “removed”, which accurately corresponds to common problems in power maintenance operations. In the "correctly connected" judgment, not only are the contact angle and gap between the clamp and the connection point required to meet preset standards, but it is also necessary to confirm that the connecting bolts are locked to avoid hidden risks caused by loose bolts. In the "missed connection" judgment, by identifying the clamp's suspended state and the absence of a matching connection point in the image, combined with the timing information of the "grounding wire connection" task phase, the operator's operational error of not connecting the grounding wire can be quickly identified. In the "poor connection" judgment, three scenarios are subdivided: "clamp in contact with a non-target connection point," "contact angle exceeding the standard," and "contact gap exceeding the standard," to accurately locate the specific cause of the connection abnormality and provide clear guidance for subsequent rectification. In the "removed" judgment, by detecting the absence of clamps at the original connection point and matching the planned time of the "grounding wire removal" task, the risk of power supply to the line due to the omission of the grounding wire can be avoided.

[0033] Taking the "poor connection" scenario in power maintenance as an example, when a clamp is mistakenly attached to a non-target connection point (such as the grounding terminal of an adjacent line), the system detects the deviation between the clamp's position and the preset connection point through image recognition. Simultaneously, inertial sensor data shows that the clamp's tilt angle exceeds the preset range of 5°. Combined with the expected state of the "grounding wire connection" task phase, the system quickly determines it as a "poor connection" and triggers an early warning. This comprehensive, categorized state determination mechanism covers the entire power maintenance process, from pre-connection preparation, the connection process, post-connection monitoring to dismantling verification. It meets the monitoring needs of different operational stages, solves the problem of "incomplete state coverage" in the background technology, and provides comprehensive protection for the safety of maintenance operations. (3) This application formulates differentiated early warning strategies for different risk scenarios to achieve the precise early warning goal of "rapid response to high risks, timely reminders for medium risks, and appropriate prompts for low risks". Level 1 early warning is for high-risk scenarios where poor connection leads to short circuits. In addition to activating the sound and light alarm on site, it also sends SMS and telephone notifications to management personnel to ensure that the early warning is reached within 3 minutes, thus buying time for emergency response. Level 2 early warning is for medium-risk scenarios where the connection is missed for more than 10 minutes or the disconnection is missed for more than 30 minutes. It reminds the operators through APP push and SMS, which avoids the risk from expanding and does not require excessive use of emergency resources. Level 3 early warning is for low-risk scenarios where the clamp tilt angle is close to the safety threshold (such as the tilt angle reaches 4°) and the equipment battery power is less than 20%. It only displays prompt information on the APP to reduce interference with normal operations.

[0034] In practical applications, this tiered early warning mechanism demonstrates significant resource optimization effects. For example, when the system detects a sudden increase in the rate of change of the clamp acceleration to 0.8 m / s², it can effectively mitigate these effects. 2 (far exceeding 0.5m / s) 2 When the image shows that the gap between the clamp and the mounting point has widened to 3mm (exceeding the preset fluctuation threshold), it is judged as "poor mounting" and a level one warning is triggered. Management personnel can then notify the site immediately by phone to handle the situation and prevent short circuits. Conversely, when the device battery level drops to 18%, a level three warning is only displayed on the app, allowing operators to replace the battery during maintenance breaks without interrupting current work. This differentiated warning strategy ensures rapid response to high-risk hazards while avoiding resource waste caused by excessive warnings, solving the problem of "low warning efficiency" in the background technology and achieving a balance between safety and resource efficiency.

[0035] (4) This application constructs a fully automated system of "data acquisition - data processing - status determination - early warning push - data storage" to achieve intelligent monitoring without human intervention, which greatly improves monitoring efficiency and reduces labor costs. In the data acquisition stage, miniature high-definition cameras and inertial measurement units (IMUs) can automatically acquire image and attitude data in real time. The PC / APP terminal supports operators to quickly enter time-series task data without manual on-site recording. In the data processing stage, deep learning algorithms automatically complete the identification and feature extraction of key components, replacing manual image analysis. In the status determination stage, the system automatically generates preliminary and final determination results based on multimodal data fusion logic, without manual judgment. In the early warning push stage, the early warning module automatically triggers the corresponding level of early warning information according to the final status and directly pushes it to the relevant personnel terminals. In the data storage stage, the cloud database automatically stores monitoring data, determination results and early warning records, supporting subsequent retrospective analysis.

[0036] Taking the monitoring of grounding wires in a large substation as an example, this substation has 50 grounding wires. Traditional manual inspection requires four maintenance personnel and takes a full day to complete comprehensive monitoring, and there is a risk of missed inspections. With the automated monitoring system proposed in this application, real-time monitoring of all 50 grounding wires can be completed automatically. Monitoring data is transmitted to the backend in real time, and abnormal conditions trigger an early warning within 10 seconds. Only one manager needs to view the status of all grounding wires via an app, improving monitoring efficiency by over 80% and reducing labor costs by 75%. Simultaneously, the system automatically stores historical data that can be used to analyze trends in grounding wire status changes, such as statistically analyzing the tilt angle changes of a specific clamp, to predict potential faults in advance. This achieves a shift from "passive monitoring" to "proactive prevention," improving the safety and economy of power maintenance operations and solving the problems of "high reliance on manual labor and low efficiency" in the background technology.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent monitoring of grounding wire status based on multimodal information fusion, characterized in that, Includes the following steps: Image acquisition: Acquire multimodal monitoring data of the grounding wire, including image data, inertial sensing data, and time-series task data; Data processing: A deep learning object detection algorithm is used to identify key components in the image data. The key components include at least wire clamps, connecting bolts, and hanging points. Preliminary judgment: Based on the identified key components, the spatial relationship characteristics, geometric characteristics and visual characteristics of each component are analyzed, and a preliminary result of the grounding wire status is generated according to the preset judgment rules; Fusion verification: The preliminary judgment results are fused with attitude data and temporal task data for multimodal verification, and the final state judgment result is output. Warning Trigger: If the final status determination result is an abnormal status, the corresponding warning information will be triggered.

2. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 1, characterized in that, The deep learning object detection algorithm is either YOLO or SSD deep learning object detection algorithm.

3. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 1, characterized in that, The clamp attitude data includes the clamp's tilt angle, sway amplitude, and acceleration change rate.

4. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 1, characterized in that, The time-series task data includes task stage data for "connecting the grounding wire" and "removing the grounding wire" and corresponding expected status data.

5. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 4, characterized in that, The preset determination rules include: Correctly connected: The angle between the wire clamp and the connection point is within the preset range, the gap between them is less than or equal to the preset threshold, and the connecting bolts are locked. Missing hook: The wire clamp is suspended and there is no matching hook point in the image; Poor connection: The clamp is in contact with a non-target connection point, or the contact angle or gap between the clamp and the connection point exceeds the preset range. Removed: The wireless clip at the original attachment point has been removed, and it matches the scheduled mission time.

6. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 5, characterized in that, The logic for the multimodal fusion verification is as follows: When the initial determination is "correctly connected", the attitude data must meet the preset threshold and be consistent with the expected state data in the "connecting the ground wire" stage; When initially determined to be "removed", it must be consistent with the expected status data of the "removal of ground wire" stage and the characteristics of the original wireless clip at the connection point; If the initial judgment is "missing" or "poorly attached", further verification is required based on attitude data.

7. The intelligent monitoring method for grounding wire status based on multimodal information fusion according to claim 1, characterized in that, The warning information includes: Level 1 warning: This includes the risk of short circuit due to poor connection, which will trigger an audible and visual alarm, SMS push or telephone notification; Level 2 warning: This includes situations where the number of missed hang-ups exceeds the first preset time threshold, or the number of missed removals exceeds the second preset time threshold, triggering an APP push or SMS reminder; Level 3 warning: This includes warnings when the cable clamp tilt angle approaches the safety threshold or the battery level is lower than the set threshold, which will only be displayed on the app.

Images