Power transmission maintenance safety early warning method based on inference engine and power transmission line maintenance platform

Abstract

The present application relates to a power transmission maintenance safety early warning method based on a reasoning machine, based on a reasoning machine and a job behavior knowledge base, a job environment knowledge base and a safety early warning knowledge base, after the behavior and scene of the maintenance personnel are obtained by using an image recognition method, the job behavior and scene of the maintenance personnel are matched according to rules, the behavior type is inferred; the environment characteristics are matched according to rules, the dangerous type of the job environment is obtained; then the safety type of the behavior is inferred by combining the behavior type and the dangerous type of the environment; the safety type of the behavior of the maintenance personnel is judged, if the judgment result is a dangerous behavior, a warning information is sent out. The present application also discloses a power transmission line maintenance platform. The present application analyzes the dangerous degree of the environment and the behavior action of the maintenance personnel in real time, obtains the feedback information of the work data, has good accuracy, can be widely applied to the maintenance of power transmission lines in various environments, has high reliability, saves time and labor, and saves cost.

Description

Technical Field

[0001] This invention belongs to the field of power transmission and transformation facility maintenance, specifically relating to a power transmission line maintenance platform with hazard warning based on an inference engine and its usage method. Background Technology

[0002] Currently, the demand for electricity in various regions of my country is constantly increasing. Due to natural weather conditions and various human factors, line faults are occurring frequently. A typical piece of equipment for circuit maintenance is the insulated bucket truck. Large trucks carry the insulated bucket truck to the vicinity of the transmission line, where maintenance personnel enter the insulated bucket. The bucket is moved by the bucket's boom to transport the bucket and personnel to the faulty line area to carry out the work. However, the working environment and functions of this maintenance equipment have certain limitations. For example, when the faulty line is located near a narrow road, large trucks cannot enter, which to some extent affects the progress of maintenance work.

[0003] Existing power transmission line maintenance equipment is mostly single-function and cannot adapt to increasingly complex working environments. For example, when using insulated bucket trucks to assist maintenance personnel, the lack of real-time sensing components and monitoring units forces maintenance personnel to rely solely on their experience to assess hazardous conditions. Furthermore, the large size of the insulated bucket trucks makes the entire device cumbersome, resulting in high manufacturing and operating costs. Such devices cannot meet the line maintenance needs of personnel in different environments and pose certain safety hazards. Summary of the Invention

[0004] The technical problem of this invention is that existing power transmission line maintenance devices lack the function of monitoring and early warning of dangerous scenarios or dangerous behaviors that may occur during line maintenance operations.

[0005] The purpose of this invention is to address the aforementioned problems by providing a power transmission maintenance safety early warning method and system based on an inference engine. This method utilizes an inference engine and a knowledge base for work behavior, a knowledge base for the work environment, and a knowledge base for safety early warning. After obtaining the behavior and scenario of maintenance personnel using image recognition methods, it performs rule matching based on the personnel's work behavior and the scenario to infer the behavior type. It then performs rule matching on environmental features to obtain the hazard type of the work environment. Finally, combining the behavior type and the hazard type of the environment, it infers the safety type of the behavior. This allows for timely warnings of dangerous behaviors by maintenance personnel, thereby improving the safety of power transmission maintenance operations.

[0006] The technical solution of this invention is a power transmission maintenance safety early warning method based on an inference engine, the safety early warning method comprising the following steps:

[0007] Step 1: Collect images of maintenance personnel on the maintenance work platform, use the ST-GCN motion recognition network to identify the work behavior of the maintenance personnel, and use the YoLo v4 model to identify the scene in which the maintenance personnel are located.

[0008] Step 2: Based on the work behavior knowledge base, and according to the work behaviors and work scenarios obtained in Step 1, infer the behavior type of the maintenance personnel;

[0009] Step 3: Collect environmental characteristics and electric field strength of the work environment, and infer the hazard type of the work environment based on the work environment knowledge base;

[0010] Step 4: Based on the safety early warning knowledge base, infer the safety type of the maintenance personnel's behavior according to the behavior type of the maintenance personnel and the hazard type of the working environment;

[0011] Step 5: Determine the safety type of the maintenance personnel's behavior obtained in Step 4. If the determination result is dangerous behavior, issue a warning message; if the determination result is abnormal behavior, issue a prompt message.

[0012] Preferably, in step 1, the Alphapose model is first used to detect and identify the posture of the maintenance personnel in the image, and then the ST-GCN motion recognition network is used to determine the motion coordinates of the maintenance personnel based on their posture, thereby determining the limb movements of the personnel.

[0013] Furthermore, the Alphapose model includes a spatial transformation network (STN), a single-person pose estimation network (SPPE), a spatial inverse transformation network (SDTN), and a pose nonmaximum suppressor (PPNMS).

[0014] Furthermore, the ST-GCN action recognition network includes a normalization layer, multiple ST-GCN units, a pooling layer, and a fully connected layer. The ST-GCN units include an attention layer (ATT), a graph convolutional network (GCN), and a temporal convolutional network (TCN).

[0015] Preferably, the inference engine uses the Drools rule engine.

[0016] The power transmission line maintenance platform employing a safety early warning method includes a work bucket, a base, a lifting mechanism, and an embedded system. The embedded system's memory stores a computer program, which, when executed by the embedded system's processor, implements the aforementioned safety early warning method. The bottom of the work bucket is connected to the base via the lifting mechanism, which includes a first telescopic frame and a second telescopic frame driven by a hydraulic telescopic column. The tops of the first and second telescopic frames are respectively connected to the front and rear edges of the bottom of the work bucket. The base has multiple support legs, with support plates connected to the bottom of the legs. The base has a first wheel set and a second wheel set. The connecting shaft of the wheel center of the first wheel set is rotatably connected to the base via a steering column; the axle of the second wheel set is fixedly connected to the base.

[0017] One upper end of the X-shaped rod at the top of the first telescopic frame is hinged to the bottom of the working bucket, and the other upper end is connected to the slider of the slider mechanism at the bottom of the working bucket. The connection between the second telescopic frame and the working bucket is the same as that of the first telescopic frame, and they are symmetrical. A hydraulic telescopic column is provided on the diagonal direction of two adjacent X-shaped rods in the first telescopic frame. The structure of the second telescopic frame is the same as that of the first telescopic frame, and they are symmetrical. Corresponding to the slider mechanism at the bottom of the working bucket, a slider mechanism is also provided on the base. The ends of the slider mechanisms of the bottom X-shaped rods of the first and second telescopic frames near the base are connected to the sliders of the slider mechanism. When the lifting mechanism is lifted upward under the action of the hydraulic telescopic column, the slider of the slider mechanism slides adaptively with the movement of the end of the lifting mechanism connected to it.

[0018] The work bucket is equipped with a sensor unit and a camera, and the output terminals of the sensor unit and the camera are electrically connected to the embedded system respectively; the work bucket is equipped with a first operating table electrically connected to the embedded system; the base is equipped with a second operating table electrically connected to the embedded system.

[0019] Preferably, the sensor unit includes an electric field strength sensor and a wind speed sensor.

[0020] Preferably, the hinge joints of the X-shaped rods on both sides of the first telescopic frame are connected to the corresponding rod hinge joints on the second telescopic frame via crossbars.

[0021] Preferably, the base is provided with a hydraulic rod, and the rod head of the hydraulic rod is connected to the slider of the slider mechanism of the base.

[0022] Furthermore, a hydraulic pump is installed on the base, and the hydraulic telescopic column and hydraulic rod are connected to the hydraulic pump via oil pipelines; the pump shaft of the hydraulic pump is connected to the rotating shaft of the motor, and the control terminal of the motor is connected to the control signal output terminal of the embedded system.

[0023] Compared with the prior art, the beneficial effects of the present invention include:

[0024] 1) The power transmission maintenance safety early warning method of the present invention classifies behaviors by matching the features of collected images and environmental information with the knowledge base established. By using three different types of knowledge bases, the accuracy of behavior recognition is improved.

[0025] 2) The method of the present invention analyzes the environment and the degree of danger of the maintenance personnel's work in real time, obtains feedback information of work data, has good accuracy, can be widely used in the maintenance of transmission lines in various environments, has high reliability, saves time and effort, and saves costs.

[0026] 3) The transmission line maintenance platform of the present invention collects the physical state of the working environment, such as temperature, humidity, and electric field strength, in real time, and also collects the working status of maintenance personnel in real time, and issues timely warnings for dangerous behaviors of maintenance personnel, so as to better protect the personal safety of maintenance personnel.

[0027] 4) The power transmission line maintenance platform of the present invention can adapt to various power transmission line maintenance environments and has low manufacturing cost. Attached Figure Description

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0029] Figure 1 This is a schematic diagram of real-time human pose estimation according to an embodiment of the present invention.

[0030] Figure 2 This is a flowchart of action recognition based on ST-CGN according to an embodiment of the present invention.

[0031] Figure 3 This is a structural diagram of the ST-CGN model according to an embodiment of the present invention.

[0032] Figure 4 This is a schematic diagram illustrating the working principle of the rule engine Drools in an embodiment of the present invention.

[0033] Figure 5 This is a flowchart of the safety early warning method according to an embodiment of the present invention.

[0034] Figure 6 This is a schematic diagram of the neural network model of the embedded system according to an embodiment of the present invention.

[0035] Figure 7 This is a schematic diagram illustrating the principle of security early warning implemented in an embedded system according to an embodiment of the present invention.

[0036] Figure 8 This is a front view of the power transmission line maintenance platform according to an embodiment of the present invention.

[0037] Figure 9 This is a left view of the power transmission line maintenance platform according to an embodiment of the present invention.

[0038] Figure 10 This is a partially enlarged left view of the power transmission line maintenance platform according to an embodiment of the present invention.

[0039] Figure 11 This is an overall structural view of the power transmission line maintenance platform according to an embodiment of the present invention.

[0040] Figure 12 This is a schematic diagram of the rules of the knowledge base in an embodiment of the present invention.

[0041] Explanation of reference numerals in the attached drawings: Embedded system 1, base 2, support leg 201, support plate 202, first operating table 3, lifting mechanism 4, first telescopic frame 401, second telescopic frame 402, X-shaped rod 403, working bucket 5, second operating table 6, slider mechanism 7, hydraulic pump 8, electric motor 9, hydraulic telescopic column 10, hydraulic rod 11, sensor unit 12, camera 13, traction rod 14, crossbar 15, first wheel set 16, second wheel set 17, steering column 18. Detailed Implementation

[0042] like Figure 8-11 As shown, the transmission line maintenance platform of this embodiment includes a work bucket 5, a base 2, a lifting mechanism 4, and an embedded system 1. The memory of the embedded system 1 stores a computer program. When the computer program is executed by the processor of the embedded system, it implements the transmission line maintenance safety early warning method provided in this application. The bottom of the work bucket 5 is connected to the base 2 via the lifting mechanism 4. The lifting mechanism 4 includes a first telescopic frame 401 and a second telescopic frame 402 driven by hydraulic telescopic columns. The tops of the first telescopic frame and the second telescopic frame are respectively connected to the front and rear edges of the bottom of the work bucket 5. One upper end of the first X-shaped rod 403 at the top of the first telescopic frame is hinged to the bottom of the work bucket 5, and the other upper end is connected to the slider of the slider mechanism 7 at the bottom of the work bucket 5. The connection method between the second telescopic frame and the work bucket is the same as that of the first telescopic frame, and they are symmetrical. A hydraulic telescopic column 10 is provided in the diagonal direction of two adjacent X-shaped rods 403 in the first telescopic frame. The structure of the second telescopic frame is the same as that of the first telescopic frame. The hinge points of the X-shaped rods on both sides of the first telescopic frame 401 are respectively connected to the corresponding rod hinge points on the second telescopic frame via crossbars 15. The base 2 is provided with multiple support legs 201, and the bottom of the support legs 201 is connected to the support plate 202; the base 2 is provided with a first wheel group 16 and a second wheel group 17, the connecting shaft of the wheel center of the first wheel group 16 is rotatably connected to the base via the steering column 18; the axle of the second wheel group 17 is fixedly connected to the base.

[0043] Corresponding to the slider mechanism 7 at the bottom of the working bucket 5, a slider mechanism 7 is also provided on the base. The ends of the slider mechanisms 7 near the base of the X-shaped rods at the bottom of the first and second telescopic frames are connected to the sliders of the slider mechanisms 7. When the lifting mechanism is lifted upward under the action of the hydraulic telescopic column, the slider of the slider mechanism 7 slides adaptively with the movement of the end of the lifting mechanism connected to it. A hydraulic rod 11 is provided on the base 2, and the rod head of the hydraulic rod 11 is connected to the slider of the slider mechanism 7 on the base. A hydraulic pump 8 is provided on the base 2, and the hydraulic telescopic column 10 and the hydraulic rod 11 are respectively connected to the hydraulic pump 8 through oil pipes; the pump shaft of the hydraulic pump 8 is connected to the rotating shaft of the motor 9, and the control terminal of the motor 9 is connected to the control signal output terminal of the embedded system.

[0044] The work hopper 5 is equipped with a sensor unit 12 and a camera 13. The output terminals of the sensor unit 12 and the camera 13 are electrically connected to the embedded system 1, respectively. The sensor unit 12 includes an electric field strength sensor and a wind speed sensor, and can be equipped with corresponding sensors as needed. The work hopper 5 contains a first operating platform 3 electrically connected to the embedded system 1. In this embodiment, the sensor unit 12 and the camera 13 are respectively mounted on a frame above the insulated work hopper 5, ensuring their monitoring height is consistent with the head and shoulders of the maintenance personnel. This allows for accurate monitoring of the physical data of the working environment, while the camera can comprehensively monitor the actions and behaviors of the maintenance personnel. In this embodiment, a 620A electromagnetic radiation detector is used as the electric field strength sensor, and a 485-type wind speed sensor is used.

[0045] The first control panel 3 is equipped with "Up", "Down" and "Stop" buttons to control the hydraulic telescopic column of the lifting mechanism. When the hydraulic telescopic column extends, the first telescopic frame and the second telescopic frame unfold accordingly, and the working bucket rises; when the hydraulic telescopic column retracts, the first telescopic frame and the second telescopic frame fold accordingly, and the working bucket descends.

[0046] like Figure 10 As shown, the base 2 is provided with a second operating console 6 that is electrically connected to the embedded system 1. The function of the second operating console 6 is the same as that of the first operating console 3.

[0047] In this embodiment, the working bucket 5 is an insulated working bucket with dimensions of 3.3m × 2.1m × 3.5m, and the maximum working height of the power transmission line maintenance platform is 15 meters. In this embodiment, the processor of the embedded system 1 is an NVIDIA Jetson Nano. The embedded system includes a knowledge base, an inference engine, an image processing module, and an alarm module.

[0048] When maintenance personnel perform maintenance work on a transmission line maintenance platform, image recognition methods are used to obtain their behavior and the surrounding environment. Based on the personnel's work behavior and the environment, rule matching is performed to infer the behavior type, such as... Figure 4 As shown; rule matching is performed on environmental characteristics to obtain the hazard type of the work environment; then, by combining the behavior type and the environmental hazard type, the safety type of the behavior is inferred. The inference engine uses the Drools rule engine, and the rules in the knowledge base are as follows: Figure 12 As shown.

[0049] The process by which an embedded system monitors and issues early warnings in real time based on images of maintenance personnel captured by a camera is as follows: Figure 7 As shown.

[0050] like Figure 5 As shown, the power transmission maintenance safety early warning method based on inference engines includes the following steps:

[0051] Step 1: Based on the images of maintenance personnel captured by the camera, the ST-GCN model is used to identify the work behavior of the maintenance personnel; and the YoLo v4 model is used to identify the scene in which the maintenance personnel are located.

[0052] First, the Alphapose model is used to detect and identify the poses of personnel in the images of maintenance workers, such as... Figure 1 As shown; then, the ST-GCN motion recognition network is used to determine the motion coordinates of the maintenance personnel based on their posture, thus determining the personnel's limb movements, such as... Figure 2 As shown.

[0053] The ST-GCN action recognition network consists of a normalization layer, multiple ST-GCN units, pooling layers, and fully connected layers. Each ST-GCN unit includes an attention layer (ATT), a graph convolutional network (GCN), and a temporal convolutional network (TCN). Figure 3 As shown.

[0054] Step 2: Based on the work behavior knowledge base, and according to the work behaviors and work scenarios obtained in Step 1, infer the behavior type of the maintenance personnel;

[0055] Step 3: Use electric field strength sensors and wind speed sensors to collect environmental characteristics, and infer the hazard type of the work environment based on the work environment knowledge base.

[0056] Step 4: Based on the safety early warning knowledge base, infer the safety type of the maintenance personnel's behavior according to the behavior type of the maintenance personnel and the hazard type of the working environment.

[0057] Step 5: Determine the safety type of the maintenance personnel's behavior obtained in Step 4. If the maintenance personnel's behavior is risky, issue a prompt or warning message. If the maintenance personnel's behavior is not risky, proceed with the work normally.

[0058] The image processing module of the embedded system employs neural network models, including the Alphapose model, ST-GCN model, and YoLo v4 model. The training and usage principles of neural network models are as follows: Figure 6 As shown.

[0059] The basis for safety warning methods comes from two sources: first, environmental changes, such as the impact of electric field strength exceeding safe working levels, excessively high wind speeds, and extreme weather conditions like rain and snow on maintenance work; and second, safety hazards arising from the constantly changing behavior of maintenance personnel, such as maintenance personnel excessively extending their bodies out of the work area or their limbs potentially coming into direct contact with live lines.

[0060] The slider mechanism 7 of the embodiment refers to the slider mechanism disclosed in the paper "Design of Passenger-Carrying Scissor Lift Platform for Amusement Industry" by Xie Yinguo, Meng Jie, Ye Shuai et al., published in the first issue of Metallurgical Equipment in 2021.

[0061] The YoLo v4 model of the embodiment is the YoLo v4 object detection network disclosed in the paper "Human-Computer Collaborative Assembly Scene Cognition Method Based on Multi-Scale Object Detection" by Dong Yuanfa, Yan Huabing et al., published in the second issue of Computer Integrated Networks in 2022.

Claims

1. A power transmission maintenance safety early warning method based on inference engines, characterized in that, The safety early warning method is based on an inference engine and a knowledge base of work behavior, a knowledge base of work environment, and a knowledge base of safety early warning. After obtaining the behavior and scenario of maintenance personnel using image recognition methods, it performs rule matching based on the work behavior and scenario of maintenance personnel to infer the behavior type. By performing rule matching on environmental characteristics, the hazard type of the work environment is obtained; then, by combining the behavior type and the hazard type of the environment, the safe type of the behavior is deduced; by using three different types of knowledge bases, the accuracy of identifying hazardous behaviors of maintenance personnel is improved. The security early warning method includes the following steps: Step 1: Collect images of maintenance personnel on the maintenance work platform, extract the skeleton of the maintenance personnel from the images using the Alphapose model, then use the ST-GCN motion recognition network to determine the motion coordinates of the maintenance personnel based on the skeleton, and determine the limb movements of the personnel; and use the YoLo v4 model to identify the scene in which the maintenance personnel are located. The Alphapose model includes a spatial transformation network (STN), a single-person pose estimation network (SPPE), a spatial inverse transformation network (SDTN), and a pose nonmaximum suppressor (PPNMS). The ST-GCN action recognition network includes a normalization layer, multiple ST-GCN units, a pooling layer, and a fully connected layer. The ST-GCN unit includes an attention layer (ATT), a graph convolutional network (GCN), and a temporal convolutional network (TCN). Step 2: Based on the work behavior knowledge base, and according to the work behaviors and work scenarios obtained in Step 1, infer the behavior type of the maintenance personnel; Step 3: Collect environmental characteristics and electric field strength of the work environment, and infer the hazard type of the work environment based on the work environment knowledge base; Step 4: Based on the safety early warning knowledge base, infer the safety type of the maintenance personnel's behavior according to the behavior type of the maintenance personnel and the hazard type of the working environment; Step 5: Determine the safety type of the maintenance personnel's behavior obtained in Step 4. If the determination result is dangerous behavior, issue a warning message; if the determination result is abnormal behavior, issue a prompt message.

2. The power transmission maintenance safety early warning method according to claim 1, characterized in that, The inference engine uses the Drools rule engine.

3. The transmission line maintenance platform of the transmission line maintenance safety early warning method as described in claim 1 or 2, characterized in that, The power transmission line maintenance platform includes a work bucket (5), a base (2), a lifting mechanism (4), and an embedded system (1). The memory of the embedded system (1) stores a computer program. When the computer program is executed by the processor of the embedded system, it implements the safety warning method of claim 1. The bottom of the work bucket (5) is connected to the base (2) via the lifting mechanism (4). The lifting mechanism (4) includes a first telescopic frame (401) and a second telescopic frame (402) driven by a hydraulic telescopic column. The tops of the first telescopic frame and the second telescopic frame are respectively connected to the front and rear edges of the bottom of the work bucket (5). One upper end of the X-shaped rod (403) at the top of the first telescopic frame is hinged to the bottom of the working bucket (5), and the other upper end is connected to the slider of the slider mechanism (7) at the bottom of the working bucket (5). The connection between the second telescopic frame and the working bucket is the same as that of the first telescopic frame. In the first telescopic frame, hydraulic telescopic columns (10) are provided on the diagonal direction of two adjacent X-shaped rods (403), and the structure of the second telescopic frame is the same as that of the first telescopic frame; Corresponding to the slider mechanism (7) at the bottom of the working bucket (5), a slider mechanism (7) is also provided on the base. The ends of the slider mechanism (7) near the base of the X-shaped rod at the bottom of the first telescopic frame and the second telescopic frame are connected to the slider of the slider mechanism (7). When the lifting mechanism is lifted upward under the action of the hydraulic telescopic column, the slider of the slider mechanism (7) slides left and right adaptively with the movement of the end of the lifting mechanism connected to it. The work hopper (5) is equipped with a sensor unit (12) and a camera (13), and the output terminals of the sensor unit (12) and the camera (13) are electrically connected to the embedded system respectively; The work hopper (5) is equipped with a first operating table (3) that is electrically connected to the embedded system.

4. The power transmission line maintenance platform according to claim 3, characterized in that, The base (2) is provided with multiple legs (201), and the bottom of the legs (201) is connected to the support plate (202).

5. The power transmission line maintenance platform according to claim 4, characterized in that, The hinged ends of the X-shaped rods on both sides of the first telescopic frame (401) are connected to the corresponding rod hinged ends on the second telescopic frame via crossbars (15).

6. The power transmission line maintenance platform according to claim 5, characterized in that, The base (2) is provided with a first wheel group (16) and a second wheel group (17). The connecting shaft of the first wheel group (16) is rotatably connected to the base via a steering column (18); the shaft of the second wheel group (17) is fixedly connected to the base.

7. The power transmission line maintenance platform according to claim 6, characterized in that, A hydraulic rod (11) is provided on the base (2), and the rod head of the hydraulic rod (11) is connected to the slider of the slider mechanism (7) of the base.

8. The power transmission line maintenance platform according to claim 7, characterized in that, A hydraulic pump (8) is provided on the base (2). The hydraulic telescopic column (10) and hydraulic rod (11) are connected to the hydraulic pump (8) via oil pipelines. The pump shaft of the hydraulic pump (8) is connected to the rotating shaft of the motor (9). The control terminal of the motor (9) is connected to the control signal output terminal of the embedded system.

Images