LiDAR-based hot-line work equipotential monitoring method

Abstract

The invention relates to the field of hot-line work monitoring, in particular to a LiDAR-based hot-line work equipotential monitoring method, which comprises the following steps: S1, positioning and tracking a hot-line worker through an infrared sensor, and scanning the hot-line worker through binocular laser radars which are mutually redundant after determining the position of the hot-line worker; s2, human body contour extraction, key node extraction and posture parameter calculation are carried out, a posture model is established, posture recognition and classification are carried out, and point cloud data of a live working site are obtained; s3, according to the position change of the live working personnel and the live equipment, analyzing and evaluating the posture of the human body to obtain an identification result, and performing safety specification judgment on the safety distance and the action specification; and S4, when the distance is smaller than the safe distance or the action is detected to be nonstandard, an alarm signal is sent out, and feedback and guidance are given in time. The posture recognition and monitoring can be carried out on the live working personnel under the condition that obstacles exist, and the working safety is guaranteed.

Description

Technical Field

[0001] The invention relates to the field of live working monitoring, and in particular to a LiDAR-based live working equipotential monitoring method. Background Art

[0002] During live working, workers need to enter an equipotential environment to operate. Due to the existence of the electric field and the live state of the equipment, workers face the risk of direct electric shock. Through equipotential monitoring, the distance between the worker and the live object, the worker's operating actions, etc. can be monitored in real time, and dangerous situations that may cause electric shock can be discovered in time, and corresponding measures can be taken to prevent and correct them, avoiding the occurrence of electric shock accidents and protecting the lives of workers.

[0003] In the prior art, the patent publication number is CN112785455B, and the patent name is "A system and method for monitoring and warning live wires based on multiple laser radars". It discloses a system including several laser radars installed on different sides of a boom truck, and the several laser radars are respectively connected to several data processing platforms; the several laser radars do not have line of sight with each other, and each laser radar senses information in one direction. This patent installs multiple laser radars around the power operation boom truck. The number of laser radars is large, which directly increases the application cost, and can only be applied to power operation boom trucks. The application environment is limited, and there are still many inconveniences in actual application. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a LiDAR-based live working equipotential monitoring method, which can perform body posture recognition and monitoring of live working personnel in the presence of obstacles, thereby ensuring the working safety of live working personnel.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A LiDAR-based live working equipotential monitoring method, the method comprising:

[0007] S1: Use infrared sensors to locate and track live-line workers. After determining their positions, use redundant binocular laser radars to scan them and collect point cloud data in different postures and movements.

[0008] S2: Extract human body contours, extract key nodes, and calculate posture parameters, establish a body model, perform body recognition and classification, and obtain point cloud data of live working sites, including the location information of operators and live equipment;

[0009] S3: According to the position changes of live workers and live equipment, as well as the analysis and evaluation of human body posture, the recognition results are obtained, and the safety distance and action specifications are judged;

[0010] S4: When the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and timely feedback and guidance are given.

[0011] Further, in step S1, after determining the position of the live-line worker, scanning the live-line worker by using mutually redundant binocular laser radars to collect point cloud data in different postures and actions includes the following steps:

[0012] S11: Start two laser radars at the same time to perform synchronous scanning on the live-line workers, obtain point cloud data from different perspectives of the live-line workers in the same time period, and record the emission time, return time and scanning angle of each laser pulse during the scanning process;

[0013] S12: Remove noise points through median filtering, convert the point cloud data collected by the two laser radars into the same coordinate system for coordinate conversion, and extract feature points from the point cloud data of the two laser radars using the SIFT feature extraction algorithm;

[0014] S13: Using the extracted feature points, the point cloud data of the two lidars are matched and aligned through the NICP registration algorithm, and then the registered point cloud data are fused through the distance-based fusion method to obtain a complete point cloud model of the live working personnel;

[0015] S14: Use the Poisson reconstruction algorithm to reconstruct the fused point cloud data into a three-dimensional model of the live working personnel.

[0016] Further, in step S13, the extracted feature points are used to match and align the point cloud data of the two laser radars through the NICP registration algorithm, and then the registered point cloud data are fused through the distance-based fusion method to obtain a complete point cloud model of the live working personnel, including the following steps:

[0017] S131: The initial transformation matrix is ​​obtained based on the feature-based preliminary registration method, and the nearest point in the target point cloud is found in the point cloud after the initial transformation through the Euclidean distance between the points.

[0018] S132: The objective function based on NICP involves the position information and normal vector information of the point. For each corresponding point The objective function is defined as:

[0019]

[0020] in, is the transformation matrix to be solved, which contains the rotation R and translation t;

[0021] and They are point p i and The unit normal vector of the y-axis. α is the weight parameter used to balance the position error and the normal vector error. w i is the weight of the corresponding point pair;

[0022] S133: Minimize the objective function E(T) by the Levenberg-Marquardt algorithm to perform iterative optimization. For each iteration, firstly, calculate the partial derivatives of the objective function E(T) with respect to the elements of the transformation matrix T, and then update the transformation matrix T according to the derivative results using the update rule of the Levenberg-Marquardt algorithm. Repeat the above steps until the value of the objective function E(T) converges to a stable minimum value or reaches a preset number of iterations to complete the registration.

[0023] S134: Using the distance-based fusion method, input the two sets of point cloud data P′ and Q′ after NICP registration, for each point p′ in the point cloud P′ i Calculate the distance to all points in point Q'. For each point q' in point cloud Q' j Calculate the distance to all points in the point cloud P′ and set a distance threshold;

[0024] S135: When point p′ i If the minimum distance to the point cloud Q′ is less than the distance threshold, p′ i Add to the fused point cloud set; when point q′ j If the distance to the nearest point in the point cloud P′ is less than the distance threshold, then q′ j Add to the fused point cloud collection;

[0025] S136: De-duplication processing is performed on the fused point cloud set through a hash table.

[0026] Furthermore, in step S14, the fused point cloud data is reconstructed into a three-dimensional model of the live working personnel using a Poisson reconstruction algorithm, comprising the following steps:

[0027] S141: defining a voxel grid in a three-dimensional space to surround the space where the point cloud data is located, constructing an indicator function for each voxel in the voxel grid according to whether there is a point cloud data point inside, and calculating the gradient field of the indicator function using a central difference method;

[0028] S142: constructing a Poisson equation according to the gradient field, solving the Poisson equation using a multigrid method, setting an isosurface value, extracting the isosurface in the space of the solved function, expressing the isosurface in the form of a triangular mesh using a Marching Cubes algorithm, and reconstructing the surface of the three-dimensional model using the triangular mesh;

[0029] S143: Smoothing the surface of the three-dimensional model, checking for errors in the model and repairing them.

[0030] Furthermore, in step S2, the human body contour extraction, key node extraction and posture parameter calculation, the establishment of a body model and the body posture recognition and classification include the following steps:

[0031] S21: The density clustering algorithm is used to separate the human body from the background and other objects in the point cloud based on the spatial range. The Marching Cubes algorithm extracts the human body contour from the segmented human body point cloud area and smoothes the extracted contour.

[0032] S22: The trained convolutional neural network predicts the key node positions in the point cloud, which correspond to the joints of the human body, and the predicted key nodes are screened and optimized;

[0033] S23: Establish a local coordinate system with a fixed part of the human body as the origin, calculate the angles between adjacent joints according to the positions of key nodes in the local coordinate system, calculate the limb length ratio, and determine the body inclination angle by calculating the angle between the center of gravity of the human body and the vertical direction;

[0034] S24: The body model is obtained by training the model through support vector machine and a large amount of collected point cloud data of different human bodies in various postures, and the characteristics and patterns of different postures are learned;

[0035] S25: The newly collected point cloud data of the human body to be identified is input into the trained body model, the features of the input point cloud data are extracted and matched and compared with the features of different postures learned during the training process, and the human body posture category is output according to the matching result model to complete the identification and classification.

[0036] Furthermore, the acquisition of point cloud data of the live working site, including the location information of the workers and the live equipment, comprises the following steps:

[0037] A1: Before live-line work begins, the work area is scanned by LiDAR, the acquisition time of each point cloud data is recorded, and it is transmitted to the data processing equipment via a wireless network;

[0038] A2: Pre-process the transmitted original point cloud data, use the point cloud segmentation algorithm to segment the live workers, live equipment and other background objects from the point cloud data based on the spatial distribution of the point cloud, and perform feature extraction and recognition on the segmented point cloud data of the workers and live equipment;

[0039] A3: Calculate the position coordinates of live workers and live equipment in the established three-dimensional coordinate system based on the processed point cloud data, and determine the real-time position changes through point cloud registration and target tracking algorithms.

[0040] Furthermore, in step S3, the following steps are included: obtaining identification results based on the position changes of the live working personnel and the live equipment, and analyzing and evaluating the body posture of the human body, and making safety standard judgments on the safety distance and action standards:

[0041] S31: setting a safety distance threshold for live working personnel and live equipment, calculating the real-time distance based on the position coordinates of the live working personnel and live equipment, and comparing the distance with the safety distance threshold;

[0042] S32: Establish an action specification model based on the safe operation procedures and standards for live working, and match and compare the action posture and action process of the operator extracted from the point cloud data with the action specification model;

[0043] S33: Record and store the collected point cloud data, position and posture information, and the judgment result data of the safety distance and action specification.

[0044] Further, in step S32, the action specification model is established based on the safe operation procedures and standards for live working, and the action posture and action process of the operator extracted from the point cloud data are matched and compared with the action specification model, including the following steps:

[0045] S321: The safety operation procedures and standards for live working are decomposed into a series of action units, and clear action characteristics are defined for each basic action unit. The allowable action range of each basic action unit is determined in combination with ergonomic principles, and the speed and force limits of the action are set;

[0046] S322: Determine the operation sequence and process of different operation tasks according to the actual live operation process, model the sorted operation sequence and process through a directed graph, and verify the established operation process model using model verification technology;

[0047] S323: Use laser scanning to build a three-dimensional model of the site, convert environmental factors into constraints on action specifications, and use a bounding box-based collision detection algorithm to perform collision detection between the operator's actions and obstacles in the environment;

[0048] S324: Use virtual reality technology to simulate the live working environment and conduct simulated operation to verify the model. Select some actual work tasks for real-time comparison and verification at the live working site, and optimize and improve the model based on the verification results.

[0049] Further, in step S4, when the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and feedback and guidance are given in time, including the following steps:

[0050] S41: When the safety distance threshold is exceeded, the alarm condition is triggered. When the actual action of the live working personnel is detected to be different from the standard action defined in the model, the alarm condition is triggered.

[0051] S42: Once the alarm condition is triggered, an alarm signal is generated, an alarm sound with a specific frequency, duration and intensity is generated using an audio synthesis algorithm, and a warning effect is achieved by controlling the flashing frequency and color of the light emitting diode;

[0052] S43: When a safety distance violation occurs: generate feedback and output guidance information based on the location information of the operator and the live equipment; when an irregular action is detected: generate feedback and output guidance suggestion information based on the action specification model and combined with the actual action data comparison results of the live operator.

[0053] Furthermore, the binocular laser radar is composed of two laser radars, which are respectively arranged on a rotating assembly, the rotating assembly is fixed at both ends of the first telescopic support rod, the middle part of the first telescopic support rod is fixedly connected to one end of the second telescopic support rod, the other end of the second telescopic support rod is connected to the rotating assembly, the infrared sensor is arranged on the second telescopic support rod, and the rotating assembly is a device that can drive the equipment and control the angle for rotation.

[0054] The beneficial effects of the present invention are:

[0055] 1. The present invention arranges two mutually redundant laser radars, which can penetrate obstructions from different directions and reduce the impact of obstructions on monitoring.

[0056] 2. When two radars work simultaneously, the present invention can reduce the influence of interference on the measurement results and improve the adaptability of the system in complex environments by comprehensively analyzing and processing multiple sets of data.

[0057] 3. For different types of live working tasks and working environments, such as outdoor overhead lines, equipment in substations or cable tunnels, the present invention can flexibly adjust the installation positions and angles of the two laser radars to adapt to various complex working scene requirements.

[0058] 4. Two redundant LiDARs can realize real-time, all-round monitoring of live workers and live equipment, and promptly detect safety hazards such as abnormal distance between live workers and live equipment, irregular movements of live workers, etc., and quickly issue alarm signals to remind live workers and supervisors to take measures and give corresponding guidance, effectively preventing the occurrence of electric shock accidents and ensuring the life safety of live workers.

[0059] 5. Based on the real-time data obtained by the LiDAR, digital modeling and analysis of the operation process are carried out to identify bottlenecks and problems in the operation process, optimize the operation steps and operation sequence, and improve operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of a method for monitoring equipotential in live working based on LiDAR;

[0061] Figure 2 It is a schematic diagram of the structure of binocular laser radar;

[0062] Figure 3 It is a flowchart of the steps for obtaining point cloud data of live working sites.

[0063] Among them, 1: rotating component; 2: first telescopic support rod; 3: second telescopic support rod; 4: laser radar. DETAILED DESCRIPTION

[0064] See also Figure 1 , Figure 2 and Figure 3 As shown, the present invention relates to a LiDAR-based live working equipotential monitoring method, the method comprising:

[0065] S1: Use infrared sensors to locate and track live-line workers. After determining their positions, use redundant binocular laser radars to scan them and collect point cloud data in different postures and movements.

[0066] S2: Extract human body contours, extract key nodes, and calculate posture parameters, establish a body model, perform body recognition and classification, and obtain point cloud data of live working sites, including the location information of operators and live equipment;

[0067] S3: According to the position changes of live workers and live equipment, as well as the analysis and evaluation of human body posture, the recognition results are obtained, and the safety distance and action specifications are judged;

[0068] S4: When the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and timely feedback and guidance are given.

[0069] Specifically, in step S1, the infrared sensor is used to locate and track the live-line workers. The infrared sensor works based on the infrared radiation characteristics of the object. The pyroelectric infrared sensor detects the changes in infrared emitted by the object to track the movement of the object. The role of the infrared sensor is to detect, locate and track the live-line workers through background subtraction algorithm, maximum likelihood estimation method and mean shift algorithm. When the infrared sensor determines the target object, if the temperature of the target object is different from the surrounding environment, the infrared sensor can detect this temperature change. If the detected infrared radiation intensity and distribution are different from the preset background or environmental radiation pattern, and this difference exceeds a certain threshold, it can be determined that the target object exists.

[0070] Further, in step S1, after determining the position of the live-line worker, scanning the live-line worker by using mutually redundant binocular laser radars to collect point cloud data in different postures and actions includes the following steps:

[0071] S11: simultaneously start two laser radars 4 to perform synchronous scanning on the live-line workers, obtain point cloud data of the live-line workers from different perspectives in the same time period, and record the emission time, return time and scanning angle of each laser pulse during the scanning process;

[0072] S12: removing noise points by median filtering, converting the point cloud data collected by the two laser radars 4 into the same coordinate system for coordinate conversion, and extracting feature points from the point cloud data of the two laser radars 4 by using a SIFT feature extraction algorithm;

[0073] S13: using the extracted feature points, matching and aligning the point cloud data of the two laser radars 4 through the NICP registration algorithm, and then fusing the registered point cloud data through a distance-based fusion method to obtain a complete point cloud model of the live-line worker;

[0074] S14: Use the Poisson reconstruction algorithm to reconstruct the fused point cloud data into a three-dimensional model of the live working personnel.

[0075] Specifically, the mutually redundant binocular laser radars are set at different positions and angles to ensure that they can scan the target object from different directions, and at the same time ensure that their measurement ranges have a certain overlap area to cover the obscured parts, and the two laser radars 4 are calibrated to ensure that their measurement data can be accurately matched and fused. Parameter calibration involves internal parameter calibration and external parameter calibration, where: internal parameters are calibrated, including laser emission power, pulse width, scanning frequency, angular resolution, etc. The calibration of these parameters is performed through the calibration program or tool provided by the laser radar 4, and the external parameter calibration is achieved through a method based on feature point matching to determine the relative position and posture relationship between the two laser radars 4.

[0076] In step S11, two laser radars are connected using a hardware synchronization line or a synchronization signal generator to ensure that they can start and stop scanning at the same time. The emission time, return time, scanning angle and other information of each laser pulse are recorded to facilitate the subsequent calculation of the distance and position of the object.

[0077] In step S12, due to measurement errors, environmental interference and other factors, there may be noise points in the collected point cloud data. The Gaussian filter algorithm is used to remove the noise points to improve the data quality. The point cloud data collected by the two laser radars are converted to the same coordinate system for data fusion and processing. For coordinate transformation, the rotation matrix R and translation vector T from the source coordinate system to the target coordinate system are first determined. The rotation matrix R can be expressed in the form of Euler angles, and the translation vector T is a three-dimensional vector that represents the displacement between the origins of the two coordinate systems. For point P in the source coordinate system s (x s ,y s , z s ) is converted to point P in the target coordinate system t (x t ,y t , z t ), through the following formula:

[0078] Feature points, such as the edge, corner, and plane features of an object, are extracted through the SIFT feature extraction algorithm. The original point cloud image is subjected to Gaussian blurring of different scales to construct a multi-scale space. In the space of different scales, the size of each point is compared with its adjacent points, and the extreme points are found as feature points. The gradient direction histogram is calculated in the neighborhood of the feature point. The peak value of the histogram is the main direction of the feature point. With the feature point as the center, a neighborhood of a certain size is taken, and the gradient direction and size of the points in the neighborhood are statistically calculated to generate a feature descriptor with rotation invariance, thus completing the extraction of feature points.

[0079] Further, in step S13, the extracted feature points are used to match and align the point cloud data of the two laser radars 4 through the NICP registration algorithm, and then the registered point cloud data are fused through the distance-based fusion method to obtain a complete point cloud model of the live working personnel, including the following steps:

[0080] S131: The initial transformation matrix is ​​obtained based on the feature-based preliminary registration method, and the nearest point in the target point cloud is found in the point cloud after the initial transformation through the Euclidean distance between the points.

[0081] S132: The objective function based on NICP involves the position information and normal vector information of the point. For each corresponding point The objective function is defined as:

[0082]

[0083] in, is the transformation matrix to be solved, which contains the rotation R and translation t;

[0084] and They are point p i and The unit normal vector of the y-axis. α is the weight parameter used to balance the position error and the normal vector error. w i is the weight of the corresponding point pair;

[0085] S133: Minimize the objective function E(T) by the Levenberg-Marquardt algorithm to perform iterative optimization. For each iteration, firstly, calculate the partial derivatives of the objective function E(T) with respect to the elements of the transformation matrix T, and then update the transformation matrix T according to the derivative results using the update rule of the Levenberg-Marquardt algorithm. Repeat the above steps until the value of the objective function E(T) converges to a stable minimum value or reaches a preset number of iterations to complete the registration.

[0086] S134: Using the distance-based fusion method, input the two sets of point cloud data P′ and Q′ after NICP registration, for each point p′ in the point cloud P′ i Calculate the distance to all points in point Q'. For each point q' in point cloud Q' j Calculate the distance to all points in the point cloud P′ and set a distance threshold;

[0087] S135: When point p′ i If the minimum distance to the point cloud Q′ is less than the distance threshold, p′ i Add to the fused point cloud set; when point q′j If the distance to the nearest point in the point cloud P′ is less than the distance threshold, then q′ j Add to the fused point cloud collection;

[0088] S136: De-duplication processing is performed on the fused point cloud set through a hash table.

[0089] Specifically, in step S131, two sets of point cloud data P obtained by scanning two laser radars 4 are input. 1 , p 2 , ..., p m} and Q = {q 1 ,q 2 , ..., q n} and the pre-extracted feature points, the initial transformation matrix T is obtained through the feature-based preliminary registration method 0 , for each point p in the point cloud P i After the initial transformation, the point cloud T 0 (P) Find the closest point in the point cloud Q Form corresponding point pairs

[0090] In step S133, a nonlinear optimization method is used for iterative optimization, and the objective function E(T) is minimized by the Levenberg-Marquardt algorithm. For each iteration, the partial derivative of the objective function E(T) with respect to the elements of the transformation matrix T is first calculated: and

[0091] (where i, j = 1, 2, 3; k = 1, 2, 3)

[0092] Then, according to the derivative results, the update rule of the Levenberg-Marquardt algorithm is used to update the transformation matrix T: T k+1 =T k +ΔT

[0093] Repeat the above steps and continuously update the transformation matrix T until the value of the objective function E(T) converges to a stable minimum value or reaches a preset number of iterations.

[0094] In step S134, two sets of point cloud data P′ and Q′ after NICP registration are input, where P′={p′ 1 , p′ 2 , ..., p′ m′}Q′={q′ 1 , q′ 2 , ..., q′ n′}

[0095] For a point in the point cloud Q′ The Euclidean distance between them is:

[0096] Record each point p′ i The closest distance to the point cloud Q′ And the corresponding nearest point index, also record each point q′ j The closest distance to the point cloud P′ and the corresponding nearest point index.

[0097] Furthermore, in step S14, the fused point cloud data is reconstructed into a three-dimensional model of the live working personnel using a Poisson reconstruction algorithm, comprising the following steps:

[0098] S141: defining a voxel grid in a three-dimensional space to surround the space where the point cloud data is located, constructing an indicator function for each voxel in the voxel grid according to whether there is a point cloud data point inside, and calculating the gradient field of the indicator function using a central difference method;

[0099] S142: constructing a Poisson equation according to the gradient field, solving the Poisson equation using a multigrid method, setting an isosurface value, extracting the isosurface in the space of the solved function, expressing the isosurface in the form of a triangular mesh using a MarchingCubes algorithm, and reconstructing the surface of the three-dimensional model using the triangular mesh;

[0100] S143: Smoothing the surface of the three-dimensional model, checking for errors in the model and repairing them.

[0101] Specifically, in S141, the point cloud data P after registration and fusion is inputted as {p 1 , p 2 , ..., p n}, where each point p i =(x i ,y i , z i ) is the coordinate point in the three-dimensional space, and then the voxel grid in the three-dimensional space is defined to construct the indicator function χ(x): if there is a point cloud data point in the voxel, then χ(x) = 1; otherwise χ(x) = 0.

[0102] In step S142, the Poisson equation is constructed based on the gradient field Build, for:

[0103]

[0104] Where f is the function to be solved, Δ is the Laplace operator, is a known vector field.

[0105] Furthermore, in step S2, the human body contour extraction, key node extraction and posture parameter calculation, the establishment of a body model and the body posture recognition and classification include the following steps:

[0106] S21: The density clustering algorithm is used to separate the human body from the background and other objects in the point cloud based on the spatial range. The Marching Cubes algorithm extracts the human body contour from the segmented human body point cloud area and smoothes the extracted contour.

[0107] S22: The trained convolutional neural network predicts the key node positions in the point cloud, which correspond to the joints of the human body, and the predicted key nodes are screened and optimized;

[0108] S23: Establish a local coordinate system with a fixed part of the human body as the origin, calculate the angles between adjacent joints according to the positions of key nodes in the local coordinate system, calculate the limb length ratio, and determine the body inclination angle by calculating the angle between the center of gravity of the human body and the vertical direction;

[0109] S24: The body model is obtained by training the model through support vector machine and a large amount of collected point cloud data of different human bodies in various postures, and the characteristics and patterns of different postures are learned;

[0110] S25: The newly collected point cloud data of the human body to be identified is input into the trained body model, the features of the input point cloud data are extracted and matched and compared with the features of different postures learned during the training process, and the human body posture category is output according to the matching result model to complete the identification and classification.

[0111] Specifically, in step S21, for human body area segmentation, the point cloud corresponding to the human body is clustered into an independent area according to the spatial distribution and distance relationship of the points, and then the contour is extracted through the edge detection algorithm. The MarchingCubes algorithm is used to extract the human body contour from the segmented human body point cloud area, and the extracted contour is smoothed to remove jagged and irregular parts to make the contour more continuous and natural.

[0112] In step S22, the predicted key nodes are screened and optimized to remove misdetected points. The positions of the key nodes may also be constrained and adjusted according to the physiological structure of the human body and prior knowledge to make them more consistent with the actual joint positions of the human body.

[0113] In step S23, the angles between adjacent joints are calculated according to the positions of the key nodes in the local coordinate system; for example, for the knee joint, the bending angle of the knee joint can be determined by calculating the angle between the thigh and calf direction vectors. The limb length ratio is calculated, such as the ratio of arm length to body height, the ratio of leg length to body height, etc.

[0114] In step S24, a large amount of point cloud data of different human bodies in various postures are collected and annotated, and the annotation content includes human body posture categories (such as standing, walking, sitting, bending, etc.), key node positions, posture parameters, etc. A support vector machine is selected as a model, and the annotated point cloud data is used as a training set and input into the selected model for training, so that the model learns the characteristics and patterns of different postures.

[0115] Furthermore, the acquisition of point cloud data of the live working site, including the location information of the workers and the live equipment, comprises the following steps:

[0116] A1: Before live working begins, the working area is scanned by the laser radar 4, the acquisition time of each point cloud data is recorded and transmitted to the data processing equipment via the wireless network;

[0117] A2: Pre-process the transmitted original point cloud data, use the point cloud segmentation algorithm to segment the live workers, live equipment and other background objects from the point cloud data based on the spatial distribution of the point cloud, and perform feature extraction and recognition on the segmented point cloud data of the workers and live equipment;

[0118] A3: Calculate the position coordinates of live workers and live equipment in the established three-dimensional coordinate system based on the processed point cloud data, and determine the real-time position changes through point cloud registration and target tracking algorithms.

[0119] Specifically, in step A1, during the scanning process, the laser radar 4 emits laser pulses and receives reflected signals, measures the flight time, angle and other information of each pulse, and generates point cloud data. At the same time, the acquisition time of each point cloud data is recorded for subsequent data synchronization and processing.

[0120] In step A2, the transmitted raw point cloud data is preprocessed, including removing noise points and outliers, and smoothing the data using a filtering algorithm to improve data quality. A point cloud segmentation algorithm based on the spatial distribution of point clouds is used to segment operators, live equipment, and other background objects from the point cloud data. Feature extraction and recognition are performed on the segmented point cloud data of operators and live equipment, such as distinguishing different target objects by features such as shape, size, and position.

[0121] Furthermore, in step S3, the following steps are included: obtaining identification results based on the position changes of the live working personnel and the live equipment, and analyzing and evaluating the body posture of the human body, and making safety standard judgments on the safety distance and action standards:

[0122] S31: setting a safety distance threshold for live working personnel and live equipment, calculating the real-time distance based on the position coordinates of the live working personnel and live equipment, and comparing the distance with the safety distance threshold;

[0123] S32: Establish an action specification model based on the safe operation procedures and standards for live working, and match and compare the action posture and action process of the operator extracted from the point cloud data with the action specification model;

[0124] S33: Record and store the collected point cloud data, position and posture information, and the judgment result data of the safety distance and action specification.

[0125] Specifically, in step S31, after the positions of the live workers and the live equipment are extracted and the posture of the workers is extracted, the safety distance threshold between the live workers and the live equipment is determined according to the voltage level, operation type and relevant safety standards of the operation scene. The real-time distance between the live workers and the live equipment is calculated based on the extracted position coordinates. The real-time distance is compared with the set safety distance threshold.

[0126] In step S33, the judgment results of the safety distance and action specifications are output in an intuitive manner, and the collected point cloud data, position and posture information, judgment results of the safety distance and action specifications and other data are recorded and stored for subsequent accident analysis, work evaluation and safety training.

[0127] Further, in step S32, the action specification model is established based on the safe operation procedures and standards for live working, and the action posture and action process of the operator extracted from the point cloud data are matched and compared with the action specification model, including the following steps:

[0128] S321: The safety operation procedures and standards for live working are decomposed into a series of action units, and clear action characteristics are defined for each basic action unit. The allowable action range of each basic action unit is determined in combination with ergonomic principles, and the speed and force limits of the action are set;

[0129] S322: Determine the operation sequence and process of different operation tasks according to the actual live operation process, model the sorted operation sequence and process through a directed graph, and verify the established operation process model using model verification technology;

[0130] S323: Use laser scanning to build a three-dimensional model of the site, convert environmental factors into constraints on action specifications, and use a bounding box-based collision detection algorithm to perform collision detection between the operator's actions and obstacles in the environment;

[0131] S324: Use virtual reality technology to simulate the live working environment and conduct simulated operation to verify the model. Select some actual work tasks for real-time comparison and verification at the live working site, and optimize and improve the model based on the verification results.

[0132] Specifically, in step S321, the national, industry and enterprise internal safety operating procedures, standards and operation guidance documents related to live working are collected. These documents specify in detail the allowed and prohibited actions, operation sequence, safety precautions and other contents in specific live working scenarios. The collected safety operating procedures are deeply analyzed to understand the safety requirements, purpose and principles behind each step and action, and extract the key action specification elements. The live working process is decomposed into a series of basic action units, such as raising hands, stretching, bending, climbing, tool operation, etc. Clear action characteristics are defined for each basic action unit, including the starting state, end state, motion trajectory, involved body parts, required strength and speed, etc. According to the safety operating procedures and ergonomic principles, the allowable action range of each basic action unit is determined, such as: the height limit of raising hands, the distance limit of stretching, and the angle limit of bending, etc., and the speed and force limits of the action are set to prevent safety risks caused by too hasty actions or excessive force, such as: the maximum swing speed and maximum applied force when operating tools, etc.

[0133] In step S322, the operation sequence and process of different operation tasks are sorted out by observing and analyzing the actual live operation process. For example, for the live maintenance operation of overhead lines, the process may include: arriving at the operation site, setting up safety protection facilities, climbing poles and towers, approaching live wires, performing maintenance operations, evacuating the operation site, etc. The sorted out operation sequence and process are modeled through a directed graph. For example, the nodes of the directed graph can be used to represent the various action units in the operation process, and the directed edges represent the sequence and logical relationship between the action units. The established operation process model is verified using model verification technology to check whether the model meets the safety operation procedures and logical rationality requirements. According to the verification results, the operation process model is optimized and adjusted to ensure that the model can accurately reflect the actual operation process and safety requirements.

[0134] In step S323, the influence of environmental factors must also be considered, and the environmental factors of the live working site must be analyzed in detail, including the size and shape of the working space, the distribution of obstacles, the layout of equipment, and weather conditions. Use laser scanning, photogrammetry and other technologies to perform three-dimensional modeling of the working site to obtain the geometric information and spatial layout of the environment. Convert environmental factors into constraints on action specifications. For example, if the working space is narrow, then limit the operator's body extension range and movement range accordingly. Use collision detection algorithms such as bounding box-based collision detection algorithms to detect collisions between the operator's actions and obstacles in the environment to ensure that the actions are performed within a safe space.

[0135] In step S324, virtual reality technology is used to embed the established action specification model into a virtual live-line operation scene, and experienced operators are invited to perform simulated operation in a virtual environment to verify the accuracy and practicality of the model. During the simulated operation, the action data of the live-line operators are collected, and compared and analyzed with the action specifications in the model to check whether there are deviations and non-conformities. At the actual live-line operation site, some work tasks are selected. During the execution of the tasks by the live-line operators, sensors (such as inertial sensors, optical sensors, etc.) are used to collect the action data of the live-line operators in real time, and the action data is compared and verified with the action specification model in real time. According to the results of simulation verification and field verification, the action specification model is optimized and improved, and the parameters such as the range, limit, and sequence of the action are adjusted to make the model more in line with the needs and safety requirements of the actual operation.

[0136] Further, in step S4, when the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and feedback and guidance are given in time, including the following steps:

[0137] S41: When the safety distance threshold is exceeded, the alarm condition is triggered. When the actual action of the live working personnel is detected to be different from the standard action defined in the model, the alarm condition is triggered.

[0138] S42: Once the alarm condition is triggered, an alarm signal is generated, an alarm sound with a specific frequency, duration and intensity is generated using an audio synthesis algorithm, and a warning effect is achieved by controlling the flashing frequency and color of the light emitting diode;

[0139] S43: When a safety distance violation occurs: generate feedback and output guidance information based on the location information of the operator and the live equipment; when an irregular action is detected: generate feedback and output guidance suggestion information based on the action specification model and combined with the actual action data comparison results of the live operator.

[0140] Specifically, in step S41, according to the established safety distance standard, the corresponding distance threshold parameter is set in the system. When the calculated real-time distance between the operator and the live equipment is less than this threshold, the alarm condition is triggered; the established action specification model is converted into rules and parameters that can be recognized by the system. When it is detected that the actual action of the operator deviates from the standard action defined in the model, such as the movement amplitude, speed, posture, etc. exceed the allowable range, the alarm condition is triggered.

[0141] In step S42, once the alarm condition is triggered, the alarm module of the system generates a corresponding alarm signal. The alarm signal may include a sound signal, a light signal or an electrical signal. For a sound alarm signal, an audio synthesis algorithm can be used to generate an alarm sound with a specific frequency, duration and intensity; for a light signal, the warning effect can be achieved by controlling the flashing frequency and color of a light emitting diode (LED) or other lighting equipment. The generated alarm signal is transmitted wirelessly to the corresponding output device, such as a speaker, a warning light, a smart bracelet or a helmet display worn by the operator, etc.

[0142] In step S43, when a safety distance violation occurs, the system generates feedback information based on the real-time position information of the operator and the live equipment, combined with the three-dimensional model of the on-site environment, to prompt the operator of the direction and approximate distance in which the current distance is too close, and gives guidance information, such as "Please move back [X] meters", "Please raise [X] meters", etc., to guide the operator to adjust the position to return to the safe distance range. For the detected irregular movements, the system generates detailed feedback information based on the comparison results of the movement standard model and the actual operator's movement data, describing the specific irregular movement content and degree of deviation, and gives corresponding guidance information, such as "Please slow down the movement speed" or "Please adjust the arm posture", etc., to help the operator correct the movement in time.

[0143] Feedback and guidance information is presented to operators and on-site supervisors through display devices (such as large-screen displays, tablet computers), voice broadcasting devices (such as speakers, headphones) or wearable devices (such as smart bracelets, augmented reality glasses) installed on site. For example, text prompts and arrow instructions are displayed on the operator's helmet display to guide the operator to adjust the position or action; or voice guidance information is played in a loop through the on-site speakers. Feedback and guidance information is transmitted to the remote monitoring end (such as the computer system of the monitoring center) at the same time, so that remote monitoring personnel can understand the safety status of the work site in real time and provide remote support and guidance when necessary. The interface of the remote monitoring end can display real-time video, point cloud data, alarm information, feedback and guidance content of the work site, etc. The monitoring personnel can communicate and guide the on-site operators through voice calls, text messages, etc.

[0144] Furthermore, the binocular laser radar is composed of two laser radars 4, which are respectively arranged on a rotating component 1, the rotating component 1 is fixed at both ends of the first telescopic support rod 2, the middle part of the first telescopic support rod 2 is fixedly connected to one end of the second telescopic support rod 3, the other end of the second telescopic support rod 3 is connected to the rotating component 1, and the infrared sensor is arranged on the second telescopic support rod 3. The rotating component 1 is a device that can drive the equipment and control the angle for rotation.

[0145] Specifically, the first telescopic support rod 2 and the second telescopic support rod 3 can be electrically controlled to change the length of the support rods, or can be manually adjusted to complete the length change of the support rods. The two laser radars 4 are respectively arranged on the rotating assembly 1, and the rotating assembly 1 drives the laser radar 4 to rotate. The other end of the second telescopic support rod 3 is connected to the rotating assembly 1, and the rotating assembly 1 drives the second telescopic support rod 3 and other devices arranged on the support rod to rotate together. The infrared sensor is arranged on the second telescopic support rod 3. During the tracking and positioning process, the infrared sensor drives the second telescopic support rod 3 to rotate through the rotating assembly 1 connected to the second telescopic support rod 3, thereby changing and adjusting the scanning direction of the infrared sensor for tracking and positioning. The detection directions of the infrared sensor and the binocular laser radar are both in the same direction. When the infrared sensor tracks and locates the live working personnel, the binocular laser radar can directly perform the scanning operation after fine-tuning.

[0146] The two laser radars 4 are respectively arranged on the rotating assembly 1. In order to prevent the live-line workers from suffering from pulse damage from the laser radar 4, the laser power is usually within the laser safety level of Class 1 or Class 2. In order to ensure that the two laser radars 4 can scan the target object from different directions, and at the same time ensure that the measurement ranges of the two laser radars 4 have a certain overlapping area to cover the obstructed parts, the two rotating assemblies 1 adjust their respective corresponding angles. The binocular laser radar can be installed on a power operation boom truck, a drone or a movable track. When the working environment requires higher scanning accuracy, two binocular laser radars can be set up to work simultaneously, and two or more binocular laser radars are set on both sides or around the live-line workers.

[0147] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for monitoring potential of live working based on LiDAR, characterized in that: The method includes: S1: Use infrared sensors to locate and track live-line workers. After determining their positions, use redundant binocular laser radars to scan them and collect point cloud data in different postures and movements. S2: Extract human body contours, extract key nodes, and calculate posture parameters, establish a body model, perform body recognition and classification, and obtain point cloud data of live working sites, including the location information of operators and live equipment; S3: According to the position changes of live workers and live equipment, as well as the analysis and evaluation of human body posture, the recognition results are obtained, and the safety distance and action specifications are judged; S4: When the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and timely feedback and guidance are given.

2. The method for monitoring equipotential in live working based on LiDAR according to claim 1, characterized in that: In step S1, after determining the position of the live-line worker, the live-line worker is scanned by mutually redundant binocular laser radars to collect point cloud data in different postures and actions, including the following steps: S11: Start two laser radars at the same time to perform synchronous scanning on the live-line workers, obtain point cloud data from different perspectives of the live-line workers in the same time period, and record the emission time, return time and scanning angle of each laser pulse during the scanning process; S12: Remove noise points through median filtering, convert the point cloud data collected by the two laser radars into the same coordinate system for coordinate conversion, and extract feature points from the point cloud data of the two laser radars using the SIFT feature extraction algorithm; S13: Using the extracted feature points, the point cloud data of the two lidars are matched and aligned through the NICP registration algorithm, and then the registered point cloud data are fused through the distance-based fusion method to obtain a complete point cloud model of the live working personnel; S14: Use the Poisson reconstruction algorithm to reconstruct the fused point cloud data into a three-dimensional model of the live working personnel.

3. The method for monitoring equipotential in live working based on LiDAR according to claim 2, characterized in that: In step S13, the extracted feature points are used to match and align the point cloud data of the two laser radars through the NICP registration algorithm, and then the registered point cloud data are fused through the distance-based fusion method to obtain a complete point cloud model of the live working personnel, including the following steps: S131: The initial transformation matrix is ​​obtained based on the feature-based preliminary registration method, and the nearest point qi in the target point cloud is found in the point cloud after the initial transformation through the Euclidean distance between the points. closcst ; S132: Based on the NICP objective function, the position information and normal vector information of the point are involved. For each corresponding point (p i ,qi closcst ), the objective function is defined as: in, is the transformation matrix to be solved, which contains the rotation R and translation t; and They are point p i and qi closcst The unit normal vector of the y-axis. α is the weight parameter used to balance the position error and the normal vector error. w i is the weight of the corresponding point pair; S133: Minimize the objective function E(T) by the Levenberg-Marquardt algorithm to perform iterative optimization. For each iteration, firstly, calculate the partial derivatives of the objective function E(T) with respect to the elements of the transformation matrix T, and then update the transformation matrix T according to the derivative results using the update rule of the Levenberg-Marquardt algorithm. Repeat the above steps until the value of the objective function E(T) converges to a stable minimum value or reaches a preset number of iterations to complete the registration. S134: Using the distance-based fusion method, input the two sets of point cloud data P′ and Q′ after NICP registration, for each point p′ in the point cloud P′ i Calculate the distance to all points in point Q′, for each point q′j in point cloud Q′ calculate the distance to all points in point cloud P′, and set a distance threshold; S135: When point p′ i If the minimum distance to the point cloud is less than the distance threshold, p′ i Add to the fused point cloud set; when point q′ j If the distance to the nearest point in the point cloud P′ is less than the distance threshold, then q′ j Add to the fused point cloud collection; S136: De-duplication processing is performed on the fused point cloud set through a hash table.

4. The method for monitoring potential of live working based on LiDAR according to claim 2, characterized in that: In step S14, the fused point cloud data is reconstructed into a three-dimensional model of the live working personnel using the Poisson reconstruction algorithm, including the following steps: S141: defining a voxel grid in a three-dimensional space to surround the space where the point cloud data is located, constructing an indicator function for each voxel in the voxel grid according to whether there is a point cloud data point inside, and calculating the gradient field of the indicator function using a central difference method; S142: constructing a Poisson equation according to the gradient field, solving the Poisson equation using a multigrid method, setting an isosurface value, extracting the isosurface in the space of the solved function, expressing the isosurface in the form of a triangular mesh using a Marching Cubes algorithm, and reconstructing the surface of the three-dimensional model using the triangular mesh; S143: Smoothing the surface of the three-dimensional model, checking for errors in the model and repairing them.

5. The method for monitoring equipotential in live working based on LiDAR according to claim 1, characterized in that: In step S2, the human body contour extraction, key node extraction and posture parameter calculation are performed, a body shape model is established, and body shape recognition and classification are performed, including the following steps: S21: The density clustering algorithm is used to separate the human body from the background and other objects in the point cloud based on the spatial range. The Marching Cubes algorithm extracts the human body contour from the segmented human body point cloud area and smoothes the extracted contour. S22: The trained convolutional neural network predicts the key node positions in the point cloud, which correspond to the joints of the human body, and the predicted key nodes are screened and optimized; S23: Establish a local coordinate system with a fixed part of the human body as the origin, calculate the angles between adjacent joints according to the positions of key nodes in the local coordinate system, calculate the limb length ratio, and determine the body inclination angle by calculating the angle between the center of gravity of the human body and the vertical direction; S24: The body model is obtained by training the model through support vector machine and a large amount of collected point cloud data of different human bodies in various postures, and the characteristics and patterns of different postures are learned; S25: The newly collected point cloud data of the human body to be identified is input into the trained body model, the features of the input point cloud data are extracted and matched and compared with the features of different postures learned during the training process, and the human body posture category is output according to the matching result model to complete the identification and classification.

6. The method for monitoring potential of live working based on LiDAR according to claim 1, characterized in that: The step of obtaining point cloud data of the live working site, including the location information of the workers and the live equipment, comprises the following steps: A1: Before live-line work begins, the work area is scanned by LiDAR, the acquisition time of each point cloud data is recorded, and it is transmitted to the data processing equipment via a wireless network; A2: Pre-process the transmitted original point cloud data, use the point cloud segmentation algorithm to segment the live workers, live equipment and other background objects from the point cloud data based on the spatial distribution of the point cloud, and perform feature extraction and recognition on the segmented point cloud data of the workers and live equipment; A3: Calculate the position coordinates of live working personnel and live equipment in the established three-dimensional coordinate system based on the processed point cloud data, and determine the real-time position changes through point cloud registration and target tracking algorithms.

7. The method for monitoring equipotential in live working based on LiDAR according to claim 1, characterized in that: In step S3, the following steps are included: according to the position changes of the live working personnel and the live equipment, and the analysis and evaluation of the human body posture to obtain the recognition result, the safety distance and the action specification are judged as safe norms: S31: setting a safety distance threshold for live working personnel and live equipment, calculating the real-time distance based on the position coordinates of the live working personnel and live equipment, and comparing the distance with the safety distance threshold; S32: Establish an action specification model based on the safe operation procedures and standards for live working, and match and compare the action posture and action process of the operator extracted from the point cloud data with the action specification model; S33: Record and store the collected point cloud data, position and posture information, and the judgment result data of the safety distance and action specification.

8. The method for monitoring equipotential in live working based on LiDAR according to claim 7, characterized in that: In step S32, the action specification model is established based on the safe operation procedures and standards for live working, and the action posture and action process of the operator extracted from the point cloud data are matched and compared with the action specification model, including the following steps: S321: The safety operation procedures and standards for live working are decomposed into a series of action units, and clear action characteristics are defined for each basic action unit. The allowable action range of each basic action unit is determined in combination with ergonomic principles, and the speed and force limits of the action are set; S322: Determine the operation sequence and process of different operation tasks according to the actual live operation process, model the sorted operation sequence and process through a directed graph, and verify the established operation process model using model verification technology; S323: Use laser scanning to build a three-dimensional model of the site, convert environmental factors into constraints on action specifications, and use a collision detection algorithm based on a bounding box to detect collisions between the operator's actions and obstacles in the environment; S324: Use virtual reality technology to simulate the live working environment and conduct simulated operation to verify the model. Select some actual work tasks for real-time comparison and verification at the live working site, and optimize and improve the model based on the verification results.

9. The method for monitoring equipotential in live working based on LiDAR according to claim 1, characterized in that: In step S4, when the distance between the operator and the live equipment is less than the safe distance, or when the operator's action is detected to be irregular, an alarm signal is issued and feedback and guidance are given in time, including the following steps: S41: When the safety distance threshold is exceeded, the alarm condition is triggered. When the actual action of the live working personnel is detected to be different from the standard action defined in the model, the alarm condition is triggered. S42: Once the alarm condition is triggered, an alarm signal is generated, an alarm sound with a specific frequency, duration and intensity is generated using an audio synthesis algorithm, and a warning effect is achieved by controlling the flashing frequency and color of the light emitting diode; S43: When a safety distance violation occurs: generate feedback and output guidance information based on the location information of the operator and the live equipment; when an irregular action is detected: generate feedback and output guidance suggestion information based on the action specification model and combined with the actual action data comparison results of the live operator.

10. The method for monitoring equipotential in live working based on LiDAR according to claim 1, characterized in that: The binocular laser radar is composed of two laser radars, which are respectively arranged on a rotating assembly. The rotating assembly is fixed at both ends of a first telescopic support rod. The middle part of the first telescopic support rod is fixedly connected to one end of a second telescopic support rod. The other end of the second telescopic support rod is connected to the rotating assembly. The infrared sensor is arranged on the second telescopic support rod. The rotating assembly is a device that can drive the equipment and control the angle for rotation.

Images