Power distribution network operation abnormal behavior management method and system based on panoramic video analysis

Abstract

The invention discloses a power distribution network operation abnormal behavior management method and system based on panoramic video analysis, and relates to the field of abnormal behavior management. Scene information and operator appearance information are generated according to a panoramic video of a power distribution network operation scene; setting operation standard information of the power distribution network; analyzing the operator data information and the operator appearance information under the scene information according to the power distribution network operation standard information to generate first behavior information; generating a personnel attitude structure according to the operator data information, analyzing the panoramic video of the power distribution network operation scene, obtaining gradient histogram direction features, and generating personnel prediction behavior information; analyzing the personnel prediction behavior information based on the power distribution network operation standard information to generate second behavior information; and giving a prompt according to the first behavior information and the second behavior information, and performing operation behavior process auditing by a manager according to the prompt to judge whether an abnormal behavior exists or not.

Description

Technical Field

[0001] The present invention relates to the field of abnormal behavior management, and in particular to a method and system for managing abnormal behavior of distribution network operations based on panoramic video analysis. Background Art

[0002] Panoramic video is a method of capturing image information from multiple angles at the same time by using a special panoramic camera or a combination of multiple cameras, and then stitching and processing the image data to generate a video that can freely select viewing angles and view content in all directions. Abnormal distribution network operation behavior refers to various illegal operating behaviors that occur during the operation of the distribution network when the operating personnel violate relevant safety regulations, operating specifications, process flows and industry standards, such as performing maintenance operations without shutting off the power, testing the electricity, and hanging the ground wire as required, failing to maintain a safe distance when working with power on, failing to wear and use personal protective equipment correctly, operating electrical equipment in violation of regulations, and arbitrarily changing the operating process or simplifying the operating steps.

[0003] The existing abnormal behavior management methods of distribution networks are difficult to fully capture anomalies in data, and the data processing capabilities are weak. Faced with massive real-time data, they are unable to efficiently screen and analyze key information, resulting in delayed anomaly identification and prone to misjudgment. When faced with difficult and complex distribution network operations, there is a lack of standardized operating specifications, and the handling methods of different regions and personnel vary greatly, resulting in low coordination efficiency, which makes the operating personnel behave differently in emergency handling, resulting in insufficient awareness of the hazards of abnormal behavior, and unable to effectively prevent the recurrence of abnormal behavior, making it difficult to meet the growing needs of safe and efficient operation of distribution networks. Summary of the invention

[0004] In order to solve the above technical problems, a method and system for managing abnormal behavior of distribution network operations based on panoramic video analysis are provided. This technical solution solves the problems raised in the above background technology, such as the difficulty in comprehensively capturing anomalies, weak data processing capabilities, and the inability to efficiently screen and analyze key information in the face of massive real-time data, resulting in delayed anomaly identification and prone to misjudgment, as well as the lack of standardized operating specifications when faced with difficult and complex distribution network operations.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] A method for managing abnormal behavior of distribution network operations based on panoramic video analysis, characterized by comprising:

[0007] Obtain panoramic video of distribution network operation scenes and operator information;

[0008] Generate scene information and operator appearance information based on the panoramic video of the distribution network operation scene;

[0009] Collect relevant specifications for distribution network operations and set distribution network operation standard information;

[0010] Based on the distribution network operation standard information, analyze the operator profile information and operator appearance information in the scenario information to generate the first behavior information;

[0011] According to the operator profile information, generate a human pose structure, and combine it with the panoramic video of the distribution network operation scenario to obtain the histogram of oriented gradients (HOG) features;

[0012] Generate operator predicted behavior information through the histogram of oriented gradients (HOG) features;

[0013] Based on the distribution network operation standard information, analyze the operator predicted behavior information to generate the second behavior information;

[0014] Give a prompt according to the first behavior information and the second behavior information. The manager conducts an audit of the operation behavior process according to the prompt to determine whether there is abnormal behavior.

[0015] Preferably, the generation of the first behavior information specifically includes:

[0016] Obtain the time when the operator enters the scenario, the facial information of the person entering the scenario, and the required operation content;

[0017] According to the time when the operator enters the scenario, extract the panoramic video data of the hour before the operator enters the distribution network scenario to generate a pre-operation panoramic record;

[0018] Extract image frames from the pre-operation panoramic record at a rate of one frame per second to generate each static image;

[0019] Based on the region growing method, analyze each static image to generate the scene object boundaries of each image;

[0020] Perform image boundary matching on the scene object boundaries of each image to obtain boundary rule parameters, and judge whether the object boundary rule parameters are greater than the rule threshold. If so, it indicates that there is no interfering weather in the scene. If not, it indicates that there is interfering weather in the scene;

[0021] According to the judgment result, analyze the relevant specifications for distribution network operations, adjust the distribution network operation standard information, and generate the operation wearing standard for the current scenario;

[0022] According to the operator profile information, set and bind a unique identifier for each operator. The operator profile information includes personnel operation qualification information and personnel facial feature information;

[0023] Determine the operator qualification information by comparing the facial information of the person entering the scene with the personnel facial feature information;

[0024] According to the relevant specifications for distribution network operations, analyze the required operation content, generate the qualifications that the operators should have, compare them with the operator qualification information, judge whether the personnel entering the scene are qualified to perform the operations, and generate a qualification judgment result;

[0025] Compare the appearance information of the operators with the current scene operation wearing standards, judge whether the wearing appearance of the operators can perform safe operations, and generate an appearance judgment result;

[0026] Comprehensively analyze the qualification judgment result and the appearance judgment result to generate the first behavior information.

[0027] Preferably, the generation of the personnel posture structure specifically includes:

[0028] According to the operator profile information, obtain the height, body proportion and arm span data of the operators, divide the operator types according to the operator qualification information, and generate ground operation types and aerial operation types;

[0029] Obtain the standard bone structure diagram and standard body proportion corresponding to each height of the human body;

[0030] Take the ratio of the body proportion of the operators at each height to the standard body proportion as the individual bone difference, and combine it with the standard bone structure diagram to generate an individual bone structure diagram;

[0031] Perform body posture superposition on the individual bone structure diagram according to the arm span data to generate an individual form fitting diagram;

[0032] Obtain the distribution characteristics of human body joint points, analyze the individual form fitting diagram of the ground operation type, and generate upper limb joint point information;

[0033] According to the distribution characteristics of human body joint points, analyze the individual form fitting diagram of the aerial operation type, and generate torso joint point information;

[0034] Substitute the torso joint point information and the upper limb joint point information into the individual form fitting diagram respectively, and generate the personnel posture structure based on the kinematic principle.

[0035] Preferably, the acquisition of the gradient histogram direction characteristics specifically includes:

[0036] Extract the image frames of the panoramic video at one frame per second, and convert the image frames into grayscale images;

[0037] Locate the operators according to the human body structure posture, and generate a personnel bounding box;

[0038] Obtain the left pixel value, right pixel value, upper pixel value and lower pixel value of each pixel within the personnel bounding box in the grayscale image;

[0039] The horizontal gradient of a pixel is obtained by subtracting the left pixel value from the right pixel value, and the vertical gradient of a pixel is obtained by subtracting the lower pixel value from the upper pixel value;

[0040] Based on the horizontal gradient and vertical gradient of the pixel, the gradient magnitude and direction of each pixel are generated;

[0041] The person bounding box is divided into 16×16 cells, and the gradient magnitudes of the pixels within the cells are statistically analyzed to generate a gradient magnitude histogram;

[0042] According to the pixel direction, the gradient magnitude histograms are superimposed to generate the gradient histogram orientation feature;

[0043] Among them, the specific calculation formulas for generating the gradient magnitude and direction of a pixel are:

[0044]

[0045] In the formula, G represents the gradient magnitude of the pixel, G x represents the horizontal gradient of the pixel, G y represents the vertical gradient of the pixel, and θ represents the pixel direction angle.

[0046] Preferably, the generation of the second line of information specifically includes:

[0047] Analyze each frame of the grayscale image converted from the panoramic video to obtain the gradient magnitude histogram features of adjacent frames;

[0048] Compare the gradient magnitude histogram features of adjacent frames, extract the movement of the person's posture in adjacent frames, and generate the predicted change range and movement direction of the person's posture;

[0049] According to the distribution network operation standard information, expand the information of the person bounding box to generate the standard operation process posture and the risk operation area;

[0050] By comparing the standard operation process posture with the predicted change range of the person's posture, determine whether the operation process posture of the person is reasonable, and generate an operation judgment result;

[0051] According to the predicted change range and movement direction of the person's posture, generate the movement trend of the operating personnel;

[0052] By comparing the movement trend of the operating personnel with the location of the risk operation area, determine whether the operating personnel are about to enter the risk operation area, and generate a movement judgment result;

[0053] Combine the operation judgment result and the movement judgment result to generate the second line of information.

[0054] Preferably, the determination of whether there is an abnormal behavior specifically includes:

[0055] When the management personnel receive the first-line information, check whether there is any abnormality in the first-line information. If so, conduct a review of the first-line information to find out the cause of the abnormality. If not, allow the operators to start working;

[0056] The management personnel give safety reminders to the operators according to the cause of the abnormality;

[0057] When the second-line information is abnormal and a reminder is issued, the management personnel need to manually analyze the real-time operation steps of the operators, judge whether the current operators' operations are abnormal. If so, guide the subsequent operation steps of the current operators and train the current operators. If not, cancel the abnormal reminder and record it.

[0058] Furthermore, a management system for abnormal behavior in distribution network operations based on panoramic video analysis is proposed to implement the above-mentioned abnormal behavior management method, including:

[0059] A collection module, which is used to obtain the panoramic video of the distribution network operation scene and the information of the operators, collect the relevant specifications of the distribution network operation, and transmit the collected data to the processing module and the judgment module;

[0060] A processing module, which is used to process the received data. According to the panoramic video of the distribution network operation scene, generate scene information and the appearance information of the operators. According to the distribution network operation standard information, analyze the operator information and the appearance information of the operators under the scene information to generate the first-line information, analyze the predicted behavior information of the personnel to generate the second-line information, generate the personnel posture structure according to the operator information, and transmit the data to the judgment module;

[0061] A judgment module, which is used to analyze and judge the received data, send a prompt to the management personnel through the first-line information and the second-line information. The management personnel conduct a review of the operation behavior process according to the prompt, judge whether there is any abnormal behavior, and transmit the data to the management module;

[0062] A management module, which is used to receive the prompt transmitted by the judgment module, further review and confirm the received prompt through the management personnel, and then sort and store the results of the further review.

[0063] Preferably, the collection module specifically includes:

[0064] A first collection unit, which is used to obtain the time when the operator enters the scene, the facial information of the person entering the scene, and the required operation content, and transmit the data to the processing module;

[0065] A second collection unit, which is used to obtain the height, body proportion and arm span data of the operator according to the operator profile information and transmit the data to the processing module;

[0066] A third collection unit, which is used to obtain the panoramic video of the distribution network operation scenario and transmit the data to the processing module and the judgment module.

[0067] Preferably, the processing module specifically includes:

[0068] A first processing unit, which is used to extract image frames from the pre-operation panoramic record at a rate of one frame per second to generate static images, analyze each static image based on the region growing method to generate the scene object boundaries of each image, and transmit the data to the judgment module;

[0069] A second processing unit, which is used to perform body posture superposition on the individual bone structure diagram according to the arm span data to generate an individual form fitting diagram, analyze the individual form fitting diagrams of the ground operation types and the high-altitude operation types respectively according to the distribution characteristics of human joint points to obtain the upper limb joint point information and the trunk joint point information, generate a personnel posture structure, and transmit the data to the judgment module;

[0070] A third processing unit, which is used to locate the operator according to the human body structure posture to generate a personnel bounding box, generate the gradient amplitude and direction of each pixel point through the horizontal gradient and vertical gradient of the pixel points, statistically analyze the gradient amplitude of each pixel point in the unit to generate a gradient amplitude histogram, and superimpose the gradient amplitude histogram according to the pixel point direction to generate a gradient histogram direction feature, and transmit the data to the judgment module.

[0071] Preferably, the judgment module specifically includes:

[0072] A first judgment unit, which is used to perform image boundary matching on the scene object boundaries of each image to obtain boundary rule parameters, judge whether the object boundary rule parameters are greater than the rule threshold, and transmit the result to the management module;

[0073] A second judgment unit, which is used to compare with the operator's due qualifications according to the operator qualification information to judge whether the personnel entering the scene are qualified to perform the operation, compare the appearance information of the operator with the current scene operation wearing standard to judge whether the wearing appearance of the operator can perform safe operation, and transmit the result to the management module;

[0074] A third judgment unit, which is used to judge whether the operation process posture of the person is reasonable by comparing the standard operation process posture and the predicted change range of the person's posture, generate an operation judgment result, judge whether the person to be operated is going to enter the risk operation area by comparing the movement trend of the operator and the location of the risk operation area, generate a movement judgment result, and transmit the result to the management module;

[0075] A fourth judgment unit, which is used for the management personnel to manually analyze the real-time operation steps of the operator, judge whether the operation of the current operator is abnormal, and transmit the result to the management module.

[0076] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0077] The present invention proposes a management scheme for abnormal behavior in distribution network operations based on panoramic video analysis. According to the panoramic video of the distribution network operation scenario, scene information and operator appearance information are generated. By collecting relevant specifications for distribution network operations, distribution network operation standard information is set. According to the distribution network operation standard information, the operator profile information and operator appearance information in the scene information are analyzed to generate first behavior information. According to the operator profile information, a personnel posture structure is generated. The panoramic video of the distribution network operation scenario is analyzed to obtain the histogram of oriented gradients (HOG) features, and personnel predicted behavior information is generated. Based on the distribution network operation standard information, the personnel predicted behavior information is analyzed to generate second behavior information. A prompt is issued according to the first behavior information and the second behavior information. The management personnel conduct a review of the operation behavior process according to the prompt to judge whether there is abnormal behavior. In this way, abnormalities can be effectively captured. In the face of a large amount of real-time data, key information can be efficiently screened and analyzed, reducing the lag in anomaly recognition and the occurrence of misjudgment situations. When facing high-difficulty and complex distribution network operation behaviors, the work efficiency of personnel is improved, the emergency handling ability of operators is improved, and the recurrence of abnormal behaviors is effectively prevented. Description of the Drawings

[0078] Figure 1 It is a flow chart of the method for managing abnormal behavior in distribution network operations based on panoramic video analysis proposed by the present invention;

[0079] Figure 2 It is a flow chart of the method for generating the first behavior information in the present invention;

[0080] Figure 3 It is a flow chart of the method for generating the personnel posture structure in the present invention;

[0081] Figure 4 It is a flow chart of the method for obtaining the histogram of oriented gradients (HOG) features in the present invention;

[0082] Figure 5Flowchart of the method for generating the second behavior information in the present invention;

[0083] Figure 6 Flowchart of the method for determining whether there is an abnormal behavior in the present invention;

[0084] Figure 7 Structural diagram of the abnormal behavior management system for distribution network operation based on panoramic video analysis proposed by the present invention. Detailed implementation manners

[0085] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations.

[0086] Refer to Figure 1 As shown, an abnormal behavior management method for distribution network operation based on panoramic video analysis includes:

[0087] Obtain the panoramic video of the distribution network operation scene and the information of the operation personnel;

[0088] Generate the scene information and the appearance information of the operation personnel according to the panoramic video of the distribution network operation scene;

[0089] Collect the relevant specifications of distribution network operation and set the standard information of distribution network operation;

[0090] Based on the standard information of distribution network operation, analyze the information of the operation personnel and the appearance information of the operation personnel under the scene information to generate the first behavior information;

[0091] Generate the personnel posture structure according to the information of the operation personnel, combine the panoramic video of the distribution network operation scene, and obtain the histogram of oriented gradients (HOG) features; generate the predicted behavior information of the personnel through the HOG features;

[0092] Based on the standard information of distribution network operation, analyze the predicted behavior information of the personnel to generate the second behavior information;

[0093] Give a prompt according to the first behavior information and the second behavior information, and the management personnel conduct an audit of the operation behavior process according to the prompt to determine whether there is an abnormal behavior.

[0094] Based on the panoramic video of the distribution network operation scenario, this solution generates scenario information and the appearance information of operating personnel. By collecting relevant specifications for distribution network operations, it sets the standard information for distribution network operations. According to the standard information for distribution network operations, it analyzes the operator profile information and the appearance information of operating personnel in the scenario information to generate the first-line behavior information. Based on the operator profile information, it generates the personnel posture structure, analyzes the panoramic video of the distribution network operation scenario, obtains the histogram of oriented gradients (HOG) features, and generates the predicted behavior information of the personnel. Based on the standard information for distribution network operations, it analyzes the predicted behavior information of the personnel to generate the second-line behavior information. It issues a prompt based on the first-line behavior information and the second-line behavior information. The management personnel conduct a review of the operation behavior process based on the prompt to determine whether there are abnormal behaviors. Among them, the operator profile information includes the facial features of the operating personnel, the power qualification certificate information of the operating personnel, and the pre-employment physical examination information. The relevant specifications for the distribution network include "GB26859-2011 Electric Safety Work Regulations - Electric Power Line Part", "GB / T 36047-2018 Judgment and Handling of Abnormal Voltages in Distribution Networks", "DL / T 1476-2025 Distribution Network Safety Regulations", and "GB / T 36047-2018 Judgment and Handling of Abnormal Voltages in Distribution Networks". The standard information for distribution network operations includes the wearing standards for operating personnel and the standard operation process. The appearance information of operating personnel represents the equipment wearing situation of operating personnel, such as standard insulating gloves, insulating boots, electric insulating safety helmets, goggles and other equipment.

[0095] Refer to Figure 2 As shown, the first-line behavior information is generated, specifically including:

[0096] Obtain the time when the operating personnel enter the scene, the facial information of the personnel entering the scene, and the required operation content;

[0097] According to the time when the operating personnel enter the scene, extract the panoramic video data of the previous hour before the operating personnel enter the distribution network scene to generate a pre-operation panoramic record;

[0098] Extract image frames from the pre-operation panoramic record at a rate of one frame per second to generate each static image;

[0099] Based on the region growing method, analyze each static image to generate the scene object boundaries of each image;

[0100] Perform image boundary matching on the scene object boundaries of each image to obtain boundary rule parameters. Determine whether the object boundary rule parameters are greater than or equal to the rule threshold. If so, it indicates that there is no interfering weather in the scene. If not, it indicates that there is interfering weather in the scene. Divide the logarithm of the object boundary perimeter to the base 10 by the logarithm of the object boundary area to the base 10 to obtain the fractal dimension. Divide the product of 4π and the object boundary area by the square of the object boundary perimeter to obtain the shape factor. Use 0.6 as the weight of the fractal dimension and 0.4 as the weight of the shape factor for weighted summation to generate the boundary rule parameters. Set 0.7 as the rule threshold;

[0101] According to the judgment result, analyze the relevant specifications for distribution network operations, adjust the standard information for distribution network operations, and generate the current scene operation wearing standards;

[0102] Based on the operator's profile information, set and bind a unique identifier for each operator. The operator's profile information includes the operator's operation qualification information and the operator's facial feature information;

[0103] Determine the operator's qualification information by comparing the facial information of the person entering the scene with the operator's facial feature information;

[0104] According to the relevant specifications for distribution network operations, analyze the required operation content, generate the qualifications that the operator should have, compare them with the operator's qualification information, and determine whether the person entering the scene is qualified to perform the operation, generating a qualification judgment result;

[0105] Compare the operator's appearance information with the current scene operation wearing standards to determine whether the operator's wearing appearance can perform safe operations, generating an appearance judgment result;

[0106] Conduct a comprehensive analysis of the qualification judgment result and the appearance judgment result to generate the first-line information. First, judge the qualifications of the operator. After the judgment result shows that the operator is qualified to perform the operation, then conduct an appearance judgment to confirm whether the operator's equipment meets the requirements of the current operation scene, obtaining the first-line information.

[0107] It is understandable that the operating environment of the distribution network is complex and changeable, covering different scenarios. Among them, rainy, snowy weather and sunny days have completely different requirements for wearable equipment. When operating on sunny days, with sufficient light and relatively dry and stable climate, operators usually wear light and breathable cotton work clothes, which are conducive to heat dissipation, wear ordinary safety helmets to protect the head from accidental collisions, and wear anti-slip insulating work shoes on their feet to ensure stable walking on dry ground and effective insulation. In rainy and snowy weather, the environment is slippery and there is a risk of electric leakage. Operators need to change into waterproof raincoats and rain pants. The materials of which are not only waterproof but also have certain windproof performance to prevent rainwater from seeping in. A rainproof cover should be worn outside the safety helmet to avoid the accumulation of rainwater affecting the line of sight and the performance of the safety helmet. At the same time, high-top waterproof insulating rubber boots need to be worn, which can not only prevent water and slip but also provide reliable insulation protection to prevent electric shock accidents caused by accumulated water on the ground, comprehensively ensuring the safety of operators in bad weather. The region growing method is an image segmentation algorithm based on image region features. It starts from multiple seed points and, according to predefined growth criteria, gradually merges adjacent pixels with similar features to the seed points, such as gray values, colors, and textures, into the region where the seeds are located, continuously expanding the region until no pixels that meet the growth criteria can be added, thereby realizing the segmentation of the image into different regions with similar characteristics. The SIFT feature extraction algorithm is used to extract representative feature points and their descriptors on the boundary. Then, using these feature points and their descriptors, through the nearest neighbor matching algorithm, the corresponding relationships are found between the feature points on the object boundaries of different images to form matching pairs. After obtaining a sufficient number of matching pairs, the affine transformation model is used to analyze the matching pairs and calculate the transformation parameters between the object boundaries in different images. Finally, an error threshold of 3-5 pixels is set to determine whether the calculated transformation parameters are within the error threshold. If they are within the error threshold, it is determined that the object boundaries are consistent; otherwise, they are inconsistent. The qualification certificates for the distribution network mainly include electrician certificates, access network operation permits, and high-altitude operation certificates, each corresponding to specific work contents. Among them, electrician certificates are divided into low-voltage and high-voltage. Holders of low-voltage electrician certificates can engage in the installation, commissioning, maintenance, and fault repair of low-voltage equipment in the distribution network below 1 kV, such as the daily inspection and simple fault handling of low-voltage distribution cabinets in residential communities. High-voltage electrician certificates allow operators to operate high-voltage equipment above 1 kV, such as the switching operation of high-voltage switches in substations and the maintenance of high-voltage lines. The access network operation permit can prove that the operator can engage in electrical installation, testing, maintenance, operation, etc. on the receiving device or the power transmission device, including professional testing of large electrical equipment in the distribution network. The high-altitude operation certificate is used for scenarios involving high-altitude operations, such as climbing electric poles for the erection and maintenance of overhead lines, and the installation and commissioning of power equipment on iron towers, ensuring the safe operation of operators in the high-altitude environment.

[0108] Refer to Figure 3 As shown, generate the human pose structure, specifically including:

[0109] Obtain the height, body proportion and arm span data of the operator according to the operator's profile information, classify the operator's job type according to the operator's job qualification information, and generate a ground operation job type and an aerial operation job type;

[0110] Obtain the standard bone structure diagram and standard body proportion corresponding to each human height;

[0111] Take the ratio of the body proportion of the operator with each height to the standard body proportion as the individual bone difference, and combine it with the standard bone structure diagram to generate an individual bone structure diagram;

[0112] Perform posture superposition on the individual bone structure diagram according to the arm span data to generate an individual form fitting diagram. According to human anatomy knowledge, determine the corresponding relationship between the arm span data and the length and position of the upper limb bones, and superimpose the upper limb bone length information reflected by the arm span data on the upper limb part of the individual bone structure diagram to obtain the individual form fitting diagram;

[0113] Obtain the distribution characteristics of human joint points, analyze the individual form fitting diagram of the ground operation job type, and generate upper limb joint point information;

[0114] According to the distribution characteristics of human joint points, analyze the individual form fitting diagram of the aerial operation job type, and generate trunk joint point information;

[0115] Substitute the trunk joint point information and the upper limb joint point information into the individual form fitting diagram respectively, and generate the personnel posture structure based on the kinematic principle.

[0116] It is understandable that there is a close internal connection and specific proportional relationship between the human skeletal structure and height, body proportion, and arm span data. The height of the human body is determined by the lengths of bones such as the spine and lower limb bones. The body proportion reflects the relative lengths and positional relationships of the bones in various parts of the body. There is a certain proportional relationship between the arm span and height as well as the length of the upper limb bones, and this proportional relationship can be obtained by querying relevant medical materials. Through image recognition technology and human pose estimation algorithms, the distribution characteristics of human joint points in images can be accurately captured. These characteristics include the spatial positions of joint points, the distance ratios between them, and the angular relationships. In live working, operations mostly rely on the upper limbs. For example, when holding insulated tools to connect lines and repair equipment, precise control of the upper limb joint points is crucial for ensuring accurate operations and avoiding electric shock. The accuracy of its movements is directly related to the success or failure of the operation and the safety of personnel. Therefore, emphasis is placed on the analysis of upper limb joint points. When working at heights, workers need to maintain body balance at high altitudes. Trunk joint points such as the spine and hips are the keys to maintaining body stability. Through these joint points, changes in the body's center of gravity can be effectively monitored, postures can be adjusted in a timely manner, and the risk of falling can be prevented, ensuring the safety and stability of the entire process of working at heights. Therefore, jobs at heights focus on the joint points of the trunk. Based on the inherent distribution patterns of human joint points in the common action postures of ground jobs, joint point information related to the upper limbs can be extracted from the individual form fitting diagram. For example, by analyzing the relative positions of joint points such as the shoulders, elbows, and wrists, the degree of extension and bending angles of the upper limbs can be determined, thereby generating detailed and accurate upper limb joint point information. By analyzing the relative positions of joint points such as the spine, shoulders, and hips, the degree of bending, torsion angle, and the bias of the body's center of gravity of the trunk can be clarified, and then comprehensive and detailed trunk joint point information can be generated. Substitute the trunk joint point information, such as the position and angle data of joints such as the spine, shoulders, and hips, and the upper limb joint point information, including the corresponding data of joints such as the shoulders, elbows, and wrists, into the individual form fitting diagram. Then, based on the principles of kinematics, considering the degrees of freedom and motion constraints of the joints, calculate the relative positions and motion relationships between joint points. By analyzing the connection methods and motion ranges between adjacent joint points, determine the motion directions and angles such as flexion, extension, and rotation of the joints. Using these data and based on a mathematical model, simulate the motion states of the human body in different postures, and gradually construct a complete personnel posture structure, thereby intuitively presenting the posture characteristics of personnel in specific actions or scenarios.

[0117] Refer to Figure 4 As shown, obtain the gradient histogram direction features, specifically including:

[0118] Extract image frames from the panoramic video at a rate of one frame per second, and convert the image frames into grayscale images;

[0119] Locate the operating personnel according to the human body structure posture, and generate a personnel bounding box;

[0120] Obtain the left pixel value, right pixel value, upper pixel value, and lower pixel value of each pixel within the person bounding box in the grayscale image;

[0121] Derive the horizontal gradient of the pixel point by subtracting the left pixel value from the right pixel value, and derive the vertical gradient of the pixel point by subtracting the lower pixel value from the upper pixel value;

[0122] Generate the gradient magnitude and direction of each pixel point based on the horizontal gradient and vertical gradient of the pixel point;

[0123] Divide the person bounding box into 16×16 cells, count the gradient magnitudes of each pixel point within the cells, and generate a gradient magnitude histogram;

[0124] Superimpose the gradient magnitude histogram according to the pixel point direction to generate a gradient histogram orientation feature;

[0125] Among them, the specific calculation formulas for generating the gradient magnitude and direction of the pixel point are:

[0126]

[0127] In the formula, G represents the gradient magnitude of the pixel point, G x represents the horizontal gradient of the pixel point, G y represents the vertical gradient of the pixel point, and θ represents the pixel point direction angle.

[0128] It is understandable that the computational complexity on grayscale images is lower, the running speed is faster, and color interference is eliminated, making features such as shapes, contours, and textures in the images more prominent, facilitating subsequent processing operations such as edge detection, feature extraction, and image segmentation, and being able to improve the accuracy of image recognition and analysis. Through the human pose estimation algorithm, the human body structure pose in the image is accurately detected, and the position information of each joint point is identified. Then, based on the distribution and connection relationship of these joint points, the overall shape and position range of the human body are analyzed, and an external rectangle with the outermost joint points as the boundary is generated as the personnel bounding box for positioning the operator. The already delimited personnel bounding box is precisely divided and evenly segmented into small units of 16×16 to ensure that each unit can cover a certain number of pixel points. For the pixel points within each unit, the gradient magnitude of each pixel point is calculated to measure the degree of change of the pixel point in the image. Then, several gradient magnitude intervals are set for each unit, all the pixel points within the unit are traversed, and according to the magnitude of their gradient, the pixel points are classified into the corresponding intervals, and the number of pixel points in each interval is counted. Finally, based on the number of pixel points in each interval obtained from the statistics, the gradient magnitude histogram of the unit is constructed. By dividing 0° - 360° into an interval every 20°, 18 angular intervals are obtained, and the pixel points are classified into the corresponding intervals according to their directions. For the gradient magnitude histogram of each unit, the gradient magnitudes within the same direction interval are superimposed according to the direction interval to which the pixel points belong. After the above operations are completed for all 16×16 units within the personnel bounding box, the histograms of each unit are connected in sequence to form the gradient histogram orientation feature.

[0129] Refer to Figure 5 As shown, generate the second row of information, specifically including:

[0130] Analyze each frame of the grayscale image converted from the panoramic video to obtain the gradient magnitude histogram features of adjacent frames;

[0131] Compare the gradient magnitude histogram features of adjacent frames, extract the movement situation of the personnel pose in adjacent frames, and generate the predicted change range and movement direction of the personnel pose;

[0132] According to the distribution network operation standard information, expand the information of the personnel bounding box to generate the standard operation process pose and the risk operation area;

[0133] By comparing the standard operation process pose and the predicted change range of the personnel pose, judge whether the operation process pose of the personnel is reasonable and generate an operation judgment result;

[0134] Generate the movement trend of the operator according to the predicted change range and movement direction of the personnel pose;

[0135] By comparing the movement trend of the operator and the location of the risk operation area, it is determined whether the operator is about to enter the risk operation area, and a movement judgment result is generated;

[0136] Combining the operation judgment result and the movement judgment result, a second-line message is generated.

[0137] It can be understood that the cosine similarity is used to compare the histogram of gradient magnitude feature vectors of adjacent frames to quantify the degree of difference between them. The change in the magnitude of each interval in the feature vector corresponds to the change in the limb movements of the operator. According to the feature differences, the change in the human posture between adjacent frames is analyzed. Through the correlation relationship between the histogram of gradient magnitude features and the human body joint points, the change trend of the joint point positions is deduced. By analyzing the change trend of the joint point positions, the moving direction of the human posture is determined. At the same time, based on the magnitude of the posture change between adjacent frames, relevant data on the human posture in past operation scenarios are collected, including the joint point positions and the histogram of gradient magnitude features under different actions. These historical data are classified and sorted according to the behavior categories. For example, the posture data of common walking, climbing, bending, etc. are classified separately. By extracting the centroid displacement trajectory of the person in consecutive frames and combining the optical flow method to calculate the motion vector, if the speed > 1.5 m / s, it is determined as running. By detecting and identifying the contact points between the torso joint points and the tower pole, if the height change rate is greater than 0.3 m / s, it is determined as a climbing action. Using the analysis of the angles of the human body bone joint points, if the angle between the hip joint and the knee joint is less than 90° and lasts for more than 3 seconds, it is determined as a squatting posture. Then, using statistical methods, the mean and standard deviation of the posture parameters under various behaviors are calculated, such as the mean and standard deviation of the joint angle changes. During prediction, according to the posture change characteristics between the current adjacent frames, the behavior category to which it belongs is determined. Then, based on the statistics of the historical data of this category, with the mean as the basis and combined with the standard deviation, a reasonable prediction range is set. For example, if the current posture is determined as walking, and the mean of the joint angle changes in the walking posture is X and the standard deviation is Y, the predicted change range is set as [X - Y, X + Y]. In this way, the predicted change range of the human posture is estimated. Through the estimated predicted change range of the human posture, the predicted change range and the moving direction of the human posture are finally generated.

[0138] Refer to Figure 6 As shown, it is judged whether there is an abnormal behavior, specifically including:

[0139] When the manager receives the first-line message, check whether there is an abnormality in the first-line message. If so, review the first-line message to check the cause of the abnormality. If not, allow the operator to start working;

[0140] The manager gives a safety reminder to the operator according to the cause of the abnormality.

[0141] When an abnormality occurs in the information of the second line and a reminder is issued, the management personnel need to manually analyze the real-time operation steps of the operators, judge whether the operations of the current operators are abnormal. If so, guide the subsequent operation steps of the current operators and train the current operators. If not, cancel the abnormality reminder and record it.

[0142] It can be understood that abnormal situations often have diversity and particularity. Some abnormalities exceed the conventional patterns set by existing algorithms. Manual review can flexibly consider comprehensively from multiple dimensions such as the overall operation process and the operation logic of equipment, and dig out the real reasons behind the abnormalities. During the operation process of the distribution network, due to sudden changes in the natural environment, the wearing requirements do not meet the standards, and the operators have different handling methods. Although the methods are reasonable, there are differences from the standard process, resulting in the appearance of abnormal reminder phenomena. For example, in the monitoring of personnel postures, special emergency avoidance postures may be misreported by the system, and manual operation can accurately judge in combination with the actual scenario, so as to ensure the accuracy and reliability of abnormal information.

[0143] Further, referring to Figure 7 As shown, based on the same concept as the method for managing abnormal behaviors in distribution network operations based on panoramic video analysis, an abnormal behavior management system is proposed, including:

[0144] A collection module, which is used to obtain the panoramic video of the distribution network operation scenario and the information of the operators, collect the relevant specifications of the distribution network operation, and transmit the collected data to the processing module and the judgment module;

[0145] A processing module, which is used to process the received data. According to the panoramic video of the distribution network operation scenario, generate the scenario information and the appearance information of the operators. According to the standard information of the distribution network operation, analyze the operator information and the appearance information of the operators under the scenario information to generate the first behavior information, analyze the predicted behavior information of the personnel to generate the second behavior information, and generate the personnel posture structure according to the operator information, and transmit the data to the judgment module;

[0146] A judgment module, which is used to analyze and judge the received data, send a prompt to the management personnel through the first behavior information and the second behavior information. The management personnel conduct a review of the operation behavior process according to the prompt, judge whether there are abnormal behaviors, and transmit the data to the management module;

[0147] A management module, which is used to receive the prompt transmitted by the judgment module, and further review and confirm the received prompt through the management personnel, and then sort and store the results of the further review.

[0148] Preferably, the collection module specifically includes:

[0149] The first collection unit is used to obtain the time when the operator enters the scene, the facial information of the person entering the scene, and the required operation content, and transmit the data to the processing module;

[0150] The second collection unit is used to obtain the height, body proportion, and arm span data of the operator according to the operator's profile information, and transmit the data to the processing module;

[0151] The third collection unit is used to obtain the panoramic video of the distribution network operation scene, and transmit the data to the processing module and the judgment module.

[0152] Preferably, the processing module specifically includes:

[0153] The first processing unit is used to extract image frames from the pre-operation panoramic record at a rate of one frame per second to generate static images, analyze each static image based on the region growing method to generate the scene object boundaries of each image, and transmit the data to the judgment module;

[0154] The second processing unit is used to perform body posture superposition on the individual bone structure diagram according to the arm span data to generate an individual morphology fitting diagram, analyze the individual morphology fitting diagrams of the ground operation types and high-altitude operation types respectively according to the distribution characteristics of human joint points to obtain upper limb joint point information and torso joint point information, generate a personnel posture structure, and transmit the data to the judgment module;

[0155] The third processing unit is used to locate the operator according to the human body structure posture to generate a personnel bounding box, generate the gradient amplitude and direction of each pixel point through the horizontal gradient and vertical gradient of the pixel points, statistically analyze the gradient amplitude of each pixel point in the unit to generate a gradient amplitude histogram, and superimpose the gradient amplitude histogram according to the pixel point direction to generate a gradient histogram direction feature, and transmit the data to the judgment module.

[0156] Preferably, the judgment module specifically includes:

[0157] The first judgment unit is used to perform image boundary matching on the scene object boundaries of each image to obtain boundary rule parameters, judge whether the object boundary rule parameters are greater than the rule threshold, and transmit the result to the management module;

[0158] The second judgment unit is used to compare with the operator's qualification information according to the operator's due qualification, judge whether the person entering the scene is qualified to perform the operation, compare the appearance information of the operator with the current scene operation wearing standard, judge whether the wearing appearance of the operator can perform safe operation, and transmit the result to the management module;

[0159] A third judgment unit, which is used to judge whether the operation process posture of the personnel is reasonable by comparing the standard operation process posture and the predicted change range of the personnel posture, generate an operation judgment result, judge whether the operator is going to enter the risk operation area by comparing the movement trend of the operator and the location of the risk operation area, generate a movement judgment result, and transmit the result to the management module;

[0160] A fourth judgment unit, which is used for the management personnel to manually analyze the real-time operation steps of the operator, judge whether the current operation of the operator is abnormal, and transmit the result to the management module.

[0161] In summary, the advantages of the present invention are as follows: it can effectively capture anomalies, efficiently screen and analyze key information in the face of a large amount of real-time data, reduce the lag of anomaly recognition, reduce the occurrence of misjudgment situations, improve the work efficiency of personnel when facing high-difficulty and complex distribution network operation behaviors, improve the emergency handling ability of operators, and effectively prevent the recurrence of abnormal behaviors.

[0162] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and what is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A management method for abnormal behaviors in distribution network operations based on panoramic video analysis, characterized in that, Including: Obtain the panoramic video of the distribution network operation scenario and the information of the operation personnel; Generate the scenario information and the appearance information of the operation personnel according to the panoramic video of the distribution network operation scenario; Collect the relevant specifications for distribution network operation and set the standard information for distribution network operation; Based on the standard information for distribution network operation, analyze the information of the operation personnel and the appearance information of the operation personnel under the scenario information, and generate the first behavior information; Generate the personnel posture structure according to the information of the operation personnel, and combine with the panoramic video of the distribution network operation scenario to obtain the histogram of oriented gradients (HOG) features; Generate the personnel predicted behavior information through the HOG features; Based on the standard information for distribution network operation, analyze the personnel predicted behavior information and generate the second behavior information; Send a prompt according to the first behavior information and the second behavior information, and the management personnel conduct a review of the operation behavior process according to the prompt to judge whether there is any abnormal behavior.

2. The method for managing abnormal behaviors in distribution network operations based on panoramic video analysis according to claim 1, wherein, The generation of the first behavior information specifically includes: Obtain the time when the operation personnel enter the scenario, the facial information of the personnel entering the scenario, and the required operation content; According to the time when the operation personnel enter the scenario, extract the panoramic video data of the previous hour before the operation personnel enter the distribution network scenario, and generate a pre-operation panoramic record; Extract image frames from the pre-operation panoramic record at a rate of one frame per second to generate each static image; Based on the region growing method, analyze each static image to generate the scene object boundaries of each image; Perform image boundary matching on the scene object boundaries of each image to obtain the boundary rule parameters, and judge whether the object boundary rule parameters are greater than the rule threshold. If so, it indicates that there is no interference weather in the scenario. If not, it indicates that there is interference weather in the scenario; According to the judgment result, analyze the relevant specifications for distribution network operation, adjust the standard information for distribution network operation, and generate the current scenario operation wearing standard; According to the information of the operation personnel, set and bind a unique identifier for each operation personnel, and the information of the operation personnel includes the personnel operation qualification information and the personnel facial feature information; Determine the operation personnel qualification information by comparing the facial information of the personnel entering the scenario with the personnel facial feature information; According to the relevant specifications for distribution network operation, analyze the required operation content, generate the required qualifications of the operation personnel and compare them with the operation personnel qualification information to judge whether the personnel entering the scenario are qualified to perform the operation, and generate a qualification judgment result; Compare the appearance information of the operation personnel with the current scenario operation wearing standard to judge whether the wearing appearance of the operation personnel can perform safe operation, and generate an appearance judgment result; Comprehensively analyze the qualification judgment result and the appearance judgment result to generate the first behavior information.

3. A method for managing abnormal behaviors in distribution network operations based on panoramic video analysis according to claim 2, characterized in that, The generation of the personnel posture structure specifically includes: According to the information of the operation personnel, obtain the height, body proportion and arm span data of the operation personnel, divide the personnel work types according to the personnel operation qualification information, and generate ground operation work types and high-altitude operation work types; Obtain the standard bone structure diagram and standard body proportion corresponding to each height of the human body; Use the ratio of the body proportion of the operation personnel at each height to the standard body proportion as the individual bone difference, and combine with the standard bone structure diagram to generate an individual bone structure diagram; Perform body posture superposition on the individual bone structure diagram according to the arm span data to generate an individual morphology fitting diagram; Obtain the distribution characteristics of human joint points, analyze the individual morphology fitting diagram of ground operation workers, and generate upper limb joint point information; According to the distribution characteristics of human joint points, analyze the individual morphology fitting diagram of high-altitude operation workers, and generate trunk joint point information; Substitute the trunk joint point information and the upper limb joint point information into the individual morphology fitting diagram respectively, and generate the personnel posture structure based on the kinematic principle.

4. The method for managing abnormal behaviors in distribution network operations based on panoramic video analysis according to claim 3, wherein The obtaining of the gradient histogram direction feature specifically includes: Extract image frames from the panoramic video at a rate of one frame per second, and convert the image frames into grayscale images; Locate the operating personnel according to the human body structure posture, and generate a personnel bounding box; Obtain the left pixel value, right pixel value, upper pixel value, and lower pixel value of each pixel within the personnel bounding box in the grayscale image; Calculate the horizontal gradient of the pixel point by subtracting the left pixel value from the right pixel value, and calculate the vertical gradient of the pixel point by subtracting the lower pixel value from the upper pixel value; Generate the gradient magnitude and direction of each pixel point through the horizontal gradient and vertical gradient of the pixel point; Divide the personnel bounding box into 16×16 cells, and statistically analyze the gradient magnitudes of the pixel points within the cells to generate a gradient magnitude histogram; Superpose the gradient magnitude histogram according to the pixel point direction to generate the gradient histogram direction feature; Among them, the specific calculation formulas for generating the gradient magnitude and direction of the pixel point are: where G represents the gradient magnitude of a pixel, G x represents the horizontal gradient of the pixel, and G y represents the vertical gradient of the pixel, and θ represents the direction angle of the pixel.

5. A method for managing abnormal behaviors in distribution network operations based on panoramic video analysis according to claim 4, characterized in that, The generation of the second line of information specifically includes: Analyze each frame of the grayscale image converted from the panoramic video to obtain the gradient magnitude histogram features of adjacent frames; Compare the gradient magnitude histogram features of adjacent frames, extract the movement of the personnel posture in adjacent frames, and generate the predicted change range and movement direction of the personnel posture; Expand the information of the personnel bounding box according to the distribution network operation standard information to generate the standard operation process posture and the risk operation area; Judge whether the operation process posture of the personnel is reasonable by comparing the standard operation process posture with the predicted change range of the personnel posture, and generate an operation judgment result; Generate the movement trend of the operating personnel according to the predicted change range and movement direction of the personnel posture; Judge whether the operating personnel are about to enter the risk operation area by comparing the movement trend of the operating personnel with the location of the risk operation area, and generate a movement judgment result; Combine the operation judgment result and the movement judgment result to generate the second line of information.

6. The management method for abnormal behavior of distribution network operation based on panoramic video analysis according to claim 5, wherein The judgment of whether there is abnormal behavior specifically includes: When the management personnel receive the first line of information, check whether there is any abnormality in the first line of information. If so, review the first line of information to check the reason for the abnormality. If not, allow the operating personnel to start working; The management personnel give safety reminders to the operating personnel according to the reason for the abnormality; When the second line of information is abnormal and a reminder is issued, the management personnel need to manually analyze the real-time operation steps of the operating personnel to judge whether the current operation of the operating personnel is abnormal. If so, guide the subsequent operation steps of the current operating personnel and train the current operating personnel. If not, cancel the abnormal reminder and record it.

7. A management system for abnormal behavior in distribution network operations based on panoramic video analysis, characterized in that, For implementing the abnormal behavior management method described in any one of claims 1-6, including: A collection module, which is used to obtain the panoramic video of the distribution network operation scenario and the information of the operation personnel, collect the relevant specifications of the distribution network operation, and transmit the collected data to the processing module and the judgment module; A processing module, which is used to process the received data, generate scene information and the appearance information of the operation personnel according to the panoramic video of the distribution network operation scenario, analyze the operation personnel information and the appearance information of the operation personnel under the scene information according to the distribution network operation standard information, generate the first behavior information, analyze the personnel predicted behavior information, generate the second behavior information, generate the personnel posture structure according to the operation personnel information, and transmit the data to the judgment module; A judgment module, which is used to analyze and judge the received data, send a prompt to the management personnel through the first behavior information and the second behavior information, the management personnel conduct a review of the operation behavior process according to the prompt, judge whether there is an abnormal behavior, and transmit the data to the management module; A management module, which is used to receive the prompt transmitted by the judgment module, further review and confirm the received prompt through the management personnel, and then sort and store the result of the further review.

8. An abnormal behavior management system for distribution network operations based on panoramic video analysis according to claim 7, characterized in that, The collection module specifically includes: A first collection unit, which is used to obtain the time when the operation personnel enter the scene, the facial information of the personnel entering the scene, and the required operation content, and transmit the data to the processing module; A second collection unit, which is used to obtain the height, body proportion and arm span data of the operation personnel according to the operation personnel information, and transmit the data to the processing module; A third collection unit, which is used to obtain the panoramic video of the distribution network operation scenario, and transmit the data to the processing module and the judgment module.

9. The management system for abnormal behavior of distribution network operation based on panoramic video analysis according to claim 8, wherein The processing module specifically includes: A first processing unit, which is used to extract image frames from the pre-operation panoramic record at a rate of one frame per second, generate each static image, analyze each static image based on the region growing method, generate the scene object boundaries of each image, and transmit the data to the judgment module; A second processing unit, which is used to overlay the body posture on the individual bone structure diagram according to the arm span data, generate an individual shape fitting diagram, analyze the individual shape fitting diagrams of the ground operation types and the high-altitude operation types respectively according to the distribution characteristics of the human body joint points, obtain the upper limb joint point information and the torso joint point information, generate the personnel posture structure, and transmit the data to the judgment module; A third processing unit, which is used to locate the operation personnel according to the human body structure posture, generate a personnel bounding box, generate the gradient amplitude and direction of each pixel point through the horizontal gradient and vertical gradient of the pixel points, statistically analyze the gradient amplitude of each pixel point in the unit, generate a gradient amplitude histogram, and overlay the gradient amplitude histogram according to the pixel point direction to generate a gradient histogram direction feature, and transmit the data to the judgment module.

10. The management system for abnormal behavior in distribution network operations based on panoramic video analysis according to claim 9, characterized in that, The judgment module specifically includes: The first judgment unit is used to perform image boundary matching on the scene object boundaries of each image, obtain boundary rule parameters, judge whether the object boundary rule parameters are greater than the rule threshold, and transmit the result to the management module; The second judgment unit is used to compare with the operator's qualification information according to the required qualifications of the operator, judge whether the personnel entering the scene are qualified to perform the operation, compare the appearance information of the operator with the current scene operation wearing standard, judge whether the wearing appearance of the operator can perform safe operation, and transmit the result to the management module; The third judgment unit is used to judge whether the operation process posture of the personnel is reasonable by comparing the standard operation process posture and the predicted change range of the personnel posture, generate an operation judgment result, judge whether the operator is about to enter the risk operation area by comparing the movement trend of the operator and the location of the risk operation area, generate a movement judgment result, and transmit the result to the management module; The fourth judgment unit is used for the management personnel to manually analyze the real-time operation steps of the operator, judge whether the operation of the current operator is abnormal, and transmit the result to the management module.

Images