A power construction monitoring system, monitoring method, and computer equipment

Abstract

The present invention belongs to the technical field of safety monitoring, and discloses a power construction monitoring system, including: a flying device, a microphone array and an image acquisition device are mounted on the flying device, the microphone array is used to obtain acoustic information of the power equipment, and the image acquisition device is used to collect image information of workers; and an identification module, which identifies the state of the power equipment or the behavior of the workers according to the information transmitted by the microphone array or the image acquisition device. The monitoring system disclosed in the present invention can monitor the working behavior of the construction workers or the state of the power equipment, find problems in time, take corresponding measures, and effectively ensure the safety of the workers or power equipment at the distribution network construction site.

Description

Technical Field

[0001] The present invention relates to the technical field of safety monitoring, and in particular to an electric power construction monitoring system, a monitoring method, and a computer device. Background Art

[0002] With the development of science and technology, industrial production and residents' lives have higher and higher requirements for the continuity and stability of electricity. Power distribution is the last mile for customers to use electricity, and it is also the link that is most susceptible to environmental and force majeure damage, which will directly affect customers' electricity use. In recent years, with the continuous increase in customer electricity load, the number of new power distribution and emergency repair construction sites has increased dramatically, and the safety risk factors such as high mobility of on-site workers, few safety management personnel, and complex distribution network construction have also increased. There is a lack of relevant safety monitoring auxiliary technology, and the main problems are as follows:

[0003] (1) Workers on construction sites have a weak sense of safety. They often do not wear safety helmets or cotton long-sleeved work clothes. Common illegal work behaviors pose risks of electric shock and injury. There is a lack of technical means for real-time identification and monitoring.

[0004] (2) Equipment at the construction site has the risk of electric discharge and injury, and high-temperature explosion. In particular, local discharge occurs frequently in old equipment and parts with loose connections at the repair site. In the summer, equipment at the construction site is exposed to sunlight, which poses a risk of explosion and scalding.

[0005] (3) In the case of overhead lines at high altitudes, equipment repairs, important risky construction links, complex construction areas on site, and concealed projects, it is impossible for the work supervisor on the ground to provide all-round supervision of the workers at high altitudes. Moreover, it is extremely inconvenient for the work supervisor and the supervisor to look up for a long time to supervise the construction workers at high altitudes, which poses a safety hazard.

[0006] Therefore, how to provide a system that can monitor the safety of workers on construction sites and the status of power equipment is a problem that needs to be solved urgently. Summary of the invention

[0007] The embodiments of the present invention provide a power construction monitoring system to solve the problem of lack of monitoring of the behavior of operators and the status of power equipment during the current power construction process. In order to have a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a general review, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its only purpose is to present some concepts in a simple form as a preface to the detailed description that follows.

[0008] According to a first aspect of an embodiment of the present invention, a power construction monitoring system is provided.

[0009] In one embodiment, a power construction monitoring system includes:

[0010] The flying equipment is equipped with a microphone array and an image acquisition device, the microphone array is used to obtain acoustic information of the power equipment, and the image acquisition device is used to collect image information of the operating personnel;

[0011] An identification module, used to identify the state of power equipment or the behavior of operators based on information transmitted by a microphone array or an image acquisition device;

[0012] The application side is used to interact with the flight device, microphone array and image acquisition device to control the flight status of the flight device and the working status of the microphone array and image acquisition device.

[0013] Optionally, the system also includes a sound insulation structure surrounding the outside of the microphone array.

[0014] Optionally, the image acquisition device also includes an infrared acquisition device for acquiring the temperature of the monitored area.

[0015] Optionally, the flying device includes a lighting device.

[0016] Optionally, the microphone array includes a control module, and the control module of the microphone array is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, and transmit the acoustic information collected by the microphone array to the recognition module.

[0017] Optionally, the system further comprises a ground angle adjustment module, and the ground angle adjustment module is used to adjust the ground angle of the microphone array.

[0018] Optionally, the system further includes an application end, which is used to interact with the flight device, the microphone array and the image acquisition device to control the flight status of the flight device and the working status of the microphone array and the image acquisition device.

[0019] Optionally, the identification module is used to identify the state of the power equipment according to the information transmitted by the microphone array, including:

[0020] The recognition module compares the acoustic information of the power equipment transmitted by the microphone array with the voiceprint signal of the power equipment operating normally in the database, and identifies the state of the power equipment according to the comparison result.

[0021] Optionally, the recognition module is used to recognize the behavior of the operator according to the information transmitted by the image acquisition device, including:

[0022] The recognition module transmits the image acquired by the image acquisition device to the pre-trained neural network model, determines the target to be tracked through the neural network model, and tracks the target based on the pre-trained tracking algorithm;

[0023] The recognition module recognizes the behavior of the tracking target through a neural network model.

[0024] According to a second aspect of an embodiment of the present invention, a method for monitoring electric power construction is provided.

[0025] In one embodiment, a method for monitoring electric power construction is based on a system, the system comprising a flying device, an identification module and an application end, the flying device being equipped with a microphone array and an image acquisition device; the method comprising:

[0026] Interact with the flight device, microphone array and image acquisition device through the application end to control the flight status of the flight device and the working status of the microphone array and image acquisition device;

[0027] Acoustic information of power equipment and image information of operators are obtained through microphone array and image acquisition equipment respectively;

[0028] The recognition module identifies the state of the power equipment or the behavior of the operator based on the information transmitted by the microphone array or the image acquisition device.

[0029] Optionally, the step of acquiring acoustic information of the power equipment through a microphone array includes:

[0030] The microphone array includes a control module, which is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, and transmit the acoustic information collected by the microphone array to the recognition module.

[0031] Optionally, the control module of the microphone array is used to adjust the working state of the microphone array according to the control instruction of the flight device or the application end, including:

[0032] The control module of the microphone array uses the Chebyshev synthesis method to adjust the excitation current distribution of the microphone array, so that the directivity diagram of the microphone array presents a central symmetrical form that gradually decreases from the main lobe to the side lobe, so that the main lobe is aimed at the power equipment to obtain the acoustic information of the power equipment.

[0033] Optionally, the control module of the microphone array is used to adjust the working state of the microphone array according to the control instruction of the flight device or the application end, including:

[0034] The control module of the microphone array applies the particle swarm algorithm in the sparse distribution operation of the microphone array, takes the array element amplitude, array element phase and array element spacing as parameters, and the main beam gain and side lobe level of the microphone array as optimization targets, obtains the optimal solution of the sparse distribution of the microphone array, and provides feeding for the microphone array according to the optimal solution of the sparse distribution of the microphone array.

[0035] Optionally, the step of applying a particle swarm algorithm in the sparse distribution operation of the microphone array by the control module of the microphone array comprises:

[0036] The array element amplitude, array element phase and array element spacing of the microphone array are used as parameters as input, and the calculation formula of the main beam gain and side lobe level of the microphone array is set as the fitness function;

[0037] Randomly initialize the position and speed of each particle, and calculate the fitness function value to obtain the historical local optimal solution and historical global optimal solution of each initialized particle;

[0038] Each iteration corresponds to a set of main beam gain values ​​and side lobe level values. The main beam gain and side lobe level are assigned corresponding weights. After multiple iterations, the optimal solution for the sparse distribution of the microphone array is obtained.

[0039] Optionally, the step of the identification module identifying the state of the power equipment according to the information transmitted by the microphone array includes:

[0040] The recognition module compares the acoustic information of the power equipment transmitted by the microphone array with the voiceprint signal of the power equipment operating normally in the database, and identifies the status of the power equipment according to the comparison result.

[0041] Optionally, the step of the recognition module recognizing the behavior of the operator according to the information transmitted by the image acquisition device includes:

[0042] The recognition module transmits the image acquired by the image acquisition device to the pre-trained neural network model, determines the target to be tracked through the neural network model, and tracks the target based on the pre-trained tracking algorithm;

[0043] The recognition module recognizes the behavior of the tracking target through a neural network model.

[0044] Optionally, the step of constructing the neural network model includes:

[0045] Build data sets based on work scenarios;

[0046] Annotate the images in the dataset;

[0047] Divide the dataset into training, validation and test sets;

[0048] The training set is trained using a convolutional neural network algorithm to obtain a neural network model.

[0049] Optionally, the method further comprises the step of optimizing the neural network model, specifically:

[0050] The neural network model is optimized by any one or more of a model weight quantization model, a model weight sparse model, a model channel pruning compression model, a TVM optimization model, a tensorRT optimization model, and an OpenVINO optimization model.

[0051] Optionally, the system further includes a ground angle adjustment module, and the method further includes: adjusting the ground angle of the microphone array by means of the ground angle adjustment module.

[0052] According to a third aspect of an embodiment of the present invention, a computer device is provided.

[0053] In one embodiment, the computer device includes a memory and a processor, the memory stores a computer program, and is characterized in that the processor implements the steps of the method described in any of the above embodiments when executing the computer program.

[0054] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0055] It can monitor the working behavior of construction personnel or the status of power equipment, discover problems in time, and take corresponding measures to effectively ensure the safety of workers or power equipment at the distribution network construction site.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1 is a schematic diagram of a power construction monitoring system according to an exemplary embodiment;

[0059] Figure 2 is a schematic diagram of a microphone array and a sound insulation structure according to an exemplary embodiment;

[0060] Figure 3 The figure is a schematic diagram showing the structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0061] The following description and accompanying drawings fully illustrate the specific embodiments of this article so that those skilled in the art can practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments of this article includes the entire scope of the claims, as well as all available equivalents of the claims. Herein, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the structure, device or equipment including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such structure, device or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the structure, device or equipment including the elements. Each embodiment is described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0062] The terms "longitudinal", "lateral", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like used herein to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this article and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In the description herein, unless otherwise specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a mechanical connection or an electrical connection, it can also be the internal connection of two elements, it can be a direct connection, or it can be an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0063] As used herein, the term "plurality" means two or more than two, unless otherwise specified.

[0064] In this document, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B.

[0065] In this article, the term "and / or" is a description of the association relationship between objects, indicating that three relationships may exist. For example, A and / or B means: A or B, or, A and B.

[0066] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0067] Figure 1 An embodiment of the electric power construction monitoring system of the present invention is shown.

[0068] In this optional embodiment, the power construction monitoring system includes: a flying device, which is equipped with a microphone array and an image acquisition device, the microphone array is used to obtain acoustic information such as local discharge and abnormal noise of the power equipment, and the image acquisition device is used to acquire image information of the workers; an identification module, which identifies the state of the power equipment or the behavior of the workers according to the information transmitted by the microphone array or the image acquisition device; and an application end, which is used to interact with the flying device, the microphone array and the image acquisition device to control the flight state of the flying device and the working state of the microphone array and the image acquisition device.

[0069] The monitoring system of the embodiment of the present invention can analyze the violations of regulations and rules at the power operation site and the fault status of the power equipment in real time.

[0070] The microphone array is a sound source positioning and acoustic imaging device that can collect sound in the space in front of the array. Through spatial and frequency filtering, abnormal acoustic signals can be detected. The flying device carries a microphone array and flies on the designated route at the construction site. The main beam direction of the microphone array is aimed at the power transmission line, high-voltage equipment in the substation, reactors, distribution switch cabinets and other power equipment for scanning. If the power equipment fails, partial discharge, abnormal noise and other phenomena that are difficult to detect with the naked eye will occur. The microphone array can collect the acoustic information of these abnormal phenomena and transmit the collected acoustic information to the recognition module. The recognition module includes a database of voiceprint signals of normally operating power equipment. The recognition module compares the acoustic information of the power equipment transmitted by the microphone array with the voiceprint signals of normally operating power equipment in the database, identifies the state of the power equipment based on the comparison results, and marks the power equipment if a high-frequency abnormal signal with strong power is found, so as to timely discover the fault state of the power equipment.

[0071] The recognition module transmits the images collected by the image acquisition device to the pre-trained neural network model, determines the target to be tracked through the neural network model, tracks the target based on the pre-trained tracking algorithm, obtains the control parameters of the flight device based on the position information of the tracked target, controls the flight status of the flight device and the working status of the image acquisition device based on the control parameters, and continuously tracks the tracked target. The recognition module recognizes the behavior of the tracked target through the neural network model.

[0072] The application end is used to interact with the flight equipment, microphone array and image acquisition equipment to control the flight status of the flight equipment and the working status of the microphone array and image acquisition equipment. Optionally, the application end is an APP, which is developed based on the MSDK of the drone to realize the interaction between the application and the drone payload, image data transmission, and control signal transmission; realize mobile terminal analysis, identification of illegal operations, and snapshot alarm functions.

[0073] In this embodiment, the tracking target is the operator or power equipment in the construction process, and the information that needs to be collected and identified can be the operator behavior information or equipment status information, for example:

[0074] (1) Whether the workers' safety helmets, work clothes, reflective clothing, safety belts and other wearing equipment are in proper use;

[0075] (2) Tracking and identification of operators and intrusion of non-operators on the work site;

[0076] (3) Safe distance between operators and important live parts;

[0077] (4) Behaviors of people in the work area such as making phone calls, smoking, falling down, and leaving the work area;

[0078] (5) Equipment status of safety isolation equipment such as high-voltage disconnectors, circuit breakers, fuses, etc.

[0079] The recognition module can be run in the server carried by the flight equipment, or in the ground server, or can be run jointly by the server carried by the flight equipment and the ground server. For example, the recognition module determines that the target to be tracked is an operator who is not wearing a helmet based on the image output by the image acquisition device, tracks the operator based on the pre-trained tracking algorithm, and converts the position information of the operator into a control parameter. Based on the control parameter, the flight state of the flight equipment and the working state of the image acquisition device are controlled to track the operator and identify the operator's behavior of not wearing a helmet. For another example, the recognition module determines that the target to be tracked is a high-voltage isolating switch based on the information transmitted by the microphone array. If the high-voltage isolating switch fails, a high-frequency partial discharge signal will be generated. The microphone array captures the signal, and the recognition module compares the signal with the voiceprint signal of the normally operating isolating switch in the database. If a high-frequency abnormal signal with a strong power is found, the high-voltage isolating switch is marked and the faulty device is located. The process of the recognition module identifying the equipment failure based on the information transmitted by the microphone array is judged by the partial discharge, abnormal sound and other phenomena that are difficult to see with the naked eye. Since the abnormal signals collected by the microphone array are often generated by power equipment that is fixed and does not move, the power equipment can be marked and an alarm signal can be issued in a timely manner so that subsequent maintenance workers can conduct targeted inspections.

[0080] Optionally, the flying device is a drone. To avoid the influence of the rotation of the drone motor on the partial discharge detection effect, such as Figure 2 As shown, the outer side of the microphone array is surrounded by a sound insulation structure, which is used to isolate the ultrasonic interference generated by the motor and rotor of the drone. Optionally, the material of the sound insulation structure is polyurethane foam plastic, which can absorb sound intensity, achieve sound absorption and sound insulation, and improve the stability and anti-interference performance of the microphone array in collecting acoustic information.

[0081] Optionally, the system of the embodiment of the present invention also includes: a ground angle adjustment module, which is used to adjust the ground angle of the microphone array, and the ground angle of the microphone array is optimized and changed to reduce the noise reflected by the drone and environmental noise on the ground. Since the microphone array only needs to receive ultrasonic signals in front and on the left and right sides of the front, a narrow beam can be formed by the microphone array. By adjusting the amplitude and phase of the array element, beam scanning is achieved on the two axes of the main radiation direction, and the receiving direction of the microphone array is adjusted to achieve the purpose of directional reception, thereby realizing the function of increasing the main beam radiation power (that is, increasing the main lobe level) and reducing the side lobe level, thereby minimizing the interference from the drone body noise. Optionally, the ground angle adjustment module adjusts the ground angle of the microphone array plane to 45°, which can effectively reduce the ground noise reflection and achieve an ideal noise reduction function.

[0082] Optionally, the flight equipment also includes lighting equipment. By installing lighting equipment, sufficient light source is ensured at the construction site. For example, by mounting the lighting equipment on the gimbal of the drone, light source can be provided to the ground and overhead line working areas.

[0083] The microphone array includes a control module, and the control module of the microphone array is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, and transmit the acoustic information collected by the microphone array to the recognition module. Optionally, in the process of the microphone array acquiring the acoustic information of the power equipment, when the control module of the microphone array receives the instructions issued by the flight equipment or the application end and adjusts the directivity of the microphone array, the control module of the microphone array uses the Chebyshev synthesis method to adjust the excitation current distribution of the microphone array, so that the directional diagram of the microphone array presents a central symmetrical form that gradually decreases from the main lobe to the side lobe, so that the main lobe is aligned with the power equipment to acquire the acoustic information of the power equipment. In the case of a small number of array elements, by making the array element excitation current amplitude satisfy the Chebyshev polynomial, the function of suppressing the side lobe level and improving the main beam radiation power is achieved, and local discharge or abnormal noise is more accurately collected.

[0084] Specifically, the control module of the microphone array uses the Chebyshev synthesis method to adjust the excitation current distribution of the microphone array so that the directivity diagram of the microphone array presents a central symmetrical form that gradually decreases from the main lobe to the side lobe. Therefore, the Chebyshev array factor can be expanded into an odd array and an even array.

[0085] The array factor of an odd array (N=2M+1) can be expressed as:

[0086]

[0087] I1,……,I M+1 is the excitation current value of the array element of 1, ..., M+1, k is the wave number (2π / wavelength), d is the array element spacing, θ is the scanning angle, θ is 90° in the normal direction, and ɑ is the initial phase.

[0088] The array factor of an even array (N=2M) can be expressed as:

[0089]

[0090] According to the two combinations of odd-numbered arrays and even-numbered arrays, the amplitude and phase characteristics of the excitation current of each array element can be obtained, that is,

[0091] Even arrays satisfy:

[0092]

[0093] Among them, q is the serial number of the array element.

[0094] Odd arrays satisfy:

[0095]

[0096]

[0097]

[0098] Among them, R is the ratio of the main lobe level to the side lobe level, that is, the ratio of the main lobe level to the side lobe level.

[0099] The existing distribution line inspection uses a microphone array to detect whether the equipment has partial discharge, and adopts the most basic same-amplitude and same-phase array distribution. The system of the above embodiment of the present invention adopts the Chebyshev synthesis method to make the microphone array element excitation current different in amplitude and phase, thereby increasing the main beam gain and suppressing the side lobe level.

[0100] Optionally, in the process of the microphone array acquiring acoustic information of the power equipment, the control module of the microphone array applies the particle swarm algorithm in the sparse distribution operation of the microphone array, takes the array element amplitude, array element phase and array element spacing of the microphone array as parameters, takes the main beam gain and side lobe level of the microphone array as optimization targets, obtains the sparse distribution optimal solution of the microphone array, and provides power feeding for the microphone array according to the sparse distribution optimal solution of the microphone array. The application of the particle swarm algorithm in the sparse distribution operation of the microphone array can further reduce the number of array elements, suppress the side lobe level, reduce the impact of the UAV flight noise on the main beam gain, reduce the load weight, reduce the power loss, and make it more suitable for UAV application scenarios. The application of the particle swarm algorithm in the sparse distribution operation of the microphone array in this embodiment can be used in combination with the Chebyshev synthesis method of the above embodiment, or it can be used alone. For example, the Chebyshev synthesis method is used alone to make the excitation current of the microphone array elements different in amplitude and phase, thereby increasing the main beam gain and suppressing the sidelobe level; for another example, the particle swarm algorithm is used alone to reduce the number of array elements and suppress the sidelobe level; for another example, the Chebyshev synthesis is first used to make the excitation current of the microphone array elements different in amplitude and phase, thereby increasing the main beam gain and reducing the sidelobe, and on this basis, the particle swarm algorithm is introduced to reduce the number of array elements through iterative optimization, so that the energy is more concentrated in the main beam, further improving the efficiency.

[0101] Optionally, the step of applying a particle swarm algorithm in the sparse distribution operation of the microphone array by the control module of the microphone array includes:

[0102] The array element amplitude, array element phase and array element spacing of the microphone array are used as parameters as input, and the calculation formula of the main beam gain and side lobe level of the microphone array is set as the fitness function. According to the actual situation, the two optimization variables of the main beam gain and the side lobe level are given corresponding weights, which are the ratio R of the main lobe level;

[0103] Randomly initialize the position x of each particle i With speed v i , and calculate the fitness function value fit i , obtain the historical local optimal solution p of each particle best and the historical global optimal solution g best ;

[0104] Each iteration corresponds to a set of main beam gain values ​​G i and the sidelobe level δ i , assign corresponding weights to the main beam gain and side lobe level, and obtain the optimal solution for the sparse distribution of the microphone array after multiple iterations.

[0105] Due to the unknown nature of on-site conditions, the two optimization variables, main beam gain and side lobe level, are assigned corresponding weights according to actual conditions. The requirements for main beam gain and side lobe level are different in open scenes and scenes with more walls. In open scenes, the main lobe beam is required to be as narrow as possible so that the recognition distance can be farther, and even if the side lobe is slightly higher, it will hardly be affected. In an environment with more walls, the side lobe needs to be minimized to reduce the impact of echo and noise.

[0106] A specific embodiment of the particle swarm algorithm of the present invention is given below.

[0107] In this embodiment, the particle swarm algorithm includes the following steps:

[0108] Step S11, set the algorithm inertia weight ω, acceleration constants c1, c2, particle swarm number Q, solution space dimension D and maximum number of iterations M and other parameters, the main beam gain of the microphone array in the full array state is set to G0, the ideal side lobe level of the microphone array is set to δ0, and the calculation formula of the main beam gain and side lobe level of the microphone array is set to the fitness function fit i , according to the actual situation, the two optimization variables of main beam gain and side lobe level are given corresponding weights;

[0109] Step S12: Randomly initialize the position x of each particle i With speed v i , and calculate the fitness function value fit i , obtain the historical local optimal solution p of each particle best and the historical global optimal solution g best ;

[0110] Step S13, iterative calculation of fitness function fit i , if there is a fitness function value fit that is better than the local optimal solution and the global optimal solution p If , replace the optimal solution;

[0111] Step S14, update the particle position and velocity according to the following formula, where r1 and r2 are random numbers in the range [0,1];

[0112]

[0113]

[0114] Step S15: determine whether the end condition is met, if not, repeat steps S13 and S14.

[0115] The image acquisition device includes a visible light acquisition device, which acquires the scene image through the visible light acquisition device and transmits it to the recognition module. For example, the visible light camera carried by the drone sends the scene image to the remote control end through the remote control radio frequency link, and the remote control end is connected to the handheld tablet end through the USB data line. The tablet end runs the recognition module to realize the construction process monitoring of the operators.

[0116] Optionally, the image acquisition device also includes an infrared acquisition device for acquiring the temperature of the monitoring area. For example, the infrared camera carried by the drone measures the temperature of the construction area in real time, and an alarm is issued when the temperature of the monitoring area exceeds a threshold.

[0117] Optionally, the recognition module identifies and tracks the target's behavior based on an image output by an image acquisition device, including: dividing the acquired image into grids through a neural network model and predicting the class probability and bounding box of each grid, calculating the probability values ​​of the bounding box and each class, and using a non-maximum suppression algorithm to calculate a bounding box with a confidence level less than a threshold as a result output.

[0118] Optionally, the recognition module uses a YOLO model to track the target to be tracked, and the images captured in real time by the image acquisition device are transmitted to a pre-trained YOLO model. The target to be tracked is determined by the YOLO model, and the tracking target is tracked based on a pre-trained tracking algorithm. The control parameters of the flight equipment are obtained according to the position information of the tracking target, and then one or more of the flight mechanism, gimbal, and image acquisition device of the flight equipment are controlled.

[0119] Furthermore, the YOLO model applies a convolutional neural network (CNN) to the entire image, divides it into grids and predicts the class probability and bounding box of each grid. The algorithm calculates the bounding box and the probability value of each class (operator, equipment, behavior, etc.), and uses the non-maximum suppression algorithm (Non Max Suppression) to calculate the bounding box with a confidence level less than a threshold as the result output.

[0120] Optionally, the YOLO model is a YOLO V5 model, which includes four common modules, specifically: an input end, a reference network, a Neck network, and a Head output end, and deepens and widens the network.

[0121] Optionally, the step of constructing a neural network model includes:

[0122] Build data sets based on work scenarios;

[0123] Annotate the images in the dataset;

[0124] Divide the dataset into training, validation and test sets;

[0125] The training set is trained using a convolutional neural network algorithm to obtain a neural network model.

[0126] Optionally, the step of constructing a data set according to the work scenario specifically includes:

[0127] According to the nature of the work and technical feasibility, the on-site work scenes are divided into multiple categories, and flying equipment is used to collect image samples of multiple categories of scenes to construct a data set. For example, according to the nature of the work and technical feasibility, the on-site work scenes are divided into 9 categories of scenes, including construction, acceptance and emergency repair of distribution network overhead line equipment, construction, acceptance and emergency repair of distribution network pole-mounted transformers and ancillary equipment, and construction, acceptance and emergency repair of distribution network ground box-type transformers and ancillary equipment. UAVs are used to collect image samples of 9 categories of scenes to construct a data set.

[0128] Optionally, the collected image samples are preprocessed to filter out poor quality images such as blur, ghosting, and overexposure. Optionally, the preprocessing of the collected original images also includes normalizing the image samples to make the image samples the same size, which is convenient for later unified processing and application. For example, DJI series drones are commonly used in power inspections. The image samples taken have a high resolution and occupy a large space, which puts a great burden on subsequent model training. Therefore, the original images are compressed while ensuring the quality of the image samples. The resolution of the compressed images is 416*416.

[0129] Optionally, the step of annotating the images in the data set specifically includes: annotating the work behaviors and equipment status in the work scene. For example, the images are manually annotated, mainly annotating the important work behaviors and equipment status in the above 9 scenes, such as the correct wearing of safety helmets, work clothes, reflective clothing and safety belts, whether there are people crossing the safety area, and annotating work behaviors such as making phone calls, smoking, and falling.

[0130] Optionally, after the step of labeling the image of the data set, the step further includes: deforming the image to expand the data set. For example, operations such as twisting and rotating the image samples can be performed to expand the data set, thereby improving the generalization ability and robustness of the neural network model.

[0131] Optionally, the data set obtained after the above steps is divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0132] Optionally, after the above step of obtaining the neural network model, the step of optimizing the neural network model is also included, specifically: optimizing the neural network model through any one or more of a model weight quantization model, a model weight sparse model, a model channel pruning compression model, a TVM optimization model, a tensorRT optimization model, and an OpenVINO optimization model.

[0133] Optionally, after the above step of obtaining the neural network model, a step of verifying the accuracy of the neural network model is also included, specifically: inputting the picture or video into the neural network model to extract relevant features, and then performing feature matching. The higher the similarity, the closer to the correct result.

[0134] Optionally, the trained neural network model can be reduced in parameters and calculations by using decomposition convolution, group convolution, structures with strong feature representation capabilities, and 1x1 convolution instead of 3x3 convolution to achieve lightweight. Optionally, an Android mobile tablet SOC chip with eight or more CPU cores (such as Kirin 820, Qualcomm Snapdragon 865) is selected for algorithm adaptation to complete the transplantation and operation of the algorithm.

[0135] Optionally, the flight device can be connected to the microphone array, image acquisition device, and lighting device by changing the load mode. The communication and power supply of the module load response are realized through the flight device gimbal interface. The power supply standard refers to the flight device (for example, the power supply standard of DJI M300 drone is 17V, 2A). The digital signal is transmitted using USART, and the video stream uses WLAN. The flight device and the remote controller use radio frequency communication to complete the image and data transmission. The remote controller transmits the image and data with the mobile tablet through the USD data cable, using the MQTT protocol, and the program API interface is unified.

[0136] Optionally, the system of the present invention can replace the response load according to the working process. On the one hand, the load hardware is replaced manually, and the load interface is unified, mainly including the power supply interface, time synchronization interface, high power application interface, UART and network communication interface; on the other hand, the remote control end develops and runs the corresponding application program of the load through SDK, including the lighting equipment load control program, the image acquisition equipment load control program, and the partial discharge imaging detection program. Among them, the lighting equipment load program includes the linear adjustment function of the lighting equipment brightness; the image acquisition equipment load control program includes a one-key photo taking function, a one-key switching function of infrared and visible light display screens, an infrared manual point selection temperature measurement function, a video recording function, etc.; the partial discharge imaging detection program includes the monitoring ultrasonic signal frequency band adjustment function, the hologram display and cancellation function, the discharge intensity display and cancellation function, the monitoring area selection function, etc.

[0137] After the work is completed, the person in charge can use drones to check the construction process and acceptance, and use the APP to view key detail photos and violations to complete the online scoring and evaluation of the construction quality.

[0138] In other embodiments, the present invention further proposes a method for monitoring electric power construction, which monitors the behavior of operators or the status of electric power equipment based on the system in the above embodiment. The working principle of the method for monitoring electric power construction in this embodiment is the same as the working principle of the electric power construction monitoring system in the above embodiment, and will not be repeated here.

[0139] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above method embodiment are implemented.

[0140] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiment when executing the computer program.

[0142] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0144] The present invention is not limited to the structures which have been described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A power construction monitoring system, characterized in that: include: The flying equipment is equipped with a microphone array and an image acquisition device, the microphone array is used to obtain acoustic information of the power equipment, and the image acquisition device is used to collect image information of the operating personnel; An identification module, used to identify the state of power equipment or the behavior of operators based on information transmitted by a microphone array or an image acquisition device; The application side is used to interact with the flight device, microphone array and image acquisition device to control the flight status of the flight device and the working status of the microphone array and image acquisition device; The microphone array includes a control module, which is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, and transmit the acoustic information collected by the microphone array to the recognition module; the control module of the microphone array applies the particle swarm algorithm in the sparse distribution operation of the microphone array, takes the array element amplitude, array element phase and array element spacing as parameters, takes the main beam gain and side lobe level of the microphone array as optimization targets, obtains the optimal solution of the sparse distribution of the microphone array, and provides feeding for the microphone array according to the optimal solution of the sparse distribution of the microphone array.

2. The electric power construction monitoring system according to claim 1, characterized in that: It also includes a sound-isolating structure surrounding the outside of the microphone array.

3. The electric power construction monitoring system according to claim 1, characterized in that: The image acquisition device also includes an infrared acquisition device for acquiring the temperature of the monitored area.

4. The electric power construction monitoring system according to claim 1, characterized in that: The flying equipment includes lighting equipment.

5. The electric power construction monitoring system according to claim 1, characterized in that: The system also includes a ground angle adjustment module, which is used to adjust the ground angle of the microphone array.

6. The electric power construction monitoring system according to claim 1, characterized in that: The identification module is used to identify the state of the power equipment according to the information transmitted by the microphone array, including: The recognition module compares the acoustic information of the power equipment transmitted by the microphone array with the voiceprint signal of the power equipment operating normally in the database, and identifies the state of the power equipment according to the comparison result.

7. The electric power construction monitoring system according to claim 1, characterized in that: The recognition module is used to recognize the behavior of the operator according to the information transmitted by the image acquisition device, including: The recognition module transmits the image acquired by the image acquisition device to the pre-trained neural network model, determines the target to be tracked through the neural network model, and tracks the target based on the pre-trained tracking algorithm; The recognition module recognizes the behavior of the tracking target through a neural network model.

8. A method for monitoring electric power construction, characterized in that: Based on a system, the system includes a flight device, an identification module and an application end, and the flight device is equipped with a microphone array and an image acquisition device; the method includes: Interact with the flight device, microphone array and image acquisition device through the application end to control the flight status of the flight device and the working status of the microphone array and image acquisition device; Acquiring acoustic information of power equipment through a microphone array, and acquiring image information of operating personnel through an image acquisition device; the microphone array includes a control module, the control module of the microphone array is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, and transmit the acoustic information collected by the microphone array to the recognition module; the control module of the microphone array is used to adjust the working state of the microphone array according to the control instructions of the flight equipment or the application end, including: the control module of the microphone array applies a particle swarm algorithm in the sparse distribution operation of the microphone array, takes the array element amplitude, array element phase and array element spacing as parameters, takes the main beam gain and side lobe level of the microphone array as optimization targets, obtains the sparse distribution optimal solution of the microphone array, and provides feeding for the microphone array according to the sparse distribution optimal solution of the microphone array; The recognition module identifies the state of the power equipment or the behavior of the operator based on the information transmitted by the microphone array or the image acquisition device.

9. A method for monitoring electric power construction as claimed in claim 8, characterized in that: The control module of the microphone array is used to adjust the working state of the microphone array according to the control instruction of the flight equipment or the application end, including: When the control module of the microphone array receives instructions from the flight equipment or the application end to adjust the directivity of the microphone array, the control module of the microphone array uses the Chebyshev synthesis method to adjust the excitation current distribution of the microphone array so that the directivity diagram of the microphone array presents a centrally symmetrical form that gradually decreases from the main lobe to the side lobe, so that the main lobe is aligned with the power equipment to obtain the acoustic information of the power equipment.

10. A method for monitoring electric power construction according to claim 8, characterized in that: The step of applying the particle swarm algorithm in the sparse distribution operation of the microphone array by the control module of the microphone array comprises: The array element amplitude, array element phase and array element spacing of the microphone array are used as parameters as input, and the calculation formula of the main beam gain and side lobe level of the microphone array is set as the fitness function; Randomly initialize the position and speed of each particle, and calculate the fitness function value to obtain the historical local optimal solution and historical global optimal solution of each initialized particle; Each iteration corresponds to a set of main beam gain values ​​and side lobe level values. The main beam gain and side lobe level are assigned corresponding weights. After multiple iterations, the optimal solution for the sparse distribution of the microphone array is obtained.

11. A method for monitoring electric power construction according to claim 8, characterized in that: The step of the identification module identifying the state of the power equipment according to the information transmitted by the microphone array comprises: The recognition module compares the acoustic information of the power equipment transmitted by the microphone array with the voiceprint signal of the power equipment operating normally in the database, and identifies the status of the power equipment according to the comparison result.

12. A method for monitoring electric power construction according to claim 8, characterized in that: The step of the recognition module recognizing the behavior of the operator according to the information transmitted by the image acquisition device includes: The recognition module transmits the image acquired by the image acquisition device to the pre-trained neural network model, determines the target to be tracked through the neural network model, and tracks the target based on the pre-trained tracking algorithm; The recognition module recognizes the behavior of the tracking target through a neural network model.

13. A method for monitoring electric power construction according to claim 12, characterized in that: The steps of constructing the neural network model include: Build data sets based on work scenarios; Annotate the images in the dataset; Divide the dataset into training, validation and test sets; The training set is trained using a convolutional neural network algorithm to obtain a neural network model.

14. A method for monitoring electric power construction according to claim 13, characterized in that: The method further comprises the step of optimizing the neural network model, specifically: The neural network model is optimized by any one or more of a model weight quantization model, a model weight sparse model, a model channel pruning compression model, a TVM optimization model, a tensorRT optimization model, and an OpenVINO optimization model.

15. The electric power construction monitoring method according to claim 8, characterized in that: The system further comprises a ground angle adjustment module, and the method further comprises: adjusting the ground angle of the microphone array by means of the ground angle adjustment module.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 8 to 15 are implemented.

Images