Unmanned aerial vehicle peccancy patrol intervention method and system for electric power infrastructure construction site

Abstract

The invention discloses an electric power infrastructure site unmanned aerial vehicle peccancy inspection and intervention method and system, and relates to the technical field of electric power engineering site safety supervision. Unmanned aerial vehicles are used for autonomous flight inspection and peccancy identification in an infrastructure site, and peccancy identification is sent to a ground end background for peccancy evaluation and peccancy intervention; the problems that in the prior art, risk identification cannot be carried out on violation consequences, risk evaluation cannot be carried out, and the site cannot be effectively restrained are solved, and the beneficial effect of improving the construction operation safety management level of the capital construction site is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of on-site safety supervision of electric power projects, and in particular to a method and system for intervening in illegal inspections by unmanned aerial vehicles on electric power infrastructure sites. Background Art

[0002] Safety is the primary concern in power operations. Currently, the safety management of power grid infrastructure mostly relies on manual inspections, which have the following problems: traditional ground monitoring equipment has poor mobility, a small monitoring field of view, and is suitable for monitoring on-site scenes, but cannot monitor aerial operations; traditional monitoring equipment can only monitor passively, lacks recognition and judgment of defective behaviors, cannot interact with on-site workers, and cannot flexibly monitor based on on-site conditions.

[0003] Existing technologies use some drones to identify violations and issue warnings at power construction sites. However, this type of technical solution can only generally compare violations on site, but cannot identify or assess the consequences of violations, nor can it effectively constrain the site. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a method and system for unmanned aerial vehicle (UAV) inspection and intervention in violation of regulations at power infrastructure sites, so as to solve the problems that the prior art cannot identify risks, conduct risk assessments, or effectively constrain the consequences of violations.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] A method for intervening in illegal inspections by drones at power infrastructure sites, comprising the following steps:

[0007] The S1 drone automatically generates flight missions based on the coordinates of the operation points;

[0008] S2 takes photos of the work situation at the work site with the camera downward according to the inspection rules, stitches the images to generate a panoramic view of the work site, and dynamically updates it at predetermined intervals;

[0009] After the S3 drone takes pictures, it automatically identifies the violations through the edge computing unit it carries, and the drone system reports the defect information to the backend;

[0010] S4 evaluates the violation defects by reporting the defect information to the backend and checking and matching it with the backend compliance database;

[0011] S5 displays and intervenes in the violation through the violation intervention device according to the violation evaluation result.

[0012] Preferably, step S2 also includes the step of automatically generating an electronic fence to restrict the drone to a predetermined area.

[0013] Preferably, step S1 comprises the following steps:

[0014] S1.1 The step of collecting the work points is to use a drone to collect a picture with geographic location information at the center of the construction site, and obtain the center position of the construction area through the picture with geographic information.

[0015] S1.2 The generation step of the patrol task is to obtain the current construction position and input the coordinates into the system to generate a patrol area with the on-site operation center as the circle. The patrol radius is half of the distance from the patrol area boundary to the on-site operation center, and the patrol points are determined at fixed angles in the horizontal circumferential direction.

[0016] S1.3 sets the interval time to determine the next patrol parameters, and patrols according to the next patrol parameters.

[0017] Preferably, the fixed angle in step S1.2 is 90 degrees, and four photos are used in the step of stitching images to generate a panoramic view of the on-site operation in step S2.

[0018] Preferably, the step of identifying illegal defects in step S3 uses a target detection algorithm based on adaptive feature cross-fusion.

[0019] Preferably, in the step of evaluating the violation defect, the evaluation value P=Max(P i ), P i A score value for each violation item for each on-site unit.

[0020] Preferably, in step S5, the step of displaying the violation through the violation intervention device according to the violation evaluation result uses a multi-color vertical bar sound and light alarm to display different colors and sounds for different evaluation results to alarm.

[0021] Preferably, in step S5, the step of intervening in the violation through the violation intervention device according to the violation evaluation result uses the water and electricity control device to control the water and electricity of the on-site unit according to different evaluation results, and cuts off the water and electricity to require on-site rectification when the violation evaluation result is greater than or equal to a specified threshold.

[0022] Preferably, a buffer time is provided in the step of cutting off water and electricity.

[0023] Based on the same inventive concept, the present solution also provides a UAV violation inspection and intervention system for power infrastructure sites, including a UAV and a ground end, wherein the UAV includes a camera, an image transmission transmitter and a loudspeaker with reception, and the ground end includes a walkie-talkie, a remote control, a receiver, a display device, a violation intervention device and a background, wherein the camera is used to take a panoramic view and feature picture of the on-site operation, the image transmission transmitter is used to transmit the image to the background of the ground end, the loudspeaker with reception is used to play back the voice information received by the walkie-talkie on the UAV end, the walkie-talkie is used to send the voice of the remote operator to the UAV, the remote control is used to manually remotely control the UAV, and the receiver is used to For receiving pictures and texts sent by the drone, the display device is used to display the real-time situation of the site photographed by the drone or the processed or unprocessed images photographed by the drone, the violation intervention device includes a current unit violation indication device and a water and electricity control device, which are used to indicate the violation situation on site and control the water and electricity, the background is used to evaluate the violation defects and send the violation situation evaluation situation to the unit violation indication device on site and control the water and electricity control device to intervene in the construction on site, the violation indication device uses a multi-color vertical bar sound and light alarm and includes a review request device, and the review request device is used to re-evaluate the violation defects of the review request of the on-site unit.

[0024] Compared with the prior art, the beneficial effect of this solution is that the method and system for unmanned aerial vehicle inspection of violations at power infrastructure sites are applied to infrastructure site management. After actual application at infrastructure sites, the unmanned aerial vehicle autonomous flight inspection and infrastructure site violation identification realizes regular and autonomous inspections of infrastructure sites by unmanned aerial vehicles, a bird's-eye view of the entire site, remote inspections and shouting from the background, and automatic identification, evaluation, and intervention of violations. The overall level of safety management and control at power grid infrastructure sites is effectively improved. UAVs provide a wide field of view in the air, effectively making up for the defect that ground-based control balls cannot provide an aerial perspective, facilitating the identification and supervision of violations by background inspectors. The high-low combination with ground-based control balls can quickly discover illegal operations without blind spots, improving the quality and efficiency of traditional ground-based control balls and manual inspection solutions. The remote control function of drones can be used to realize remote inspection of violations, which allows inspectors to find defects through videos or pictures, thereby improving the quality of inspections. The intelligent recognition algorithm carried by drones can automatically identify defects, avoiding manual review of massive videos and effectively improving work efficiency. Overall, this solution provides a new inspection solution for infrastructure violations, and realizes autonomy and intelligence through drone systems, solving the problems of existing technologies that are unable to identify risks, conduct risk assessments, and effectively constrain the consequences of violations, achieving the beneficial effect of improving the safety management level of infrastructure construction site operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1This is a schematic diagram of the composition of an embodiment of a drone in the system of this solution;

[0026] Figure 2 This is a schematic diagram of the composition of an embodiment of the ground terminal in the system of this solution;

[0027] Figure 3 This is a schematic diagram of a certain implementation method of stitching panoramic images of on-site operations in an embodiment of the first solution;

[0028] Figure 4 This is an implementation example of the violation identification step 1 of the power infrastructure site drone violation inspection intervention method of this scheme;

[0029] Among them, 1-UAV, 11-camera, 12-image transmission transmitter, 13-loudspeaker with reception, 1-camera, 2-ground end, 21-walkie-talkie, 22-remote control, 23-receiver, 24-display device, 25-violation intervention device, 26-backstage. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0031] This embodiment provides a technical solution: a method for intervening in illegal inspections by drones at power infrastructure sites, the method comprising:

[0032] The S1 drone automatically generates flight missions based on the coordinates of the operation points;

[0033] S2 takes photos of the work situation at the work site with the camera downward according to the inspection rules, stitches the images to generate a panoramic view of the work site, and dynamically updates it at predetermined intervals;

[0034] After the S3 drone takes pictures, it automatically identifies the violations through the edge computing unit it carries, and the drone system reports the defect information to the backend;

[0035] S4 evaluates the violation defects by reporting the defect information to the backend and checking and matching it with the backend compliance database;

[0036] S5 performs early intervention on the violation through the violation intervention device according to the violation evaluation result.

[0037] In the above steps, step S1 also includes the following steps:

[0038] S1.1 The step of collecting the work points is to use a drone to collect a picture with geographic location information at the center of the construction site, and obtain the center position of the construction area through the picture with geographic information.

[0039] S1.2 The generation step of the patrol task is to obtain the current construction position and input the coordinates into the system to generate a patrol area with the on-site operation center as the circle. The patrol radius is half of the distance from the patrol area boundary to the on-site operation center, and the patrol points are determined at fixed angles in the horizontal circumferential direction.

[0040] S1.3 sets the interval time to determine the next patrol parameters, and patrols according to the next patrol parameters.

[0041] During specific implementation, the interval time can be set to 30 minutes, and the UAV can perform automatic flight safety inspections regularly at intervals of 30 minutes. The inspection path and operation parameters can be automatically planned according to the above typical plan, and can be flexibly modified according to on-site needs.

[0042] In step S1.2, this embodiment uses a fixed angle of 90 degrees. In step S2, four photos are used to stitch images to generate a panoramic view of the on-site operation. That is, during the inspection process, the drone first flies clockwise around the construction center in the four directions of southeast, northwest, and northeast, with the camera facing downward to take four pictures. Of course, you can also use counterclockwise flight or use other fixed angles and corresponding numbers of photos, such as six photos at 60 degrees. The process of generating a bird's-eye view: stitch the pictures to generate a bird's-eye view. The bird's-eye view will be used to view the overall picture of the construction site. The drone will continuously update the bird's-eye view during the flight.

[0043] The principle of image stitching is to perform image calibration preprocessing on the images to be stitched. After the images are calibrated, feature extraction algorithms such as S IFT, SURF, and ORB are used to extract the feature points of each image, and feature descriptors are selected to achieve feature point matching. The homography matrix of the image is obtained by using the matched feature points, and one image is transformed to the relative position of another image. Then, the overlapping areas of the stitched images are fused, and finally a large scene image with no stitching traces and high stitching quality is obtained, realizing the automatic stitching of drone images; for desert Gobi landforms where features cannot be extracted, the basic geographic error elimination alignment method is used for image stitching. For the stitched images, image projection conversion, image data tiling, image database generation and other operations are continued, and the situation can finally be viewed on the GIS map.

[0044] The steps of automatically generating an electronic fence and restricting the drone to a given area can be implemented as follows: the drone automatically identifies and tracks the target, or controls the flight in the front and back ends, collects RTK photos of the scene according to predetermined rules, and AI recognizes or manually recognizes the photos in the front and back ends respectively to calculate key parameters. According to the predetermined rules, the horizontal and vertical boundaries of the electronic fence are automatically generated with manual intervention. Three-dimensional safety boundaries are set around tall targets in the field, such as towers, cranes, pump trucks, excavators, and forklifts. When taking photos and observing the ground, the drone can be set to be no less than 6 meters away from the ground. When remotely controlled or automatically controlled, it can automatically avoid dangerous areas.

[0045] Restricting drones to a given area includes both autonomous and controlled flight. The latter is the remote control process, which can be implemented as follows: After the overall observation of the construction site is completed, the drone enters the remote control mode. Safety inspectors can use the mouse single-click, double-click, scroll wheel and other functions in the background to flexibly control the drone to perform up, down, left, right, forward, backward, rotation and other actions. It can perform continuous flight and step-by-step movement. The speed can be fixed or quickly adjusted by the scroll wheel. The safe speed can be intelligently set according to the application scenario. The drone is always restricted to a safe electronic fence during the control process. The remote intelligent auxiliary control function of the drone can remotely control the drone to provide on-site operation guidance, routine safety inspections or surprise inspections, thereby realizing remote guidance and inspection functions.

[0046] Through the above steps, you can use AI intelligence and manual assistance to set up electronic fences to determine the safe operating airspace for drones, and automatically avoid dangerous areas and perform flight operations in safe areas, thereby realizing the intelligent guarantee function of operation safety.

[0047] The implementation method of step S3 for identifying violation defects is as follows: the drone takes a picture through a camera, then performs scene segmentation on the picture to obtain a feature map of the picture, and then compares the feature map with the sample library feature map features of the edge computing box violation algorithm. Based on the comparison results, it is determined whether there is a violation. If there is a violation, the edge computing box will mark the picture and then send it back to the background server.

[0048] For example, for violations such as not wearing a helmet or not dressing properly, we use an object detection algorithm based on adaptive feature cross fusion. On the basis of the original single shot multi-box detection SSD (Single Shot Multi Box Detector) network: we add a feature cross pyramid FCP (Feature Cross Pyramid) structure to enhance the visual and semantic information of the feature map of the violation behavior; we introduce an adaptive feature fusion AFF (Adaptive Feature Fusion) mechanism for feature Figure 2 The feature maps of different scales are fused with weighted features using adaptively learned weights; the original single multi-frame detection SSD network prior frame is optimized, the network structure is improved, and then the final detection result is obtained through post-processing steps such as non-maximum suppression NMS (Non-Maximum Suppression i on), which effectively improves the detection accuracy of illegal behaviors at risky operation sites of power grid projects.

[0049] In summary, the intelligent hardware and intelligent recognition software on the front end of the drone can automatically identify violations through AI intelligence, determine the nature of the violations, and automatically take measures such as reminders to correct violations and reporting to higher authorities, thereby realizing the functions of intelligent identification, determination, and handling of violations.

[0050] In the step of evaluating the violation defect, the evaluation value P=Max(P i ), P i It is the score value of each violation item for each field unit. In this embodiment, the background compliance database includes descriptions and ratings of various violation items. The ratings are divided into 1 to 4 levels according to the severity of the violation. Each violation item identified by the drone is scored separately according to the severity of the violation, and then an overall evaluation is calculated for all violation items of the field unit, with the highest evaluation score being used as the evaluation value of the field unit. The evaluation of violation defects can also be carried out using other effective evaluation methods in combination with violation intervention steps.

[0051] In step S5, the step of displaying the violation through the violation intervention device according to the violation evaluation result uses a multi-color vertical bar sound and light alarm to display different colors and sounds for different evaluation results. For example, red, orange, yellow and green represent serious violation, heavy violation, violation and no violation respectively, and different alarm voices are set.

[0052] When the safety inspectors on the ground find pictures of illegal operations, they can also use the loudspeaker 13 with a receiver on the drone, such as a 4G loudspeaker, to conduct remote supervision through the background, and play the matching compliance database, which clause of the regulation document the illegal item violates and the severity level of the violation, so as to dissuade and correct the illegal behavior in time, and realize the remote supervision function.

[0053] In step S5, the step of intervening in the violation through the violation intervention device according to the violation evaluation result uses the water and electricity control device to control the water and electricity of the on-site unit according to different evaluation results. When the violation evaluation result is greater than or equal to the specified threshold, such as level 3, the violation is serious, and the water and electricity are cut off to require on-site rectification.

[0054] A buffer time is set in the step of cutting off water and electricity. For some situations where it is not suitable to cut off water and electricity immediately, this embodiment sets a buffer time to give on-site personnel the buffer time to complete certain tasks that must be completed.

[0055] Based on the same inventive concept, the present application also provides a UAV violation inspection and intervention system for power infrastructure sites, including a UAV 1 and a ground terminal 2, wherein the UAV 1 includes a camera 11, an image transmission transmitter 12 and a loudspeaker 13 with reception, and the ground terminal 2 includes a walkie-talkie 21, a remote control 22, a receiver 23, a display device 24, a violation intervention device 25 and a background 26, wherein the camera 11 is used to take panoramic views and feature pictures of on-site operations, the image transmission transmitter 12 is used to transmit images to the background of the ground terminal, the loudspeaker 13 with reception is used to play back the voice information received by the walkie-talkie on the UAV side, the walkie-talkie 21 is used to send the voice of the remote operator to the UAV, and the remote control 22 is used to control the UAV Manual remote control is performed, the receiver 23 is used to receive the images and texts sent by the drone, the display device 24 is used to display the real-time situation of the scene photographed by the drone or the processed or unprocessed images photographed by the drone, the violation intervention device 25 includes a current unit violation indication device and a water and electricity control device, which are used to indicate the violation situation on the scene and control the water and electricity, the background 26 is used to evaluate the violation defects and send the violation situation evaluation situation to the unit violation indication device on the scene and control the water and electricity control device to intervene in the construction on the scene, the violation indication device uses a multi-color vertical bar sound and light alarm and includes a review request device, and the review request device is used to re-evaluate the violation defects for the review request of the on-site unit. When the violation defect is eliminated, the water and electricity control device is automatically controlled to restore the water and electricity supply to the on-site unit. The violation indication device of the multi-color vertical bar sound and light alarm including the review request device in this embodiment is powered by solar energy and batteries, and does not rely on the water and electricity power supply of the original site.

[0056] The above is only a preferred specific implementation method of the present scheme, but the protection scope required by the present scheme is not limited thereto. Any technician familiar with the technical field can make equivalent substitutions or changes according to the technical scheme and inventive concept of the present application within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present application.

Claims

1. A method for intervening in illegal inspections by drones at power infrastructure sites, characterized in that Use the following steps: The S1 drone automatically generates flight missions based on the coordinates of the operation points; S2 takes photos of the work situation at the work site with the camera downward according to the inspection rules, stitches the images to generate a panoramic view of the work site, and dynamically updates it at predetermined intervals; After the S3 drone takes pictures, it automatically identifies the violations through the edge computing unit it carries, and the drone system reports the defect information to the backend; S4 evaluates the violation defects by reporting the defect information to the backend and checking and matching it with the backend compliance database; S5 displays and intervenes in the violation through the violation intervention device according to the violation evaluation result.

2. The method for intervening in illegal inspections by unmanned aerial vehicles at power infrastructure sites according to claim 1 is characterized by: Step S2 also includes the step of automatically generating an electronic fence to restrict the drone to a predetermined area.

3. The method for intervening in illegal inspections by unmanned aerial vehicles at power infrastructure sites according to claim 1 is characterized in that: Step S1 includes the following steps: S1.1 The step of collecting the work points is to use a drone to collect a picture with geographic location information at the center of the construction site, and obtain the center position of the construction area through the picture with geographic information. S1.2 The generation step of the patrol task is to obtain the current construction position and input the coordinates into the system to generate a patrol area with the on-site operation center as the circle. The patrol radius is half of the distance from the patrol area boundary to the on-site operation center, and the patrol points are determined at fixed angles in the horizontal circumferential direction. S1.3 sets the interval time to determine the next patrol parameters, and patrols according to the next patrol parameters.

4. The method for intervening in illegal inspections by unmanned aerial vehicles at power infrastructure sites according to claim 3 is characterized by: The fixed angle in step S1.2 is 90 degrees, and four photos are used in the step of stitching images to generate a panoramic view of the on-site operation in step S2.

5. The method for intervening in illegal inspections by unmanned aerial vehicles at power infrastructure sites according to claim 1 is characterized in that: The step of identifying illegal defects in step S3 uses a target detection algorithm based on adaptive feature cross-fusion.

6. The method for intervening in illegal inspections by unmanned aerial vehicles at power infrastructure sites according to claim 1 is characterized by: In the step of evaluating the violation defect, the evaluation value P=Max(P i ), P i A score value for each violation item for each on-site unit.

7. The power infrastructure site unmanned aerial vehicle violation inspection intervention system according to claim 1 is characterized by: In step S5, the step of displaying the violation through the violation intervention device according to the violation evaluation result uses a multi-color vertical bar sound and light alarm to display different colors and sounds for different evaluation results to alarm.

8. The power infrastructure site unmanned aerial vehicle violation inspection intervention system according to claim 1 is characterized by: In step S5, the step of intervening in the violation through the violation intervention device according to the violation evaluation result uses the water and electricity control device to control the water and electricity of the on-site unit according to different evaluation results. When the violation evaluation result is greater than or equal to the specified threshold, the water and electricity are cut off to require on-site rectification.

9. The power infrastructure site unmanned aerial vehicle violation inspection intervention system according to claim 8 is characterized by: A buffer time is provided in the step of cutting off water and electricity.

10. A UAV illegal inspection and intervention system for power infrastructure sites, comprising a UAV and a ground terminal, characterized in that: The UAV includes a camera, an image transmission transmitter and a loudspeaker with reception function. The ground end includes an intercom, a remote control, a receiver, a display device, a violation intervention device and a background. The camera is used to take a panoramic view of the on-site operation and a feature picture. The image transmission transmitter is used to transmit the image to the background of the ground end. The loudspeaker with reception function is used to play back the voice information received by the intercom on the UAV end. The intercom is used to send the voice of the remote operator to the UAV. The remote control is used to manually control the UAV. The receiver is used to receive the pictures and texts sent by the UAV. The display device is used to display the pictures and texts sent by the UAV. The real-time situation of the site photographed by the drone or the processed or unprocessed images photographed by the drone are shown. The violation intervention device includes a current unit violation indication device and a water and electricity control device, which are used to indicate the violation situation on site and control the water and electricity. The background is used to evaluate the violation defects and send the violation situation evaluation situation to the unit violation indication device on site and control the water and electricity control device to intervene in the construction on site. The violation indication device uses a multi-color vertical bar sound and light alarm and includes a review request device. The review request device is used to re-evaluate the violation defects of the on-site unit review request.

Images