A safety inspection and early warning system for power construction sites based on drones

Abstract

The present invention relates to the field of unmanned aerial vehicle technology and construction inspection technology, and provides a safety inspection and early warning system for electric power construction sites based on unmanned aerial vehicles. A swarm unmanned aerial vehicle module performs all-round monitoring of the construction site, a ground control module is communicatively connected to the swarm unmanned aerial vehicle module, remotely controls the unmanned aerial vehicle and receives information sent back by the unmanned aerial vehicle in real time, and a dynamic task allocation module dynamically allocates inspection tasks to the swarm unmanned aerial vehicle module; an early warning analysis module is connected to the ground control module, and performs early warning analysis in combination with images transmitted by the unmanned aerial vehicle; a safety confirmation module is connected to the early warning analysis module, and when the early warning analysis module analyzes the location of a potential safety hazard, the unmanned aerial vehicle is called for monitoring and confirmation; an emergency response module is connected to the early warning analysis module, and when the potential safety hazard is confirmed, an emergency response is automatically initiated; the present invention combines the collected image information through the early warning analysis module to analyze the potential safety hazard location, thereby improving the accuracy and reliability of potential safety hazard identification.

Description

Technical Field

[0001] The invention relates to the fields of unmanned aerial vehicle (UAV) technology and construction inspection technology, and in particular to an electric power construction site safety inspection and early warning system based on unmanned aerial vehicles (UAVs). Background Art

[0002] With the rapid development of drone technology, it plays an important role in safety inspections at power construction sites with its advantages of flexibility, efficiency and low cost. Power construction sites are often complex in terrain, and traditional inspection methods are limited by the terrain and difficult to reach certain areas. Drones can easily fly over obstacles and quickly inspect places that are difficult to reach manually, such as the top of towers and mountainous areas. For example, it can efficiently check the condition of overhead lines and towers at construction sites in mountainous areas. At the same time, compared with manual inspections, swarm drones can cover larger areas in a shorter time, and multiple continuous inspections ensure that safety conditions are monitored in a timely manner, and in the long run, it can reduce manpower and equipment maintenance costs.

[0003] By analyzing the images collected by drones in real time, it is possible to automatically detect potential safety hazards at the construction site, such as using target detection algorithms to identify whether construction workers are wearing helmets, wearing seat belts, and illegal operations. It is also possible to monitor the status of power equipment, determine whether the equipment is damaged or overheated, and issue early warnings in time to avoid safety accidents. In addition, with the popularization of 5G technology, the construction of dedicated communication networks, and the application of satellite communication technology, drones can transmit the collected data to the ground control center in real time, and in the process, advanced technical means are used to ensure the stability of data transmission. At the same time, drones receive instructions to provide strong support for safety inspections at power construction sites.

[0004] However, the existing background technology still has the following shortcomings:

[0005] 1. The current UAV patrol warning system lacks scientific and reasonable planning in the task allocation process. When faced with complex construction site environments, it is difficult for a single or a few UAVs to respond quickly to various emergencies. Moreover, once an emergency occurs in a UAV, the existing task allocation model is difficult to adjust in a timely and effective manner, which greatly affects the efficiency and reliability of the patrol warning work. This task allocation method cannot fully cover and monitor large-scale power construction sites in real time, which seriously restricts the application effect of the UAV patrol warning system at power construction sites.

[0006] 2. In existing patrol warning systems, target detection and identification of areas with potential safety hazards are often performed based on simple image features such as color, shape, and texture. This approach has significant limitations. Simple image target recognition is easily affected by factors such as lighting, angle, and occlusion, leading to misjudgment or missed judgments. In addition, relying solely on image target recognition, it is impossible to conduct in-depth analysis and evaluation of potential safety hazards. Summary of the invention

[0007] The present invention provides a safety inspection and early warning system for electric power construction sites based on unmanned aerial vehicles, which solves the problems in the prior art by allocating inspection tasks to swarm unmanned aerial vehicles and performing three-dimensional modeling analysis on collected image information by an early warning analysis module.

[0008] The technical solution of the present invention is as follows:

[0009] A safety inspection and early warning system for power construction sites based on drones, including:

[0010] A swarm drone module, which is composed of multiple drones and is used to monitor the construction site in all directions and from multiple angles;

[0011] A ground control module, the ground control module is communicatively connected to the swarm drone module, and is used to remotely control and monitor the swarm drones and receive images and data sent back by the drones in real time;

[0012] A dynamic task allocation module, which is connected to the ground control module through data, dynamically allocates inspection tasks to the swarm drone module through the ground control module according to the real-time situation of the construction site and the status of each drone;

[0013] The dynamic task allocation module first randomly generates a set of initial task allocation schemes, represents them as a matrix, and calculates the fitness function value of each task allocation scheme, that is, the total task completion time, which is determined by the completion time of each task and the task priority, which also involves the power of the drone. When the power does not meet the constraint conditions, the fitness function value is reduced;

[0014] By selecting the solution with the largest fitness function value as the parent, crossover and mutation operations are performed on it to avoid the solution falling into the local optimal solution; then continuously receiving the real-time task progress feedback and UAV status information from the ground control module, recalculating the fitness function value and adjusting the allocation plan according to the real-time information, and performing real-time dynamic allocation;

[0015] An early warning analysis module, which is connected to the ground control module and performs early warning analysis in combination with the image data transmitted back by the swarm drone module, including the specific location of potential safety hazards;

[0016] The early warning analysis module includes an analysis unit and an early warning unit;

[0017] The analysis unit combines the image information collected by multiple drones to establish a three-dimensional model based on the image data transmitted by the swarm drone module, and analyzes and confirms the hidden danger location;

[0018] First, images with potential safety hazards are collected, feature point sets are extracted and classified and marked; then, feature points are extracted and matched from the images transmitted in real time by the swarm drone module to generate a feature point pair set, in which the feature point pairs of potential safety hazards are marked; the camera posture is calculated based on the feature point pairs to reconstruct a 3D point cloud; then, a model is reconstructed to generate a surface model of the 3D point cloud, and the location of the potential safety hazard points is determined by extracting and comparing the point cloud features;

[0019] The early warning unit will display the monitoring information of the confirmed drone to the staff for further confirmation;

[0020] A safety confirmation module, which is data-connected to the early warning analysis module. When the early warning analysis module analyzes the location of a potential safety hazard, it notifies nearby drones to conduct multi-angle monitoring, and then transmits the monitored data to the early warning analysis module for further confirmation;

[0021] The emergency response module is connected to the early warning analysis module. When the early warning analysis module confirms a potential safety hazard, the emergency response mechanism is automatically activated, and the drone is guided to conduct continuous monitoring to provide real-time information support for emergency rescue.

[0022] Preferably, the specific process of calculating and optimizing the fitness function by the dynamic task allocation module is as follows:

[0023] Randomly generate a set of initial task allocation schemes, using The matrix V is represented by m, where m is the number of drones and n is the number of inspection tasks. represents an element in the matrix V, which means whether the i-th UAV performs the j-th mission. =1 means execution, =0 means not to execute;

[0024] For each task allocation scheme, let it be matrix V and calculate its fitness function value ;

[0025]

[0026] Among them, the total time T for task completion is:

[0027]

[0028] represents the completion time of the jth task, and T is the total time to complete the task; represents the priority of task j, which is set according to the on-site conditions of the construction environment;

[0029] Considering the state of the drone, let the power of drone i be , the power consumed by executing task j is ,

[0030] when

[0031]

[0032] When , the fitness function value is reduced ;

[0033]

[0034] in, It is a weight coefficient used to balance the importance of task completion time and task priority, and is set according to the specific construction site conditions; It is a penalty coefficient, which is set according to the importance of the task and the impact of power shortage. The more important the task is and the more impact the power shortage has on the task execution, the larger the penalty coefficient is set. When the UAV is low on power, it is used to penalize infeasible task allocation schemes in the fitness function and reduce the fitness function value.

[0035] According to the fitness function, the UAV status and task completion time are compared, and the task allocation scheme with the largest fitness function is selected as the parent;

[0036] Perform a crossover operation on the selected parent generation task allocation scheme matrix V, determine the crossover point with the crossover probability, and then generate a new child generation task allocation scheme. The crossover probability is between 0.4 and 0.9. However, the crossover operation will fall into a local optimal solution during the solution process, so continue to perform a mutation operation on the child generation task allocation scheme, that is, randomly select individuals from the offspring with the mutation probability, mutate after determining the mutation position, and take the mutation probability between 0.001 and 0.1. Add the child generation task allocation scheme to the population to replace the task allocation scheme with the lowest fitness.

[0037] The specific process of cross operation is as follows:

[0038] Determine the crossover probability. First, randomly determine a crossover probability value between 0.4 and 0.9. For each pair of parent individuals, decide whether to perform a crossover operation based on the crossover probability.

[0039] Select the intersection point, for the parent generation task allocation matrix, that is, Matrix V, where m is the number of drones and n is the number of inspection tasks; suppose a pair of parent generation task allocation matrix and ;

[0040] Randomly select a row or a column in the matrix as the intersection point. After selecting a row as the intersection point, The part before the intersection The partial combination after the intersection, The money part of the intersection The partial combination after the intersection point obtains two offspring individuals as the new offspring task allocation scheme, before or after the intersection point or above or below it, depending on whether the intersection point is a row or a column;

[0041] The specific process of mutation operation is as follows:

[0042] Determine the mutation probability, randomly determine a mutation probability value between 0.001 and 0.1, and the mutation probability determines the proportion of mutation operations in the offspring individuals;

[0043] Select mutant individuals and assign tasks to offspring In , the individuals to be mutated are randomly selected according to the mutation probability;

[0044] Determine the mutation position,For the generation task allocation scheme selected to be mutated, randomly determine an element position in the matrix or a sub-matrix area as the mutation position;

[0045] Perform mutation, at the determined mutation position, change the value of the position according to the predetermined mutation rule, the preset mutation rule includes changing the specific value of the element information;

[0046] Continuously receive real-time mission progress feedback and drone status information from the ground control module, including completed missions and the remaining battery power of the drone. If the drone fails, it will be considered as having no battery power.

[0047] Based on this real-time information, the fitness function value of the task allocation plan is recalculated, and the parameters in the fitness function are adjusted in time according to changes in the task environment until the maximum number of iterations is reached.

[0048] Preferably, the analysis unit performs the following analysis steps:

[0049] First, a large number of images with potential safety hazards are collected, and feature extraction algorithms such as SIFT or SURF are used to extract feature points in the images of the places with potential safety hazards as a separate set of potential safety hazard feature points S; and different feature points are classified according to the degree of harm of potential safety hazards;

[0050] Secondly, during the extraction process, the extracted feature points are marked by manual annotation or by using a known safety hazard pattern library to clarify the type of safety hazard they represent;

[0051] Then, for each drone image input in real time, feature points in the image are extracted; these feature points have unique texture or shape information and can be matched in different images;

[0052] Suppose the set of feature points extracted from an image L is , where p represents a feature point, r is the index of the feature point, and each feature point , It is a feature point The coordinates in the image, It is a feature point Descriptor of;

[0053] Feature matching: feature point matching between different images. By comparing the descriptors of feature points, we can find the corresponding feature point pairs in different images. For the feature point set S, these feature points carry specific safety hazard related features. When matching the feature points with the real-time input drone image, we can preliminarily determine the type of safety hazard that may exist in the image corresponding to the feature point set S. Then, we can further analyze and verify the matched feature points.

[0054] In a feature point pair, one feature point comes from the safety hazard image and has safety hazard characteristics, and the other comes from the real-time input image. A feature point in the real-time input drone image has the same descriptor as the feature point from the safety hazard image and has the same position in the image, that is, the two feature points are considered to match each other; then this feature point pair is marked as a matching pair with safety hazards;

[0055] Suppose two images and , the set o of matching feature point pairs between them is:

[0056] ;in , ;

[0057] Among them, the camera attitude calculation is performed based on the matched feature point pairs, and the attitude of the drone camera at different positions is calculated. The camera attitude is represented by the rotation matrix H and the translation vector k. By minimizing the reprojection error, the camera attitude and reprojection error are solved. The calculation formula is:

[0058]

[0059] in is the camera projection function, which is determined by the fixed parameters of the camera itself;

[0060] The camera pose provides basic spatial positioning information for 3D reconstruction and provides a measurement basis for subsequent triangulation methods. The 3D point cloud of the scene is reconstructed using the camera pose and matching feature point pairs. For each matching feature point pair, the coordinates of the corresponding 3D space point are calculated by triangulation.

[0061] Set a feature point pair , the corresponding three-dimensional space point is , according to the camera projection function, let and are the coordinates of the feature points in the two images, f is the focal length of the camera, is the coordinate of the camera optical center, is the camera translation between the two images, resulting in the following system of equations:

[0062]

[0063] and

[0064]

[0065] Solve the system of equations to obtain the coordinates of the three-dimensional space point q, and calculate the coordinates of each point in the three-dimensional point cloud in turn;

[0066] The reconstructed 3D point cloud is subjected to surface reconstruction to generate a continuous 3D model. The feature point set S extracted on the 3D model is used to determine which 3D point clouds in the 3D model have safety hazard characteristics. The features of each point in the 3D point cloud are compared with the feature points in the feature point set S one by one. The Euclidean distance between the two is calculated. If the distance value is less than a preset threshold, the threshold is set according to the degree of hazard of the safety hazard feature point. The higher the degree of hazard, the greater the threshold. The 3D point cloud is determined to have safety hazard characteristics. According to the coordinates of the 3D point cloud, the position of each safety hazard feature point in the 3D model in the actual geographic space is determined.

[0067] Preferably, the warning unit is equipped with a data storage medium to accurately record warning information and related data in real time, and immediately store it in a specific database when the warning is triggered, automatically recording all warning information and processing situations to form a historical record.

[0068] Preferably, the swarm drone module is equipped with a high-definition camera and sensor equipment, has communication conditions, and is capable of receiving instructions from a ground control module and transmitting information to the ground control module in real time;

[0069] In addition, the number of drones in the swarm can be flexibly increased or decreased according to the size and complexity of the construction site. When faced with a large and complex construction site, the drones' own monitoring pressure is too great, so the swarm drones coordinate with the ground control module's communication network to automatically increase the number of drones, while the number is reduced at a small construction site. The drones can also automatically monitor the surrounding environment through sensors including air pressure and wind speed sensors, adjust flight parameters according to environmental changes to cope with various complex environments, and ensure continuous monitoring of the construction site.

[0070] Preferably, the ground control module includes a remote control and navigation software, which generates a patrol path and controls the UAV to complete the assigned patrol tasks, and provides an intuitive and convenient intelligent interactive interface. The operator remotely controls the swarm UAV through a touch screen or remote control. The status of each UAV, monitoring screen and key data indicators are displayed in real time on the intelligent interactive interface.

[0071] Preferably, the safety confirmation module has a rapid response mechanism. When the early warning analysis module analyzes the location of a potential safety hazard, the safety confirmation module can quickly notify nearby drones to conduct multi-angle monitoring and obtain all-round image data of the location of the potential safety hazard.

[0072] Preferably, when the emergency response module confirms a potential safety hazard, it automatically triggers an emergency response of a related safety system, and the emergency response includes an audible and visual alarm and a mobile terminal push.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] 1. The present invention realizes all-round and no-dead-angle monitoring of the construction site by adopting a swarm drone module. At the same time, the dynamic task allocation module flexibly adjusts the number of drones according to the scale of the construction site, improves resource utilization efficiency, and can quickly respond to sudden situations of drones, ensuring the continuous progress of inspection tasks and ensuring the safety and stability of the construction site.

[0075] 2. The present invention combines the image information collected by multiple drones through the early warning analysis module to establish a three-dimensional model to analyze and confirm the location of hidden dangers. This method can deeply analyze and evaluate potential safety hazards, reduce misjudgments or missed judgments caused by factors such as lighting, angles, and occlusions, improve the accuracy and reliability of safety hazard identification, and further confirm through the safety confirmation module, saving human resources for safety confirmation and avoiding misjudgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0077] Figure 2 It is a schematic diagram of the workflow of the dynamic task allocation module of the present invention;

[0078] Figure 3 It is a schematic diagram of the workflow of the early warning analysis module of the present invention. DETAILED DESCRIPTION

[0079] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0080] The present invention provides a safety inspection and early warning system for electric power construction sites based on unmanned aerial vehicles, comprising:

[0081] A swarm drone module, which is composed of multiple drones and is used to monitor the construction site in all directions and from multiple angles;

[0082] A ground control module, the ground control module is communicatively connected to the swarm drone module, and is used to remotely control and monitor the swarm drones and receive images and data sent back by the drones in real time;

[0083] A dynamic task allocation module, which is connected to the ground control module through data, dynamically allocates inspection tasks to the swarm drone module through the ground control module according to the real-time situation of the construction site and the status of each drone;

[0084] The dynamic task allocation module first randomly generates a set of initial task allocation schemes, represents them as a matrix, and calculates the fitness function value of each task allocation scheme, that is, the total task completion time, which is determined by the completion time of each task and the task priority. It also involves the power of the drone. When the power does not meet the constraint conditions, the fitness function value is reduced;

[0085] By selecting the solution with the largest fitness function value as the parent, crossover and mutation operations are performed on it to avoid the solution falling into the local optimal solution; then continuously receiving the real-time task progress feedback and UAV status information from the ground control module, recalculating the fitness function value and adjusting the allocation plan according to the real-time information, and performing real-time dynamic allocation;

[0086] An early warning analysis module, which is connected to the ground control module and performs early warning analysis in combination with the image data transmitted back by the swarm drone module;

[0087] The early warning analysis module includes an analysis unit and an early warning unit;

[0088] The analysis unit combines the image information collected by multiple drones to establish a three-dimensional model based on the image data transmitted by the swarm drone module, and analyzes and confirms the hidden danger location;

[0089] First, images with potential safety hazards are collected, feature point sets are extracted and classified and marked; then, feature points are extracted and matched from the images transmitted in real time by the swarm drone module to generate a feature point pair set, in which the feature point pairs of potential safety hazards are marked; the camera posture is calculated based on the feature point pairs to reconstruct a 3D point cloud; then, a model is reconstructed to generate a surface model of the 3D point cloud, and the location of the potential safety hazard points is determined by extracting and comparing the point cloud features;

[0090] The early warning unit will display the monitoring information of the confirmed drone to the staff for further confirmation;

[0091] A safety confirmation module, which is data-connected to the early warning analysis module. When the early warning analysis module analyzes the location of a potential safety hazard, it notifies nearby drones to conduct multi-angle monitoring, and then transmits the monitored data to the early warning analysis module for further confirmation;

[0092] The emergency response module is connected to the early warning analysis module. When the early warning analysis module confirms a potential safety hazard, the emergency response mechanism is automatically activated, and the drone is guided to conduct continuous monitoring to provide real-time information support for emergency rescue.

[0093] Example: Figure 1-3 As shown, in this embodiment, a large-scale power construction site with complex terrain and multiple construction areas, including high-voltage transmission line erection and substation construction, is used as the inspection scene. Due to the large scale of construction and complex environment, traditional safety inspection methods are difficult to meet the needs of comprehensive and timely safety monitoring, so this system is connected for safety inspection warning.

[0094] The swarm drone module is equipped with high-definition cameras and sensor equipment, and has communication means to receive instructions from the ground control module and transmit information to the ground control module in real time;

[0095] In this scenario, first of all, according to the complexity and scale of the construction site, the management team or relevant technical personnel of the power construction site deployed dozens of drones to form a swarm. These drones patrol different areas and conduct all-round monitoring of every corner of the construction site. When the construction area is found to be expanded or the task is heavier, the system automatically increases the number of drones to ensure no-blind-angle monitoring. At the same time, drones can automatically adjust flight parameters through sensors, such as automatically adjusting flight altitude and speed in strong winds, to ensure continuous and stable monitoring.

[0096] The ground control module includes a remote controller and navigation software, which generates patrol paths and controls the drones to complete assigned patrol tasks. It provides an intuitive and convenient intelligent interactive interface, and operators can remotely control the swarm drones through a touch screen or remote control. The intelligent interactive interface displays the status of each drone, monitoring images, and key data indicators in real time.

[0097] In the project, operators used the intelligent interactive interface of the ground control module to develop detailed patrol routes and monitor the working status of the drones in real time. Once an abnormality is found in a drone, such as low battery or unstable signal, corresponding measures will be taken immediately, such as arranging the drone to return for charging.

[0098] When traditional safety inspection methods are unable to meet the needs, the swarm drone module of this system, with its high-definition camera and sensor equipment, can transmit images and key data of the construction site in real time, providing clear and accurate information for ground control personnel, greatly improving the timeliness and comprehensiveness of safety monitoring. Compared with existing technologies, traditional inspection methods are often limited by terrain and manpower, and cannot effectively monitor every corner of the construction site. This system breaks these limitations and ensures that safety hazards are nowhere to hide.

[0099] The specific process of calculating and optimizing the fitness function of the dynamic task allocation module is as follows;

[0100] Randomly generate a set of initial task allocation schemes, using The matrix V is represented by m, where m is the number of drones and n is the number of inspection tasks. Indicates whether the i-th UAV performs the j-th mission, =1 means execution, =0 means not to execute;

[0101] For each task allocation scheme, let it be matrix V and calculate its fitness function value , the fitness function includes the task completion time, task priority and the battery status of the drone; the fitness function is designed as:

[0102]

[0103] in:

[0104]

[0105] represents the completion time of the jth task, and T is the total time to complete the task; represents the priority of task j, which is set according to the on-site conditions of the construction environment;

[0106] Considering the state of the drone, let the remaining power of drone i be , the power consumed by executing task j is ,

[0107] when

[0108]

[0109] When , reduce the fitness function value;

[0110]

[0111] in, It is a weight coefficient used to balance the importance of task completion time and task priority, and is set according to the specific construction site conditions; It is a penalty coefficient, which is set according to the importance of the task and the impact of power shortage. The more important the task is and the more impact the power shortage has on the task execution, the larger the penalty coefficient is set. When the UAV is low on power, it is used to penalize infeasible task allocation schemes in the fitness function and reduce the fitness function value.

[0112] According to the fitness function, the UAV status and task completion time are compared, and the task allocation plan with the largest fitness function is selected as the parent generation;

[0113] Perform a crossover operation on the selected parent generation task allocation scheme matrix V, determine the crossover point with the crossover probability, and then generate a new child generation task allocation scheme. The crossover probability is between 0.4 and 0.9. However, the crossover operation will fall into a local optimal solution during the solution process, so continue to perform a mutation operation on the child generation task allocation scheme, that is, randomly select individuals from the offspring with the mutation probability, mutate after determining the mutation position, and take the mutation probability between 0.001 and 0.1. Add the child generation task allocation scheme to the population to replace the task allocation scheme with the lowest fitness.

[0114] The specific process of cross operation is as follows:

[0115] Determine the crossover probability. First, randomly determine a crossover probability value between 0.4 and 0.9. For each pair of parent individuals, decide whether to perform a crossover operation based on the crossover probability.

[0116] Select the intersection point, for the parent generation task allocation matrix, that is, Matrix V, where m is the number of drones and n is the number of inspection tasks; suppose a pair of parent generation task allocation matrix and ;

[0117] Randomly select a row or a column in the matrix as the intersection point. After selecting a row as the intersection point, The part before the intersection The partial combination after the intersection, The money part of the intersection The partial combination after the intersection point obtains two offspring individuals as the new offspring task allocation scheme, before or after the intersection point or above or below it, depending on whether the intersection point is a row or a column;

[0118] The specific process of mutation operation is as follows:

[0119] Determine the mutation probability, randomly determine a mutation probability value between 0.001 and 0.1, and the mutation probability determines the proportion of mutation operations in the offspring individuals;

[0120] Select mutant individuals and assign tasks to offspring In , the individuals to be mutated are randomly selected according to the mutation probability;

[0121] Determine the mutation position,For the generation task allocation scheme selected to be mutated, randomly determine an element position in the matrix or a sub-matrix area as the mutation position;

[0122] Perform mutation, at the determined mutation position, change the value of the position according to the predetermined mutation rule, the preset mutation rule includes changing the specific value of the element information;

[0123] Continuously receive real-time mission progress feedback and drone status information from the ground control module, including completed missions and the remaining battery power of the drone. If the drone fails, it will be considered as having no battery power.

[0124] Based on this real-time information, the fitness function value of the task allocation plan is recalculated, and the parameters in the fitness function are adjusted in time according to changes in the task environment until the maximum number of iterations is reached.

[0125] The dynamic task allocation module can flexibly adjust the number of drones according to the scale of the construction site, which not only improves the efficiency of resource utilization, but also responds quickly when the construction area is expanded or the task is heavier, ensuring all-round monitoring. At the same time, the module quickly generates a task allocation plan and continuously adjusts and optimizes it based on real-time task progress feedback and drone status information, which can quickly respond to drone emergencies and ensure the continuous progress of inspection tasks.

[0126] Compared with existing technologies, traditional task allocation methods are often fixed and rigid, and cannot adapt to the complex and changeable construction site environment. The dynamic task allocation module of this system can continuously receive real-time task progress feedback from the ground control module and the status information of the drone, including the status of completed tasks and the remaining power of the drone. When the drone fails, it is regarded as having no power. Intelligent adjustments are made according to the actual situation, which greatly improves the inspection efficiency and safety.

[0127] In short, the swarm drone module and dynamic task allocation module of this system cooperate with each other to provide strong technical support for the safety inspection of large-scale power construction sites. Its necessity lies in solving the complex terrain and large-scale construction problems that traditional inspection methods cannot cope with. Its outstanding effect is that it realizes all-round and no-dead-angle monitoring, improves resource utilization efficiency, responds quickly to emergencies, and ensures the safety and stability of the construction site.

[0128] The early warning analysis module includes an analysis unit and an early warning unit;

[0129] The analysis unit combines the image information collected by multiple drones to establish a three-dimensional model based on the image data transmitted by the swarm drone module, and analyzes and confirms the hidden danger location;

[0130] First, a large number of images with potential safety hazards are collected, and feature extraction algorithms such as SIFT or SURF are used to extract feature points in the images of the places with potential safety hazards as a separate potential safety hazard feature point set S, and different feature points are classified according to the degree of harm of potential safety hazards;

[0131] Then, for each drone image input in real time, feature points in the image are extracted; these feature points have unique texture or shape information and can be matched in different images;

[0132] Suppose the set of feature points extracted from an image L is , where p represents a feature point, r is the index of the feature point, and each feature point , It is a feature point The coordinates in the image, It is a feature point Descriptor of;

[0133] Feature matching: feature point matching between different images. By comparing the descriptors of feature points, we can find the corresponding feature point pairs in different images. For the feature point set S, these feature points carry specific safety hazard related features. When matching the feature points with the real-time input drone image, we can preliminarily determine the type of safety hazard that may exist in the image corresponding to the feature point set S. Then, we can further analyze and verify the matched feature points.

[0134] In a feature point pair, one feature point comes from the safety hazard image and has safety hazard characteristics, and the other comes from the real-time input image. A feature point in the real-time input drone image has the same descriptor as the feature point from the safety hazard image and has the same position in the image, that is, the two feature points are considered to match each other; then this feature point pair is marked as a matching pair with safety hazards;

[0135] Suppose two images and , the set o of matching feature point pairs between them is:

[0136] ;in , ;

[0137] Then, the camera attitude is calculated based on the matched feature point pairs to calculate the attitude of the drone camera at different positions. The camera attitude is represented by the rotation matrix H and the translation vector k. The camera attitude and reprojection error are solved by minimizing the reprojection error. The calculation formula is:

[0138]

[0139] in is the camera projection function, which is determined by the fixed parameters of the camera itself;

[0140] The camera pose and the matched feature point pairs are used to reconstruct the 3D point cloud of the scene. For each matched feature point pair, the coordinates of the corresponding 3D space point are calculated by triangulation.

[0141] Set a feature point pair , the corresponding three-dimensional space point is , according to the camera projection function, let and are the coordinates of the feature points in the two images, f is the focal length of the camera, is the coordinate of the camera optical center, is the camera translation between the two images, resulting in the following system of equations:

[0142]

[0143] and

[0144]

[0145] Solve the system of equations to obtain the coordinates of the three-dimensional space point q, and calculate the coordinates of each point in the three-dimensional point cloud in turn;

[0146] The reconstructed 3D point cloud is subjected to surface reconstruction to generate a continuous 3D model. The feature point set S extracted on the 3D model is used to determine which 3D point clouds in the 3D model have safety hazard characteristics. The features of each point in the 3D point cloud are compared with the feature points in the feature point set S one by one. The Euclidean distance between the two is calculated. If the distance value is less than a preset threshold, the threshold is set according to the degree of hazard of the safety hazard feature point. The higher the degree of hazard, the greater the threshold. The 3D point cloud is determined to have safety hazard characteristics. According to the coordinates of the 3D point cloud, the position of each safety hazard feature point in the 3D model in the actual geographic space is determined.

[0147] In traditional safety hazard inspections, manual inspections or single-view image analysis are often relied upon, which are greatly affected by factors such as lighting, angle, and occlusion, and are prone to misjudgment or omission, making it difficult to fully and accurately identify safety hazards. This system uses multiple drones to collect image information, and through advanced feature extraction algorithms and 3D reconstruction technology, it can construct a high-precision 3D model to present the actual situation of the construction site from multiple angles and in all directions. Compared with existing technologies, this method greatly improves the accuracy and reliability of safety hazard identification, providing a solid guarantee for the safety and stability of the construction site.

[0148] The early warning analysis module analyzes and confirms the location of hidden dangers by combining image information collected by multiple drones to create a three-dimensional model, which solves the limitations of traditional safety hazard inspection methods, improves the accuracy and reliability of safety hazard identification, reduces misjudgments and missed judgments, and provides strong technical support for the safety management of large-scale power construction sites.

[0149] The early warning unit is equipped with data storage media, which will display the monitoring information of the drone that has been received and confirmed to the staff for further confirmation, and automatically record all early warning information and processing situations to form a historical record, providing a basis for formulating more effective safety management strategies.

[0150] The safety confirmation module has a quick response mechanism. At the construction site, when the early warning analysis module finds that a certain area may have safety hazards, the safety confirmation module immediately notifies the nearby drones based on the signal strength and the camera status calculated by the early warning unit. The ground control module notifies the drones to go to the area for multi-angle monitoring. The notified drones decide whether to interrupt the current task and go to the safety hazard area based on their own task priority. In the end, these drones were mobilized to take images from different angles. They can also be equipped with infrared thermal imaging sensors to collect infrared thermal imaging image data, providing workers with more comprehensive safety hazard information so that they can make accurate judgments and take effective measures. Further confirmation through the safety confirmation module saves human resources and avoids misjudgment.

[0151] In the project, the emergency response module confirms safety hazards and immediately and automatically activates the emergency response mechanism, triggers sound and light alarms, and guides drones to conduct continuous monitoring, providing real-time information support for emergency rescue.

[0152] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A safety inspection and early warning system for power construction sites based on drones, characterized in that: include: A swarm drone module, which is composed of multiple drones and is used to monitor the construction site in all directions and from multiple angles; A ground control module, the ground control module is communicatively connected to the swarm drone module, and is used to remotely control and monitor the swarm drones and receive images and data sent back by the drones in real time; A dynamic task allocation module, which is connected to the ground control module through data, dynamically allocates inspection tasks to the swarm drone module through the ground control module according to the real-time situation of the construction site and the status of each drone; The dynamic task allocation module first randomly generates a set of initial task allocation schemes, represents them as a matrix, and calculates the fitness function value of each task allocation scheme, that is, the total task completion time, which is determined by the completion time of each task and the task priority, which also involves the power of the drone. When the power does not meet the constraint conditions, the fitness function value is reduced; By selecting the solution with the largest fitness function value as the parent, crossover and mutation operations are performed on it to avoid the solution falling into the local optimal solution; then continuously receiving the real-time task progress feedback and UAV status information from the ground control module, recalculating the fitness function value and adjusting the allocation plan according to the real-time information, and performing real-time dynamic allocation; An early warning analysis module, which is connected to the ground control module and performs early warning analysis in combination with the image data transmitted back by the swarm drone module, including the specific location of potential safety hazards; The early warning analysis module includes an analysis unit and an early warning unit; The analysis unit combines the image information collected by multiple drones to establish a three-dimensional model based on the image data transmitted by the swarm drone module, and analyzes and confirms the hidden danger location; First, images with potential safety hazards are collected, feature point sets are extracted and classified and marked; then, feature points are extracted and matched from the images transmitted in real time by the swarm drone module to generate a feature point pair set, and the feature point pairs with potential safety hazards are marked; The camera posture is calculated based on the feature point pairs to reconstruct the 3D point cloud. Then, the 3D point cloud surface is reconstructed to generate a model, and the location of the potential safety hazard point is determined by extracting and comparing the point cloud features. The early warning unit will display the monitoring information of the confirmed drone to the staff for further confirmation; A safety confirmation module, which is data-connected to the early warning analysis module. When the early warning analysis module analyzes the location of a potential safety hazard, it notifies nearby drones to conduct multi-angle monitoring, and then transmits the monitored data to the early warning analysis module for further confirmation; An emergency response module, wherein the emergency response module is data-connected to the early warning analysis module. When the early warning analysis module confirms a potential safety hazard, the emergency response mechanism is automatically activated, and the drone is guided to conduct continuous monitoring to provide real-time information support for emergency rescue; The specific steps of analysis by the analysis unit are as follows: First, a large number of images with potential safety hazards are collected, and feature extraction algorithms such as SIFT or SURF are used to extract feature points in the images of the places with potential safety hazards as a separate set of potential safety hazard feature points S; and different feature points are classified according to the degree of harm of potential safety hazards; Secondly, during the extraction process, the extracted feature points are marked by manual annotation or by using a known safety hazard pattern library to clarify the type of safety hazard they represent; Then, for each drone image input in real time, feature points in the image are extracted; these feature points have unique texture or shape information and can be matched in different images; Suppose the set of feature points extracted from an image L is , where p represents a feature point, r is the index of the feature point, and each feature point , It is a feature point The coordinates in the image, It is a feature point Descriptor of; Feature matching: feature point matching between different images. By comparing the descriptors of feature points, we can find the corresponding feature point pairs in different images. For the feature point set S, these feature points carry specific safety hazard related features. When matching the feature points with the real-time input drone image, we can preliminarily determine the type of safety hazard that may exist in the image corresponding to the feature point set S. Then, we can further analyze and verify the matched feature points. In a feature point pair, one feature point comes from the safety hazard image and has safety hazard characteristics, and the other comes from the real-time input image. A feature point in the real-time input drone image has the same descriptor as the feature point from the safety hazard image and has the same position in the image, that is, the two feature points are considered to match each other; then this feature point pair is marked as a matching pair with safety hazards; Suppose two images and , the set o of matching feature point pairs between them is: ;in , ; Among them, the camera attitude calculation is performed based on the matched feature point pairs, and the attitude of the drone camera at different positions is calculated. The camera attitude is represented by the rotation matrix H and the translation vector k. By minimizing the reprojection error, the camera attitude and reprojection error are solved. The calculation formula is: in is the camera projection function, which is determined by the fixed parameters of the camera itself; The camera pose provides basic spatial positioning information for 3D reconstruction and provides a measurement basis for subsequent triangulation methods. The 3D point cloud of the scene is reconstructed using the camera pose and matching feature point pairs. For each matching feature point pair, the coordinates of the corresponding 3D space point are calculated by triangulation. Set a feature point pair , the corresponding three-dimensional space point is , according to the camera projection function, let and are the coordinates of the feature points in the two images, f is the focal length of the camera, is the coordinate of the camera optical center, is the camera translation between the two images, resulting in the following system of equations: and Solve the system of equations to obtain the coordinates of the three-dimensional space point q, and calculate the coordinates of each point in the three-dimensional point cloud in turn; The reconstructed 3D point cloud is subjected to surface reconstruction to generate a continuous 3D model. The feature point set S extracted on the 3D model is used to determine which 3D point clouds in the 3D model have safety hazard characteristics. The features of each point in the 3D point cloud are compared with the feature points in the feature point set S one by one. The Euclidean distance between the two is calculated. If the distance value is less than a preset threshold, the threshold is set according to the degree of hazard of the safety hazard feature point. The higher the degree of hazard, the greater the threshold. The 3D point cloud is determined to have safety hazard characteristics. According to the coordinates of the 3D point cloud, the position of each safety hazard feature point in the 3D model in the actual geographic space is determined.

2. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: The specific process of calculating and optimizing the fitness function by the dynamic task allocation module is as follows: Randomly generate a set of initial task allocation schemes, using The matrix V is represented by m, where m is the number of drones and n is the number of inspection tasks. represents an element in the matrix V, which means whether the i-th UAV performs the j-th mission. =1 means execution, =0 means not to execute; For each task allocation scheme, let it be matrix V and calculate its fitness function value ; Among them, the total time T for task completion is: represents the completion time of the jth task, and T is the total time to complete the task; represents the priority of task j, which is set according to the on-site conditions of the construction environment; Considering the state of the drone, let the power of drone i be , the power consumed by executing task j is , when When , the fitness function value is reduced ; in, It is a weight coefficient used to balance the importance of task completion time and task priority, and is set according to the specific construction site conditions; It is a penalty coefficient, which is set according to the importance of the task and the impact of power shortage. The more important the task is and the more impact the power shortage has on the task execution, the larger the penalty coefficient is set. When the UAV is low on power, it is used to penalize infeasible task allocation schemes in the fitness function and reduce the fitness function value. According to the fitness function, the UAV status and task completion time are compared, and the task allocation plan with the largest fitness function is selected as the parent generation; Perform a crossover operation on the selected parent generation task allocation scheme matrix V, determine the crossover point with the crossover probability, and then generate a new child generation task allocation scheme. The crossover probability is between 0.4 and 0.

9. However, the crossover operation will fall into a local optimal solution during the solution process, so continue to perform a mutation operation on the child generation task allocation scheme, that is, randomly select individuals from the offspring with the mutation probability, mutate after determining the mutation position, and take the mutation probability between 0.001 and 0.

1. Add the child generation task allocation scheme to the population to replace the task allocation scheme with the lowest fitness. Continuously receive real-time mission progress feedback and drone status information from the ground control module, including completed missions and the remaining battery power of the drone. If the drone fails, it will be considered as having no battery power. Based on this real-time information, the fitness function value of the task allocation plan is recalculated, and the parameters in the fitness function are adjusted in time according to changes in the task environment until the maximum number of iterations is reached.

3. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: The warning unit is equipped with a data storage medium to accurately record warning information and related data in real time. When the warning is triggered, it is immediately stored in a specific database, automatically recording all warning information and processing conditions to form a historical record.

4. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: The swarm drone module is equipped with high-definition cameras and sensor equipment, has communication conditions, and is capable of receiving instructions from the ground control module and transmitting information to the ground control module in real time; And the number of drones in the swarm can be flexibly increased or decreased depending on the size and complexity of the construction site; When faced with a large and complex construction site, the drone's own monitoring pressure is too great, so the swarm drones coordinate with the ground control module communication network to automatically increase the number of drones, and reduce the number at a small construction site. The drones can also automatically monitor the surrounding environment through sensors including air pressure and wind speed sensors, adjust flight parameters according to environmental changes to cope with various complex environments, and ensure continuous monitoring of the construction site.

5. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: The ground control module includes a remote control and navigation software, which generates patrol paths and controls the UAVs to complete assigned patrol tasks, and provides an intuitive and convenient intelligent interactive interface. The operator remotely controls the swarm UAVs through a touch screen or remote control. The intelligent interactive interface displays the status of each UAV, monitoring images and key data indicators in real time.

6. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: The safety confirmation module has a rapid response mechanism. When the early warning analysis module analyzes the location of a potential safety hazard, the safety confirmation module can quickly notify nearby drones to conduct multi-angle monitoring and obtain all-round image data of the location of the potential safety hazard.

7. The electric power construction site safety inspection and early warning system based on unmanned aerial vehicle as claimed in claim 1, characterized in that: When the emergency response module confirms the potential safety hazard, it automatically triggers the emergency response of the relevant safety system, and the emergency response includes sound and light alarm and mobile terminal push.

Images