Engineering structure safety status monitoring method and system based on UAV formation

Abstract

The present invention discloses a method and system for monitoring the safety status of an engineering structure based on a formation of unmanned aerial vehicles. The method comprises the following steps: acquiring images of a target area at different time periods by means of unmanned aerial vehicles, identifying feature points in the images, matching feature points corresponding to the same target area at a first time and a second time, calculating and obtaining the coordinates of the feature points in the image at the first time in the image at the second time, and actual displacement data of the feature points in space; analyzing the safety status of the structure according to the actual displacement data, theoretical displacement data and historical displacement changes of the feature points in the target area. The present invention can improve the efficiency and accuracy of data collection, reduce errors and omissions in manual operations, provide more comprehensive and accurate monitoring data, provide the possibility for early warning of engineering structures, make the analysis of the safety status of structures more efficient and accurate, and can deal with complex monitoring environments in a targeted manner.

Description

Technical Field

[0001] The invention relates to the technical field of power systems, and in particular to a method for dynamically increasing the capacity of a cable group based on environmental parameters. Background Art

[0002] Safety status monitoring of engineering structures, especially large-scale engineering structures such as bridges, high-rise buildings, tunnels, etc., is of great significance due to their large scale and complex environment. It can prevent possible structural damage, reduce safety accidents, and protect people's lives and property.

[0003] Existing technologies mainly use manual inspection, fixed sensor monitoring and ground vehicle patrol to monitor the safety status of engineering structures. These methods can monitor and warn to a certain extent, but there are also some problems. For example, manual inspection is inefficient and easily affected by human factors; fixed sensors have limited coverage and cannot achieve comprehensive monitoring; ground vehicle patrols are restricted by terrain and cannot cover all areas.

[0004] The defects of the prior art mainly include:

[0005] Low efficiency: Manual inspections or ground vehicle patrols are inefficient and cannot perform real-time or frequent monitoring;

[0006] Limited coverage: Fixed sensors have limited coverage and cannot provide comprehensive spatial monitoring; ground vehicle patrols are limited by terrain and cannot cover all areas;

[0007] Accuracy issues: The accuracy of manual inspections or ground vehicle patrols is affected by human factors and cannot provide accurate data;

[0008] Failure to fully consider environmental factors: Existing monitoring methods mainly focus on changes in the structure itself and often fail to fully consider the impact of environmental factors such as temperature, humidity, wind direction and wind speed. Summary of the invention

[0009] In order to solve the many problems existing in the above-mentioned prior art in engineering structure domain monitoring, the present invention provides an engineering structure safety status monitoring method and system based on drone formation. The present invention automatically acquires and processes monitoring data through drone formation, comprehensively and accurately analyzes and warns the safety status of the engineering structure, and effectively copes with complex environmental conditions.

[0010] The engineering structure safety status monitoring method based on UAV formation includes the following steps:

[0011] The drone acquires images of the target area at different time periods, identifies feature points in the images, matches feature points corresponding to the same target area at the first time and the second time, and calculates the coordinates of the feature points in the image at the first time in the image at the second time, as well as the actual displacement data of the feature points in space;

[0012] The safety status of the structure is analyzed based on the actual displacement data, theoretical displacement data and historical displacement changes of the characteristic points in the target area.

[0013] Preferably, the area to be detected is segmented to obtain at least two target areas, and images of the designated areas are captured.

[0014] Preferably, the coordinates of the feature points in the image at the first time in the image at the second time are calculated through the relative posture information and displacement data of the drone at the first time and the second time.

[0015] Preferably, the actual displacement data of the feature point in space is calculated through the relative displacement data of the feature point on the second image.

[0016] Preferably, the relative coordinate of the first coordinate on the corresponding image at the second time is calculated, and the difference between the relative coordinate and the second coordinate is the relative displacement data;

[0017] The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

[0018] Preferably, numerical simulation calculations are performed through structural load and environmental information to obtain theoretical displacement values ​​of each characteristic point.

[0019] Wherein, the structural load is the load-bearing and distribution information of the structure; and

[0020] The environmental information includes: temperature, humidity, wind direction and wind speed.

[0021] Preferably, if the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradually increasing trend in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be dangerous; if the actual displacement value of the feature point is less than the theoretical displacement value, and the trend is stable or the change values ​​are less than the preset threshold in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be good.

[0022] The engineering structure safety status monitoring system based on drone formation includes:

[0023] A drone, wherein the drone acquires images of a target area at different time periods;

[0024] A feature recognition module, the feature recognition module is used to recognize feature points in the image and match feature points corresponding to the same target area at the first time and the second time;

[0025] An information processing module, the information processing module is used to calculate and obtain the coordinates of the feature points in the first time image in the second time image, and the actual displacement data of the feature points in space;

[0026] A determination module analyzes the safety status of the structure according to actual displacement data, theoretical displacement data and historical displacement changes of characteristic points in the target area.

[0027] Preferably, the information processing module calculates and obtains the actual displacement data of the feature point in space through the relative displacement data of the feature point on the second image;

[0028] The information processing module calculates and obtains the relative coordinate of the first coordinate on the corresponding image at the second time, and the difference between the relative coordinate and the second coordinate is the relative displacement data;

[0029] The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

[0030] Preferably, the determination module analyzes the safety status of the structure including:

[0031] If the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradual increase trend in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be dangerous;

[0032] If the actual displacement value of the feature point is smaller than the ideal displacement value, and the trend is stable in the historical displacement change or the change value is smaller than the preset threshold, then the structural safety status corresponding to the feature point is judged to be good.

[0033] Advantages of the present invention:

[0034] (1) Automation and intelligence: Based on the UAV formation, image data at different times and locations can be automatically acquired, greatly improving the efficiency and accuracy of data collection. At the same time, through image processing and data analysis, the safety status of engineering structures can be intelligently monitored and evaluated, reducing errors and omissions in manual operations.

[0035] (2) Comprehensive and accurate: This solution not only fully covers the target area in space, but also performs multiple measurements in time to obtain the actual displacement data of feature points, providing more comprehensive and accurate monitoring data;

[0036] (3) Early warning: This solution can detect abnormal changes in the structure by comparing the actual displacement data, theoretical displacement data and historical displacement changes of feature points, providing the possibility of early warning of engineering structures;

[0037] (4) Efficient analysis: This solution uses digital simulation calculations and compares the results with actual observation data, making the analysis of the safety status of the structure more efficient and accurate;

[0038] (5) Flexibility: The solution improves flexibility by segmenting the area to be detected and performing separate image acquisition and analysis on each area, making it possible to handle complex monitoring environments in a targeted manner.

[0039] (6) Consideration of environmental factors: This scheme also considers the influence of environmental information such as temperature, humidity, wind direction and wind speed, which enables a more comprehensive assessment and understanding of the various factors that may affect structural safety;

[0040] The present invention can timely discover and warn potential problems of the structure, thereby ensuring the safety of the engineering structure.

[0041] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0043] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0044] Figure 2 It is a schematic diagram of a flow chart of determining a structural safety state of the present invention;

[0045] Figure 3 It is a structural block diagram of the system of the present invention;

[0046] Figure 4 is a schematic diagram of single-strip monitoring in an embodiment of the present invention;

[0047] Figure 5 Schematic diagram of multi-strip monitoring in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0049] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0050] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0051] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0052] The engineering structure safety status monitoring method based on UAV formation includes the following steps:

[0053] The drone acquires images of the target area at different time periods, identifies feature points in the images, matches feature points corresponding to the same target area at the first time and the second time, and calculates the coordinates of the feature points in the image at the first time in the image at the second time, as well as the actual displacement data of the feature points in space;

[0054] The safety status of the structure is analyzed based on the actual displacement data, theoretical displacement data and historical displacement changes of the characteristic points in the target area.

[0055] The present invention uses a drone to fly over a target area at different times to capture high-definition images. By dividing the area to be detected, specific images can be collected for the designated area.

[0056] Through image processing technology, feature points in the image are identified. These feature points may represent key parts of the engineering structure or key areas that can reflect its structural changes.

[0057] Matching the feature points corresponding to the same target area at different times may require considering the relative posture information and displacement data of the drone at different times.

[0058] The coordinates of the feature points in the first time image in the second time image are calculated, that is, the relative displacement of these feature points on the image. Then, the actual displacement of the feature points in space is calculated based on the relative position of the drone and the feature points.

[0059] The theoretical displacement value of each characteristic point is obtained by numerical simulation calculation based on structural load and environmental information.

[0060] By comparing actual displacement data, theoretical displacement data and historical displacement changes, the safety status of the engineering structure is analyzed.

[0061] The present invention can monitor the safety status of engineering structures in real time, comprehensively and accurately, detect structural anomalies at an early stage, provide reliable data support and reduce safety risks.

[0062] Taking a bridge as an example, a drone can fly over the bridge and take high-definition images of the bridge. Through image processing technology, key feature points of the bridge, such as beams, columns, arch feet, etc., can be identified. At different times, the drone flies over the same area again, obtains new images, and then matches the feature points and calculates the displacement of the feature points. At the same time, structural engineers can perform numerical simulation calculations based on the structural load and environmental information of the bridge to obtain the theoretical displacement value of the feature point. Compare the actual displacement with the theoretical displacement. If it is found that the actual displacement is greater than the theoretical displacement and shows a trend of gradual increase, then it can be judged that the bridge may have safety hazards and further inspection and maintenance are required.

[0063] Preferably, the area to be detected is segmented to obtain at least two target areas, and images of the designated areas are captured.

[0064] In the present invention, the area to be detected is segmented to obtain at least two target areas, and then images are collected for these designated areas. The purpose of segmenting the target area is to collect images more specifically and efficiently.

[0065] For large engineering structures, such as bridges, buildings or tunnels, due to their large size and complex shape, if regional segmentation is not performed, comprehensive image acquisition at one time may lead to difficulties in processing and analysis. Through regional segmentation, large structures can be divided into multiple smaller and more regular target areas, which facilitates more accurate positioning and flight of drone formations, while reducing the difficulty and complexity of image processing and feature recognition.

[0066] Through regional segmentation and targeted image acquisition, this preferred solution can improve the efficiency of image acquisition and processing, increase monitoring accuracy, and reduce the complexity of calculation and analysis, thereby more effectively monitoring the safety status of engineering structures.

[0067] For example, a bridge can be divided into multiple target areas, such as piers, bridge decks, main bridge towers, etc. Then, according to the characteristics of the target areas and monitoring requirements, the flight routes and shooting plans of the drones are formulated so that each drone can correspond to one or several target areas for efficient and accurate image acquisition. After that, the images of different areas can be processed and analyzed, feature points can be extracted, displacements can be calculated, and safety status analysis can be performed. This not only improves work efficiency, but also allows for more targeted monitoring and analysis based on the characteristics and requirements of different areas.

[0068] Preferably, the coordinates of the feature points in the image at the first time in the image at the second time are calculated through the relative posture information and displacement data of the drone at the first time and the second time.

[0069] In the present invention, the coordinates of the feature points in the image at the first time in the image at the second time are calculated based on the relative posture information and displacement data of the drone at the first time and the second time. This step is to convert the two-dimensional image information into the actual displacement data of the feature points in space.

[0070] During the flight, the drone will move and may have posture changes, such as rotation, tilt, etc. This means that the coordinates of the same feature point on the image may change at two different time points. Therefore, it is necessary to perform coordinate transformation based on the displacement and posture changes of the drone to obtain the accurate coordinates of the feature point in the first time image in the second time image.

[0071] This step requires some advanced calculation and transformation methods, such as computational geometry, coordinate transformation, and 3D reconstruction.

[0072] This can accurately track and locate the coordinate changes of feature points at different times of the drone flight, thereby obtaining the actual displacement data of the feature points in space. This is very important for the safety status monitoring of engineering structures, because it can accurately reflect the changes and displacements of the structure and provide strong data support.

[0073] For example, for a column of a bridge (assuming it is a feature point), the drone photographed the column in the first flight (first time), and then photographed the column again in the second flight (second time). Since the flight path, flight altitude, angle, etc. of the drone may be different, this may cause the coordinates of the column in the two images to be different. At this time, it is necessary to calculate the coordinates of the column photographed in the first shot in the second shot image based on the attitude information and displacement data of the drone. In this way, the position changes of the column in the two flights can be accurately tracked, thereby obtaining the actual displacement data of the column in space.

[0074] Preferably, the actual displacement data of the feature point in space is calculated through the relative displacement data of the feature point on the second image.

[0075] In the present invention, the relative displacement data of the feature point on the image at the second time is used to calculate the actual displacement data of the feature point in space, because the displacement of the feature point on the image is directly related to its displacement in the actual three-dimensional space.

[0076] The image displacement of feature points is based on the images acquired by the drone at two different time points, and then obtained through feature matching and image comparison techniques. This displacement is two-dimensional, and it reflects the change of feature points on the image plane. In order to obtain the displacement of feature points in real space, it is necessary to combine the flight data of the drone (such as height, angle, etc.) and convert and calculate through methods such as computational geometry and stereo reconstruction.

[0077] Through this method, the displacement information of structural feature points in the actual three-dimensional space can be extracted from the two-dimensional image data obtained by the drone. This is very important for the safety status monitoring of engineering structures, because the safety status of the structure is determined by its displacement and change in the actual space.

[0078] For example, when monitoring a bridge, the drone acquires images of the bridge at the first time point and the second time point, and then identifies a feature point of the bridge (such as a support column). Through analysis, it is found that the feature point has shifted in the image at the second time point compared to the image at the first time point. Then, based on the flight data of the drone and the image displacement, the displacement data of the feature point in the actual three-dimensional space is calculated. If this displacement data exceeds the normal range, it may mean that there is a problem with the structural safety of the bridge.

[0079] Preferably, the relative coordinate of the first coordinate on the corresponding image at the second time is calculated, and the difference between the relative coordinate and the second coordinate is the relative displacement data;

[0080] The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

[0081] The coordinate change of the same feature point in the images captured at two time points (first time and second time) is calculated, and this change is defined as relative displacement data. In the images at the first time and the second time, the position of the feature point is defined as the first coordinate and the second coordinate, respectively.

[0082] In this process, the relative displacement data is obtained by calculating the difference between the relative coordinate of the first coordinate in the second time image and the second coordinate. This difference reflects the displacement of the feature point between the two time points, that is, the spatial change of the feature point during this period of time.

[0083] Calculating the changes in coordinates requires the use of technologies such as digital image processing, feature matching, and computational geometry to accurately track and locate the displacement of feature points.

[0084] Through the above scheme description, the displacement of the same feature point at different time points can be accurately calculated and tracked, thereby accurately reflecting the changes in the engineering structure. This is crucial for monitoring the safety status of the engineering structure.

[0085] For example, suppose you are monitoring a large engineering structure (e.g., a bridge). At the first time, you use a drone to photograph the bridge and mark a feature point (e.g., a portion of the bridge deck). The location of this point is the first coordinate. Then, at the second time, you use a drone to photograph the bridge again and find the same feature point as at the first time. The location of this point is the second coordinate.

[0086] Next, we need to calculate the relative coordinates of the first coordinate on the corresponding image at the second time. Then, we calculate the difference between the relative coordinates and the second coordinates. This difference is the relative displacement data, which reflects the actual displacement of the feature point between the two time points. If this displacement exceeds a certain threshold, it may mean that there is a problem with the structural safety of the bridge.

[0087] Preferably, numerical simulation calculations are performed through structural load and environmental information to obtain theoretical displacement values ​​of each characteristic point.

[0088] Wherein, the structural load is the load-bearing and distribution information of the structure; and

[0089] The environmental information includes: temperature, humidity, wind direction and wind speed.

[0090] The present invention mainly performs numerical simulation calculations based on structural load and environmental information to obtain the theoretical displacement value of each characteristic point. The structural load mainly refers to the load-bearing and distribution information of the structure, while the environmental information includes temperature, humidity, wind direction and wind speed, etc.

[0091] This information is critical because the displacement and deformation of the structure is not only related to the load-bearing and distribution of the structure itself, but also affected by environmental factors. For example, temperature changes will cause thermal expansion or contraction of materials, humidity changes will affect the swelling or shrinkage of materials, and wind speed and direction will produce wind pressure.

[0092] In the present invention, structural load and environmental information are used to perform numerical simulation calculations to obtain theoretical displacement values ​​of each feature point. This theoretical displacement value is calculated based on structural mechanics and physical principles, and represents the displacement that the feature point should have under given load and environmental conditions.

[0093] Through this method, the theoretical displacement of the structure under specific environment and load can be obtained, which is very useful for monitoring and evaluating the safety status of the structure. In particular, by comparing the actual displacement data with the theoretical displacement data, the safety status of the structure can be better judged.

[0094] For example, suppose you are monitoring a bridge. You know the load information of the bridge (e.g., the traffic on the bridge and the weight distribution of the bridge body, etc.) as well as the environmental information (e.g., the current temperature, humidity, wind speed and direction, etc.). This information is then input into a numerical simulation calculation program, which, based on the principles of structural mechanics and physics, simulates the theoretical displacement of each characteristic point of the bridge under these load and environmental conditions.

[0095] These theoretical displacement values ​​can then be compared with the actual displacement data monitored by the drone. If the actual displacement data greatly exceeds the theoretical displacement, it may mean that there is a problem with the structural safety of the bridge.

[0096] Preferably, if the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradually increasing trend in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be dangerous; if the actual displacement value of the feature point is less than the theoretical displacement value, and the trend is stable or the change values ​​are less than the preset threshold in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be good.

[0097] This part mainly involves the comparison of the actual displacement data of the feature point with the theoretical displacement data, and the judgment of the structural safety status based on this comparison. When the actual displacement is greater than the theoretical displacement and shows a trend of gradual increase in the historical displacement changes, the structural safety status corresponding to the feature point is judged to be dangerous. This means that the structure may be abnormal and requires further inspection and maintenance immediately. On the other hand, if the actual displacement of the feature point is less than the theoretical displacement, and the trend is stable or the change value is less than the preset threshold in the historical displacement changes, then the structural safety status corresponding to the feature point is judged to be good, indicating that the structure is in a normal and stable state.

[0098] This method can provide a real-time, accurate and flexible way to monitor and evaluate the safety status of the structure. Compared with the traditional manual monitoring method, this method is more scientific, efficient and accurate. At the same time, it can automatically carry out large-scale and long-term monitoring, greatly improving work efficiency and safety.

[0099] For example, when monitoring the safety status of a bridge, the drone first obtains the actual displacement data of the bridge, and then compares it with the theoretical displacement data calculated by numerical simulation. Assuming that the actual displacement data of a certain feature point is greater than the theoretical displacement data, and shows a trend of gradual increase in the past monitoring data, then it can be determined that the structural safety status corresponding to the feature point is dangerous and requires further inspection and maintenance immediately. On the other hand, if the actual displacement data of a certain feature point is less than the theoretical displacement data, and the trend is stable or the change value is less than the preset threshold in the past monitoring data, then it can be determined that the structural safety status corresponding to the feature point is good, indicating that the bridge is still safe and stable under the current environment and load conditions.

[0100] The engineering structure safety status monitoring system based on drone formation includes:

[0101] A drone, wherein the drone acquires images of a target area at different time periods;

[0102] A feature recognition module, the feature recognition module is used to recognize feature points in the image and match feature points corresponding to the same target area at the first time and the second time;

[0103] An information processing module, the information processing module is used to calculate and obtain the coordinates of the feature points in the first time image in the second time image, and the actual displacement data of the feature points in space;

[0104] A determination module analyzes the safety status of the structure according to actual displacement data, theoretical displacement data and historical displacement changes of characteristic points in the target area.

[0105] The system of the present invention integrates multiple functions such as image acquisition, feature point recognition, information processing and safety status determination, and can realize comprehensive, real-time and accurate monitoring of the safety status of engineering structures, and can timely discover potential safety hazards of the structures, thereby improving the efficiency and accuracy of monitoring.

[0106] This system can be used to monitor the safety status of a bridge. First, the drone acquires images of the bridge at different time periods. Then, the feature recognition module identifies the feature points in the image and matches the feature points corresponding to the same target area acquired at the first time and the second time. Next, the information processing module calculates and obtains the coordinates of these feature points in the images at the two time points, as well as the actual displacement data of the feature points in space. Finally, the judgment module analyzes the safety status of the bridge based on the actual displacement data, theoretical displacement data and historical displacement changes of the feature points, thereby achieving an accurate assessment of the safety status of the bridge.

[0107] Preferably, the information processing module calculates and obtains the actual displacement data of the feature point in space through the relative displacement data of the feature point on the second image;

[0108] The information processing module calculates and obtains the relative coordinate of the first coordinate on the corresponding image at the second time, and the difference between the relative coordinate and the second coordinate is the relative displacement data;

[0109] The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

[0110] The working principle and method of the information processing module is one of the key parts of the system of the present invention.

[0111] This module first calculates the actual displacement data of the feature point in space through the relative displacement data of the feature point on the image obtained at the second time. The relative displacement data here is actually the coordinate difference between the feature point on the corresponding image at the first time and the second time, or the change in the position of the feature point. The purpose of this calculation is to infer the displacement data in the actual three-dimensional space through the displacement change on the two-dimensional image.

[0112] Then, the module calculates the relative coordinates of the feature point on the image corresponding to the first time (first coordinate) and on the image corresponding to the second time, that is, the position change of the feature point on the image from the first time to the second time. The difference between this relative coordinate and the coordinate of the feature point on the image corresponding to the second time (second coordinate) is defined as the relative displacement data. The calculation goal of this step is to clarify the coordinate changes between the same feature points on images at different times.

[0113] Through such calculations and processing, the information processing module can accurately obtain the actual displacement data of each feature point at different times, providing important data support for subsequent structural safety status analysis.

[0114] Through this processing method, two-dimensional image data can be converted into actual displacement data in three-dimensional space, which will greatly enhance the accuracy and reliability of the system. At the same time, this method can effectively process a large amount of images and data and improve the efficiency of engineering structure safety status monitoring.

[0115] For example, suppose that at the first time, a feature point A is identified in an image of a bridge, and its coordinates are (x1, y1). At the second time, the image of the bridge is acquired again, and a feature point B matching A is found, and its coordinates are (x2, y2). Then, the displacement data of the feature point from A to B can be calculated, that is, (x2-x1, y2-y1), which is the relative displacement data of the feature point on the two-dimensional image. Then, this two-dimensional displacement data can be converted into actual displacement data in three-dimensional space through a specific calculation method, and this data can be used to evaluate the safety status of the bridge.

[0116] Preferably, the determination module analyzes the safety status of the structure including:

[0117] If the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradual increase trend in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be dangerous;

[0118] If the actual displacement value of the feature point is smaller than the ideal displacement value, and the trend is stable in the historical displacement change or the change value is smaller than the preset threshold, then the structural safety status corresponding to the feature point is judged to be good.

[0119] The judgment module is the core part of the safety status assessment of engineering structures. This module judges the structural safety status corresponding to the feature points based on the actual displacement data, theoretical displacement data and historical displacement change trends of the feature points.

[0120] First, if the actual displacement data of the feature point is greater than the theoretical displacement data, and shows a trend of gradual increase in the historical displacement changes, the judgment module will determine that the structural safety status corresponding to the feature point is dangerous. This is because the increase in actual displacement, especially exceeding the theoretical predicted value, and continuing to increase in historical data, is usually a sign of decreased structural safety, which may indicate that the structure has cracks, deformation and other problems.

[0121] Secondly, if the actual displacement value of the feature point is smaller than the theoretical displacement value, and the trend is stable in the historical displacement change or the change value is smaller than the preset threshold, then the judgment module will judge that the structural safety status corresponding to the feature point is good. This is because the actual displacement data is smaller than the theoretical displacement data, and the change trend is stable, indicating that the actual deformation of the structure under the influence of the environment and load does not exceed the expected range, and the structural safety is good.

[0122] This judgment mechanism can promptly detect possible safety problems in the structure at an early stage, provide timely and accurate information for the maintenance and repair of the structure, avoid possible safety accidents, and greatly improve the safety of the project.

[0123] For example, a column of a building has multiple feature points set for monitoring. After a period of observation, it is found that the actual displacement data of a certain feature point is greater than the theoretical displacement data, and it shows a gradual increase trend in the historical displacement changes. In this case, the judgment module will judge the structural safety status corresponding to this feature point as dangerous, which means that the building needs to be inspected and repaired. If the actual displacement value of another feature point is less than the theoretical displacement value, and the trend is stable in the historical displacement changes, then the structural safety status corresponding to the feature point is judged to be good, and there is no problem with the structural safety of this part of the building.

[0124] In the following embodiments of the present invention, the working mode of the drone formation and the final determination of the safety status of the engineering structure are described in detail:

[0125] S1. To meet the monitoring accuracy requirements of large-size structures, the detection area is segmented, such as Figure 4-Figure 5 As shown, in the present invention, the best segmentation method can be selected according to the size of the detection area and the area that the drone can capture images. Single-strip monitoring and multi-strip monitoring are commonly used. i =(a i ×b i ,i=(0,1,2,…)), where i is the number of the target area to be divided, A i is the area of ​​the target area, a i , b i is the corresponding length and width, the chip area of ​​the drone airborne image sensor z = (α×β), α, β are the corresponding image sensor chip length and width, and the pixel arrangement n α ×n β , where n α 、n β is the number of pixels in the α and β directions of the corresponding image sensor chip, the focal length f, and the distance d between the image sensor and the structure monitoring surface must satisfy ε α ,ε β They are the minimum accuracy in the monitoring scale direction;

[0126] S2. According to the segmented area in the previous step, set the target area of ​​each drone in the formation. The segmented area corresponds to the drones in the formation one by one. The number of drones required is n is the number of required flight strips, L is the overall length of the structure monitoring surface, is the average length of the segmented region;

[0127] The texture features of the structure surface are enhanced, and the speckle is sprayed on the structure monitoring surface by using a drone. The speckle should be significantly different from the speckle in the neighborhood.

[0128] S3. Use drone formations to monitor the deformation of the structure based on image information, that is, at the monitoring moment, the image sensors onboard the drones in the drone formation simultaneously expose the monitoring surface of the structure, and use the image information to comprehensively record the external loads (except environmental loads) and structural deformations on the structure, that is, obtain digital images of each monitoring area of ​​the structure at the time moment, in which the structural load and the structural deformation correspond one to one. During the monitoring period, the image sensors onboard the drones in the drone formation continuously and simultaneously expose the monitoring surface of the structure, so as to obtain a series of digital images of each area under the action of the structural load in the time course;

[0129] S4. Use the image information analysis terminal to process the digital images collected by the structure monitoring, that is, load the digital images collected by the drone image sensor in each monitoring area of ​​the structure corresponding to the time node t. Among them, the image corresponding to time t in area i, the spatial coordinates of each drone recorded by RTK, satellite navigation system, and drone airborne inertial navigation unit With displacement increment The displacement increment of region i at time t, dx, dy, dz represent the increments in the x, y, and z directions in space respectively, and the attitude data recorded by the drone gyroscope unit Among them, the angular displacement of region i at time t, rx, ry, rz represents the rotation around the x, y, z axis, and the distance {d i , i=(0,1,2,…)}, all i represent the area numbers divided above; Image sensor internal parameter matrix (provided by the manufacturer) Where K i is the intrinsic parameter matrix of the image sensor corresponding to each region, fα i is α i The focal length in the direction, fβ i β i Focal length in direction, uα i Main point α i Direction coordinate, vβ i Main point β i Direction coordinates:

[0130] (1) Setting There is a digital image in the area at time t0 There is a digital image in area t0 at the next moment t1

[0131] (2) Yes Perform feature recognition separately, with a set of feature points

[0132] (3) Calculation The feature points of the region are matched at time t0 and time t1, and the structure is obtained. The pixel coordinates of the actual spatial feature points of the region in the digital image at time t0 and time t1 are: Among them, this represents a processing process when performing feature matching, j and k represent the symbols of the feature points, (u j ,v j ) This is the coordinate corresponding to the j feature point. It means that these two points match and are pixels with the same name;

[0133] (4) Calculation The attitude and displacement of the drone relative to the aircraft at time t0 and time t1 in the region, that is, Where R represents and The spatial angle between [RX a ,RY a ,RZ a ] and [RX a ,RY a ,RZ a ] are all recorded by the drone’s inertial unit, and T represents the displacement increment;

[0134] (5) Calculation The real pixel coordinates of the matching feature points in the region at time t0 under the drone posture at time t1 are:

[0135]

[0136] (4) Calculation The relative pixel displacement between the matching feature points in the region at time t0 and time t1 is:

[0137]

[0138] (5) Calculation The actual displacement of the space point corresponding to the matching feature point at time t0 and time t1 in the region is: d is The distance between the regional drone image sensor unit and the monitoring plane, and the focal length of the drone image sensor unit in the monitoring direction are obtained. The displacement of several feature points with the same name in the region at time t1 relative to time t0;

[0139] S5. Calculate the theoretical displacement value of the engineering structure under monitoring status;

[0140] S6. Evaluate the safety status of the engineering structure.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0146] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0147] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for monitoring the safety status of engineering structures based on UAV formations, characterized in that: The following steps are involved: Using drone formations, deformation monitoring of structures based on image information is performed. That is, at the monitoring moment, the image sensors onboard the drones in the drone formation simultaneously expose the monitoring surface of the structure, and use the image information to comprehensively record the external load and structural deformation of the structure. The drones acquire images of the target area at different time periods, identify the feature points in the images, match the feature points corresponding to the same target area at the first time and the second time, and calculate the coordinates of the feature points in the image at the first time in the image at the second time, as well as the actual displacement data of the feature points in space. The coordinates of the feature point in the first time image in the second time image are calculated by using the relative posture information and displacement data of the drone at the first time and the drone at the second time; The actual displacement data of the feature point in space is obtained by calculating the relative displacement data of the feature point on the second image; The theoretical displacement value of each characteristic point is obtained by numerical simulation calculation of structural load and environmental information, wherein the structural load is the load-bearing and distribution information of the structure; and the environmental information includes: temperature, humidity, wind direction and wind speed; The safety status of the structure is analyzed based on the actual displacement data, theoretical displacement data and historical displacement changes of the characteristic points in the target area.

2. The monitoring method according to claim 1, characterized in that: By segmenting the area to be detected, at least two target areas are obtained, and images of the designated areas are collected.

3. The monitoring method according to claim 1, characterized in that: Calculate and obtain the relative coordinate of the first coordinate on the corresponding image at the second time, and the difference between the relative coordinate and the second coordinate is the relative displacement data; The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

4. The monitoring method according to claim 1, characterized in that: If the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradually increasing trend in the historical displacement changes, the structural safety status corresponding to the feature point is judged to be dangerous; if the actual displacement value of the feature point is less than the theoretical displacement value, and the trend is stable or the change values ​​are less than the preset threshold in the historical displacement changes, the structural safety status corresponding to the feature point is judged to be good.

5. The engineering structure safety status monitoring system based on UAV formation is characterized by: include: UAV formation, using the UAV formation to monitor the deformation of the structure based on image information, that is, at the monitoring moment, the image sensors on the UAVs in the UAV formation simultaneously expose the monitoring surface of the structure, and use the image information to comprehensively record the external load and structural deformation of the structure. The UAVs obtain images of the target area at different time periods; A feature recognition module, the feature recognition module is used to recognize feature points in the image and match feature points corresponding to the same target area at the first time and the second time; An information processing module, the information processing module is used to calculate the coordinates of the feature points in the first time image in the second time image, and the actual displacement data of the feature points in space; perform numerical simulation calculations through structural load and environmental information to obtain theoretical displacement values ​​of each feature point, wherein the structural load is the load-bearing and distribution information of the structure; and the environmental information includes: temperature, humidity, wind direction and wind speed; The coordinates of the feature point in the first time image in the second time image are calculated by using the relative posture information and displacement data of the drone at the first time and the drone at the second time; The actual displacement data of the feature point in space is obtained by calculating the relative displacement data of the feature point on the second image; A determination module analyzes the safety status of the structure according to actual displacement data, theoretical displacement data and historical displacement changes of characteristic points in the target area.

6. The monitoring system according to claim 5, characterized in that: The information processing module calculates the actual displacement data of the feature point in space through the relative displacement data of the feature point on the second image; The information processing module calculates and obtains the relative coordinate of the first coordinate on the corresponding image at the second time, and the difference between the relative coordinate and the second coordinate is the relative displacement data; The coordinates of the feature point on the image corresponding to the first time are the first coordinates, and the coordinates of the feature point on the image corresponding to the second time are the second coordinates.

7. The monitoring system according to claim 5, characterized in that: The determination module analyzes the safety status of the structure including: If the actual displacement data of the feature point is greater than the theoretical displacement data and shows a gradual increase trend in the historical displacement changes, the structural safety state corresponding to the feature point is judged to be dangerous; If the actual displacement value of the feature point is smaller than the ideal displacement value, and the trend is stable in the historical displacement change or the change value is smaller than the preset threshold, then the structural safety status corresponding to the feature point is judged to be good.

Images